AINoon Lesson 3

Understanding what’s possible will enable you to spot opportunities.

Clever preparation of prompt/context
+

GenAI model to generate text, images, etc.

+
Clever use of model outputs

Problem: LLMs can answer questions about general knowledge from training data, but have no knowledge of your domain/company
Expensive solution: Continue to train (“fine-tune”) the model on your own text
Pragmatic solution: Retrieve relevant text and add it to augment the prompt

Break up documents into chunks of text
When a user’s question comes in, search for relevant chunks of text
- Usually by finding chunks with the most similar “meaning” to the question according to a specialised text embedding model
Add relevant chunks to the prompt

Key idea: The LLM decides when to call a tool

Fetch data:
- Web or document search (RAG tools)
- Run code (e.g. to analyse a spreadsheet)
- Fetch data from a system (e.g. task management)
- Ask human for input (e.g. review a decision)
Take action:
- Write to a file
- Send a message
- Update data in a system

Many chatbots have built-in tools, or allow you to provide your own
- E.g. You can enable Web Search in ChatGPT
Many systems now provide an MCP server that provides tools in a standard format for any AI app
- E.g. An MCP server for files on your computer allows the LLM to read/write any file it chooses

Vague term with lots of meanings, but one useful definition is:

An LLM agent runs tools in a loop to achieve a goal - Simon Willison

(so that tool example was technically an agent)

Let’s say we’re a restaurant who wants to attract diners with desserts themed for upcoming movie releases.

We’ll call the LLM with instructions for the agent’s task
- Talk through each step, including the overall goal
We need to provide any tools that the agent needs to complete its goal:
- A tool to search the web for movies
- A tool to update the spreadsheet
These are good examples of tools that fetch data and take action
We’ll configure a trigger to run the agent on the first day of every month
As we saw in the tools example, the LLM will be called in a loop as it chooses to execute tools, makes use of the results, executes more tools…
…and so on until the goal is achieved

Some things to note about this pattern:

A variety of events can be used to trigger an agent
- When a chatbot receives a message
- When an email lands in your inbox
Agents typically use a reasoning LLM
- Specially trained to approach complex tasks with chain-of-thought
- It basically “talks to itself”
You can expand this into a multi-agent system
- A top-level coordinator LLM calls other agents like they are tools
- Each agent has instructions and tools to perform a specific task

Let’s build that movie desserts agent!

We’ll also use Zapier’s “copilot” agents along the way to help us make the table and agent!

Try ChatGPT’s Think longer (reasoning) mode:
- “Make a plan to increase visitors to my website”
- Compare level of detail with/without reasoning
Try ChatGPT’s Deep Research agent:
- “Find a cheap washing machine for a family of four”
- Searches the web and summarises in a few minutes
Extend your Desserts agent into a multi-agent system:
- Make a new Recipe agent that takes a dessert description and generates a recipe for it
- Use the Call an agent tool from your Desserts agent to call the Recipe agent for each dessert description
- Add a Recipe long-text column to your table so that the Desserts agent can include the recipe in each row
  - Remove and re-add the Zapier Tables tool to see the new column