How an AI agent actually works, in five pieces

You hand an agent a task and it comes back with the work done. It is easy to assume something mysterious happens in between. Nothing does. An agent is five ordinary pieces stacked on top of each other, and once you can see them, you can tell why an agent works well, or why it quietly goes wrong.

1. The model only guesses the next word

A large language model (LLM) does one thing: given some text, it predicts the next word. Chain enough of those guesses together and you get a sentence, a plan, a piece of code.

That is also the limit. On its own it cannot look anything up, cannot press a button and cannot remember yesterday. Ask it for today's gold price and it will give you a plausible number. It is guessing.

2. The prompt is one sheet of paper

The model remembers nothing between calls. Everything it knows is on the sheet you send it this time:

Role:      You are Mai, a friendly support agent.
Rules:     Refund up to $20. Verify the order first.
Tools:     check_order(id), refund(id)
History:   (earlier turns)
New:       Order #123 is 3 days late. Refund me!

Role, rules, the list of tools and the conversation so far are all just text added to the sheet. Nothing here is magic, and the whole sheet is sent again on every call.

3. The loop turns a chatbot into an agent

So far the model can only ask for a tool by writing a request. The loop does the rest:

1. Think. The model reads the sheet and decides: call a tool, or answer.
2. Act. If it asked for a tool, the program around it runs the tool for real.
3. Observe. The result is pasted back onto the sheet.
4. Go back to step 1. When the model asks for no more tools, the loop stops.

That is the whole difference between a chatbot and an agent. The loop also needs a ceiling. Without one, an agent that thinks "not enough yet" will run forever, so real runtimes cap the number of turns and tool calls. A typical default is a couple of dozen turns and a few dozen tool calls.

4. Tools are hands, skills are recipes

The model never acts directly. It writes a request, and the host program decides whether to run it. A sensible host checks three things before anything happens:

• Does this tool exist?
• Are the arguments well-formed?
• Is the agent allowed to use it?

The default is deny. A tool is usable only once someone grants it. When a request is blocked, the reason goes back to the model, so it can change course instead of crashing.

Tools say what the agent *can do*. A skill says *how* to do it, for example how your company's events table is laid out. The agent sees a one-line summary of each skill and loads the full text only when it needs it. Memory is a third thing: what the agent already knows from before.

5. Context is a desk with limited space

Every result pasted onto the sheet takes room, and the sheet has a fixed size. Run long enough and it overflows. So a good agent tidies up as it goes:

• Prune old tool results once the desk is about half full. It is cheap and needs no model call.
• Summarise the early part of the conversation when the desk is mostly full. This costs one model call.
• Shelve huge outputs. Keep a ticket on the desk and fetch the full text only if it is needed.
• Pin the plan outside the history, so that summarising can never swallow it.

Guardrails catch one kind of failure each

Each guardrail targets a specific way an agent goes wrong: running forever, repeating the same failing call, declaring it is done without checking, forgetting the plan, flooding the desk, waiting on a slow job, deciding something irreversible on its own, losing its model provider mid-run, forgetting yesterday, or reaching outside its folder. Each is a small separate piece. Remove any of them and the agent still runs. It just handles fewer situations.

Try it yourself

We turned all of this into an interactive page, where you can guess words as the model, grant and deny tools, and try to keep an agent alive for 20 turns while the desk fills up. Try it here.

If you want to see these pieces working in a real agent, start with the Soot docs: trysoot.com/docs.