How AI agents actually work: a non-technical primer.

"AI agent" is one of the most-used phrases in AI right now, and one of the most loosely defined. Here is the clearest one-sentence version, and then how agents actually work underneath it, so you can tell when someone is selling you one for the right reasons.

The sentence: an AI agent is not a smarter AI model; it is a loop that lets a regular AI model take an action, see what happened, and decide what to do next.

The interesting thing about an agent is not the model inside it. The interesting thing is the loop around the model.

What an AI agent actually is

A regular AI call works like this: you ask a question, the model answers, the conversation ends. One turn. Static. The model gives you the best answer it can with the information it has, and then it's done.

An agent works differently. You give it a goal ("find the three pension funds in Texas worth contacting this quarter") and the agent does not try to answer all at once. It works through the goal in steps. At each step, it can call a tool: a function the agent is allowed to invoke. The tool might search a database. It might look up the day's news. It might query an API. It might write to a file. It might send an email. The agent does not perform these actions itself; it asks the system to perform them and then reads the result.

So the cycle is:

1. The agent considers the goal and decides which tool to use first.

2. The system runs that tool and returns the result.

3. The agent reads the result and decides what to do next: call another tool, refine its approach, or finish.

4. Repeat until the goal is met or the agent gives up.

This is a loop. The agent is the orchestrator. The model decides what to do. The tools do the work. The system runs the tools and feeds the results back. The agent is "smart" not because the underlying model is smarter than usual, but because it can take action, see the result of that action, and adapt.

A concrete example

Pretend a marketing lead at an asset management firm needs a list of pension funds in Texas that have changed CIOs in the last six months.

Without an agent, the marketing lead opens four databases, runs four searches, copies the results into a spreadsheet, dedupes the rows, and hand-checks against the firm's CRM. Two days of work, mostly tedious.

With an agent, the lead types the goal and the agent works through it. It calls the pension-database tool to query Texas funds. It reads the results, picks the ones that name a CIO, and calls another tool to check whether the CIO has changed recently. It calls the CRM tool to check whether the firm already has contacts at those funds. It assembles a list, ordered by relevance, and writes it to a document. Fifteen minutes of compute, one minute of human review.

What made this possible was not a smarter model. The underlying model could not have answered the question by itself. It has no access to the pension database or the CRM. What made it possible was the loop, the tools the agent could call, and the system that runs the loop reliably.

Where agents work well

Agents are the right tool when the work has three properties.

The work requires multiple steps, and each step depends on the previous one. If the first query returns five candidates, the second query has to be shaped by those five. A regular AI call cannot do this. It answers once and stops. An agent can adapt as it goes.

Each step involves a judgment that genuinely benefits from a smart model. The model is choosing what to query, what to keep, what to drop, how to rank. If every step is a fixed rule, a regular script is cheaper and more reliable. Agents are worth their cost when there are judgment calls along the way.

The tools needed to complete the work are well-defined. "Search this database." "Look up this address." "Send a notification to this Slack channel." If the agent's available tools cover what the goal needs, the agent can complete the goal. If the tools are vague or missing, the agent flails.

Where agents fail

Three failure modes are common enough to name.

The tools are poorly defined. If the search tool returns inconsistent results, the look-up tool times out half the time, or the database is missing the fields the agent assumed, the agent fails reliably. Agents are only as good as the tools they can act on. Most production agent failures are tool failures, not model failures.

The work has one right answer that can be computed directly. If you need the sum of a column in a spreadsheet, do not use an agent. Use a formula. Agents are for judgment work, not arithmetic.

The loop is allowed to run forever. A well-built agent has a maximum number of steps, a maximum cost, and a clean way to give up. A badly-built agent will burn through a budget trying to complete a goal it cannot complete. The harness around the agent (limits, timeouts, fallbacks) is often more engineering work than the agent itself.

The harness around the agent

This is the part most agent demos do not show you, and it is where the engineering actually lives.

In production, an agent needs guardrails. It needs structured output so the system knows what the agent decided to do at each step. It needs typed schemas so when the agent calls a tool, the tool gets the inputs it expects. It needs retries with fallbacks so a single failing tool call does not crash the whole loop. It needs telemetry so the team can see what the agent did and what it cost. It needs route-layer guards in regulated industries so the agent cannot take an action that would violate a compliance rule.

When you read about a new "AI agent product," what you are actually buying is the harness around the model, not the model itself. The model is the same one everyone else uses. The harness is the differentiator. A good agent vendor talks about the harness as much as the model. A bad one talks only about the model.

What most people get wrong about agents

An agent is not a smarter model. Same model, different shape. The agent's apparent intelligence comes from the loop, not from extra training.

An agent is not a chatbot. A chatbot answers questions. An agent takes actions and adapts based on what the actions returned. They look similar in the UI sometimes; they are different categories of system underneath.

An agent does not need to be autonomous to be useful. Many of the most useful agents in production are "human-in-the-loop." They propose actions, a human approves, the agent executes. The loop and the tool-use cycle are still what makes the system valuable.

Bigger context windows do not replace agents. Some commentary in 2026 suggested that million-token context windows would make agents unnecessary. They have not. The reason is that agents are not about how much information the model can read; they are about giving the model the ability to act, see results, and adapt.

The decision rule

The next time a vendor pitches you an AI agent, ask one question: *what tools can the agent call, and what does the harness around the loop look like?*

If they can name the tools, describe the loop, and talk honestly about the failure modes (limits, retries, guardrails, human-in-the-loop), you are talking to someone who has built the thing. If they only talk about the model, the benchmarks, or the size of the context window, they are selling something they do not understand.

That is what an agent is. The model is the easy part. The loop and the harness are the rest of the work.

— Charlie