AI Agent Architecture: How Agents Use Tools, State and Guardrails
A timeout tells you the connection failed. It doesn’t tell you whether the system on the other end saved the change. For an AI agent that can create records or send messages, that missing confirmation can lead to the same action happening twice.
This is one reason agent architecture matters as much as the model. The application needs to know what the agent was allowed to do, which actions went through and what still needs checking. It also needs protection against instructions hidden inside the webpages and files the agent reads.
When an Agent Is Worth the Extra Complexity
Scheduled reports, fixed approval flows and database lookups rarely need an agent deciding every step. Ordinary software can carry out known rules reliably. A language model might help explain a result, but it needn’t control the workflow.
Agents become useful when the next action depends on evidence the software hasn’t seen yet. Troubleshooting code may require opening one file, discovering an unfamiliar dependency and looking somewhere else. The path changes as the investigation proceeds. Anthropic’s engineering guidance makes a similar distinction between fixed workflows and agents.
The model can propose its next step. Application code provides the tools and checks permission. State records progress between steps, while a tool’s external system confirms whether an action took effect. People should be able to review important proposed changes before they run.
If the system does need that flexibility, decide what completion means before deploying it. Put a limit on tool calls and running time. When a tool returns an uncertain result, the agent needs a way to reconcile it or ask for help. Otherwise an elegant demonstration can turn into a very expensive loop.
Tools Need Narrow Permissions
Tool access is where an agent begins to affect other systems. A function that reads one customer’s order is easier to secure than one that accepts arbitrary database commands. A tool description helps the model select the operation, but the application still decides whether the signed-in user can perform it.
Model Context Protocol (MCP) lets applications discover tools offered by other services. It can reduce the work of writing integrations. It doesn’t automatically give those tools permission to see every file or edit every record. Review what a connector can access before exposing it to an agent.
Tools can also affect one another. A search tool may return a webpage containing an instruction to send information through an email tool. The agent was asked to read the page. Its text cannot authorize a message. The execution layer must keep those two permissions separate.
Keep a Record That Survives a Failed Connection
When the agent creates a ticket, the ticketing system owns the result. The conversation alone cannot establish whether the ticket exists. Save the requested action and the destination’s identifier when it confirms the write. If the connection ends early, mark the outcome uncertain until a lookup settles it.
Checkpointing records the agent’s progress so it can resume after an interruption. LangGraph documents one approach. A saved checkpoint won’t tell you whether an outside service completed an unconfirmed request, so the resumed run still needs to reconcile that operation.
For APIs that support it, an idempotency key lets the receiving service recognize a repeated request as the same intended action. Amazon’s engineering explanation of safe retries goes into the tradeoffs. Without that support, check for the existing record before trying again. A blind retry can create duplicate tickets or send the same message twice.
Memory needs a different kind of check. An agent may remember last month’s customer policy correctly and still give wrong advice today. Retrieve the current policy from its source before acting on it. Keep approvals attached to the specific proposed action too. If the action changes after approval, it needs to be reviewed again.
Approvals Belong at the Point of Action
An instruction telling the model to ask before deleting records is useful guidance. The software that performs the deletion must enforce the rule. It needs to check the user’s identity and the exact target before the write happens. Approval for one operation should never silently apply to a changed request.
These boundaries matter even for read access. A search tool connected to many customer accounts should filter results using the authenticated user’s permissions. The model must not be allowed to select another tenant simply by supplying an account number. AWS’s guidance on tenant isolation explores that risk.
In practice, reading public documentation may need ordinary source checks. Sending a message or changing production data requires a more consequential authorization decision. The application can pause for human review where appropriate, but it must still validate the tool arguments and destination when execution resumes. OWASP’s agent security guidance covers those independent controls.
When a Webpage Tries to Give the Agent Orders
Agent tools often return webpages, search results and documents written by people outside the application. Those documents may contain instructions. A webpage might tell an assistant to ignore the original task and send private data somewhere else.
This is prompt injection. Security researcher Simon Willison describes the particularly dangerous combination of untrusted input, access to private information and a way to send it out. The assistant may need all three for legitimate work. The problem arises when content from one channel gets treated as an instruction with authority over the others.
Protect the tools at execution time. A fetched page can supply facts to examine. It cannot expand the user’s permissions, change the recipient of a message or approve a destructive action. Test that boundary with harmless hostile content in an isolated environment.
Handoffs Need More Than a Summary
Some applications split work between agents. One investigates a problem while another prepares a change. The second agent needs the evidence, its source and any uncertainty that might alter the decision. Passing along a neat paragraph without those details can make a weak conclusion sound settled.
OpenAI’s orchestration guidance describes ways specialists can hand work back or pass control onward. Start with one agent when it can do the job. More agents mean more opportunities to lose context or confuse which component may act.
Follow a Code Change Through to the Repository
For a maintenance task, finding the bug and deploying a fix are separate responsibilities. An agent can read source files using limited access and propose an edit. Before it writes, the application must confirm the repository and branch, then check the exact operation the user authorized.
- Identify the requested repository and issue from a trusted source.
- Read the relevant code and keep links to the files that support the diagnosis.
- Prepare a proposed change and identify which tests would catch regressions.
- Check the target and permissions before writing, then verify the resulting commit or pull request in the destination system.
The commit identifier matters more than the model’s final “Done.” If creating the branch times out, check whether it already exists. If a repository comment tells the agent to copy credentials elsewhere, treat it as untrusted content. A good workflow can stop and report uncertainty without inventing a success.
Test What the Agent Did, Not What It Said
The assistant can confidently report success after a failed tool call. It can also report failure when the external system completed a change but lost the confirmation. Engineers need a trace of the requested actions and tool responses, with private data excluded wherever possible. The trace should identify the destination’s confirmed result.
Build tests around the uncomfortable cases. Make a tool time out immediately after committing a write, then resume the agent and check for duplicates. Change a proposed action after it receives approval and confirm that the previous approval no longer applies. Ask for a record outside the test account’s scope and verify that the tool refuses access.
Run prompt injection tests through the same channels that supply real untrusted material. Place harmless adversarial instructions inside a retrieved document and confirm they cannot expand tool permissions. Cap the run’s calls and execution time so an unresolved task stops cleanly. Use test accounts and isolated systems for this work.
Measure the full time to a verified result, including retrieval and recovery. A quick model response can mask minutes of failed tool calls or manual correction. Keep failures in the evaluation set after fixing them so a later update doesn’t reintroduce the problem.
What Happens if the Record Changes During Approval?
Long tasks introduce another failure path. An agent can prepare an update, leave it waiting for review and return after somebody else changed the record. The original proposal might now overwrite newer information.
Concurrency control prevents that by checking the record’s version before committing a write. If it changed, the application refuses the old update. The agent can retrieve the current state and prepare a new proposal for approval. An idempotency key protects against repeating the same action, while a version check protects against acting on an outdated decision. Both can be necessary.
These safeguards belong in the software executing the work. CMX’s AI automation services cover integrating agents with existing systems, including the checks that determine when an action may proceed.
