The word agent is applied to everything from a chatbot with one tool to a long-running autonomous workflow. A useful way through the ambiguity is to separate the parts and be precise about what each one does.
One component among several
The LLM is one box in the architecture. The orchestration layer, tools and controls are ordinary software around it.
The LLM is the reasoning component
The model interprets a goal and the evidence in front of it, decides what to do next, emits structured requests such as tool calls, and summarises results. It does not execute anything, it does not remember between calls unless the application supplies state, and it can be wrong.
Orchestration is not the model
Frameworks such as Semantic Kernel provide the plumbing: registering tools, managing prompts, calling the model, handling its function-call responses and routing results back. They are deterministic software around a probabilistic model. Treating the framework as the intelligence leads to wrong assumptions about where reasoning, and therefore error, comes from.
Tools and APIs
Good tools are narrow, typed and validated. They distinguish reads from writes, define timeouts and failure behaviour, and run with the minimum permissions required. The tool boundary is where the application keeps control of what actually happens.
Memory and state
Agents need working state for the current task: the goal, evidence gathered, hypotheses considered and progress made. Explicit, persisted state is easier to inspect, resume and test than state that exists only inside a growing prompt. Longer-term memory is a separate design decision with its own privacy and correctness questions.
Enterprise knowledge, observability and governance
Retrieval is one capability an agent can use to bring enterprise knowledge into its reasoning. Observability records each reasoning step, tool call, input, output and cost so behaviour can be reviewed. Governance, meaning risk assessment, policy and approval, sits outside the model and decides what the agent is permitted to do.
Key takeaways
- An agent is a system. The LLM is its reasoning component, not the whole.
- Orchestration frameworks are software that coordinates the model and tools; they are not the model.
- Explicit state and tool boundaries make agents inspectable and testable.
- Observability and governance are architectural components, not features to add later.