Most agentic AI implementations collapse in production because they force a single context window to act as strategist, runtime engine, and quality control inspector simultaneously. When a model tries to handle high-level reasoning while parsing raw tool outputs and formatting edge-case payloads, prompt bloat and hallucinated parameters inevitably follow.
Reliable autonomous behavior requires structural separation of concerns. Instead of relying on a monolithic prompt, production systems must split cognition into distinct phases: strategic planning, isolated tool execution, and reflective critique, all coordinated by an explicit state container.
+-----------------------------------------------------------------------+
| AgentState |
| - Goal - Memory / Context Window - Observability Traces |
+-----------------------------------------------------------------------+
| ^ ^
v | |
[ 1. Planner ] [ 2. Executor ] [ 3. Critic ]
- Emits strict JSON - Dispatches Tools - Analyzes Traces
- Formulates Strategy - Resolves Calls - Finalizes Artifact
|
v
{ Execution Runtime }
- Safe Calculator
- Knowledge Retrieval
- JSON Extractor
- File Writer
The Core Triage: Decoupling Strategy, Execution, and Critique
Monolithic prompt engineering fails when task complexity scales. Splitting the cognitive pipeline into three dedicated personas stabilizes execution boundaries and enforces determinism across steps.
| Persona | Core Responsibility | Primary Output Mode |
|---|---|---|
| The Planner | Evaluates raw goals, maps sequential dependencies, establishes constraints. | Strict, deterministic JSON schema |
| The Executor | Dispatches tool calls, handles runtime payload validation, digests tool feedback. | Interleaved Tool Call / Response Loop |
| The Critic | Inspects intermediate steps, traces errors, formats and refines final output. | Grounded response validation |
1. The Planner
The Planner focuses solely on strategy. It does not touch runtime APIs or write final deliverables. It processes the operational goal and outputs a structured execution plan. By constraining this phase to strict JSON outputs, downstream workers receive predictable, machine-readable specifications.
2. The Tool-Using Executor
The Executor turns the plan into action. It interfaces directly with the execution environment, translating abstract steps into concrete function arguments. It listens for model-generated tool calls, dispatches internal routines, and feeds the resulting tool payloads back into context until the step reaches completion.
3. The Critic
The Critic handles validation. It cross-examines the original goal, the execution traces, and the draft artifacts. By using execution traces as concrete ground truth, the Critic catches arithmetic errors, omissions, or malformed data before declaring a job finished.
State Observability and Runtime Plumbing
Decoupling roles requires an explicit data layer to manage information handoffs. The AgentState object anchors the system runtime by holding the goal, the active memory window, and execution traces.
Agent Execution Lifecycle:
[Goal Input]
│
▼
┌───────────┐ Structured JSON Plan
│ Planner │ ──────────────────────────────┐
└───────────┘ │
▼
┌───────────┐ Tool Execution Loop ┌───────────┐
│ Trace Log │ ◄────────────────────── │ Executor │
└───────────┘ └───────────┘
│ │
│ Intermediate Output │
└─────────────────────────────────────┤
▼
┌───────────┐
│ Critic │
└───────────┘
│
▼
[Final Artifact]
This trace record provides deep observability into model decisions:
- Auditability: You can inspect the exact function arguments generated during execution.
- Deterministic Debugging: When an execution fails, traces pinpoint whether the failure occurred during planning, tool formatting, or critique.
- Context Isolation: Workers ingest only the trace data relevant to their specific role.
The Robust API Wrapper
A frequent source of API failures in tool-based pipelines is misconfigured parameter payloads. Passing an empty tool_choice or providing tool configuration objects when no actual tools are exposed triggers immediate 400 Bad Request errors from OpenAI endpoints.
A resilient client wrapper intercepts each call, dynamically stripping tool_choice parameters unless executable tools are explicitly loaded for that invocation.
Deterministic Tool Design
Tools should operate like standard microservices. They must accept strongly typed inputs and return structured dictionary responses, ensuring their outputs remain machine-readable and easy for the model to parse.
+--------------------------------------------------------------------+
| Tool Interface Map |
+-----------------------------+--------------------------------------+
| Function | Operational Role |
+-----------------------------+--------------------------------------+
| Safe Calculator | Isolated mathematical computation |
| Keyword Knowledge Retrieval | Internal documentation query lookup |
| JSON Extractor | Structured data isolation and parse |
| File Writer | Output artifact persistence to disk |
+-----------------------------+--------------------------------------+
Dynamic tool execution uses a two-part design:
- Tool Schemas: Strict definitions passed to the model endpoint to specify property types, required fields, and functional descriptions.
- Python Dispatch Registry: A lookup table mapping the function name string directly to its callable implementation.
Model Output
│
▼
┌──────────────────────┐
│ name: "calc" │
│ args: {"expr":...} │
└──────────────────────┘
│
▼
┌──────────────────────┐
│ Dispatch Registry │ ───► Python: safe_calculator(**args)
└──────────────────────┘
When the Executor detects a tool call in the model response, it routes the payload through the registry, catches execution exceptions internally, and formats the output dictionary directly back to the message thread.
Orchestrating the End-to-End Lifecycle
The full execution pipeline runs through an orchestrator function that manages state transitions:
[Initialize State] ──► [Generate Plan] ──► [Tool Execution Loop] ──► [Run Critique] ──► [Final Artifact]
- Initialize State: The pipeline accepts the user prompt, stores it in
AgentState, and instantiates logging buffers. - Plan: The Planner generates a strict JSON roadmap based on the core goal.
- Execute: The Executor runs the roadmap steps, calling the safe calculator, retrieval functions, and JSON extractors as needed.
- Critique: The Critic inspects the final draft alongside the run traces, ensuring all requirements are met.
- Finalize: The validated deliverable is persisted to disk using the file writer tool.
Production Horizons
This structured pipeline delivers predictable agent execution, but deploying mission-critical systems requires pushing these concepts further:
- Automated Tool Retries: Wrapping the tool dispatch loop with exponential backoff and schema repair prompts when parameters fail type checks.
- Hierarchical Sub-Agents: Breaking the single Executor role into parallel sub-agents for concurrent data gathering.
- Hybrid Memory: Integrating symbolic state stores with semantic vector recall to maintain long-horizon task context.
- Continuous Evaluation: Building automated test harnesses to benchmark planner accuracy, tool efficiency, and critique quality across model versions.