Agentic Systems & LLM Application Engineering

1. Earn the right to be agentic

An agent is a system in which a model chooses at least part of the action sequence at runtime. That flexibility is useful when the task is open-ended, the correct path depends on observations, and tool selection cannot be exhaustively encoded. It also expands the state space: more trajectories, model calls, permissions, latency, cost, and failure combinations. “Agentic” is therefore a design choice, not a maturity level.

Apply the uncertainty–consequence test. Agent value rises with path uncertainty: research, diagnosis, codebase exploration, or heterogeneous support requests. Required control rises with consequence: moving money, deleting data, changing production, contacting a customer, or disclosing sensitive information. High uncertainty plus high consequence calls for a bounded agent that prepares evidence and a plan, then an explicit approval before action.

Define success and terminal outcomes

“Helpful response” is not an operational contract. Define allowed terminal states such as completed, needs_user_input, awaiting_approval, blocked_by_policy, budget_exhausted, and dependency_failed. For each, specify what the user sees and whether resumption is possible. A loop without a terminal-state model is a reliability bug waiting for traffic.

2. Choose an orchestration pattern from the control problem

Framework names matter less than who decides the next step and where state lives. Model the workflow as states, events, guarded transitions, side effects, and terminal states. Then choose a framework—or plain code—that expresses this model clearly.

Worked example: bounded refund investigation

Consider a support workflow that may inspect an order and draft a refund, but cannot issue it without policy checks and approval above a threshold. A useful graph is:

ALLOWED = {
    "validated": {"looked_up", "needs_user_input"},
    "looked_up": {"proposed", "not_found"},
    "proposed": {"policy_denied", "awaiting_approval", "approved"},
    "awaiting_approval": {"approved", "rejected"},
    "approved": {"completed", "dependency_failed"},
}

def transition(state, next_status):
    if next_status not in ALLOWED.get(state["status"], set()):
        raise ValueError("invalid workflow transition")
    return {**state, "status": next_status}

The model may propose an action and rationale, but deterministic code verifies policy and authorization. The approval stores the exact proposed action, resource ID, amount, policy version, and expiry. Execution uses an idempotency key. The system never interprets approval as permission for a later, altered action.

3. Treat tool calls as untrusted requests

Function calling is a protocol round trip: the application describes tools; the model emits a structured request; application code validates and authorizes it; the application executes the operation; and a structured result returns to the model. Both OpenAI and Anthropic document this separation: the model proposes arguments, while client-side application code performs the function. Never let the model’s selection bypass normal service controls.

A strong tool contract

• Narrow intent: get_order_status is safer and easier to select than run_api_request.
• Constrained schema: use enums, bounded lengths/ranges, required fields, and reject unknown fields where supported.
• Trusted identity: derive tenant, actor, and scopes from authenticated context; do not accept them as model-controlled arguments.
• Clear effects: distinguish read, reversible write, irreversible write, and external communication.
• Idempotency: mutation tools accept or derive a key tied to workflow and semantic operation.
• Typed result: separate ok, retryable error, permanent error, policy denial, and not-found; keep user-safe and operator detail distinct.
• Limits: server-enforced timeout, pagination, result-size cap, rate/quota, and redaction.

{
  "name": "prepare_refund",
  "description": "Create a reviewable refund proposal; does not issue funds.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string", "pattern": "^ord_[A-Za-z0-9]+$"},
      "reason": {"type": "string", "maxLength": 500},
      "amount_minor": {"type": "integer", "minimum": 1}
    },
    "required": ["order_id", "reason", "amount_minor"],
    "additionalProperties": false
  }
}

Structured-output support can guarantee or improve schema conformance depending on provider and mode, but schema validity is not semantic validity. A perfectly formed order ID can belong to another tenant; a valid amount can exceed the refundable balance. Revalidate business invariants at execution time. Keep tool descriptions accurate and treat third-party descriptions as untrusted metadata.

Return errors the orchestrator can act on

A timeout, invalid input, expired credential, policy denial, and missing record need different control flow. Avoid returning a prose blob that the model must reinterpret. Use stable error codes, retryability, safe user message, and correlation ID. Do not expose tokens, stack traces, raw database errors, or private tool results to the model unnecessarily.

4. Design for interruption and replay

Multi-step systems fail between steps. A process can crash after an external write but before recording success; a user can approve hours later; a provider can time out after completing a request. Durable execution requires explicit persisted state and replay-safe side effects—not merely “retry three times.”

Checkpoint at meaningful boundaries

Persist input references, validated state, chosen route, model/prompt/tool versions, tool request and result references, budget consumption, approvals, and terminal status. Store only the content required for recovery and audit; encrypt or reference sensitive payloads and apply retention. LangGraph’s official persistence documentation describes thread-scoped checkpoints, pending writes, fault recovery, and time-travel debugging. Its interrupt documentation notes that state is saved for human-in-the-loop resumption; side effects before an interrupt must be idempotent because the node can replay.

Retry ownership

Place retries at one layer whenever possible. SDK, HTTP client, orchestrator, queue, and tool service each retrying can multiply calls. Retry only transient failures, with exponential backoff, jitter, a deadline, and a total attempt budget. Respect provider retry hints. Do not retry policy denials or deterministic validation failures. If a request has ambiguous outcome, reconcile using an operation key or read-before-retry.

Termination is a product feature

Enforce maximum wall time, model calls, tool calls, repeated identical calls, replans, tokens, and monetary budget. Detect no-progress cycles using normalized action/result fingerprints. On exhaustion, preserve a partial result and missing requirements where safe. A hard “something went wrong” after ten hidden retries wastes both evidence and trust.

5. Separate context, state, and memory

Conversation history is not a database, and a vector store is not automatically memory. Use three distinct concepts:

• Working context: bounded messages, evidence, tool results, and instructions needed for the current model call.
• Workflow state: authoritative typed fields required to resume and enforce transitions.
• Long-term memory: intentionally retained facts or summaries available across sessions, with provenance, consent, correction, and deletion.

Keep raw authoritative values in workflow state; format them into prompts at call time. Summaries are lossy and should carry source references and version. Treat retrieved memory as untrusted context, because it can be stale, incorrectly attributed, or poisoned. Enforce tenant/user boundaries before retrieval. Do not store secrets or sensitive tool results merely because they might help a later response.

Manage the context window as a budget

Reserve space for system/tool schemas, current request, evidence, and output. Drop irrelevant history; summarize older turns with explicit unresolved commitments; retrieve only task-relevant memory; and cap tool outputs. Compaction can lose an instruction or pending constraint, so evaluate long-running conversations and resume cases. Cache stable prefixes only when provider semantics, privacy, and version invalidation are understood.

Models and prompts are policy versions

Route by task risk and measured capability, not prestige. A small model may classify or extract; a stronger model may plan or resolve ambiguity; deterministic code validates and executes. Fallback is not simply “another model”: providers differ in schema subsets, tool-call formats, stop reasons, safety behavior, and context handling. Create a provider adapter and run the same contract/evaluation suite per route. Version system prompt, tool catalog, schemas, model snapshot, temperature/effort settings, and retrieval configuration with every trace.

6. Put authority outside the model

Prompt injection is a control-flow attack: untrusted content attempts to redefine instructions or induce tool use. Label and delimit external content, but do not depend on prompting alone. Authorization must be enforced by trusted code at the tool boundary. Give each workflow an explicit capability set and derive scopes from the authenticated actor, tenant, environment, and approved purpose.

Approval design

Request approval for the exact action, not a vague plan. Show target, effect, sensitive fields, cost/amount, environment, and why the action is requested. Bind approval to a content hash, policy version, actor, and expiry; invalidate it when material arguments change. Separate proposer from executor where risk warrants. Log the decision and make rejection a first-class state that can include feedback.

MCP is interoperability, not an authorization shortcut

The Model Context Protocol standardizes connections among hosts, clients, and servers and exposes resources, prompts, and tools. The current specification explicitly treats tool execution and arbitrary data access as high-risk and emphasizes consent and control. Its authorization specification requires resource-bound tokens for HTTP flows and forbids token passthrough. An MCP server still needs authentication, per-tool scopes, tenant isolation, input validation, output redaction, rate limits, audit logs, and downstream credentials distinct from inbound credentials.

7. Operate the agent as a distributed system

One user request may span router, model, retriever, several tools, approval wait, and final synthesis. Give it a trace ID and create spans for each meaningful step. Capture workflow/step name, model and prompt version, tool name, attempt, status, latency, token usage, candidate/argument size, cache outcome, budget remaining, and safe error category. Do not record raw prompts, tool inputs, or outputs by default; OpenTelemetry’s GenAI attribute documentation warns that message content can contain sensitive information.

Observe outcomes and trajectories

End-to-end task success alone hides inefficient or unsafe routes. Measure correct tool selection, argument validity, authorization denials, unnecessary calls, repeated calls, plan changes, step success, approval rate/time, recovery success, and terminal-state distribution. Pair quality with latency and cost. Sample failed, expensive, long, denied, and novel traces into evaluation datasets.

Use hierarchical budgets

Start with an end-to-end deadline and cost ceiling. Allocate child timeouts per dependency with room for response construction. Enforce model-call, tool-call, parallel-branch, output-token, and retry budgets. Cancel abandoned work when the client disconnects or the outcome becomes terminal. Streaming improves perceived latency but complicates error semantics: distinguish provisional progress from committed result, and never stream a claim of success before a side effect is confirmed.

Degrade by preserving the user’s goal

• If the planner fails, fall back to a fixed supported workflow or ask a targeted question.
• If a nonessential tool fails, return a partial result with the missing source named.
• If the primary model is unavailable, use a validated fallback only for routes it passed.
• If approval infrastructure is unavailable, pause; do not silently auto-approve or discard the proposal.
• If a mutating tool has ambiguous outcome, reconcile before telling the user to retry.

Syllabus checkpoint: API contracts, memory, routing, and model adaptation

Tool calling or function calling is a typed proposal from a model, not authorization to execute. Validate structured output against a schema, enforce policy in code, convert provider-specific tool messages into an internal contract, and test malformed arguments, unknown tools, duplicate calls, timeouts, and partial streaming failures. OpenAI, Anthropic, and Gemini expose different request/response details; the durable design is a narrow internal tool interface plus versioned adapters.

Short-term and long-term memory are different data products

Short-term memory keeps the current task coherent—recent messages, tool results, and summarized state—within an explicit token and privacy budget. Long-term memory retrieves durable facts or prior outcomes and therefore needs provenance, tenant isolation, retention/deletion, conflict handling, and a rule for when old memory is untrusted. Do not silently promote a model inference into a user fact.

Model routing and fallback

Route only on observable requirements such as modality, context size, risk class, tool reliability, latency target, or evaluated task performance. A fallback must preserve the contract: schema, safety policy, tool permissions, and user-visible limitations. Record route, model/version, trigger, tokens, latency, cost, and result so the policy can be evaluated rather than becoming invisible complexity.

Fine-tuning and distillation decision criteria

Prefer prompting, retrieval, deterministic rules, or tool design when the problem is missing knowledge, fresh data, or unsafe authority. Consider fine-tuning when a stable, well-labeled behavior or format repeatedly resists prompting and the evaluation set proves the gap. Consider distillation when a capable teacher produces a large, quality-controlled dataset and a smaller model can meet a measured quality threshold with better latency or cost. Include data rights, privacy, training/evaluation leakage, maintenance, rollback, and provider portability in the decision.

Interview playbook

Use AGENTS to structure a design answer:

1. A — Aim and authority: user outcome, risk, actor, tenant, and actions the system may never take.
2. G — Graph and state: deterministic baseline, model decisions, transitions, invariants, terminal states.
3. E — Execution contracts: narrow tools, schemas, validation, idempotency, typed results, deadlines.
4. N — Non-happy paths: retries, ambiguous writes, dependency failure, no progress, rejection, resumption.
5. T — Telemetry and tests: traces, versions, trajectory/outcome evals, safety cases, release gates.
6. S — Spend and safe rollout: token/tool/time budgets, shadow mode, approvals, canary, fallback.

Common traps

• Calling a prompt chain an “agent” without identifying any runtime decision.
• Letting model-generated tenant IDs, URLs, SQL, or scopes reach a tool unvalidated.
• Assuming structured output means business-correct or authorized output.
• Retrying a timed-out mutation without idempotency or reconciliation.
• Using conversation history as authoritative workflow state.
• Adding multiple agents when independent tools or parallel functions would suffice.
• Describing human approval without binding it to exact, expiring action arguments.
• Tracing sensitive content by default or omitting model/prompt/tool versions.

Question bank

Answer with control boundaries, measurable failure behavior, and the smallest justified architecture.

Proof artifact: a resumable approval-bound agent

Build a support workflow against a fake order service. It may read an order, retrieve a policy, propose a refund, and—only after deterministic policy checks—execute a low-value simulated refund or request human approval. Use synthetic data and fake funds. The artifact demonstrates controls, not production outcomes.

Steps

1. Write the authority matrix and terminal states first. Mark which transitions are deterministic, model-selected, or human-controlled.
2. Implement typed state and narrow read/proposal/execution tools. Inject actor and tenant server-side; add idempotency keys to execution.
3. Persist checkpoints, version prompt/model/tool schemas, and support resume after process restart.
4. Add end-to-end traces with safe metadata, per-step timing/tokens, attempts, budget, approval, and terminal reason.
5. Create an evaluation set for route choice, argument accuracy, policy result, trajectory length, final outcome, injection resistance, and user-visible recovery.
6. Run in shadow mode over synthetic scenarios, then enable only the reversible fake action behind a feature flag.

Metrics

• Task completion and correct terminal-state rate by scenario.
• Tool selection precision/recall, argument validity, and business-invariant pass rate.
• Unauthorized action attempts and cross-tenant disclosures—both must be zero in the test suite.
• Median/p95 model calls, tool calls, tokens, wall time, and example cost per terminal state.
• Duplicate side effects after replay—must be zero with idempotent fake execution.
• Checkpoint recovery and user-visible safe-degradation success rate.

Deliberate failure injection

Crash after the fake refund service commits but before the workflow records success; resume and prove no duplicate. Time out policy retrieval; exhaust the model-call budget; change a proposal after approval and verify approval invalidation; return malicious instructions inside a policy document; submit a model-generated tenant ID; and make the approval service unavailable. Capture trace and terminal behavior for each.

What to present

Present the state diagram, authority matrix, one tool schema, a crash-and-resume trace, an injection-denial trace, evaluation results by slice, and a short argument for which steps were deliberately kept non-agentic. Label every metric as a synthetic artifact measurement.

Chapter review

Reliable agent engineering is control engineering around probabilistic decisions. Keep authority in trusted code, express state and termination explicitly, make tools narrow and replay-safe, persist meaningful checkpoints, bound every resource, and evaluate both outcome and trajectory. Add autonomy only where runtime uncertainty creates measured value.

Glossary

Agent

A system in which a model chooses part of the action sequence at runtime.

Capability

An explicitly granted operation available to a workflow, distinct from what a model requests.

Checkpoint

A persisted workflow boundary from which execution can be inspected or safely resumed.

Idempotency key

A stable operation identifier that lets a service return the same semantic result without repeating the effect.

Interrupt

A deliberate pause that persists state and awaits external input such as approval.

Trajectory

The ordered sequence of model decisions, tool calls, observations, and transitions.

Working context

The bounded information supplied to one model call, not an authoritative state store.

MCP

Model Context Protocol, a standard for connecting hosts/clients with servers exposing resources, prompts, and tools.

Mastery checklist

• I can justify every model-selected step against a deterministic baseline.
• I can draw states, guarded transitions, terminal outcomes, and recovery paths.
• I can design a tool whose schema, authority, effect, and errors are explicit.
• I can explain ambiguous mutation outcomes and idempotent recovery.
• I can distinguish working context, workflow state, and long-term memory.
• I can bind human approval to one exact, expiring action.
• I can explain MCP’s integration value without outsourcing security to the protocol.
• I can define trace fields, privacy controls, and hierarchical budgets.
• I can evaluate trajectory quality as well as final task success.