Model selection & granularity

• Match model capability to the agent role. Use smaller, cheaper models for high-volume routing/triage and larger, more capable models for specialist reasoning or multimodal work.
• Prefer the latest provider-recommended model classes for specialist reasoning when available (example: OpenAI GPT-5.4 as the current high-capability class at the time of writing). Benchmark latency and cost for both router and specialist roles on representative workloads.
• Recent community signals and vendor updates show managed agent hosting and model-native harnesses are now common; test managed offerings for latency, cost, and sandboxing trade-offs before committing to a deployment architecture (see provider docs linked in Tooling & libraries).
• For routers, optimize for classification accuracy and latency; for specialists, optimize for reasoning depth and tool integration (code execution, file system access, multimodal inputs).

Agent authorization and credentials

• Two common authorization patterns: per-user delegation (agents act using the end-user's credentials) and fixed-agent credentials (agents use a service account or fixed key). Choose per workload.
• Always explicitly include an auth/credential field in handoff metadata when routing to agents that will access protected resources.
• When passing user credentials, include scope-limited tokens and refresh/expiration metadata to enable safe short-lived access.

Handoff protocol

Handoffs transfer control from one agent to another. The handoff carries:

• target agent — which specialist to invoke
• context — accumulated conversation state (or pointer/token to external store)
• instructions — what the target should focus on
• metadata — routing reason, priority, constraints
• auth — credential or auth type required (e.g., "user-delegated" or "service-account")
• model_hint — optional preferred model or model class for the target

Notes:

• Keep the schema vendor-agnostic (target, contextPointer, instructions, metadata, auth, model_hint) so routing logic can span multiple providers and platforms (OpenAI, Vercel AI Gateway, Anthropic/Claude deployments).
• Prefer pointers (contextPointer) for long-lived context to avoid token bloat; fetch and rehydrate only the slices the specialist needs.

State management patterns

Shared state approaches:

• Conversation history passthrough — Simple, stateless agents; token bloat on long chains

• External store (Redis/DB) — Persistent, queryable, can store pointers and compact summaries; adds infra and security considerations

• Vector DB + persistent memory — Retrieval-augmented agents; requires index maintenance

• Structured context object — Typed and compact; requires schema discipline and migrations

• Tool-based state — Agents read/write via tools (file store, DB APIs); natural LLM interface but needs transactional semantics if concurrent

• Pattern: prefer pointers (contextPointer) + on-demand retrieval for long-lived chains to avoid token bloat.

• Use a dedicated vector DB and memory layer for agents that require personal or long-lived context; enforce access controls per-credential.

Async & long-running tasks

• Adopt task lifecycle semantics: start_async_task, check_async_task, update_async_task, cancel_async_task, list_async_tasks. Persist task IDs, status, provenance, expected duration, and owner in your external store.
• LangChain's Deep Agents work and related community implementations formalize async subagent lifecycle semantics for background work (start/check/update/cancel/list) — adopt similar lifecycle semantics when implementing supervisors and UIs (see LangChain Deep Agents: https://blog.langchain.com/deep-agents).
• OpenAI's Agents SDK now includes native sandbox execution primitives suitable for long-running, file-based workflows where an agent must inspect or modify a controlled workspace. Use sandboxed agents to isolate untrusted execution and restrict filesystem/tool access; see the OpenAI sandboxes guide for primitives such as SandboxAgent and SandboxRunConfig (OpenAI docs: https://developers.openai.com/api/docs/guides/agents/sandboxes).
• Design supervisors to avoid blocking on long-running subagents: allow the supervisor to continue interacting with the user, poll or subscribe to task completion events, and provide follow-up instructions to running tasks when necessary.
• Ensure idempotency and cancellation semantics for background tasks. Provide meaningful timeouts and resource limits; surfaced task metadata should include expected duration and resource class.

Skills, Fleet, and reuse

• Reuse "skills" (shareable, versioned behavior modules) across fleets of agents. Skills capture domain knowledge, tools, and test suites that you can attach to agents for consistent behavior.
• OpenAI's Custom GPTs (GPTs) and LangChain Fleet patterns provide packaging and distribution patterns; treat skills and GPTs as code: version them, include automated tests, and attach identity/permission metadata so they can be safely shared across teams.
• Agent identity and permissions: assign each agent an identity and an access policy (which skills it may use, which credentials it may request) and enforce at runtime.

Workflow

Step 0: Define evaluation and safety requirements before building

• Draft an agent evaluation checklist (routing accuracy, task completion, guardrail precision) and include both offline and online tests.
• Specify telemetry, failure modes, and human escalation paths. Consider integrating safety bug-bounty signals and anomalous-behavior reports into your issue triage.

Step 1: Define your agent graph

Map out which agents exist and how control flows between them.

Step 2: Implement agents with focused instructions

• Keep instructions narrow and deterministic where possible. Provide explicit success/failure signals and named outputs (e.g., {status: "APPROVED"}).

Step 3: Run the orchestration loop — secure execution

• For agents that execute code or write files, run untrusted code in isolated sandboxes. Use timeouts and resource limits. OpenAI's Agents SDK provides sandbox primitives and examples for controlled workspaces — consult the SDK docs for configuration and runtime examples (https://developers.openai.com/api/docs/guides/agents/sandboxes).
• Signal streaming data with agent identity so consumers can attribute output to the correct agent.

Step 4: Add guardrails and safety processes

• Align guardrails with authoritative model behavior specifications and monitor for agentic vulnerabilities (prompt injection, hidden tool-use, and data exfiltration).
• Vendor guidance (OpenAI, Anthropic) and community safety patterns should inform guardrail ordering and policy enforcement.

Tooling & libraries

• OpenAI Agents SDK (Apr 2026): model-native harness + native sandbox execution. See the SDK docs for SandboxAgent, SandboxRunConfig, and client examples (e.g., UnixLocalSandboxClient) for file-based workflows and isolated execution (OpenAI docs: https://developers.openai.com/api/docs/guides/agents/sandboxes).
• LangChain Deep Agents (blog + docs): async subagents, planning tools, subagent lifecycle patterns, and Agent Protocol concepts for cross-deployment interoperability (LangChain blog: https://blog.langchain.com/deep-agents; docs: https://docs.langchain.com/oss/python/deepagents/overview).
• Vercel / third-party hosting: managed AI hosting (Vercel AI, Cloudflare Agent Cloud, etc.) can simplify deployment of agent fleets; evaluate provider sandboxes and hosting isolation guarantees before relying on managed runtimes.
• Prefer a stable, vendor-agnostic handoff schema in your orchestration layer. Normalize vendor-specific fields in an adapter layer and implement provider adapters for OpenAI, Anthropic (Claude), Vercel AI Gateway, and any in-house agent servers.
• Use vector DBs, Redis, or managed memory layers that integrate with your agent harness. Monitor open-source model releases and evaluate them on your agent-specific tasks before committing to a vendor.

Examples

Example: OpenAI Agents SDK — SandboxAgent (Python)

The OpenAI Agents SDK demonstrates creating sandboxed agents that run in a temporary workspace and operate only on files provided to that workspace. Consult the OpenAI sandboxes guide for SDK primitives and example clients. Example (pseudocode):

• Create SandboxRunConfig that limits filesystem access and defines allowed tools
• Start SandboxAgent with the RunConfig and a UnixLocalSandboxClient
• Submit a task payload and stream results back to the orchestrator

Refer to the OpenAI sandboxes docs for exact code samples and SDK version guidance (developers.openai.com API docs).

Example: Async background task lifecycle (pattern)

• Supervisor launches a long-running research task and receives a taskId immediately
• Persist taskId and owner in your DB
• Supervisor returns to the user and continues the conversation
• Provide endpoints to checkAsyncTask(taskId), getAsyncTaskResult(taskId), updateAsyncTask(taskId), cancelAsyncTask(taskId)

Cross-vendor routing

• Keep the handoff payload vendor-agnostic and normalize vendor-specific fields in the adapter layer. Where possible, target Agent-Protocol-compliant endpoints for remote agents (LangChain Agent Protocol).

Decision tree

Is one LLM call enough? ├── Yes → Single agent with tools (no orchestration needed) └── No ├── Is the flow linear (A → B → C)? │ ├── Yes → Pipeline pattern (chain agents sequentially) │ └── No → Router + Specialists or Loop patterns based on routing needs

Additional checks:

• Which model class for each agent? (fast/mini for routers, larger for specialists)
• Which authorization pattern is required? (user-delegated vs service-account)
• Do any agents execute code? If yes, run in sandboxes with resource limits
• Where is state stored? Use pointers for long-lived context and vector DBs for retrieval memory
• Do you need shareable skills across teams? Consider Fleet + skills patterns for governance and reuse

Edge cases and gotchas

• Infinite loops: Always set maxTurns — agents handing off to each other can loop forever
• Context bloat: Summarize before handoff for long chains
• Error propagation: Orchestrator must handle specialist failures gracefully
• Model mismatch: Router agents can use cheaper/faster models; specialists may need more capable ones
• Streaming across handoffs: The stream must indicate which agent is responding
• Tool overlap: If two specialists share a tool, ensure they don't conflict on shared state
• Latency multiplication: Each handoff adds a round-trip; budget per-hop latency accordingly
• Guardrail ordering: Input guardrails run before the agent; output guardrails after — both can block
• Agentic vulnerabilities: Monitor for prompt injection, hidden tool-use, and data exfiltration

Evaluation criteria

• Routing accuracy — % dispatched to correct specialist
• Task completion rate — % of tasks resolved without human escalation
• Handoff efficiency — average number of handoffs per task
• Latency budget — total wall-clock time
• Error recovery rate — % of tool failures gracefully retried or escalated
• Context preservation — does the specialist have all needed info after handoff?
• Guardrail precision — false positive rate on blocked legitimate requests

Research-backed changes

• OpenAI (Apr 15, 2026): Agents SDK update adds a model-native harness and native sandbox execution; the SDK docs describe sandbox agents and run configurations (SandboxAgent, SandboxRunConfig) and include example clients for local sandboxes (https://developers.openai.com/api/docs/guides/agents/sandboxes).
• LangChain Deep Agents (Jul 30, 2025): formalizes async subagents, planning tools, and subagent lifecycle semantics that are useful patterns for designing supervisors and background tasks (https://blog.langchain.com/deep-agents).
• Continue to monitor vendor changelogs, LangChain releases, and the Agent Protocol repo for protocol-level changes that affect cross-vendor routing and long-running orchestration.