AI Research
Vol. 1·Friday, May 15, 2026
The Orchestration Stack: A Practitioner's Guide to Multi-Agent Coordination
Tips, corrections, or questions? support@omniscient.media
Tips, corrections, or questions? support@omniscient.media
Sign in to join the discussion.
Multi-agent orchestration is the discipline of coordinating multiple AI agents - each with its own instructions, tools, and context - to complete tasks that no single agent can reliably handle alone. It covers how work is divided, how agents communicate, how state is maintained across steps, and how control is returned to a human when the system reaches the limits of its confidence.
A single agent working in isolation hits predictable ceilings: context windows overflow on long tasks, a monolithic prompt degrading in adherence as instruction complexity grows, and a single failure point when anything goes wrong. The microservices lesson transfers directly to agent design: a monolithic prompt that grows to cover too many responsibilities degrades in coherence before it fails outright, and when it does fail, there is nowhere to isolate the cause. Google's ADK documentation captures this precisely, noting that an overloaded agent becomes "a jack of all trades, master of none" and that "error rates compound" as instruction complexity increases.[1]
But orchestration is not free. Every agent boundary you introduce creates a new seam: a handoff point where context can be lost, a trust boundary where permissions must be defined, and a failure surface where cascading errors become possible. The discipline is knowing where to draw those seams - and the frameworks below each make different bets about where the right answer lies.
Before evaluating frameworks, it helps to have names for the building blocks they all implement in different ways. Four primitives appear in every serious orchestration system:
Handoffs vs. agent-as-tool. The most fundamental design choice: does a specialist agent take over the conversation entirely, or does it serve as a bounded capability called by a manager that retains control? OpenAI's Agents SDK formalizes this distinction explicitly - handoffs transfer ownership of the reply; agent-as-tool keeps the manager responsible for the final answer.[2] The choice shapes tracing, auditability, and where errors surface.
Shared state. Agents in a workflow need a common surface to read from and write to without stepping on each other. LangGraph models this as a typed state graph - each node reads from and writes to a shared state object, and the graph structure defines which transitions are permitted.[3] Google ADK uses session.state with named output_key fields per agent, a deliberately narrow contract that reduces race conditions in parallel workflows.[1]
Control flow topology. Sequential pipelines (A finishes, then B starts), parallel fan-out/gather (multiple agents run simultaneously, a synthesizer aggregates), hierarchical decomposition (a top-level agent delegates sub-tasks recursively), and loops (a generate-critique cycle that runs until a quality gate passes) - these are distinct topologies with different debugging profiles and failure modes.
Human-in-the-loop (HITL). The mechanism by which an agent pauses execution and surfaces a decision to a human before proceeding. HITL is not a fallback; for high-stakes or irreversible actions - financial transactions, code deploys, operations on sensitive data - it is a first-class architectural requirement. LangGraph exposes explicit interrupt points; ADK implements HITL via custom approval tools; CrewAI routes decisions back to a deterministic Flow backbone before proceeding.
Across all four major frameworks, the same handful of patterns appear under different names. Understanding them framework-agnostically is more durable than memorizing any SDK's API.
The simplest and most debuggable topology: Agent A completes, writes its output to shared state, Agent B reads that output and proceeds. Because execution is strictly ordered and deterministic, trace analysis is straightforward - you always know exactly where in the chain a failure originated. This pattern suits document processing, data transformation, and any workflow where each step has a well-defined input contract from the previous one. ADK's SequentialAgent primitive and LangGraph's linear graphs both model this directly.
A central agent analyzes intent and routes tasks to specialists. The supervisor may retain ownership of the final answer (manager pattern) or transfer it entirely to the specialist (handoff pattern). OpenAI's Agents SDK documents both variants with explicit guidance on when each applies: use handoffs when a specialist should own the next response, use agent-as-tool when the manager should synthesize the final answer.[2] LangGraph's multi-agent supervisor library implements the tool-calling variant, which it describes as giving "more control over context engineering."[3]
Tasks that are independent of each other run simultaneously, then a synthesizer aggregates the results. The latency benefit is obvious; the risk is subtler - agents running in parallel share state, so each must write to a unique key to avoid race conditions. ADK's ParallelAgent handles concurrent execution but leaves the developer responsible for non-overlapping output keys.[1] A common application: running security, style, and performance audits on a pull request simultaneously, then consolidating findings into a single review.
When a task exceeds any single agent's effective context window, a top-level agent breaks it into sub-tasks and delegates them - potentially to agents that themselves delegate further. The key implementation detail is whether the sub-agent is invoked as a tool call (parent retains explicit control over when it fires) or as an autonomous sub-agent (parent hands off and waits). ADK's AgentTool wrapper supports the tool-call variant; AutoGen's layered Mixture of Agents pattern takes a different approach, passing previous-layer outputs to the next layer as synthesis material rather than as explicit sub-task assignments.[4]
A generator produces a draft; a critic evaluates it against defined criteria; the loop continues until the critic signals a pass condition or a maximum iteration count is reached. ADK distinguishes two variants - a correctness-focused pass/fail gate (Generator and Critic pattern) and a qualitative refinement cycle (Iterative Refinement pattern) - noting that the exit mechanism differs: the former exits on a pass signal, the latter on an escalate=True flag from an agent that judges quality sufficient.[1] Both are implemented via LoopAgent.
Four frameworks dominate serious production discussion in 2026. They are not interchangeable - each reflects a distinct philosophy about where control should live and how much abstraction is appropriate.
LangGraph positions itself as a "low-level orchestration framework" that trades abstraction for control.[3] Workflows are modeled as typed state graphs - nodes are agents or functions, edges are transitions, and the state schema is explicitly typed. This makes the graph introspectable, debuggable, and deterministic in ways that higher-abstraction systems are not. LangGraph supports single-agent, multi-agent, and hierarchical topologies in one framework, and integrates natively with LangSmith for observability. The tradeoff is that you write more code and maintain more graph configuration as workflows grow - a complaint some developers voice about graph-based frameworks generally.
LangGraph is MIT-licensed and free; LangSmith (the observability layer) is a separate commercial product. It is the most-used framework for production agent engineering in the LangChain ecosystem; LangChain claims its products are in use at 35% of Fortune 500 companies, a figure cited in its October 2025 Series B announcement.[5]
OpenAI's SDK provides the tightest integration with OpenAI's own model stack (GPT-5.5, o-series reasoning models) and is the natural choice for teams already on the Responses API. Its orchestration primitives are intentionally minimal: handoffs and agent-as-tool are the two core patterns, and the SDK's guidance is explicit about not splitting agents prematurely - "start with one agent whenever you can," adding specialists "only when they materially improve capability isolation, policy isolation, prompt clarity, or trace legibility."[2]
The SDK supports guardrails, resumable state, background mode, and streaming natively. Its observability story integrates with the broader OpenAI platform tracing tooling. The limitation is obvious: it is optimized for OpenAI models, and while third-party model providers are listed as supported, the ergonomics favor the first-party stack.
ADK is Google's most direct answer to LangGraph - an open framework that supports the full range of orchestration patterns via named primitives (SequentialAgent, ParallelAgent, LoopAgent) and integrates with Vertex AI and Gemini models. Its design philosophy emphasizes composition: primitives combine predictably, and the session.state contract is narrow by design to reduce coupling between agents.
The framework is notable for its explicit documentation of eight production patterns - from sequential pipelines through human-in-the-loop - which makes it the most thoroughly specified framework for practitioners coming from an enterprise background.[1] Like LangGraph, it skews toward developers comfortable with explicit state management; like OpenAI's SDK, it favors its own model stack in practice.
CrewAI's central argument is that most agentic deployments fail for the same reason: teams treat the system as a reasoning problem when it is actually a structural one. Smarter prompts and better models cannot compensate for an architecture that gives the agent no reliable boundaries, no deterministic escape path, and no audit trail.[6] Its answer is the "Agentic Systems" architecture - a deterministic Flow backbone (explicit code-level control over branching, state, and sequencing) into which intelligent Crews (collaborative multi-agent teams) are inserted at specific steps. Control always returns to the Flow when a Crew finishes. This separation of structure from intelligence is CrewAI's core bet: it makes systems observable, governable, cost-controllable, and auditable in ways that purely LLM-driven routing is not.
The philosophy has production evidence behind it. CrewAI reports approximately 2 billion agentic workflows processed through its enterprise platform, and its public case studies - DocuSign building sales outreach automation, US Department of Defense deployments - skew toward regulated industries where auditability is non-negotiable.[6] The tradeoff is opinionation: teams that want arbitrary graph topologies will find CrewAI's Flow model constraining compared to LangGraph.
Microsoft's AutoGen (v0.4, released January 17, 2025) is architecturally the most research-oriented of the group - its actor-model runtime, asynchronous message passing, and layered Mixture of Agents pattern reflect academic design patterns more than product ergonomics.[4] AutoGen's Mixture of Agents is particularly distinctive: worker agents are organized into layers (analogous to a feed-forward neural network), with each layer's outputs concatenated and passed to the next layer for synthesis. This enables emergent quality improvement through multi-model deliberation rather than explicit critic prompting.
AutoGen is best suited for research teams and engineers building novel agent architectures who want low-level runtime primitives. For teams prioritizing production deployment speed and framework stability, LangGraph or CrewAI are more practical starting points.
Framework | Control model | Best for | Watch out for |
|---|---|---|---|
LangGraph | Graph-based, explicit state | Complex custom topologies, teams wanting full control | Graph maintenance overhead at scale |
OpenAI Agents SDK | Handoffs + agent-as-tool | OpenAI-native stacks, rapid prototyping | First-party model bias, limited topology expressiveness |
Google ADK | Named primitives, session state | Vertex AI / Gemini integration, enterprise pattern library | Google cloud lock-in in practice |
CrewAI | Flow backbone + Crew intelligence | Regulated industries, production reliability, auditability | Less flexibility for non-standard topologies |
AutoGen | Actor model, async messaging | Research, novel architectures, multi-model ensembles | Steeper learning curve, less production polish |
The honest answer is that framework choice is less consequential than architectural clarity. CrewAI's CEO João Moura puts the point bluntly: the teams that fail are not using the wrong SDK - they are "optimizing for agent intelligence" when the real gap is "system architecture."[6] OpenAI's documentation makes a similar point from the opposite direction, warning against premature decomposition: adding specialists creates more prompts, more traces, and more approval surfaces, and is only justified when the seam materially improves the system.
The patterns above describe the happy path. Production adds five requirements that none of the framework documentation front-pages as prominently as it should.
Observability from day one. Without trace-level visibility into every agent decision, handoff, and tool call, debugging a multi-agent failure is nearly impossible. All four frameworks have observability integrations (LangSmith, OpenAI platform traces, Cloud Trace for ADK, CrewAI's enterprise platform) but these are often treated as after-thoughts. They should be the first dependency you wire up.
Explicit trust boundaries. When one agent calls another, which permissions does the callee inherit? In most frameworks, this is left to the developer - there is no automatic permission scoping at handoff boundaries. Failing to define this explicitly is one of the primary attack surfaces for prompt injection propagation across agents, a topic covered in depth in our LLM Security deep-dive.
State persistence across failures. Long-running workflows will fail mid-execution. Whether you are using LangGraph's checkpoint system, OpenAI SDK's resumable state, or ADK's session persistence, the question of "where does the workflow restart after a crash?" needs an answer before you go to production - not after.
Cost discipline. Multi-agent workflows compound token costs. A generate-critique loop with a liberal exit condition and an expensive model can spend an order of magnitude more than a single-agent equivalent. CrewAI's Flow architecture addresses this by scoping agency to specific steps - you pay for reasoning only where it adds value.
Model-agnostic interfaces. Tying your orchestration architecture to one model provider's SDK is a deployment risk. Models change, APIs change, and the model that performs best today may not be optimal six months from now. Where possible, design agent interfaces against an abstraction layer that permits model substitution without rewriting orchestration logic.
The current frameworks are still young. LangGraph's core primitives, OpenAI's Agents SDK, and Google's ADK are all post-2024 products; CrewAI's Flows feature shipped experimentally in late 2024; the broader "Agentic Systems" conceptual framing - positioning Flows as a deterministic backbone for production deployments - was articulated publicly in December 2025. The patterns are stabilizing, but the tooling is not. Three developments are worth watching.
First, the Model Context Protocol (MCP) is becoming a de facto standard for tool connectivity - meaning the "what tools can this agent access" question is increasingly solved at the protocol layer rather than the framework layer. This could flatten one axis of framework differentiation.
Second, observability and eval tooling (LangSmith, Weights and Biases, Braintrust) is maturing faster than the orchestration frameworks themselves. The debugging and evaluation story - which today still requires significant custom instrumentation - will likely be the next major battleground.
Third, the human-in-the-loop problem remains unsolved at scale. Current HITL implementations are essentially synchronous interrupts: the workflow pauses, a human reviews, execution resumes. For asynchronous, long-running workflows with many decision points, this model does not compose well. Expect significant framework innovation here as enterprise deployments grow in complexity.
Developer's guide to multi-agent patterns in ADK - Google Developers Blog Inline ↗
Orchestration and handoffs - OpenAI API Documentation Inline ↗
LangGraph: Agent Orchestration Framework - LangChain Inline ↗
Mixture of Agents - AutoGen Documentation, Microsoft Inline ↗
How to build Agentic Systems: The Missing Architecture for Production AI Agents - CrewAI Blog Inline ↗