This article was originally published on the Dataiku medium publication data from the trenches.

Building generative AI applications is no longer just about using one powerful language model. Today, the real challenge is designing systems made of several specialized LLM agents (or AI agents) that work together to solve complex problems. The hard part is not only deciding what to build, but also figuring out how to build it in practice.

You are facing a complex problem, and you know that using several LLM agents together is the right approach. But a key question quickly comes up: How should these agents be organized? Should they follow a clear hierarchy, operate as a flexible group with more dynamic, non-deterministic interactions, or adopt a hybrid approach with a portion of deterministic task flows? And which framework should you use to build this system, i.e., LangGraph, AutoGen, CrewAI, or something else?

This represents a new form of choice overload in the field. The way you organize your agents acts like a playbook: It can either help your system run smoothly or lead to confusion and poor results.

What are multi-agent systems?

A multi-agent system is a setup where several LLM agents interact with each other to solve a problem. Instead of relying on one single, all-purpose agent, you build a small team of specialists, such as a researcher, a writer, or a reviewer, who work together on the same task.

The architecture is the rulebook for this team, including:

• Who can talk to whom

• Who makes decisions

• How tasks move from one agent to another

Choosing the right architecture is critical. It directly affects how much control you have over the system and how well it can scale. In this blog post, you'll find practical advice to help you understand these options and pick the right design and associated framework for your own application.

Defining the landscape

Before comparing solutions, let's clarify two key ideas:

• Architectures: How agents are organized

• Frameworks: The software libraries you use to build them

You can think of architectures as blueprints, and frameworks as toolkits.

Common multi-agent architectures (the "blueprints")

We will focus on the most widely used MAS architectures.

Hierarchical (or supervisor)

This is a top-down model, similar to a manager leading a team. One supervisor agent is in charge: It evaluates the request, selects the appropriate next actions, and routes well-defined tasks to worker agents.

Each agent runs, produces an output, and hands the result back to the supervisor, which then determines what to do next.

Several common forms of this hierarchical pattern include:

• Simple Supervisor (Tool Calling):

  In the simplest version, the supervisor treats other agents as callable tools. Using the LLM's tool-calling feature, it selects which agent to invoke and provides the corresponding input.

• Complex Hierarchy:

  For larger or more complex systems, the hierarchy can be extended. Instead of one supervisor managing all agents, you may have multiple layers of supervision. In this setup, a top-level supervisor delegates work to sub-supervisors, each of which manages its own small group of agents. These "pods" allow the system to handle more complex workflows while keeping responsibilities clearly separated.

Network (or swarm)

This is a decentralized architecture where all agents are equals and there is no single leader controlling the system. Agents communicate directly with one another, often through a shared conversation, and adjust their actions based on the existing conversation state.

Any agent can contribute at any time by proposing ideas, responding to others, or refining previous outputs. Through this ongoing interaction, agents can challenge, review, and improve each other's work, allowing the system to gradually converge toward a higher-quality final result.

Custom workflow

In a custom workflow architecture, you explicitly design the execution flow between agents. You decide how agents are connected and how tasks move through the system, using sequential steps, conditional branches, loops, or parallel execution. This approach does not rely on open-ended agent interactions or a central supervisor; instead, the behavior of the system is defined by the workflow itself, giving you full control over how it operates.

Popular agent frameworks (the "toolkits")

Frameworks provide the building blocks to implement these architectures.

• LlamaIndex:

  In LlamaIndex, an agent combines an LLM with memory and tools. You can create agent workflows using ready-made components (such as FunctionAgent or AgentWorkflow) or build fully custom, event-driven processes.

• LangGraph: LangGraph models systems as graphs. Each node represents a tool, a decision step, or an agent. Multi-agent systems are created by connecting these nodes and defining how messages and state move between them. For very common architectures, pre-built graphs exist (e.g., create_swarm, create_supervisor).

• AutoGen: AutoGen lets you create agents using the AssistantAgent class. It is excellent for conversational patterns like multi-agent debates or group chats.

• CrewAI: CrewAI is a lightweight Python framework focused on production use. It enables role-based agents with clear goals to collaborate using tools, memory, and task orchestration.

• OpenAI Agents SDK: This SDK (Software Development Kit) enables you to build agentic AI apps in a lightweight package. Its core primitives are agents, handoffs (agents delegating to other agents), and guardrails.

• Google ADK: Google’s Agent Development Kit is an enterprise-grade toolkit designed for modular, maintainable hierarchies and structured workflows.

Key factors to guide your choice

Choosing a multi-agent architecture or framework is not about picking the most popular option; it's about matching the design to the problem you are trying to solve. Before diving into specific architectures and recommendations, it's important to step back and think about a few core dimensions that shape how your system should behave.

Number of agents

Start by identifying how many distinct roles or specializations your system really needs. Some problems can be handled by a small group of three to five agents, each with a clear responsibility. Others require a much larger setup, closer to a full organization, with many specialized agents working in parallel. The number of agents you plan to use will strongly influence both the architecture and the framework you choose.

Level of decision autonomy

Next, consider how much decision-making power each agent should have. In some systems, agents simply execute well-defined and assigned tasks. In others, agents independently determine the next action or which other agents to invoke. Higher autonomy can unlock more flexible behavior, but it also increases complexity and the risk of unpredictable outcomes.

Task trajectory complexity

Think about how the task is executed from start to finish. Is it a simple, sequential workflow with a clear path to the result, or does each request require complex branching decisions to reach a solution? This includes whether tasks follow a fixed order or can split into parallel paths. The more complex and variable the path to a solution, the more important the choice of architecture becomes.

Modularity & reusability

Finally, consider how your system will evolve over time. Will you need to add new agents, replace existing ones, or reuse the same agent across multiple systems? A modular design makes experimentation, maintenance, and scaling much easier, especially as your application grows beyond a single use case.

Taken together, these questions form a practical lens for evaluating different multi-agent designs. With these criteria in mind, we can now look at relevant examples for each multi-agent architecture.

Multi-agent architectures: scenario examples

The easiest way to understand when to use each architecture is to see how they behave in real situations. Below are three concrete use cases, each illustrating a different multi-agent design pattern.

Hierarchical architecture: quarterly competitive report

You ask the system:

"Generate a Q3 competitive analysis report on 'Competitor X'. Focus on product features, marketing campaigns, and customer sentiment."

How it works:

• Supervisor (the "lead analyst agent"):

  The supervisor receives the request and breaks it into clear sections (product, marketing, sentiment).

• Specialized Agents:

  Each section is assigned to a dedicated agent (e.g.,

  product analyst agent

  ,

  marketing analyst agent

  ,

  sentiment analyst agent

  ).

• Parallel Execution:

  Agents work independently and do not need to coordinate with each other.

• Synthesis:

  The supervisor collects the outputs to assemble and review the final report.

This is a strong fit for a hierarchical architecture because the task is structured, predictable, and can be split into independent research efforts.

Network architecture: purchase customer support

A customer writes:

"My order #123 hasn't arrived, and I'd also like to upgrade my subscription."

How it works:

• Specialized Agents: The system includes a shipping agent to handle delivery issues and a billing agent to manage subscription and payment requests.

• Initial Agent: One of these agents receives the message first and determines whether it can fully handle the request or whether part of it should be routed to another agent.

• Autonomous Handoffs: After addressing the shipping issue, the shipping agent detects that the subscription request remains unresolved and transfers the conversation to the billing agent to complete the task.

This is perfect for a network architecture because the problem is unpredictable. Control naturally moves between specialists. This approach is flexible and adaptive, which is essential for conversations that cannot be fully planned in advance.

Custom workflow architecture: technical customer support

A customer writes:

"Help! My dashboard isn't loading"

How it works:

• Routing Step: A routing component classifies the request into technical and billing categories.

• Conditional Execution: Depending on the classification, the request is sent to either a technical support agent or a billing agent.

• Convergence Point: Regardless of the path taken, the response is sent to a quality-control agent.

• Final Polish: The quality-control agent rewrites the answer to match the company’s tone and standards before sending it to the customer.

This use case fits a custom workflow because it follows explicit logic rules and requires dynamic controlled routing while ensuring consistent quality through a final review step.

These examples highlight how different architectures excel under different constraints. In the next section, we'll compare hierarchical, network, and custom workflow architectures across the key dimensions introduced earlier — number of agents, autonomy, task trajectory complexity, and modularity — to help you choose the right design for your own application.

The great multi-agent debate: practical guides & recommendations

Now that we've explored concrete examples, let's compare the three architectures across the four key dimensions introduced earlier.

Architecture vs. technical dimensions

Number of agents

• Hierarchical (High): Best suited for large systems. You can organize dozens of agents into small teams managed by intermediate supervisors, keeping the overall system structured and understandable.

• Network (Low): Works best with small groups. As the number of agents grows, the system quickly becomes hard to follow, making handoffs, loops, and agent interactions difficult to track.

• Custom Workflow (Medium): Scalability depends on the complexity of the workflow graph. It handles mid-sized teams well, as long as routing logic and execution paths remain clear.

Level of Decision Autonomy

• Hierarchical (Low): Agents act as focused specialists. They execute tasks only when instructed and operate strictly within the scope defined by the supervisor.

• Network (High): Agents are independent peers. Each agent determines when its task is complete and triggers the next appropriate agent.

• Custom Workflow (Medium): Agents have freedom within clearly defined boundaries. For example, a technical agent determines the remediation steps, while a router controls whether that agent is invoked at all.

Task Trajectory Complexity

• Hierarchical (Medium): The path is centrally managed. A supervisor defines the plan and splits the work across agents, making it easy to manage multiple tasks in parallel. Individual tasks can be complex, but the overall flow stays clear, structured, and easy to debug.

• Network (Low): The path emerges naturally from agent conversations. There is no predefined plan, often moving one step at a time, which limits parallel execution. This makes the setup simple, but the sequence of actions can quickly become hard to follow or predict.

• Custom Workflow (High): The path is fully predefined. Each step, condition, and transition is explicitly designed. This requires more upfront work, but it gives you full visibility, precise control, potential parallelization and complete traceability of how decisions are made.

Modularity & Reusability

• Hierarchical (High): Individual agents can be upgraded or replaced without impacting the rest of the system. A new agent can be swapped in without changing the supervisor logic.

• Network (Medium): The system is flexible. New agents can be added easily, as long as existing agents are aware of their role.

• Custom Workflow (Medium): Modularity is tied to the workflow graph. Adding new behavior usually requires updating routing logic and defining new paths.

Framework Recommendations

Choosing a framework is about aligning its design philosophy with your architectural needs.

• LangGraph: Particularly well-suited for all architectures. Its stateful graph model gives you fine-grained control over execution paths.

• Google ADK: It excels at large hierarchical systems with strong modularity and clear management layers.

• AutoGen: A reference framework for network and debate-based systems, where agents collaborate through open-ended conversations.

• CrewAI: Well suited for role-based hierarchical teams. It offers a good balance between structure and flexibility.

• LlamaIndex: Can support all three architectures, but with varying levels of effort. It is easy to use for network-style systems, requires more setup for hierarchical designs, and involves the most complexity when implementing fully custom workflows.

• OpenAI Agents SDK: A strong choice for simple hierarchical systems.

Other elements, such as the overall enterprise technology landscape and the comfort level of the development team, will also influence the choice of framework.

Conclusion: architecture first, framework second

Multi-agent systems are not about chasing the latest framework or recreating human organizations for the sake of it. They are about making complexity manageable. The biggest mistake teams make today is jumping straight to a toolkit without first deciding how agents should collaborate.

None of the three architectures described in this article are "better" in absolute terms; each one is optimized for a different trade-off between control, flexibility, and complexity. Frameworks like LangGraph, AutoGen, CrewAI, Google ADK, or the OpenAI Agents SDK simply operationalize these choices. Each framework comes with its own strengths and limitations, depending on the architecture you choose.

If there is one key takeaway from this article, it's this: Architecture is the strategic decision; frameworks are tactical enablers.

This article is illustrated with a working implementation available on the Dataiku Public Gallery.