Agent Roundtable: Building a Local Multi-Agent Expert Roundtable System

I recently built a multi-agent chat project: agent_roundtable.

The idea is simple: I provide a topic, and the system invites several expert agents with different perspectives to speak in turn. A moderator is responsible for follow-up questioning, connecting the discussion, and summarizing, then the entire exchange is saved as a Markdown report.

It is not just a simple chat bot, but a local agent workflow that can configure roles and models, connect to a local knowledge base, and capture the discussion process as documentation.

I prefer to think of it as an "expert roundtable utility": instead of asking one model to answer everything at once, I split a question across agents with different perspectives and let them discuss the same topic.

1. Why I built this project

I had been thinking about one question for a long time: if I were to build a truly useful AI Agent system, what should it actually look like?

A single agent is easy to get started with. You give it a prompt, and it answers questions, calls tools, and generates summaries, which already does a lot. But as questions get more complex, the limits of a single agent become obvious:

• It tends to blend multiple perspectives together.
• It struggles to reliably act as multiple professionals.
• Its output often reads like a blended essay rather than a real discussion.
• It cannot clearly separate "macroeconomic perspective", "investment perspective", "AI technical perspective", "philosophical perspective", and "historical-strategy perspective".

So I wanted a multi-agent roundtable.

A single topic can be discussed by different experts in parallel. A macro expert looks at institutions and cycles, an investment expert looks at cash flow and risk, an AI expert looks at technological evolution, a philosophy expert looks at concepts and value judgments, and a historical-strategy expert looks at long-term structural change.

After doing this, the output is no longer just "one model’s answer"; it becomes something closer to a structured discussion.

2. What the project can do today

agent_roundtable is currently a local Python project that supports both CLI and local web UI modes.

Right now it can do the following:

• Start a multi-agent roundtable via command line or local Streamlit UI.
• Configure each agent’s provider, model, and API key environment variable individually.
• Keep API keys only in .env; they are not written into JSON, YAML, logs, or reports.
• Main experts can read their own local long-form materials and retrieve relevant content via RAG.
• Outputs are automatically saved to logs/ as Markdown reports.
• Each utterance in the report is tagged with the provider and model used.
• You can run the full flow with --mock even without API keys.

There are currently two built-in councils:

One important note: persona_inspired is only "style-inspired," not impersonation, and does not represent any person’s official views. Its purpose is to borrow a thought style to organize responses.

3. My understanding of agents

The term agent is hot right now, but if you make it too abstract, it becomes harder to understand.

My understanding is this: an Agent is not simply a model that can chat; it is a system unit that executes tasks around a goal. It usually includes several components:

• A role or objective.
• A set of prompts and behavioral constraints.
• A set of callable tools.
• A mutable state.
• Optional memory, a knowledge base, and workflow control.

Ordinary chat is "ask one question, get one answer." An agent is more like "given one task, it decides what to do next in order to complete it."

If you add Tool Use, an agent can call search, file, database, and code execution tools. If you add RAG, the agent can consult sources first and then answer. If you add workflow orchestration, multiple agents can cooperate in a defined sequence.

That is why I built agent_roundtable: I wanted to move beyond prompt-only behavior and operationalize role, knowledge, model invocation, and execution flow. If you want a more systematic entry point from agent fundamentals through ReAct and into memory and RAG, you can refer to Hello-Agents: Open Source Tutorial for Building AI Agents from Scratch.

4. Why multi-agent instead of one super-agent

A super-agent can of course answer complex questions, but I prefer the multi-agent form.

The reason is simple: real-world complex judgment is not completed by one voice.

Take the long-term impact of AI on investment and employment, for example. At minimum it can be split into several perspectives:

If one model tries to answer this in one shot, it may still cover these points, but it is hard for it to produce true perspective divergence.

The advantage of multi-agent is that each agent has its own role card, knowledge scope, speaking style, and blind spots. During discussion, different roles enter from different angles, and the moderator weaves them together.

That is more robust than writing one huge prompt, and easier to scale.

5. The role of RAG in this project

RAG can be understood simply as an "open-book exam."

Ordinary LLM responses rely mainly on knowledge embedded in model parameters. RAG first retrieves relevant materials from an external knowledge base before generating answers.

In agent_roundtable, I place long-form documents in the knowledge/ directory. Each expert can have its own knowledge folder, for example:

knowledge/macro_economist/
knowledge/investing_master/
knowledge/ai_researcher/
knowledge/philosophy_expert/
knowledge/history_strategist/

Then build the index with:

python -m rag.ingest --expert-name macro_economist --embedding-provider keyword

I currently prioritize keyword retrieval first because it does not require an additional API key, making it suitable for local testing and onboarding. If stronger semantic retrieval is needed later, we can still plug in embedding models.

This design is important to me.

I do not want expert agents to answer only from the model’s own "impression." I want them to read books, papers, and long-form sources I have prepared beforehand, and use those materials as their knowledge background.

That way, the agent is no longer just a role-playing prompt; it becomes a prototype expert system with a local knowledge source.

6. Project structure

The project structure is roughly:

agent_roundtable/
├── main.py
├── ui/
│   └── app.py
├── src/
│   ├── graph.py
│   ├── agents.py
│   ├── llm.py
│   ├── model_catalog.py
│   ├── agent_llm_config.py
│   ├── loader.py
│   ├── logger.py
│   ├── prompts.py
│   └── state.py
├── agents/
│   ├── domain_experts/
│   └── persona_inspired/
├── councils/
├── configs/
│   └── agent_llms.json
├── knowledge/
├── rag/
├── vector_db/chroma/
├── logs/
├── tests/
├── requirements.txt
└── .env.example

The key directories work as follows:

I prefer this layered style. Roles are roles, model configuration is model configuration, knowledge base is knowledge base, and execution logs are execution logs. They should not be mixed.

7. How agent role cards are designed

In this project, each agent has its own YAML role card.

A typical agent role card includes:

• Name.
• Role.
• Worldview.
• Speaking style.
• Strengths.
• Weaknesses.
• Corresponding RAG knowledge directory.
• Agent type.

For example, an energy expert might be designed like this:

name: "Energy Expert"
role: "Energy and Industrial Policy Expert"
worldview: "Analyze issues through energy supply and demand, infrastructure, geopolitics, and technological substitution"
speaking_style: "Clear and careful, explain constraints before judgments"
strengths:
  - "Energy supply-demand analysis"
  - "Value-chain decomposition"
weaknesses:
  - "May underestimate short-term financial market fluctuations"
catchphrases:
  - "Start by checking energy constraints"
rag_expert_name: "energy_expert"
agent_type: "domain_expert"
profile:
  focus:
    - "Energy security"
    - "Power systems"
    - "Oil, gas, and renewables"

The point here is not to write a character profile, but to fix the agent’s analytical boundaries.

A good expert agent should know what it is good at and also know what it may ignore. Otherwise, all agents eventually sound the same.

8. Why model configuration is kept separate

I keep each agent’s model selection in configs/agent_llms.json.

One benefit is role-model decoupling.

The role card only describes "who this agent is." The JSON config only describes "which provider, which model, and which API key environment variable this agent uses."

For example:

{
  "agents": {
    "macro_economist": {
      "provider": "openrouter",
      "model": "nvidia/nemotron-3-super-120b-a12b:free",
      "api_key_env": "OPENROUTER_API_KEY_1"
    },
    "ai_researcher": {
      "provider": "openrouter",
      "model": "nvidia/nemotron-3-ultra-550b-a55b:free",
      "api_key_env": "OPENROUTER_API_KEY_1"
    }
  }
}

Note that api_key_env is not the API key itself. It only tells the program which variable to read from .env.

Real API keys should not be written into JSON, YAML, README files, logs, or screenshots. This is important.

9. Why local UI matters

I added a local Streamlit UI.

It is not to create a polished commercial product; it is to make configuration more intuitive.

In the UI you can:

• Choose a council.
• Set which provider each agent uses.
• Set which model each agent uses.
• Choose which .env API key each agent uses.
• Save configuration to configs/agent_llms.json.
• Enter a topic and run with real LLMs directly.
• Monitor run progress, current stage, and recent events.
• Preview final summaries and conversation transcripts.

This UI is practical for me.

As a multi-agent system starts to scale, manually editing config files becomes increasingly cumbersome. The UI does not need to be complex, but it should at least let me quickly switch models, switch councils, and test different combinations.

10. MCP and Tool Use: where to go next

agent_roundtable currently focuses on local multi-agent roundtables, RAG, and report generation. MCP is not yet a core implementation, but it is a natural next step.

MCP stands for Model Context Protocol, which can be understood as an open protocol for connecting AI applications with external tools and data sources. It tries to standardize how models connect to tools, databases, filesystems, and business systems.

If RAG mainly addresses "look up information before answering," then MCP focuses more on how an agent can connect to external tools in a standard way and execute actions.

For that reason, I am not presenting agent_roundtable as a fully MCP-integrated project. More accurately, I am first getting the local multi-agent workflow stable, then leaving interface space for MCP and Tool Use later.

For this project, possible MCP capabilities to consider in the future include:

• Integrating filesystem MCP so agents can read local resources more standardly.
• Integrating search MCP so certain agents can verify facts online.
• Integrating database MCP so financial or macro agents can query structured data.
• Integrating GitHub MCP so technical agents can analyze repositories, issues, and code.
• Integrating calendar, email, or task systems so roundtable results turn into follow-up actions.

But I am not making MCP complicated from the start.

I prefer to first stabilize the core path: local RAG, role configuration, model invocation, and report generation. Once that core is stable, MCP can be added as the tooling layer.

11. Relationship to ReAct and LangGraph

ReAct is a classic agent idea where the language model alternates between reasoning and acting. In plain terms, the model does not just output an answer directly; it thinks, calls tools, and then adjusts the next step based on tool results.

LangGraph is more about workflow orchestration. It models agent processes as a graph: nodes can be agents, tools, decision logic, or summarizers, and edges represent how flow moves through the process.

A multi-agent roundtable is very suitable to understand with a graph:

flowchart TD
    A["Moderator raises question"] --> B["Experts speak in turn"]
    B --> C["Moderator summarizes"]
    C --> D["Next round of follow-up"]
    D --> B
    C --> E["Final summary"]

This is not a simple chain call; it is a stateful, ordered, role-based workflow.

My current project is designed in this direction: council flow first, then agent utterances, then summary and logging. If we enhance it further later, we can add more complex conditional branches, such as:

• Trigger retrieval when an agent finds evidence insufficient.
• Ask follow-ups from the moderator when two agents disagree.
• Automatically add an extra rebuttal round when a discussion round is weak.
• Add a fact-checking agent before final report generation.

That is where agent workflows become genuinely interesting.

12. What to watch for when open-sourcing

This project involves local knowledge bases, API keys, and execution logs, so open-sourcing requires care.

I currently exclude several categories by default in .gitignore:

• .env: real API keys.
• knowledge/**/*.md: local books, papers, and long-form content.
• logs/**: run reports.
• vector_db/chroma/**: local vector indexes.
• Temporary files such as __pycache__/, .pytest_cache/.

Generally, it is safe to commit:

• Code.
• README.
• agents/*.yaml.
• councils/*.yaml.
• configs/agent_llms.json.
• knowledge/README.md.
• .gitkeep placeholder files.

The knowledge/ directory needs especially careful handling.

If it contains copyrighted books, full papers, private material, or paid content, it should not be uploaded to a public repository. Open-source projects can show directory structure and usage patterns, but should not include materials that are not suitable for public sharing.

13. What this project means to me

For me, agent_roundtable is not just a toy project.

It connects several directions I have long wanted to work on:

• AI Agents.
• Multi-agent collaboration.
• RAG knowledge bases.
• Tool Use.
• Local workflow.
• Model configuration management.
• Markdown report generation.
• UI for real usage.

It is also easy to extend.

In the future I can build separate expert systems for different domains—macroeconomics, investment, AI, history, philosophy, divination, programming, and more. Each expert can have its own knowledge base, role card, and model configuration. Users only need to enter a question and let different experts discuss it around a shared topic.

That feels more like a system than just writing a few prompts.

This project is fully implemented in Python and is still more research-and-experimentation oriented. If I later turn it into a user-facing product, language choice becomes another decision point. I discussed this in What Language to Use for AI Agents: Python, TypeScript, and Next-Generation Product Engineering.

I am increasingly convinced that many future AI applications will not be solved by "one chat box." Instead, they will be assembled from multiple roles, tools, knowledge bases, and workflows.

agent_roundtable is my current attempt in that direction.

FAQ

What is agent_roundtable?

It is a local multi-agent expert roundtable project. You provide a topic, and the system invites experts with different backgrounds to speak in turn, while a moderator handles follow-up questions, connects discussion threads, and summarizes, with the full discussion saved as a Markdown report. It supports command-line and local Streamlit UI modes, and each agent can be configured with its own provider, model, and API key environment variable.

Why use multiple agents instead of one super-agent?

Complex real-world judgment is not completed by one voice. The advantage of multiple agents is that each one has its own role card, knowledge scope, speaking style, and blind spots, so they approach a problem from different angles before the moderator connects the perspectives. This is more stable and easier to scale than writing one giant prompt.

Can it run without API keys?

Yes. The project supports --mock mode, which allows you to run the full workflow without API keys; RAG retrieval also defaults to keyword mode first, which does not require an additional embedding API key, making it suitable for local testing and onboarding.

Has this project integrated MCP yet?

Not yet. agent_roundtable currently focuses on local multi-agent roundtables, RAG, and report generation. MCP and Tool Use are natural next steps, and the project leaves room for future integration, but they are not core features yet.

How do I avoid exposing API keys and private materials when open-sourcing?

Keep real API keys only in .env, and only store the api_key_env variable names in configuration files. Do not write keys themselves anywhere else. .gitignore already excludes .env, local materials under knowledge/, logs/ reports, and vector_db/chroma/ indexes by default; public repositories should commit only code, README, role cards, configs/agent_llms.json, and placeholder files.

References

• Random-Walk2026/agent_roundtable GitHub repository
• Model Context Protocol official documentation
• Anthropic: Introducing the Model Context Protocol
• LangGraph official page
• LangGraph: Multi-Agent Workflows
• ReAct: Synergizing Reasoning and Acting in Language Models
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks