- Published Date
Mohamed HabbatAuthor
What 17 Days of 24 AI Agents in Production Actually Looks Like
Share:
What 17 Days of 24 AI Agents in Production Actually Looks Like
Twenty-four agents, one orchestrator, a system I'd been building toward for months. On day one, one agent did almost everything. Not because the others failed. Because Midas, the orchestrator, was executing tasks instead of dispatching them.
That's the version nobody writes about. Not "here's our architecture diagram." Here's what the first 17 days of multi-agent AI in production actually looked like: the stalls, the skill drift, the two agents I had to kill mid-task, and what we fixed each time. If you're evaluating whether to build something like this, or you're already building it and things aren't compounding the way you expected, this is the account I wish I'd had on day one.
Day 1: One Agent Did All the Work
Midas is the orchestrator. His job is to receive tasks, classify them, and dispatch them to the right specialist. On day one, he wasn't doing that. He was doing the work himself.
This is what I call the coordinator trap — and it's more common than anyone admits. You configure a multi-agent setup with twenty, thirty agents. You give them names. You write their system prompts. You assign them roles. Then you ask the orchestrator to do something, and it does it. No dispatch. No handoff. Just one LLM call through one path, the same as a single-agent setup with a fancier config file.
The problem wasn't Midas failing — he was following his instructions. The routing architecture wasn't clear enough. There was no typed dispatch. Nothing that explicitly said: this type of task goes to this agent, and you do not execute it yourself. Midas was capable of executing tasks directly, so he did. That's a rational response to an under-specified system.
The fix wasn't a new system prompt. It was architecture. I restructured the dispatch logic so Midas routes by task type and never executes. Every task has a declared type. Every type has an owner. Midas matches them and hands off. Full stop. The moment dispatching was the only option and execution structurally impossible, the system started behaving like a multi-agent system.
The lesson: an orchestrator that can do the work will do the work. You cannot fix that with better prompting. It's the dominant failure mode in multi-agent AI production. The reason teams configure ten agents and consistently get one agent's results.
The First Thing That Broke: Skill Chaos
By day three I had a different problem. Skills.
In OpenClaw, the multi-agent runtime we run on, each agent has a skill set: capability modules that define what tools they can use, what external systems they can access, what they're equipped to do. When I built the initial setup, I gave each agent their skills locally. Agent-specific directories. That felt clean.
What actually happened: drift.
One agent's postgresql-db skill was on one version. Another's was different. Nobody knew which was current. When I updated a skill for one agent, nothing propagated to the others. I had to track down every agent using that skill and update it separately, and I didn't always know which agents used which skills.
By today, March 29, the system had accumulated 23 stale .agents/ skill directories, each holding their own version of capabilities that may or may not have matched what the agent actually needed. Four specific issues had developed: a duplicate postgresql-db skill, canvas-design linked to the wrong agent, agent-browser missing entirely from Eris, and brand-protection missing from the security agent.
None of those were catastrophic individually. Together they meant agents were operating with an inconsistent, partially broken capability set. An agent that should have had browser automation access didn't have it. An agent responsible for brand checks wasn't equipped to run them. Tasks that depended on those capabilities were running against the wrong foundation.
The fix: centralised skill library. Today we migrated 16 skills to a shared workspace/skills/ directory and removed all 23 stale per-agent directories. One skill, one location. Every agent links to it. An update to the postgresql-db skill now propagates to every agent that uses it, immediately, with no manual tracking required.
If you're coming from no-code automation tools where integrations are point-and-click, the skill concept might sound abstract. Think of skills as capability contracts. An agent without the right skill can still attempt a task, silently, with no error, and produce wrong output. The failure is invisible until you trace the result back to a missing capability.
The migration took most of a day. It should have been the first decision, not one made on day 17.
What 37 Pull Requests Taught Us About Agent Specialisation
We shipped 37+ pull requests in 17 days. That number is from the W13 weekly review. At a two-person company, that throughput is not possible without a functioning multi-agent pipeline.
Specialist agents catch specialist issues. When a PR went through Hephaestus (the code review agent), he caught architecture patterns and design concerns. When the same PR went through Argus — the security agent — he caught entirely different things. Neither surfaced what the other found.
What didn't work: agents attempting tasks outside their domain when routing was ambiguous. Early in the system, not every task type was explicitly declared. When a task was ambiguous, Midas would sometimes dispatch to a capable-looking agent rather than the right one. The capable agent would attempt it. The output was always technically valid and almost always wrong for the context.
One example: a content task routed to a technical agent because both had overlapping skills. The output was accurate but had none of the SEO structure required. Task ran clean. Output was unusable. Second pass required.
The fix was enforced skill routing: not just role names, but explicit task-type-to-agent mappings. Every declared task type has one owner. Ambiguous tasks get flagged, not guessed at. Specialisation only works when routing is enforced. Role names don't prevent cross-domain execution. Typed dispatch does.
The 37 PRs tell both sides: what a functioning specialist pipeline can produce in 17 days, and how much early friction came from routing that wasn't tight enough.
The Agents That Stalled And What That Cost
I had two real production stalls in 17 days. Both were instructive.
Pythia, week one. Pythia is the research agent. She hit the session timeout limit on a deep web research task. 30 minutes in, she was mid-analysis and ran out of runway. No output. The task was real: keyword research and competitive analysis for an SEO content sprint. I had to steer her manually and ask her to wrap up with what she had.
What that revealed: I had given her an open-ended research task with no time constraint and no explicit scope limit. "Research X" is not a production-ready task brief. A production task brief says: research X, in no more than 30 minutes, output a structured report covering Y and Z. Without that constraint, the agent optimises for completeness, not for shipping. Pythia was doing exactly what she was built to do: going deep. I hadn't told her when to stop.
Orpheus, week two. Orpheus handles de-AI cleanup: stripping AI voice from written content. I sent him a piece for cleanup. An hour and 24 minutes later: nothing. No output, no partial result, no error. Session still active.
I killed it and did the pass directly. Twenty minutes.
What that revealed: I hadn't given Orpheus a hard output requirement. The task was ambiguous on what "done" looked like. He was likely revising iteratively, trying to hit a quality standard that hadn't been defined. Agents don't have an internal clock telling them when good enough is sufficient. They have an instruction set. If the instruction set doesn't include a stopping condition, the agent fills the available time.
Both stalls came from the same root cause: task scoping. Before you worry about what AI agents are or what frameworks to use, get this right. Every agent task in production needs a hard time budget and an explicit output spec. Not guidance. Requirements. That's the difference between a task that ships and a session that runs until timeout.
What Flows Changed
Before every multi-agent task was ad-hoc. An article: ask Clio for a brief, ask Calliope to write it, ask Orpheus to clean it up, ask Dike for meta structure. The sequence worked because I remembered it. If I was in a hurry, steps got skipped. There was no gate. Nothing enforced that Calliope couldn't start until Clio's brief was complete. Steps got skipped. Downstream agents worked from incomplete inputs.
Today we have 32 flows defined as JSON, across five categories. Every multi-step process has a spec: what triggers it, what each step does, what gate must pass before the next step starts.
The SEO content sprint that produced this article ran through a defined flow. Pythia did the research. Clio wrote the brief. Calliope wrote the article. Orpheus will do the de-AI pass. Dike will handle meta structure. Each step has a gate. Clio's brief can't be bypassed. Orpheus can't be skipped. The flow enforces the sequence whether I'm paying close attention or not.
What compounded: each time a flow ran and something failed, I had a trace. Which step failed, which agent, what the input was, what the output was. That's not possible with ad-hoc dispatch. With flows, the second run of a sprint was tighter than the first. The third better than the second. Each failure had a location, not just a symptom.
32 flows sounds like a lot. Some are three steps. But every recurring process is now repeatable, auditable, and improvable. Ad-hoc isn't any of those things. Once you see what a gate actually prevents (a half-built draft in the pipeline, a security review skipped, an incomplete research task used as a writing brief), you stop treating flow specs as overhead.
The Numbers After 17 Days
Here's what's real after 17 days of multi-agent AI in production:
- 24 agents: each with a defined domain and enforced skill set, running on OpenClaw
- 32 flows across 5 categories, all defined as JSON
- 16 skills migrated to centralised workspace library today
- 23 stale directories removed today
- 37+ PRs shipped through the pipeline in 17 days
- 2 production stalls: Pythia (30-min timeout) and Orpheus (1h24m). Both resolved, both traceable to task scoping.
- 0 broken symlinks after today's audit
- 4 skill issues fixed: duplicate postgresql-db, canvas-design on wrong agent, agent-browser missing from Eris, brand-protection missing from security agent
On the industry side: 85% of enterprises are using AI in some form (Wiz, 2025). Only 2% are at full agent deployment scale (index.dev). That gap is the whole story. Having AI tools and having a production multi-agent system with governed flows and centralised skills are different things entirely. Most organisations are firmly in the first category. We're in the second.
Trust in fully autonomous agents has dropped from 43% to 27% over the past year (index.dev). That number makes sense to me. Teams set up agents, see inconsistent output, lose confidence. The inconsistency is almost always architecture — not the agents, not the models. When routing is enforced, tasks are scoped, and flows have gates, output is consistent. The 73% who don't trust autonomous agents yet mostly haven't seen a governed system.
What I'd Do Differently From Day 1
Three things. Honest ones.
1. Define flows before building agents. I built all 24 agents first: names, system prompts, skills, domains — before writing a single flow spec. Then I retrofitted flows onto an existing agent structure. Some flows were awkward because the agents were designed without a clear step sequence in mind. If I'd mapped the workflows first — what tasks run in what order, what handoffs exist, what gates matter — the agents would have been designed to fit. Instead, the flows bent around the agents. It works, but the seams show.
2. Centralise skills on day one. The migration I ran today should have been setup step one. It took hours to untangle 17 days of skill drift across 24 agents. If the centralised library had existed from the start, there would have been nothing to migrate. Every new skill would go in one place. Every agent links to it. The four issues I fixed today (wrong agent, missing skills, duplicates) don't happen when there's a single source of truth from the beginning.
3. Set hard time bounds on every agent task. Every task brief I send to an agent now has a maximum time budget and explicit output requirements. Not guidelines — requirements. "Research X in 30 minutes and produce a structured report with sections Y, Z, and W" is a production task brief. "Research X thoroughly" is not. The Pythia and Orpheus stalls were both preventable with a single line added to the task spec. I didn't add it because I assumed the agents would know when to stop. They don't. That's not a limitation. It's a contract gap on my end.
Seventeen days in, the system works. Not because everything went right from the start (it didn't), but because each failure was traceable, each fix was architectural, and the foundation was solid enough to build on. That's what multi-agent AI in production actually requires: not perfect agents, but a system that makes failures visible and fixable.
Ready to Build It Right From Day One?
If you're putting together a multi-agent setup, or you've already built one and you're hitting the coordinator trap, skill drift, or agent stalls, the mistakes above are avoidable. They just need to be caught before 17 days go by, not after.
Book a discovery call and we'll look at your current architecture together: where routing breaks down, where skills are drifting, where tasks need hard bounds. One hour. No pitch. Just a clear-eyed look at what's actually happening and what to fix first.
At Bi·Catalyst, we specialize in engineering and developing custom software tailored to your unique needs. If you have an idea you want to bring to life, don't hesitate to get in touch. with us, and let's transform your vision into reality. Your journey to bespoke software solutions begins here with Bi·Catalyst.💡
