Open-Source Multi-Agent AI Pipeline with 12 Agents and 5 Quality Gates

Open-source multi-agent pipeline: 61K Python, 12 agents, 5 quality gates…

Alex has developed a self-hosted, MIT-licensed multi-agent pipeline consisting of 61,503 lines of Python code. The system orchestrates 12 specialized agents through a rigorous 11-state machine to transform plain-language ideas into functional products.

Why This Matters

While LLM-driven development tools like Bolt.new exist, production-grade reliability requires moving beyond simple API calls to handle model-specific output variances and state corruption. This project demonstrates that technical success hinges on rigorous quality gates and state recovery mechanisms rather than just the underlying LLM’s intelligence, addressing the reality that AI outputs often fail to meet production standards without external verification.

Key Insights

• The AICOM project consists of 61,503 Python LOC and 22,997 TypeScript LOC as of its 2026 release.
• Task-level pinning prevents architectural mismatches when failing over between different LLM providers like Claude and DeepSeek.
• Playwright is utilized by the Quality Gate agent to perform automated E2E crawls for JS errors and 404s.
• State machines require dual JSON and SQLite persistence to survive corrupted artifacts during task loads.
• The Director AI manages a 6-phase autonomous cycle that requires noop detection to prevent infinite feedback loops.

Working Examples

Example of a visual QA failure where the model assumed a dark theme not present in the application, caught by the visual quality gate.

color: white;
background: white;

Practical Applications

• Use Case: Sandbox preview systems for AI code generation. Pitfall: Missing base tags or absolute URL links cause 404s in iframe environments.
• Use Case: Autonomous feedback loops in AI Directors. Pitfall: Recursive decision-making where the model reads its own output, leading to infinite loops without noop detection.
• Use Case: Multi-provider failover routing based on a model capability matrix for performance optimization. Pitfall: Switching models mid-pipeline without task pinning causes structural inconsistencies.

References:

• https://dev.to/alex_ea31e0501f5eb2156ecc/open-source-multi-agent-pipeline-61k-python-12-agents-5-quality-gates-4hl4
• github.com/alexar76/aicom