Plain asyncio orchestration
- Instead of
- LangGraph or CrewAI
- Why
- The four agents are independent, so they only need gather, a timeout and failure isolation, and I can see exactly what runs.
Multi-agent · RAG · Jul 2026 – Sep 2026
Four LLM agents review every pull request and block merges on critical bugs.
01
Code review is a bottleneck, and LLM review bots fail in three ways. They see only the diff, so they miss who calls the changed function. They rate real vulnerabilities too low to act on: in the first eval the agents found 5 of the 6 blocking issues but rated them HIGH. And a diff is attacker-controlled text, so a planted comment can talk the model out of reporting a bug.
02
PR webhook
GitHub App webhook verified with HMAC-SHA256; job stored in SQLite with a duplicate-delivery guard and a stale-commit check
Filter & screen
Skips lockfiles, generated and oversized patches; Bedrock Guardrails screens each patch for prompt injection and quarantines flagged files
Context retrieval
Callers, callees and imports from an AST call graph, plus similar code from Qdrant (Titan embeddings), capped at 8,000 chars
4 agents in parallel
Security, architecture, test coverage and documentation on Bedrock gpt-oss-120b / 20b; 45 s timeout each; untrusted text in randomly named tags
Post-processing
Re-anchors line numbers to the quoted snippets, applies the deterministic severity floor and ceiling, de-duplicates and ranks
Synthesizer
gpt-oss-120b writes the PR comment, with a deterministic fallback
Merge gate
Any CRITICAL finding fails the ai-code-review status until a reviewer with write access approves or blocks
03
04
05
06
| Merge-gate block recall (33 cases) | 17% → 83%, block precision 100%, 0 false blocks |
|---|---|
| Recall, all categories | 81% (25 / 32) |
| Exact-severity match | 46% → 73% |
| Latency per file (Bedrock) | p50 5.5 s, p95 10.9 s |
| LLM cost | $0.0017 per reviewed file |
| Injection guardrail | 5 / 8 caught, 0 / 37 false positives |
| Tests | 597 passed with the network blocked, 84% app line coverage |
Every eval number comes from a single run, so differences of 1–2 issues are noise. The call graph parses Python only.
07
| Provider | AWS ap-south-1 (Mumbai), defined in Terraform (50 resource blocks) |
|---|---|
| Edge | CloudFront serves the dashboard from private S3 and forwards API, WebSocket and webhook paths to an ALB that only accepts CloudFront |
| Compute | ECS Fargate, 1 task (0.5 vCPU / 1 GB); SQLite and call-graph JSON on EFS |
| AI services | Amazon Bedrock (gpt-oss, Titan embeddings, Guardrails), Qdrant Cloud, Groq as fallback |
| Secrets & monitoring | Secrets Manager; CloudWatch logs, EMF metrics and 7 alarms → SNS email |
| CI/CD | 5 required checks (tests, dashboard lint + build, Docker build + Trivy, LLM eval gate, the bot's own review); deploys via GitHub OIDC after manual approval, with ECS circuit-breaker rollback |
| Cost | About $1.5 a day while running |
08