Aakash Devkar
← All projects

Agentic AI · Trading · Oct 2026 – Nov 2026

QuantAxis

LLMs interpret, code decides, a human approves: a fail-closed AI trading decision system.

Live demoRepo going public soon
backtest–live decision mismatches
0 / 1,790
backtest–live decision mismatches
LLM eval sign accuracy (42 / 44)
95%
LLM eval sign accuracy (42 / 44)
test coverage on the core packages
98%
test coverage on the core packages

01

The problem

LLMs read macro text such as central-bank statements, rate gaps and events well, but they aren't calibrated for numbers, can invent them, and can be steered by injected text. A trading system that lets a model set prices, sizes or stops is untestable and unsafe. QuantAxis uses the LLM only to turn text into bounded scores with citations. Everything numeric is deterministic and tested, every gate fails closed, and a person approves every order. Live and backtest runs share one decision engine.

02

Architecture: decision pipeline

  1. 1

    Ingest

    Celery workers pull MT5 bars, FRED, the economic calendar, RSS headlines (scored by a local FinBERT) and CFTC positioning into TimescaleDB

  2. 2

    Bias graph

    LangGraph + Groq gpt-oss-120b (20b fallback) score USD, EUR and JPY from cited facts; every number must appear in a cited fact

  3. 3

    Decision graph

    On each H1 close: regime → bias → sentiment → intermarket → event guard → conviction → SMC trigger; stale data or any error fails closed

  4. 4

    Risk veto

    Deterministic sizing, portfolio limits and breakers; signs the exact order with an HMAC token (single-use nonce, 15 min TTL)

  5. 5

    Human approval

    The graph pauses with interrupt(), checkpointed in Postgres; the operator approves with TOTP

  6. 6

    Execution

    Re-checks risk, plan hash and kill switch, consumes the token's nonce, then places an idempotent order via the replay broker or the MT5 bridge

  7. 7

    Manage & journal

    Every 60 s under an account lock: TP1 partial → breakeven → reconcile. Outbox → Redis Streams → WebSocket UI; closed trades get a grounded LLM review

14 services in Docker Compose locally, with OpenTelemetry → Jaeger, Prometheus → Grafana and LangSmith on the LLM graphs. The public demo runs 5 of them.

03

Design decisions

Modular monolith (7 Django apps) with dedicated Celery worker pools

Instead of
Microservices, or one shared queue
Why
One developer and low load. Bulkhead queues mean a slow LLM call never delays an order or a stop move, and 9 import-linter contracts enforce the boundaries.

A fixed LangGraph state machine

Instead of
ReAct / supervisor agents or role-playing crews
Why
Deterministic order and cost, no LLM path to trading tools, pause and resume via interrupt(), and it can be backtested.

The LLM returns only scores and cited text

Instead of
The LLM proposes the whole trade
Why
No ungrounded numbers on the money path, and every citation can be checked in code.

One replay broker shared by live replay and the backtester

Instead of
A separate vectorised backtester
Why
One set of fill rules for both, checked by a parity test.

04

What I built

  • Django 5 modular monolith with 7 bounded-context apps, a Django Ninja API with JWT (30 endpoints + 1 WebSocket), and Celery worker pools on separate queues for agents, trading, ingest and sentiment.
  • Data layer on TimescaleDB hypertables: JForex CSV history, FRED macro series, an economic calendar, live MT5 bars, RSS headlines scored by a local FinBERT model, and CFTC COT positioning.
  • Deterministic core: indicators, regime, smart-money-concepts structure, intermarket, event guard, conviction scoring with a sentiment guard (sentiment can shrink a bias but never flip it), and a risk engine with sizing, portfolio limits and breakers.
  • LLM gateway with a Groq fallback chain, circuit breaker and token budget, plus a grounding validator and a prompt-injection sanitiser.
  • Two LangGraph graphs: bias_graph for validated, cited currency bias, and decision_graph, which pauses with interrupt() for approval and is checkpointed in Postgres.
  • Execution safety: HMAC-signed risk tokens with nonce replay protection, deterministic idempotent order ids, a kill switch, flatten, and TOTP on approve, release and flatten.
  • A replay broker shared with the backtester (bid/ask fills, spread, slippage, stop-first, TP1 partial, breakeven) and an MT5 bridge in FastAPI on Windows with HMAC-signed requests.
  • Live updates via a transactional outbox → Redis Streams → Django Channels WebSocket → React SPA, with one-time ticket auth.
  • Observability with OpenTelemetry (redacting exporter), Jaeger, Prometheus and 2 Grafana dashboards; two read-only MCP servers for market and macro data.
  • A pre-registered 30-case LLM eval suite that replays 148 recorded cassettes offline in CI behind a regression gate.

05

Hard problems

MT5 silently cut order comments to 16 characters

Problem
The first live smoke order filled, but the bridge answered 504 because it couldn't find the deal. The broker keeps only 16 comment characters, not the documented 31, so the 19-character idempotency id was cut and lookups could never match.
Fix
Order ids became a 13-character base32 digest, the bridge refuses an over-long comment before sending, and the fake terminal used in tests now truncates like the broker.
Verified
Regression tests for the short id, the length check and the truncating fake terminal; one broker contract suite runs against the replay, fake and MT5 bridge adapters.

Backtest and live had to decide identically

Problem
The live LangGraph pipeline and the bar-by-bar backtester each had their own gate logic. Separately, re-running a pinned config gave 18 trades instead of 13, because cached config was read before the config files were swapped.
Fix
Gate steps and pending-setup resolution moved into one shared qa_core.decision module, each run parses one config snapshot, and runs are compared by a results hash over trades, summary, gates and costs only.
Verified
A parity test over 1,790 decision points and 5 scenarios shows 0 mismatches, and the train re-run reproduces its results hash (13 trades).

A prompt injection that passed every validator

Problem
In an eval run on prompt v4, a fact carrying a fake operator rule made the model score JPY +1 and USD −1. The output was valid by shape, because schema and grounding checks validate form, not intent.
Fix
Prompt v5 added an input sanitiser (NFKC, invisible-character removal, rule-based removal repeated until clean) and an output screen that rejects drivers repeating input instructions. It was pre-registered and recorded once.
Verified
5 of 6 injection cases pass. One forged-source case still passes silently with wrong signs and is documented as a known limitation; the LLM has no tools, and orders still need a signed token and human approval.

06

Results

Backtest–live parity1,790 decision points, 5 scenarios, 0 mismatches
LLM eval (30 cases, prompt v5)pass 28/30 (93%), sign accuracy 42/44 (95%), stable pair decisions 14/21 (67%)
Prompt-injection cases5 / 6 pass
Python tests1,846 collected; 1,147 passed, 0 failed in the no-DB run; 98% coverage on the core
Static checksmypy strict: 0 issues in 419 files; 9 / 9 import-linter contracts kept
LLM latencymean 1.69 s per call (32 live calls)
Public demo load testk6, 20 users × 2 min: p95 621 ms, 34.2 req/s, 0% errors
Surface30 REST endpoints + 1 WebSocket, 19 Celery tasks, 14 services

Backtests have 13 trades per split, too few to show an edge. A forward test has run on an MT5 demo account since 19 Oct 2026, but there aren't enough closed trades to publish. The public demo is a read-only replay snapshot with no workers or LLM calls.

07

Deployment

ProviderAWS ap-south-1 (Mumbai), public read-only demo
ComputeOne EC2 t3.small running Docker Compose: db, redis, migrate, web and spa (no workers, LLM calls or tracing)
Images & configECR with immutable tags; config and secrets in SSM Parameter Store; Session Manager instead of SSH
TLS & opsLet's Encrypt via certbot; CloudWatch Logs; daily EBS snapshots
CI/CDGitHub Actions: lint, mypy, tests and a compose smoke test on every push; evals on path changes; manual deploy via OIDC → ECR → SSM Run Command
TradingThe MT5 forward test runs locally (WSL2 + Windows bridge), not on AWS

08

Stack

PythonDjangoDjango NinjaCeleryLangGraphLangSmithGroqMCPFinBERTTimescaleDBRedisFastAPIMetaTrader 5OpenTelemetryPrometheusGrafanaReactTypeScriptDockerAWS