Decision record · D005

Which LLM supply powers the bots?

Aquila POC · decided 2026-09-16 by Bhavya (reviewed via this page) · status: locked · prices verified Sep 2026

Every bot talks only to a LiteLLM gateway that holds the real keys, mints a virtual key per bot, tracks spend per bot, and enforces a monthly ₹ budget. The upstream chain is configuration, not code — swapping providers is a config edit. The POC runs on what Bhavya already has; production leans on DeepSeek-class pricing.

POC chain

OpenCode Go→ OpenRouter free / high-limit models

Per-bot virtual keys + ₹ budgets + spend log = the "packaged inference" metering the economics report needs. If OpenCode Go exposes no programmatic API, OpenRouter free becomes primary — same config file.

Options compared

Provider	Price in/out per 1M tokens	India access	Role	Notes
OpenCode Go	flat subscription (₹~900/mo, already held)	✅ working today (this build runs on it)	POC primary	Cheap, covers DeepSeek/GLM/Kimi-class models. Subscription model — suitable for POC; production needs per-token accounting (below).
OpenRouter (free tier)	$0 on `*:free` variants (rate-limited); ~5% markup paid models	✅ one key	POC fallback / primary-if-needed	Fronts DeepSeek/GLM/Kimi/Qwen; ":free" variants have daily caps — good for routines and demos, not constant chat. Universal escape hatch.
DeepSeek V4 Flash (direct)	$0.22 / $0.66 off-peak (2× peak: 6:30–9:30 & 11:30–15:30 IST)	✅ easy signup	Production primary	Cheapest credible agentic model; 1M context; cron jobs scheduled pre-6:30am IST always bill off-peak.
GLM-4.7-Flash (Z.ai)	free	✅ (intl. card may be needed)	Production batch tier	Free named model from a major provider; rate-limited — ideal for cron/batch.
GLM-5.3-Flash (Z.ai)	$0.15 / $0.50	✅	Cheap escalation	Promo ended Sep 9, 2026; still cheap at list.
On-node Ollama (Gemma-class)	₹0 marginal	n/a (local)	Optional tier — off by default	Costs ~3GB RAM ≈ 2 bots of density. 12–18 tok/s on N100-class. Fine for classification/summaries, weak for chat. Measure later.
Kimi K2.6 / K3	$0.95/$4.00 → $3/$15	✅	skip for POC	Priced out of ₹250; K3 carries a custom license anyway.
OpenAI / Anthropic / Gemini frontier	$1–5 / $5–30	✅ (USD billing)	not for Aquila	10–50× the cost; breaks the price point and the India data story.

Why this shape

Metering is the product. The POC's key output is ₹/bot/month. That requires per-bot keys, budgets and a spend log — none of which a shared subscription gives you. Hence the gateway exists before any bot does.
Cheap + free tiers carry the routine work. Morning digests and price checks are batch jobs: free/cheap models are plenty. Chat gets the better model. This routing is the single biggest cost lever toward ₹250.
Peak/off-peak is a scheduling feature. Routines at 6:00–6:30 IST = off-peak on DeepSeek in production. Design the schedule, then the bill.
No frontier dependency. Per the founding call: "we're not selling inference, we're selling packaged inference." The owner buys outcomes; the model is our business.

Full rationale in DECISIONS.md (D005). Sibling comparison: bot runtime.