Multi-vendor LLM cost accounting
Cost-tracking system that attributes AI spend across providers automatically, including calls that would otherwise go unlogged.
A system that automatically records and attributes the cost of every AI model call across multiple providers, tied back to the exact request that caused it. The per-request cost lands on the same row as the request, so any ops view can answer not just 'how much did we spend' but 'how much did this specific feature spend on this specific user'.
When cost tracking depends on engineers remembering to log each call, coverage silently breaks. Someone ships a new feature, forgets to wrap the call, and the cost shows up as zero. The hard part is making coverage automatic so the numbers stay honest as the codebase grows. This makes tracking impossible to forget: a new call by a future engineer is tracked the moment it ships, without them knowing the tracker exists.
A Python module monkey-patches the Gemini, Claude, and OpenAI SDKs at import time. Every patched method records the call against a per-request cost tracker held in a contextvar, so concurrent requests do not bleed into each other's totals. Dedup keys on Python object identity so a legacy manual call site does not double-count if it still has its own logging. Pricing comes from LiteLLM's model-cost table, so bumping the package keeps prices current without code changes. The tracker writes per-call rows that feed both the ops dashboard's per-user spend view and the cohort-level model spend cuts in eval observability.
Sole author of the cost tracker, the import-time SDK patches, the per-request contextvar attribution, the dedup-by-object-identity guard, and the LiteLLM pricing integration.
Want the full technical depth, the tradeoffs, what broke, what I'd do differently? Ask the agent about this project.