cost.observer: Turn LLM Spend Into a Managed, Attributed Line Item

AI FinOps Cost Management LLM Gateway Budget DLP

What if your Anthropic and OpenAI bills broke down by project, team, and use case — and your budgets were actually enforced before the request left the wire?

cost.observer is an AI gateway that turns opaque LLM spend into a managed, attributed, governable line item. Drop it in front of Anthropic, OpenAI, Ollama, LM Studio, and on-prem Spark — and start tagging, budgeting, downgrading, and auditing every prompt that leaves your network.

Explore cost.observer →

cost.observer

  1. Overview — KPIs (requests, total cost, tokens, error rate) plus cost broken down by project, model, and source over 24h
    Overview — KPIs (requests, total cost, tokens, error rate) plus cost broken down by project, model, and source over 24h
  2. Recent requests — every prompt logged with project, source, model, token counts, cost, latency, and status
    Recent requests — every prompt logged with project, source, model, token counts, cost, latency, and status
  3. Connected providers — health-checked upstreams (Anthropic, OpenAI, Spark) with model counts and live latency
    Connected providers — health-checked upstreams (Anthropic, OpenAI, Spark) with model counts and live latency
  4. Analytics — time-window filters across projects / models / sources / providers with cost-over-time, request volume, token usage, latency trends
    Analytics — time-window filters across projects / models / sources / providers with cost-over-time, request volume, token usage, latency trends
  5. Live tail — server-sent stream of in-flight requests, scoped by project, with pause / clear controls
    Live tail — server-sent stream of in-flight requests, scoped by project, with pause / clear controls
  6. Budgets — declarative spend limits per scope, period, and action (warn / throttle / block) enforced before the request leaves
    Budgets — declarative spend limits per scope, period, and action (warn / throttle / block) enforced before the request leaves
  7. Routing config — per-project provider/model overrides, applied per-request at the gateway
    Routing config — per-project provider/model overrides, applied per-request at the gateway

The Problem with AI Spend

AI API spend is opaque. The Anthropic and OpenAI bills arrive as a single monthly number with no breakdown by project, team, or use case. Budgets are unenforceable until the invoice lands.

Provider outages take applications down because there’s no built-in failover.

And the same prompt a developer sends from a laptop is sent again — untouched — from production. With no DLP. No caching. No audit.

You need a gateway.

What cost.observer Does

A drop-in proxy that sits in front of Anthropic, OpenAI, Ollama, LM Studio, and on-prem Spark. Every request is:

  1. Tokenized
  2. Attributed to a project
  3. Cost-estimated before it leaves
  4. Recorded after it returns
  5. Optionally inspected by a DLP service mid-flight

Budgets are enforced in-flight — over the limit, the request is downgraded to a cheaper model or blocked. Not silently overrun.

Three Reasons CTOs Choose It

1. Cost Attribution That Actually Works

Every request is tagged to a project via header, API key, system-prompt fingerprint, or session lineage. SQLite log gives per-project, per-model, per-day, per-token rollups. Export to CSV or query the API.

Your FinOps lead finally has an answer.

2. Budget Enforcement, Not Budget Alerts

Daily / weekly / monthly budgets per project or per user. When a limit is hit, the gateway downgrades (Opus → Sonnet → Haiku → GPT-4o → local Llama) instead of failing.

Applications stay up. Costs stay capped. Nobody’s pager goes off at 3 AM because the team blew past their quota.

3. Resilience-First Design

  • Multi-provider failover
  • Configurable timeouts tuned for streaming
  • Cluster-mode workers with graceful shutdown
  • Cluster-aware circuit breaker around the DLP inspector

An upstream blip never takes traffic offline.

What You Get on Day One

Capability Day One
Cost attribution by project Yes
Budget enforcement (downgrade on cap) Yes
Multi-provider failover Yes
Per-project, per-model, per-day rollups Yes
CSV export Yes
DLP integration hook Yes
Self-hosted, SQLite-backed Yes
Per-seat pricing No

Who Buys It

VP Engineering / CTO

Predictable AI cost. A single answer to “where is our LLM money going?” No more invoice-day surprises.

FinOps Lead

Per-project, per-team rollups for chargeback and forecasting. The numbers reconcile to the provider invoice — because cost.observer is the one talking to the provider.

Security / DLP Owner

Every outbound prompt inspected before it crosses the trust boundary. The circuit breaker means DLP failures degrade gracefully — they never take traffic down.

Better Together: The Observer Suite

cost.observer pairs naturally with:

  • shield.observer — every prompt inspected for PII, secrets, and policy violations
  • tasks.observer — project attribution derived from the same project identity tasks already tracks
  • transmission.observer — every Claude Code session, IDE call, and production agent shares one billed, audited, governed pipe
  • logging.observer — structured access logs land in Loki with pre-built Grafana dashboards

Or run it standalone in front of any Anthropic- or OpenAI-compatible workload. Point your existing apps at it and you have day-one cost attribution, budget enforcement, and provider failover.

Getting Started

  1. docker run against localhost:4007
  2. Point one project at the gateway with a project header
  3. Watch the cost dashboard populate within minutes
  4. Set a budget — daily, weekly, monthly
  5. Configure failover order — Opus → Sonnet → Haiku → local

TL;DR

cost.observer = your LLM bill, governable.

Per-project attribution. In-flight budget enforcement. Multi-provider failover. DLP-ready. Self-hosted.

📞 Contact us to put cost.observer in front of your AI workloads.

Written on February 12, 2026