# Marginal — AI cost tracking: integration guide

Marginal ingests cost events for AI/LLM usage and shows spend sliced by
model, provider, and custom fields (customer, feature, environment, ...).
This document is the complete integration contract, written for a coding
agent instrumenting an existing codebase. Human docs:
https://marginalhq.com/docs

## Your task

1. Find every AI/LLM call site: OpenAI, Anthropic, Vercel AI SDK, Gemini,
   Bedrock, OpenAI-compatible hosts (Groq, Mistral, xAI, DeepSeek,
   Together, Fireworks, Ollama, Azure), and raw HTTP calls to any of them.
2. Install the Marginal SDK and construct one shared client.
3. After each AI response, call track() with provider, the
   response-reported model, the response's usage object, and fields.
4. Report to the user: which call sites you instrumented, which fields you
   attached, and remind them to register those field keys in Marginal's
   Fields page BEFORE traffic flows (unregistered keys are stripped).

## Rules

- Construct the client once (module scope), read the API key from the
  MARGINAL_API_KEY env var. Never hardcode keys. Server-side code only.
- track() is synchronous, buffered, and never throws — call it inline
  after the AI call. Do not await it, wrap it in try/catch, or move it
  off the request path.
- Pass the provider's usage object AS-IS wherever a recipe below says so.
  Do not convert, rename, or sum token counts yourself — the server
  detects each provider's wire dialect (OpenAI Chat Completions, OpenAI
  Responses, Anthropic, Gemini, Bedrock Converse) by its key names and
  normalizes the cached/uncached split. Hand-converted counts are the #1
  source of cost bugs; don't be that integration.
- model must be the RESPONSE-reported model (response.model / modelId),
  not the alias that was requested — dated snapshots price differently.
- fields: flat object, string/number/boolean values, no nesting. Attach
  what identifies the spend: customer (who), feature (which part of the
  product), environment (production/staging). Derive values from
  surrounding code where obvious; otherwise ask the user. Never put
  prompts, completions, or PII in fields.
- On shutdown paths that exist anyway (SIGTERM handlers, serverless
  handler end), call marginal.shutdown() / flush() so buffered events
  drain. Do not invent new lifecycle hooks just for this.

## SDK setup

TypeScript/JavaScript (Node 18+):
    npm install marginal-sdk

    import { Marginal } from "marginal-sdk";
    const marginal = new Marginal({ apiKey: process.env.MARGINAL_API_KEY });

    marginal.track({ provider, model, usage, fields });   // LLM call
    marginal.track({ cost: 0.05, fields });               // pre-computed dollars

Python (3.9+):
    pip install marginal-sdk

    from marginal import Marginal
    marginal = Marginal(api_key=os.environ["MARGINAL_API_KEY"])

    marginal.track(provider=..., model=..., usage=..., fields=...)
    marginal.track(cost=0.05, fields=...)

Events buffer and flush automatically (every 5 s / at 100 events).
Integration warnings arrive via onError / on_error: "ignored-fields"
(a field key is not registered) and "unpriced-models" (model missing
from the price catalog) — surface these to the user, they are actionable.

## Event contract

Each event needs EITHER cost (finite number >= 0, dollars) OR the complete
triple provider + model + usage. Both -> cost wins. Neither -> rejected.

- provider: lowercased string, <= 64 chars. Namespaces the price lookup.
- model: <= 200 chars, response-reported.
- usage: the provider's usage object. Recognized token counts must be
  non-negative integers; unrecognized keys are ignored (kept for audit),
  never rejected. Do not send null values for token counts — omit or 0.
- fields: only keys registered in the project land; the rest are stripped
  and reported back in the response's "ignored" array.
- Timestamps are server-assigned on arrival; the API accepts none.

Pricing happens at ingest: project custom prices, then catalog
"provider/model", then bare "model". Unknown model -> event lands with
cost = null (never a silent $0) and the model is reported in the
response's "unpriced" array; the user can add a custom price on the
Models page.

## Per-library recipes

### OpenAI (Node + Python) — also Azure and all OpenAI-compatible hosts
Chat Completions and the Responses API: pass usage as-is.

    const c = await openai.chat.completions.create({ model, messages });
    marginal.track({ provider: "openai", model: c.model, usage: c.usage, fields });

Python: usage=completion.usage.model_dump()

Streaming REQUIRES stream_options: { include_usage: true } (Python:
stream_options={"include_usage": True}) or usage never arrives. The final
chunk carries chunk.usage (other chunks: null); read model from that chunk.

For Azure / Groq / Mistral / xAI / DeepSeek / Together / Fireworks /
Ollama: identical recipe, change only the provider string ("azure",
"groq", "mistral", "deepseek", "xai", "together_ai", "fireworks_ai",
"ollama").

### Anthropic (Node + Python)
    const m = await anthropic.messages.create({ model, max_tokens, messages });
    marginal.track({ provider: "anthropic", model: m.model, usage: m.usage, fields });

Python: usage=message.usage.model_dump()

Streaming: usage is split across stream events — do NOT reassemble it.
Use the SDK helper: TS stream.finalMessage(), Python
stream.get_final_message(); then track the final message's model + usage.

### Vercel AI SDK (v7)
Map the SDK's normalized usage onto Marginal's canonical shape:

    function toMarginalUsage(usage) {
      return {
        input_tokens: usage.inputTokenDetails.noCacheTokens ?? usage.inputTokens ?? 0,
        cache_read_tokens: usage.inputTokenDetails.cacheReadTokens ?? 0,
        cache_write_tokens: usage.inputTokenDetails.cacheWriteTokens ?? 0,
        output_tokens: usage.outputTokens ?? 0,
      };
    }

    generateText: model: result.response.modelId,
                  usage: toMarginalUsage(result.usage)   // summed across steps
    streamText:   model: (await result.response).modelId,
                  usage: toMarginalUsage(await result.usage)

provider = whichever provider the model was routed to ("openai",
"anthropic", ...). AI SDK <= 6 has a flat usage object instead:
input_tokens: usage.inputTokens - (usage.cachedInputTokens ?? 0),
cache_read_tokens: usage.cachedInputTokens ?? 0,
output_tokens: usage.outputTokens.

### Google Gemini
JS: pass response.usageMetadata as-is; model: response.modelVersion.
Python renames the counts to snake_case — send them back under their wire
names (do not send None; use 0):

    um = response.usage_metadata
    usage = {
      "promptTokenCount": um.prompt_token_count or 0,
      "candidatesTokenCount": um.candidates_token_count or 0,
      "thoughtsTokenCount": um.thoughts_token_count or 0,
      "cachedContentTokenCount": um.cached_content_token_count or 0,
    }

provider: "gemini". Streaming: the last chunk's usageMetadata has totals.

### AWS Bedrock
Converse: usage: response.usage as-is; model: the modelId you requested
(Converse does not echo it back); provider: "bedrock".
InvokeModel: the response body carries the model vendor's native usage
(Anthropic-shaped for Claude) — pass that as-is.

### Anything else (self-hosted, custom fine-tunes)
Canonical shape — input_tokens is the UNCACHED input count:

    usage: { input_tokens, output_tokens, cache_read_tokens?, cache_write_tokens? }

Tell the user to add a custom price on the Models page or events stay
unpriced (they still land, with token counts and cost = null).

### Non-LLM spend (voice, images, tool calls, pre-computed costs)
    marginal.track({ cost: 0.05, fields });

## HTTP fallback (no SDK)

    POST https://api.marginalhq.com/v1/events
    Authorization: Bearer mgl_...
    Content-Type: application/json

    { "events": [ { "provider": "...", "model": "...", "usage": { ... },
                    "fields": { ... } } ] }

Limits: 500 events/request, 1 MB body. Response is 202 with per-event
acceptance: { accepted, rejected?, ignored?, unpriced? }.

## Verify before finishing

Trigger one request through each instrumented call site, then check:
(1) the project's Ingest log page shows each request tagged "success" —
a "warn" or "error" row spells out rejected events (with payloads),
stripped field keys, and unpriced models,
(2) the events appear in the project's Explorer page with the expected
model, cost, and fields,
(3) no "ignored-fields" / "unpriced-models" warnings from onError.
Nothing fails silently — if these pass, the integration is complete.