The dataset

Source and licence

The catalog is model_prices_and_context_window.json from LiteLLM, MIT licensed. It is vendored into this repository under data/, about 1.7 MB, and redistributed normalized.

Both this project and LiteLLM are MIT, so redistribution is straightforward; the upstream copyright and licence terms cover the data file itself.

By the numbers

The snapshot documented here. Live values are always available at /v1/meta — this table is the shape, not a contract.

The file on disk is the source of truth

data/model_prices_and_context_window.json is versioned in git and baked into the container image. It is read at boot, and it is what the service falls back on when the network is not there. There is no database and no cache layer — the snapshot lives in memory with its indexes prebuilt, and gets swapped atomically.

Next to it sits model_prices_and_context_window.meta.json, a sidecar holding { etag, sha256, parseVersion, fetchedAt }. It exists because a file has no ETag of its own — the server issues one, so if you want to make a conditional request later you have to store it yourself.

Normalization: one boundary, on the way in

The dataset is downloaded verbatim but not stored verbatim. It goes through parse immediately before being written to disk, and that output is what everything else reads — the snapshot, the indexes and the API responses all see the same normalized data.

The edits are numbered and applied in order. Today there is exactly one:

1. litellm_provider → provider, applied recursively, preserving key position and never overwriting an existing provider.

Doing it here rather than at the API layer means there is no second place where field names get adjusted, and no chance of the file on disk and the responses disagreeing. parse is idempotent, so running it twice changes nothing.

What a boot does

1. Read both files and check that the sidecar's sha256 matches the JSON on disk. A mismatch means the pair drifted apart, so the ETag is discarded.
2. Send a conditional GET to UPSTREAM_URL with If-None-Match.
3. Build the in-memory snapshot and start serving.

The only fatal case is having neither a usable local copy nor a reachable upstream. Since the copy ships inside the image, that means a genuinely broken deployment.

Writes are atomic (temporary file plus rename) and never fatal: on Cloud Run the filesystem is an ephemeral tmpfs, so a failed write-back should not take down a service that is otherwise perfectly able to answer.

How the data gets refreshed

On Cloud Run the write-back lasts only as long as the instance; it is the copy in the image that keeps cold boots cheap. A cold start with a current copy transfers zero bytes.

Upstream risk

UPSTREAM_URL defaults to LiteLLM's litellm_internal_staging branch. That is an internal branch: it can be force-pushed, carry unvalidated data, or disappear.

A cold start with upstream down is survivable — the copy in the image is served and /health reports startup_error. The quieter failure is the branch going away for good: every boot then keeps shipping whatever version was last baked into the image, without anything looking broken. Watch startup_error and dataset.loaded_at if that matters to you.

For production, consider pointing at main instead:

UPSTREAM_URL=https://raw.githubusercontent.com/BerriAI/litellm/refs/heads/main/model_prices_and_context_window.json

Decisions the data forced

The context tier is a flag, not a marginal band

If the prompt crosses the threshold, the whole request is repriced — output included. The cleanest evidence is that the dataset defines output_cost_per_token_above_200k_tokens: an output price gated on an input threshold only makes sense as a flag. 116 of the 117 threshold pairs have a ratio of exactly 2.0, LiteLLM picks a single rate the same way, and Anthropic, Google and OpenAI all document it that way.

_above_1hr is not a volume threshold

It is Anthropic's 1 hour cache TTL, and it composes with the context tier — which is how a key like cache_creation_input_token_cost_above_1hr_above_200k_tokens comes to exist, and why the cost engine treats TTL and threshold as separate axes rather than parsing key names ad hoc.

A missing rate is never billed as 0

258 models legitimately declare output_cost_per_token: 0. If an absent key were also treated as zero, "free" and "unknown" would be indistinguishable in the total. Unknown usage goes to unpriced[] instead.

Decimal arithmetic, not number

The file ships floating point noise already serialized into the JSON literal (1.2999000000000001e-07) and rates as small as 1.3e-10. Summing those in binary floating point drifts visibly across an aggregate, so everything accumulates in Decimal.

Two incompatible key families

The catalog spells cost keys two ways: <direction>_cost_per_<unit>, and the cache family where _cost comes last (cache_read_input_token_cost). A single *_cost_per_* regex would miss roughly 1,100 cache key occurrences, so the engine keeps an explicit registry of 33 base keys instead of guessing — and a test walks the whole dataset to fail the build when upstream introduces a pricing key that registry cannot describe.

Databricks DBU rates are not dollars

47 models carry input_dbu_cost_per_token / output_dbu_cost_per_token, which is why totals is keyed by currency rather than being one number. No usage field maps to them yet — see the limitations.

Implausible rates are flagged, not corrected

18 models carry a rate above roughly $500/MTok — wandb/* entries at $55,000/MTok and beyond. They are almost certainly upstream errors, and they are served as they are, with a SUSPICIOUS_RATE_MAGNITUDE warning on the affected line.

Correcting them would mean this API silently disagreeing with its own stated source, and the threshold that catches the errors also catches the genuine outliers: o1-pro really is $600/MTok, and it sits in an explicit allowlist so it produces no warning.

Quirks in the ids

• 82% of the ids contain a /, 244 contain :, and four contain a literal *. A conventional :id path parameter is useless here, so the lookup route is a wildcard that rejoins every segment. How to call it →
• One id ends in a slash — fireworks_ai/accounts/fireworks/models/ — which no path can carry, because the router normalizes the trailing slash away. That is the entire reason /v1/models/by-id exists.
• Two entries differ only by case: together_ai/baai/… and together_ai/BAAI/…, with identical content. Lookup is exact first, so each resolves to itself; an id that matches several only case-insensitively is a 409 rather than an arbitrary pick.
• The id is not a field. Upstream is an object keyed by model id, so id is injected into each entry on the way out — which is why it shows up last.

Next

The API reference for the endpoints that expose all of this, or the estimator for how these decisions turn into a number.