Prompt caching

When a request starts with the same long prefix as an earlier one (a system prompt, tool definitions, a document), the provider can reuse it from cache. Cached tokens are billed at the model's cache read price instead of the full input price.

Each model page in the catalog lists its input, output, cache read and cache write prices per 1M tokens. A cache read price on the model page is the first thing to check before you rely on caching for that model.

Whether and when a model caches is decided by its provider. Whenever the upstream reports cached tokens, BigModel.io bills them at the cache price, and the usage log shows them as cache read tokens.

#How caching is billed

• Cache read tokens are billed at the cache read price. On Chat Completions, cached_tokens are subtracted from input tokens and billed as cache read, so you never pay for them twice.
• Cache write tokens (Messages cache_creation_input_tokens) are billed at the cache write price. If the model has separate 5-minute and 1-hour write prices, each part is billed at its own price.
• The total is then multiplied by your group's rate multiplier, as for all usage. See Billing.

The Usage page shows cache creation (5m / 1h) and cache read tokens in the tokens tooltip of each request.

#Cache with cache_control on the Messages API

Add cache_control to the last block of the prefix you want cached. Everything up to and including that block becomes the cached prefix.

curl https://bigmodel.io/v1/messages \
  -H "Authorization: Bearer $BIGMODEL_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3",
    "max_tokens": 256,
    "system": [
      {"type": "text", "text": "You are a support agent for Acme. <paste a long policy document here>",
       "cache_control": {"type": "ephemeral"}}
    ],
    "messages": [{"role": "user", "content": "How do refunds work?"}]
  }'

import os
from anthropic import Anthropic

client = Anthropic(base_url="https://bigmodel.io", api_key=os.environ["BIGMODEL_API_KEY"])
policy = open("policy.md").read()  # a long, stable prefix

for question in ["How do refunds work?", "Can I change my plan?"]:
    msg = client.messages.create(
        model="glm-5.3",
        max_tokens=256,
        system=[{"type": "text", "text": policy, "cache_control": {"type": "ephemeral"}}],
        messages=[{"role": "user", "content": question}],
    )
    u = msg.usage
    print(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens)

import Anthropic from "@anthropic-ai/sdk";
import { readFileSync } from "node:fs";

const client = new Anthropic({ baseURL: "https://bigmodel.io", apiKey: process.env.BIGMODEL_API_KEY });
const policy = readFileSync("policy.md", "utf8");

for (const question of ["How do refunds work?", "Can I change my plan?"]) {
  const msg = await client.messages.create({
    model: "glm-5.3",
    max_tokens: 256,
    system: [{ type: "text", text: policy, cache_control: { type: "ephemeral" } }],
    messages: [{ role: "user", content: question }],
  });
  console.log(msg.usage.cache_creation_input_tokens, msg.usage.cache_read_input_tokens);
}

The first call should report cache creation tokens, the second cache read tokens. Whether the upstream honours the breakpoint depends on the model behind your group.

#Rules the gateway applies

• cache_control is passed through to the provider.
• At most 4 breakpoints are kept per request. Extra ones are removed, first from tools, then from messages, then from system.
• cache_control on thinking blocks is not allowed and is removed.

#Automatic caching on Chat Completions

For providers that cache on their own, you do not send anything. Keep the start of your prompt identical between calls (same system prompt, same tool list, same order) and put the parts that change at the end. Read the result:

completion = client.chat.completions.create(model="deepseek-v4-flash", messages=messages)
details = completion.usage.prompt_tokens_details
print(details.cached_tokens if details else 0)

#Verify the setup

Send the same long-prefix request twice within a few minutes. On the second response, cache_read_input_tokens (Messages) or prompt_tokens_details.cached_tokens (Chat Completions) should be above zero. In Usage, the second request's tokens tooltip shows cache read tokens and a lower cost.

#Troubleshooting

Cache read is always 0. The prefix changed between calls (a timestamp in the system prompt, a different tool order), the prefix is shorter than the provider's minimum, the cache expired, or the model does not cache. Make the prefix byte-identical and check the model page for a cache read price.

Cache creation tokens on every call. Each call writes a new cache entry because the prefix differs. Move changing content after the breakpoint.

Some breakpoints seem to be ignored. You sent more than 4; the extras were removed. Keep the most valuable ones (usually the system prompt and the end of the tools list).

400 about cache_control from the provider. The upstream model does not accept the field. Remove cache_control for that model.

404 on /v1/messages. The key's group serves Chat Completions only, where you rely on automatic caching.

#Related

• Billing & top-ups for the full cost formula.
• Messages API reference.
• Usage to check cache tokens per request.