everyais
← All docs

Guides

Prompt caching

Updated

Caching helps when every request starts with the same long prefix, such as a system prompt or tool definitions. Cached prefixes are read at about 10% of the input price. If the prefix changes on every request, leave caching off: cache writes cost more than normal input.

Turn it on

json
{
  "model": "everyais/claude-sonnet-5-5",
  "messages": [
    {"role": "system", "content": "(the long instructions you send every time)"},
    {"role": "user", "content": "First question"}
  ],
  "everyais": { "cache": "on", "cache_ttl": "5m" }
}

With the OpenAI SDK, pass it in extra_body.

python
response = client.chat.completions.create(
    model="everyais/claude-sonnet-5-5",
    messages=messages,
    extra_body={"everyais": {"cache": "on", "cache_ttl": "5m"}},
)
print(response.usage.prompt_tokens_details.cached_tokens)
OptionBehavior
cache: "on"Adds up to 4 cache points: tool definitions, system prompt, then the last user turn
cache: "off"Removes automatic cache points and any cache_control you sent
cache_ttl"5m" (default) or "1h". One-hour writes cost more

Gemini models cache automatically, with no setup and no write charge. On /v1/messages, cache_control on blocks is passed through unchanged.

Cost example

A cache read costs about 10% of the input price. A write costs about 1.25 times input for 5 minutes, or 2 times for 1 hour. For a 10,000-token prefix sent five times within five minutes, in input-token equivalents:

CaseInput-token equivalent
Caching off10,000 × 5 = 50,000
Caching on, same prefix12,500 (first-request write) + 1,000 × 4 = 16,500
Caching on, prefix changes every time12,500 × 5 = 62,500

Messages after the prefix are billed at the normal input price.

Check for cache hits

usage.prompt_tokens_details.cached_tokens counts tokens read from cache, and cache_creation_tokens counts tokens written, when the provider reports them. Both are billed at the provider's cache prices. prompt_tokens (input_tokens on Responses) is the total input including both. On /v1/messages, input_tokens excludes cache reads and writes, as in Anthropic's API.

If cached_tokens stays at 0:

  • The prefix must match exactly. Keep changing values, such as dates or request IDs, out of the start of the system prompt.
  • The cache expires after cache_ttl.
  • Prefixes shorter than the model's minimum are not cached.
Contact support