Reuse stable prefixes; verify cache hits
Use prompt caching as a provider-specific billing and latency tool, keep prefixes stable, and confirm cache reads in usage — without assuming a universal discount or a smaller request.
Step 1 of 4
Four outcomes, only one of which is “fewer tokens”
Token reduction means you sent or generated fewer tokens. Billing discount means the same tokens were priced cheaper (cache reads, batch APIs). Latency includes time to first token and total time to finish; measure them separately. Quality is whether the answer still meets the check.
Prompt caching is usually a discount + latency story. Official docs for OpenAI, Anthropic, and xAI all state that caching does not change how output is generated, and cached prefixes still occupy the context window. Anthropic: caching “changes what you pay for those tokens, not whether they count.” xAI: cached prompt tokens still count toward TPM, and long-context price tiers use total prompt tokens including cached tokens. OpenAI: cached prompts still count toward rate limits. Do not report a cache hit as “we reduced input_tokens to zero.”