Agentic Development · Infrastructure
Storing a prefix of a prompt (like a long system prompt or document) on the provider's side so repeated calls skip re-processing it, cutting cost and latency.
Example: An agent that sends the same 10k-token tool spec on every call caches it once, then pays only for the new user turn each time.
In practice: Cache hits require the prefix to match exactly — reordering tools or editing the system prompt invalidates the cache and you eat the full cost again.