Open
Description
It's a lot cheaper to use cache control, but the npm libs we use don't fully support it yet:
- Caching tools isn't supported: Support Anthropic prompt caching for tool definitions vercel/ai#3820
- Cached token usage isn't returned: Anthropic Cache Control not returning cached token usage vercel/ai#4335
- Bedrock SDK doesn't yet support it: Support prompt caching in bedrock API (@anthropic-ai/bedrock-sdk) anthropics/anthropic-sdk-typescript#507
We'd probably need to implement it manually if we want this to work right now:
https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
Or make PRs to the above libraries.
We'd only use it on one ai call (Choose rule
), although we use it use a lot, so it can lead to large cost savings.
Here's the WIP PR: https://github.com/elie222/inbox-zero/tree/anthropic-cache-control
Metadata
Assignees
Labels
No labels
Activity