What Survives the Cache
1.1 · Step 2 · tools go at the front

What survives the cache

This is your coding agent's last request after the chat template rendered it: one system prompt, two tools and four turns, measured with the real Qwen2.5 tokenizer. The width of each block is its token count.

The request the model actually sees

Rendered text
Predict first

Your server reuses the longest unchanged token prefix, as vLLM does. You add one tool, git_diff, to the end of the tools list. Nothing else changes. What can the next call reuse from the prefix cache?

How sure are you?
Pick an answer and how sure you are.

Now edit the request

Each edit is measured against the request above: the next call reuses the longest run of identical tokens from the start.

–tokens reused
–tokens processed again
–reused in 16-token blocks
Tell your tutor what you predicted and what surprised you.