GitHub Copilot Token Efficiency in VS Code: What Cache Explorer Can Show
Technology News & Updates
VS Code’s November 4, 2026 guidance explains how developers can investigate AI credit usage, including by checking prompt-cache reuse in Cache Explorer. GitHub documented the underlying Copilot improvements in its April release roundup in May, and the VS Code team explained them in more detail in June. The November update is useful because it shows developers how to inspect the behavior in their own sessions.
Why can a short prompt still use many tokens?
An agent request can contain system instructions, tool definitions and conversation history before the model sees the latest message. If that early part stays consistent, a provider may reuse the matching prefix through prompt caching. A small change near the start of the request can reduce reuse in the following request. The user’s visible message length tells only part of the story.
What is deferred tool loading?
Deferred tool loading makes less frequently used tool definitions available on demand. An agent keeps a set of common tools ready and can look up additional capabilities when a task calls for them. The VS Code engineering post describes different implementations for different model providers, so the exact behavior depends on the model and configuration.
Consider an agent that normally reviews a few C# files but also has access to tools for browsing, deployment and issue tracking. Sending descriptions for every tool on every turn adds repeated context. Deferral limits how much of that catalog needs to be present initially. A later lookup still takes work, which is one reason commonly used tools can remain immediately available.
What can Cache Explorer tell you?
VS Code’s documentation says Cache Explorer compares consecutive requests and shows cache hits and input-token reuse. It can help narrow down whether changes to instructions, tools or messages affected the repeated prompt prefix. It does not by itself show that a single change caused every latency or billing difference.
What should teams check before changing their workflow?
Compare two similar sessions using the same model and a similar task. Look for changes in reused input tokens and which tools the agent actually needed. If a session has drifted across unrelated work, a new session can make the next task easier to understand. If the work is a direct follow-up, keeping the existing context may be more useful. Token use, credit use and task quality are separate measures.
Related on ASP.NET PB · How MCP servers expose tools.
Comments
Post a Comment