95% Fewer Tokens.
Same Answer Quality.
How DataGrout's Platform Advisor eliminated 95% of prompt bloat, cut latency by 93%, and reduced cost-per-query by 95% — using a technique called Selective Context Hydration.
The Naive RAG Trap
When building a documentation-grounded AI chatbot, the obvious first instinct is to inject everything. It's safe. It's simple. And it's extremely expensive.
Token Cost at Scale
Dumping 22 tool descriptions into every query costs ~3,240 context tokens — regardless of whether 20 of those tools are relevant.
Unnecessary Latency
Larger prompts mean slower first-token times. At ~4,800ms per query, a bloated catalog turns a snappy chatbot into a frustrating one.
Diluted Answer Quality
LLMs lose focus when the context is packed with irrelevant noise. Injecting 20 off-topic tools alongside 2 relevant ones degrades precision.
Selective Context Hydration
Instead of feeding the LLM everything and hoping it filters, we filter first — locally, for free — and only hand the model what it actually needs to answer the question.
Load All Docs
0 tokens · <50msAll tool documentation is read from a structured database. This is a fast database read — no LLM involved, no tokens spent.
Score & Rank Locally
0 tokens · <10msEach doc is scored against the user's question using keyword overlap and category matching — pure JavaScript, no model call. Top 3–6 docs are selected.
Inject Only What's Relevant
~148 tokens · LLM focusedOnly the highest-scoring docs enter the prompt. The LLM receives a tight, relevant context window — signal, not noise.
The key insight: The scoring step costs nothing — no LLM, no API call, no tokens. It's a fast local computation that makes every subsequent LLM call dramatically cheaper and more accurate. This is the same principle behind DataGrout's Logic tool's logic.hydrate primitive.
See It Side by Side
Step through the same query processed both ways — and watch the difference unfold.
The Numbers
| Metric | Before | After | Change |
|---|---|---|---|
| Context tokens per query | ~3,240 | ~148 | 95% reduction |
| Time to first answer | ~4,800ms | ~310ms | 93% faster |
| Estimated cost per query | $0.0097 | $0.00044 | 95% cheaper |
| Answer specificity | Broad, diluted | Precise, focused | Qualitative win |
| Scoring overhead | N/A | <10ms local | Zero tokens spent |
Why This Matters
Generalizable Architecture
This pattern works for any RAG-style agent operating over a known, bounded knowledge base — product docs, internal wikis, support playbooks.
"We eat our own cooking"
DataGrout built its own Platform Advisor using the same context hydration techniques available to all customers via the Logic tool.
Linear Cost Scaling
Because scoring is free and local, query costs scale with relevance — not catalog size. Adding 100 more tools doesn't increase per-query cost.
Auditable & Transparent
Every answer is traceable: you can see exactly which docs scored highest and why. Full observability without extra tooling.
Build Leaner, Smarter Agents
The same selective hydration technique is available to all DataGrout users via the Logic tool. Stop paying for context you don't need.