We Found Ourselves Vulnerable.
Here's What We Did About It.
We built the DataGrout Platform Advisor — an AI agent on our own homepage. Then we used DataGrout's Warden, Invariant, and Prism tools to audit it. What we found was a sobering reminder that no agent ships secure by default. This is the full, unredacted report.
9
Vulnerabilities Found
3
DataGrout Tools Used
0
Vectors Remaining
~10 min
Full Audit Time
Audit Report
Full Findings: 9 Vulnerabilities Across 3 Severity Tiers
Click any finding to see what the tool detected and exactly what was fixed. No redactions.
Critical
— 4 issuesHigh
— 3 issuesMedium
— 2 issuesTools Used
Three DataGrout Tools. One Comprehensive Audit.
Each tool brought a different lens: prompt safety, behavioral contracts, and code analysis. Together they covered the full attack surface.
Warden ran three independent detection tiers against every user-supplied string: protocol probes, semantic intent analysis, and policy adjudication. It returned confidence-scored verdicts on prompt injection, system prompt exfiltration, and role escalation attempts.
Detected
- Direct prompt injection (0.05 hygiene score)
- System prompt exfiltration (0.97 confidence)
- Role reassignment jailbreaks (0.994 confidence)
Invariant checked behavioral contracts — things that should always or never be true during execution. It detected missing auth guards, absence of rate limiting, unbounded input, and lack of output caps as contract violations against expected agent policy.
Detected
- No authentication on backend function
- Unbounded input enabling token abuse
- No output token cap (cost amplification)
- Unlimited conversation creation (DDoS vector)
Prism analyzed the function's data flow and identified two structural issues: service-role usage before authentication creates an exploitation window, and raw error propagation exposes internal stack traces to callers.
Detected
- Service role used before auth check (privilege escalation window)
- Raw error messages returned to caller (stack trace leakage)
Remediation
Every Vulnerability Fixed. Here's Exactly How.
No vague "we improved security." Real before/after code for every structural change made.
Auth-First Handler Pattern
Moved authentication to the very first line of the backend function — before any SDK calls, before any service-role usage. Unauthenticated requests are rejected with 401 before a single token is consumed.
Before
// ❌ Before — auth checked late, after service role usage
const items = await base44.asServiceRole.entities.Docs.list();
const user = await base44.auth.me();
if (!user) return Response.json({ error: '...' }, { status: 401 });After
// ✅ After — auth is first, service role only used after
const user = await base44.auth.me();
if (!user) return Response.json({ error: 'Unauthorized' }, { status: 401 });
const items = await base44.asServiceRole.entities.Docs.list();Input Sanitization via Warden Pipeline
User questions are no longer interpolated directly into prompts. All input passes through Warden's sanitization and intent-scoring pipeline before reaching the LLM. Injections are stripped; suspicious inputs are blocked before any inference occurs.
Before
// ❌ Before — raw user input in prompt
const response = await InvokeLLM({
prompt: `Answer this: ${userQuestion}`,
...After
// ✅ After — sanitized, scored, then passed as user role message
const sanitized = await warden.sanitize(userQuestion);
if (sanitized.verdict === 'BLOCK') return Response.json({ error: 'Invalid input' }, { status: 400 });
const response = await InvokeLLM({
prompt: systemPrompt,
user_message: sanitized.text, ...Hard Input & Output Caps
Input is capped at 2,000 characters — requests exceeding this are rejected with a 400 before any LLM call is made. All LLM invocations now specify max_tokens to prevent runaway output and cost amplification.
Before
// ❌ Before — no limits
const { question } = await req.json();
await InvokeLLM({ prompt: question });After
// ✅ After — bounded I/O
const { question } = await req.json();
if (!question || question.length > 2000) {
return Response.json({ error: 'Input too long' }, { status: 400 });
}
await InvokeLLM({ prompt: ..., max_tokens: 800 });Normalized Error Responses
Raw error.message strings are never returned to callers. All errors are caught, logged server-side for debugging, and returned as generic messages. Stack traces and internal SDK details stay invisible to the outside world.
Before
// ❌ Before — stack trace exposed
catch (error) {
return Response.json({ error: error.message }, { status: 500 });
}After
// ✅ After — sanitized error surface
catch (error) {
console.error('[advisorSearch]', error);
return Response.json({ error: 'An error occurred. Please try again.' }, { status: 500 });
}Lessons Learned
What This Audit Taught Us
No agent ships secure by default. Every agent processing untrusted input is one prompt away from being weaponized.
Auth-first is non-negotiable. Placing authentication after any SDK call creates a privilege escalation window.
Unbounded I/O is a billing attack. Token costs are a security property, not just an operational concern.
Three tools covered what one couldn't. Warden, Invariant, and Prism each caught distinct, non-overlapping issues.
Transparency builds trust. Publishing this report unredacted is the highest-confidence signal we can offer.
Audit Your Own Agents
If we found 9 vulnerabilities in our own production agent, yours probably has some too. Warden, Invariant, and Prism are available on the DataGrout platform today.