We benchmarked AquilaX's purpose-built Securitron AI against five leading general-purpose models — Claude, GPT-5.5, DeepSeek, Kimi, and Grok — on the job every AppSec team lives or dies by: telling a real vulnerability apart from scanner noise. Same 340 findings, same ground truth, six models, one independent manual review.
AquilaX's 32-engine scanner suite ran against a Python app, a Java/Maven app, and OWASP Juice Shop — plus its Solidity/Web3 and Terraform IaC challenges — producing 340 distinct findings. A human analyst independently labelled every one True Positive or False Positive (225 confirmed real, 115 confirmed benign). All six models then classified the same findings, blind to that ground truth.
Securitron leads every metric that matters — and the margin is driven overwhelmingly by recall.
| Model | Overall Accuracy | Real Vulns Caught | Real Vulns Missed | Benign Dismissed Correctly |
|---|---|---|---|---|
| Securitron | 90.6% (308/340) | 213 / 225 (94.7%) | 12 | 95 / 115 (82.6%) |
| Claude | 75.9% (258/340) | 159 / 225 (70.7%) | 66 | 99 / 115 (86.1%) |
| DeepSeek | 73.8% (251/340) | 151 / 225 (67.1%) | 74 | 100 / 115 (87.0%) |
| OpenAI | 70.9% (241/340) | 146 / 225 (64.9%) | 79 | 95 / 115 (82.6%) |
| Kimi | 69.7% (237/340) | 129 / 225 (57.3%) | 96 | 108 / 115 (93.9%) |
| Grok | 65.9% (224/340) | 121 / 225 (53.8%) | 104 | 103 / 115 (89.6%) |
Overall triage accuracy vs. manual-review ground truth, 340 findings.
Split the set into code/config-level findings (SAST, secrets, IaC, Solidity, API spec) and SCA/dependency/repo-policy findings (license, CVE, hygiene checks) and the gap widens sharply on the second.
| Model | Code & Config-Level (n=243) | SCA / Dependency & Repo Policy (n=97) |
|---|---|---|
| Securitron | 89.7% | 92.8% |
| Claude | 83.5% | 56.7% |
| DeepSeek | 81.9% | 53.6% |
| OpenAI | 79.8% | 48.5% |
| Kimi | 80.7% | 42.3% |
| Grok | 77.4% | 37.1% |
On code-level vulnerabilities — injection flaws, hardcoded secrets, access-control gaps, IaC misconfigurations — Securitron leads at 89.7%, with Claude closest at 83.5%. On dependency and repo-policy findings, matching a specific license/CVE/hygiene policy matrix rather than reasoning about code semantics, every general-purpose model drops well below its own code-level number — from 37.1% (Grok) to 56.7% at best (Claude) — while Securitron holds at 92.8%.
Five ecosystems: Python, Java/Maven, TypeScript/JavaScript, Solidity (Web3), and Terraform (IaC).
| Model | Python | Java (Maven/JSP) | TypeScript/JS | Solidity | Terraform | Other |
|---|---|---|---|---|---|---|
| Securitron | 100% | 100% | 86.8% | 98.0% | 78.7% | 91.8% |
| Claude | 100% | 77.8% | 94.7% | 91.8% | 72.1% | 50.0% |
| DeepSeek | 100% | 66.7% | 93.4% | 91.8% | 68.9% | 50.0% |
| OpenAI | 100% | 86.1% | 93.4% | 83.7% | 72.1% | 34.7% |
| Kimi | 95.0% | 80.6% | 93.4% | 89.8% | 72.1% | 30.6% |
| Grok | 95.0% | 66.7% | 89.5% | 89.8% | 70.5% | 26.5% |
Securitron leads four of five languages outright and ties for first on Python — with its widest lead on Solidity and Terraform, where general-purpose training data is thinnest. The one honest exception: on TypeScript/JavaScript, Securitron (86.8%) trails every general-purpose model, with Claude ahead at 94.7% — JS/TS security patterns are some of the most heavily represented examples in general LLM training data.
Five representative findings — no competing model calls all five correctly. Securitron does.
Securitron's 12 misses vs. Grok's 104, out of 225 confirmed real vulnerabilities.
It did not miss a single SQL/command injection, authentication, or access-control vulnerability across any codebase, and caught every seeded Java/Maven CVE.
An honest trade-off, not hidden: Securitron trails Claude/DeepSeek/Kimi/Grok slightly on specificity (82.6% vs. up to 93.9%) — but only Securitron pairs its precision with 94.7% recall.
Securitron isn't one model — it's a suite of components fine-tuned for each step of the triage pipeline, plus continuous per-customer learning that general-purpose LLMs simply don't have.
| Component | Base Model | Role |
|---|---|---|
| Review Model | GraphCodeBERT (encoder classifier), trained on verified findings + per-customer feedback | Classifies every finding TP/FP with a confidence score |
| AI Scanner Model | Qwen2.5-Coder-3B-Instruct, LoRA rank 512, 100K-file corpus | Deep semantic vulnerability analysis on source files |
| Security Assistant | Qwen2.5-Coder-0.5B, LoRA rank 256 | Answers developer questions about findings |
| Securitron Chat | ChatML, 8192-token context | Multi-turn conversational triage support |
| Query Model | FLAN-T5-base, fine-tuned NL→SQL | Translates natural-language questions into PostgreSQL queries |
Beyond the base training run, Securitron trains a dedicated model per customer on top of the shared foundation. Every accept/reject decision a security team makes feeds back into that customer's model — accuracy compounds sprint over sprint instead of staying fixed at deployment, the way a cold-start general-purpose LLM does on every single scan.
Methodology, the complete results tables, per-language and per-category breakdowns, representative finding evidence, and how Securitron is trained — all 11 pages, free to download, no form required.
Run AquilaX free and watch AI-driven triage cut through your scanner noise in minutes, not sprints.
Methodology note: All figures on this page are drawn directly from AquilaX's "AquilaX AI (Securitron) VS Frontier AI Models" benchmark report (August 2026), which classified 340 scanner findings from AquilaX-AI/vulnapp-python, AquilaX-AI/vulnapp-java, and OWASP Juice Shop against an independent manual security review as ground truth. Model versions tested: Securitron (per-customer Review Model, production build), DeepSeek V4 Flash, Claude Sonnet 4.6, GPT-5.5, Kimi K3, and Grok 3. See the full report for complete methodology and source data.