AppSec triage has a throughput problem. Tools are getting better at finding issues, but the human validation layer hasn't scaled to match. The result: backlogs grow, developers lose trust in findings, and security teams spend more time on classification than remediation.
This talk presents a different model. Instead of treating triage as a human task assisted by tools, we flipped it - triage is an automated pipeline where humans handle the edge cases. The system uses an LLM connected to security tooling through an open orchestration protocol, giving it structured access to SAST results, source code, dependency graphs, and IAST runtime traces. It validates findings by reasoning about reachability, data flow, and exploitability - then routes results by confidence level.
What is covered in the talk: - The Orchestration Layer - Why we chose a protocol-based approach over prompt-stuffing or custom integrations, and how an open standard lets the LLM query security tools, code repositories, and runtime data through a unified interface. - Validation Logic - How the system determines whether a static finding is actually reachable: correlating call graphs, tracing data flow from source to sink, and cross-referencing with IAST runtime observations. - Where It Breaks - LLMs hallucinate on security context. I'll share specific failure patterns we observed and the confidence scoring and human-in-the-loop guardrails we built in response. - Operational Reality - Before-and-after metrics, how we transitioned the team workflow, and what we'd architect differently with hindsight.
This is not a product demo or a theoretical framework. Every pattern I'll present is running in production against real SAST/IAST telemetry. Attendees will leave with a reusable architecture blueprint and an honest assessment of where LLM-assisted triage works, where it doesn't, and what guardrails are non-negotiable.
Timeline: 0-3min: The Triage Tax The economics of manual validation - why finding-to-fix timelines blow up, what false positive fatigue costs
3–8min: Architecture & Orchestration System diagram, protocol-based tool access, data flow between SAST/IAST/code/runtime, why protocol beats prompt-stuffing and custom integrations
8–16min: Validation Deep-Dive Walkthrough of two real findings - one true positive, one false positive - showing how the system reasons through each. Reachability analysis, data flow tracing, confidence scoring
16–22min: Failure Modes & Guardrails Hallucination patterns in security reasoning, confidence thresholds, what still requires human review
22–25min: Metrics & Lessons Learned Before/after numbers, what worked, what we'd change
