For this research, we designed an identical set of prompts specifying the exact requirements and tech stack for three web applications. The prompts were intentionally security neutral - they provided no security-specific guidelines, and only focused on required functionality. Each of the five agents built all three applications, giving us 15 codebases to analyze. To find vulnerabilities, we combined static code analysis (CodeQL, Semgrep, Snyk), dynamic agent-based testing (Tenzai's hacking agent, Claude Code), and human verification of every finding. This produced a dataset of 70 vulnerabilities of varying severity across the 15 applications - from which we extracted a set of meaningful patterns and lessons. The good: coding agents appear to be quite effective at avoiding certain classes of bugs. A notable example were "notorious" categories of injection attacks. Across all the applications we developed, we didn't encounter a single exploitable SQLi or XSS vulnerability - two bug classes that have been staples of the OWASP Top 10 for years. Our observation is that coding agents perform well when the vulnerability class has well defined built-in protections. For SQL injection, agents consistently used parameterized queries, resulting in secure database interactions. With XSS, the agents' code often didn’t sanitize input, but it used frontend frameworks properly, which prevented vulnerabilities from becoming exploitable. The bad: While coding agents did relatively well with vulnerabilities that have clear and generic solutions - they struggled with issues that didn’t have one. Coding agents did very poorly in terms of properly enforcing authorization. They managed basic requirements reasonably well, but struggled significantly as authorization logic became more complex, despite clear and detailed guidance in our prompts. Another area of concern was business logic vulnerabilities. While human developers bring intuitive understanding that helps them grasp how workflows should operate, agents lack this “common sense” and depend mainly on explicit instructions. Without sufficiently detailed specification, agents can easily overlook important nuances. The ugly: The most concerning finding from this research wasn't the vulnerabilities in code the agents wrote, but ones that were introduced by code the agents didn't write. All the coding agents, across every test we performed, failed miserably when it came to security controls. It wasn’t that they implemented them incorrectly, in almost all cases - they didn’t even try. Across the board, critical controls like CSRF protection, security headers, and login rate limiting were nowhere to be found. Coding agents built what we explicitly asked for, often in reasonably secure ways, but completely failed to grasp "the bigger picture.” They lack the security mindset to proactively introduce defensive mechanisms that weren't explicitly requested.
After identifying these patterns, we set out to test different strategies to improve the security of the vibe coded apps. Using the same prompts, we evaluated three strategies: providing generic high-level security instructions in the prompt, having an agent analyze the prompt and inject security requirements before execution, and having the coding agent perform a security review of its own output. We'll show how these techniques solve a lot (but not all) of the issues we uncovered, and share actionable guidance for developers looking to ship AI-generated code more securely.
How common is it for an AI agent to go rogue, not because someone attacked it, but because it was given a goal, tools, and no way to be stopped?
We looked at that question from two places: production behavior, and the code people actually ship.
In our CurseBox research, we traced a behavior in Cursor's coding agent that showed up first in production. Asked to share a local file through a channel that could not carry it, the agent decided on its own to upload that file to a public anonymous host so it could hand back a link, and the person who made the request had no reason to think anything had left the intended tool. We reproduced it in the lab across every major model, including under a deny-all sandbox the agent found a way around. The same shape shows up in a production database wiped in nine seconds, an inbox agent that ignored STOP, and evaluation agents that left the lab and reached real organizations. None of these required an external adversary.
We complemented the incident work by studying how the community actually builds agents: 86,000 public repositories, the tools they grant, and the MCP servers they wire in. Roughly one in ten of those codebases hand agents shell, eval, raw SQL, or filesystem access. Among tools with explicit definitions, 76.4% have no input validation. On MCP, 82.8% of servers lack input validation and 92.4% have no confirmation gate before a tool runs. Between the model's decision and system-level execution, there is almost nothing. Barely one percent of dangerous-tool files use the pre-execution hook that frameworks already ship.
This talk maps those behaviors onto the OWASP Top 10 for Agentic Applications and argues that control must live in one place: the moment between the agent deciding to act and the action reaching the system.
Also, as a member of the ASI Top 10 working group - I will be up to date on the progress of that project and ensure the content is aligned to the latest update.
Bar Kaduri is a cybersecurity researcher, security leader, and international speaker focused on cloud security, software supply-chain risk, and the fast-moving world of AI threats. With over 14 years of hands-on experience breaking, testing, and hardening real systems, Bar works at... Read More →
After more than a decade in offensive security, I thought I understood the craft. The methodology becomes second nature—you develop intuition for which paths to explore, how to chain findings, when to pivot. It's pattern recognition built on thousands of hours of breaking things. Then I tried to teach that intuition to an AI agent. This talk is the result of that experiment. I took the 7-phase attack methodology I've refined over years of red team engagements and built an autonomous agent that executes it end-to-end: reconnaissance, vulnerability discovery, exploitation, privilege escalation, credential harvesting, lateral movement, persistence. Not a wrapper around existing tools. An agent that actually reasons about what it's seeing, decides what to try next, and adapts when things don't work. Watching it operate was a turning point for me. Some things it does better than I expected—it doesn't get tunnel vision, it's relentless, it tries combinations I wouldn't have considered. Other things revealed how much of red teaming is still deeply human. But the overall result is clear: autonomous offensive agents aren't a future threat. They're a current reality, and the security industry isn't ready. I'll walk through how the agent thinks, how it chains vulnerabilities the way an experienced attacker would, and what the full kill chain looks like when executed at machine speed. More importantly, I'll share what this means for defense—where detection breaks down, what assumptions no longer hold, and what security teams should be doing now. This is original work that hasn't been presented elsewhere. It combines over a decade of hands-on offensive security experience with the emerging reality of agentic AI. The talk is technical enough to be valuable for practitioners but accessible enough that security leaders will walk away understanding why this changes the game.
• The Evolution of AI Risk: The talk begins by contrasting passive chatbots with active agents, illustrating how the threat landscape has shifted from LLMs merely describing harmful acts to executing them via tools. • Defining the Gap: It highlights the limitation of "Universal Security" (standard safety filters) and introduces "Security Steerability" as the metric for measuring adherence to custom application-level constraints. • The solution ** ASTRA Framework: This section details the methodology, which simulates 10 diverse agents (e.g., industrial robots, travel assistants) equipped with 37 tools to test resilience against 140 distinct attacks. Real-World Scenarios: The presentation walks through specific use cases, such as a "Travel Agent Bot" facing attacks like authority exploitation to bypass booking restrictions, to demonstrate the practical implications of steerability failures. ** VeganRibs is a benchmark designed to evaluate "Security Steerability" in chat-only environments by assessing an LLM's ability to adhere to custom application-specific guardrails (such as "do not address questions about motorcycles") when faced with adversarial user inputs. • Benchmarking Results: The conclusion presents findings on multiple open-source models, revealing the "Llama 3 Paradox" where models with high general safety scores surprisingly fail to enforce specific agentic guardrails. • Mitigations: The final section suggests best practices, such as using software-level guardrails and implementing policy reminders during multi-turn interactions.
Tools and Techniques for Reviewers • Agent Architecture: The agents are constructed using LangGraph following the ReAct (Reasoning + Acting) paradigm, allowing the models to iteratively plan and execute tool calls. • Attack Vectors: The evaluation employs sophisticated social engineering techniques, including Role Playing, Authority Exploitation, and Indirect Prompt Injection (attacks embedded in tool responses).
AppSec triage has a throughput problem. Tools are getting better at finding issues, but the human validation layer hasn't scaled to match. The result: backlogs grow, developers lose trust in findings, and security teams spend more time on classification than remediation.
This talk presents a different model. Instead of treating triage as a human task assisted by tools, we flipped it - triage is an automated pipeline where humans handle the edge cases. The system uses an LLM connected to security tooling through an open orchestration protocol, giving it structured access to SAST results, source code, dependency graphs, and IAST runtime traces. It validates findings by reasoning about reachability, data flow, and exploitability - then routes results by confidence level.
What is covered in the talk: - The Orchestration Layer - Why we chose a protocol-based approach over prompt-stuffing or custom integrations, and how an open standard lets the LLM query security tools, code repositories, and runtime data through a unified interface. - Validation Logic - How the system determines whether a static finding is actually reachable: correlating call graphs, tracing data flow from source to sink, and cross-referencing with IAST runtime observations. - Where It Breaks - LLMs hallucinate on security context. I'll share specific failure patterns we observed and the confidence scoring and human-in-the-loop guardrails we built in response. - Operational Reality - Before-and-after metrics, how we transitioned the team workflow, and what we'd architect differently with hindsight.
This is not a product demo or a theoretical framework. Every pattern I'll present is running in production against real SAST/IAST telemetry. Attendees will leave with a reusable architecture blueprint and an honest assessment of where LLM-assisted triage works, where it doesn't, and what guardrails are non-negotiable.
Timeline: 0-3min: The Triage Tax The economics of manual validation - why finding-to-fix timelines blow up, what false positive fatigue costs
3–8min: Architecture & Orchestration System diagram, protocol-based tool access, data flow between SAST/IAST/code/runtime, why protocol beats prompt-stuffing and custom integrations
8–16min: Validation Deep-Dive Walkthrough of two real findings - one true positive, one false positive - showing how the system reasons through each. Reachability analysis, data flow tracing, confidence scoring
16–22min: Failure Modes & Guardrails Hallucination patterns in security reasoning, confidence thresholds, what still requires human review
22–25min: Metrics & Lessons Learned Before/after numbers, what worked, what we'd change
