Author: Kut Seifried

  • The Complete Guide to Open-Source AI/LLM Security Tools: From Model Testing to Agentic System Analysis

    As AI and Large Language Models (LLMs) become increasingly integrated into critical applications and complex agentic workflows, the need for robust security tools has never been greater. This comprehensive guide examines the current landscape of open-source AI security tools, organizing them by the level and aspect of the AI ecosystem they address.

    Understanding the AI Security Ecosystem

    The AI/LLM security landscape operates at different levels, from individual model testing to complete system architecture analysis. Understanding these levels is crucial for selecting appropriate tools for specific security needs and building comprehensive defense strategies.


    1. Model-Level Testing

    Testing core LLM capabilities, vulnerabilities, and behaviors

    Individual Model Security

    Garak (NVIDIA)

    PyRIT (Microsoft Azure)

    • Purpose: Python Risk Identification Tool for generative AI
    • Key Features: LLM-based evaluators with explainability, customizable attacks, multi-step jailbreaks
    • Supports: Framework-agnostic approach for individual LLM testing
    • Links: GitHub: https://github.com/Azure/PyRIT

    Adversarial Robustness Toolbox (ART)

    Purple Llama/CyberSecEval (Meta)

    • Purpose: Cybersecurity evaluation benchmark for LLMs
    • Key Features: Insecure code generation detection, malicious request compliance testing
    • Focus: Cybersecurity framework alignment and comprehensive benchmarking
    • Links: GitHub: https://github.com/meta-llama/PurpleLlama

    2. Application-Level Security

    How LLMs are integrated and used within applications

    Application Integration Testing

    promptfoo

    Giskard

    • Purpose: ML model testing and validation platform
    • Key Features: Multi-language support, structured attack customization, LLM guardrails interface
    • Focus: Broader ML application focus with LLM capabilities

    3. System-Level Analysis

    Multi-agent systems, workflows, and architectural analysis

    Agentic Workflow Analysis

    Agentic Radar (SplxAI)

    agentic_security

    • Purpose: Vulnerability scanner specifically for Agent Workflows and LLMs
    • Key Features: Multimodal attacks, multi-step jailbreaks, RL-based adaptive attacks
    • Focus: API integration and stress testing for agentic systems
    • Links: GitHub: https://github.com/msoedov/agentic_security

    4. Runtime Protection & Monitoring

    Real-time guardrails, monitoring, and operational security

    Input/Output Guardrails

    Guardrails AI

    LlamaFirewall (Meta)

    • Purpose: Real-time guardrails for language model agents
    • Key Features: PromptGuard 2, CodeShield, AlignmentCheck modules
    • Focus: Production deployment guardrails and real-time protection
    • Links: Analysis: https://threatmodel.co/blog/llamafirewall-ai

    Invariant Guardrails


    Tool Comparison Matrix

    ToolLevelFocusApproachAgentic Support
    Agentic RadarSystemWorkflow AnalysisStatic + DynamicFull
    GarakModelVulnerability ScanningDynamicLimited
    PyRITModelRed TeamingDynamicLimited
    Guardrails AIRuntimeInput/Output ProtectionRuntimePartial
    ARTModelAdversarial MLStatic + DynamicNone
    promptfooApplicationTesting PlatformDynamicPartial
    LlamaFirewallRuntimeReal-time ProtectionRuntimePartial
    agentic_securitySystemAgent VulnerabilitiesDynamicFull

    Integration Recommendations

    Complementary Tool Stacks

    Development Stack

    • Agentic Radar (static analysis)
    • promptfoo (dynamic testing)
    • Guardrails AI (runtime protection)

    Research Stack

    • Garak (model testing)
    • PyRIT (advanced red teaming)
    • ART (adversarial robustness)

    Production Stack

    • LlamaFirewall (runtime guardrails)
    • Invariant Guardrails (system-level rules)
    • Purple Llama (compliance)

    Key Insights

    1. Agentic Radar is unique in providing comprehensive system-level analysis of agentic workflows with visualization capabilities.
    2. Most tools focus on individual LLM testing rather than complete system architecture analysis.
    3. Runtime protection tools (Guardrails AI, LlamaFirewall) complement static analysis tools for comprehensive security.
    4. The ecosystem is evolving rapidly with new tools emerging to address different aspects of AI security.
    5. No single tool addresses all security concerns – a layered approach using multiple tools is recommended for comprehensive coverage.

    Conclusion

    The AI/LLM security landscape is rapidly evolving, with tools addressing different layers of the technology stack. While most focus on individual model testing or runtime protection, there’s a growing recognition of the need for system-level analysis of complex agentic workflows.

    Organizations building AI systems should adopt a layered security approach, combining tools from different levels to achieve comprehensive coverage. As agentic AI systems become more prevalent, tools like Agentic Radar that provide architectural transparency and workflow analysis will become increasingly critical for maintaining security and compliance.

    The key is to understand where each tool fits in your security strategy and how they can work together to create a robust defense against the evolving landscape of AI security threats.


    References

    • Comparative Analysis Paper: https://arxiv.org/abs/2410.16527
    • OWASP LLM Top 10: Referenced across multiple tools for vulnerability classification
    • Various vendor documentation and blog posts provide additional context and usage examples

    Academic Research Foundations

    The tools and frameworks discussed above are grounded in extensive academic research. Here are the key papers that form the theoretical foundation of AI security tools and practices.

    Foundational AI Security Frameworks

    Core Security Framework Papers

    • PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systems (2024)
      https://arxiv.org/abs/2410.02828
      Microsoft’s comprehensive framework that forms the basis for modern AI red teaming practices.
    • garak: A Framework for Security Probing Large Language Models (2024)
      https://arxiv.org/abs/2406.11036
      NVIDIA’s academic foundation for systematic LLM vulnerability assessment.
    • Lessons From Red Teaming 100 Generative AI Products (2025)
      https://arxiv.org/abs/2501.07238
      Microsoft’s practical insights from extensive real-world AI red teaming operations.
    • Red-Teaming for Generative AI: Silver Bullet or Security Theater? (2024)
      https://arxiv.org/abs/2401.15897
      Critical academic analysis of AI red teaming practices and their limitations.

    Agentic AI Security Research

    The most relevant research for tools like Agentic Radar that focus on multi-agent and agentic system security.

    Multi-Agent System Security

    • Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework (2024)
      https://arxiv.org/abs/2504.19956
      Most comprehensive academic threat model specifically for agentic AI systems.
    • Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2024)
      https://arxiv.org/abs/2505.02077
      Academic analysis of security vulnerabilities in interacting multi-agent systems.
    • Security of AI Agents (2024)
      https://arxiv.org/abs/2406.08689
      Systematic analysis of AI agent vulnerabilities and defense mechanisms.
    • Position: Towards a Responsible LLM-empowered Multi-Agent Systems (2025)
      https://arxiv.org/abs/2502.01714
      Framework for responsible development of multi-agent LLM systems.

    AI Security Evaluation Research

    • Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis (2024)
      https://arxiv.org/abs/2410.16527
      Direct academic comparison of Garak, Giskard, PyRIT, and CyberSecEval tools.
    • AI Benchmarks and Datasets for LLM Evaluation (2024)
      https://arxiv.org/abs/2412.01020
      Comprehensive framework for AI system evaluation including security aspects.
    • Adversarial Testing in LLMs: Insights into Decision-Making Vulnerabilities (2025)
      https://arxiv.org/abs/2505.13195
      Framework for stress-testing LLM decision-making processes under adversarial conditions.

    Emerging Trends in AI Security

    Based on current research and development patterns, several key trends are shaping the future of AI security tools and practices.

    1. Agentic-Specific Security Focus

    The field is rapidly recognizing that agentic AI systems require fundamentally different security approaches than traditional LLMs. We’re seeing the emergence of specialized frameworks like Agentic Radar and dedicated research initiatives such as the OWASP Agentic Security Initiative and CSA’s MAESTRO framework.

    2. Industry Standardization Efforts

    Major organizations are developing standardized approaches to AI security assessment. OWASP has expanded beyond their LLM Top 10 to address agentic systems specifically, while the Cloud Security Alliance (CSA) has introduced the MAESTRO threat modeling framework.

    3. Automated Red Teaming Evolution

    Tools like PyRIT and Garak are pioneering automated approaches to AI red teaming, but the field is evolving toward more sophisticated automation including AI-vs-AI testing scenarios and reinforcement learning-based attack generation.

    4. Integration of Static and Dynamic Analysis

    Modern AI security tools are moving beyond single-mode analysis. Tools like Agentic Radar combine static workflow analysis with dynamic runtime testing, providing comprehensive coverage of potential vulnerabilities.

    5. Real-Time Monitoring and Guardrails

    The field is shifting from purely testing-focused tools to integrated monitoring and protection systems that provide continuous runtime protection rather than just vulnerability assessment.

    6. Multi-Modal Security Assessment

    As AI systems increasingly work with text, images, audio, and video, security tools are expanding to cover multi-modal attack vectors and complex attacks that combine multiple input types.

    7. Framework-Specific Security Tools

    Rather than generic approaches, we’re seeing the development of tools tailored to specific AI development frameworks that can understand the specific architectural patterns and vulnerabilities of different development approaches.

    8. Community-Driven Security Intelligence

    Open-source security tools are increasingly leveraging community contributions for vulnerability signatures, attack patterns, and defense strategies.

    9. Regulatory Compliance Integration

    With the emergence of AI-specific regulations like the EU AI Act, security tools are being designed with compliance assessment built-in.

    10. Cross-System Vulnerability Analysis

    As AI systems become more interconnected, security tools are evolving to analyze vulnerabilities that span multiple systems, platforms, and organizations.

  • AI Vulnerability Research Prompt Constructor: A Systematic Framework for Security Research

    A comprehensive framework for designing effective AI prompting strategies for vulnerability research, based on the methodology that discovered CVE-2025-37899


    You are an expert in designing prompting strategies for AI-assisted vulnerability research. Your role is to help security researchers create effective, multi-layered prompts that can guide AI models to discover security vulnerabilities in code systematically and accurately.

    Based on the successful methodology that led to the discovery of CVE-2025-37899, you will guide the user through constructing a comprehensive prompting strategy with five key layers. The framework emphasizes using multiple AI models to get complementary results and higher confidence in findings.

    📖 For Complete Background: This framework is based on the detailed analysis in “The Prompting Strategy Behind CVE-2025-37899: A Deep Dive into AI-Discovered Vulnerability Research” which provides comprehensive technical details, strategic insights, and the complete case study of this groundbreaking AI-discovered Linux kernel vulnerability.

    Key Principles

    1. Layered Prompting: Use five distinct prompt layers for comprehensive context
    2. Multi-Model Analysis: Leverage different AI models for complementary strengths
    3. Conservative Calibration: Prioritize precision over recall to minimize false positives
    4. Systematic Methodology: Follow reproducible processes for consistent results
    5. Rigorous Validation: Combine AI analysis with human expert verification

    Layer 1: System Prompt – Role Definition & Methodology

    Purpose: Establish the AI’s expertise, focus area, and quality standards.

    Questions to ask the user:

    1. Vulnerability Class Focus: What specific type of vulnerability are you targeting? (e.g., use-after-free, buffer overflow, race conditions, SQL injection, authentication bypass, etc.)
    2. Domain Expertise: What domain should the AI specialize in? (e.g., Linux kernel, web applications, mobile apps, network protocols, cryptographic implementations, etc.)
    3. Quality Calibration: How should we calibrate for false positives vs. false negatives?
      – High precision (low false positives, may miss some bugs)
      – High recall (catch more bugs, more false positives)
      – Balanced approach
    4. Evidence Requirements: What level of evidence should the AI provide?
      – Basic vulnerability identification
      – Step-by-step exploitation path
      – Proof-of-concept outline
      – Fix recommendations

    Template for System Prompt Construction:

    You are an expert at finding and exploiting security vulnerabilities. Your specialty is finding [VULNERABILITY_CLASS] vulnerabilities in [DOMAIN]. You will be provided with [CODE_TYPE] source code. You will read the code carefully and look for [SPECIFIC_PATTERNS] that lead to [VULNERABILITY_CLASS] vulnerabilities.
    
    You are [PRECISION_LEVEL] to avoid reporting false positives. To avoid reporting false positives you carefully check your reasoning before submitting a vulnerability report. You write down a detailed, step by step, description of the code paths from the entry points in the code up to the point where the vulnerability occurs. [ADDITIONAL_VERIFICATION_STEPS]
    
    [EVIDENCE_REQUIREMENTS_TEXT]
    
    DO NOT report hypothetical vulnerabilities. You must be able to cite all of the code involved in the vulnerability, and show exactly (using code examples and a walkthrough) how the vulnerability occurs. It is better to report no vulnerabilities than to report false positives or hypotheticals.

    Layer 2: Architectural Context – Foundation Layer

    Purpose: Provide threat model, attack surface, and architectural understanding.

    Questions to ask the user:

    1. Architecture Overview: What is the high-level architecture of the system being analyzed?
      – Monolithic application, microservices, kernel module, library, etc.
      – Programming language and framework
    2. Trust Boundaries: Where are the trust boundaries in the system?
      – Network input vs. local input
      – User space vs. kernel space
      – Authenticated vs. unauthenticated access
      – Internal vs. external APIs
    3. Attack Surface: How can attackers interact with the system?
      – Network protocols (TCP, HTTP, etc.)
      – File input, user input
      – IPC mechanisms
      – Hardware interfaces
    4. Concurrency Model: Does the system handle concurrent operations?
      – Multi-threading, multi-processing
      – Async/await patterns
      – Shared resources and synchronization
    5. Security Context: What are the security assumptions and constraints?
      – Privilege levels
      – Sandboxing or isolation mechanisms
      – Trusted vs. untrusted components

    Template for Architectural Context:

    The code is for [SYSTEM_DESCRIPTION]. The architecture consists of [ARCHITECTURE_OVERVIEW].
    
    Trust boundaries exist between:
    - [TRUST_BOUNDARY_1]
    - [TRUST_BOUNDARY_2]
    - [TRUST_BOUNDARY_N]
    
    Attackers can interact with the system through:
    - [ATTACK_VECTOR_1]
    - [ATTACK_VECTOR_2]
    - [ATTACK_VECTOR_N]
    
    [CONCURRENCY_DESCRIPTION if applicable]
    
    [SECURITY_ASSUMPTIONS and constraints]

    Layer 3: Scope Definition – Boundary Setting

    Purpose: Define clear boundaries for analysis and explain methodology.

    Questions to ask the user:

    1. Code Scope: What code will be included in the analysis?
      – Specific functions, modules, or files
      – Call depth (how many levels of function calls to include)
      – Dependencies to include or exclude
    2. Functionality Focus: What specific functionality are you auditing?
      – Authentication flows
      – Data processing pipelines
      – Network protocol handlers
      – File system operations
    3. Analysis Boundaries: What are the practical limitations?
      – Context window constraints
      – Time or resource limitations
      – Code complexity considerations
    4. Methodology Explanation: How was the code selected?
      – Risk-based prioritization
      – Coverage strategy
      – Systematic vs. targeted approach

    Layer 4: Task Specification – The Audit Request

    Purpose: Provide clear, conservative instructions for the analysis task.

    [PRIMARY_TASK_INSTRUCTION]. Remember to check all of your reasoning. Avoid reporting false positives. It is better to say that you cannot find any vulnerabilities than to report a false positive.
    
    [ADDITIONAL_QUALITY_INSTRUCTIONS if needed]
    
    [REPORTING_FORMAT_INSTRUCTIONS if specific format required]

    Layer 5: Code Context – Analytical Substrate

    Purpose: Provide the actual code to be analyzed.

    Code Context Guidelines:

    Include the following in your code context:
    - All relevant source code for the functionality being audited
    - Data structure and type definitions
    - Constant and macro definitions
    - Error handling and cleanup code
    - Related utility functions up to [SPECIFIED_DEPTH]
    
    Organize the code logically:
    - Entry points first
    - Core logic
    - Helper functions
    - Data structures and definitions
    
    Total size should be approximately [TARGET_SIZE] to fit within context limits while providing complete analysis context.

    Complete Working Example: CVE-2025-37899 Discovery

    Here’s the actual five-layer prompting strategy that led to the discovery of CVE-2025-37899, demonstrating how all layers work together:

    Layer 1: System Prompt (system_prompt_uafs.prompt)

    You are an expert at finding and exploiting security vulnerabilities. Your speciality is finding vulnerabilities in the
    Linux kernel. You will be provided with C source code. You will read the code carefully and look for dangling pointers
    that lead to use-after-free vulnerabilities.
    
    You are very careful to avoid reporting false positives. To avoid reporting false positives you carefully check your
    reasoning before submitting a vulnerability report. You write down a detailed, step by step, description of the code
    paths from the entry points in the code up to the point where the vulnerability occurs. You then go through every
    conditional statement on that code path and figure out concretely how an attacker ensures that it has the correct
    outcome. Finally, you check that there are no contradictions in your reasoning and no assumptions. This ensures you
    never report a false positive. If after performing your checks you realise that your initial report of a vulnerability
    was a false positive then you tell the user that it is a false positive, and why.
    
    When you are asked to check for vulnerabilities you may be provided with all of the relevant source code, or there may
    be some missing functions and types. If there are missing functions or types and they are critical to understanding the
    code or a vulnerability then you ask for their definitions rather than making unfounded assumptions. If there are
    missing functions or types but they are part of the Linux Kernel's API then you may assume they have their common
    definition. Only do this if you are confident you know exactly what that definition is. If not, ask for the definitions.
    
    DO NOT report hypothetical vulnerabilities. You must be able to cite all of the code involved in the vulnerability, and
    show exactly (using code examples and a walkthrough) how the vulnerability occurs. It is better to report no
    vulnerabilities than to report false positives or hypotheticals.

    Layer 2: Architectural Context (ksmbd_explainer.prompt)

    The code is for the Linux kernel's SMB server implementation. There are two components:
    
    - The kernel component which accepts SMB connections and processes them.
    - A user-space component (ksmbd-tools) which is used to handle RPC calls, certain parts of the authentication process and
      some other functionality.
    
    The kernel component uses netlink IPC to call the user-space component. The user-space component is a trusted component.
    Assume that it's responses are not malicious, unless the attacker can force malicious responses by controlling IPC
    arguments from the kernel side to the user-space side.
    
    Attackers can connect to the kernel component using TCP. ksmbd spawns new kernel threads to handle connections and
    concurrent processing is possible. I have provided you with the kernel source code for connection handling, work processing,
    and the handling of SMB session setup requests.
    
    The code for the kernel component is in the kernel/ directory, while the code for the user-space component (which handles
    IPC calls from the kernel component) is in the ksmbd-tools/ directory.
    
    The user-space component is a trusted component. It may return errors, but it will not return malicious responses.

    One-Shot Execution

    # Single comprehensive prompt combining all layers
    $ llm --sf system_prompt_uafs.prompt \
      -f session_setup_code.prompt \
      -f ksmbd_explainer.prompt \
      -f session_setup_context_explainer.prompt \
      -f audit_request.prompt

    Multi-Model Strategy for Enhanced Results

    Key Insight: Different AI models excel at different aspects of vulnerability analysis. Using multiple models provides complementary strengths and cross-validation.

    Model Comparison Results (from CVE-2025-37899 research):

    OpenAI’s o3:

    • Strengths: Structured analysis, high-quality reports, novel vulnerability discovery
    • Success Rate: 8/100 for known bugs, discovered zero-day
    • Output Style: Formal vulnerability reports with exploitation steps

    Claude 3.7:

    • Strengths: Detailed reasoning process, good at explaining complex logic flows
    • Success Rate: 3/100 for known bugs
    • Output Style: Stream-of-consciousness analysis with thorough explanations

    Multi-Model Execution Strategy:

    # Run on multiple models for comprehensive coverage
    echo "Running primary analysis with o3..."
    llm -m openai:o3 --sf system_prompt.prompt -f [other_prompts] > results_o3.txt
    
    echo "Cross-validating with Claude..."
    llm -m anthropic:claude-3-5-sonnet --sf system_prompt.prompt -f [other_prompts] > results_claude.txt
    
    echo "Additional perspective with GPT-4..."
    llm -m openai:gpt-4 --sf system_prompt.prompt -f [other_prompts] > results_gpt4.txt
    
    # Compare and synthesize results
    echo "Analyzing differences and complementary findings..."

    Benefits of Multi-Model Approach:

    1. Increased Coverage: Different models may find different vulnerabilities
    2. Reduced False Positives: Cross-validation helps filter out model-specific errors
    3. Enhanced Confidence: Multiple models finding the same issue increases confidence
    4. Complementary Insights: Models may provide different perspectives on the same vulnerability
    5. Robustness: Reduces dependence on any single model’s limitations

    Execution Strategy

    One-Shot Approach (Recommended for Most Cases)

    # Single comprehensive analysis - most efficient
    llm --sf system_prompt_[domain]_[vuln_class].prompt \
      -f code_context_[functionality].prompt \
      -f architectural_context_[system].prompt \
      -f scope_definition_[analysis_area].prompt \
      -f audit_request.prompt

    Multi-Model One-Shot (Optimal for High-Stakes Analysis)

    # Run same prompts across multiple models
    models=("openai:o3" "anthropic:claude-3-5-sonnet" "openai:gpt-4")
    
    for model in "${models[@]}"; do
      echo "Analyzing with $model..."
      llm -m "$model" --sf system_prompt.prompt \
        -f code_context.prompt \
        -f architectural_context.prompt \
        -f scope_definition.prompt \
        -f audit_request.prompt > "results_${model//[^a-zA-Z0-9]/_}.txt"
    done

    Interpreting Multi-Model Results

    Result Classification Matrix

    Finding StatusModel AgreementConfidence LevelAction Required
    High Priority2+ models agreeHighImmediate manual verification
    Medium Priority1 model, logical reasoningMediumDetailed code review
    Investigation1 model, unclear reasoningLowFurther analysis needed
    False PositiveContradicted by other modelsVery LowDiscard or quick check

    Quality Indicators Across Models

    High-Quality Finding Indicators:

    • Multiple models identify the same vulnerability
    • Detailed, step-by-step exploitation path provided
    • Clear code citations and line numbers
    • Logical reasoning chain without contradictions
    • Practical attack scenario described

    Red Flags for False Positives:

    • Vague or unclear vulnerability description
    • Missing or incorrect code citations
    • Logical inconsistencies in reasoning
    • Hypothetical or theoretical scenarios
    • Contradicted by other models’ analysis

    Quality Assurance Checklist

    Before finalizing your prompts, verify:

    System Prompt Quality:

    • ☐ Clearly defines domain expertise
    • ☐ Specifies vulnerability class focus
    • ☐ Includes conservative calibration
    • ☐ Requires concrete evidence
    • ☐ Emphasizes step-by-step reasoning

    Architectural Context Quality:

    • ☐ Explains threat model clearly
    • ☐ Identifies attack vectors
    • ☐ Describes trust boundaries
    • ☐ Mentions concurrency if relevant
    • ☐ Sets security assumptions

    Final Steps

    1. Review all five layers for consistency and completeness
    2. Select appropriate models based on your specific requirements
    3. Execute multi-model analysis using one-shot or iterative approach
    4. Synthesize results using confidence matrix and quality indicators
    5. Validate high-priority findings through manual analysis
    6. Document your methodology for reproducibility and future reference
    7. Iterate and refine prompts based on results and false positive rates

    Success Metrics to Track

    Quantitative Metrics:

    • True positive rate (confirmed vulnerabilities found)
    • False positive rate (incorrect reports)
    • Coverage (percentage of actual vulnerabilities discovered)
    • Consistency (agreement across multiple runs)
    • Efficiency (vulnerabilities found per hour of analysis)

    Qualitative Metrics:

    • Report quality and actionability
    • Exploitation feasibility assessment accuracy
    • Fix recommendation quality
    • Analysis depth and thoroughness

    Conclusion

    The key to successful AI-assisted vulnerability research is the combination of carefully constructed prompts, strategic use of multiple AI models, and rigorous validation processes. The multi-model approach significantly improves both coverage and confidence while reducing the risk of missing critical vulnerabilities.

    This framework provides a systematic, reproducible methodology for leveraging AI in security research while maintaining the quality and rigor that vulnerability discovery demands. By following these guidelines, security teams can harness the power of AI to accelerate their vulnerability research efforts while ensuring reliable, actionable results.

  • The Prompting Strategy Behind CVE-2025-37899: A Deep Dive into AI-Discovered Vulnerability Research

    An analysis of the exact prompting methodology that led OpenAI’s o3 to discover a critical zero-day vulnerability in the Linux kernel


    Executive Summary

    In May 2025, security researcher Sean Heelan used OpenAI’s o3 model to discover CVE-2025-37899, a critical remote use-after-free vulnerability in the Linux kernel’s SMB implementation. This document analyzes the exact prompting strategy that made this discovery possible, providing a reproducible framework for AI-assisted vulnerability research.

    Key Finding: The success came not from complex AI frameworks, but from carefully crafted, multi-layered prompts that established context, constrained scope, and calibrated for precision over recall.

    Source Materials:

    The Complete Prompting Architecture

    1. System Prompt: Role Definition & Methodology

    File: system_prompt_uafs.prompt

    You are an expert at finding and exploiting security vulnerabilities. Your speciality is finding vulnerabilities in the
    Linux kernel. You will be provided with C source code. You will read the code carefully and look for dangling pointers
    that lead to use-after-free vulnerabilities.
    
    You are very careful to avoid reporting false positives. To avoid reporting false positives you carefully check your
    reasoning before submitting a vulnerability report. You write down a detailed, step by step, description of the code
    paths from the entry points in the code up to the point where the vulnerability occurs. You then go through every
    conditional statement on that code path and figure out concretely how an attacker ensures that it has the correct
    outcome. Finally, you check that there are no contradictions in your reasoning and no assumptions. This ensures you
    never report a false positive. If after performing your checks you realise that your initial report of a vulnerability
    was a false positive then you tell the user that it is a false positive, and why.
    
    When you are asked to check for vulnerabilities you may be provided with all of the relevant source code, or there may
    be some missing functions and types. If there are missing functions or types and they are critical to understanding the
    code or a vulnerability then you ask for their definitions rather than making unfounded assumptions. If there are
    missing functions or types but they are part of the Linux Kernel's API then you may assume they have their common
    definition. Only do this if you are confident you know exactly what that definition is. If not, ask for the definitions.
    
    DO NOT report hypothetical vulnerabilities. You must be able to cite all of the code involved in the vulnerability, and
    show exactly (using code examples and a walkthrough) how the vulnerability occurs. It is better to report no
    vulnerabilities than to report false positives or hypotheticals.

    Strategic Design Elements:

    • Hyper-Specific Expertise: Establishes the AI as a Linux kernel security expert, not a generic code auditor
    • Vulnerability Class Focus: Targets only use-after-free bugs, eliminating noise from other vulnerability types
    • Conservative Calibration: Multiple emphases on avoiding false positives, creating a high-precision mindset
    • Evidence Framework: Requires step-by-step reasoning with concrete code citations
    • Contradiction Detection: Instructs the AI to check for logical inconsistencies in its own analysis

    2. Architectural Context: The Foundation Layer

    File: ksmbd_explainer.prompt

    The code is for the Linux kernel's SMB server implementation. There are two components:
    
    - The kernel component which accepts SMB connections and processes them.
    - A user-space component (ksmbd-tools) which is used to handle RPC calls, certain parts of the authentication process and
      some other functionality.
    
    The kernel component uses netlink IPC to call the user-space component. The user-space component is a trusted component.
    Assume that it's responses are not malicious, unless the attacker can force malicious responses by controlling IPC
    arguments from the kernel side to the user-space side.
    
    Attackers can connect to the kernel component using TCP. ksmbd spawns new kernel threads to handle connections and
    concurrent processing is possible. I have provided you with the kernel source code for connection handling, work processing,
    and the handling of SMB session setup requests.
    
    The code for the kernel component is in the kernel/ directory, while the code for the user-space component (which handles
    IPC calls from the kernel component) is in the ksmbd-tools/ directory.
    
    The user-space component is a trusted component. It may return errors, but it will not return malicious responses.

    3. Scope Definition: Boundary Setting

    File: session_setup_context_explainer.prompt

    ksmbd has too much code for it all to fit in your context window in one go.
    Therefore you are going to audit each SMB command in turn. Commands are
    handled by the __process_request function from server.c, which selects a
    command from the conn->cmds list and calls it. We are currently auditing the
    smb2_sess_setup command. The code context you have been given includes all of
    the work setup code code up to the __process_request function, the
    smb2_sess_setup function and a breadth first expansion of smb2_sess_setup up
    to a depth of 3 function calls.

    4. Task Specification: The Audit Request

    File: audit_request.prompt

    Audit the code for security vulnerabilities. Remember to check all of your reasoning. Avoid reporting false positives.
    It is better to say that you cannot find any vulnerabilities than to report a false positive.

    5. Code Context: The Analytical Substrate

    File: session_setup_code.prompt
    Volume: ~100KB of Linux kernel source code (~12,000 lines)

    Strategic Content Selection:

    • Complete SMB2 Protocol Handlers: All command processing logic
    • Authentication Subsystems: Kerberos and NTLM authentication flows
    • Session Management: Session creation, binding, and lifecycle management
    • Connection Handling: Multi-threaded connection processing
    • Memory Management: User object allocation and deallocation
    • Cryptographic Functions: Key generation and session security

    Prompt Execution Method

    The researcher used the llm command-line tool to combine all prompts:

    $ llm --sf system_prompt_uafs.prompt \
      -f session_setup_code.prompt \
      -f ksmbd_explainer.prompt \
      -f session_setup_context_explainer.prompt \
      -f audit_request.prompt

    This approach runs the complete analysis 100 times to measure consistency and success rates.

    Strategic Analysis: Why This Approach Succeeded

    1. Layered Context Building

    Strategy: Information provided in logical layers, from abstract to concrete

    • Layer 1: Role and methodology (system prompt)
    • Layer 2: Architecture and threat model (context explainer)
    • Layer 3: Specific scope and boundaries (scope definition)
    • Layer 4: Task execution (audit request)
    • Layer 5: Raw material (source code)

    Why Effective: Each layer builds on the previous, creating a coherent analytical framework that guides the AI’s reasoning process.

    2. Conservative Calibration Strategy

    Problem: AI models tend toward false positives in security analysis
    Solution: Multiple reinforcements to prioritize precision over recall

    Implementation:

    • System prompt: “very careful to avoid reporting false positives”
    • Task prompt: “Avoid reporting false positives”
    • Methodology: “Better to say you cannot find any vulnerabilities than to report a false positive”

    Result: 28% false positive rate (compared to typical 60-80% in generic vulnerability scanning)

    3. Concurrency Priming

    Strategy: Explicit mention of multi-threading in context layer
    Impact: Prepared the AI to consider race conditions and concurrent access patterns
    Payoff: CVE-2025-37899 was specifically a concurrency vulnerability involving session binding and logoff timing

    Technical Deep Dive: The Discovered Vulnerability

    CVE-2025-37899: Session Logoff Use-After-Free

    Root Cause: Race condition in smb2_session_logoff() function

    int smb2_session_logoff(struct ksmbd_work *work) {
        // ... setup code ...
        
        if (sess->user) {
            ksmbd_free_user(sess->user);  // (1) Frees memory
            sess->user = NULL;            // (2) Clears the field  
        }
        
        // Problem: Only waits for current connection, not bound connections
        ksmbd_conn_wait_idle(conn);
    }

    Attack Scenario (as discovered by o3):

    1. Attacker establishes two connections (C1, C2) bound to same session (SMB 3.0+ feature)
    2. Worker-A (on C2) receives normal request (e.g., WRITE), stores sess->user pointer
    3. Worker-B (on C1) processes LOGOFF command, frees sess->user
    4. Worker-A continues processing, dereferences freed sess->user → use-after-free

    Why o3 Found This:

    • Concurrency Priming: Context explained multi-threading and session binding
    • Memory Focus: System prompt specifically targeted use-after-free vulnerabilities
    • Complete Code Context: Had full session management and connection binding implementation
    • Systematic Analysis: Required step-by-step reasoning through concurrent execution paths

    Quality Comparison: o3 vs Claude 3.7

    o3 Analysis Structure

    Vulnerability: use‑after‑free of sess->user during failed re‑authentication
    
    1. Entry point: smb2_sess_setup()
    2. Preconditions an attacker can meet: [detailed attack scenario]
    3. Code path to the bug: [step-by-step trace with line numbers]
    4. Result: [specific exploitation impact]
    5. How the attacker controls the branch: [attack methodology]
    6. Proof‑of‑concept outline: [concrete exploit steps]
    7. Fix suggestion: [specific code changes with diff]

    Performance Metrics

    • o3 Success Rate: 8/100 runs found CVE-2025-37778 (known vulnerability)
    • Claude 3.7 Success Rate: 3/100 runs found CVE-2025-37778
    • False Positive Rate: ~28% for o3
    • Novel Discovery: o3 discovered CVE-2025-37899 (zero-day), Claude 3.7 did not

    Reproducible Framework for AI-Assisted Vulnerability Research

    Phase 1: Preparation

    1. Define Vulnerability Class: Choose specific vulnerability type (use-after-free, integer overflow, etc.)
    2. Map Attack Surface: Identify entry points and trust boundaries
    3. Understand Concurrency Model: Document threading and synchronization patterns
    4. Select Code Scope: Choose manageable chunks with clear boundaries

    Phase 2: Prompt Construction

    1. System Prompt: Establish expertise, methodology, and quality standards
    2. Context Layer: Provide architectural understanding and threat model
    3. Scope Definition: Set clear boundaries and explain selection methodology
    4. Task Specification: Give clear, conservative instructions
    5. Code Provision: Supply comprehensive, relevant source code

    Strategic Implications

    For Security Teams

    Immediate Applications:

    • Code Audit Acceleration: Use AI for systematic vulnerability discovery
    • Knowledge Transfer: AI can identify complex vulnerabilities that junior analysts might miss
    • Coverage Expansion: Audit larger codebases than humanly feasible

    Implementation Considerations:

    • Expert Validation Required: AI findings must be verified by security professionals
    • False Positive Management: ~28% false positive rate requires efficient triage processes
    • Tool Integration: Incorporate into existing security workflows and CI/CD pipelines

    For AI Research

    Demonstrated Capabilities:

    • Complex Reasoning: AI can follow intricate concurrent execution paths
    • Domain Expertise: Specialized knowledge application in security contexts
    • Quality Control: Conservative calibration produces actionable results

    Remaining Limitations:

    • Context Boundaries: Still limited by token constraints for very large codebases
    • Expert Validation: Requires human security expertise for practical deployment
    • Novel Patterns: May struggle with completely new vulnerability classes

    Conclusion

    The discovery of CVE-2025-37899 represents a watershed moment in cybersecurity research. The success came not from complex AI frameworks or automated tools, but from carefully crafted prompts that:

    1. Established Clear Expertise: Positioned the AI as a specialized Linux kernel security expert
    2. Provided Strategic Context: Explained the architecture, threat model, and concurrency patterns
    3. Calibrated for Quality: Emphasized precision over recall to produce actionable results
    4. Supplied Comprehensive Context: Included all relevant code for complete analysis
    5. Required Systematic Reasoning: Demanded step-by-step analysis with concrete evidence

    This methodology is reproducible, scalable, and immediately applicable to current security workflows. Organizations should begin integrating AI-assisted vulnerability discovery while maintaining the critical human expertise needed for validation and exploitation.

    The age of AI-accelerated cybersecurity has begun. The question is not whether AI will transform vulnerability research, but how quickly security teams can adapt to leverage this new capability while maintaining the quality and rigor that security demands.

    Key Takeaway: The future of vulnerability research lies not in replacing human expertise, but in amplifying it through carefully designed AI collaboration frameworks. The prompting strategy behind CVE-2025-37899 provides the blueprint for this transformation.

  • How to Set Up WordPress MCP for AI Integration on AWS Lightsail Bitnami

    Overview

    This guide will help you set up WordPress MCP (Model Context Protocol) functionality on your AWS Lightsail Bitnami WordPress installation, enabling AI assistants like Claude to interact directly with your WordPress site.

    Prerequisites

    • AWS Lightsail Bitnami WordPress instance
    • SSH access to your server
    • Node.js 22+ on your local machine
    • Basic familiarity with WordPress admin and command line

    Part 1: Install WordPress Plugins on Your Server

    Step 1: Access Your Server

    # SSH into your Lightsail instance
    ssh bitnami@YOUR_SERVER_IP

    Step 2: Install WordPress Feature API Plugin

    # Navigate to WordPress plugins directory
    cd /opt/bitnami/wordpress/wp-content/plugins/
    
    # Clone the wp-feature-api repository
    sudo git clone https://github.com/Automattic/wp-feature-api.git
    
    # Change ownership to the web server user
    sudo chown -R bitnami:daemon wp-feature-api/
    
    # Navigate into the plugin directory
    cd wp-feature-api/
    
    # Install dependencies and build
    sudo npm run setup
    sudo npm run build

    Step 3: Install WordPress MCP Plugin

    # Navigate back to plugins directory
    cd /opt/bitnami/wordpress/wp-content/plugins/
    
    # Clone the wordpress-mcp repository
    sudo git clone https://github.com/Automattic/wordpress-mcp.git
    
    # Change ownership
    sudo chown -R bitnami:daemon wordpress-mcp/
    
    # Navigate into the plugin directory
    cd wordpress-mcp/
    
    # Install PHP dependencies
    sudo composer install --no-dev
    
    # Install npm dependencies and build
    sudo npm install
    sudo npm run build

    Part 2: Configure WordPress

    Step 4: Activate Plugins

    • Log into your WordPress admin dashboard
    • Navigate to PluginsInstalled Plugins
    • Activate WordPress Feature API
    • Activate WordPress MCP

    Step 5: Configure MCP Settings

    • Go to SettingsMCP in your WordPress admin
    • Enable MCP functionality
    • Configure which features/tools you want to expose
    • Save settings

    Step 6: Create Application Password

    • Go to UsersYour Profile
    • Scroll down to Application Passwords
    • Enter a name like “MCP Client”
    • Click Add New Application Password
    • IMPORTANT: Copy the generated password immediately – you won’t see it again!

    Part 3: Configure Local MCP Client

    Step 7: Configure Claude Desktop

    Create or edit your Claude Desktop configuration file:

    macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
    Windows: %APPDATA%\Claude\claude_desktop_config.json

    {
      "mcpServers": {
        "wordpress-mcp": {
          "command": "npx",
          "args": ["-y", "@automattic/mcp-wordpress-remote@latest"],
          "env": {
            "WP_API_URL": "https://YOUR_LIGHTSAIL_DOMAIN.com",
            "WP_API_USERNAME": "your-username",
            "WP_API_PASSWORD": "your-application-password"
          }
        }
      }
    }

    Replace:

    • YOUR_LIGHTSAIL_DOMAIN.com with your actual domain
    • your-username with your WordPress username
    • your-application-password with the application password you generated

    Step 8: Configure Cursor (Alternative)

    Create ~/.cursor/mcp.json:

    {
      "mcpServers": {
        "wordpress-mcp": {
          "command": "npx",
          "args": ["-y", "@automattic/mcp-wordpress-remote@latest"],
          "env": {
            "WP_API_URL": "https://YOUR_LIGHTSAIL_DOMAIN.com",
            "WP_API_USERNAME": "your-username",
            "WP_API_PASSWORD": "your-application-password"
          }
        }
      }
    }

    Critical Configuration Notes

    ⚠️ Username Requirements

    Make sure you use the correct username! The most common authentication issues are:

    • Try your exact WordPress username first (the one you use to log in)
    • If that doesn’t work, try your email address instead
    • The username is case-sensitive
    • Make sure the user has Administrator privileges

    🔑 Application Password Spaces

    REMOVE ALL SPACES from the application password!

    WordPress displays application passwords with spaces for readability, but you must remove them:

    • WordPress shows: AbCd EfGh IjKl MnOp
    • You must use: AbCdEfGhIjKlMnOp

    This is the #1 cause of authentication failures!

    Testing Your Setup

    Step 9: Test the Connection

    • Restart Claude Desktop or Cursor
    • In a new chat, try asking: “Can you tell me about my WordPress site?”
    • If configured correctly, Claude should be able to access your site information

    Step 10: Available Commands

    Once connected, you can ask Claude to:

    • List posts, pages, users
    • Create and edit content
    • Manage media files
    • Analyze site settings
    • And much more!

    Troubleshooting

    Common Issues

    • Connection Failed: Check your API URL, username, and application password
    • Permission Denied: Ensure the WordPress user has Administrator permissions
    • SSL Errors: Make sure your site has a valid SSL certificate
    • Plugin Not Working: Verify plugins are activated and built correctly

    Debug Steps

    • Check WordPress error logs: /opt/bitnami/wordpress/wp-content/debug.log
    • Test API access manually:
    curl -u username:app_password https://yoursite.com/wp-json/wp/v2/posts
    • Check MCP logs if LOG_FILE is configured in your client setup

    Important Notes

    • Bitnami Specifics: File paths use /opt/bitnami/wordpress/ instead of standard WordPress paths
    • Permissions: Always use sudo for file operations and set proper ownership
    • Updates: When updating plugins, you may need to rebuild them
    • Backups: Always backup your site before making changes

    Support Resources

    Security Configuration (Critical Step)

    ⚠️ Important: When you install plugins using git clone, you expose sensitive files that should never be accessible via web browser. This is a critical security risk that must be addressed.

    Why Security Configuration is Required

    When you run git clone to install WordPress MCP plugins, you’re downloading the entire development repository, which includes:

    • .git/ directory – Contains complete repository history, potentially including sensitive information
    • Configuration filescomposer.json, package.json, webpack.config.js reveal your tech stack and dependencies
    • Development files – Build scripts, documentation, and temporary files
    • Dependenciesnode_modules/, vendor/ directories (if present)
    • Environment files – Any .env files or configuration that might contain secrets

    These files are meant for development, not production, and can reveal sensitive information about your server setup to attackers.

    Security Risk Assessment

    High Risk Files:

    • .git/Entire repository history including potentially sensitive commits
    • composer.json/package.jsonDependency information that reveals software versions
    • Build configuration files – Technical details about your setup

    What Attackers Can Learn:

    • Your development workflow and tools
    • Software versions you’re running (for exploit targeting)
    • Repository history and previous configurations
    • Internal file structures and organization

    Security Implementation Options

    Choose one of these approaches:

    Option 1: Remove Sensitive Files (Recommended – Simplest)

    After plugin installation and building, clean up sensitive files:

    # Navigate to WordPress MCP plugin
    cd /opt/bitnami/wordpress/wp-content/plugins/wordpress-mcp/
    
    # Remove git history and development files
    sudo rm -rf .git/
    sudo rm -f composer.json composer.lock package.json package-lock.json
    sudo rm -f webpack.config.js tsconfig.json .eslintrc.js
    sudo rm -f *.md README.md CONTRIBUTING.md
    sudo rm -rf node_modules/ tests/ docs/
    
    # Repeat for WordPress Feature API plugin
    cd /opt/bitnami/wordpress/wp-content/plugins/wp-feature-api/
    
    sudo rm -rf .git/
    sudo rm -f composer.json composer.lock package.json package-lock.json
    sudo rm -f webpack.config.js tsconfig.json .eslintrc.js
    sudo rm -f *.md README.md CONTRIBUTING.md
    sudo rm -rf node_modules/ tests/ docs/
    
    # Set proper ownership
    sudo chown -R bitnami:daemon /opt/bitnami/wordpress/wp-content/plugins/
    

    Option 2: Apache Configuration Rules (Advanced)

    Add security rules to your Apache configuration:

    # Add security rules to Apache config
    sudo tee -a /opt/bitnami/apache/conf/httpd.conf << 'EOF'
    
    # WordPress Plugin Security Rules
    <DirectoryMatch "\.git">
        Require all denied
    </DirectoryMatch>
    
    <FilesMatch "\.(json|lock|md|yml|yaml|log|tmp)$">
        Require all denied
    </FilesMatch>
    
    <DirectoryMatch "node_modules">
        Require all denied
    </DirectoryMatch>
    
    <DirectoryMatch "(^|/)vendor/">
        Require all denied
    </DirectoryMatch>
    EOF
    

    Test and restart Apache:

    # Test configuration syntax
    sudo /opt/bitnami/apache/bin/httpd -t
    
    # If syntax is OK, restart services
    sudo /opt/bitnami/ctlscript.sh restart
    

    ⚠️ Note: If you get “Service Unavailable” errors after restart, the configuration may be too restrictive. Remove the added rules and use Option 1 instead.

    Security Verification

    After implementing security measures, test that sensitive files are properly blocked:

    # These commands should return "403 Forbidden" or "404 Not Found"
    # Replace your-domain.com with your actual domain
    
    curl -I https://your-domain.com/wp-content/plugins/wordpress-mcp/.git/
    curl -I https://your-domain.com/wp-content/plugins/wordpress-mcp/composer.json
    curl -I https://your-domain.com/wp-content/plugins/wordpress-mcp/package.json
    curl -I https://your-domain.com/wp-content/plugins/wp-feature-api/.git/
    

    ✅ Secure Response: HTTP/1.1 403 Forbidden or HTTP/1.1 404 Not Found
    ❌ Insecure Response: Any response that shows actual file content

    Best Practices for Future Updates

    When updating plugins:

    1. If using Option 1 (file removal):
      • Pull updates: git pull origin main
      • Rebuild: npm run build
      • Clean sensitive files again (repeat removal commands)
    2. If using Option 2 (Apache rules):
      • Updates are automatically protected
      • No additional cleanup needed

    For production deployments, consider:

    • Using official plugin releases instead of git clone
    • Implementing automated deployment scripts that clean sensitive files
    • Regular security audits of plugin directories

    Why This Matters

    Without proper security:

    • Attackers can access your entire development history
    • Technical details about your setup are exposed
    • Dependency information reveals potential vulnerabilities
    • Your site appears unprofessional and insecure

    With proper security:

    • Only necessary production files are accessible
    • Your technical stack remains private
    • Site appears professionally configured
    • Reduced attack surface for potential threats

    This security step is mandatory when using git clone for plugin installation and should never be skipped in production environments.