Skip to main content

​
Rogue - The AI Agent Evaluator & Red Team Platform

Rogue is a powerful tool designed to evaluate the performance, compliance, security, and reliability of AI agents. It combines dynamic policy evaluation with comprehensive red teaming capabilities to test your agents against 87+ vulnerability types using 30+ attack techniques.
Rogue FlowStar

​
Architecture

Rogue operates on a client-server architecture:
  • Rogue Server: Contains the core evaluation logic including the Red Team Orchestrator
  • Client Interfaces: Multiple interfaces that connect to the server:
    • TUI (Terminal UI): Modern terminal interface built with Go and Bubble Tea
    • Web UI: Gradio-based web interface
    • CLI: Command-line interface for automated evaluation and CI/CD
This architecture allows for flexible deployment and usage patterns, where the server can run independently and multiple clients can connect to it simultaneously.

​
Key Features

​
Policy Evaluation

  • 🔄 Dynamic Scenario Generation: Automatically creates a comprehensive test suite from your high-level business context.
  • 👀 Live Evaluation Monitoring: Watch the interaction between the Evaluator and your agent in a real-time chat interface.
  • 📊 Comprehensive Reporting: Generates a detailed summary of the evaluation, including pass/fail rates, key findings, and recommendations.

​
Red Teaming & Security Testing

  • 🛡️ 87+ Vulnerability Types: Test against comprehensive vulnerability categories including prompt injection, PII exposure, content safety, bias, and more.
  • ⚔️ 30+ Attack Techniques: Single-turn, multi-turn, and agentic attacks including Base64 encoding, roleplay, social engineering, and advanced jailbreaking.
  • 📋 Compliance Framework Mapping: Automatically map findings to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, EU AI Act, GDPR, and more.
  • 📈 CVSS-Based Risk Scoring: Industry-standard risk scoring with severity levels, exploitability metrics, and remediation recommendations.

​
Platform Capabilities

  • 🤖 Broad Model Support: Compatible with a wide range of models from providers like OpenAI, Google (Gemini), and Anthropic.
  • 🎯 Multiple Interfaces: Choose from TUI, Web UI, or CLI interfaces depending on your workflow needs.
  • 🚀 Easy Installation: Get started quickly with uvx rogue-ai - no complex setup required.
  • 🔌 Multi-Protocol Support: Works with A2A (Agent-to-Agent) and MCP (Model Context Protocol) agents.