Skip to content

docs: add 'How A.I.G Compares' section to README #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
boy-hack wants to merge 1 commit into
base: main
Choose a base branch
from
+19 −0
Open

docs: add 'How A.I.G Compares' section to README #635

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ Help us improve A.I.G! Please take 3-5 minutes to fill out our [User Feedback Su
## Table of Contents
- [🚀 Quick Start](#-quick-start)
- [✨ Features](#-features)
- [🆚 How A.I.G Compares](#-how-aig-compares)
- [🖼️ Showcase](#-showcase)
- [📖 User Guide](#-user-guide)
- [🔧 API Documentation](#-api-documentation)
Expand Down Expand Up @@ -245,6 +246,24 @@ For full leaderboard and details, visit [SkillTrustBench](https://matrix.tencent

<br />

## 🆚 How A.I.G Compares

AI red-teaming/security tools tend to specialize in one attack surface. A.I.G is one of the few that scans across five: AI infra CVEs, MCP servers, Agent Skills, Agent workflows, and jailbreak/prompt-injection evaluation, in a single self-hostable repo.

| | **A.I.G**&nbsp;(this&nbsp;repo) | NVIDIA [garak](https://github.com/NVIDIA/garak) | Microsoft [PyRIT](https://github.com/Azure/PyRIT) | [Promptfoo](https://github.com/promptfoo/promptfoo) | Invariant [mcp-scan](https://github.com/invariantlabs-ai/mcp-scan) |
|:--|:--|:--|:--|:--|:--|
| AI infra CVE scan (Ollama, vLLM, ComfyUI, n8n, Triton, etc.) | ✅ 146 components / 2000+ CVE rules | ❌ | ❌ | ❌ | ❌ |
| MCP server security scan (tool poisoning, RCE, credential exfil) | ✅ 14 risk categories, source or remote URL | ❌ | ❌ | ⚠️ limited (via redteam plugins) | ✅ (MCP-focused) |
| Agent Skill / plugin bytecode & supply-chain scan | ✅ incl. `.pyc` bypass & charset-smuggling defense, benchmarked on [SkillTrustBench](https://matrix.tencent.com/skilltrustbench/) | ❌ | ❌ | ❌ | ❌ |
| Agent workflow scan (Dify, Coze, multi-agent) | ✅ | ❌ | ❌ | ⚠️ limited | ❌ |
| LLM jailbreak / prompt-injection evaluation | ✅ multi-turn (Many-Shot, PAIR, GOAT, ActorAttack) | ✅ (probe-based, single/few-turn focus) | ✅ (attack orchestration framework) | ✅ (redteam plugin) | ❌ |
| Black-box LLM API poisoning / model-substitution detection | ✅ (API Checker, v4.6.0) | ❌ | ❌ | ❌ | ❌ |
| Deployment | Docker / CLI / Web UI, self-hosted | CLI (Python lib) | Python lib / SDK | CLI / cloud | CLI |
| License | Apache-2.0 | Apache-2.0 | MIT | MIT (core) | Apache-2.0 |

This isn't a claim that A.I.G is strictly "better" — garak, PyRIT, Promptfoo, and mcp-scan are each excellent at what they specialize in, and are worth running alongside A.I.G rather than instead of it. The table above is about scope: most teams end up needing more than one of these tools, and A.I.G is built to cover the infra + MCP + Skill + Agent + jailbreak surfaces from one place instead of stitching several together. Corrections welcome via issue or PR if any row is out of date.

<br />

## 🖼️ Showcase

Expand Down