Everyone Is Shipping an AI Pentester. Most of Them Are Expensive Autocomplete.

GitHub is currently drowning in “agentic” security tools that pinky-promise to find your bugs while you sleep. Some of them are interesting. Most of them are a chatbot with a scanner bolted on and a DEF CON sticker.

The last week of GitHub trending has been wall-to-wall AI agents that claim to pentest for you. ARTEX, BugTraceAI, ExhibitA, NVIDIA’s SkillSpector. The marketing copy all sounds the same: autonomous, evidence-first, offline-capable, DEF CON stage-ready. Cool. Now can we talk about which ones are actually useful and which ones are going to get you sued when they crawl a production host you forgot about.

What happened

A few real releases landed around October 7 to 11, 2026:

• ARTEX (mhtsec): a Go-written AI autonomous pentesting system. Its page calls it the champion of Baidu’s “agent+” offense-defense challenge. Third-party trending charts put it near the top of the AI Agent category with roughly 1,400 stars in a day and about 2,713 total as of October 10. Chinese open-source project, vulnerability detection and security automation.
• BugTraceAI: Apache-2.0, self-hosted agentic pentesting framework. CLI with a TUI, web dashboard, API over REST and MCP, and a Launcher for Docker or local setup. Presented at DEF CON 34, RootedCON 2026, and HKOSCon 2026. Authors are explicit that component versions are in beta and outputs are leads to verify, not verdicts. That honesty alone puts them ahead of half the field.
• ExhibitA (exhibita on PyPI, v0.1.0 uploaded 2026-10-11): MIT-licensed AI security reviewer that does something different. It does not just flag. For serious findings it spins the app in a sandbox, confirms the bug with a black-box exploit, applies a proposed fix, and checks the exploit no longer works. Runs as CLI, GitHub Action, Agent Skill, or MCP server. Built on Semgrep, Gitleaks, Hadolint, Checkov, and Google’s open Gemma 4 model. Offline-capable.
• NVIDIA SkillSpector: not a pentester, a scanner for AI agent skills. Detects prompt injection, data exfiltration, malicious patterns, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them. Apache-2.0, part of NVIDIA’s Verified Skills pipeline. About 20k stars. This is the one you actually want if your team is installing random agent skills like they are Chrome extensions.

Who should care

Red teamers who want a second set of eyes that does not bill by the hour. AppSec folks tired of SARIF dumps that cry wolf. CISOs who keep getting pitched “AI-powered continuous penetration testing” by vendors who cannot spell XSS. And anyone who has installed an MCP skill from a stranger on the internet, which, based on the star counts, is most of you.

Technical breakdown: what is actually different

Most of these tools share a skeleton: an LLM planner, a tool-use loop, and a scanner backend. The differentiator is what they do with a finding.

ARTEX is an autonomous offense agent. Useful for lab work and CTF-style ranges. Treat it like a junior pentester with a caffeine problem: fast, enthusiastic, needs adult supervision. Do not point it at anything you do not own.

BugTraceAI is the self-hosted platform play. The evidence-first design is the right call. A finding with a reproducible payload and a screenshot beats a confidence score every time. The MCP and REST surfaces mean you can plug it into your existing agent stack instead of living inside their UI. Beta means beta. Verify every finding before you file a bounty or a ticket.

ExhibitA is the one that made me stop scrolling. Exploit, patch, verify, on an open model, offline. That closes the loop most SAST tools refuse to close. A “critical” that dies when you actually try to exploit it is a false positive you just spent money on. A finding that survives a black-box exploit and then dies after the proposed fix is a finding you can ship a PR for. The GitHub Action posting inline review comments with one-click fixes is the kind of thing that actually moves the needle for engineering teams, not SOC theater.

SkillSpector is a different category and, honestly, more important right now. AI agent skills are a new supply chain. People are installing skill packs the way they used to install npm packages in 2016, which is to say: recklessly. Scanning for prompt injection and exfiltration before install is table stakes. NVIDIA shipping this under Apache-2.0 and wiring it into a signed catalog is the grown-up move.

Quick try (SkillSpector)

# install via uv
uv tool install git+https://github.com/NVIDIA/skillspector.git

# scan a skill directory before you install it
skillspector scan ./some-skill/

For ExhibitA:

pip install exhibita
exhibita scan ./my-app
exhibita verify ./my-app --finding findings.json

What to do Monday morning

1. If your team installs agent skills, put SkillSpector (or an equivalent) in the install path. No scan, no install.
2. Pilot ExhibitA on one internal app as a GitHub Action. Measure how many “criticals” survive the exploit-and-verify loop. Use that number to roast your existing SAST vendor.
3. If you try BugTraceAI or ARTEX, start on a deliberately vulnerable lab (their BugStore demo, OWASP Juice Shop, or your own box). Never production. Never a customer. Never “just this once.”
4. Ask any commercial “AI pentest” vendor three questions: do findings come with a reproducible exploit? does it verify the fix? where does the data go? If they dodge any of them, hang up.
5. Stop confusing star counts with readiness. Stars mean the marketing worked. They do not mean the tool will not nuke your CI bill.

Sources

• HIMIAO Horizon on ARTEX (2026-10-10): himiao.cc
• BugTraceAI announcement (2026-10-07): dev.to and github.com/BugTraceAI
• ExhibitA on PyPI (v0.1.0, 2026-10-11): pypi.org/project/exhibita
• NVIDIA SkillSpector: github.com/NVIDIA/SkillSpector