1. Red
  2. Learn
  3. Landscape and analysis

Autonomous pentesting in 2026: a reality check for owners

Autonomous pentest 2026: what DARPA, curl and vendor pages show about AI penetration testing, where people still validate, and what to ask before you pay.

By Timothy Lindblom, FounderPublished 4 November 20267 min readHow we check facts
On this page
  1. What does "autonomous pentesting" mean in 2026?
  2. What has independent evidence shown?
  3. Where does it still fall short?
  4. How do vendors handle the human step?
  5. Who is allowed to be tested?
  6. What should I ask before paying for one?
  7. What does Red do, and what can it not tell you?
  8. How current is this page?

AI penetration testing works in 2026 on defined tasks, and people still check the output. Machines have found and patched real bugs in source code, and maintainers now accept AI-assisted reports. Vendors also describe human validation, scoped authorization and false positives on their own pages. Read "autonomous" as a claim about the search step, then ask who confirms the result.

FactDetailSource
DARPA AI Cyber Challenge, final, 8 August 2025Systems found 54 of 63 synthetic vulnerabilities and patched 43. They also found 18 real vulnerabilities and patched 11DARPA
curl, October 2026"Hundreds" of reports in recent months, each with AI involved to some extent; the project's maintainer says slop reports have been practically gone since spring 2026curl
Practitioner survey, 17 to 18 June 202687.8% of 158 AI-tool users hit findings needing significant manual validationPentest-Tools.com (vendor survey)
Academic benchmark, 16 December 2025End-to-end LLM pentest pipelines reached 31% successPentestEval
Published prices$100 to $4,000 for a single automated test, depending on depth and reviewVendor pages, table below
Checked24 October 2026this page
54/63synthetic vulnerabilities found, DARPA 2025
87.8%of surveyed users needed manual validation
31%end-to-end success, PentestEval

What does "autonomous pentesting" mean in 2026?

It names a pipeline, not a single capability. A tool lists what it can reach, tries known weaknesses, links findings and writes a report. For a smaller owner's view, what to expect from an automated pentest is on its own page. The word "autonomous" usually describes how much of that runs without a person starting each step.

Validation is the step that varies most. A finding can be confirmed by a fixed, non-AI check, by another AI agent, or by a person. XBOW's documentation draws exactly this line. Where a non-AI validator confirms a class, "the class reports no false positives". Where agents confirm it, "some false positives are possible".

Five stages of an automated pentest and who confirms the result at each one Find Try Validate Report Fix AI search AI agents Fixed check, agent or person Tool or analyst You or your host The validate stage decides how many false positives reach you.
Where the work happens in an automated pentest. Searching and trying are the parts machines do well; the validate stage is where vendors differ, and the fix is always the owner's or the host's.

What has independent evidence shown?

Three public sources are worth reading in full.

DARPA's AI Cyber Challenge. In the final on 8 August 2025, teams' systems found 54 of 63 synthetic vulnerabilities and patched 43, across 54 million lines of C and Java. They also found 18 real vulnerabilities and patched 11. DARPA reports an average cost of about $152 per competition task. The setting was source code with known scoring, which is a long way from a black-box scan of a live shop.

curl. Maintainer Daniel Stenberg wrote on 22 October 2026 that every report in recent months had AI involved and that the slop reports of earlier years had practically stopped since spring 2026. His remaining complaint is length. A separate audit by Aisle produced 35 published CVEs, backed, in his words, by "a team of skilled humans". On 14 October a use-after-free, CVE-2026-92392, was reported with Anthropic's OSS Scanner and fixed in curl 8.23.0. curl itself estimated that a crash is the likeliest outcome and that a successful attack is unlikely.

The pattern across all three is the same. The machine widens the search and a human decides what counts.

Where does it still fall short?

Accuracy and volume. The Pentest-Tools.com survey of 158 practitioners, run on 17 and 18 June 2026, found 87.8% hit findings needing significant manual validation, and 63% ranked the false positive rate above cost. About one in four reworked more than a quarter of AI findings, and only 20.3% had a workflow for 500 or more candidates. It is a vendor survey of people who already use AI tools, so read it as a signal from that group only, with no claim about all businesses.

The academic picture is cautious as well. PentestEval, submitted on 16 December 2025, reported that end-to-end pipelines reached only 31% success. Benchmarks move fast, and a figure from last winter is not a ceiling. It does show that a result on one task does not transfer to a whole engagement.

Two limits apply to any outside tool. It cannot see inside your server, so it cannot read a package changelog or know whether workers restarted. It also cannot know your business rules, such as who may see which invoice.

How do vendors handle the human step?

Their own pages differ more than their headlines do. At our check on 24 October 2026, with the Intruder, Scantinel and Barrion pricing pages re-read and unchanged on 4 November:

ProviderWhat its own page says about validationPublished price
XBOWNon-AI validator for some classes, agents for others; "some false positives are possible" for the latter. Testing "without scoping calls"From $4,000, results in five business days (announcement of 13 November 2025)
IntruderSame-day reports; refund in full if an auditor rejects the report$3,500 per test, $12,000 for four
BreachLockAI for discovery and enumeration; "every finding surfaced is manually validated by a certified pentester"Not published on the page
ScantinelOptional analyst review of AI findings adds 1 to 3 business days$100 full scan, $399 AI pentest
BarrionEngineer review by depth: none at Light, 1 hour at Standard, up to 8 hours at MaximumCredit based, see page
Pentera"Evidence review and remediation decisions remain with human teams"; throttling and emergency stop controlsNo public price on the platform page

Prices and page text change. Open the linked page before you decide, and see how Barrion compares if it is on your shortlist. We describe what each page says and have not tested any of these products.

Who is allowed to be tested?

Only systems you control, and what counts as authorization is worth knowing before you start. A testing tool that sends real requests to a domain is a risk to whoever owns that domain, and the better vendors say so.

Detectify's documentation says its tests send real requests, and that verification "protects third parties from unauthorized scanning". It offers a DNS TXT record, a verification file, or a meta tag. Horizon3.ai's documentation separates discovery from authorization and tells users to "authorize only the assets you are legally permitted to pentest". That puts the legal decision on the customer.

Red asks for one DNS TXT record before anything runs, so nobody can point us at a site that is not theirs. Agencies need written consent from each client before any test of a client site.

What should I ask before paying for one?

If you are unsure a pentest is what you need, the pentest-or-scan question comes first. Then send the provider this, and keep the reply:

For a test of [your domain]: which findings in the report are confirmed by a non-AI check, by an AI agent, or by a person? What is your policy when a finding is wrong, and what do you check about the scope before you start? Does the report show the requests that were sent, and will your tool ever change or delete data?

Then work through the list:

  • Ask your auditor or insurer in writing what test evidence they accept, before you buy anything.
  • Confirm the provider verifies you control the domain, and that you can name the person who authorized it.
  • Find out what a wrong finding costs you: a refund, a corrected report, or nothing.
  • Check whether human review is included, optional or absent, and what it adds in days.
  • Keep the fix step in your plan. A test finds issues. Your host or your developer repairs them, and who patches what depends on your plan.
A test result describes the day it ran. After a new vulnerability is disclosed, as with PHP2Shell, the fix still has to be confirmed on the server by the host or by you.

What does Red do, and what can it not tell you?

Red by run.cloud is a free website vulnerability scan, and a full scan that sends an AI red team. The free scan identifies the technologies your site serves, the way Wappalyzer does, then looks each version up in public vulnerability records. It is a lookup. It can say a version is affected, not whether your setup is exposed. One free scan per domain, no card.

The full scan costs $100 per scan. An AI red team lists every page, route and service it can reach from outside, ranks the paths worth trying, links small findings into one path, proves access once with evidence, stops, and writes up the path and the fix. It never deletes, changes or downloads your data, requests are rate limited, and every step it took is in your report. The fixing is yours or your host's.

That means an outside result sits at the first two of three states. Something we detected is an indication to check on the server. Something we could not see is unknown. Only your host or your own server can give you confirmed by owner evidence, such as a package changelog and a restart time. For the wider October picture see the security tracker and the other landscape pieces, and if you suspect a problem now, use the step-by-step website incident response.

How current is this page?

This page reports what public pages said on 24 October 2026, with three vendors' prices re-read on 4 November. Vendor accuracy numbers are the vendor's own, and we found no independent test of the commercial products named above at a primary source. Prices move, so the table is a dated snapshot.

Questions people ask

Can AI really pentest my website?

It can do parts of the job well: listing what is reachable, trying known weaknesses and linking small findings. DARPA's 2025 challenge and curl's 2026 reports show machines finding real bugs in source code. For a live website the open question is who confirms each result, so ask the provider that before you pay.

Is an AI pentest as good as a human one?

Public evidence does not support a blanket yes or no. A June 2026 survey of 158 practitioners found 87.8% hit AI findings that needed significant manual validation. Vendors that sell human review say it covers business logic and chained paths, which a single automated pass is weakest at.

Will an automated pentest satisfy my auditor or insurer?

That is the auditor's decision, not the tool's. Ask your auditor or insurer in writing what they accept before you buy. One vendor publishes a refund if an auditor rejects its report, which tells you the risk exists.

Is it legal to run an AI pentest on my site?

Only on systems you own or have written permission to test. Reputable tools make you prove control of the domain first, usually with a DNS record, a file or a meta tag. Never point one at a site you cannot prove is yours.

What is a false positive and why does it matter here?

It is a finding that looks real and is not. Each one costs someone time to disprove, and a pile of them hides the real issue. XBOW's own documentation says findings confirmed by agents can include false positives, while ones confirmed by a non-AI validator do not.

How much does an AI pentest cost in 2026?

Published prices at our 24 October 2026 check ran from $100 for a full automated scan at one vendor to $3,500 and $4,000 for single tests at others. Enterprise platforms publish nothing. Price tells you little without knowing what is validated and by whom.

Sources

  1. DARPA: AI Cyber Challenge marks pivotal inflection point for cyber defense (8 August 2025)Final results: 54 of 63 synthetic vulnerabilities found, 43 patched, 18 real vulnerabilities discovered, 11 patched. Checked 24 October 2026.
  2. Daniel Stenberg: ignore the slop (curl, 22 October 2026)First-party statement from curl's maintainer on AI-assisted vulnerability reports. Checked 24 October 2026.
  3. Daniel Stenberg: a curl audit by Aisle (14 October 2026)35 published CVEs from an AI-assisted audit, backed by human engineers.
  4. Daniel Stenberg: a UAF turned RCE (curl, 14 October 2026)CVE-2026-92392, reported by Anthropic's OSS Scanner, fixed in curl 8.23.0, with curl's own estimate of how likely an attack is.
  5. Pentest-Tools.com: State of AI Pentesting survey (17 to 18 June 2026)Vendor survey of 158 practitioners who already use AI tools. Not a representative sample of all businesses.
  6. PentestEval: Benchmarking LLM-based Penetration Testing (arXiv, 16 December 2025)Academic benchmark. End-to-end pipelines reached 31% success.
  7. XBOW docs: vulnerability classificationSeparates findings confirmed by a non-AI validator from findings confirmed by agents, and says the latter can include false positives.
  8. XBOW: Pentest On-Demand announcement (13 November 2025)Starting price and delivery time as published by the vendor.
  9. Pentera: platform pageStates that evidence review and remediation decisions remain with human teams.
  10. Intruder: AI pentest pricingPer-test price, same-day reports, refund if an auditor rejects the report.
  11. BreachLock: penetration testing serviceDescribes AI for discovery and enumeration and manual validation of every finding by a certified pentester.
  12. Scantinel: pricingFull scan, AI pentest and optional analyst review with a stated delay.
  13. Barrion: pricingEngineer review hours by test depth.
  14. Detectify docs: asset verificationThree ownership verification methods and the reason for them.
  15. Horizon3.ai docs: authorize assets for an external pentestDiscovery is separate from authorization; the user authorizes assets they are legally permitted to test.

Facts on this page are dated and point at the primary source next to them. If a vendor status changes after the date shown, the vendor's page wins; tell us at support@run.cloud and we will update it.