Visa open-sourced something interesting this year: vvaharness, an agentic SAST pipeline under Apache-2.0. Not a linter with a rule pack, but an eleven-stage thing that maps the attack surface, builds a threat model, decomposes the repo into hunting assignments, and then runs an adversarial verification pass against its own findings before it writes a report.

We wanted to know whether that actually works on a real codebase we did not write. So we pointed it at a third-party open-source project: a peer-to-peer encrypted mesh networking daemon in Rust, roughly 133,000 lines of code, of which 96,011 were in scope. Permissionless network, cryptographic identities, privileged network access on the host. A genuinely hard target, and exactly the kind of software where a bug is somebody's outage.

Here is what happened, including the parts that do not flatter the tool.

Detection only, stages one through nine. The remediation and validation stages are disabled by an explicit flag, because the shipped default profile will otherwise continue into fix mode and edit the source files of the repository you were only supposed to look at. On code you do not own, that is not a spoiled measurement, it is touching someone else's project. That flag is the single most important thing to get right.

The numbers, unembellished:

Seven real issues in an hour, on a codebase whose maintainer had shipped a batch of security fixes two days earlier. That is a good result. It is also not the interesting part.

The standard failure mode of a language model asked to audit code is confident fabrication. Plausible file paths, plausible line numbers, plausible quoted code, none of it real. We have been burned by it enough that we now mechanically verify every citation before we believe a single sentence.

Every file and line reference we checked matched the source exactly. Every quoted snippet was verbatim. Zero fabrications.

That is worth saying plainly, because it changes what the tool is useful for. If the citations hold, a human can adjudicate a finding in a couple of minutes instead of re-auditing from scratch. The tool is not replacing the reviewer; it is aiming the reviewer's attention, and aiming it accurately.

Two of the nine were miscalibrated, and both failures were the same underlying mistake.

The critical that was not. One finding came back rated 9.3, critical, with a CVSS vector asserting no privileges required and low attack complexity. Its own preconditions section, three lines further down, said the attacker needed an authenticated membership in the network and a privileged position between two other participants. The vector contradicted the finding's own text. Recomputed honestly it lands in the medium-to-high band.

The underlying issue was real, and the genuinely valuable insight was buried: the protocol field in question is deliberately unauthenticated and the code documents that choice, so that part is not news to the maintainer. What was news is a consequence one layer downstream, where a value only ever moves in one direction and never resets, turning a piece of routing metadata into a condition that does not repair itself. The tool found the right code and told the wrong story about it.

The high that the maintainer had already written down. Another finding came back at 8.5. The maintainer's own source comment, inside the exact line range the tool cited, states the limitation, explains the deployment assumption under which it is acceptable, and says it should be documented. The tool read that comment and rated it high anyway.

This is the sharpest lesson of the exercise. An agentic scanner cannot distinguish an undiscovered vulnerability from a documented, accepted engineering tradeoff. Both look like a gap in the code. Only one is worth a maintainer's afternoon. Telling them apart requires reading the surrounding comments and the changelog for the author's own reasoning, and that is a judgement call, not a pipeline stage.

The harness has a deterministic first stage that builds a call graph to seed the hunt. It supports C#, Go, Java, JavaScript, Python and TypeScript.

Our target was Rust. That stage returned zero entry points and zero sinks, and contributed nothing at all. All 170 hunting assignments came from language-model decomposition instead.

The report then states 100% line coverage, which is true and also the most misreadable number in the document. It means every line reached a model. It does not mean any line received data-flow analysis. If you quote that figure without the qualification, you are overclaiming, and if you are selling the result to somebody, you are misselling it.

Anyone running this on Rust, C, C++, Ruby, PHP or Swift should know that going in.

Seven real issues, unpatched, in software running on a live network. We wrote none of them up publicly.

The findings went to the maintainer by email, in full, with root causes and suggested fixes, before this post existed. The project ships no security policy file, so there was no published channel and direct contact was the only correct route. Nothing is published, and nothing will be until fixes land, at which point we will name the project and link to the maintainer's own commits.

That sequencing is not politeness. A public issue describing an unpatched, remotely reachable defect is not a report, it is a working recipe, and shipping one to look productive would make us the problem. If a security practice will not hold that line when holding it costs a good story, the practice is decorative.

We also told the maintainer what we had not checked. One of the nine we did not verify ourselves, so we said so and declined to assert it. An earlier draft of that email claimed the findings were human-verified. They were not, not all of them, and a blanket claim like that poisons every honest line around it. It got cut before sending.

The shape of the work has changed and the judgement has not moved an inch.

A machine can now read 96,000 lines of unfamiliar Rust in an hour and come back with seven real issues and accurate citations. That is a genuine capability increase, and anyone claiming these tools produce only noise has not run one recently.

What the machine cannot do is decide what a finding means. It cannot weigh a tradeoff the author already made deliberately. It cannot tell that a panic which unwinds a task while the process survives is operationally worse than a crash, because nothing restarts a process that is still running. It cannot decide whether a defect goes in a public issue or a private email. Every one of those is a judgement about consequences, and consequences are what security work is actually about.

So the division of labour we settled on: the machine generates candidates, a human adjudicates every one against the source, severities are attributed to us and never to the tool, and the disclosure route is decided before a single line of public copy is written.

We run this as a service, and we would rather describe it accurately than impressively.

What you get: an AI-assisted security review, pinned to an exact commit, with a run manifest recording the tool version, the model roles, the configuration hash and the target hash, so the run is reproducible rather than asserted. Every finding is adjudicated by a human against your source. You get told which ones we verified and which ones we did not. Machine output is never handed to you raw, and it never feeds automated gating or ticketing.

What it is not: a certification, an assurance engagement, or a penetration test. There is no badge and there will not be one. It is a signed, structured, honest first pass, which is a useful thing to have and a different thing from a guarantee.

If your codebase is the sort where a defect becomes somebody's outage, that is the kind of review worth having before a customer finds it for you.

Aunova builds verifiable, privacy-first infrastructure. This review was run by Cowork:Alice RED, our own agent system, under the discipline described above: Reproducible Evidence and Disclosure.

Candidates generated with Visa's open-source `vvaharness` (Apache-2.0) in detection-only mode. Severity assessments are ours. Not a certification, not an assurance engagement. Visa has neither reviewed nor endorsed this work. `vvaharness`, Copyright 2026 Visa, Inc., licensed under the Apache License, Version 2.0.