Back to BlogAnyone Can Find Your Bugs Now. Find Yours First.

Anyone Can Find Your Bugs Now. Find Yours First.

Octopus TeamĀ·

A developer recently chained 35 prompts to an LLM and walked out with a working remote kernel RCE. Not a CTF challenge. A real one.

That story is making the rounds because it captures something a lot of us have been quietly noticing: AI is now better at breaking software than it is at writing it. Browser zero-days, full-chain TCP/IP exploits, kernel escalations, all coming out of agent loops that a teenager can run from a laptop.

The uncomfortable conclusion: every line of code you ship is now being audited. Not by your team. By strangers with AI agents pointed at your repo.

The asymmetry just flipped

For two decades, the rough deal was that finding a serious vulnerability took years of specialized skill, expensive tooling, and a lot of luck. Defenders could lean on that asymmetry. Most bugs would never be discovered, and the ones that were got found by people with reputations to protect.

That world is ending fast. Veteran vulnerability researchers are openly saying that "agents will generate working zero-days in well, everything," and that the only sliver of human exploit work left is at the absolute frontier. The boring middle, the SQL injections, the deserialization bugs, the auth bypasses hiding in a helper file three directories away from the diff, is being commoditized.

A few signals from the last few months:

  • Frontier labs now ship vulnerability research benchmarks in their model cards.
  • Some firms have quietly started running simple agent loops across all their targets and watching previously-missed sev-highs fall out by the handful.
  • OpenAI is reportedly downgrading certain requests to weaker models when they suspect security work, which only confirms how strong the new models are at it.
  • Teenagers are filing validated CVEs in popular open source projects.

If your defensive posture still assumes "nobody will look that hard at our code," it has a short shelf life. Open weight models from anywhere in the world will close any policy-based gap inside of a year. The asymmetry is gone.

You ship it, or someone else audits it for you

Here is the part most teams have not internalized yet. The window between "code merged" and "code audited by a hostile agent" is collapsing toward zero. Public repos get scanned constantly. Private repos leak through dependencies, contractors, breaches, and packaged binaries.

So the only winning move is the one defenders have always had on paper but rarely executed: review every change with the same depth an attacker would, before it ships.

That used to be impossible. Human reviewers do not have time to trace every helper, every auth path, every input boundary on every PR. Diff-only AI tools see a 30-line patch and miss that the function it calls has been reachable from an unauthenticated endpoint since last quarter. That is exactly the gap attackers walk through.

What codebase-aware review actually does

Octopus Review was built for this gap. Instead of looking at a PR diff in isolation, it indexes your full repository into a vector database and pulls the relevant context every time it reviews a change.

So when a PR touches validateSession, the reviewer already knows which routes call it, which middleware wraps it, what the auth model looks like, and what the last three security-relevant changes to that area were. It reasons about the change the way a senior engineer who has lived in the codebase for two years would.

The output looks like this on a real PR:

šŸ”“ Critical — Prototype pollution via unsanitized merge
src/api/preferences.ts:84
The user-supplied `patch` object is deep-merged into the session preferences
without filtering `__proto__` or `constructor`. Reachable from the unauthenticated
`/api/preferences/preview` route registered in `src/router/public.ts:22`.

🟔 Major — Missing rate limit on auth-adjacent endpoint
src/api/reset-password.ts:31
This route bypasses the global limiter applied in `src/middleware/throttle.ts`
because it is registered before the limiter is mounted.

That kind of comment is only possible because the reviewer has the whole codebase loaded as context, not just the patch. Diff-only tools cannot tell you "this is reachable from an unauthenticated route" because they cannot see the router.

Trigger it however you want:

# Auto-review on PR open in CI
octopus review 1842

# Or kick it off locally before you push
octopus pr review https://github.com/your-org/your-repo/pull/1842

For codebases where you would rather not send source to a third party at all, Octopus is open source and self-hostable. Spin it up next to your repos, bring your own Claude or OpenAI key, and process code in memory only. Embeddings stay on your infrastructure. Source code is never persisted.

git clone https://github.com/octopusreview/octopus.git
cd octopus
docker compose up -d

If you want the reviewer to enforce your own rules, drop your security standards into the Knowledge Base and they become part of every review:

octopus knowledge add ./docs/security-standards.md --title "Security Standards"

Now the review does not just catch generic CWEs. It catches the specific patterns your team has decided are not allowed.

The new floor

For a long time, "we passed code review" meant a teammate gave it a thumbs up and CI was green. That floor is no longer enough. The floor is now: a codebase-aware reviewer with full context has looked at this change the way an attacker with an agent loop will look at it tomorrow.

You can hit that floor today. Try Octopus Review at https://octopus-review.ai, star the repo at github.com/octopusreview/octopus, or come argue about agentic exploit research with us on Discord. The window to find your bugs first is open. It is not staying open forever.