Introducing Open-Kritt

The clearest advantage of AI is scale.

Agents perform better when the task is specific. If tokens were free, you could give one agent every line of code and ask it one narrow question. Tokens are not free, so the useful question is where brute force gives the best return.

Two places have worked especially well for us: entry points and bounty scopes.

The first agent finds all entry points. Open-Kritt can then launch one agent for every combination of entry point and relevant scope. For the same entry point, one agent might look for a node crash, another for a chain fork, and another for a path that could lead to a loss of funds.

This is one of our simplest and most battle-tested workflows, and it is included in this release. Brute force becomes even more effective when the bug pattern is precise. We encourage researchers to take bugs they already understand and turn them into targeted workflows. Open-Kritt also includes an AI assistant that helps create workflows from a research idea or known pattern.

And yes, the more you spend, the more results you'll have. To reach first place in the Firedancer competition, we spent over $2,000 on tokens.

In our experience, stronger models generally find more complex vulnerabilities.

We use the models that perform best on our private benchmark. Right now, our strongest results come from GPT-5.6, Claude Opus 4.8, and Kimi K3.

We have also found that models often perform best inside their native coding harness. GPT models work best for us with Codex, while Opus works best with Claude Code.

Open-Kritt supports both harnesses, Codex and Claude Code, from day one, along with three model providers. You can use your existing paid ChatGPT or Claude account and its usage limits, or connect models through OpenRouter. Use whichever combination works best for your workflow and budget.

The right tools matter, but we found that vulnerability research tools overlap heavily with what Codex and Claude Code already provide.

Both harnesses can inspect code, search the web, and run command-line tools. Because each Open-Kritt agent has its own container, it can also install libraries or utilities when it needs them. In most cases, that has been enough.

We have not found generic skills to be a major improvement over focused prompts and our own written research tips. Niche skills are different. A skill that captures knowledge about a specific protocol, language, or bug class can be useful.

Open-Kritt leaves that choice to you. You can attach open-source skills to a scan, including the excellent security skills shared by Pashov and others.

A bug in the code is not always reachable in production.

A workflow may report findings that cannot happen on mainnet because the current configuration disables the relevant feature. The opposite can also happen. A model may ignore code that is disabled by default even though it might be enabled in production.

This is why configuration gathering needs to be meticulous.

Production configuration belongs to the scan, since it describes the specific target being researched rather than the reusable research method.

Give the scan the relevant feature flags, deployment parameters, genesis values, chain settings, contract addresses, and bounty scope rules. The more accurately the agents understand the production environment, the less likely they are to waste time on impossible paths or dismiss reachable ones.

This part sucks.

The newest models increasingly apply stricter cyber safeguards, and legitimate vulnerability research can get interrupted or blocked.

We recommend that security researchers using Claude or GPT accounts complete the providers' cyber verification processes.

Open-Kritt detects when a provider appears to cyber-limit a request and pauses the scan. We do not recommend retrying a blocked request again and again. Repeated attempts may put the account at risk, and we have also seen harder-to-notice cases where the model continues responding but its cyber performance appears degraded.

Verification is currently available here:

• Claude Cyber Verification Program
• OpenAI Trusted Access for Cyber

These systems change frequently. Verification does not guarantee that every legitimate request will pass, and we still see inconsistent behavior across verified accounts. We do not yet have a good explanation or a complete solution besides using the best available open-source model, Kimi-K3.

Sorry.

A dependency is any codebase outside the main repository that the target interacts with or relies on.

For many vulnerabilities, one repository is not a wide enough view. A bug may sit across several repos. One component may assume that another component validates an input when it does not. For example, relying on a library for verification or cryptography.

Open-Kritt lets you add multiple repositories alongside the main codebase when configuring a scan. Agents can access them when needed instead of carrying every repository in every context.

This also helps with discovery. A workflow can identify which external repositories, contracts, clients, or libraries are relevant before deeper agents inspect them.

This one became more important as the models got smarter.

Instead of giving an agent the exact tool for every possible task, we give it guidance and room to work.

Each Open-Kritt agent runs as root inside its own isolated container. It gets easy access to the target codebase and its dependencies, while remaining free to install tools, execute commands, compile the project, run tests, fuzz inputs, or build a proof of concept.

This is where a lot of the magic happens.

Agents can use dynamic analysis during the workflow to test their theories. After the workflow finishes, the scan can run a stricter validation or false-positive filtering step.

Do not give the agent every tool in advance. Tell it what kinds of tools may help, then let it choose for itself which tools to use and when.

The workflow can produce a large set of raw findings. Some will be unsupported, several may describe the same underlying bug, and many will not deserve the same amount of attention.

This is handled after the workflow at the scan level.

Depending on how the scan is configured, its post-script can ask validation agents to reproduce findings, test the claimed behavior, check whether the required configuration is reachable, or reject reports that do not meet the evidence requirements.

Deduplication also happens here.

One annoying thing about using AI for research is that it keeps finding the same bugs.

Giving one agent 100 detailed vulnerability reports and asking it to deduplicate them usually works poorly. The context becomes too large, and important details get mixed together.

We use a method we call anchored deduplication.

The system takes a small batch, such as ten findings, and merges duplicates into a set of canonical findings. It then processes the next batch against those canonical findings. The surviving set becomes the anchor for the next round, and the process continues recursively.

This keeps each comparison small while preserving the best current description of every distinct issue. Anchored deduplication is only enabled per scan currently; it does not work across separate scans yet.

Another annoying thing about AI is the amount of slop :-)

AI often struggles to decide whether a vulnerability is truly Critical. It tends to exaggerate impact, especially when each finding is judged in isolation.

We found that it performs better when comparing findings.

It may be unreliable when asked, “Is this Critical?” It is much more useful when asked, “Which of these two findings is more promising, and which one should a researcher inspect first?”

Our ranker uses that relative strength.

After the workflow and the other scan-level processing are complete, the ranker compares the remaining vulnerabilities and gives each one a numerical score. The findings are then ordered by that score so you can start with the results the system considers most promising.

The number is not meant to be a final severity verdict. It is a prioritization tool. The goal is to help you decide where to spend your attention when a scan returns too many results to review manually.

Ranker logic is configured per scan. Open-Kritt includes our generic ranker, but it works better when you add the exact program scope, accepted impacts, exclusions, production assumptions, and any other rules that matter for the target.

LLMs are statistical creatures.

We do not measure a model by asking, “Can it find this vulnerability?” We ask, “Out of 100 attempts, how often does it find it?”

Running the same scan or step more than once can improve the chance of finding something. Repeats also help with enumeration. When a task requires a long list, such as every entry point in a large codebase, one run may stop before it finds them all.

A scan can configure repeats for workflow steps.

The same task can run several times, but later runs do not need to start from zero. They can receive the previous run's outputs and focus on extending them. An entry-point agent, for example, can be asked to find what the earlier run missed instead of producing the same obvious list again.

How do you know whether a prompt is good?

You test it against vulnerabilities you already know.

We use a private benchmark that currently consists mostly of bounties we have found. It helps us compare models and workflows, but it is also useful for improving prompts.

When a workflow misses a known bug, we can give the failed scan to another model and ask it to identify why the reasoning broke down. The model then proposes changes to the prompts, the workflow runs again, and the process repeats. The goal is to remove a specific failure mode until the workflow finds the bug or we understand why it cannot.

This kind of self-improvement loop is part of how we develop Open-Kritt internally. It is not implemented in the open-source release yet, but it is planned.