Code Quality Metrics

The CRAP Metric: Finding Untested Complexity

CRAP stands for Change Risk Anti-Patterns. It multiplies how tangled a method is by how untested it is, and it answers a question no single metric can: which code is dangerous to change? That question got a lot more urgent once agents started writing the code.

Assess Our Code Quality Gates View QA Automation Services

A Deliberately Blunt Instrument

Alberto Savoia and Bob Evans introduced CRAP in 2007 alongside Crap4j, a Java tool that scored every method in a build. The premise was that neither complexity nor coverage means much alone. A gnarly method with thorough tests is workable. A trivial method with no tests is fine. What actually hurts is complexity nobody has covered, because that is the code where a small edit produces a surprise nobody catches. CRAP puts both terms in one expression so the dangerous combination scores badly and the harmless combinations do not.

The name is doing deliberate work. A metric called Change Risk Anti-Patterns gets discussed once; a metric that tells you a method is crap gets fixed.

What It Is Actually For

CRAP is not a dashboard ornament. Each score resolves to one of four decisions, which is the whole reason to compute it.

Where the next test goes

Sorted descending, the report is a work queue for testing effort. The top of the list is where a test buys the most reduction in risk per hour spent, which is a far better answer than chasing a global coverage percentage.

What to refactor before touching it

A high score driven by complexity is a warning that the method will fight you. Splitting it before adding behaviour is usually cheaper than adding behaviour and then trying to test the result.

Which change needs a careful human read

A diff that raises CRAP on methods it touches is a diff that deserves line-by-line review. A diff that lowers it can be skimmed. That routing decision is where the metric pays for itself in review time.

What to regression-test before a release

Change risk plus recent churn tells you which modules earned a regression pass this release, and which ones you can take on trust because nothing in them moved.

The Formula, and Why the Exponents Are Uneven

Complexity is squared. The uncovered fraction is cubed. That asymmetry is the entire design.

CRAP(m) = CC² × (1 − cov)³ + CC
  • • CC is the cyclomatic complexity of the method: the number of independent paths through it
  • • cov is the coverage fraction for that method, from 0 to 1, so (1 − cov) is the untested part
  • • Cubing the untested fraction makes the first term collapse fast as coverage rises, so testing an ugly method pays off immediately
  • • At full coverage the first term is zero and CRAP equals CC, the residual risk the tests cannot remove
  • • The trailing + CC is what stops the metric from ever pretending complexity is free

What the Numbers Actually Demand

Using the conventional threshold of 30, here is how much coverage each complexity level needs before a method falls below the line.

CC 0% cov 50% cov 100% cov To clear 30
5 30.0 8.1 5 None. Simple code passes untested.
10 110.0 22.5 10 About 42%
15 240.0 43.1 15 About 60%
20 410.0 70.0 20 About 71%
25 650.0 103.1 25 Exactly 80%
30 930.0 142.5 30 100%, and it lands exactly on the line
31+ 961.0 155.2 31 Unreachable. Tests cannot fix this one.

What the Curve Is Telling You

Past complexity 30, coverage stops helping

With a threshold of 30, a method at complexity 31 is over the line even at 100% coverage. That is not a flaw in the formula, it is the message: the only move left is to split the method.

At full coverage, CRAP is just complexity

Tests never forgive complexity, they only stop it being amplified. A well-covered complex method still carries its complexity as acknowledged, managed risk rather than hidden risk.

Simple code is left alone on purpose

A complexity 5 method sits exactly at 30 with no tests at all. The metric routes attention towards tangled code instead of generating busywork on getters and mappers.

Coverage is the weak leg

cov measures execution, not assertion. A suite of assertion-free tests raises coverage and lowers CRAP while changing nothing about real risk. That specific gap is what mutation testing exists to close, and it is the gap an LLM will walk straight into.

How a QA Team Uses It

This is the part usually left out of metric write-ups. CRAP is a targeting instrument, and QA is who it is aimed by.

Aim the regression suite instead of growing it

  • - Rank methods by CRAP multiplied by commit frequency, and build the release regression pass from the top of that list
  • - Low-CRAP modules with no churn do not need a fresh pass every release, which is where the time for the top of the list comes from
  • - Re-rank per release rather than maintaining a static suite that grows forever and is never pruned

Decide automation versus exploration

  • - High complexity with low coverage is a unit-test gap, not an exploratory testing gap — asking manual QA to cover 20 branches by hand is a waste of a good tester
  • - High complexity with high coverage is where exploratory testing earns its keep, because the paths are exercised but the requirements may still be wrong
  • - Undefined behaviour hides in uncovered branches of complex methods, so those are the places to design negative and boundary cases first

Make release gates arguable in numbers, not opinions

  • - A gate of no changed method above threshold is enforceable, reviewable and hard to argue with at the end of a sprint
  • - Report the two inputs alongside the score so the fix is unambiguous: split it, or test it
  • - Track escaped defects against the CRAP of the method they came from — that correlation is how you tune your own threshold instead of inheriting ours

Use it as the shared language with development

  • - Quality objections framed as this method is messy lose to deadlines; framed as this method is complexity 24 at 30% coverage they usually do not
  • - It gives QA a legitimate, pre-agreed reason to ask for a refactor before a feature lands
  • - It also protects developers from busywork, because the metric explicitly says trivial code needs no tests

Using CRAP to Judge Code an LLM Wrote

Agents changed the economics of this metric. They produce plausible code faster than anyone can read it, and they will happily write the tests that measure themselves. Complexity against coverage is one of the cheapest honest signals you have left.

The specific hazard is that an LLM optimises for the visible objective. Ask for tests and you get tests; ask for coverage and you get coverage. Both halves of CRAP are things an agent can move, but it can only move one of them honestly. Complexity is a structural property of the code it wrote — it cannot be talked up or down. Coverage is a number the agent can inflate by executing lines without asserting anything about them. So the pairing is diagnostic: complexity tells you what the agent built, coverage tells you what it claims about it, and the gap between them is where you look.

Score the diff, not the repository

  • - Compute CRAP on the methods an agent touched, before and after, and post the delta on the pull request
  • - Rising complexity with flat coverage is the signature of behaviour bolted onto an existing function, which is the default way an agent adds a feature
  • - Falling complexity with rising coverage is what a good agent run looks like, and it is worth rewarding by reviewing it faster

Put a complexity ceiling in the agent's own instructions

  • - Uncle Bob's swarm does exactly this: the cleaner role runs the CRAP tool first and drives complexity to 6 or below before anything else happens
  • - That is a smarter use of the metric than gating on the score, because at complexity 6 the number can barely rise no matter what coverage does
  • - Give the agent the threshold and the command, not a paragraph about clean code, and it will comply because the check has an exit code

Feed the tool output back into the loop

  • - A CRAP report in a dashboard changes nothing; the same report piped into the agent's next turn as a failing check gets fixed
  • - Run it as a separate step from the one that wrote the code, so the agent cannot grade its own homework
  • - Cap the retries — an agent that cannot get under the threshold in two attempts is telling you the design is wrong, not that it needs another go

Never let one agent both raise coverage and report the metric

  • - Coverage is the gameable half, and an agent asked to improve its own score will find the cheapest path to the number
  • - Pair every CRAP gate with a mutation run, which asks whether those new tests actually assert anything
  • - Treat the agent's explanation of a score as marketing copy; the tool output is the evidence

Reading the Signal on Agent-Written Code

The same numbers mean specific, recognisable things when the author is a model.

Signal What it usually means What to do
Complexity climbed, coverage flat New behaviour was added as extra branches inside an existing function. Ask for extraction before review. This is the most common shape of agent code growth.
Coverage jumped, complexity unchanged Tests were added. Whether they assert anything is not yet established. Run mutation testing on the changed files before you believe the number.
One method far above the threshold The model kept appending cases to the place it first found, prompt after prompt. Split by rule, then re-run. The score usually collapses without any new tests.
Complexity in code nobody asked for Speculative branches, defensive paths and options with no requirement behind them. Delete it. Untested complexity that no requirement names is the cheapest thing to remove.
Score good, coverage from snapshot tests The suite executes everything and checks almost nothing. The metric is lying to you through its coverage input. Fix the tests, not the score.

Working With It

The formula is two lines of code, which is most of why it keeps being reimplemented in new ecosystems.

The metric itself

js

Any coverage report plus any complexity tool gives you everything the formula needs.

// crap.js — coverage is a fraction, 0..1
export function crap(complexity, coverage) {
	const untested = 1 - coverage;
	return complexity ** 2 * untested ** 3 + complexity;
}

crap(15, 0);    // 240
crap(15, 0.5);  // 43.125
crap(15, 1);    // 15

Gate the change, not the codebase

sh

A ratchet on changed methods keeps new risk out without opening a cleanup project nobody funded. Legacy scores stay visible on a dashboard instead of blocking every build. For agent-authored branches this is the whole gate.

# CI: score only the methods this branch touched
git diff --name-only origin/main... -- '*.js' \
  | xargs node ./tools/crap-report.mjs --threshold 30 --changed-only

# exit non-zero when a touched method crosses the line;
# print the untouched offenders as a report, not a failure

The refactor the score is asking for

js

When the score comes from complexity rather than coverage, testing harder is the wrong response. Pull each branch out into something with complexity 1 and the driver stays flat no matter how many rules arrive — including rules a future agent appends.

// Before: complexity climbs with every rule anyone adds.
function validate(order) {
	if (!order.id) return 'missing id';
	if (order.items.length === 0) return 'no items';
	if (order.total < 0) return 'negative total';
	if (order.currency !== 'USD' && order.currency !== 'EUR') return 'bad currency';
	if (order.customer && !order.customer.email) return 'customer without email';
	return null;
}

// After: each rule is trivially testable, the driver stays at 2.
const RULES = [
	[(o) => !o.id, 'missing id'],
	[(o) => o.items.length === 0, 'no items'],
	[(o) => o.total < 0, 'negative total'],
	[(o) => !['USD', 'EUR'].includes(o.currency), 'bad currency'],
	[(o) => Boolean(o.customer) && !o.customer.email, 'customer without email']
];

function validate(order) {
	for (const [fails, message] of RULES) if (fails(order)) return message;
	return null;
}

The gate an agent cannot talk its way past

sh

Two checks, in this order, run by a step that did not write the code. The first says the code is shaped like something you can change; the second says the tests protecting it actually react when it changes.

# 1. structural: is this changeable code?
crap-report --changed-only --threshold 30 || exit 1

# 2. behavioural: do the new tests assert anything?
stryker run --incremental --mutate "$(git diff --name-only origin/main...)"

# report both on the PR. one number is about the code,
# the other is about the tests that claim to cover it.

How to Use It Without Annoying Everyone

Read the two levers separately

  • - A high score driven by complexity is a refactoring task, not a testing task
  • - A high score driven by missing coverage is a testing task, and a cheap one
  • - Always show CC and coverage next to the score, because the number alone does not say which

Weight by churn

  • - Change risk only matters where change happens
  • - A score of 200 in a file untouched for four years is less urgent than 60 in a file edited weekly
  • - Sort by score multiplied by commit frequency and work down the top of that list

Make exclusions explicit

  • - Generated code, adapters and exhaustive switch statements inflate complexity without inflating real risk
  • - Exclude them in configuration, with a comment saying why, rather than quietly raising the threshold
  • - Revisit the exclusion list when the code it protects stops being generated

Common Misreadings

Treating it as a quality score

CRAP estimates the risk of changing code. It says nothing about whether the code is correct, well named, or well designed. A clean, well-covered method with genuine domain complexity scores the same as an unpleasant one.

Gaming it with coverage

Snapshot-everything tests and assertion-free walkthroughs move the number without moving the risk. If CRAP is a gate, something has to keep the tests honest — review, mutation testing, or both. With an agent in the loop, assume this will happen unless a separate check prevents it.

Launching a cleanup epic

A repository-wide CRAP report on legacy code produces a number so large it gets ignored. Ratchet on changed code instead, and let the dangerous parts get fixed by the people, or agents, who were already going to touch them.

Expecting a maintained tool

The original Crap4j has been dormant for years. NDepend carries the metric on .NET, there are community implementations for Rust, .NET and Groovy, and Uncle Bob maintains crap4j, crap4go and crap4clj for his agent swarm. On most stacks you compute it yourself from data you already collect.

One Number, One Question

CRAP is not a quality score and was never meant to be one. It answers a narrower and more useful question: if someone edits this method next week, how likely is it that something breaks quietly? Complexity says how many ways there are to get it wrong, coverage says how many of them anyone is watching, and the formula weights the second more heavily than the first.

That makes it a targeting tool for QA and an audit tool for AI-assisted work. Point it at the diff, gate new and changed code at a threshold, rank the rest by score against churn, and keep the two inputs visible so the number resolves to an action: split this method, or test it. Just remember which half an author can fake. Complexity is what the code is; coverage is only what the tests claim, and mutation testing is how you check the claim.

Related Engineering Articles

The coverage half of this formula is exactly what mutation testing interrogates, and both metrics have already been assigned to dedicated roles in a working agent swarm.

Back to QA automation
Test Design

Mutation Tests Are Negative. TDD Tests Are Positive.

Why mutation runs can only report what a suite misses, how to run them affordably, and how to use them to judge tests an LLM wrote for its own code.

Agentic Engineering

SwarmForge Reviewed: How Uncle Bob's Agent Swarm Actually Works

A deep review of SwarmForge's role pipeline, handoff daemon and executable quality gates, with an explicit verdict on what to adopt and what to skip.

FAQ

What counts as a bad CRAP score?

Thirty is the conventional default threshold, inherited from Crap4j and reused by most later implementations. Lower it for greenfield code where the cost of holding the line is small, and keep it while ratcheting on legacy code rather than raising it to make a report look better.

Is CRAP useful for reviewing AI-generated code?

It is one of the most useful cheap signals available, because complexity is a structural fact about the code the model produced and cannot be argued with. Score the diff rather than the repository, watch for complexity rising while coverage stays flat, and always pair it with a mutation run so the coverage half cannot be inflated by tests that assert nothing.

What threshold should we set for agent-written branches?

Tighter than for humans, and expressed as complexity rather than score. The practical pattern, and the one Uncle Bob's swarm uses, is a hard complexity ceiling around 6 on changed methods: at that level the CRAP score cannot climb much whatever happens to coverage, so there is nothing left to negotiate.

Can an agent fix its own CRAP score?

Usually yes for the complexity half, which is genuine improvement, and that is worth automating. For the coverage half be careful: the cheapest way to raise coverage is to execute code without checking it. Run the gate in a step that did not write the code, and verify the new tests with mutation testing.

Why is complexity squared but the untested fraction cubed?

So that coverage moves the score faster than complexity does. The cubed term collapses towards zero as coverage approaches full, which rewards testing complex code immediately, while the squared term plus the trailing complexity keeps a floor under the score no amount of testing can remove.

Should CRAP fail the build?

As a ratchet on new and changed methods, yes, because that is enforceable and nobody has to schedule it. As a repository-wide gate on an existing codebase, no. That produces a backlog rather than a behaviour change.

© 2026 - Ryware.