Skip to content
denoise

How it is tested

This page says what the numbers on the home page measure, and what nothing here measures. denoise draws what the code says. It does not say whether the code is right, and it has no figure for how often a drawing is the one a reviewer needed.

What the corpus run measures

Section titled “What the corpus run measures”

The home page says there were 0 failures across 70,811 functions in 22 real repositories. That comes from the corpus run: every function in the source folders listed for 22 open-source repositories, each pinned to one commit, goes through the same code the app runs. The folders are chosen, not the whole of each repository: corpus/repos.json names them. scripts/corpus.mjs is the whole check, and the nightly workflow runs it.

For each function it does these things:

  1. Reads the file and lists its functions, then builds the function’s tree.
  2. Checks the tree against the front end’s contract (src/tree-check.js): node shapes, line spans that sit inside their parents and inside the function, siblings in source order, and as many exits on the rail as the tree has.
  3. Draws the function as HTML at the four renderer levels, and as an image.
  4. Draws the Minimal level a second time and compares the two byte for byte.
  5. Diffs the function against itself, which must show no changes.

A failure is any step that throws, any broken contract, any pair of draws that differs, and any self-diff that shows a change, unless corpus/known-issues.json lists it. That list is empty. A function that takes over a second is reported as slow, which is not a failure.

The figure on the home page is from the run of 2026-10-25: 4,885 files, 70,811 functions, 0 failures and 1 slow function. The nightly workflow reruns it and fails on any new failure.

So “0 failures” means: nothing crashed, every drawing met its structural rules, and the same input drew the same way twice. It does not mean a drawing is a correct reading of what the function does. A tree can satisfy every rule and still be wrong, for example by resolving a call to the wrong function by name. Parsing and drawing without an error is not evidence of that. The next section covers what is.

Does the drawing carry the facts?

Section titled “Does the drawing carry the facts?”

A blind test gave models the picture of a function they had never seen, and nothing else: no code. Their answers were scored against facts written down before anything was drawn. On five fresh functions, used once after the renderer was tuned, the Outline picture carried about 87% of the facts and the Full picture about 96%. On an earlier holdout, Outline scored 76 to 78 percent, so the tuning had overstated the general case.

The limits are plain. The sets are small (five to six functions each), most of the functions come from the author’s own projects, and the test measures whether the picture holds the facts, not whether it makes a review faster or catches more. A second test on a large Go codebase is in eval/tsgo/results.md. The questions, the answer keys and every score are in eval/functions/results.md.

Flow changes

Section titled “Flow changes”

Flow changes say how the control flow around a call changed between two revisions of a function. It reads JavaScript and TypeScript only, and one function at a time.

It reports three kinds of fact, each about one call that both revisions make:

  • a condition that governs the call changed (an enclosing if or else if, or an early exit in front of it);
  • an early return or throw in front of the call was added or removed;
  • the call moved into or out of an if branch.

A call is paired across revisions only when its callee and arguments are the same text, and it appears exactly once on each side. Everything else is not paired. The wording is a template: a changed condition is “changed”, never “stronger” or “weaker”, and nothing is read from names. No code is run, and there is no model in the loop.

When part of a function falls outside what it can read, it says so. A change in the result is then listed as not analyzed, and the card shows “analysis incomplete” rather than “none detected”. The analysis understands:

  • if / else if / else
  • early return or throw as if (cond) return|throw
  • blocks
  • calls and new in statements and expressions

and it leaves these out, each reported as skipped when a call needs it:

  • for, while, do, for…in/of
  • switch
  • try / catch / finally
  • labeled statements
  • with
  • class bodies
  • functions and callbacks nested in the function
  • calls in ?: branches
  • calls on the right of &&, || or ??
  • optional calls (?.)
  • a return or throw nested in another statement before the call
  • the same call (callee and arguments) more than once
  • an early exit that moved from before the call to after it (or the reverse)
  • branch nesting changed in a way not summarized
  • a version the parser could not read cleanly
  • a function that could not be located in its file

What there is to check

Section titled “What there is to check”
  • test/flowchanges.test.js has a test for each kind of observation and for each reason to skip, including the cases that must stay silent. The example on the home page is the first of them.
  • docs/data.md specifies the JSON every observation comes in, with the evidence and its exact ranges.
  • The home page example is not a picture. site/scripts/flow-demo.mjs runs the engine on the two fixture files in test/fixtures/flow when the page is built, and a test fails if the page and the engine disagree. node scripts/flow-demo.mjs in the repository serves the same pull request in the real app, on your machine.

What has not been measured

Section titled “What has not been measured”

There is no figure for how often a flow observation is the thing a reviewer cared about, or for how many real changes it does not see. It was built from cases and is tested on cases. scripts/flow-demo.mjs --pr owner/repo#number runs it on any public pull request, but that is a demonstration, not a measurement. Treat a quiet result as “no supported change found”, never as “nothing changed”. An observation is a fact about the shape of the code. It is not a score and not a verdict that the change is safe or wrong.

Where to look

Section titled “Where to look”