Screenshot Tests Where the Input Is State

Most screenshot-testing tools take a composable and some arguments and give you a PNG. f(@Composable args) → PNG. Paparazzi and Roborazzi both work this way, and they work well.

rk snapshot takes a store state and gives you a PNG. f(state) → PNG. You seed a store, dispatch into it, and it renders your real Compose screen off-screen against the resulting state.

That difference sounds academic. It’s not, and I want to show you the moment it earned its keep.

Install

1
brew install reduxkotlin/tap/rk        # macOS / Linux

1
scoop bucket add reduxkotlin https://github.com/reduxkotlin/scoop-bucket
2
scoop install rk                       # Windows

No JDK required: the archives ship a jlink-minimized JRE. The macOS bottle is Apple Silicon only right now; on an Intel Mac, build from source.

The thing everyone gets wrong

The rk binary can only render its own built-in demo scenes. It has no idea what your app looks like.

To render your screens, you depend on the library and give it a scene registry:

1
// build.gradle.kts — test scope, so Skiko never ships in your app
2
testImplementation("org.reduxkotlin:redux-kotlin-snapshot:1.0.0-alpha04")

1
val mySnapshots: SnapshotApp = snapshotApp {
2
    defaults { width = 411; height = 891; density = 2f; theme = "dark" }
3

4
    scene("board") {
5
        presets("seeded", "empty")
6
        render { args -> boardScene(presetOf(args), themeOf(args.theme)) }
7
    }
8
}
9

10
fun main(args: Array<String>) {
11
    mySnapshots.runCli(args)
12
    kotlin.system.exitProcess(0) // Skiko leaves non-daemon threads alive
13
}

A scene builds a real store, dispatches real actions, and renders the real screen. The library never deserializes your types. You hand it a preset name or a JSON blob, and your scene decodes it.

The commands

rk snapshot has no subcommands. It’s one command driven by flags.

1
# what can this binary render? (machine-readable)
2
rk snapshot --list
3

4
# render one shot
5
rk snapshot --scene board --preset seeded --out goldens/board-seeded.png
6

7
# verify it against a golden
8
rk snapshot --scene board --preset seeded --verify goldens/board-seeded.png
9

10
# render from arbitrary state — no preset needed
11
rk snapshot --scene counter --state-json '{"count":3}' --out c3.png
12

13
# render the whole manifest and build a browsable dashboard
14
rk snapshot --batch shots.json --out-dir build/snapshots --dashboard

Exit codes are 0 ok, 1 render or verify failure, 2 usage, because CI reads exit codes, not prose.

One flag worth calling out: in batch mode, verification is turned on by --golden-dir, not by --verify. --verify is single-shot only.

The manifest and the dashboard

The manifest is boring on purpose. This is TaskFlow’s, in full:

1
{
2
  "shots": [
3
    { "id": "board-seeded",     "scene": "board",    "preset": "seeded" },
4
    { "id": "board-empty",      "scene": "board",    "preset": "empty" },
5
    { "id": "settings-default", "scene": "settings", "preset": "default" },
6
    { "id": "settings-offline", "scene": "settings", "preset": "offline-failing" },
7
    { "id": "settings-bot",     "scene": "settings", "preset": "online-bot" }
8
  ]
9
}

--dashboard writes a self-contained index.html over the run’s report.json. Cards are sorted worst-first, each showing a golden / actual / diff triptych. Click any image to zoom to 1:1 native pixels. Changed pixels are painted magenta over a dimmed copy of the actual.

Where pixel diffing quietly fails

Here is the part I actually want you to read.

While writing this post I was generating TaskFlow screenshots, and I noticed a bug in my own sample app. The card footer said “1 labels”. Classic. So I fixed it:

1
- text = "$labelCount labels",
2
+ text = if (labelCount == 1) "1 label" else "$labelCount labels",

Then I re-ran the batch against the goldens I had rendered five minutes earlier, expecting the golden gate to flag the change.

It didn’t.

1
batch: 5/5 ok, 0 failed, 0 mismatched, 0 semantics-mismatched -> build/snapshots

The batch tolerance is 0.5%. Changing labels to label moved 0.0072% of the pixels: one character, in body-small type, on one card. It sailed under the threshold. As far as the pixel gate was concerned, nothing had changed.

That’s not a bug in the differ. Any tolerance you pick has this property, and you need a tolerance, because sub-pixel anti-aliasing noise across a batch will fail you constantly if you set it to zero. But it means a pixel gate can’t be trusted to catch a text change, which is a large fraction of the changes anyone actually makes.

The semantics dump

This is why --semantics exists. It dumps the Compose semantic node tree as text: the content, not the pixels.

1
rk snapshot --batch shots.json --golden-dir goldens --update-semantics   # record
2
rk snapshot --batch shots.json --golden-dir goldens --verify-semantics   # gate

Same code change, same run, with the semantics gate turned on:

1
batch: 4/5 ok, 0 failed, 0 mismatched, 1 semantics-mismatched -> build/snapshots
2
  drift board-seeded: pixel=match semantics=mismatch png=board-seeded.png diff=- sidecar=-
3
    -                           "1 labels",
4
    -                           "1 labels",
5
    +                           "1 label",
6
    +                           "1 label",
7
batch had 0 failed / 0 mismatched / 1 semantics-mismatched

Exit code 1, and it hands you the exact diff. That’s real output, not a mock-up: pixel=match and semantics=mismatch, on the same shot, in the same run.

The dashboard puts it at the top, sorted worst-first:

Two gates, and they catch different things. Pixels catch layout, color, spacing, and anything that moves a lot of them. Semantics catch content: the text and structure, the things a human would call “what the screen says.” Neither is a superset of the other, which is why both ship.

The three-tier loop

That gives a verify loop with three tiers, cheapest first. This ordering matters most when the thing running the loop is a coding agent, but it’s a good ordering for humans too:

1. Tier 0, semantics. A text diff. Assert content deterministically: the title is "Ship v1", the column has two cards. Zero vision tokens. Start here.
2. Tier 1, the golden verdict. A pass/fail line out of report.json. Still text. No image is opened.
3. Tier 2, read the PNG. Only when a shot actually drifted and you need to decide whether it drifted correctly.

Most iterations never leave Tier 0. Looking at a picture is the expensive, last-resort operation, and it should be treated that way. Part 9 is where this pays off.

Sharp edges I’ve actually hit

I run this in anger on another app of mine, which has 130-odd committed goldens across 20 scenes. Three things it taught me:

• Keep per-OS golden sets. CoreText and FreeType disagree about glyph advances by enough to flip a line-wrap in a long paragraph, which then shifts an entire layout. I’ve measured a 22% pixel diff from nothing but a macOS-vs-Linux font difference. That app keeps snapshots/macos/ for the local loop and snapshots/linux/ for CI.
• Pin the clock. If a scene renders a date, an unpinned clock will rot your goldens overnight. Inject it.
• Tolerance is a real dial. I run at maxDiffPercent = 4.0, which sounds enormous until you’ve watched anti-aliasing noise fail a build for the fourth time.

The limitation

This is the least cross-platform thing in the whole series.

redux-kotlin-snapshot is JVM-only. It’s a Skiko renderer, so it needs a desktop graphics stack: it renders your Compose Multiplatform screens on the JVM and that’s that. The rk binary’s macOS bottle is Apple Silicon only. The library runs on nine targets. The tooling does not.

For an Android/desktop/iOS app sharing one Compose codebase, rendering on the JVM is fine, because they’re the same composables. If your UI diverges meaningfully per platform, this tool tests the shared part.

It’s also experimental and exempt from semver until it settles. It will move.

Full details in the snapshot testing doc.

Happy loop closing!

Sources of truth: reduxkotlin.org · github.com/reduxkotlin/redux-kotlin · the TaskFlow sample

Part 8 of ReduxKotlin 1.0.