How it works

A tracer, a proof, and a comparison. Plus the long list of reasons it would rather run than guess.

A murmuration of starlings and one peregrine diving at a single bird at its edge.

The trace

stoop starts the command under ptrace with a seccomp filter in front of it. The filter lets ordinary system calls through untouched and stops only the ones that name a file, a directory or a connection, so a traced build pays for its file activity, not for every call it makes. Children, threads and anything they exec are followed.

From those stops stoop collects every path the process tree:

  • opened for reading,
  • executed (the compiler, the interpreter, a helper script),
  • looked up, including lookups that found nothing,
  • listed as a directory,
  • followed as a symlink,

and every path it created or rewrote inside the repository.

The proof

When the command exits 0, stoop writes a proof: a JSON file under .stoop/proofs/ holding

PartWhat is stored
InputsEach path with an id: a hash of every byte for a file, a hash of the entry list for a listed directory, the target for a symlink, and "absent" for a path inside the repository that was looked for and not found.
EnvironmentA hash of each variable's value. See the environment below.
OutputsWhat the run left inside the repository, with hashes. A skip needs them still in place.
ProbesFor each --probe command, a hash of what it printed.
Directory and repositoryThe working directory relative to the root and, for an external store, a repository identity.
PlatformThe operating system, architecture, tracer and tracer version that observed the command.

Inputs and outputs inside the root are stored as relative paths, so a proof stored with the checkout can move with it. Paths outside the root stay absolute. A failing run records nothing. Neither does a run started with --no-record.

Proofs are specific to that platform and tracer. A Linux x86-64 proof cannot skip a Linux arm64 command, even when they share a store or Git remote. Proofs without platform information are treated as cold; run the command on a supported Linux platform to record a fresh one. New proofs use a format older stoop versions cannot read, so update runners sharing a store. stoop ls shows the recorded platforms. macOS has no tracer, so it runs commands without recording or skipping; stoop explain reports that they would run.

The comparison

Next time, before running anything, stoop loads the proofs for the key, newest first, and checks each in turn: same platform and tracer, not expired, same environment, every input still the same, every output still in place, every probe still printing the same thing. The first proof that passes means a skip. If none passes, the command runs and the reason given is what differed from the newest proof. stoop explain prints what differed in each proof it checked.

Checking costs one hash of every recorded input. For the cargo build in the getting started transcript that was 668 inputs, the Rust toolchain included, in 289 ms. Every file is hashed byte for byte, with no sampling and no trust in timestamps.

Keys

A key names one command in one place. By default it is the program's name plus a hash of the full argument list and the working directory relative to the repository root, for example cargo-46fcb2881638d3cc. The root is the nearest ancestor containing a .git directory or file. A nested repository uses the inner root; without .git, the working directory is the root. --root DIR selects a different root. Symlinked paths are resolved to their physical directories.

Proofs live under <root>/.stoop by default, so separate roots have separate stores. With an external --store or STOOP_STORE, automatic keys also include a repository identity. The configured Git origin URL identifies the repository, so checkouts with the same origin can share proofs. Local origin paths are resolved relative to the checkout. Without an origin, the physical root path is the identity: separate roots stay separate, and moving one starts with a cold cache in an external store. The URL or path is hashed before it is stored.

--key NAME keeps the chosen name, while each proof still has to match the working directory and repository identity. Proofs recorded before directory tracking are rerun once. stoop explain and stoop ls show the directory, with literal percent signs and non-UTF-8 bytes percent-escaped.

In shell mode (stoop sh, used by GitHub Actions), the key comes from the step's script contents, its arguments, the workflow name, the job name and the working directory. The script file itself is not an input: the runner gives it a new name on every run.

Branches, reverts and history

Proofs are content-addressed. They don't know which branch or commit made them, only what the command read. Each key keeps its 8 most recent proofs, so a branch that matches what main already proved can skip, and so can a revert, as long as the earlier proof's conditions still hold.

That last part matters for builds. A proof requires the outputs its run left in the repository to still be there and unchanged. If a later build rewrote target/, an earlier build proof no longer matches even after you revert the source, so the build runs again. Steps with no outputs in the repository (tests, linters, type checks, or anything writing elsewhere) do reuse earlier proofs across a revert. Restoring outputs on a skip is on the roadmap, not built.

The environment

A variable that a program reads never appears in a file trace, so the whole environment counts. Two groups are left out:

  • The runner's per-run bookkeeping: anything starting GITHUB_, RUNNER_, ACTIONS_, INPUT_ or STOOP_, unless the command text names it.
  • Values that differ on every start of the same machine: _, PWD, OLDPWD, SHLVL, TERM, HOSTNAME, INVOCATION_ID, JOURNAL_STREAM, SSH_AUTH_SOCK, XDG_RUNTIME_DIR.

--ignore-env NAME leaves out one more. --env NAME puts one back in, for example a GITHUB_ variable the step depends on without naming it.

When it runs anyway

stoop fails wide. Each of these means the command runs, and the ones marked "not recorded" also mean no proof is written:

SituationWhat happens
No proof for the keyRuns, then records.
Newest proof older than --max-age (7 days by default)Runs, then records.
STOOP_FULL=1Runs even when a proof matches, then records.
The command failedExits with its code; not recorded.
The trace may be incompleteNot recorded, with the reason.
A traced path cannot be represented as UTF-8Not recorded: the proof must name the exact path.
The command read .gitNot recorded: a skip could hide a history change. Add .git to .stoopignore if that read cannot change the result.
The command used the network or a local serviceNot recorded, naming the destination, unless a --probe covers it or --allow-network is set.
A probe failed, or changed while the command ranNot recorded.
Tracing is unavailable (including macOS)Runs untraced; never skips or records.
STOOP_OFF=1Runs the command directly; no tracing, no skipping.