Verifying a pull request

maintainer-merge verify earns the receipt that maintainer-merge merge asks for. It proves one thing: a test fails when the guard it names is removed. A green board cannot show that, so verify runs the test twice, once on the pull request as it is and once with a mutation applied, and records a receipt only if the first run passes and the second fails.

maintainer-merge verify <pr> <head-sha> <test-filter> <sed-mutation> [suite]

What it does

  1. Takes a per-pull-request lock, so two verifies of one pull request cannot build the same workspace side by side.
  2. Reads the changed paths and picks a suite from the profile's verify.d/: the one you named, or the one suite that covers every changed path, or the one that covers every production path. When none does, it refuses and prints which suite covers which path, instead of running the wrong one.
  3. Fetches the pull request head, checks it is the sha you named, extracts it to a scratch directory, and refuses a tree with a symlink that points outside itself.
  4. If the suite defines suite_prefetch_command, runs it once with a network (the rust suite runs cargo fetch --locked), so a dependency bump can find the crates it names. The runs that execute tests never have a network.
  5. Runs <test-filter> in a container: --network=none, --cap-drop=ALL, --read-only, no-new-privileges, a /tmp tmpfs, and the memory, CPU, process and time limits in configuration. It must pass, and suite_ran must count at least one test that executed.
  6. Applies <sed-mutation> to the files matching suite_mutate_glob. The mutation must change at least one file, and one the profile calls production (PROD_GLOBS), or the receipt would say a test notices its own edit.
  7. Runs the container again. It must fail. A container that was killed (memory, timeout) is reported as that, never as a guard that bites or a contributor whose test broke.
  8. Writes a signed receipt of kind observed under the state directory, with the first failing line of the mutated run.

verify-deps <pr> <head-sha> is the path for a pull request that only bumps dependencies or pins: there is no guard to mutate, so the gate classifies the diff and checks pins and image digests instead. verify-part proves one suite's share of a diff that no single suite covers, and merge checks that the parts add up to every production path.

A suite

A profile's verify.d/<name>.sh is sourced by the gate and defines functions. The file name is the suite name.

functionrequiredwhat it answers
suite_covers <path>yesdoes this suite run that path
suite_imageyesthe image to run in
suite_command <filter>yesthe shell command to run in the container
suite_mutate_globyesone glob per line, the files a mutation may touch
suite_ran <log>yeshow many tests the clean run executed, read from its log
suite_needsnobinaries the image must contain; maintainer-doctor checks them
suite_named_onlynodefined means the suite runs only when named, never by inference
suite_requires <runtime>norefuse a runtime the suite cannot use
suite_podman_args <runtime> <tree>noextra container flags, one per line
suite_prefetch_command, suite_prefetch_guardnothe networked fetch, and a host-side check that runs first
suite_memory, suite_cpus, suite_pids, suite_tmpfsnothis suite's own resources, above the profile and below MAINTAINER_VERIFY_*

The magent profile has a suite named characterization that runs one case file of tests/characterization/ through tests/run-characterization.sh. The gate's filter argument is the case file name, and it must match ^[a-z]+-[a-z0-9-]+\.sh$ and exist. The suite defines suite_named_only, which tells pick_suite and coverage_map never to infer it: it covers the same bin/ and lib/ paths as the shell suite, so you name it:

maintainer-merge verify <pr> <sha> <case-file> '<sed>' characterization

Use it for a change to the orchestrator, the backends, the failure-shape classifier, the installers, the doctor or the repo tool, which tests/run-tests.sh does not pin. CI runs the characterization runner as its own job; the pre-commit hook does not, because the cases take about 155 s.

suite_ran cannot be generic: cargo prints test result: ok. N passed, and a shell test prints whatever its author chose. A suite that cannot say how many tests ran gets no receipt, because a filter that matches nothing passes.

A profile with no verify.d/ gets no merge. Copy profiles/_template/verify.d/shell.sh and change the five required functions.

After the receipt: bringing a BEHIND branch up to date

merge refuses a pull request that is BEHIND its base, because its checks describe a tree that is not the one being merged. When the pull request is approved and that is the only thing missing, the agent runs maintainer-merge update <pr>. The order is: verify, approve, update if BEHIND, wait for green, merge.

update applies the same receipt checks merge does (signature, the pull request the receipt names, its kind, coverage of the production paths, and the tree-merge comparison that lets GitHub's own merge commit pass), so a branch nobody verified is not updated. It asks GitHub to update the branch pinned to the head it checked, then reads the pull request back until the head has moved and the state is neither BEHIND nor UNKNOWN. The update starts the required checks again; merge refuses until they pass. A fork whose author has not allowed maintainer edits exits 3, and the author presses the button.