CI / Build Gate Mistakes
Generated from
.claude/rules/anti-patterns.md(the same source as MCPget_anti_patterns). Each entry is a real mistake + its fix; where a detector code is listed, the linter /pyreon doctor/ MCPvalidatecatches it automatically.
The same cache artifact saved under TWO key names by two workflows — a full actions cache evicts exactly the entries every PR depends on
(2026-09). ~/.bun/install/cache (~1 GB per lockfile variant) was saved by ci.yml as bun-install-cache-<os>-<lockhash> AND by release.yml + bundle-size-diff.yml as bun-<os>-<lockhash> — one artifact, two writers, two copies. Five live lockfile variants × 2 = 4.8 GB of pure duplication in a 10 GB budget that was already at 10.02 GB, and the actions cache evicts by LRU with no notion of value: the ~6 MB bootstrap lib/, the <1 MB native verdict store and the 280 MB Playwright browsers were the entries that went, so a main run paid ~60 min of cold kotlinc/swiftc (1325 s + 1312 s + 931 s on run 33532534054, vs ~45 s warm) because a duplicate tarball store was more recently touched. This is the THIRD time the budget hit capacity (2026-08 twice, both "fixed"); each fix trimmed one writer and missed a sibling. Rule: one artifact ⇒ one key prefix ⇒ one writer. Every other workflow that needs it does actions/cache/restore under that exact key, never actions/cache (which saves). check-cache-key-sync now polices BOTH directions: a prefix's hashFiles() inputs must agree (key drift), and a path: may be WRITTEN (actions/cache or actions/cache/save) under only ONE literal key prefix, with every restore-only prefix having a writer somewhere (an orphan restore can never hit). On its first run it found two more instances the eye had missed — the native verdict store (native-validate-* vs native-verdicts-ci-*, same path) and the cargo registry (cargo-release-* vs cargo-registry-*). Lineage goes in the key SUFFIX (native-verdicts-ci-<os>-<cell>-), never in a second prefix. Companion trap in the same pass: "save on main only" is a rule about SIZE, not a rule. It exists so a 1 GB store is not duplicated per branch; applied to the <1 MB content-addressed verdict store it was a bug, because main's push run is CANCELLED whenever a newer merge queues behind it (cancel-in-progress: false keeps one running + one pending and cancels the rest — 27 of the last 60 main runs), so the store went unsaved for hours while every PR restored a stale prefix and re-derived the same verdicts. Save small, content-addressed, poison-proof stores from every run; reserve main-only for what is actually big. Reference: ci.yml Install job (the one writer) + the native-verdicts save condition; the eviction cascade is measured in [[project_ci_deep_audit_2026_09]].
A gate that takes seconds as its own JOB — under a slot cap, a job is a queue entry every other PR pays for
(2026-09). The org is on the free plan: 20 concurrent jobs ORG-WIDE. Measured on an ordinary code PR (run 33544139801): 29 jobs, 24 min of work, 641 min of summed queue wait, 90 min wall; the four matrix AGGREGATORS (Test/Typecheck/E2E/Scaffold Smoke) each waited ~50 min for a runner to execute 2 s of shell; a docs-only PR waited 48 min for 5 min of work; and ~17 singleton gate jobs (20-45 s of work each) plus 6 standalone single-job workflows were, per PR, ~23 slot acquisitions for under 10 min of total work. Every one of them was written in good faith with a queue-time argument ("no needs, starts at t=0", "lib-free jobs shouldn't wait for the build") that is TRUE while runners are free and INVERTED once they are not: under contention, wall-clock is governed by DAG LEVELS (each a full queue round) and org throughput by JOB COUNT. Rule: a gate that takes seconds is a STEP of an existing job in the same tier (lib-free → Fast Gates, lib-needing → Build); a matrix needs ONE aggregator, not one per matrix; a new JOB needs a written reason it cannot be a step (a different runner OS, a genuinely different toolchain lifecycle, a required name that must report independently). Two consequences worth stating: (a) a step-based gate must carry if: ${{ !cancelled() && steps.setup.outcome == 'success' }} so one push reports EVERY red gate rather than the first — otherwise consolidation trades queue time for round-trips; (b) folding a gate into a required job changes only its CHECK NAME, so the protection edit is a DROP of the old names from BOTH authorities (classic + ruleset) — never an ADD, which hangs every open PR on "Expected". The consolidation this entry records: 29 ci.yml jobs + 6 standalone workflows → 13 jobs + 1 (pr-gates.yml, kept separate ONLY because its two gates are label-sensitive and ci.yml's trigger must not include labeled). Reference: ci.yml header job map; the measurement is [[project_ci_deep_audit_2026_09]].
A type that exists ONLY in a validation stub — the emit compiles against it and references nothing real
(the <Audio> instance, 2026-08). Stubs are what let the Swift/Kotlin gates run without an Apple/Android SDK, and they are therefore the one place where DECLARING something hides its absence. <Audio> emitted PyreonAudioPlayer(url:…, engine: AVFoundationAudioEngine()) on iOS and Media3AudioEngine(…) on Android; all three names lived only in swift-stubs.ts / kotlin-stubs.ts, so the primitive had never compiled on either platform while every gate stayed green for its whole life. Two checks, both needed: (1) every OWNED type a stub declares AND an emitter emits must be declared in that language's REAL runtime — paired BY LANGUAGE, since a combined corpus finds Swift's PyreonAudioPlayer for the Kotlin check and passes; (2) compile a probe against the real SDK WITH the real runtime sources linked in, which additionally catches a wrong signature, an unaccepted modifier, an availability mismatch — none of which a name check can see. The mirror failure is a stub NARROWER than the runtime, which manufactures a phantom bug in correct codegen; both are stub-fidelity defects, one quiet and one loud. Reference: packages/native/compiler/src/tests/{emitted-runtime-types-exist,real-runtime-typecheck}.test.ts.
A primitive/feature no example uses is one no gate ever compiles
(same instance). The device gates are the only configuration without stubs, and they build examples — so a capability absent from every example is verified by nothing, however many gates exist. <Audio> was the single canonical primitive no native example used, and the single one that had never built. Ask the coverage question directly rather than leaving it a property nobody tracks (scripts/check-native-primitive-coverage.ts), and derive the list from the package's own exports — the first cut keyed on the compiler's SWIFT_NAMES map, which omits three primitives with dedicated emitters, and so reported "all 16 covered" while the one that motivated it was not in the set.
A probe that cannot fail, and a diagnostic truncated at its tail
(2026-08, two shapes of the same waste). A reactivity sweep grepped the emit for the signal's NAME to prove a prop stayed live — but the name always appears, because the signal is declared, so the probe could only ever report "clean". A device-gate timeout message put its most decisive field (the tree BEFORE the action) last, after a multi-line dump, and the runner cut the message at the dump — a ~30-minute round producing a diagnostic that stopped one line short of its own answer. Give every sweep a POSITIVE CONTROL — a case it must report — and order a failure message by DECISIVENESS, not narrative, because it competes for a truncation budget. Same family as "verify the harness before trusting its result": a surprising-in-a-convenient-direction result is the tell.
STACK_TRACE_ERROR with no assertion text has a SECOND mechanism — a per-test TIMEOUT under load — and the gate's own diagnostic names only OOM
(2026-09, @pyreon/loom on Coverage (Full)). The check-coverage message asserted "no text ⇒ the worker died ⇒ OOM under 4-way parallelism; do NOT re-run past it", and it was wrong: the whole loom package peaks at ~630 MB under coverage, while the whole-repo scan spec (strip-equivalence.test.ts, "agrees on every source file in this repo") measured 3.6 s on one run and 26.4 s on another of the SAME tree — a ~7× inflation under the job's 4-way package parallelism that crossed the shared testTimeout: 20_000. A vitest TIMEOUT rendered through the JSON reporter drops its text exactly as a dead worker does, so the symptom is identical and the gate's heuristic sent the investigation down the memory path first. Main went red on two consecutive runs (a third was still queued) and it looked like the shard-split PR that merged at the same minute; CI retry: 2 cannot help because every retry runs under the same sustained load. Rule: on a blank-text STACK_TRACE_ERROR, discriminate by BOTH peak RSS and duration-vs-effective-timeout; and a spec whose cost is repo-size-bound must carry an explicit DERIVED timeout with the measured pass/fail durations in its comment, never the shared default. Prove the option is honored by forcing 1 ("Test timed out in 1ms") — the structural derivation is the bisect, since the load failure itself is not reproducible on demand. Reference: strip-equivalence.test.ts:WHOLE_REPO_SCAN_TIMEOUT_MS; the gate text now names both mechanisms.
A tool that shells out to vite build INHERITS the caller's NODE_ENV, and Vite only sets it when UNSET — so a stray value silently yields a non-production bundle
(the loom build instance, 2026-08). vite build does NODE_ENV ||= 'production', and Vite derives isProduction from that VARIABLE — not from mode, so passing mode: 'production' does nothing (verified: still 3894 MB, dev branches intact). Any caller with NODE_ENV already set — development in a dev shell, test under ANY test runner — therefore got a build with every process.env.NODE_ENV !== 'production' branch in the graph retained: dev-only warnings shipped to users, and 3894 MB of build memory against 952 MB. The rule: a purpose-built "emit the production artifact" command must FORCE NODE_ENV=production (and restore the caller's value in a finally — including the was-unset case, or you mutate the caller's environment). Vite's ||= is correct for a general-purpose command where NODE_ENV=staging may steer a user's config; it is wrong wherever nothing can read the value — here the build runs configFile: false, so no user config is loaded at all, and there is no dev variant of the command. The test-side half is the sharper lesson: vitest sets NODE_ENV=test, so ANY build a test runs — in-process or spawned — is a non-production build by default. A suite that asserts on build output is then asserting on an artifact no consumer ever gets, which is exactly what this suite (whose whole reason for existing is that a green build once emitted a 356-byte shell page) had been doing. Do NOT fix that by sanitising the child's env: that MASKS a regression of the product fix instead of catching it — spawn with the hostile NODE_ENV inherited and assert the OUTPUT is production (a dev-only warning string absent from the emitted JS is a clean binary discriminator; minification is not — Vite minifies in both modes). Diagnostic trap that cost the most time here: the failure named an innocent test. 3.9 GB sits just under node's ~4 GB old-space cap, so under Coverage (Full) (4 packages in parallel) the worker died, and vitest attributes a dead worker to whichever spec was in flight — reported as Error: STACK_TRACE_ERROR against strip-equivalence, the longest-running spec in the package. STACK_TRACE_ERROR with no assertion text means a dead worker, not a failed assertion: attribute by measuring peak RSS per test FILE, never by reading the name in the report. Reference: packages/tools/loom/src/build/static-site.ts (forced + restored) + tests/static-site.test.ts (spawns the shipped bin, inherits NODE_ENV=test, asserts production output); bisect-verified both directions (remove the override → rebuild lib/ → the production spec fails and peak returns to 3992 MB).
A published CLI bin that relies on import.meta.main (or a bundler-strippable top-level guard) to invoke — a SILENT no-op for consumers, invisible to the monorepo
a bin that is a no-op still builds, ships, and exits 0 — so nothing flags it except a real npx <tool> run. Two shipped instances of the class (both in the 0.43.x published tarballs): (1) @pyreon/lint's bin/pyreon-lint.js did a bare import('../lib/cli.js') and relied on if (import.meta.main) main() inside cli.ts — but the bundler treats lib/cli.js as a LIBRARY entry (it has exports) and TREE-SHOOK the guarded top-level call away, so the shipped lib/cli.js is a pure re-export and npx pyreon-lint ran NOTHING (dead on every runtime); (2) @pyreon/mcp's stdio server was gated on import.meta.main alone, which Node only defines from v24.2 (undefined on 20/22 LTS) → npx pyreon-mcp under Node LTS started nothing (it worked under bunx, which masked it). Why the monorepo never caught it: dev runs bun src/cli.ts (where import.meta.main IS true), and the internal gate calls the exported entry (runCli()) PROGRAMMATICALLY — neither path exercises the published bin. Fix shape: a hand-written bin/<name>.js (shipped as-is, NEVER bundled) that imports the entry function and calls it EXPLICITLY (import { runCli } from '../lib/cli.js'; const code = runCli(process.argv.slice(2)); if (code !== null) process.exit(code)) — runtime-agnostic, bundler-proof, mirrors @pyreon/cli's unconditional main().catch(). When a bin IS the bundled lib entry (mcp) and must NOT self-run on import (tests import it), use a CROSS-RUNTIME entry check, not bare import.meta.main: a pure matchesProcessEntry(meta, moduleUrl, resolvedEntryUrl) that trusts the boolean when present, else compares the resolved process-entry URL (pathToFileURL(realpathSync(process.argv[1])), symlink-resolved for .bin) to import.meta.url — the pure matcher makes the LTS undefined-meta path unit-testable without an old Node. Prevention gate (scripts/check-bin-liveness.ts, CI job "Check Bin Liveness", needs bootstrap): spawns every published bin under REAL Node (NOT Bun — the failure is Node-runtime-specific; running under Bun masks exactly the bug) and fails naming any bin that does nothing — --version/--help must produce non-empty stdout; a stdio server (mcp) must answer a JSON-RPC initialize handshake. Bisect-verified: reverting the lint bin → the gate reports ✗ exit 0 but EMPTY stdout, exit 1. General rule: import.meta.main is a DEV convenience (Bun / Node ≥24.2), never the invocation trigger for a PUBLISHED bin — and any bin-liveness claim must be verified against the published tarball under Node, not bun src/….
A changes/decide job gating required checks with if: needs.changes.outputs.code == 'true' is FAIL-OPEN
when an affected-detection ("changes") job feeds a code output that gates the heavy required checks (Build / Verify Modes / Test (browser) / Audit Types / …), the naive if: needs.changes.outputs.code == 'true' is fail-open two ways. (1) If the detect STEP misbehaves (affected.ts non-zero exit / empty stdout), code is empty → == 'true' is false → every heavy job SKIPS → a skipped required check reports success to branch protection → a real regression PR is mergeable with all heavy checks green-via-skip. (2) If the changes JOB itself fails (infra flake / timeout-minutes trip), the output is empty AND the dependents auto-skip → same fail-open. Fixes (BOTH needed): (a) STEP-level — default the output fail-closed: CODE=$(affected.ts …) || CODE=true; if [ "$CODE" != "false" ]; then CODE=true; fi (only a clean false narrows to docs-only; anything else runs the full set). (b) JOB-level — make the gate run when the decide job didn't succeed, WITHOUT losing the skip-on-real-dep-failure optimization, via status functions: if: ${{ (success() && needs.changes.outputs.code == 'true') || (failure() && needs.changes.result != 'success') }}. This runs on a changes crash (fail-closed) but still skips when lint/typecheck/bootstrap failed (those already block via their own required checks — don't waste compute). A plain !cancelled() && (code != 'false') is WRONG: it force-runs every heavy job on every lint-failing PR. Reference: .github/workflows/ci.yml (the changes job + the 12 gated heavy jobs); the proven precedent is the E2E aggregator's if: always() + exit 1 when e2e-decide != success.
A path-scoped workflow promoted to a required check — two deadlock/laundering traps
(1) a TRIGGER-level paths: filter is deadlock-UNSAFE for required checks — a filtered-out PR creates NO check run, so branch protection waits on "Expected" forever; path scoping must move to a JOB-level skip via an always-running changes decide job (a skipped job satisfies a required check). (2) once labeled is in the trigger types, every label addition creates a NEW workflow run on the same head SHA, and the LATEST check run per name is what branch protection reads — so if the job gate skips builds on unrelated-label events, a RED required run can be LAUNDERED into a skipped-green latest check by adding any label. The path-detect arm must therefore fire on EVERY PR action including labeled (re-running the matrix on a rare label addition is the safe side; cancel-in-progress dedups). Both traps ship together in any "promote an opt-in device/e2e workflow to required" change. Reference: .github/workflows/native-device.yml (the changes job + the laundering comment on ios-build); the fail-closed dependent-gate idiom is the "changes/decide job gating required checks is FAIL-OPEN" entry above.
Classifying a doc-INPUT file as "docs-only" hides it from the package whose tests PARSE it
an affected-detection that treats .claude/** / docs/** / *.md as docs-only (so build/verify-modes skip — correct, they don't read them) ALSO makes those paths seed no workspace in the test-cell matrix, so the package whose tests ASSERT their structure never runs. E.g. @pyreon/mcp's anti-patterns.test.ts / patterns.test.ts parse .claude/rules/anti-patterns.md + docs/patterns/*.md; a reorg that breaks the parser merges unnoticed because the test (tools) cell skipped mcp. Fix: map doc-INPUT files to their consuming package as a LEAF seed in the test-affected computation (like scripts/** → @pyreon/test-utils), so the consuming cell runs — WITHOUT reclassifying them as code for the heavy-job gate (build still correctly skips). The seed MUST be ADDITIVE, not an else-only fallback (if findOwningWorkspace is null): docs/patterns/** is already OWNED by the @pyreon/docs workspace, so a fallback branch never reaches @pyreon/mcp and its patterns.test.ts silently stops running on a pattern-doc reorg. Run the consumer check for EVERY path regardless of whether it also owns a workspace (both seeds land). Test-fixture trap: a synthetic workspace fixture that OMITS the owning package (@pyreon/docs) makes the fallback form pass — the shadowing only appears when the owner is present, so the regression test must include it. Two signals, not one, gate the two job classes: code (!isDocsOnly) gates the read-nothing heavy jobs (they skip on a doc-input change — nothing to build); a separate affected (--has-affected, this computation non-empty) gates bootstrap + the test/typecheck cells (they run on a doc-input change so the parser test fires). They diverge exactly on a doc-input change (code=false, affected=true). Reference: scripts/affected.ts:DOC_INPUT_CONSUMERS / docInputConsumer (additive leaf seed) + --has-affected.
A path-scoped decide job whose regex matches a directory NAME rather than the surface it means — and a cache added to one lane of a twin
(native lanes, 2026-09-07). Both native decides matched packages/<cat>/<pkg>/native/ (co-located Swift/Kotlin) — which is ALSO packages/core/compiler/native/, the JSX compiler's napi-rs crate — and matched every package.json, so a dual-backend compiler PR or a dependency bump ran ~2h of macOS/iOS/Android lanes for nothing (6 of 40 PRs). Green lanes hide their own cost; only queue depth shows it. Meanwhile the macOS real-SDK typecheck lane re-ran every swiftc verdict cold (36–58 min) because the verdict cache had been wired into its Linux twin only. Rules: (1) a name-based path rule must be checked against ls -d of every sibling that reuses the name; (2) a decide classifier belongs in ONE unit-tested script both workflows call, fail-closed (any uncertainty RUNS); (3) when a cache is added to a lane, grep for the lane's twins. Reference: scripts/native-surface-touched.ts + test-utils/src/tests/native-surface-touched.test.ts; the macOS verdict-cache steps in native-validate.yml.
A gate that MEASURES must validate its own measurement — a bundle that shook to nothing is not a small number, it is no number
(the bundle-budget instance, 2026-09): check-bundle-budgets bundled each package's built entry and compared the bytes to a budget. When lib/index.js is a pure re-export barrel (import { t as useHead } from './_chunks/…' + an export clause, the shape rolldown emits) and the manifest says sideEffects: false, the bundler drops the imported bindings as unused and emits export { t as HeadContext } with no t in scope — an un-importable module (SyntaxError: Export 't' is not defined) weighing a couple hundred bytes. Seven packages were guarded by fiction: @pyreon/lint had a 512 B budget over a package that really measures 65,544 B (128×), @pyreon/lathe 512 B vs 25,741 (50×), @pyreon/charts 256 vs 2,779, @pyreon/loom 6,144 vs 7,202, @pyreon/head 256 vs 1,699, @pyreon/meta 1,536 vs 2,361, @pyreon/storybook 256 vs 386. loom is the instructive one: it was seeded at 6,144 when it measured correctly and silently drifted to 298 B when its build grew _chunks — the class REGRESSES a healthy package, and the gate stays green through the transition. sideEffects: false is necessary but NOT sufficient (many healthy packages declare it; @pyreon/reactivity has the same chunk layout and measures correctly because its entry carries real code of its own) — the second condition is that the entry's body is a PURE barrel, with nothing referencing the imported bindings except the export clause. Two detectors, and they are not the same kind of thing. (1) A PROOF: every export specifier must resolve to a local binding; a module that cannot be instantiated cannot be what a consumer ships, so there is no threshold to argue about. (2) A HEURISTIC backstop: the bundle's raw bytes against the bytes its entry statically reaches over RELATIVE imports (dynamic import() is the lazy boundary and is deliberately not followed), floored at 5% — measured across all 71 budgeted packages the lowest LEGITIMATE ratio is 0.2004, so ~4× margin. Bisect-verified that neither substitutes for the other: with the proof neutered, the ratio catches 5 of 7 and MISSES @pyreon/meta (ratio 0.5374 — a perfectly healthy-looking number for a bundle that cannot be imported) and @pyreon/storybook, reporting both as real sizes with zero violations; with the ratio neutered, a valid but gutted bundle (a side-effect import dropped under sideEffects: false — the registration-seam hazard) reports 35 B against 35 KB of reach and passes. Telling "shook away" from "genuinely small" needs no exemption list: the ratio is SCALE-FREE — a package is compared against ITS OWN reachable source — so @pyreon/config (a single 973-byte module measuring 237 B ⇒ 0.24) and a re-export-only package over externalised deps both pass for the same reason a large package does. Replacing it with a naive absolute floor fails the legitimately-small spec, which is the bisect that proves the scale-free choice is load-bearing rather than decorative. The repair is applied on PROOF of breakage, never as a blanket change: re-measuring through a barrel-safe export * from <entry> wrapper is exact (0.0–0.5%) for 64 of 71 packages but collapses an entry with no named exports (@pyreon/zero-cli, a side-effect-only CLI script) or one whose only export is default under sideEffects: false (@pyreon/create-zero) to 28 B — so measuring everything that way would have traded seven wrong numbers for two new ones, and re-baselined 64 budgets nobody reviewed. General rule: any gate that derives a NUMBER must be able to say the number is invalid. A measurement gate that cannot distinguish "measured 200 bytes" from "measured nothing" reports the second as the first, and a budget compared against nothing can never fail — which this repo already treats as worse than having no gate. Reference: scripts/check-bundle-budgets.ts:diagnoseMeasurement; locked by packages/internals/test-utils/src/tests/check-bundle-budgets-failures.test.ts (barrel / gutted / legitimately-small fixtures, PYREON_BUDGETS_NO_REPAIR=1 exercises the detector with the repair off — an auto-repair that is never tested against a broken bundle is a detector nobody has proof of).
A budget with less headroom than the measurement's own noise is UN-SATISFIABLE, not strict
(the bundle-budget sibling, 2026-09): gzip output differs between the machine a contributor measures on and the ubuntu runner that gates the PR — ~1.1% measured on a 16.5 KB package — so a budget set closer than that gets a DIFFERENT VERDICT PER MACHINE: validate-fast green locally, CI red, and re-running locally only reconfirms the wrong answer. Three PRs paid a CI round trip to this in a single day (@pyreon/compiler, @pyreon/create-multiplatform, @pyreon/native-cli), each on real intended growth. The gate already KNEW — it printed "that is within the ~1.5% variance — this budget has too little headroom to be measured reliably" — but only as advice attached to a failure it had ALREADY DECLARED, which is the wrong moment: by then the check is red and the round trip is spent. A diagnosis a gate can compute is worth nothing if it is only emitted after the verdict. Auditing all 71 entries found FOUR below the band, including @pyreon/url-state (50 B of headroom against a 64 B band) that nobody had noticed. Fixed by checking headroom on EVERY run, before any verdict, and reporting the exact remediation (raise to N B to retire it) rather than a description of the problem. Gated as a RATCHET, and the distinction is the load-bearing part: a thin budget is a defect in the BUDGET FILE, not in the package — so failing on the ones that already exist reddens every unrelated PR (the red-on-arrival shape that makes a gate dead), while what must actually be prevented is CREATING a new one. So an entry that is thin and NOT grandfathered in _thinHeadroom fails; listed ones warn loudly on every run, green or red, because a list consulted only on failure is a list nobody reads; and an entry that recovers is named for removal, so the list can only shrink. Each exemption carries a mandatory reason naming what retires it (the loom ignore convention — an exemption nobody can retire is permanent debt). General rule: any gate that compares a MEASUREMENT to a THRESHOLD must also police the DISTANCE between them — when that distance is below the measurement's own variance, the gate is not being strict, it is flipping a coin, and it will spend other people's CI time proving it. Note tight is not itself bad — a small budget makes the gate MORE sensitive to real growth, which is the point; the failure mode is specifically headroom below the noise. Reference: scripts/check-bundle-budgets.ts:requiredHeadroom (one constant shared with the existing failure note, so the two can never disagree); locked by the un-satisfiable budgets block in check-bundle-budgets-failures.test.ts (bisect-verified: neutering the guard passes a 1-byte-headroom budget silently; ignoring the grandfather list reddens the known entries).
A safety-net gate red-on-arrival is a DEAD gate
Coverage (Full) (push:main + merge_group) ran red on every completed main run for weeks — 12 packages sat below their configured vitest thresholds, so the gate could not distinguish "new regression" from "arrived red"; its stated purpose ("main never regresses") was structurally unmet. Rule: a threshold-style gate's thresholds must sit AT/BELOW measured reality at ALL times — ratchet UP as tests land (the lint-baseline.json discipline), never leave aspiration baked into the gate. When coverage drifts because a feature wave's tests live in another tier, the fix is per-package and honest: browser-covered files get coverageExclude (whole file) or targeted /* v8 ignore */ (mixed file) with a rationale naming the covering browser test — the @pyreon/a11y skip-link instance: skip-link.tsx was covered ONLY by skip-link.browser.test.tsx, dragging the node gate to 69% (its sibling visually-hidden.tsx had the documented exclusion; the new file didn't) — never exclude a file with NO coverage anywhere; cheap real gaps get genuine tests; the rest is an explicit threshold re-baseline documented as debt (BELOW_FLOOR_EXEMPTIONS entry when below floor). 2026-07 restoration: 15 packages triaged (a11y/store/primitives/server/lint/styler/unistyle/testing via exclusion+tests, cli/compiler/router/runtime-dom/validate/form re-baselined; testing ALSO had NO explicit thresholds — the gate assumed 95 while vitest enforced the 80/75 category default, a silent divergence to check when adding a package). Reference: scripts/check-coverage.ts + the coverage-full comment block in .github/workflows/ci.yml.
A gate's --update that RE-LOCKS every entry from the current measurement can only ratchet DOWN on a bad build — and the pure-helper unit test passes while it does
(two sibling gates, 2026-09). A budget is derived from a BUILD, so a stale or partial lib/ measures SMALLER than the real package: an unscoped relock therefore turns one wrong measurement into a committed budget BELOW what CI measures, and the gate then reds on a package nobody touched. Observed: @pyreon/validate went 15872 → 15360 (implying a ~12288 B measurement for a package that really measures 15330 locally / 15473 on CI) and travelled to two branches. Refuse by DIRECTION, not by drop size — a size threshold cannot tell a stale build from a genuinely loose budget, and this repo has plenty of the latter (loom measures 298 B against 6144, testing 1745 against 5120, both identical before and after a full rebuild), so a threshold refuses legitimate tightening while still missing a small stale drop. Raises always apply (that is the reviewed case); lowering requires NAMING the target (--update=@pyreon/pkg), which is the review signal. Three sub-traps, each of which cost a round. (a) args.includes('--update') is an exact match, so the documented scoped form --update=X never enabled update mode at all — latent for the flag's whole life, and only found by running the command end-to-end; two unit specs on the pure helper passed throughout. (b) Fixing one gate is folklore — the sibling check-import-budgets had the identical defect, measured live at 8 of 11 budgets lowered in a single unscoped run (router::basic −9.9%) while raising 3, so a person relocking for the three legitimate raises silently committed eight unreviewed tightenings. Measure the neighbour, do not read it. (c) A unit test on the policy helper cannot see whether the policy is CONSULTED — extract the decision as a pure function over (measured, previous, scope) and test the WIRING, with control specs (raise / seed / equal) that must stay green, or a rule that reports nothing passes every structural test. Reference: scripts/bundle-budget-policy.ts (one rule, both gates) + check-import-budgets.ts:relockBudgets; locks bundle-budget-drop-guard.test.ts + import-budget-relock.test.ts, both bisect-verified.
A test importing a root scripts/*.ts drags that script's WHOLE type surface into a program that may not have @types/bun
(2026-09). @pyreon/test-utils extends @pyreon/tsconfig/internal.json, whose types are ["vitest/globals", "node"] — no bun. So a test that imports a gate to unit-test its pure helpers fails typecheck on the gate's Bun.build (TS2868) plus every parameter whose type was inferred from it (TS7006), and on import.meta.dir (TS2339 — the Bun-only spelling; the STANDARD property is import.meta.dirname, which is what affected.ts and check-coverage.ts use and what works under vitest). The repo has TWO established answers and they are complementary, not alternatives: extract the pure policy into a Bun-free module (scripts/is-entry.ts, scripts/test-paths.ts, scripts/bundle-budget-policy.ts) — right when more than one consumer needs the rule; and declare a minimal local ambient declare const Bun { build(...) } (check-import-budgets.ts, serve-ssg.ts) — right so the file stays importable at all. Do both: the module is the home for shared policy, the ambient stops the next importer hitting the wall. The detection lesson is the sharp one: neither bun run test nor validate-fast typechecks, so this class is invisible to both — it is caught ONLY by the affected-typecheck leg of the pre-push hook, which is exactly what a PYREON_SKIP_PRE_PUSH=1 push skips. A grep for Bun\. is also not evidence here: three scripts carry that string only in COMMENTS documenting this very trap.
Silent-filter on aggregate CI gates
a gate that runs op(item) per package / file and aggregates with results.filter(r => !r.failed) silently hides the failure mode it was supposed to catch. The failed items disappear from the measured set without ever surfacing in the gate's output. THIRD instance (2026-07, upstream-reported): pyreon doctor's file-scanning gates hardcoded the Pyreon repo's own packages/<cat>/<pkg>/src shape, so in ANY foreign workspace (even single-level packages/*, let alone apps/*+modules/*) they scanned ZERO files and the aggregate still reported 100/100 Grade A — the empty INPUT SET is the same class as the filtered result set. Fix: one workspace-roots resolver (the repo's own workspaces globs) + scanned: 0 → skipped-with-warning (meta.emptyScan) + report.measured (nothing measured → score renders —, --ci fails). Rule: a gate must fail loudly when its input set is EMPTY, not only when items fail. Pattern surfaced TWICE before in close succession: PR #434 (bundle-budgets reported "All 49 within budget" while 5 packages silently failed Bun.build); PR #435 (bootstrap swallowed postinstall build failures, leaving partial lib/ state). Both gates were doing the right per-item work; both filters quietly threw away the diagnostic value. Defensive design rule for any aggregate gate: (1) include a failures: [{name, error}] (or equivalent) field in machine-readable output, (2) print failures to stderr in human-readable mode, (3) exit non-zero on any failure regardless of the postinstall / "transient flake" rationale. Aborting the operation loudly is always better than continuing with hidden partial state — silent partial state misleads users into thinking work succeeded, and the symptom surfaces hours later far from the cause. Bisect-verify shape: revert the failure-surfacing change → assert the measured count drops AND no failures field appears → restore → assert the count returns AND failures: []. If both assertions don't fail in the bisected state, the regression test isn't load-bearing. FOURTH instance (2026-08, Coverage (Full)): check-coverage.ts ran each package and did if (result) {…} else console.log(' (skipped)') — a package whose run produced no parseable output was printed once mid-run and then dropped from the results table, the exit code, and CI. Three packages (@pyreon/zero, @pyreon/mcp, @pyreon/vite-plugin) were absent from EVERY CI coverage table, so their thresholds had never been enforced at all — and two BELOW_FLOOR_EXEMPTIONS entries even documented the shortfall, attributing it to a timeout, which is how a hole gets written down and still not fixed. Two sub-lessons specific to measurement gates. (1) "Measured nothing" is not "measured 0%". @pyreon/config reported 0% statements (need 95%) while nine tests passed and every line of its logic ran: its whole implementation is src/index.ts, which the shared vitest config excludes as a re-export barrel, so ZERO files reached the instrumenter (Statements: Unknown% ( 0/0 )). The number sent a reader to write tests that already existed. A gate that reports a coverage figure must distinguish the empty INPUT (0/0) from a genuine zero (0/500) and name the likely cause — here includeIndexInCoverage: true, the same trap @pyreon/store hit in #2167 and @pyreon/runtime-server before it, i.e. the third recurrence. (2) Do not parse a human-facing summary TABLE when a stable summary BLOCK exists. The parser keyed on the All files | … row, which the v8 reporter OMITS for a single-file package — so fixing @pyreon/config's instrumentation would have moved it from "wrongly reported 0%" straight to "silently skipped". The Coverage summary block is always present and carries the ratio, which is the only thing that makes sub-lesson (1) decidable. Reference: scripts/check-coverage.ts:parseCoverageOutput + packages/internals/test-utils/src/tests/check-coverage-parse.test.ts (bisect-verified against the real captured output: old regex → NO MATCH → skipped; new → 100%).
core.hooksPath install treating "set to git default location" as a user override
any non-empty core.hooksPath value used to be treated as a real user override (husky / lefthook / custom path) — but if the value happens to resolve to git's default <git-dir>/hooks location (e.g. set explicitly by an older bootstrap run, or by a user who didn't realise it matched the default), the installer bailed and Pyreon's hook at .githooks/pre-push was orphaned. Result: every git push SHOULD have run lint + typecheck + tests on affected packages, but git was looking at the empty .git/hooks/ instead. Fix: distinguish "genuine custom path" from "set to default location" by comparing current against the candidate default paths (<git-dir>/hooks AND <git-common-dir>/hooks for worktrees). Reference: scripts/install-git-hooks.ts:getDefaultHooksPaths. Worktree gotcha: core.hooksPath is repo-shared (not per-worktree), but git rev-parse --git-dir returns the per-worktree git dir. Accept both <git-dir>/hooks and <git-common-dir>/hooks as default locations.
Subprocess testing as a default for shell scripts
when a Pyreon scripts/*.ts file ships with significant policy logic, tests should call the policy as a pure function (exported) rather than spawnSync-fork the script and assert on captured output. Two failure modes this rule prevents:
Subprocess output capture under parallel load is non-deterministic. Tests fail with
expected '' to contain 'X'—console.log/console.warnran correctly but the output was lost between fork and parent capture due to pipe-buffering races under heavy concurrent I/O (bun run --filter='*' testagainst 60+ packages, what the pre-push hook computes for any cross-package PR).GIT_* env-var leak inside git hooks. When the script (or its test fixture) runs
execSync('git ...', { cwd: tempDir })from inside a git pre-push / pre-commit hook, git setsGIT_DIR/GIT_WORK_TREE/GIT_INDEX_FILEetc. in the env to point at the OUTER hook's repo. Those env vars OVERRIDE both thecwdoption andgit -C <dir>— git uses GIT_DIR if set, regardless of where it was invoked. The originalinstall-git-hooks.test.tsliterally corrupted the running worktree's git config becauseexecSync('git config user.email test@test.local', { cwd: tempDir })saw GIT_DIR and wrote to the worktree's.git/configinstead of the temp dir's. Symptoms: tests fail withexpected { kind: 'no-hooks-dir' } to deeply equal { kind: 'not-a-git-repo' }(test temp dir's.gitwas never created —git initno-op'd against GIT_DIR), orexpected { kind: 'already-configured' } to deeply equal { kind: 'installed' }(script reads OUTER repo'score.hooksPath).
Fix shape:
Export
function doThing(cwd: string): StructuredResultfrom the script (cwd injected as parameter, noprocess.cwd()calls inside policy);main()translates the result to console output for the bin path.Inside the script's git invocation helper, EXPLICITLY clear every
GIT_*env var beforeexecSync(Object.keys(env).filter(k => k.startsWith('GIT_'))). Don't rely oncwdor-Calone — the env vars override both.Test fixtures must do the same for THEIR git invocations (
git init,git config user.email,git config --get) since those bypass the script's helper. AcleanGitEnv()helper in the test that returns{ ...process.env, [GIT_*]: removed }and gets passed asenvto everyexecSyncis the canonical shape.Use
git -C <dir>(git's native working-dir flag) AS WELL AS the env-clearing — defense in depth.Tests call the policy function directly and assert on the discriminated-union return shape. NO subprocess fork for the assertion path. Keep ONE thin smoke test that spawns the actual binary and asserts only on
result.status === 0(NOT on captured output).
Reference: scripts/install-git-hooks.ts:runGit (env-clearing + -C + cwd, all three) + packages/internals/test-utils/src/tests/install-git-hooks.test.ts:cleanGitEnv (post-fix). The shipped fix took TWO iterations: first replacing subprocess with direct call (closed failure mode 1 — load-dependent), then env-clearing (closed failure mode 2 — only surfaced in the real pre-push run, not in bun run --filter='*' test standalone, because standalone doesn't set GIT_*).
CODE_SIGNING_ALLOWED=NO on a simulator app that touches the KEYCHAIN — securityd denies SecItemAdd for unsigned apps
(the router-demo secure-storage CI failure, 2026-07; three fix rounds before the real cause). An UNSIGNED app carries no signature entitlements, and the iOS 18.5 simulator's securityd rejects its keychain writes — while every LOCAL build (ad-hoc signed by default) passes, the classic local-pass/CI-fail split. Two rounds of correct-but-insufficient fixes (keychain-access-groups entitlement + kSecUseDataProtectionKeychain) could not matter: the CI invocation stripped the signature that carries entitlements. Diagnosis pattern that collapsed it: make the failure message read the LIVE UI state (the app renders write-failed/read-failed/the value) — CI answered write-failed in one round, isolating the write path; then the flag was the only local-vs-CI divergence left, and adding it to a LOCAL build reproduced the failure byte-identically (the full bisect: unsigned → denied, ad-hoc → passes). Rules: (1) never pass CODE_SIGNING_ALLOWED=NO for a simulator app that uses Keychain/app-groups/any entitlement-gated service — ad-hoc signing (codesign -s -) costs nothing on a bare runner; (2) entitlements have TWO carriers on simulators (the binary's __entitlements section AND the signature) — codesign -d --entitlements reading empty on a sim app is checking one carrier, not proof of absence; xcodegen's entitlements: block alone does NOT embed under ad-hoc signing, the explicit CODE_SIGN_ENTITLEMENTS build setting does. Reference: .github/workflows/native-device.yml (router-demo step comment) + examples/native-router-demo-ios/project.yml.
A verifier that cannot run its behaviour half must SAY SO — a typecheck-only pass rendered as the same ✓ is a dead gate that looks alive
(the Kotlin co-source gate, 2026-09). verify-kotlin compiles a runtime file + its smoke main() against stubs and then RUNS the smoke — unless java is off PATH, in which case it printed skipping smoke-run (typecheck passed) and its caller rendered the identical ✓ … (1 file(s)). On a developer laptop with Homebrew's JDK not on PATH that is the normal state, so every Kotlin behaviour test in the repo was typecheck-only locally while reading as executed; the @pyreon/flow port shipped a selectAll divergence past exactly that line, and it was found only because the Swift twin (which does execute) failed the same sequence. The audit that found it also found the SWIFT half's skip message on the ubuntu job naming a macOS job that never invoked the script, and both native workflows' path filters excluding packages/*/*/native/** — three different ways for a gate to be green because it did not run. Rules: (1) a skipped half prints a marker that cannot be confused with a pass (⚠ SKIPPED … put a JDK on PATH), never a ✓; (2) a skip message that names another job as the place the check runs must be TRUE — grep that job for the invocation; (3) a path filter for a workflow that verifies co-located sources must include those sources. Harness-check any gate before trusting it: inject a failing assertion and confirm the gate reds with that exact message. Reference: packages/native/runtime-kotlin/scripts/verify-kotlin.ts, scripts/check-native-cosource.ts, .github/workflows/native-{device,validate}.yml.
A gate whose scan surface excludes the very files its rules are ABOUT — the structurally-dead rule
(three instances found together in @pyreon/lint, 2026-08). pyreon doctor's lint gate scans each package's shipped src/** minus tests, fixtures and .d.ts — right for almost every rule, and the exclusions exist because detector fixtures hold anti-patterns deliberately. But two rules' SUBJECT is exactly what that scan removes: no-query-selector-cast-in-test (about test files) and vitest-config-uses-shared (about package-root configs). Both were configured error in .pyreonlintrc.json, both were listed by --list and counted in the docs, and neither could report anything: 2,159 test files and 115 vitest configs existed, and 0 of either were in scope. The gate was green because it was blind. no-query-selector-cast-in-test exists to lock in PR #963's elimination of 122 querySelector(…) as HTMLX sites; 280 had silently re-accumulated across 92 files — the exact regression class the rule was written to prevent, in the presence of a rule that was written to prevent it. Fix: a rule DECLARES its surface (RuleMeta.scanTarget: 'source' | 'test' | 'packageConfig') and the gate collects what the enabled rules need, running each extra target as its OWN pass with every other rule off — running the full set over tests would reintroduce the noise the exclusions exist to prevent. General rule: when a gate narrows its input for a good reason, enumerate which of its checks that narrowing makes UNANSWERABLE. A scan policy and a rule's subject are two independent facts, and a rule that silently depends on a policy it cannot see is dead the moment the policy is right for everyone else. Same family as the per-file Kotlin gate that could not see a cross-file collision. Two sibling silent-holes surfaced in the same pass, both the "hand-maintained subset" shape: (a) exemptPaths was honoured per rule — a rule had to call isPathExempt itself and 55 of 101 did not, so configuring an exemption for one of those parsed, validated, and did nothing; fixed CENTRALLY in the runner's rule loop (skip before rule.create()), so support is a property of the runner, not something each rule opts into. (b) A config key naming nothing was silently ignored — this repo shipped pyreon/dangerously-set-inner-html, complete with an exemptPaths list, for a rule that has never existed; a typo'd id means the rule you meant to disable keeps running and nothing says so. Both now report as config diagnostics with a did-you-mean. Locked by exempt-paths-central.test.ts + unknown-config-keys.test.ts (bisect-verified: reverting the central skip fails the exempted-rule spec with expected [ { …(6) } ] to deeply equal []).