Skip to content

Fix oracle selftest temp-path collision flake in coverage - #1162

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/oracle-selftest-tempdirs
Oct 8, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/oracle-selftest-tempdirs

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The required coverage job failed in two of the last five merge-group runs, and both entries were evicted from the queue:

Both fail with error: 1 target failed: -p socket-patch-cli --test e2e_vendored_production. The production tests in that binary are all #[ignore]d, so the failure comes from the shared helper selftests compiled into it. The 1051 and 1030 groups, built on the same base in the same batch, passed coverage, so the change being merged doesn't cause it; it's a flake.

Root cause

tests/common/mod.rs gets compiled twice into e2e_vendored_production (and into other binaries that pull in vlt_e2e_common). It's included directly as common and again as vlt_e2e_common::common (#[path = "../common/mod.rs"] pub mod common;). As a result, each oracle_selftests test runs twice, at the same time, in one process. Three of those tests built their scratch paths from the PID alone:

  • git_sha256_file_hashes_real_bytes → /tmp/socket-patch-oracle-<pid>-{a,b}.bin
  • write_minimal_manifest_emits_apply_compatible_shape → /tmp/socket-patch-oracle-<pid>-manifest
  • write_blob_stages_exact_bytes_at_hash_path → /tmp/socket-patch-oracle-<pid>-blob

scratch_dir also ran remove_dir_all first. When the two copies ran together, each one deleted or overwrote the other's files.

Fix

Each of these tests now gets its own tempfile::tempdir(). The directory is unique per call and removed on drop, so the hand-rolled scratch_dir helper and the manual cleanup are gone. tempfile is already a dev-dependency. No assertions changed.

Proof

Looped the e2e_vendored_production test binary locally with --test-threads=16:

runs failures
before (origin/main) 300 11: all four collision signatures seen in both copies (manifest must be valid JSON: EOF, create .socket dir: NotFound, read /tmp/socket-patch-oracle-<pid>-a.bin: No such file, write_blob must stage the exact bytes with left: [])
after 500 0

cargo fmt --check passes on the file I touched. cargo clippy -p socket-patch-cli --all-targets -- -D warnings reports nothing in this file. It does report existing needless-borrow lints in tests/prebuilt_common/mod.rs and two other test files, but CI's clippy job doesn't lint test targets.

Where tests run

No tests were removed or moved. These selftests still run everywhere they ran before.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X2N3HNQzBaBgd8xyi3nxva


Generated by Claude Code


Note

Low Risk
Test-only fixture isolation change; no production or assertion logic changes.

Overview
Fixes flaky coverage failures in oracle selftests by replacing PID-based /tmp scratch paths with tempfile::tempdir() in three tests (git_sha256_file_hashes_real_bytes, write_minimal_manifest_emits_apply_compatible_shape, write_blob_stages_exact_bytes_at_hash_path).

Because tests/common/mod.rs is linked twice in binaries like e2e_vendored_production, those selftests could run in parallel on the same paths; the old scratch_dir helper (which called remove_dir_all first) could clobber sibling runs. Unique per-call temp dirs and drop-based cleanup remove that race. The scratch_dir helper and explicit file/dir teardown are deleted; test assertions are unchanged.

Reviewed by Cursor Bugbot for commit bd9c97f. Configure here.


Generated by Claude Code

tests/common/mod.rs is compiled twice into some test binaries (as
`common` and again as `vlt_e2e_common::common`), so each
oracle_selftests test runs twice in one process. The selftests built
their scratch paths from the PID alone, so the two copies shared
/tmp/socket-patch-oracle-<pid>-* and deleted or overwrote each other's
files. This failed coverage in two merge-group runs today
(e2e_vendored_production) and 11 of 300 local runs.

Use a fresh tempfile::tempdir() per test instead, which is unique per
call and removed on drop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2N3HNQzBaBgd8xyi3nxva
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Oct 8, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit bd9c97f. Configure here.

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 8, 2026
The coverage job failed on e2e_vendored_production: tests/common is
compiled twice into that binary, so its oracle selftests ran twice in
one process and clobbered each other's PID-named /tmp scratch files.
This is the same change as #1162 (per-test tempfile::tempdir()), so
the flake stops blocking this PR; it no-ops once #1162 merges.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Oct 8, 2026
Merged via the queue into main with commit f258eeb Oct 8, 2026
317 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/oracle-selftest-tempdirs branch October 8, 2026 21:56
@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 8, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Burn-down agent: labeled Ready for review at bd9c97f.

  • CI: all check suites on the head are success (317 check runs, ci-ok success); no main-wide failures.
  • Bugbot: reviewed this head (Cursor check success); no unresolved review threads.
  • GitHub still reports mergeable UNKNOWN after repeated re-queries over ~15 min; a local git merge-tree against main (830749f) is clean. No CHANGELOG.md change.
  • Slack announcement not sent this run (Slack send tool unavailable); the next run will retry.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants