Fix yarn berry project gates drifting between modes (#628, #629)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #628
Fixes #629

Summary

Hosted mode now refuses a Yarn Berry project whose root package.json mixes CRLF and LF line endings, using redirect_yarn_berry_mixed_line_endings and writing nothing. This is the same decision vendored mode already took with vendor_yarn_berry_mixed_line_endings. Before this change, hosted mode re-rendered the manifest in its majority ending, which silently rewrote lines the user never touched and left rollback with no original bytes to restore. Both modes now run one shared set of berry project gates, so they can't drift apart again.

Root cause

Each mode implemented the berry project-level gates itself (mixed line endings, cacheKey, .yarnrc.yml compressionLevel):

• vendored: SUPPORTED_CACHE_KEY, refuse_*, and yarn_berry_vendor_preflight
• hosted: YARN_BERRY_SUPPORTED_CACHE_KEY, berry_cache_key, and preflight_yarn_berry_hosted

Hosted mode also imported yarnrc_compression_level from vendor. The copies had drifted. The hosted gate looked only at yarn.lock, although the hosted rewriter also edits package.json (to add resolutions).

Change

• New formats/yarn/berry_gates.rs, a pure module with no I/O:
  • SUPPORTED_CACHE_KEY
  • one cache_key extractor
  • yarnrc_compression_level, moved here together with its tests
  • check(lock, manifest, Yarnrc) and the per-gate check_* functions, which return a BerryGate (MixedLineEndings { file }, NoMetadata, CacheKey { found }, Compression { level }, YarnrcUnreadable { error }). BerryGate owns the detail text and the code suffix.
• Vendored: refuse_* are now thin wrappers that map a BerryGate to the vendor_yarn_berry_* codes, and NoMetadata to vendor_lockfile_version_unsupported. The codes don't change.
• Hosted: preflight_yarn_berry_hosted(lock, manifest, yarnrc) calls berry_gates::check and maps results to the redirect_yarn_berry_* codes. All three callers now pass the root package.json:
  • the rewriter
  • the vendored→hosted takeover in scan/hosted.rs, which now refuses before reverting, wet and --dry-run
  • the hosted upstream restore in upstream/npm.rs, which already re-rendered that manifest
• lock_inventory::yarn reads cacheKey through the same extractor. YARN_BERRY_SUPPORTED_CACHE_KEY, berry_cache_key and berry_metadata are deleted, and patch/redirect no longer imports gate code from vendor::yarn_berry_lock.
• Docs: CLI_CONTRACT.md (the hosted berry line-endings paragraph and the takeover gates), docs/ecosystems.md, and CHANGELOG.md.

Detail text: the two modes' wording is now identical. Vendored mode's wording was kept, and it already names yarn install. A berry __metadata block with no cacheKey line now says (missing) in both modes; vendored mode used to print an empty value.

Test evidence

Issue	Test	Before fix	After fix (759933a)
#628 rewriter	patch::redirect::tests::berry_mixed_root_manifest_is_refused_untouched	FAILED on main + test (nothing written: ["package.json", "yarn.lock"])	ok
#628 fresh hosted scan	in_process_redirect::scan_redirect_refuses_a_mixed_line_ending_yarn_berry_manifest	FAILED at 2409f1a (no redirect_yarn_berry_mixed_line_endings warning)	ok
#628 vendored→hosted takeover	in_process_vendor::berry_takeovers_refuse_before_reverting_the_old_mode, new "mixed package.json" leg (wet and dry)	FAILED at 2409f1a (vendored→hosted mixed package.json dry=true: refused with redirect_yarn_berry_mixed_line_endings)	ok
#629 one decision for both modes	vendor::yarn_berry_lock::tests::both_modes_take_the_same_project_gate_decision (table: supported, BOM+CRLF, cacheKey 10, no cacheKey, compressionLevel: mixed, mixed lock, mixed manifest; asserts the same suffix and identical detail)	n/a (new API)	ok
#629 shared module	formats::yarn::berry_gates::tests::* (4 tests plus the 3 moved yarnrc_compression_level tests)	n/a	ok

Local runs:

• cargo clippy --workspace --all-features -- -D warnings: ok
• cargo fmt: the new module is rustfmt-clean. Every changed hunk is formatted. main itself isn't cargo fmt --check clean, so I did not reformat files outside this change.
• cargo test --workspace --all-features --no-fail-fast: 9731 passed, 12 failed. All 12 failures are permission-denial tests: *_state_write_failure_*, *_unremovable*, wire_*failure*, relax_loop_must_not_traverse_symlinked_root, redirect_json_mode_write_failures_*, partial_lockfile_write_failure_*, vlt_heal_invalidation_failure. They depend on chmod taking effect, and the sandbox runs as root, which ignores it. None of them touch yarn code. After the history cleanup, in_process_vendor (104/104), the core berry|yarn unit tests (233/233) and the in_process_redirect yarn_berry tests (5/5) were re-run.
• scripts/yarn-berry-vex-matrix.sh 4.18.0 (with COREPACK_NPM_REGISTRY set): e2e_redirect_yarn_berry_build, e2e_vendor_yarn_berry_build, e2e_yarn4_pnpm_linker_build and e2e_yarn4_workspaces_build all pass (56 tests) on real yarn 4.18.0. In e2e_yarn_legacy_cachekey_refusal_build the yarn 3 cells pass. Its two yarn 2.4.3 cells couldn't run locally: the sandbox proxy blocks repo.yarnpkg.com, and yarn 2 isn't on the npm registry. CI covers them.

CI on 759933a: green. 485 of 491 check runs pass and 6 are skipped. Bugbot found no issues, and there are no review threads. The Poetry backtest native (ubuntu-latest, 2.0.1) failed rescanIdempotent once; this PR touches no Poetry code, and its single re-run passed.

No wrapper changes are needed: npm/, pypi/ and gem/ only dispatch to the binary.

Follow-up, not changed here: in hosted mode, an unreadable .yarnrc.yml is still treated as absent. The rewriter receives files that were already read, so it can't tell "unreadable" apart from "missing". Vendored mode refuses it with YarnrcUnreadable. The gate now supports Yarnrc::Unreadable, so the hosted takeover could adopt it in a later change.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes hosted Yarn Berry redirect and vendored→hosted takeover preflight behavior for mixed package.json line endings; lockfile/config paths are sensitive but refusals are fail-closed with new tests.

Overview
Yarn Berry hosted and vendored modes now share one project gate module (formats/yarn/berry_gates.rs), so mixed line endings, cacheKey, and .yarnrc.yml compressionLevel decisions—and their detail text—cannot drift between modes.

#628: Hosted redirect now runs the same mixed–line-ending check on root package.json as on yarn.lock (via redirect_yarn_berry_mixed_line_endings), refusing the rewrite instead of re-rendering resolutions in a majority EOL. Vendored→hosted takeover preflight reads package.json before revert; upstream npm restore does the same.

#629: Vendored refuse_* paths and preflight_yarn_berry_hosted(lock, manifest, yarnrc) both delegate to berry_gates::check; yarnrc_compression_level / cache_key move out of yarn_berry_lock into the shared module. Contract docs (CLI_CONTRACT.md, docs/ecosystems.md) note manifest gating.

Incidental: SHA1 hashing in Gradle cache / JVM jar / Maven sidecars routes through utils::digest::sha1_hex_of.

^{Reviewed by Cursor Bugbot for commit 74629b2. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] native (ubuntu-latest, 2.0.1) failed on 759933a. This is the Poetry 2.0.1 backtest, and the only failing check is rescanIdempotent in the direct hosted cell. The other 4 cells pass.

I don't think this failure is caused by this PR:

• This PR only touches the yarn berry gates, and no Poetry or PyPI code path calls them.
• The same code (853f810, which differs only in formatting of unrelated files) passed every check suite, including this Poetry job.

PR #596 (open) targets Poetry matrix flakes caused by PyPI and patch-API transport blips, which could explain a failed hosted re-scan. I haven't confirmed that this is the same failure, so I'm not porting #596's change here; the re-run will show whether it reproduces.

A re-run of the failed job is refused with 403 while the workflow is still running. I'll re-run it once when the run completes. If it fails again, I'll treat it as real and root-cause it.

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[agent] Ready for review at head 759933a.

• CI: green. 485 of 491 check runs pass and 6 are skipped. One Poetry 2.0.1 backtest (rescanIdempotent) flaked; this PR touches no Poetry code and its single re-run passed.
• Bugbot: reviewed 759933a and found no issues. There are no open review threads.
• For reviewers: the new shared formats/yarn/berry_gates.rs module, and hosted mode now refusing a mixed-line-ending root package.json (vendored→hosted takeover included) instead of re-rendering it.

The Slack announcement is pending because no Slack send tool was available in this run.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Codex review of 759933a850676e86aeb6a89742f68c926b9dcd1a: ready to merge as-is from this review. No actionable findings.

The shared Berry gates preserve the supported cache/compression checks and warning codes. Mixed root manifests are now refused before hosted rewrites, before vendored takeover reverts, and before upstream restore writes. Uniform LF/CRLF behavior and BOM preservation remain covered.

Validation:

• 343 repository tests passed: 233 core Berry/Yarn tests, five hosted Berry CLI tests and 105 vendored CLI tests, including the wet/dry takeover regression.
• A separate public restore test passed four scenarios: wet/dry mixed-manifest refusal with files unchanged, and wet/dry uniform CRLF/BOM success.
• Independent native checks used Yarn 2.4.3, 3.8.7, 4.0.2 and 4.18.0. Fourteen metadata comparisons and six native line-ending/immutable controls support the implementation. Native Yarn can normalize a mixed manifest even under --immutable, confirming the reason for the explicit refusal.
• All 12 reviewed source hashes match this commit. Independent review and the merge check against main 045d7ec7 are clear.

Fresh CI is clear: 485 successful checks, 7 skipped; 13 successful workflows and 1 skipped. Bugbot is clear on this exact commit, with no unresolved threads or outstanding actionable feedback. The documented pre-existing hosted behavior for unreadable .yarnrc.yml remains a separate follow-up.

GitHub's normal human approval requirement remains before merge.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[agent] coverage failed on 7562f62. Two tests in socket-patch-cli --lib failed: commands::vex_consumed::tests::hosted_expands_alias_only_copies and hosted_reuses_expanded_npm_copies_and_merges_alias_variants.

This failure isn't caused by this PR. Current main (4646693) fails both tests the same way when run locally. Since #605 landed, the resolver also returns npm alias and nested store copies, and these two tests still assert the older, smaller set. This PR changes no npm or VEX code.

#851 already fixes it with a test-only change. I've ported that same change into this PR as 94dfd04; it becomes a no-op once #851 merges. Locally, cargo test -p socket-patch-cli --lib passes (840/840) and clippy is clean.

Earlier in this run I also merged main into this branch (7562f62) to resolve conflicts with main's new berry_entry module and its yarnrc_scalar helper. yarnrc_scalar now lives in formats/yarn/berry_gates.rs, and its one caller, shared_store.rs, reads it from there.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[agent] native (ubuntu-latest, 2.10.4), the PDM backtest, failed on 94dfd04. One of 36 cases failed, 2.10.4 transitive vendored, on the checks rescanAfterRelockApplies and rescanReusesWheel. The other 35 cases passed or were refused/unsupported as expected, including transitive hosted and agent and every other vendored shape.

I don't think this PR causes it:

• The PR's diff against main touches only yarn berry gate code, the berry-only takeover preflight in scan/hosted.rs, the move of yarnrc_scalar into berry_gates, and the ported Fix vex alias tests broken by store-copy merge #851 test fix. No PDM or PyPI vendoring path calls any of these.
• None of the commits on main since its last green PDM run (792e836, run 37302882874) touch PDM code either.
• The failing case took 44.4s, against 15–30s for the others. That fits a retried re-scan inside the harness.

I couldn't read the case's result.json or rescan-after-relock.log, because this sandbox can't download Actions artifacts. The workflow run is still in progress, so the failed job can't be re-run yet. I'll re-run it once when the run completes. If it fails again, I'll treat it as real and find the cause.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Burn-down agent: Ready for review at 94dfd04.

• CI: 491/491 check runs green (485 success, 6 skipped), mergeable clean. native (ubuntu-latest, 2.10.4) failed once in the PDM backtest (transitive vendored rescan cells, unrelated to this yarn berry change; the PDM transport-flake fix is Fix PDM matrix flakes on PyPI/patch API transport blips #860) and passed on re-run.
• Bugbot: reviewed 94dfd04, no findings; 0 unresolved review threads.
• Reviewer focus: the berry project gates (mixed line endings, cacheKey, compressionLevel) now live in one shared module used by both hosted and vendored modes; hosted now refuses mixed CRLF/LF root manifests with redirect_yarn_berry_mixed_line_endings instead of re-rendering them.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] coverage, test (windows-latest) and test-release failed on 236f196 because of main, not this PR. socket-patch-core --lib fails utils::digest::tests::production_digests_go_through_the_helpers, since three Gradle files on main still hash inline. I ported #878 as 74629b2. It becomes a no-op once #878 merges.

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 74629b2. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Burn-down agent: labeled Ready for review at 74629b2 (74629b2ce51cedc9629ea486aaa0b8222ab24b48).

• CI: 538/538 workflow checks green on the head commit (6 skipped by matrix rule), after one re-run of jobs the runner outage cancelled. The only non-green entries are 2 CodeQL default-setup Analyze jobs that GitHub cancelled during the outage, and GitHub doesn't allow re-running them ("This workflow run cannot be retried").
• Bugbot: reviewed 74629b2 with no new issues, and no review threads are open.
• Mergeable against main, with no conflicts.

---

Generated by Claude Code