diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 7c77f502e..ebde249c6 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -32,7 +32,7 @@ Rows are in `--help` order (v5.0): the hosted/vendored workflow (`scan` → `vex **Removed in v4.0:** the `unlock` subcommand (a leftover lock from a crashed run never blocks acquisition — the OS releases a dead holder's advisory lock — so there is no stale-lock state to inspect or clear before a mutating command; `repair` briefly owned lock-file cleanup in v4.x, and since v5.0 every lock-taking command removes its own lock file on exit). -**Lock lifecycle (v5.0).** `<.socket>/apply.lock` never outlives the command that took it: acquisition creates `.socket/` when it is missing, the guard's drop unlinks the file WHILE the lock is still held (so a waiter can never lock an orphaned inode), releases it, and then removes `.socket/` itself if that left the directory empty — a run that had nothing to persist leaves no `.socket/` behind, and there is nothing to `.gitignore`. A leftover file from a crashed (SIGKILLed) run is reclaimed in place and removed by the next lock-taking command. The lock is taken by `apply`, `rollback`, `remove`, `repair`, `vendor`, agent-mode `get` and `scan --apply`/`--sync` (download → manifest write → nested apply is ONE lock window — the nested apply never re-acquires), and `scan`/`get` in vendored **and hosted** mode — hosted acquires it around its first wet write (the takeover pre-reverts), never on `--dry-run` and never when the run would write nothing, so hosted previews and no-op runs create no `.socket/`. Dry runs of the other commands may still take the lock; it is residue-free either way. A live holder is `lock_held` (exit 1); a directory or special file squatting on `.socket/` or on the lock path is a lock I/O error — `lock_io` (exit 1, `failed to open lock file at : …`; a read-only project root surfaces the same code at the acquire, before any ledger or manifest write) — never `lock_held`. +**Lock lifecycle (v5.0).** `<.socket>/apply.lock` never outlives the command that took it: acquisition creates `.socket/` when it is missing, the guard's drop unlinks the file WHILE the lock is still held (so a waiter can never lock an orphaned inode), releases it, and then removes `.socket/` itself if that left the directory empty — a run that had nothing to persist leaves no `.socket/` behind, and there is nothing to `.gitignore`. A leftover file from a crashed (SIGKILLed) run is reclaimed in place and removed by the next lock-taking command. The lock is taken by `apply`, `rollback`, `remove`, `repair`, `vendor`, agent-mode `get` and `scan --apply`/`--sync` (download → manifest write → nested apply is ONE lock window — the nested apply never re-acquires), and `scan`/`get` in vendored **and hosted** mode — hosted acquires it before its first wet write (the staged takeover reverts), never on `--dry-run` and never when the run would write nothing, so hosted previews and no-op runs create no `.socket/`. Dry runs of the other commands may still take the lock; it is residue-free either way. A live holder is `lock_held` (exit 1); a directory or special file squatting on `.socket/` or on the lock path is a lock I/O error — `lock_io` (exit 1, `failed to open lock file at : …`; a read-only project root surfaces the same code at the acquire, before any ledger or manifest write) — never `lock_held`. **Bare-UUID fallback.** `socket-patch ` is rewritten to `socket-patch get `. The UUID shape checked is the standard 8-4-4-4-12 hex pattern (case-insensitive). See [`src/lib.rs::looks_like_uuid`](src/lib.rs). @@ -161,7 +161,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **Vendored entries and the rest of the CLI.** Because nothing is in the manifest, vendored patches are invisible to `apply` (nothing to apply in place) but fully visible to `list` (listed from the ledger, labeled `Mode: vendored (recorded in .socket/vendor/state.json)` in human mode, exit 0 on a vendored-only project), `vex` (attested from the embedded records while a lockfile still wires the artifact — see "Manifest-less VEX"), `repair` (health-checked and rebuilt from the ledger), and `scan --prune` (lockfile-driven reconcile). They are exempt from standalone `vendor`'s manifest reconcile (`reconcile_dropped` never touches `detached` entries) and exit via `remove ` (which reverts them), `vendor --revert`, or `rollback`, whose vendored leg reverts every in-scope ledger entry (unscoped and identifier-scoped runs; path-scoped runs reach them only when an installed copy matches). -`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place, never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum does not list at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and only then is it redirected. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is refused BEFORE its revert, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (`redirect.warnings[]`, and `redirect.skipped[].reason`). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are gated the same way, from the ledger entry and the lock on disk: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (vendored uv pins the entry down to the patch's version; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is refused with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is refused with `redirect_poetry_lock_unsupported`. A taken-over package whose wiring was reverted but that was then not pinned to hosted now installs the unpatched registry release in both modes. Causes include a refused lock, unavailable hosted wheel metadata, or a vendored ledger update that failed after the revert (refused with `redirect_vendored_revert_failed`). It is reported as `redirect_takeover_unpatched` with `status: "partial_failure"` and exit 1, never as success. That warning also prints under `--silent`. Human output prints no `Migrated …` progress line for the package and no "keep the hosted patches" next steps. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` around its first wet write (the takeover pre-reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON keeps the hosted shape — top-level `status: "error"`, `errorCode: "lock_held" | "lock_io"`, a string `error`, and `redirect: {mode: "hosted"}` retained (NOT the vendored `error: {code, message}` object). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` prints the results table and update detection like the other modes, then rewrites without a prompt (scan never prompts); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints the table's paid nudge, then `No downloadable patches (paid subscription required).`, and exits 0 without entering the engine (parity with the agent/vendored arms). JSON output gains a `redirect` sub-object: `{ mode: "hosted", redirected, rewrittenFiles, skipped, patches, warnings, dryRun }` (`mode` is additive so consumers can dispatch without inferring it). `patches` (additive, v5.0) is the per-purl outcome of every selected patch, sorted by purl: `{purl, uuid, action}` with `action` `pinned` (`would_pin` under `--dry-run`; `redirected` counts these), `skipped` (`errorCode` = the `skipped[]` reason, `error` = its detail when it has one), or `unpinned` (`errorCode: redirect_unconfirmed` — the patch was granted but no lockfile entry pinning it could be rewritten; the human output's `Not hosted : …` line). An `unpinned` or `skipped` row does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the per-dep `source` block to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds), `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover runs these berry gates (mixed line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) BEFORE reverting a vendored berry purl — wet and `--dry-run` alike — so a refused purl keeps its vendored wiring, ledger entry and artifact byte-identical and is skipped with the gate's code (never announced as `redirect_takeover_reverted_vendored` and then left unpatched in both modes). +`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place, never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum does not list at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and it is redirected in the same commit. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is kept vendored, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (`redirect.warnings[]`, and `redirect.skipped[].reason`). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are covered the same way: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (vendored uv pins the entry down to the patch's version; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is kept vendored with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is kept vendored with `redirect_poetry_lock_unsupported`. **Staged takeover (v5.0)**: the takeover is atomic. Each purl's vendored revert is staged in memory (the run's group commit, the same journaled commit vendored mode uses), the hosted rewrite plans against the reverted project, and a staged purl the rewrite does not pin is retracted: every staged revert is undone, that purl keeps its vendored wiring, ledger entry and artifact byte-identical, and the rest are staged and rewritten again. A retracted purl is skipped (`redirect.skipped[].reason`) with the cause: its existing skip reason (unavailable wheel metadata, …), the rewriter warning that names the package (`redirect_yarn_berry_missing_checksum`, `redirect_pypi_platform_wheel`, …), `redirect_requirements_takeover_unreachable`, the rewrite's lock-level refusal (`redirect_yarn_berry_mixed_line_endings`, `redirect_bun_lock_unsupported`, a Gradle planner refusal, …) or `redirect_takeover_not_pinned`; that warning is reported in `redirect.warnings[]`, followed by `redirect_takeover_kept_vendored` naming the package. Exit 0: the package stays vendored and patched, never unpatched in both modes. The reverts, the hosted pins and the vendored ledger then reach the disk in one commit, and the reverted artifacts are deleted only after it: a refusal or write failure before the commit writes nothing, a failed commit puts back the files it replaced, and a commit interrupted after its journal was written is finished by the next command that takes the apply lock. `--dry-run` runs the same steps and drops the staged state instead of committing, so its `redirected` count and warnings are the wet run's. A hosted run with no takeover stages its writes the same way but commits them without the journal (it writes nothing under `.socket/`): a file that fails to be replaced puts back the ones already replaced, so a failed run leaves no lock half-redirected. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` before its first wet write (the staged takeover reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON keeps the hosted shape — top-level `status: "error"`, `errorCode: "lock_held" | "lock_io"`, a string `error`, and `redirect: {mode: "hosted"}` retained (NOT the vendored `error: {code, message}` object). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` prints the results table and update detection like the other modes, then rewrites without a prompt (scan never prompts); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints the table's paid nudge, then `No downloadable patches (paid subscription required).`, and exits 0 without entering the engine (parity with the agent/vendored arms). JSON output gains a `redirect` sub-object: `{ mode: "hosted", redirected, rewrittenFiles, skipped, patches, warnings, dryRun }` (`mode` is additive so consumers can dispatch without inferring it). `patches` (additive, v5.0) is the per-purl outcome of every selected patch, sorted by purl: `{purl, uuid, action}` with `action` `pinned` (`would_pin` under `--dry-run`; `redirected` counts these), `skipped` (`errorCode` = the `skipped[]` reason, `error` = its detail when it has one), or `unpinned` (`errorCode: redirect_unconfirmed` — the patch was granted but no lockfile entry pinning it could be rewritten; the human output's `Not hosted : …` line). An `unpinned` or `skipped` row does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the per-dep `source` block to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds), `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover of a yarn-berry entry is checked against these project gates (mixed `yarn.lock` / `package.json` line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) on the project as it is before any revert, because the revert re-renders `package.json` in its majority ending — wet and `--dry-run` alike. A refused purl keeps its vendored wiring, ledger entry and artifact byte-identical, is skipped with the gate's code (reported in `redirect.warnings[]`, followed by `redirect_takeover_kept_vendored`) and is never announced as `redirect_takeover_reverted_vendored`. The rewriter reads a fixed set of candidate files from the project root: the npm-family locks (`package-lock.json`, `npm-shrinkwrap.json`, `pnpm-lock.yaml`, `shrinkwrap.yaml`, `yarn.lock`, plus `.yarnrc.yml` for the berry cache-config gate, `bun.lock` / `bun.lockb`, and `vlt-lock.json` with `vlt.json` and `node_modules/.vlt-lock.json` read only), `requirements.txt` / `uv.lock` / `Pipfile.lock` (pipfile-spec 6; see the Pipenv section below) / `poetry.lock` (every Poetry lock generation from 1.0 on — the 0.12 `[metadata.hashes]` layout is refused because that installer ignores URL sources; a Poetry < 1.4 writer additionally gets `redirect_poetry_stale_install_risk`, see `docs/testing/poetry-compatibility.md`) / `pdm.lock` (PDM lock formats `2` and `4.3`–`4.5.1`; the identity-losing `3.1` / `4.0`–`4.2` formats and unknown future formats are refused with `redirect_pdm_refused`, and a lock-format-`2` writer additionally gets `redirect_pdm_legacy_sync_required`, see `docs/testing/pdm-compatibility.md`; when `uv.lock` or `poetry.lock` sits beside it they drive and `pdm.lock` is left alone), `Cargo.toml` / `Cargo.lock` / `.cargo/config.toml` (plus the legacy extensionless `.cargo/config` — cargo reads that spelling in preference when both exist, so the managed `[registries.…]` block is written into whichever one is present; **cargo also reads every workspace-member manifest** — the `[workspace] members` globs minus `exclude` — and every in-root path-dependency manifest, recursively, reached without crossing a symbolic link and never under `.socket/`, and pins the crate in each one that declares it, so those `/Cargo.toml` files can appear in `rewrittenFiles`. A crate is redirected only when every declaration pins and every other `Cargo.lock` package depending on it is a planned member: one a registry or git crate — or a path package outside the root or behind a link — also depends on is refused `redirect_cargo_transitive_dependents` (a pin reaches only the declarations it sits on), a crate no manifest declares keeps `redirect_cargo_toml_dep_not_found` with a transitive-only detail naming `--mode vendored`, a crate every declaration of which requires another version (no requirement accepts the patched version) is refused `redirect_cargo_toml_dep_unrewritable`, and so is a requirement that also matches another locked version of the crate — each a transactional skip, never recorded or attested. With NO `Cargo.lock` there is no resolved graph to ask, so the dependents question is answered from the manifests instead: a crate declared beside any other dependency — anything but a path dependency on a manifest this run also pins, or a `workspace = true` inheritor of a table it scans — or beside a workspace member this run did not read (a `members` glob, or a member outside the project or behind a symbolic link, which member discovery drops) is refused `redirect_cargo_lockless_dependents`, whose detail names the remedies (commit a lockfile, or `--mode vendored`); a project whose only dependency is the patched crate has nothing that could pull it in and still redirects. All-CRLF manifests, locks and configs are rewritten with CRLF kept (mixed endings keep refusing where the grammar does not match), and `remove` / rollback match the recorded fragments across a later CRLF↔LF checkout conversion), `composer.lock`, `nuget.config` / `packages.lock.json`, `Gemfile` / `Gemfile.lock`, `pom.xml` (+ `.mvn/maven.config` / `.mvn/checksums/checksums.sha256` for maven Trusted Checksums merge, and, for a Gradle build, every settings, build, `buildSrc`, included-build, applied and plugin-source script, version catalog and lock file the script graph reaches, plus `gradle/verification-metadata.xml`, `gradle/wrapper/gradle-wrapper.properties` and the owned `.socket/gradle/` files), and the sbt build files (`socket-patch.sbt`, `socket-patch-vendor.sbt`, `build.sbt`, `project/build.properties`, `.sbtopts`, `.jvmopts`; `build.sbt.lock` and the Mill / scala-cli build files `build.mill`, `build.mill.yaml`, `build.sc`, `.mill-version`, `project.scala` for their presence only) — read, never edited; `socket-patch.sbt` is the only sbt file hosted mode writes (see **Hosted sbt** below). **npm-family flavor coverage**: package-lock / npm-shrinkwrap, pnpm (root OR any nested `*/pnpm-lock.yaml`), yarn classic (a yarn 2+ install migrates a v1 `yarn.lock` and drops its pins, so a run whose v1 lock carries a hosted pin warns `redirect_yarn_classic_berry_migration_risk` — the hosted twin of the vendored `yarn_classic_berry_migration_risk` — unless the root `package.json`, read as advisory input, declares `"packageManager": "yarn@1…"`), **yarn berry** (the pin yarn writes for a root `resolutions` entry: the root `package.json` — edited only beside a berry `yarn.lock` — gains one `"@npm:": ""` selector per locked range (`redirect_yarn_berry_resolution` edits), and only that `yarn.lock` entry is re-keyed `"@"` with the same `resolution:` + `yarnBerry10c0` checksum (`redirect_yarn_berry_entry`), moved to yarn's key order; never an `npm:` locator, whose fetcher sends npm registry auth to the patch host, nor a tarball locator under an `npm:` key, which hardened mode rejects (YN0078). An older release's `npm:::__archiveUrl=` pin is still recognized and is re-pinned on the next run; rollback rebuilds the key from the selectors and drops them. Refused, nothing written: a user-authored `resolutions` entry for the package `redirect_yarn_berry_resolutions_conflict`, no root manifest `redirect_yarn_berry_manifest_missing`, a builtin `patch:` entry wrapping the same descriptor `redirect_yarn_berry_shared_descriptor`, an artifact URL yarn cannot fetch as a tarball `redirect_yarn_berry_artifact_url_unsupported`; cacheKey `10c0` and `.yarnrc.yml compressionLevel 0` gated by `redirect_yarn_berry_cache_unsupported`), and **bun** (text `bun.lock` lockfileVersion 0, 1 or 2 — 0 is the `--save-text-lockfile` opt-in lock of Bun 1.1.39–1.1.45, 1 the 1.2–1.3 default, 2 the 1.4+ default; all three emit one `packages` grammar, so the registry 4-tuple → URL 3-tuple rewrite is version-independent and the lock's own version line is kept. Any other or missing version, or a `packages` section outside bun's single-line grammar, is refused `redirect_bun_lock_unsupported` — the detail is the shared version gate's text (a newer version: update socket-patch, re-locking would reproduce it; no integer: re-lock with Bun ≥ 1.2), identical to the vendored refusal. A version-0 lock holding `workspace:` packages is refused `redirect_bun_workspace_unsupported` (its 2-tuple workspace grammar cannot keep the hosted tuple through a frozen install); the remedy is to delete `bun.lock` and re-run `bun install` with Bun ≥ 1.2, which writes lockfileVersion 1 (accepted). A plain in-place `bun install` bumps the version only when a workspace depends on another workspace (e.g. root → member — the shape the matrix measured); otherwise Bun 1.2.0 keeps version 0 and Bun 1.2.23+ fail to resolve, so the in-place bump is not the documented remedy. Bun lock version, grammar and workspace compatibility are checked before a vendored takeover, including during dry-run: these refusals preserve the existing lock, artifact and vendor ledger. Version-1 and version-2 workspace locks are rewritten, nested versions included. A granted dep with no rewritable entry warns `redirect_bun_entry_not_found`, a grant without a sha512 `redirect_bun_missing_sha512`; a CRLF lock keeps `\r\n` on the rewritten line, and a hosted URL left by an earlier grant of the same `name@version` is re-pinned in place. **Digest-less re-saves (Bun 1.1.39–1.3.9)**: every text-lock Bun below 1.3.10 re-saves a URL tuple WITHOUT its `sha512` whenever the lock is re-saved for another reason (`bun add`, `bun install` after a package.json or workspace change), leaving the 2-tuple `["name@", {meta}]` — the spec Bun installs from is intact. The CLI treats that spelling as its own wiring: a repeat hosted run counts the dep as redirected (no `redirect_bun_entry_not_found`) and HEALS the line back to the 3-tuple with the current `sha512`, recording the heal as a further `redirect_bun_lock_package` edit whose `original` is the 2-tuple (a stale URL is re-pinned from either spelling); `rollback`, scoped `rollback ` / `remove ` and the vendored takeover accept the digest-less spelling of a recorded `new` line (same key, spec and meta, only the trailing `"sha512-…"` missing) and restore the recorded original over it, so the chain always unwinds to the pristine registry line. Anything else — another uuid/token, another version, a re-laid meta object — is still drift. **Native `bun.lockb`**: when no text `bun.lock` exists, binary format versions 1, 2 and 3 are read and rewritten directly. Socket Patch does not invoke Bun or convert the project to a text lockfile. Exact matching package records are rewritten to hosted tarballs with the granted integrity, preserving dependency resolution IDs, workspace/dependency topology and unrelated package metadata; binary pointers and the package metadata hash are updated. Per-package `redirect_bun_lockb_package` snapshots support scoped rollback, repeat runs, superseding grants and hosted ↔ vendored takeover. A regular binary lock is discoverable even with no Bun runtime or `node_modules`; a dry run previews the same binary edits without writing them. A malformed, unreadable, unsupported or unverified binary structure is `redirect_bun_lockb_invalid` (exit 0, `redirected: 0`), and it refuses the npm rewrite before any takeover or sibling npm-family lock mutation. A symlinked binary write target is `redirect_symlinked_file_unsupported` (exit 1, including dry-run). `bun.lock` wins when both spellings exist. Binary-only projects do not receive `redirect_npm_no_lockfile`. Measured boundaries and the real-Bun matrix: `docs/testing/bun-compatibility.md`), and **vlt** (`vlt-lock.json` without `lockfileVersion`, `0` or `1`; see the vlt hosted-mode contract below). **Rush monorepos**: when `rush.json` is present the rewriter also reads `common/config/rush/pnpm-lock.yaml` and each `common/config/subspaces//pnpm-lock.yaml` (sorted for determinism) under their repo-relative keys and repoints them in place; editing them emits `redirect_rush_repo_state_stale` when `common/config/rush/repo-state.json` exists (the `pnpmShrinkwrapHash` desync is refreshed by `rush update`, which the redirect survives). **maven** is fail-closed via version suffixing: a `mavenSuffixedVersion` + `mavenPomSha256` override pins the Socket-only `-socket.` by rewriting the literal `` (`redirect_maven_dep_version`) or adding a `` entry (`redirect_maven_dep_management_added`), plus optional Trusted Checksums (`redirect_maven_trusted_checksums`, conflicts as `redirect_maven_trusted_checksums_conflict`; when `.mvn/wrapper/maven-wrapper.properties` pins a Maven older than 3.9.4, which ignores those files, the additive warning `redirect_maven_trusted_checksums_unenforced`); a `${property}` version is refused (`redirect_maven_dep_unpinned`), a non-matching literal skipped (`redirect_maven_dep_version_mismatch`), and an override without a suffixed version falls back to same-GAV repository injection (`redirect_maven_same_gav_fallback`, NOT fail-closed). **gradle** (v5.0) is automated wiring, no longer a pasted snippet: the owned settings script `.socket/gradle/socket-patch.hosted.settings.gradle` with its index `.socket/gradle/hosted-index.tsv`, one apply line per build's settings file, every lock entry of the GA moved to the suffixed version, and the suffixed component in an existing `gradle/verification-metadata.xml`. A refused dep writes nothing and keeps `redirect_gradle_manual_snippet` as its fallback; same-GAV grants are refused (`redirect_gradle_same_gav_unsupported`). Rules, refusals and codes: [Gradle builds](#gradle-builds-v50). @@ -180,7 +180,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm **get --mode and installed narrowing (v3.6).** `get --mode hosted|vendored` consumes the resolved patch(es) through the SAME engines as `scan --mode hosted|vendored`, so for the same selected (purl, uuid) set the on-disk result is identical by construction — the per-advisory selector for hosted/vendored (`get --save-only` then `vendor` still works). **Agent mode (v5.0 lock + residue rules)**: the download phase runs under `<.socket>/apply.lock` and hands the guard to the nested apply, so download → manifest write → apply is one lock window (the nested apply never re-acquires and inherits every caller flag — `--lock-timeout` and `--verbose` included); a failed acquire is `{status: "error", errorCode: "lock_held" | "lock_io", error}` on get's legacy envelope, exit 1, before any fetch (a read-only `.socket/` fails here, naming the lock path). `.socket/` and `.socket/blobs/` are created only when a record is actually persisted — an all-skipped or all-failed run leaves no `.socket/` on a fresh project — and a same-uuid `get ` re-run rewrites neither the manifest nor the blobs. Semantics: -* **Hosted** (`get GHSA-… --mode hosted`): resolves the advisory, then hands the selected (purl, uuid) pairs to scan's hosted engine — reference grants, cross-mode takeover pre-revert, lockfile rewrite (no ledger, v5.0), gem stale-install probe, warnings, confirmation rules (cargo via `confirmed_cargo_uuids`, golang via `confirmed_golang_uuids` only) all identical to `scan --mode hosted`, and (v5.0) under the same `apply.lock` acquisition — taken around the first wet write, never on `--dry-run` or when nothing would be written; a failed acquire folds as top-level `errorCode: "lock_held" | "lock_io"` + string `error` (exit 1), and `--dry-run` under a held lock still exits 0. **No manifest write, no blobs, no ledger** — the lockfile edits are the persistence. JSON: get's legacy envelope gains the same nested `redirect` sub-object as scan's (`{mode:"hosted", redirected, rewrittenFiles, skipped, warnings, dryRun}`); the top-level shape is `{status, found, patches:[], warnings?}` — `downloaded`/`applied` are absent (nothing is downloaded into `.socket/`). Exit codes follow scan's hosted semantics: skipped grants and rewriter warnings never flip the exit; infra errors (reference fetch, file writes) exit 1. Human prompt: `Redirect N packages to the hosted patch server?` (singular for one; `--yes`/`--json`/non-TTY auto-accept as usual). This confirm is get's alone: `scan` never prompts. +* **Hosted** (`get GHSA-… --mode hosted`): resolves the advisory, then hands the selected (purl, uuid) pairs to scan's hosted engine — reference grants, staged cross-mode takeover, lockfile rewrite (no ledger, v5.0), gem stale-install probe, warnings, confirmation rules (cargo via `confirmed_cargo_uuids`, golang via `confirmed_golang_uuids` only) all identical to `scan --mode hosted`, and (v5.0) under the same `apply.lock` acquisition — taken around the first wet write, never on `--dry-run` or when nothing would be written; a failed acquire folds as top-level `errorCode: "lock_held" | "lock_io"` + string `error` (exit 1), and `--dry-run` under a held lock still exits 0. **No manifest write, no blobs, no ledger** — the lockfile edits are the persistence. JSON: get's legacy envelope gains the same nested `redirect` sub-object as scan's (`{mode:"hosted", redirected, rewrittenFiles, skipped, warnings, dryRun}`); the top-level shape is `{status, found, patches:[], warnings?}` — `downloaded`/`applied` are absent (nothing is downloaded into `.socket/`). Exit codes follow scan's hosted semantics: skipped grants and rewriter warnings never flip the exit; infra errors (reference fetch, file writes) exit 1. Human prompt: `Redirect N packages to the hosted patch server?` (singular for one; `--yes`/`--json`/non-TTY auto-accept as usual). This confirm is get's alone: `scan` never prompts. * **Vendored** (`get GHSA-… --mode vendored`): the download phase is scan's vendored posture — **manifest-free (v5.0)**: the selected records are fetched into memory (`download_patch_records`; no blob staging; nothing under `.socket/` is written; the nested apply never runs), then scan's vendor step runs under the apply lock over exactly the selected records, like `scan --mode vendored` (no whole-manifest scope and no `[note]` about other records — that blast radius is retired with the manifest; a legacy manifest record for a vendored purl is migrated out of `.socket/manifest.json` the same way scan does it). JSON: get's envelope takes the detached download envelope's shape — `{status, found, downloaded, skipped, failed, detached: true, patches: [{purl, uuid, action: "downloaded" | "skipped" | "failed", …}], warnings?}` (`applied` is absent; `detached: true` is pinned; a `downloaded` record for a purl the vendor ledger holds at another uuid carries the additive `oldUuid`, derived from the ledger — the human `[fetch]` line reads ` (replacing )`) — and gains the nested `vendor` Envelope exactly like scan's `result["vendor"]`; a vendor-step error folds the partial envelope + `{status:"error", error:{code,message}}` in (a pre-failure takeover reconcile may have already mutated the ledger — its events must reach the consumer). Exit: download failures or vendor `has_errors` → `partial_failure`/1. Human prompt: `Download and vendor N patches?`; `--dry-run` prints `[dry-run] Would download and vendor N patches. No changes made.` on both identifier paths (uuid and search). Telemetry mirrors scan's vendored arms (`track_outcomes_for_vendor` / `track_patch_vendor_failed`). **Bun vendored preflight (additive)** — shared by `get --mode vendored` on both its paths and `scan --mode vendored`: before ANY patch download, and only when the selection holds a `pkg:npm/` purl, the download phase reads `bun.lock`/`bun.lockb` once (`preflight_vendor`) and, when the vendor backend would refuse the project — a malformed, unreadable or unsupported `bun.lockb` → `vendor_bun_lockb_invalid`; an unreadable `bun.lock` → `vendor_lockfile_missing`; a `lockfileVersion` other than 0/1/2 or a non-canonical `packages` grammar → `vendor_lockfile_version_unsupported`; `workspace:` packages in a lock below version 2 → `vendor_bun_workspace_unsupported` — every `pkg:npm/` result becomes `{action:"failed", errorCode:, error:}` with NO fetch (the patch view is never requested) and no patch record; other ecosystems' results are untouched. **Search path** (`get --mode vendored`) and `scan --mode vendored`: the records ride `patches[]` / `download.patches[]` with `downloaded: 0`, the download phase writes nothing under `.socket/` (v5.0 — a pre-existing `.socket/manifest.json`, including a record seeded for another purl, is left byte-untouched), the vendor step still runs over the remaining records (no event for the refused purl), exit `partial_failure`/1. **uuid path** (`get --mode vendored`): the uuid lookup is the only fetch; the run exits 1 BEFORE the vendor step with exactly `{status:"error", found:1, downloaded:0, skipped:0, failed:1, error:{code, message}, patches:[{purl, uuid, action:"failed", errorCode, error}]}` (the `error` OBJECT is the vendored-mode error shape of the vendor-step fold-in above) and writes nothing — no `.socket/` on a fresh project; human mode prints `Error (): ` on stderr. **Already-vendored exemption**: a purl is exempt from the workspace refusal only when every instance of its `name@version` in `bun.lock` is already a `.socket/vendor/npm/…` local tuple (any uuid; the digest-less 2-tuple counts) — the engine's own criterion — so in-sync re-runs, `repair`, and a superseding patch uuid on a project vendored before it grew a workspace member all flow to the engine (re-pinning an already-local tuple adds no workspace-relative exposure); a wiped ledger alone is not a refusal (the engine path decides). UUID equality in the ledger alone never exempts a purl: `rollback --preserve-state` retains its record after unwiring. Dry-run refusal takes priority over `already_vendored`. **Unreadable vendor ledger**: a `.socket/vendor/state.json` the preflight cannot read or parse is itself the refusal — `vendor_state_unreadable` with the io/parse detail, fail-closed (nothing is exempt) — on the uuid path, the search / `scan` path and the `--dry-run` preview alike; never a Bun lock code. **`--silent`** is "errors only" and never mutes the refusal: the code-tagged `[error] (): ` (per-patch paths) / `Error (): …` (uuid path) line stays on stderr with an empty stdout. **`--dry-run`** previews the refusal as the additive `would_refuse` action (see `--dry-run` below). Agent-mode `get --save-only` is NOT preflighted (record-only intent has no consumption precondition). Pinned by `tests/vendor/in_process_vendor_bun.rs` (exact uuid-path envelope, seeded-manifest survival, `--silent`, `--dry-run`) and `tests/scan_vendor_e2e.rs`. **Lock-text refusals before the download (v5.0)** — shared by `get --mode vendored` on both its paths and `scan --mode vendored`, after the Bun preflight above and the ledger's `already vendored` skip: a `pkg:npm/` result in a **pnpm, yarn classic or yarn berry** project, or a `pkg:cargo/` result, that its vendor backend refuses on the project's lock and manifest text alone is refused BEFORE its patch view is fetched — the pnpm / classic / berry gates the backend runs before it reads the package (coordinates, the lock and manifest reads and their line-ending / version / `cacheKey` / `.yarnrc.yml` gates, override and `resolutions` conflicts, the lock entry present and rewritable) and cargo's `locked_version_mismatch` (only when it is the crate's FIRST refusal; an in-tree `cargo vendor` copy still refuses in the loop as `already_vendored_in_tree`). **Scope:** only a package the vendor loop would hand to its backend is refused early — one installed on disk (the loop's own qualified-aware resolver plus the npm identity lookup), or one the lockfile inventory resolves to a verifiable registry source (a lock entry with an integrity, or the ledger-recovered pre-vendor resolution — exactly the entry the pristine fetch would use). A package absent from the lock and not installed never reached its backend and is untouched: its view is fetched, it downloads, and the vendor loop skips it `skipped` / `package_not_installed` as in v4.x (so cargo's `locked_version_mismatch` is refused early only for a crate installed at the unlocked version). The result becomes `{action:"failed", errorCode:, error:}` in `download.patches[]` / `patches[]` with the backend's exact code and detail, no view and no pristine fetch, no patch record, and therefore no vendor event: compared with v4.x, `download.downloaded` drops and `download.failed` rises by the number of such packages, `vendor.summary.failed` and `vendor.events` lose their `failed` events, and a lockfile-only package among them loses its `vendor_fetched_missing` event (it is never fetched). Exit code and top-level `status` are unchanged (`partial_failure`/1); the nested `vendor.status` becomes `success` when those refusals were the vendor step's only failures (observed on the depscan fixture: 3 refusals, `partialFailure` → `success`), and when every selected package is refused this way the human `scan --vendor` arm prints `Nothing was vendored: N patches failed (see above).`. **Precedence:** the lock-text refusal is decided before the view, so it wins over every view-derived outcome — a package that would also have been a paid-access 403 (`[PAID]`/no access), a failed view fetch, or a no-applicable-files skip reports the lock refusal instead (the Bun refusal and the ledger's `already vendored` skip still come first). The human `[error] (): ` line is printed during the download instead of the vendor step's failure line (the human (non-`--silent`) `scan --vendor` arm's baseline pre-check still fetches the views it verifies; only the download, the pristine fetch and the vendor step skip the package there). A purl the lockfiles pin hosted keeps the loop's refusal (its takeover restore rewrites the lock the gates read); other flavors (package-lock, pnpm-legacy, bun) and ecosystems are untouched, and `--dry-run` is unchanged. `vendor` (manifest-driven, no view fetch) keeps its per-package `failed` events but no longer fetches the pristine source of a lockfile-only package it refuses this way — the source is deferred to the backend, which refuses before reading it (no `vendor_fetched_missing` event and no registry request; a refused package whose registry is unreachable reports the gate's code instead of `vendor_fetch_failed`); only a package the lock resolves to a verifiable source is deferred, and one it does not resolve keeps its `package_not_installed` skip. Pinned by `tests/scan_vendor_e2e.rs` (`exact_download_plan`: scan and exact-purl get, pnpm and cargo scope), `tests/e2e_yarn_legacy_cachekey_refusal_build.rs` and `tests/vendor/vendor_rerun_no_network_e2e.rs`. @@ -1348,7 +1348,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run ` lock`. | | `pypi_poetry_integrity_unverified` | `skipped` (warning) | vendor (pypi / poetry): the lock was written by Poetry < 1.4 (0.12 `[metadata.hashes]`, lock 1.0/1.1, or a 2.0 lock without a `@generated by Poetry X.Y.Z` header — 1.3 wrote those). That installer does not verify local wheel hashes (the committed wheel bytes are the protection) and does not replace an already-installed package at the same version; recreate the virtualenv or `pip uninstall` the package before `poetry install`, or upgrade Poetry. | | `redirect_poetry_stale_install_risk` | `redirect.warnings[]` (warning) | scan `--mode hosted` (poetry): same writer test as above — a warm virtualenv keeps the upstream package after the redirect on Poetry < 1.4 (1.4+ re-installs from the new source); fresh installs pick up the patched wheel. Emitted once per rewritten lock, only on the run that rewrites it. | -| `redirect_poetry_entry_not_found` / `redirect_poetry_missing_sha256` / `redirect_poetry_lock_unsupported` | `redirect.warnings[]` (warning) | scan `--mode hosted` (poetry): the lock has no `[[package]]` at the granted version (uv-parity twin of `redirect_uv_entry_not_found`); the grant carries no SHA-256 (gated once per dep, not per lock); the lock is refused — Poetry 0.12 layout (URL sources ignored), an unsupported `lock-version`, a forked package listed at several versions, a user-authored `[package.source]` on another origin (an earlier Socket URL for the same wheel is superseded in place), a malformed `[metadata.files]`/`[metadata.hashes]`, or a wheel whose filename does not match the locked package. Exit code and `status` unchanged (hosted-refusal posture). A vendored → hosted takeover over a Poetry 0.x lock is refused with this code BEFORE the revert (wet and `--dry-run`): the purl stays vendored and patched and is skipped with this code as `redirect.skipped[].reason`. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), upgrade to Poetry >= 1.0 and re-lock, then re-run `scan --mode hosted`. | +| `redirect_poetry_entry_not_found` / `redirect_poetry_missing_sha256` / `redirect_poetry_lock_unsupported` | `redirect.warnings[]` (warning) | scan `--mode hosted` (poetry): the lock has no `[[package]]` at the granted version (uv-parity twin of `redirect_uv_entry_not_found`); the grant carries no SHA-256 (gated once per dep, not per lock); the lock is refused — Poetry 0.12 layout (URL sources ignored), an unsupported `lock-version`, a forked package listed at several versions, a user-authored `[package.source]` on another origin (an earlier Socket URL for the same wheel is superseded in place), a malformed `[metadata.files]`/`[metadata.hashes]`, or a wheel whose filename does not match the locked package. Exit code and `status` unchanged (hosted-refusal posture). A vendored → hosted takeover over a Poetry 0.x lock is retracted with this code (wet and `--dry-run`, see **Staged takeover**): the purl stays vendored and patched and is skipped with this code as `redirect.skipped[].reason`. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), upgrade to Poetry >= 1.0 and re-lock, then re-run `scan --mode hosted`. | | `redirect_pdm_refused` / `redirect_pdm_legacy_sync_required` | `redirect.warnings[]` (warning) | scan `--mode hosted` (pdm): the `pdm.lock` rewrite was refused — an unsupported `[metadata] lock_version` (the identity-losing `3.1` / `4.0`–`4.2` formats or an untested future format), an unsupported `strategy`, a package listed at several versions (fork) or absent, a user-authored `url`/`path`/VCS/`editable` source, hash-less or malformed `files`, or a wheel whose filename does not match the locked package (`redirect_pdm_refused`); or the lock was written in format `2` (PDM 0.12–1.4), whose upstream freshness bug lets `pdm install` regenerate the lock — use `pdm sync` (`redirect_pdm_legacy_sync_required`). A refused uuid is withheld from every other PyPI rewriter when `pdm.lock` is the install driver, and its patch is not confirmed. Exit code and `status` unchanged (hosted-refusal posture). | | `redirect_bun_lock_unsupported` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (bun): the text lock's `lockfileVersion` is not 0, 1 or 2 (a newer version: update socket-patch, re-locking would reproduce it; no integer: re-lock with Bun ≥ 1.2 — the shared gate's text, identical to vendored's `vendor_lockfile_version_unsupported`), or its `packages` section is not bun's single-line grammar. Nothing rewritten; exit 0 (hosted-refusal posture). | | `redirect_bun_workspace_unsupported` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (bun): a lockfileVersion-0 lock (Bun 1.1.39–1.1.45 `--save-text-lockfile`) holds `workspace:` packages; frozen installs of that grammar cannot keep the hosted tuple. Detail: "Bun version-0 workspace locks cannot preserve hosted tarballs on frozen installs; delete bun.lock and re-run `bun install` with Bun >= 1.2 (which writes lockfileVersion 1, accepted by hosted mode) — a plain in-place `bun install` bumps the version only when a workspace depends on another workspace (e.g. root -> member); otherwise it keeps version 0 or fails to resolve" (measured: Bun 1.2.0 keeps 0, 1.2.23–1.4.2 exit 1 "failed to resolve" on a root that does not depend on its members). Version-1/2 workspace locks are rewritten. Exit 0. | @@ -1356,8 +1356,10 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `redirect_bun_entry_not_found` / `redirect_bun_missing_sha512` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (bun): the lock has no rewritable entry at the granted version (re-resolved, or occupied by an unowned URL/file spec) / the grant carries no sha512 integrity. Per-dep; nothing rewritten for it; exit 0. NOT emitted for the digest-less 2-tuple Bun 1.1.39–1.3.9 re-save our URL tuple as — that entry counts as redirected and is healed. | | `redirect_bun_patched_dependency_skipped` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (bun, `bun.lock` and `bun.lockb`): the project patches the granted `name@version` itself with `bun patch` (a `patchedDependencies` key for `name@version`, or the bare name, in the root `package.json` or mirrored in `bun.lock`). Bun applies that patch only to the registry resolution, so the entry is left on its registry tuple instead of silently losing the user's patch (#367). Per-dep; the detail names the key and the remedy (fold the Socket fix into the user's patch, or drop the `patchedDependencies` entry and re-run); the in-run VEX never assumes the uuid applied. Vendored mode refuses the same package `vendor_lock_entry_unsupported` before any write or download. Exit 0. | | `redirect_vlt_lock_unsupported` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (vlt): `vlt-lock.json` has a `lockfileVersion` other than absent, `0` or `1` (decided on the raw JSON token), is not a JSON object, starts with a UTF-8 BOM, or its `nodes` section is not vlt's one-node-per-line layout. Nothing rewritten; also refuses a vendored → hosted takeover of a `flavor: "vlt"` entry before its revert (`redirect.skipped[].reason`). Exit 0. | -| `redirect_requirements_takeover_unreachable` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (pypi / requirements.txt): a vendored → hosted takeover of a package that vendored mode wired through a pin in a `-r` include, or through a `(transitive)` line it appended to the root `requirements.txt`. Hosted mode only rewrites an existing pin in the root `requirements.txt`, so the takeover is refused before the revert (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `redirect.skipped[].reason`, and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), move the pin from the include into the root `requirements.txt` and delete it from the include (or, for a `(transitive)` line, add an exact `==` pin to the root file), then re-run `scan --mode hosted`. | -| `redirect_uv_takeover_version_unreachable` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (pypi / uv): a vendored → hosted takeover of a package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (vendored uv pins the lock entry down to the patch's version, for example when the lock resolved a newer release). Reverting would bring the lock's own version back, and hosted mode only pins the version the lock resolves, so the takeover is refused before the revert (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `redirect.skipped[].reason`, and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), make the project resolve the patch's version (for example an exact `==` requirement) and re-lock, then re-run `scan --mode hosted`. | +| `redirect_takeover_kept_vendored` | `redirect.warnings[]` (warning) | scan/get `--mode hosted`: a vendored → hosted takeover the hosted rewrite would not pin was retracted (see **Staged takeover**): the package keeps its vendored wiring, ledger entry and artifact byte-identical and stays patched. The detail names the cause code, which is also the purl's `redirect.skipped[].reason`. Exit 0. Replaces v5.0-pre `redirect_takeover_unpatched`, which reported a package left unpatched in both modes and is no longer emitted. | +| `redirect_takeover_not_pinned` | `redirect.skipped[].reason` | scan/get `--mode hosted`: the skip reason of a retracted takeover when no rewriter warning names the cause. | +| `redirect_requirements_takeover_unreachable` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (pypi / requirements.txt): a vendored → hosted takeover of a package that vendored mode wired through a pin in a `-r` include, or through a `(transitive)` line it appended to the root `requirements.txt`. Hosted mode only rewrites an existing pin in the root `requirements.txt`, so the staged takeover is retracted (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `redirect.skipped[].reason`, and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), move the pin from the include into the root `requirements.txt` and delete it from the include (or, for a `(transitive)` line, add an exact `==` pin to the root file), then re-run `scan --mode hosted`. | +| `redirect_uv_takeover_version_unreachable` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (pypi / uv): a vendored → hosted takeover of a package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (vendored uv pins the lock entry down to the patch's version, for example when the lock resolved a newer release). Reverting would bring the lock's own version back, and hosted mode only pins the version the lock resolves, so the staged takeover is retracted (wet and `--dry-run`): the vendored wiring, ledger entry and wheel stay byte-identical, the purl is skipped with this code as `redirect.skipped[].reason`, and nothing is redirected for it. Exit 0. The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), make the project resolve the patch's version (for example an exact `==` requirement) and re-lock, then re-run `scan --mode hosted`. | | `redirect_vlt_missing_sha512` / `redirect_vlt_entry_not_found` / `redirect_vlt_entry_vendored` / `redirect_vlt_unsupported_lock_key` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (vlt): the grant has no sha512 / the lock has no default-registry node for `name@version` / the only match is a vendored `file` node under `.socket/vendor/npm//` / a default-registry instance is outside vlt's node-line grammar or still unpatched after the splice. Per dep; none of the dep's instances is written. `redirect_vlt_missing_sha512` and `redirect_vlt_unsupported_lock_key` refuse the dep: it is never confirmed, whichever lock drives (a sibling lock may still carry its rewritten URL). `redirect_vlt_entry_not_found` and `redirect_vlt_entry_vendored` only say `vlt-lock.json` does not wire it: while vlt drives it is not confirmed; otherwise a sibling lock's rules may confirm it. Exit 0. | | `redirect_vlt_custom_registry_skipped` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (vlt): same-`name@version` nodes under a named alias, a scoped registry or jsr, or git, remote-tarball or local-directory nodes of the same package name (vlt records no version for those; a remote tarball whose `-.tgz` leaf names another version does not count), were left untouched (hosted mode only redirects vlt's default registry). The dep is still redirected, but the run's `--vex` does not attest it, and neither does a later `vex` from the lock alone. | | `redirect_vlt_lockfile_version_missing` / `redirect_vlt_old_lockfile_ignored` / `redirect_vlt_scalar_registry_ignored` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (vlt): the lock has no `lockfileVersion` (vlt ≥ 1.0.0-rc.15 re-resolves it) / a legacy default-registry id without `"modifiers"` in `vlt.json` (vlt 0.0.0-16 … 0.0.0-24 ignore the lock) / a scalar `registry` option that vlt 1.0.0-rc.7 … rc.29 honor over the lock. The deps stay redirected, but the run's `--vex` does not attest them. | @@ -2061,9 +2063,13 @@ the base, the row out of the index, the verification component out when it is st exactly what the planner wrote (else `gradle_verification_component_left`), and the owned files and apply lines once no row is left. -`scan --mode hosted` over a vendored Gradle entry runs this planner's checks first -(`takeover_refusal`, wet and `--dry-run` alike): a refused purl keeps its vendored -patch byte-identical and is skipped with the refusal code, whose detail says so. An +`scan --mode hosted` over a vendored Gradle entry is a staged takeover (see **Staged +takeover** under hosted mode): the vendored revert is staged in memory, this planner +plans against the reverted build, and a dep it refuses is retracted, wet and +`--dry-run` alike. The purl keeps its vendored wiring, ledger entry and +`.socket/vendor/gradle` tree byte-identical (the tree's files are deleted only after +the takeover's commit), is skipped with the refusal code and is reported with +`redirect_takeover_kept_vendored`. An eject (`vendor` over hosted pins) snapshots every Gradle wiring file before it restores, so a failed vendor step rolls the whole build back byte-exact. diff --git a/crates/socket-patch-cli/src/commands/scan/hosted.rs b/crates/socket-patch-cli/src/commands/scan/hosted.rs index fc2e10e47..175b2faf4 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted.rs @@ -22,6 +22,7 @@ use crate::commands::vex::generate_vex_from_manifest_path; use super::{discover_selected, ScanArgs}; mod python; +mod takeover; pub(crate) mod vlt; pub(crate) use vlt::rollback_heal as vlt_rollback_heal; @@ -784,8 +785,8 @@ pub(crate) async fn run_redirect_selected( // The apply lock (see `acquire_hosted_lock`), taken only by a WET run // that holds at least one granted reference — the only runs that can - // write anything: the takeover pre-reverts (lockfiles + the vendored - // ledger) and the lockfile writes. Dry runs + // write anything: the takeover's reverts (lockfiles + the vendored + // ledger) and the lockfile writes, committed together. Dry runs // and zero-grant runs never touch `.socket/`, so they never lock (a // preview must not create `.socket/`, flip to `lock_held` under a // concurrent wet run, or fail on a read-only checkout). Held to the end @@ -813,33 +814,46 @@ pub(crate) async fn run_redirect_selected( // stale-install probes and the in-run VEX attestation. A pre-v5 ledger // on disk is left untouched: it is read only for migration. // The vendored ledger, loaded ONCE per run (under the same lock, so no - // other writer can move the on-disk file under it): the takeover below - // mutates it in place per reverted purl (saving after each), and the - // post-write overlap classification reads that post-takeover state — - // never a pre-takeover snapshot, which would flag every migrated purl - // as still vendored. `Err` (unreadable / malformed) is "no vendored - // ownership known" for both consumers. + // other writer can move the on-disk file under it): a takeover drops + // the migrated entries from it, and the post-write overlap + // classification reads that post-takeover state — never a pre-takeover + // snapshot, which would flag every migrated purl as still vendored. + // `Err` (unreadable / malformed) is "no vendored ownership known" for + // both consumers. let mut vendor_state = socket_patch_core::vendor::load_state(&common.cwd).await; - // Cross-mode takeover of still-vendored purls (see `vendored_takeover`). - let Takeover { - pre_warnings: takeover_pre_warnings, - dry_run: dry_run_takeover, - migrated: takeover_migrated, - unrecorded: takeover_unrecorded, - files: takeover_files, - previews: dry_run_takeover_urls, - } = match vendored_takeover(common, &mut candidates, &mut vendor_state, &mut skipped).await { + // Every project write of this run — a takeover's vendored reverts, the + // hosted pins and the vendored ledger — is staged in one group commit + // and reaches the disk at once, after every check has passed (see + // `socket_patch_core::utils::group_commit`): an exit before the commit + // writes nothing, and a dry run drops the group instead of committing. + // Vendored artifacts the reverts delete go only after the commit. + let group = socket_patch_core::utils::group_commit::GroupCommit::begin(&common.cwd); + group.defer_removals(); + + // Cross-mode takeover of still-vendored purls (see `takeover`): their + // vendored wiring is reverted in the group's overlay, so the rewrite + // below plans against the reverted project. + let mut takeover = match takeover::Takeover::plan( + common, + &group, + &mut candidates, + &vendor_state, + &mut skipped, + ) + .await + { Ok(t) => t, Err(refusal) => return refuse(common, scan_result.take(), &refusal), }; + takeover + .stage(common, &group, &mut candidates, &mut skipped) + .await; - // Read the project's candidate files. Skipped when no candidate - // survived and no dry-run takeover preview is pending (the rewriters do - // nothing without a dep); everything after the rewrite still runs. A - // dry-run takeover preview still needs the root locks for the - // install-policy previews below. - let read = if !candidates.is_empty() || !dry_run_takeover_urls.is_empty() { + // Read the project's candidate files (through the overlay). Skipped + // when no candidate survived (the rewriters do nothing without a dep); + // everything after the rewrite still runs. + let read = if !candidates.is_empty() { engine::read_candidate_files(&view, &std::collections::BTreeSet::new(), &candidates).await } else { CandidateFiles::default() @@ -1015,13 +1029,14 @@ pub(crate) async fn run_redirect_selected( let second_pass = rollout .is_some() .then(|| (read.clone(), python_metadata.clone())); + // A retracted takeover rewrites again without it (see below). + let takeover_metadata = takeover.is_staged().then(|| python_metadata.clone()); let mut done = engine::rewrite( &view, read, &candidates, python_metadata, &vlt_preflight.withheld_from_vlt, - &dry_run_takeover_urls, rewrite_options(), ) .await; @@ -1057,7 +1072,6 @@ pub(crate) async fn run_redirect_selected( &candidates, python_metadata, &vlt_preflight.withheld_from_vlt, - &dry_run_takeover_urls, rewrite_options(), ) .await; @@ -1072,24 +1086,55 @@ pub(crate) async fn run_redirect_selected( } } } + // A staged takeover the rewrite did not pin must not happen: undo every + // staged revert, keep that purl vendored, and stage and rewrite the + // rest again. Each pass drops at least one purl, so this ends. + loop { + let unpinned = takeover.unpinned(&done.confirmed); + if unpinned.is_empty() { + break; + } + takeover + .retract( + common, + &group, + &unpinned, + &done.rewrite.warnings, + &mut candidates, + &mut skipped, + ) + .await; + let read = if candidates.is_empty() { + CandidateFiles::default() + } else { + engine::read_candidate_files(&view, &std::collections::BTreeSet::new(), &candidates) + .await + }; + done = engine::rewrite( + &view, + read, + &candidates, + takeover_metadata.clone().unwrap_or_default(), + &vlt_preflight.withheld_from_vlt, + rewrite_options(), + ) + .await; + } + let takeover::Finished { + warnings: takeover_pre_warnings, + migrated: takeover_migrated, + files: takeover_files, + } = takeover.finish(common.dry_run, &mut vendor_state, &done.rewrite.warnings); + // A takeover writes the vendored ledger: hold the lock for it too. + if lock.is_none() && !common.dry_run && !takeover_migrated.is_empty() { + match acquire_hosted_lock(common, &mut scan_result) { + Ok(guard) => lock = Some(guard), + Err(code) => return code, + } + } // Held to the end of the function. let _lock = lock; - // Dry-run mode-takeover previews were withheld from the rewriters (their - // lock fragments still carry the vendored wiring the wet run reverts - // first), so the presence probe cannot see them: the wet run reverts - // then redirects each one, and the preview's `redirected` count must - // report that outcome. Populated only under --dry-run. - let mut confirmed = done.confirmed.clone(); - confirmed.extend(dry_run_takeover); - // The takeover already reverted these purls' vendored wiring; one the - // rewrite then did not pin (a refused lock, unavailable wheel - // metadata) is left on the unpatched registry release in BOTH modes. - // That must never pass as success. - let mut stranded = stranded_takeovers(&takeover_migrated, &confirmed, common.dry_run); - // A takeover whose revert succeeded but whose ledger update failed is - // refused (never redirected), yet its vendored wiring and artifact are - // already gone: it is unpatched in both modes all the same. - stranded.extend(takeover_unrecorded); + let confirmed = done.confirmed.clone(); // Fetch the full patch view (file hashes + vulnerabilities) for each // CONFIRMED redirect and persist it so a post-install `socket-patch vex` @@ -1153,32 +1198,17 @@ pub(crate) async fn run_redirect_selected( } let rewrite = &done.rewrite; - if !common.dry_run { - for (rel, content) in rewrite - .files - .iter() - .map(|(p, s)| (p, s.as_bytes())) - .chain(rewrite.binary_files.iter().map(|(p, b)| (p, b.as_slice()))) - { - let path = common.cwd.join(rel); - if let Some(parent) = path.parent() { - let _ = std::fs::create_dir_all(parent); - } - // Atomic stage+rename, mode-preserving (the vendored backend's - // writer): a bare `fs::write` truncates first, so a crash - // mid-write could leave a torn lockfile behind. - if let Err(e) = - socket_patch_core::utils::fs::atomic_write_bytes_preserving_mode(&path, content) - .await - { - let message = format!("failed to write {rel}: {e}"); - eprintln!("{}", format_error_line(&message)); - if common.json { - emit_json_error(scan_result.take(), &message); - } - return 1; - } + if common.dry_run { + // A preview: the staged reverts never reach the disk. + drop(group); + } else if let Err(message) = + commit_hosted_writes(common, group, rewrite, &vendor_state, &takeover_migrated).await + { + eprintln!("{}", format_error_line(&message)); + if common.json { + emit_json_error(scan_result.take(), &message); } + return 1; } // Gem stale-install probe (see `gem_stale_install_warnings`): runs after @@ -1392,18 +1422,6 @@ pub(crate) async fn run_redirect_selected( warnings.extend(python_stale.warnings.iter().cloned()); warnings.extend(vlt_stale.warnings.iter().cloned()); warnings.extend(takeover_pre_warnings.iter().cloned()); - warnings.extend(stranded.iter().map(|purl| { - serde_json::json!({ - "code": "redirect_takeover_unpatched", - "detail": format!( - "{purl} was vendored and its vendored wiring was reverted, but it \ - was not pinned to hosted (see the warnings above), so the \ - project now installs the UNPATCHED registry release — fix the \ - reported cause and re-run `scan --mode hosted`, or run `scan \ - --mode vendored` to vendor it again" - ), - }) - })); warnings.extend(takeover_warnings.iter().cloned()); warnings.extend(prune_warnings.iter().cloned()); @@ -1431,9 +1449,6 @@ pub(crate) async fn run_redirect_selected( common.dry_run, ); let mut result = build_redirect_json_envelope(scan_result.take(), redirect); - if !stranded.is_empty() { - result["status"] = serde_json::json!("partial_failure"); - } if let Some(gate) = &rollout { super::finish_rollout_json(gate.stage, &mut result); } @@ -1465,16 +1480,12 @@ pub(crate) async fn run_redirect_selected( // line per sentence so CI can grep them. let width = std::io::IsTerminal::is_terminal(&std::io::stderr()).then(crate::ui::stderr_width); - // A stranded takeover was NOT migrated to hosted: its - // `redirect_takeover_unpatched` warning below says so instead. - for purl in takeover_migrated.iter().filter(|p| !stranded.contains(p)) { + for purl in &takeover_migrated { eprintln!("{}", format_takeover_line(purl, common.dry_run)); } // The files a takeover's revert touched (or, on --dry-run, - // would touch) count alongside the rewriters' own: a dry-run - // takeover is withheld from the rewriters, and a wet revert can - // touch a wiring file the hosted rewriter never rewrites. The - // same union in both modes keeps preview and wet counts equal. + // would touch) count alongside the rewriters' own: a revert can + // touch a wiring file the hosted rewriter never rewrites. let mut human_files = done.rewritten.clone(); human_files.extend(takeover_files.iter().cloned()); human_files.sort(); @@ -1565,9 +1576,7 @@ pub(crate) async fn run_redirect_selected( if let Some(line) = rollout_line { println!("{line}"); } - // "Commit … to keep the hosted patches" / "reinstall" would be - // wrong for a stranded takeover, whose warning names the remedy. - let mut next_steps = if common.dry_run || !stranded.is_empty() { + let mut next_steps = if common.dry_run { Vec::new() } else { format_next_steps(&human_files, &rewrite.edits, !takeover_migrated.is_empty()) @@ -1582,669 +1591,115 @@ pub(crate) async fn run_redirect_selected( if let Some(e) = &vex_error { e.print_embedded(common); } - if common.silent { - for w in warnings - .iter() - .filter(|w| w["code"] == "redirect_takeover_unpatched") - { - eprintln!( - "{}", - format_warning( - "redirect_takeover_unpatched", - w["detail"].as_str().unwrap_or_default(), - None - ) - ); - } - } - } - if vex_code == 0 && !stranded.is_empty() { - return 1; } vex_code } -/// The purls a WET takeover migrated (vendored wiring reverted) that the -/// rewrite did not confirm as pinned. Empty under `--dry-run`, whose -/// takeover previews are counted as confirmed without a rewrite. -fn stranded_takeovers( - migrated: &[String], - confirmed: &[(String, String)], - dry_run: bool, -) -> Vec { - use socket_patch_core::utils::purl_key::PurlKey; - if dry_run { - return Vec::new(); - } - let pinned: std::collections::HashSet = confirmed - .iter() - .map(|(purl, _)| PurlKey::new(purl)) - .collect(); - migrated - .iter() - .filter(|purl| !pinned.contains(&PurlKey::new(purl))) - .cloned() - .collect() -} - -/// Cross-mode takeover: a purl this run is about to redirect may still be -/// VENDORED — for cargo a committed `[patch.crates-io]` path entry, a -/// detached Cargo.lock entry, a committed copy, and a vendored ledger -/// entry; for the npm family a `file:./.socket/vendor/…` lock resolution -/// (plus a berry `resolutions` pin) and its committed tarball; for golang -/// the vendor-owned go.mod `replace`, its committed module copy, and its -/// ledger entry. The hosted rewriters know nothing about that wiring -/// (cargo would refuse `--locked` builds over the unused `[patch]` entry; -/// yarn classic would hijack a resolution the vendored ledger still -/// claims; yarn berry refuses `file:` outright). A takeover must leave the -/// project FULLY hosted: revert each such purl's vendored state first (the -/// per-purl machinery `vendor --revert` runs), and only then redirect — -/// which also hands the redirect the PRISTINE registry lock fragment to -/// record as its own revert original. A purl -/// whose vendored state cannot be cleanly reverted (revert failure, or -/// vendored wiring with a missing/corrupt ledger) is REFUSED — skipped -/// with an actionable error — never half-migrated. -/// -/// Refused purls are moved from `candidates` into `skipped`; dry-run -/// takeover previews leave `candidates` too (see [`Takeover::dry_run`]). -/// `Err` is the symlinked-wiring refusal (nothing was written). -async fn vendored_takeover( +/// Write the hosted pins and commit `group` (see `run_redirect_selected`): +/// the takeover's staged reverts, the pins and the vendored ledger reach +/// the disk together. A file the group does not capture (the Gradle hosted +/// index and script under `.socket/gradle/`, which only the captured settings +/// line makes live) is written first, straight to disk, and put back when a +/// later step fails. `Err` is the error line: on a failed commit nothing was +/// left changed, or an interrupted commit's journal is kept for the next +/// locked command to finish. +async fn commit_hosted_writes( common: &crate::args::GlobalArgs, - candidates: &mut Vec, - vendor_state: &mut std::io::Result, - skipped: &mut Vec, -) -> Result { - use socket_patch_core::hosted::engine::{Candidate, SkippedPatch, TakeoverPreview}; - let mut out = Takeover::default(); - // Which root locks each dry-run takeover purl is vendored into (from - // its vendor ledger wiring): the wet run reverts that wiring and then - // splices the hosted URL there, so the install-policy auto-configs - // (npm `.npmrc` allow-remote, pnpm `trustLockfile`) must be PREVIEWED - // for those locks even though the rewriters never see these purls. - let mut dry_run_locks: std::collections::HashMap> = - std::collections::HashMap::new(); - // Maven takes over only a Gradle build's vendored JVM entry (its revert - // unplans the vendored Gradle wiring); a pom-only vendored entry stays. - // PyPI: every Python rewriter (requirements.txt, Poetry, Pipenv, uv, - // Hatch, PDM, pylock) refuses a non-registry source as user-authored, - // including the vendored one socket-patch wrote itself, so a vendored - // purl must be reverted to its registry entry first (#328). - let takeover_capable = |p: &str| { - p.starts_with("pkg:cargo/") - || p.starts_with("pkg:npm/") - || p.starts_with("pkg:golang/") - || p.starts_with("pkg:pypi/") - || p.starts_with("pkg:maven/") - }; - let gradle_jvm_entry = |entry: &socket_patch_core::vendor::VendorEntry| { - entry.ecosystem == socket_patch_core::vendor::jvm::layout::LEDGER_ECOSYSTEM - && entry.wiring.iter().any(|w| { - w.file.ends_with(".gradle") - || w.file.ends_with(".gradle.kts") - || w.file == socket_patch_core::vendor::jvm::gradle::INDEX_REL - }) - }; - if !candidates.iter().any(|c| takeover_capable(&c.purl)) { - // No takeover-capable candidates — nothing to reconcile. - return Ok(out); - } - use socket_patch_core::utils::purl::strip_purl_qualifiers; - use socket_patch_core::utils::purl_key::PurlKey; - // Each takeover-capable candidate with its vendored ledger entry, if - // any (cloned out so the loop can mutate the state). - let takeover: Vec<(&Candidate, Option)> = candidates + group: socket_patch_core::utils::group_commit::GroupCommit, + rewrite: &socket_patch_core::patch::redirect::RewriteResult, + vendor_state: &std::io::Result, + takeover_migrated: &[String], +) -> Result<(), String> { + use socket_patch_core::utils::group_commit::{captures, is_pending}; + let files: Vec<(&String, &[u8])> = rewrite + .files .iter() - .filter(|c| takeover_capable(&c.purl)) - .map(|c| { - let entry = vendor_state - .as_ref() - .ok() - .and_then(|s| { - socket_patch_core::vendor::lookup_entry( - &s.entries, - strip_purl_qualifiers(&c.purl), - ) - }) - .cloned(); - (c, entry) - }) - .filter(|(c, entry)| { - !c.purl.starts_with("pkg:maven/") || entry.as_ref().is_some_and(gradle_jvm_entry) - }) + .map(|(p, s)| (p, s.as_bytes())) + .chain(rewrite.binary_files.iter().map(|(p, b)| (p, b.as_slice()))) .collect(); - if takeover.is_empty() { - return Ok(out); - } - // Compatibility must be known before the takeover removes a live - // patch. In particular, a v0 workspace can keep an existing local - // tuple even though hosted mode cannot replace it with a URL. Only - // an npm purl WITH a vendored entry can be taken over, so the bun - // locks are read here only when one exists — the candidate-file - // read below covers every other run. - let bun_takeover_refusal = if takeover - .iter() - .any(|(c, entry)| entry.is_some() && c.purl.starts_with("pkg:npm/")) - { - match socket_patch_core::utils::fs::read_regular_to_string(&common.cwd.join("bun.lock")) - .await - { - Ok(content) => socket_patch_core::patch::redirect::preflight_bun_hosted(&content).err(), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => { - match socket_patch_core::utils::fs::read_regular_to_bytes_sync( - &common.cwd.join("bun.lockb"), - ) { - Ok(bytes) => { - socket_patch_core::patch::redirect::preflight_bun_binary(&bytes).err() - } - Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, - Err(e) => Some(socket_patch_core::patch::redirect::RewriteWarning { - code: "redirect_bun_lockb_invalid".into(), - detail: format!("cannot read bun.lockb: {e}"), - }), + // Uncaptured files first: the commit below is the step that makes the + // pins live. Each one's previous bytes are kept to put it back. + let (staged, direct): (Vec<_>, Vec<_>) = files.into_iter().partition(|(rel, _)| captures(rel)); + let mut written: Vec<(std::path::PathBuf, Option>)> = Vec::new(); + let result = async { + for (rel, content) in direct.into_iter().chain(staged) { + let path = common.cwd.join(rel); + if !captures(rel) { + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent) + .map_err(|e| format!("failed to create the directory of {rel}: {e}"))?; } + let previous = + match socket_patch_core::utils::fs::read_regular_to_bytes(&path).await { + Ok(bytes) => Some(bytes), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, + Err(e) => { + return Err(format!("failed to read {rel}: {e} (nothing was changed)")) + } + }; + written.push((path.clone(), previous)); } - Err(e) => Some(socket_patch_core::patch::redirect::RewriteWarning { - code: "redirect_bun_lock_unsupported".into(), - detail: format!("cannot read bun.lock before mode takeover: {e}"), - }), - } - } else { - None - }; - // Yarn berry twin of the bun gate: the berry rewriter's project-level - // refusals (mixed yarn.lock / package.json line endings, cacheKey, - // `.yarnrc.yml` compressionLevel) must be known before the takeover reverts a - // vendored berry purl, or the revert strips the live vendored patch - // and the rewriter then refuses the lock. Only entries the - // vendor ledger wired through the yarn-berry backend are gated (the - // lock is read only when one exists); an unreadable lock is left to - // the revert's own diagnostics. - let berry_entry = |entry: &socket_patch_core::vendor::VendorEntry| { - entry.ecosystem == "npm" && entry.flavor.as_deref() == Some("yarn-berry") - }; - let berry_takeover_refusal = if takeover - .iter() - .any(|(_, entry)| entry.as_ref().is_some_and(berry_entry)) - { - match socket_patch_core::utils::fs::read_regular_to_string(&common.cwd.join("yarn.lock")) - .await - { - Ok(lock) => { - let yarnrc = socket_patch_core::utils::fs::read_regular_to_string( - &common.cwd.join(".yarnrc.yml"), - ) - .await - .ok(); - let manifest = socket_patch_core::utils::fs::read_regular_to_string( - &common.cwd.join("package.json"), - ) + // Atomic stage+rename, mode-preserving (the vendored backend's + // writer); captured by the group until the commit. + socket_patch_core::utils::fs::atomic_write_bytes_preserving_mode(&path, content) .await - .ok(); - socket_patch_core::patch::redirect::preflight_yarn_berry_hosted( - &lock, - manifest.as_deref(), - yarnrc.as_deref(), - ) - .err() - } - Err(_) => None, + .map_err(|e| format!("failed to write {rel}: {e} (nothing was changed)"))?; } - } else { - None - }; - // Yarn classic twin: an offline mirror refuses the hosted rewrite - // (vendored mode works with one), so a vendored yarn classic entry - // must stay vendored rather than be reverted into neither mode. - let classic_entry = |entry: &socket_patch_core::vendor::VendorEntry| { - entry.ecosystem == "npm" && entry.flavor.as_deref() == Some("yarn-classic") - }; - let classic_takeover_refusal = if takeover - .iter() - .any(|(_, entry)| entry.as_ref().is_some_and(classic_entry)) - { - match socket_patch_core::utils::fs::read_regular_to_string(&common.cwd.join("yarn.lock")) - .await - { - Ok(lock) => { - let read_rc = |rel: &str| { - let path = common.cwd.join(rel); - async move { - socket_patch_core::utils::fs::read_regular_to_string(&path) - .await - .ok() - } - }; - let yarnrc = read_rc(socket_patch_core::patch::redirect::YARNRC_REL).await; - let npmrc = read_rc(socket_patch_core::patch::redirect::npmrc::NPMRC_REL).await; - socket_patch_core::patch::redirect::preflight_yarn_classic_hosted( - &lock, - yarnrc.as_deref(), - npmrc.as_deref(), - &resolve_outer_yarn_mirror_for_process(&common.cwd), - ) - .err() - } - Err(_) => None, - } - } else { - None - }; - // vlt twin: the hosted rewriter's lock-level refusal must be known - // before a vendored vlt entry is reverted, or the revert strips the - // live vendored patch and the rewrite then refuses the lock. - let vlt_entry = |entry: &socket_patch_core::vendor::VendorEntry| { - entry.ecosystem == "npm" && entry.flavor.as_deref() == Some("vlt") - }; - let vlt_takeover_refusal = if takeover - .iter() - .any(|(_, entry)| entry.as_ref().is_some_and(vlt_entry)) - { - match socket_patch_core::utils::fs::read_regular_to_string( - &common - .cwd - .join(socket_patch_core::constants::npm_family::VLT_LOCK), - ) - .await - { - Ok(lock) => { - let files = std::collections::BTreeMap::from([( - socket_patch_core::constants::npm_family::VLT_LOCK.to_string(), - lock, - )]); - socket_patch_core::patch::redirect::vlt::preflight_vlt_hosted(&files).err() - } - Err(_) => None, - } - } else { - None - }; - // Gradle twin: the hosted Gradle planner refuses builds and grants the - // vendored backend accepts (a custom `lockFile`, a settings-classpath - // GA, a same-GAV or incomplete grant, ...). Each refusal must be known - // before the revert strips the live vendored patch, or the planner - // then writes nothing and the build resolves the unpatched upstream. - // Every Gradle JVM takeover purl is checked against the build on disk. - let gradle_takeover_refusals: std::collections::HashMap< - String, - socket_patch_core::patch::redirect::RewriteWarning, - > = if takeover.iter().any(|(c, entry)| { - c.purl.starts_with("pkg:maven/") && entry.as_ref().is_some_and(gradle_jvm_entry) - }) { - let build = - socket_patch_core::patch::redirect::gradle::read_build_from_disk(&common.cwd).await; - takeover - .iter() - .filter(|(c, entry)| { - c.purl.starts_with("pkg:maven/") && entry.as_ref().is_some_and(gradle_jvm_entry) - }) - .filter_map(|(c, _)| { - socket_patch_core::patch::redirect::gradle::takeover_refusal( - &build.files, - &build.unreadable, - &c.dep, - ) - .map(|w| (c.purl.clone(), w)) - }) - .collect() - } else { - std::collections::HashMap::new() - }; - // A PyPI entry is gated on the hosted rewriter's reach after the - // revert: requirements.txt pins only the root file (#699), uv pins - // only the version the restored lock resolves (#723), and Poetry - // refuses every 0.x lock (#945). Checked once per purl, here, because - // the uv and Poetry checks read the ledger and the lock on disk. - let mut pypi_takeover_refusals: std::collections::HashMap< - String, - socket_patch_core::patch::redirect::RewriteWarning, - > = std::collections::HashMap::new(); - for (c, entry) in &takeover { - let Some(entry) = entry.as_ref().filter(|_| c.purl.starts_with("pkg:pypi/")) else { - continue; - }; - if let Err(warning) = - socket_patch_core::patch::redirect::preflight_pypi_takeover(&common.cwd, entry).await - { - pypi_takeover_refusals.insert(c.purl.clone(), warning); - } - } - // The takeover refusal (if any) for one candidate: bun gates every - // npm purl, berry and vlt only their own vendored entries, Gradle each - // of its own purls, a pypi purl on a platform-tagged grant (#701), and a - // PyPI entry on the hosted rewriter's reach (`pypi_takeover_refusals` - // above). Berry also runs - // the rewriter's per-dep grant gate (a grant without the berry cache - // checksum is skipped by the rewriter, so reverting first would leave - // the package in neither mode). A refused purl is never dispatched (see - // the loop), so its wiring is not a write target here. - let takeover_refusal = |c: &Candidate, - entry: Option<&socket_patch_core::vendor::VendorEntry>| - -> Option { - if c.purl.starts_with("pkg:maven/") { - return gradle_takeover_refusals.get(&c.purl).cloned(); - } - if c.purl.starts_with("pkg:pypi/") { - // A platform-tagged grant is never pinned (#701 / #932): keep - // the vendored patch rather than revert it to nothing. - return entry.and_then(|_| { - socket_patch_core::patch::redirect::pypi_platform_wheel_refusal(&c.dep) - .or_else(|| pypi_takeover_refusals.get(&c.purl).cloned()) - }); - } - if !c.purl.starts_with("pkg:npm/") { - return None; - } - let berry = entry.is_some_and(berry_entry); - bun_takeover_refusal - .clone() - .or_else(|| berry_takeover_refusal.clone().filter(|_| berry)) - .or_else(|| { - berry - .then(|| { - socket_patch_core::patch::redirect::preflight_yarn_berry_hosted_dep(&c.dep) - .err() - }) - .flatten() - }) - .or_else(|| { - classic_takeover_refusal - .clone() - .filter(|_| entry.is_some_and(classic_entry)) - }) - .or_else(|| { - vlt_takeover_refusal - .clone() - .filter(|_| entry.is_some_and(vlt_entry)) - }) - }; - // NON-UTF-8 PRE-CHECK (#721) — the GUARD's undecodable-file rule - // (`engine::undecodable_guard`), checked BEFORE any revert dispatches - // (and under --dry-run too): a takeover that reverted first and was - // then refused by the guard would leave the reverted purls unpatched - // in both modes. - if takeover.iter().any(|(_, entry)| entry.is_some()) { - let view = socket_patch_core::vendor::lock_inventory::ProjectView::Disk(&common.cwd); - let read = socket_patch_core::hosted::engine::read_candidate_files( - &view, - &std::collections::BTreeSet::new(), - candidates, - ) - .await; - if let Some(refusal) = socket_patch_core::hosted::engine::undecodable_guard( - &read.undecodable_reads, - candidates, - ) { - return Err(refusal); - } - } - // SYMLINK PRE-CHECK for the takeover reverts — the same rule as the - // SYMLINK GUARD below, applied to each ledger entry's recorded wiring - // (the revert backends also stage and rename over the file). Checked - // BEFORE any revert dispatches (and under --dry-run too) so "nothing - // was written" stays true. - let revert_targets = takeover - .iter() - .filter_map(|(c, entry)| { - entry - .as_ref() - .filter(|e| takeover_refusal(c, Some(e)).is_none()) - }) - .flat_map(|entry| entry.wiring.iter().map(|w| w.file.as_str())); - if let Some(linked) = - socket_patch_core::utils::fs::first_symlink(&common.cwd, revert_targets).await - { - return Err(socket_patch_core::hosted::engine::symlink_refusal(linked)); - } - let mut refused: Vec = Vec::new(); - for (candidate, ledger_entry) in &takeover { - let purl = &candidate.purl; - let uuid = &candidate.dep.patch_uuid; - if let Some(entry) = ledger_entry { - if let Some(warning) = takeover_refusal(candidate, Some(entry)) { - refused.push(purl.clone()); - // Project-level refusals repeat per purl; report each once. - let warning = serde_json::json!(warning); - if !out.pre_warnings.contains(&warning) { - out.pre_warnings.push(warning); - } - continue; - } - if common.dry_run { - // Preview through the same per-purl revert machinery the - // wet run dispatches (write-free under dry_run): a - // vendored state the wet run would refuse to revert is - // refused here too, and one it would revert is announced - // as a takeover — never handed to the rewriters, which - // would refuse the still-vendored wiring. - let outcome = - crate::commands::vendor::dispatch_revert_one(entry, &common.cwd, true).await; - if outcome.success && revert_keeps_wiring(&outcome) { - refused.push(purl.clone()); - out.pre_warnings.push(drifted_takeover_warning(purl)); - continue; - } - if !outcome.success { - refused.push(purl.clone()); - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_vendored_revert_failed", - "detail": format!( - "{purl} is vendored and its vendored state could not be \ - reverted ({}); NOT switched to hosted — run `socket-patch vendor \ - --revert` to clean up, then re-run `scan --mode hosted`", - outcome.error.as_deref().unwrap_or("unknown error") - ), - })); - continue; - } - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_would_revert_vendored", - "detail": format!( - "{purl} is currently vendored; the hosted wiring will \ - revert its vendored wiring, ledger entry, and committed \ - artifact first, then switch to hosted (mode takeover)" - ), - })); - out.dry_run.push((purl.clone(), uuid.clone())); - out.migrated.push(purl.clone()); - out.files - .extend(entry.wiring.iter().map(|w| w.file.clone())); - dry_run_locks.insert( - purl.clone(), - entry.wiring.iter().map(|w| w.file.clone()).collect(), - ); - continue; - } - let outcome = - crate::commands::vendor::dispatch_revert_one(entry, &common.cwd, false).await; - if !outcome.success { - refused.push(purl.clone()); - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_vendored_revert_failed", - "detail": format!( - "{purl} is vendored and its vendored state could not be \ - reverted ({}); NOT switched to hosted — run `socket-patch vendor \ - --revert` to clean up, then re-run `scan --mode hosted`", - outcome.error.as_deref().unwrap_or("unknown error") - ), - })); - continue; - } - if revert_keeps_wiring(&outcome) { - // A wiring record drifted and was left in place, so the - // project may still resolve through the vendored artifact - // and the ledger entry holds the only recorded originals - // (the RevertOutcome contract): keep both and refuse, - // exactly as `vendor --revert` reports it skipped. - refused.push(purl.clone()); - out.pre_warnings.push(drifted_takeover_warning(purl)); - continue; - } - // Drop the reverted entry from the in-memory ledger and - // persist per purl so a crash mid-run leaves a ledger - // matching the on-disk wiring. The entry stays dropped even - // when the save fails: its wiring and artifact ARE gone, so - // a later successful save in this loop writes the truth. - let state = vendor_state - .as_mut() - .expect("a vendored ledger entry was looked up in this state, so it loaded"); - state - .entries - .retain(|k, e| !PurlKey::same(k, purl) && !PurlKey::same(&e.base_purl, purl)); - if let Err(e) = socket_patch_core::vendor::save_state(&common.cwd, state).await { - // The wiring is reverted but the ledger still claims it; - // redirecting now would leave a ledger asserting wiring - // that is gone. Fail closed for this purl — and since its - // vendored wiring is already gone, report it as stranded. - refused.push(purl.clone()); - out.unrecorded.push(purl.clone()); - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_vendored_revert_failed", - "detail": format!( - "{purl}: vendored wiring reverted but the vendored ledger \ - could not be updated ({e}); NOT switched to hosted — fix \ - .socket/vendor/state.json and re-run" - ), - })); - continue; - } - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_takeover_reverted_vendored", - "detail": format!( - "{purl} was vendored; reverted its vendored wiring, ledger \ - entry, and committed artifact before switching to hosted (mode \ - takeover: the project is now fully hosted for this package)" - ), - })); - out.pre_warnings.extend( - outcome - .warnings - .iter() - .filter(|w| w.code == socket_patch_core::vendor::vlt_lock::REINSTALL_REQUIRED) - .map(|w| serde_json::json!({ "code": w.code, "detail": w.detail })), - ); - out.migrated.push(purl.clone()); - out.files - .extend(entry.wiring.iter().map(|w| w.file.clone())); - } else { - // No usable ledger entry. If socket-owned vendored wiring for - // this crate is nevertheless present, the ledger is missing or - // corrupt — the originals needed to revert are unrecoverable, - // so redirecting on top would wedge the project. Refuse. - // (Cargo-only probe: Socket-owned `[patch.crates-io]` entries - // for exactly this name@version in the root Cargo.toml or a - // legacy `.cargo/config*` — another vendored version of the - // crate has its own ledger entry. An npm purl in this state - // falls through to the rewriters' own per-flavor - // diagnostics.) - let coords = purl - .starts_with("pkg:cargo/") - .then(|| purl_parts(purl).map(|(_, name, version)| (name, version))) - .flatten(); - let wired = match &coords { - Some((n, v)) => { - socket_patch_core::vendor::cargo::socket_wiring_present(&common.cwd, n, v).await - } - None => false, - }; - if wired { - refused.push(purl.clone()); - out.pre_warnings.push(serde_json::json!({ - "code": "redirect_vendored_revert_failed", - "detail": format!( - "{purl} has socket-owned vendored `[patch.crates-io]` \ - wiring but no usable vendored ledger entry \ - (.socket/vendor/state.json is missing or corrupt); NOT \ - redirected — restore the ledger or remove the vendored \ - wiring manually, then re-run" - ), - })); + if !takeover_migrated.is_empty() { + if let Ok(state) = vendor_state { + socket_patch_core::vendor::save_state(&common.cwd, state) + .await + .map_err(|e| { + format!( + "failed to update .socket/vendor/state.json: {e} (nothing was \ + changed: the vendored packages stay vendored)" + ) + })?; } } + Ok(()) } - for purl in &refused { - if let Some((c, entry)) = takeover.iter().find(|(c, _)| &c.purl == purl) { - let reason = takeover_refusal(c, entry.as_ref()) - .map_or_else(|| "vendored_revert_failed".to_string(), |w| w.code); - skipped.push(SkippedPatch::new(purl, &c.dep.patch_uuid, &reason)); - } + .await; + if let Err(e) = result { + put_back(&written).await; + return Err(e); } - // Purls leaving the rewrite set: refused takeovers, plus the dry-run - // takeover previews (still vendored on disk — the wet run reverts - // them before the rewriters ever see their files). - let withheld: std::collections::HashSet<&str> = refused - .iter() - .map(String::as_str) - .chain(out.dry_run.iter().map(|(p, _)| p.as_str())) - .collect(); - if !withheld.is_empty() { - // Keep the dry-run takeover candidates' URLs (and the root locks - // their purl is vendored into) for the install-policy previews. - for (purl, _) in &out.dry_run { - let locks = dry_run_locks.get(purl).cloned().unwrap_or_default(); - for c in candidates.iter().filter(|c| &c.purl == purl) { - out.previews.push(TakeoverPreview { - artifact_url: c.dep.artifact_url.clone(), - locks: locks.clone(), - }); - } + // A takeover's commit spans the project and the vendored ledger, so it + // is journaled (`.socket/vendor/` is in use anyway). A hosted-only run + // writes nothing under `.socket/vendor/`: its files are replaced one by + // one and put back if one fails. + let committed = if takeover_migrated.is_empty() { + group.commit_unjournaled().await + } else { + group.commit().await + }; + match committed { + Ok(_) => Ok(()), + // The journal finishes the commit, which the direct writes belong to. + Err(e) if is_pending(&e) => Err(format!( + "failed to write the hosted pins: {e} (the interrupted write is journaled; \ + the next socket-patch command finishes it)" + )), + Err(e) => { + put_back(&written).await; + Err(format!( + "failed to write the hosted pins: {e} (nothing was changed)" + )) } - candidates.retain(|c| !withheld.contains(c.purl.as_str())); } - Ok(out) -} - -/// Whether a takeover revert left (or, on `--dry-run`, would leave) vendored -/// wiring in place: a drift-skipped record, or a reverted file that still -/// references the artifact dir. The backends compute both signals on dry -/// runs too, while `kept_artifact` itself is set only on wet runs. -fn revert_keeps_wiring(outcome: &socket_patch_core::vendor::RevertOutcome) -> bool { - outcome.kept_artifact - || outcome.drift_skipped() - || outcome - .warnings - .iter() - .any(|w| w.code == "vendor_revert_residual_reference") -} - -/// The refusal for a takeover whose vendored wiring drifted since vendoring. -fn drifted_takeover_warning(purl: &str) -> serde_json::Value { - serde_json::json!({ - "code": "redirect_vendored_revert_failed", - "detail": format!( - "{purl} is vendored and part of its vendored wiring was edited since \ - vendoring, so it is left in place; NOT switched to hosted — restore or \ - remove that wiring (`socket-patch vendor --revert` lists it), then re-run \ - `scan --mode hosted`" - ), - }) } -/// What [`vendored_takeover`] did (or, on `--dry-run`, would do). -#[derive(Default)] -struct Takeover { - /// Its warnings, reported after the rewriters' own. - pre_warnings: Vec, - /// Dry-run takeover previews: `(purl, uuid)` pairs whose vendored state - /// the wet run would revert and then redirect. Withheld from the - /// rewriters (their lock fragments still carry the vendored wiring the - /// wet run reverts FIRST) and counted as redirected, so the preview's - /// envelope matches the wet run's outcome. - dry_run: Vec<(String, String)>, - /// Human output: the purls migrated (or, on --dry-run, to be migrated) - /// from vendored to hosted. - migrated: Vec, - /// Wet takeovers whose vendored wiring was reverted but whose ledger - /// update then failed: refused (never redirected), so unpatched in - /// both modes. - unrecorded: Vec, - /// The files their revert touches (or would touch). Both modes count - /// `rewritten ∪ files`, so the preview's file count matches the wet - /// run's even for wiring files the hosted rewriter does not also - /// rewrite (a Gemfile line, a uv source). - files: std::collections::BTreeSet, - /// The withheld dry-run takeover candidates' artifact URLs and wired - /// root locks, for the install-policy previews. - previews: Vec, +/// Put the files [`commit_hosted_writes`] wrote straight to disk back to +/// their previous bytes (removing the ones it created), best-effort. +async fn put_back(written: &[(std::path::PathBuf, Option>)]) { + for (path, previous) in written.iter().rev() { + let _ = match previous { + Some(bytes) => { + socket_patch_core::utils::fs::atomic_write_bytes_preserving_mode(path, bytes).await + } + None => tokio::fs::remove_file(path).await, + }; + } } // ── Human-output formatting ──────────────────────────────────────────────── @@ -2689,7 +2144,13 @@ fn created_settings_over_existing( || base == SBT_HOSTED_FILE) && !done.files.contains_key(rel.as_str()) }) - .find(|rel| std::fs::symlink_metadata(cwd.join(rel)).is_ok()) + // Through the run's group commit: a takeover's staged revert may + // have removed a settings file the vendored wiring created. + .find(|rel| { + let path = cwd.join(rel); + socket_patch_core::utils::group_commit::exists(&path) + .unwrap_or_else(|| std::fs::symlink_metadata(&path).is_ok()) + }) .map(|rel| { let sbt = rel.rsplit('/').next() == Some(SBT_HOSTED_FILE); socket_patch_core::hosted::engine::Refusal { diff --git a/crates/socket-patch-cli/src/commands/scan/hosted/takeover.rs b/crates/socket-patch-cli/src/commands/scan/hosted/takeover.rs new file mode 100644 index 000000000..c4225f508 --- /dev/null +++ b/crates/socket-patch-cli/src/commands/scan/hosted/takeover.rs @@ -0,0 +1,581 @@ +//! The staged vendored → hosted mode takeover. +//! +//! A purl this run is about to redirect may still be VENDORED: for cargo a +//! committed `[patch.crates-io]` path entry, a detached Cargo.lock entry and +//! a committed copy; for the npm family a `file:./.socket/vendor/…` lock +//! resolution (plus a berry `resolutions` pin) and its committed tarball; +//! for golang the vendor-owned go.mod `replace` and its module copy; for +//! pypi the vendored lock / requirements source; for a Gradle build the +//! vendored JVM wiring — each with its vendored ledger entry. The hosted +//! rewriters cannot pin over that wiring, so the takeover reverts it first +//! (the per-purl machinery `vendor --revert` runs), which also hands the +//! rewriter the PRISTINE registry lock fragment to pin. +//! +//! Every step is staged in the run's [`GroupCommit`], never on disk: +//! +//! 1. [`Takeover::plan`] picks the purls in reach +//! ([`socket_patch_core::hosted::takeover::in_reach`]) and refuses the +//! ones that cannot be reverted at all (linked wiring, cargo wiring with +//! no ledger entry). +//! 2. [`Takeover::stage`] reverts each purl into the overlay, under a +//! savepoint: a revert that fails or keeps drifted wiring is rolled back +//! and refused. A yarn-berry entry is first checked against the berry +//! project gates on the pre-revert project (its revert re-renders +//! `package.json`). The artifact deletions wait for the commit +//! ([`GroupCommit::defer_removals`]). +//! 3. The hosted rewrite reads the overlay, so it plans against the +//! reverted project. A staged purl it does not pin is RETRACTED +//! ([`Takeover::retract`]): the whole overlay goes back to its pre-revert +//! state, the purl is refused (it stays vendored, byte for byte) and the +//! others are staged and rewritten again. +//! 4. The caller writes the hosted pins into the same overlay, saves the +//! vendored ledger without the migrated entries, and commits once: the +//! revert and the hosted pins reach the disk together or not at all. +//! +//! A dry run runs the same steps and drops the overlay instead of +//! committing, so it reports exactly what the wet run would do. + +use std::collections::BTreeSet; + +use socket_patch_core::hosted::engine::{Candidate, Refusal, SkippedPatch}; +use socket_patch_core::patch::redirect::RewriteWarning; +use socket_patch_core::utils::group_commit::{GroupCommit, Savepoint}; +use socket_patch_core::utils::purl::{purl_parts, strip_purl_qualifiers}; +use socket_patch_core::utils::purl_key::PurlKey; +use socket_patch_core::vendor::{RevertOutcome, VendorEntry, VendorState}; + +/// The warning that announces a staged takeover the run did not complete: +/// the purl stays vendored. +pub(super) const KEPT_VENDORED: &str = "redirect_takeover_kept_vendored"; + +/// The skip reason of a staged takeover whose purl the hosted rewrite did +/// not pin, when no rewriter warning names the cause. +const NOT_PINNED: &str = "redirect_takeover_not_pinned"; + +/// One purl whose vendored state the overlay holds reverted. +struct Staged { + purl: String, + uuid: String, + entry: VendorEntry, + /// The revert's own advisories that outlive the takeover (vlt's + /// reinstall notice). + advisories: Vec, +} + +/// A vendored → hosted takeover in progress (see the module docs). +pub(super) struct Takeover { + /// The overlay as it was before any revert: what [`Self::retract`] + /// rolls back to. + base: Option, + /// The purls to revert, with their ledger entries, in candidate order. + attempts: Vec<(String, String, VendorEntry)>, + staged: Vec, + /// Refusals, in order. + warnings: Vec, + /// Rewriter warnings that explained a retracted purl (the final rewrite + /// no longer reports them once the purl left the rewrite set). + explained: Vec, +} + +/// What a finished takeover did (or, on `--dry-run`, would do). +pub(super) struct Finished { + /// Refusals and the per-purl takeover announcements, reported after the + /// rewriters' own warnings. + pub warnings: Vec, + /// The purls migrated (or to be migrated) from vendored to hosted. + pub migrated: Vec, + /// The files their revert touches. Human output counts `rewritten ∪ + /// files`: a revert can touch a wiring file the hosted rewriter does not + /// also rewrite (a Gemfile line, a uv source). + pub files: BTreeSet, +} + +impl Takeover { + /// Pick the candidates in a takeover's reach (step 1). Refused purls + /// move from `candidates` into `skipped`. `Err` is the linked-wiring + /// refusal: nothing was written. + pub(super) async fn plan( + common: &crate::args::GlobalArgs, + group: &GroupCommit, + candidates: &mut Vec, + vendor_state: &std::io::Result, + skipped: &mut Vec, + ) -> Result { + use socket_patch_core::hosted::takeover; + let mut out = Takeover { + base: Some(group.savepoint()), + attempts: Vec::new(), + staged: Vec::new(), + warnings: Vec::new(), + explained: Vec::new(), + }; + if !takeover::any_takeover_ecosystem(candidates.iter().map(|c| c.purl.as_str())) { + return Ok(out); + } + let mut refused: Vec = Vec::new(); + for candidate in candidates.iter() { + let purl = &candidate.purl; + let entry = vendor_state.as_ref().ok().and_then(|s| { + socket_patch_core::vendor::lookup_entry(&s.entries, strip_purl_qualifiers(purl)) + }); + if !takeover::in_reach(purl, entry) { + continue; + } + match entry { + Some(entry) => out.attempts.push(( + purl.clone(), + candidate.dep.patch_uuid.clone(), + entry.clone(), + )), + // No usable ledger entry. If socket-owned vendored wiring + // for this crate is nevertheless present, the ledger is + // missing or corrupt — the originals needed to revert are + // unrecoverable, so redirecting on top would wedge the + // project. Refuse. (Cargo-only probe: Socket-owned + // `[patch.crates-io]` entries for exactly this name@version + // in the root Cargo.toml or a legacy `.cargo/config*`. An + // npm purl in this state falls through to the rewriters' own + // per-flavor diagnostics.) + None => { + let Some((_, name, version)) = purl + .starts_with("pkg:cargo/") + .then(|| purl_parts(purl)) + .flatten() + else { + continue; + }; + if socket_patch_core::vendor::cargo::socket_wiring_present( + &common.cwd, + &name, + &version, + ) + .await + { + refused.push(purl.clone()); + skipped.push(SkippedPatch::new( + purl, + &candidate.dep.patch_uuid, + "vendored_revert_failed", + )); + out.warnings.push(serde_json::json!({ + "code": "redirect_vendored_revert_failed", + "detail": format!( + "{purl} has socket-owned vendored `[patch.crates-io]` \ + wiring but no usable vendored ledger entry \ + (.socket/vendor/state.json is missing or corrupt); NOT \ + redirected — restore the ledger or remove the vendored \ + wiring manually, then re-run" + ), + })); + } + } + } + } + // NON-UTF-8 PRE-CHECK (#721) — the GUARD's undecodable-file rule + // (`engine::undecodable_guard`), checked before any revert is + // staged (and under --dry-run too), on the project as it is on disk. + if !out.attempts.is_empty() { + let view = socket_patch_core::vendor::lock_inventory::ProjectView::Disk(&common.cwd); + let read = socket_patch_core::hosted::engine::read_candidate_files( + &view, + &std::collections::BTreeSet::new(), + candidates, + ) + .await; + if let Some(refusal) = socket_patch_core::hosted::engine::undecodable_guard( + &read.undecodable_reads, + candidates, + ) { + return Err(refusal); + } + } + // SYMLINK PRE-CHECK — the same rule as the hosted SYMLINK GUARD, + // applied to each ledger entry's recorded wiring (the revert + // backends also stage and rename over the file), dry runs included. + let wiring = out + .attempts + .iter() + .flat_map(|(_, _, entry)| entry.wiring.iter().map(|w| w.file.as_str())); + if let Some(linked) = socket_patch_core::utils::fs::first_symlink(&common.cwd, wiring).await + { + return Err(socket_patch_core::hosted::engine::symlink_refusal(linked)); + } + candidates.retain(|c| !refused.contains(&c.purl)); + Ok(out) + } + + /// Whether any purl is (still) staged. + pub(super) fn is_staged(&self) -> bool { + !self.staged.is_empty() + } + + /// Revert every pending attempt into `group`'s overlay (step 2). A + /// revert that fails, or that leaves drifted wiring in place, is rolled + /// back and refused. + pub(super) async fn stage( + &mut self, + common: &crate::args::GlobalArgs, + group: &GroupCommit, + candidates: &mut Vec, + skipped: &mut Vec, + ) { + // Judged once, before any revert of this pass (`stage` runs on the + // pre-takeover overlay, a retraction having rolled back to it). + let berry_gate = if self.attempts.iter().any(|(_, _, e)| is_berry_entry(e)) { + berry_project_gate(common).await + } else { + None + }; + for (purl, uuid, entry) in std::mem::take(&mut self.attempts) { + if let Some(gate) = berry_gate.as_ref().filter(|_| is_berry_entry(&entry)) { + self.warnings + .push(serde_json::json!({ "code": &gate.code, "detail": &gate.detail })); + self.warnings.push(kept_vendored(&purl, &gate.code)); + skipped.push(SkippedPatch::new(&purl, &uuid, &gate.code)); + candidates.retain(|c| c.purl != purl); + continue; + } + let savepoint = group.savepoint(); + let outcome = + crate::commands::vendor::dispatch_revert_one(&entry, &common.cwd, false).await; + let refusal = if !outcome.success { + Some(serde_json::json!({ + "code": "redirect_vendored_revert_failed", + "detail": format!( + "{purl} is vendored and its vendored state could not be \ + reverted ({}); NOT switched to hosted — run `socket-patch vendor \ + --revert` to clean up, then re-run `scan --mode hosted`", + outcome.error.as_deref().unwrap_or("unknown error") + ), + })) + } else if revert_keeps_wiring(&outcome) { + // A wiring record drifted and was left in place, so the + // project may still resolve through the vendored artifact + // and the ledger entry holds the only recorded originals + // (the RevertOutcome contract): keep both and refuse, + // exactly as `vendor --revert` reports it skipped. + Some(drifted_takeover_warning(&purl)) + } else { + None + }; + if let Some(warning) = refusal { + group.rollback_to(savepoint); + self.warnings.push(warning); + skipped.push(SkippedPatch::new(&purl, &uuid, "vendored_revert_failed")); + candidates.retain(|c| c.purl != purl); + continue; + } + let advisories = outcome + .warnings + .iter() + .filter(|w| w.code == socket_patch_core::vendor::vlt_lock::REINSTALL_REQUIRED) + .map(|w| serde_json::json!({ "code": w.code, "detail": w.detail })) + .collect(); + self.staged.push(Staged { + purl, + uuid, + entry, + advisories, + }); + } + } + + /// The staged purls the rewrite did not pin (`confirmed` is the + /// rewrite's `(purl, uuid)` list). + pub(super) fn unpinned(&self, confirmed: &[(String, String)]) -> Vec { + self.staged + .iter() + .filter(|s| !confirmed.iter().any(|(_, uuid)| *uuid == s.uuid)) + .map(|s| s.uuid.clone()) + .collect() + } + + /// Undo every staged revert and refuse the purls of `unpinned` (uuids): + /// they stay vendored. The rest are staged again from the pristine + /// overlay (step 3); the caller then rewrites again. `warnings` are the + /// rewrite's own warnings, which explain why a purl was not pinned. + pub(super) async fn retract( + &mut self, + common: &crate::args::GlobalArgs, + group: &GroupCommit, + unpinned: &[String], + warnings: &[RewriteWarning], + candidates: &mut Vec, + skipped: &mut Vec, + ) { + if let Some(base) = self.base.take() { + group.rollback_to(base); + } + self.base = Some(group.savepoint()); + for staged in std::mem::take(&mut self.staged) { + if !unpinned.contains(&staged.uuid) { + self.attempts.push((staged.purl, staged.uuid, staged.entry)); + continue; + } + let dep = candidates + .iter() + .find(|c| c.dep.patch_uuid == staged.uuid) + .map(|c| &c.dep); + let code = match skipped.iter().find(|s| s.uuid == staged.uuid) { + // Already skipped with its own reason (unavailable wheel + // metadata, a withheld artifact, ...). + Some(skip) => skip.reason.clone(), + None => { + let (code, explained) = + explain(&common.cwd, &staged.entry, dep, warnings).await; + skipped.push(SkippedPatch::new(&staged.purl, &staged.uuid, &code)); + for w in explained { + if !self.explained.contains(&w) { + self.explained.push(w); + } + } + code + } + }; + self.warnings.push(kept_vendored(&staged.purl, &code)); + candidates.retain(|c| c.dep.patch_uuid != staged.uuid); + } + self.stage(common, group, candidates, skipped).await; + } + + /// Settle the takeover once the rewrite pins every staged purl (step 4's + /// bookkeeping): announce each one, and on a wet run drop its entry + /// from the in-memory vendored ledger, which the caller saves into the + /// overlay before the commit. `final_warnings` are the last rewrite's + /// warnings: the explanations it no longer reports are carried here. + pub(super) fn finish( + self, + dry_run: bool, + vendor_state: &mut std::io::Result, + final_warnings: &[RewriteWarning], + ) -> Finished { + let mut warnings: Vec = self + .explained + .iter() + .filter(|w| !final_warnings.contains(w)) + .map(|w| serde_json::json!({ "code": w.code, "detail": w.detail })) + .collect(); + warnings.extend(self.warnings); + let mut migrated = Vec::new(); + let mut files = BTreeSet::new(); + for staged in self.staged { + let purl = &staged.purl; + warnings.push(if dry_run { + serde_json::json!({ + "code": "redirect_would_revert_vendored", + "detail": format!( + "{purl} is currently vendored; the hosted wiring will \ + revert its vendored wiring, ledger entry, and committed \ + artifact first, then switch to hosted (mode takeover)" + ), + }) + } else { + serde_json::json!({ + "code": "redirect_takeover_reverted_vendored", + "detail": format!( + "{purl} was vendored; reverted its vendored wiring, ledger \ + entry, and committed artifact before switching to hosted (mode \ + takeover: the project is now fully hosted for this package)" + ), + }) + }); + warnings.extend(staged.advisories); + if !dry_run { + if let Ok(state) = vendor_state.as_mut() { + state.entries.retain(|k, e| { + !PurlKey::same(k, purl) && !PurlKey::same(&e.base_purl, purl) + }); + } + } + files.extend(staged.entry.wiring.iter().map(|w| w.file.clone())); + migrated.push(staged.purl); + } + Finished { + warnings, + migrated, + files, + } + } +} + +/// Rewriter warnings that accompany pins which landed (install guidance, +/// not refusals), so they never stand as a lock-level skip reason. +const LANDED_PIN_ADVISORIES: &[&str] = &[ + "redirect_npm_allow_remote", + "redirect_pnpm_trust_lockfile", + "redirect_yarn_classic_berry_migration_risk", +]; + +/// Why the rewrite did not pin a staged purl: the skip reason, and the +/// rewriter warnings that say so. In order: for a PyPI entry, the reach the +/// hosted rewriter lacks (`preflight_pypi_takeover`: a requirements entry +/// wired outside the root file #699, a uv entry pinned down from another +/// locked version #723, a Poetry 0.x lock #945); a rewriter warning naming +/// the package; the rewrite's first warning (a lock-level refusal names no +/// package), skipping [`LANDED_PIN_ADVISORIES`]; else [`NOT_PINNED`]. +async fn explain( + root: &std::path::Path, + entry: &VendorEntry, + dep: Option<&socket_patch_core::patch::redirect::DepOverride>, + warnings: &[RewriteWarning], +) -> (String, Vec) { + if let Err(w) = socket_patch_core::patch::redirect::preflight_pypi_takeover(root, entry).await { + return (w.code.clone(), vec![w]); + } + if let Some(w) = dep.and_then(|dep| { + warnings + .iter() + .find(|w| names_package(&w.detail, &dep.name)) + }) { + return (w.code.clone(), vec![w.clone()]); + } + // A pin that did land can carry a success advisory; it never explains + // a pin that did not. + match warnings + .iter() + .find(|w| !LANDED_PIN_ADVISORIES.contains(&w.code.as_str())) + { + Some(w) => (w.code.clone(), vec![w.clone()]), + None => (NOT_PINNED.to_string(), Vec::new()), + } +} + +/// The [`KEPT_VENDORED`] warning for `purl`, naming the `code` that kept it. +fn kept_vendored(purl: &str, code: &str) -> serde_json::Value { + serde_json::json!({ + "code": KEPT_VENDORED, + "detail": format!( + "{purl} is vendored, but hosted mode would not pin it ({code}); it stays \ + vendored — its vendored wiring, ledger entry and artifact are untouched" + ), + }) +} + +/// A vendored entry wired through the yarn-berry backend. +fn is_berry_entry(entry: &VendorEntry) -> bool { + entry.ecosystem == "npm" && entry.flavor.as_deref() == Some("yarn-berry") +} + +/// The yarn berry project gates (lock and root `package.json` line endings, +/// `cacheKey`, `.yarnrc.yml` `compressionLevel`), judged on the project as +/// it is BEFORE any yarn-berry entry's revert. The revert re-renders +/// `package.json` in its majority line ending, so a mixed manifest would +/// pass the hosted rewriter's own check afterwards, while both modes refuse +/// to rewrite a mixed one (#628): the takeover keeps such a package +/// vendored. The gate itself is the rewriter's +/// (`preflight_yarn_berry_hosted`, over the shared `berry_gates`). +async fn berry_project_gate(common: &crate::args::GlobalArgs) -> Option { + use socket_patch_core::utils::fs::read_regular_to_string; + // An unreadable lock is left to the revert's own diagnostics. + let lock = read_regular_to_string(&common.cwd.join("yarn.lock")) + .await + .ok()?; + let manifest = read_regular_to_string(&common.cwd.join("package.json")) + .await + .ok(); + let yarnrc = read_regular_to_string(&common.cwd.join(".yarnrc.yml")) + .await + .ok(); + socket_patch_core::patch::redirect::preflight_yarn_berry_hosted( + &lock, + manifest.as_deref(), + yarnrc.as_deref(), + ) + .err() +} + +/// Whether `detail` names the package `name` as a whole word. +fn names_package(detail: &str, name: &str) -> bool { + if name.is_empty() { + return false; + } + let is_word = |c: char| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/' | '@'); + detail.match_indices(name).any(|(at, _)| { + let before = detail[..at].chars().next_back(); + let after = detail[at + name.len()..].chars().next(); + // A `/` is a boundary after a path segment (`node_modules/six`), + // but not after a scope: `node` is not named by `@types/node`. + let scoped = before == Some('/') + && detail[..at - 1] + .rsplit(|c: char| !is_word(c) || c == '/') + .next() + .is_some_and(|seg| seg.starts_with('@')); + !scoped + && !before.is_some_and(|c| is_word(c) && c != '@' && c != '/') + && !after.is_some_and(|c| is_word(c) && c != '@' && c != '.') + }) +} + +/// Whether a takeover revert left vendored wiring in place: a drift-skipped +/// record, or a reverted file that still references the artifact dir. +fn revert_keeps_wiring(outcome: &RevertOutcome) -> bool { + outcome.kept_artifact + || outcome.drift_skipped() + || outcome + .warnings + .iter() + .any(|w| w.code == "vendor_revert_residual_reference") +} + +/// The refusal for a takeover whose vendored wiring drifted since vendoring. +fn drifted_takeover_warning(purl: &str) -> serde_json::Value { + serde_json::json!({ + "code": "redirect_vendored_revert_failed", + "detail": format!( + "{purl} is vendored and part of its vendored wiring was edited since \ + vendoring, so it is left in place; NOT switched to hosted — restore or \ + remove that wiring (`socket-patch vendor --revert` lists it), then re-run \ + `scan --mode hosted`" + ), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn names_package_matches_whole_names_only() { + assert!(names_package("the patched wheel for six==1.16.0 is", "six")); + assert!(names_package("left-pad@1.3.0 has no checksum", "left-pad")); + assert!(names_package("`six` is missing", "six")); + assert!(!names_package("sixteen entries", "six")); + assert!(!names_package("left-pad-extra@1", "left-pad")); + assert!(!names_package("anything", "")); + assert!(names_package("node_modules/six is stale", "six")); + assert!(!names_package("@types/node@20.0.0 is missing", "node")); + assert!(!names_package("node_modules/@types/node is stale", "node")); + assert!(names_package( + "@types/node@20.0.0 is missing", + "@types/node" + )); + } + + #[tokio::test] + async fn explain_skips_landed_pin_advisories_as_the_lock_level_cause() { + let warn = |code: &str| RewriteWarning { + code: code.into(), + detail: "lock-wide detail".into(), + }; + let entry: VendorEntry = serde_json::from_value(serde_json::json!({ + "ecosystem": "npm", + "basePurl": "pkg:npm/left-pad@1.3.0", + "uuid": "u1", + "artifact": { "path": ".socket/vendor/npm/u1/left-pad.tgz" }, + "wiring": [], + })) + .expect("minimal vendor entry"); + let warnings = [ + warn("redirect_npm_allow_remote"), + warn("redirect_pnpm_trust_lockfile"), + warn("redirect_yarn_classic_berry_migration_risk"), + ]; + let root = std::path::Path::new("."); + assert_eq!(explain(root, &entry, None, &warnings).await.0, NOT_PINNED); + let mut with_refusal = warnings.to_vec(); + with_refusal.push(warn("redirect_lock_refused")); + let (code, explained) = explain(root, &entry, None, &with_refusal).await; + assert_eq!(code, "redirect_lock_refused"); + assert_eq!(explained, vec![warn("redirect_lock_refused")]); + } +} diff --git a/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs b/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs index ed0b996b1..baf7da1f9 100644 --- a/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs +++ b/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs @@ -293,6 +293,26 @@ fn vendored_project(root: &Path) { ); } +/// Every file under `root`, `.socket/` included (relative path → bytes): +/// a dry-run takeover stages its revert in memory and must leave all of +/// it byte-identical, the vendored artifacts included. +fn tree_snapshot(root: &Path) -> std::collections::BTreeMap> { + fn walk(root: &Path, dir: &Path, out: &mut std::collections::BTreeMap>) { + for e in std::fs::read_dir(dir).unwrap() { + let p = e.unwrap().path(); + if std::fs::symlink_metadata(&p).unwrap().is_dir() { + walk(root, &p, out); + } else { + let rel = p.strip_prefix(root).unwrap().to_string_lossy().into_owned(); + out.insert(rel, std::fs::read(&p).unwrap()); + } + } + } + let mut out = std::collections::BTreeMap::new(); + walk(root, root, &mut out); + out +} + fn warning_detail<'a>(doc: &'a Value, code: &str) -> Option<&'a str> { doc["redirect"]["warnings"] .as_array()? @@ -332,9 +352,15 @@ async fn dry_run_over_vendored_project_previews_the_wet_takeover() { vendored_project(root); let vendored_lock = std::fs::read(root.join("pnpm-lock.yaml")).unwrap(); let vendored_state = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); + let vendored_tree = tree_snapshot(root); let (code, doc) = scan_hosted_json(root, &server.uri(), /*dry_run=*/ true); assert_eq!(code, 0, "dry-run scan --mode hosted must succeed: {doc:#}"); + assert_eq!( + tree_snapshot(root), + vendored_tree, + "dry-run must leave every file, the vendored tarball included, byte-identical" + ); assert_eq!(doc["redirect"]["dryRun"], true, "envelope: {doc:#}"); let codes = warning_codes(&doc); @@ -364,10 +390,11 @@ async fn dry_run_over_vendored_project_previews_the_wet_takeover() { // writes (the takeover splices the root v9 lock) must be previewed too. let trust = warning_detail(&doc, "redirect_pnpm_trust_lockfile") .unwrap_or_else(|| panic!("the trust config must be previewed: {doc:#}")); - // (The vendor run already created pnpm-workspace.yaml for its own - // wiring, so the wet run MERGES the key into it.) + // The vendor run created pnpm-workspace.yaml for its own wiring, and + // the takeover's revert removes it again, so the wet run writes a new + // one: the preview plans against the same reverted project. assert!( - trust.contains("would be merged into the existing pnpm-workspace.yaml"), + trust.contains("would be written to a new pnpm-workspace.yaml"), "{trust}" ); assert!( @@ -418,6 +445,12 @@ async fn dry_run_over_vendored_project_previews_the_wet_takeover() { workspace(root).is_some_and(|w| w.contains("trustLockfile: true")), "the wet run writes what the preview promised: {wet:#}" ); + let wet_trust = warning_detail(&wet, "redirect_pnpm_trust_lockfile") + .unwrap_or_else(|| panic!("the wet run reports the trust config: {wet:#}")); + assert!( + wet_trust.contains("to a new pnpm-workspace.yaml"), + "the preview named the wet run's file: {wet_trust}" + ); } /// Refusal parity: a vendored purl whose revert the wet run would REFUSE @@ -620,9 +653,11 @@ async fn dry_run_package_lock_takeover_previews_the_npmrc_write() { assert_eq!(code, 0, "fixture vendor run must succeed: {env:#}"); let vendored_lock = std::fs::read_to_string(root.join("package-lock.json")).unwrap(); assert!(vendored_lock.contains(".socket/vendor/"), "{vendored_lock}"); + let vendored_tree = tree_snapshot(root); let (code, doc) = scan_hosted_json(root, &server.uri(), /*dry_run=*/ true); assert_eq!(code, 0, "{doc:#}"); + assert_eq!(tree_snapshot(root), vendored_tree, "dry run writes nothing"); assert!( warning_codes(&doc).contains(&"redirect_would_revert_vendored"), "{doc:#}" @@ -702,8 +737,10 @@ async fn vlt_dry_run_over_vendored_project_previews_the_wet_takeover() { let vendored_lock = std::fs::read(root.join("vlt-lock.json")).unwrap(); let vendored_state = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); let vendored_pkg = std::fs::read(root.join("package.json")).unwrap(); + let vendored_tree = tree_snapshot(root); let (_, doc) = hosted::scan_hosted(root, &server, &["--dry-run"], &[]); + assert_eq!(tree_snapshot(root), vendored_tree, "dry run writes nothing"); let codes = hosted::warning_codes(&doc); assert!( codes.contains(&"redirect_would_revert_vendored".to_string()), diff --git a/crates/socket-patch-cli/tests/covgap_commands_scan_hosted.rs b/crates/socket-patch-cli/tests/covgap_commands_scan_hosted.rs index 3a3c2d8c1..d2c7c3e93 100644 --- a/crates/socket-patch-cli/tests/covgap_commands_scan_hosted.rs +++ b/crates/socket-patch-cli/tests/covgap_commands_scan_hosted.rs @@ -2146,16 +2146,13 @@ async fn human_rush_run_prints_the_repo_state_stale_warning_line() { // ───────── ledger save failure after a successful revert ───────── -/// save_state failure AFTER a successful takeover revert: the wiring is gone -/// but the vendored ledger still claims it, so the purl must fail CLOSED — -/// `redirect_vendored_revert_failed` with the could-not-be-updated detail, a -/// `vendored_revert_failed` skip, and no redirect — and, since the package -/// is now unpatched in both modes, `redirect_takeover_unpatched` with -/// `partial_failure` and exit 1. Reached by making -/// `.socket/vendor` itself read-only (0o555): the entry's empty wiring -/// reverts trivially and its artifact dir under the still-writable -/// `.socket/vendor/npm/` is removed, but persisting the now-empty ledger -/// needs a write in `.socket/vendor` and fails. +/// The vendored ledger cannot be updated (`.socket/vendor` itself is +/// read-only, 0o555): the takeover's revert, the hosted pin and the ledger +/// are one commit, which then fails before it replaces anything. The run +/// fails (exit 1) and NOTHING changed — the lock, the ledger and the +/// artifact are byte-identical, so the package stays vendored and patched. +/// Before the staged takeover, the revert was already on disk, leaving the +/// package unpatched in both modes. #[cfg(unix)] #[tokio::test] async fn ledger_save_failure_after_successful_revert_fails_closed() { @@ -2185,6 +2182,7 @@ async fn ledger_save_failure_after_successful_revert_fails_closed() { std::fs::create_dir_all(&artifact_dir).unwrap(); std::fs::write(artifact_dir.join(format!("{NAME}-{VERSION}.tgz")), b"tgz").unwrap(); let lock_before = std::fs::read(root.join("package-lock.json")).unwrap(); + let state_before = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); let vendor_dir = root.join(".socket/vendor"); std::fs::set_permissions(&vendor_dir, std::fs::Permissions::from_mode(0o555)).unwrap(); @@ -2198,36 +2196,24 @@ async fn ledger_save_failure_after_successful_revert_fails_closed() { let (code, doc) = scan_hosted_json(root, &server.uri(), &[], &[]); - // The vendored wiring and artifact are already gone, so the package is - // unpatched in both modes: a stranded takeover, never a success. - assert_eq!(code, 1, "a stranded takeover exits 1: {doc:#}"); - assert_eq!(doc["status"], "partial_failure", "envelope: {doc:#}"); - assert!( - warning_detail(&doc, "redirect_takeover_unpatched").contains(PURL), - "the stranded package is named: {doc:#}" - ); - let detail = warning_detail(&doc, "redirect_vendored_revert_failed"); - assert!( - detail.contains("could not be updated"), - "the post-revert ledger-save failure must be named: {detail}" - ); - assert!( - doc["redirect"]["skipped"].as_array().is_some_and(|s| s - .iter() - .any(|e| e["purl"] == PURL && e["reason"] == "vendored_revert_failed")), - "the refusal must be accounted as skipped: {doc:#}" + assert_eq!(code, 1, "the failed commit fails the run: {doc:#}"); + assert_eq!(doc["status"], "error", "envelope: {doc:#}"); + let text = doc.to_string(); + assert!(text.contains("nothing was changed"), "{doc:#}"); + assert!(!text.contains("redirect_takeover_unpatched"), "{doc:#}"); + assert_eq!( + std::fs::read(root.join("package-lock.json")).unwrap(), + lock_before, + "no redirect lands" ); - assert_eq!(doc["redirect"]["redirected"], 0, "envelope: {doc:#}"); - let lock_after = std::fs::read(root.join("package-lock.json")).unwrap(); assert_eq!( - lock_after, lock_before, - "no redirect may land when the ledger cannot record the takeover" + std::fs::read(root.join(".socket/vendor/state.json")).unwrap(), + state_before, + "the ledger still claims the package" ); - // Fail-closed residue this warning exists to explain: the wiring/artifact - // are reverted but the ledger still claims the entry. assert!( - root.join(".socket/vendor/state.json").exists(), - "the stale ledger survives (the warning tells the user to fix it)" + artifact_dir.join(format!("{NAME}-{VERSION}.tgz")).exists(), + "the artifact is kept" ); } @@ -2493,10 +2479,9 @@ async fn human_pnpm_rerun_prints_only_the_reminder_and_heal_restores_guidance() // ───────────────────────────── vlt ───────────────────────────── -/// A vendored vlt entry is never reverted for a hosted takeover the vlt -/// rewriter would then refuse: the lock-level refusal (here a BOM) is known -/// first, the purl is skipped with that code, and the vendored ledger and -/// the lock stay byte-identical. +/// A vendored vlt entry over a lock vlt cannot read (here a BOM) is never +/// taken over: the staged revert refuses the unreadable lock itself, so the +/// purl is skipped and the vendored ledger and the lock stay byte-identical. #[tokio::test] async fn vlt_takeover_refusal_before_revert() { let server = MockServer::start().await; @@ -2521,11 +2506,11 @@ async fn vlt_takeover_refusal_before_revert() { assert!( doc["redirect"]["skipped"].as_array().is_some_and(|s| s .iter() - .any(|e| e["purl"] == PURL && e["reason"] == "redirect_vlt_lock_unsupported")), + .any(|e| e["purl"] == PURL && e["reason"] == "vendored_revert_failed")), "{doc:#}" ); - assert!(warning_detail(&doc, "redirect_vlt_lock_unsupported").contains("BOM")); - assert!(!warning_codes(&doc).contains(&"redirect_vendored_revert_failed".to_string())); + assert!(warning_detail(&doc, "redirect_vendored_revert_failed").contains("vlt-lock.json")); + assert!(!warning_codes(&doc).contains(&"redirect_takeover_reverted_vendored".to_string())); assert_eq!( std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), state_before diff --git a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs index 99f1ce393..0451e6975 100644 --- a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs +++ b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs @@ -101,24 +101,36 @@ async fn mount_sumdb(server: &MockServer) { } fn get_hosted(consumer: &Path, server: &MockServer, modcache: &Path) -> serde_json::Value { + get_hosted_with(consumer, server, modcache, &[]) +} + +fn get_hosted_with( + consumer: &Path, + server: &MockServer, + modcache: &Path, + extra: &[&str], +) -> serde_json::Value { + let uri = server.uri(); + let mut args = vec![ + "get", + UUID_H, + "--mode", + "hosted", + "--json", + "--yes", + "--cwd", + consumer.to_str().unwrap(), + "--api-url", + &uri, + "--org", + ORG, + "--api-token", + "fake", + ]; + args.extend_from_slice(extra); let (code, stdout, stderr) = run_with_prebuilt( consumer, - &[ - "get", - UUID_H, - "--mode", - "hosted", - "--json", - "--yes", - "--cwd", - consumer.to_str().unwrap(), - "--api-url", - &server.uri(), - "--org", - ORG, - "--api-token", - "fake", - ], + &args, &[("GOMODCACHE", modcache.to_str().unwrap())], ); assert_eq!( @@ -128,6 +140,24 @@ fn get_hosted(consumer: &Path, server: &MockServer, modcache: &Path) -> serde_js serde_json::from_str(&stdout).unwrap_or_else(|e| panic!("not JSON: {e}\n{stdout}")) } +/// Every file under `root`, `.socket/` included (relative path → bytes). +fn tree_snapshot(root: &Path) -> std::collections::BTreeMap> { + fn walk(root: &Path, dir: &Path, out: &mut std::collections::BTreeMap>) { + for e in std::fs::read_dir(dir).unwrap() { + let p = e.unwrap().path(); + if std::fs::symlink_metadata(&p).unwrap().is_dir() { + walk(root, &p, out); + } else { + let rel = p.strip_prefix(root).unwrap().to_string_lossy().into_owned(); + out.insert(rel, std::fs::read(&p).unwrap()); + } + } + } + let mut out = std::collections::BTreeMap::new(); + walk(root, root, &mut out); + out +} + fn write_consumer(consumer: &Path, go_mod_tail: &str, go_sum: &str) { std::fs::create_dir_all(consumer).unwrap(); std::fs::write( @@ -296,6 +326,22 @@ async fn hosted_takeover_of_vendored_module_removes_vendored_state() { let server = MockServer::start().await; mount_hosted_grant(&server).await; + // The dry run stages the same revert in memory and drops it: every + // byte of the project, `.socket/` included, stays as it was. + let before = tree_snapshot(&consumer); + let preview = get_hosted_with(&consumer, &server, &modcache, &["--dry-run"]); + assert!( + preview + .to_string() + .contains("redirect_would_revert_vendored"), + "the takeover is previewed: {preview}" + ); + assert_eq!( + tree_snapshot(&consumer), + before, + "a dry-run takeover changes nothing: {preview}" + ); + let env = get_hosted(&consumer, &server, &modcache); assert_eq!(env["redirect"]["redirected"], 1, "envelope: {env}"); diff --git a/crates/socket-patch-cli/tests/e2e_redirect_gradle_build.rs b/crates/socket-patch-cli/tests/e2e_redirect_gradle_build.rs index 470581be1..9fc9f8c24 100644 --- a/crates/socket-patch-cli/tests/e2e_redirect_gradle_build.rs +++ b/crates/socket-patch-cli/tests/e2e_redirect_gradle_build.rs @@ -2172,8 +2172,10 @@ fn gradle_hosted_vendored_takeover_and_eject() { } /// A vendored Gradle build the hosted planner would refuse (a custom -/// `lockFile`): `scan --mode hosted` refuses the takeover BEFORE reverting -/// anything, so the vendored patch keeps working. +/// `lockFile`): `scan --mode hosted` stages the vendored revert in memory, +/// the planner refuses the pin, and the takeover is retracted before +/// anything reaches the disk (the tree's jars included), so the vendored +/// patch keeps working. #[test] #[ignore = "real Gradle; run with --ignored"] fn gradle_hosted_takeover_refusal_keeps_vendored() { diff --git a/crates/socket-patch-cli/tests/hosted_memory_engine.rs b/crates/socket-patch-cli/tests/hosted_memory_engine.rs index e092e4c6d..b9246fcb3 100644 --- a/crates/socket-patch-cli/tests/hosted_memory_engine.rs +++ b/crates/socket-patch-cli/tests/hosted_memory_engine.rs @@ -591,6 +591,52 @@ async fn vendored_takeover_is_refused() { assert!(output.changed_files.is_empty()); } +/// The in-memory engine refuses exactly the takeovers the disk flow +/// performs (one shared predicate): a vendored PyPI package is one, so it +/// is refused as a takeover rather than handed to the Python rewriters, +/// which would refuse socket-patch's own vendored source as user-authored. +#[tokio::test] +async fn vendored_pypi_takeover_is_refused_like_the_disk_flow() { + const PYPI_FIXTURE: &str = "redirect/pypi/requirements/basic"; + let server = MockServer::start().await; + let patches = patches_from_overrides( + &fixtures_root().join(PYPI_FIXTURE).join("overrides.json"), + None, + ); + mount_api(&server, &patches).await; + let mut files = fixture_files(&fixtures_root().join(PYPI_FIXTURE).join("input")); + let uuid = "33333333-3333-3333-3333-333333333333"; + files.insert( + ".socket/vendor/state.json".into(), + serde_json::to_vec(&serde_json::json!({ + "version": 1, + "entries": { + "pkg:pypi/requests@2.28.1": { + "ecosystem": "pypi", + "basePurl": "pkg:pypi/requests@2.28.1", + "uuid": uuid, + "flavor": "requirements", + "artifact": {"path": format!(".socket/vendor/pypi/{uuid}/requests-2.28.1-py3-none-any.whl")}, + "wiring": [] + } + } + })) + .unwrap(), + ); + let output = run_engine(&server, build_input(&files, &[], options(false))).await; + let project = &output.projects[0]; + assert!( + project + .skipped + .iter() + .any(|s| s.reason == "vendored_takeover_unsupported_in_memory"), + "{:?}", + project.skipped + ); + assert!(project.redirected.is_empty()); + assert!(output.changed_files.is_empty()); +} + /// A pre-v5 redirect ledger (`.socket/vendor/redirect-state.json`) is /// never read by the v5 engine: a torn one neither fails its project nor /// changes its plan, and the engine never emits (or rewrites) the file. diff --git a/crates/socket-patch-cli/tests/in_process_redirect.rs b/crates/socket-patch-cli/tests/in_process_redirect.rs index 536de40b5..431e1cf8a 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect.rs @@ -2191,10 +2191,10 @@ async fn directory_at_the_legacy_ledger_path_does_not_block_the_run() { } /// A MID-RUN lockfile write failure (second of two locks unwritable) exits -/// 1; the first lock landed, the failed lock stays byte-untouched (atomic -/// stage+rename, no truncation), and no ledger is written (v5: a landed -/// hosted pin is undone by `rollback`'s upstream restore, which needs no -/// recorded originals). +/// 1 and changes NOTHING: the run's writes are one commit, so the lock +/// already replaced is put back, the failed lock stays byte-untouched +/// (atomic stage+rename, no truncation), and no ledger is written. Before, +/// the first lock stayed redirected while the second did not. /// /// unix-only: a read-only directory does not block file creation on Windows. #[cfg(unix)] @@ -2214,6 +2214,8 @@ async fn partial_lockfile_write_failure_exits_1_and_writes_no_ledger() { // (common/config/rush/…) is written before the subspace lock // (common/config/subspaces/…). write_rush_project(tmp.path(), false); + let common_path = tmp.path().join("common/config/rush/pnpm-lock.yaml"); + let before_common = std::fs::read_to_string(&common_path).unwrap(); let subspace_dir = tmp.path().join("common/config/subspaces/frontend"); let before_subspace = std::fs::read_to_string(subspace_dir.join("pnpm-lock.yaml")).unwrap(); std::fs::set_permissions(&subspace_dir, std::fs::Permissions::from_mode(0o555)).unwrap(); @@ -2223,12 +2225,11 @@ async fn partial_lockfile_write_failure_exits_1_and_writes_no_ledger() { std::fs::set_permissions(&subspace_dir, std::fs::Permissions::from_mode(0o755)).unwrap(); assert_eq!(code, 1, "a mid-run lockfile write failure must exit 1"); - // The first lock landed before the failure… - let common = - std::fs::read_to_string(tmp.path().join("common/config/rush/pnpm-lock.yaml")).unwrap(); - assert!( - common.contains(HOSTED_URL), - "the common lock was written before the subspace failure; got:\n{common}" + // The first lock was put back when the second failed… + assert_eq!( + std::fs::read_to_string(&common_path).unwrap(), + before_common, + "the common lock is restored after the subspace failure" ); vlt_hosted_common::assert_no_ledger(tmp.path()); diff --git a/crates/socket-patch-cli/tests/mode_migration_cargo.rs b/crates/socket-patch-cli/tests/mode_migration_cargo.rs index 65753a34b..287c771cf 100644 --- a/crates/socket-patch-cli/tests/mode_migration_cargo.rs +++ b/crates/socket-patch-cli/tests/mode_migration_cargo.rs @@ -624,6 +624,26 @@ fn read(proj: &Path, rel: &str) -> String { std::fs::read_to_string(proj.join(rel)).unwrap_or_default() } +/// Every project file outside `target/` (relative path → bytes). +fn project_snapshot(root: &Path) -> std::collections::BTreeMap> { + fn walk(root: &Path, dir: &Path, out: &mut std::collections::BTreeMap>) { + for e in std::fs::read_dir(dir).unwrap() { + let p = e.unwrap().path(); + if std::fs::symlink_metadata(&p).unwrap().is_dir() { + if p != root.join("target") { + walk(root, &p, out); + } + } else { + let rel = p.strip_prefix(root).unwrap().to_string_lossy().into_owned(); + out.insert(rel, std::fs::read(&p).unwrap()); + } + } + } + let mut out = std::collections::BTreeMap::new(); + walk(root, root, &mut out); + out +} + fn vendor_ledger_claims(proj: &Path, purl: &str) -> bool { read(proj, ".socket/vendor/state.json").contains(purl) } @@ -666,25 +686,38 @@ async fn vendored_then_hosted_takeover_leaves_pure_hosted() { let crate_bytes = build_patched_crate(&tmp.path().join("stage"), &crate_dir, &version, &patched); mount_hosted_mocks(&server, &purl, &version, &crate_bytes, &orig, &patched).await; - let (code, stdout, stderr) = run_socket( - &proj, - &[ - "scan", - "--mode", - "hosted", - "--json", - "--yes", - "--cwd", - proj.to_str().unwrap(), - "--api-url", - &server.uri(), - "--org", - ORG, - "--api-token", - "fake", - ], - &cargo_home, + let uri = server.uri(); + let hosted_args = [ + "scan", + "--mode", + "hosted", + "--json", + "--yes", + "--cwd", + proj.to_str().unwrap(), + "--api-url", + &uri, + "--org", + ORG, + "--api-token", + "fake", + ]; + // The dry run stages the same takeover in memory and drops it: every + // project byte, `.socket/` and the committed crate copy included, stays. + let before = project_snapshot(&proj); + let dry: Vec<&str> = hosted_args.iter().copied().chain(["--dry-run"]).collect(); + let (code, stdout, stderr) = run_socket(&proj, &dry, &cargo_home); + assert_eq!(code, 0, "hosted dry run failed: {stdout}\n{stderr}"); + assert!( + stdout.contains("redirect_would_revert_vendored"), + "the takeover is previewed: {stdout}" ); + assert!( + project_snapshot(&proj) == before, + "a dry-run takeover changes nothing: {stdout}" + ); + + let (code, stdout, stderr) = run_socket(&proj, &hosted_args, &cargo_home); assert_eq!(code, 0, "hosted scan failed: {stdout}\n{stderr}"); let envelope: serde_json::Value = serde_json::from_str(&stdout).expect("json envelope"); assert_eq!(envelope["redirect"]["redirected"], 1, "{stdout}"); diff --git a/crates/socket-patch-cli/tests/mode_migration_npm.rs b/crates/socket-patch-cli/tests/mode_migration_npm.rs index 5ee80a051..0e1c11255 100644 --- a/crates/socket-patch-cli/tests/mode_migration_npm.rs +++ b/crates/socket-patch-cli/tests/mode_migration_npm.rs @@ -1220,10 +1220,12 @@ async fn berry_vendored_then_hosted_takeover_leaves_pure_hosted() { !proj.join(format!(".socket/vendor/npm/{UUID_V}")).exists(), "the orphaned committed artifact must be removed" ); - assert_eq!( - read(&proj, "package.json"), - pkg_json_pristine, - "the berry resolutions entry must be reverted" + // The vendored `file:` resolutions entry is reverted; the hosted berry + // pin routes the same selector to the hosted tarball instead (#465). + let pkg_json = read(&proj, "package.json"); + assert!( + !pkg_json.contains(".socket/vendor/") && pkg_json.contains(&hosted_url), + "the berry resolutions entry is repointed hosted:\n{pkg_json}\npristine:\n{pkg_json_pristine}" ); let lock = read(&proj, "yarn.lock"); assert!( @@ -1264,7 +1266,12 @@ async fn berry_vendored_then_hosted_takeover_leaves_pure_hosted() { ); // MANIFEST-LESS VEX over the pure hosted state (see yarn_berry_common). - let registry_state = [("yarn.lock", lock_pristine)]; + // The hosted berry pin routes `resolutions` too (#465), so the registry + // state restores the manifest as well as the lock. + let registry_state = [ + ("yarn.lock", lock_pristine), + ("package.json", pkg_json_pristine.clone().into_bytes()), + ]; let yarn = |cwd: &Path, args: &[&str], env: &[(&str, &str)]| corepack(cwd, YARN_BERRY, args, env); let api_url = server.uri(); diff --git a/crates/socket-patch-cli/tests/mode_migration_pypi.rs b/crates/socket-patch-cli/tests/mode_migration_pypi.rs index 5bf9251cc..a72b85a1e 100644 --- a/crates/socket-patch-cli/tests/mode_migration_pypi.rs +++ b/crates/socket-patch-cli/tests/mode_migration_pypi.rs @@ -857,10 +857,10 @@ async fn hatch_unrelated_guard_refuses_superseding_patch_before_unwinding() { } } -/// The uv lock rewrite needs the hosted wheel's METADATA, fetched only -/// after the takeover reverted the vendored wiring. When it is unavailable -/// the package is left on the unpatched registry release in both modes, so -/// the run must fail loudly instead of reporting success. +/// The uv lock rewrite needs the hosted wheel's METADATA. When it is +/// unavailable the rewrite cannot pin the package, so the staged takeover +/// is retracted: the package stays vendored (and patched), byte for byte, +/// and the run neither strands it unpatched nor fails. /// A vendored uv project whose hosted wheel the API cannot serve. async fn stranded_uv_project() -> (tempfile::TempDir, std::path::PathBuf, MockServer) { let (tmp, root) = project(); @@ -882,17 +882,144 @@ async fn stranded_uv_project() -> (tempfile::TempDir, std::path::PathBuf, MockSe (tmp, root, server) } +/// The vendored wiring, ledger and artifact files of a project, byte for +/// byte. +fn vendored_snapshot(root: &Path) -> Vec<(String, Option>)> { + let mut out: Vec<(String, Option>)> = + ["uv.lock", "pyproject.toml", ".socket/vendor/state.json"] + .iter() + .map(|rel| (rel.to_string(), std::fs::read(root.join(rel)).ok())) + .collect(); + let unit = root.join(format!(".socket/vendor/pypi/{UUID}")); + let mut artifacts: Vec<(String, Option>)> = std::fs::read_dir(&unit) + .map(|dir| { + dir.flatten() + .map(|e| { + ( + e.file_name().to_string_lossy().into_owned(), + std::fs::read(e.path()).ok(), + ) + }) + .collect() + }) + .unwrap_or_default(); + artifacts.sort(); + out.push(( + "artifact files".into(), + (!artifacts.is_empty()).then(Vec::new), + )); + out.extend(artifacts); + out +} + +#[tokio::test] +async fn uv_takeover_without_wheel_metadata_keeps_the_package_vendored() { + for dry_run in [true, false] { + let (_tmp, root, server) = stranded_uv_project().await; + let before = vendored_snapshot(&root); + assert!( + before.iter().all(|(_, bytes)| bytes.is_some()), + "the fixture is vendored: {before:?}" + ); + let uri = server.uri(); + let mut args = hosted_scan_args(&uri); + if dry_run { + args.push("--dry-run"); + } + let (code, env) = run_cli(&root, &args, &[]); + let ctx = format!("dry_run={dry_run}: {env:#}"); + assert_eq!(code, 0, "a retracted takeover is not a failure: {ctx}"); + assert_eq!(env["redirect"]["redirected"], 0, "{ctx}"); + let text = env.to_string(); + assert!(!text.contains("redirect_takeover_unpatched"), "{ctx}"); + assert!( + !text.contains("redirect_takeover_reverted_vendored") + && !text.contains("redirect_would_revert_vendored"), + "no takeover is announced: {ctx}" + ); + assert!(text.contains("redirect_takeover_kept_vendored"), "{ctx}"); + assert!( + env["redirect"]["skipped"] + .as_array() + .is_some_and(|s| s.iter().any(|s| s["uuid"] == UUID)), + "the purl is skipped with its cause: {ctx}" + ); + assert_eq!( + vendored_snapshot(&root), + before, + "the vendored wiring, ledger and wheel stay byte-identical: {ctx}" + ); + } +} + +/// Human output for a retracted takeover: no "Migrated … to hosted" +/// progress line and no "keep the hosted patches" next steps. #[tokio::test] -async fn uv_takeover_without_wheel_metadata_fails_loudly() { +async fn retracted_takeover_human_output_is_not_a_migration() { let (_tmp, root, server) = stranded_uv_project().await; - let (code, env) = hosted_scan(&root, &server); - assert_eq!(code, 1, "a stranded takeover is a failure: {env:#}"); - assert_eq!(env["status"], "partial_failure", "{env:#}"); - assert_eq!(env["redirect"]["redirected"], 0, "{env:#}"); + let uri = server.uri(); + let (code, stdout, stderr) = run_raw(&root, &hosted_scan_args(&uri), &[]); + assert_eq!(code, 0, "stdout:\n{stdout}\nstderr:\n{stderr}"); assert!( - env.to_string().contains("redirect_takeover_unpatched"), - "the unpatched package is named: {env:#}" + !stderr.contains("Migrated pkg:pypi/six@1.16.0"), + "a retracted package is not reported migrated:\n{stderr}" ); + assert!( + !stdout.contains("keep the hosted patches") && !stdout.contains("Reinstall"), + "no next steps for a retracted takeover:\n{stdout}" + ); + assert!(stderr.contains("stays vendored"), "{stderr}"); +} + +/// The takeover's revert, the hosted pin and the vendored ledger reach the +/// disk in ONE journaled commit. A crash inside that commit (the journal +/// written, no file or the first file replaced) never leaves the package +/// unpatched in both modes: the next run that takes the apply lock finishes +/// the commit first, and the project ends purely hosted. Before, the revert +/// was written first and the hosted pin later, by separate writes. +#[tokio::test] +async fn a_crash_inside_the_takeover_commit_is_finished_by_the_next_run() { + for failpoint in ["group_commit_journal", "group_commit_file@1"] { + let (_tmp, root) = project(); + let files = stage_requirements(&root); + vendor_project(&root, files); + let vendored = std::fs::read_to_string(root.join("requirements.txt")).unwrap(); + let server = MockServer::start().await; + let hosted_url = mount_hosted_api(&server, true).await; + let uri = server.uri(); + let (code, stdout, stderr) = run_raw( + &root, + &hosted_scan_args(&uri), + &[("SOCKET_PATCH_FAILPOINT", failpoint)], + ); + assert_eq!( + code, 86, + "{failpoint}: the run crashes inside the commit\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let journal = root.join(".socket/vendor/.commit-journal.json"); + assert!(journal.exists(), "{failpoint}: the commit is journaled"); + if failpoint == "group_commit_journal" { + assert_eq!( + std::fs::read_to_string(root.join("requirements.txt")).unwrap(), + vendored, + "nothing is replaced before the journal is complete" + ); + } + + let (code, env) = run_cli(&root, &hosted_scan_args(&uri), &[]); + assert_eq!(code, 0, "{failpoint}: the next run: {env:#}"); + assert!(!journal.exists(), "{failpoint}: the journal was replayed"); + let text = std::fs::read_to_string(root.join("requirements.txt")).unwrap(); + assert!( + text.contains(&hosted_url) && !text.contains(".socket/vendor/"), + "{failpoint}: the project is purely hosted:\n{text}" + ); + let state = std::fs::read_to_string(root.join(".socket/vendor/state.json")); + assert!( + state.as_deref().map_or(true, |s| !s.contains(UUID)), + "{failpoint}: the ledger no longer claims the package: {state:?}" + ); + } } /// A vendored requirements line edited since vendoring is left in place by @@ -998,40 +1125,6 @@ async fn drifted_vendored_line_refuses_takeover() { ); } -/// Human output for a stranded takeover: no "Migrated … to hosted" progress -/// line and no "keep the hosted patches" next steps, only the warning. -#[tokio::test] -async fn stranded_takeover_human_output_is_not_a_migration() { - let (_tmp, root, server) = stranded_uv_project().await; - let uri = server.uri(); - let (code, stdout, stderr) = run_raw(&root, &hosted_scan_args(&uri), &[]); - assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); - assert!( - !stderr.contains("Migrated pkg:pypi/six@1.16.0"), - "a stranded package is not reported migrated:\n{stderr}" - ); - assert!( - !stdout.contains("keep the hosted patches") && !stdout.contains("Reinstall"), - "no next steps for a stranded takeover:\n{stdout}" - ); - assert!(stderr.contains("UNPATCHED"), "{stderr}"); -} - -/// `--silent` keeps errors: the stranded takeover's exit 1 is explained. -#[tokio::test] -async fn stranded_takeover_is_reported_under_silent() { - let (_tmp, root, server) = stranded_uv_project().await; - let uri = server.uri(); - let mut args = hosted_scan_args(&uri); - args.push("--silent"); - let (code, stdout, stderr) = run_raw(&root, &args, &[]); - assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); - assert!( - stderr.contains("UNPATCHED") && stderr.contains("pkg:pypi/six@1.16.0"), - "the failure is diagnosable under --silent:\n{stderr}" - ); -} - /// `--dry-run` predicts the drifted-wiring refusal instead of previewing a /// takeover the wet run would refuse. #[tokio::test] @@ -1068,18 +1161,19 @@ async fn dry_run_predicts_drifted_takeover_refusal() { ); } -/// The revert succeeds but the vendored ledger cannot be updated (a -/// read-only `.socket/vendor/`): the wiring and wheel are already gone, -/// so the package is unpatched in both modes. That is a stranded takeover -/// (exit 1, `partial_failure`, `redirect_takeover_unpatched`), never a -/// success. +/// The vendored ledger cannot be updated (a read-only `.socket/vendor/`): +/// the revert, the hosted pin and the ledger are one commit, which fails +/// before it replaces anything. The run fails and nothing changed, so the +/// package stays vendored and patched — never unpatched in both modes. #[cfg(unix)] #[tokio::test] -async fn ledger_update_failure_after_revert_is_stranded() { +async fn ledger_update_failure_changes_nothing() { use std::os::unix::fs::PermissionsExt as _; let (_tmp, root) = project(); std::fs::write(root.join("requirements.txt"), "six==1.16.0\n").unwrap(); vendor_project(&root, &["requirements.txt"]); + let vendored = std::fs::read(root.join("requirements.txt")).unwrap(); + let state = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); let vendor_dir = root.join(".socket/vendor"); let set_mode = |mode| { std::fs::set_permissions(&vendor_dir, std::fs::Permissions::from_mode(mode)).unwrap() @@ -1099,11 +1193,12 @@ async fn ledger_update_failure_after_revert_is_stranded() { mount_hosted_api(&server, true).await; let (code, env) = hosted_scan(&root, &server); set_mode(0o755); - let text = env.to_string(); - assert!(text.contains("redirect_vendored_revert_failed"), "{env:#}"); - assert!(text.contains("redirect_takeover_unpatched"), "{env:#}"); - assert_eq!(env["status"], "partial_failure", "{env:#}"); assert_eq!(code, 1, "{env:#}"); + assert_eq!(env["status"], "error", "{env:#}"); + assert!(!env.to_string().contains("redirect_takeover_unpatched"), "{env:#}"); + assert_eq!(std::fs::read(root.join("requirements.txt")).unwrap(), vendored); + assert_eq!(std::fs::read(root.join(".socket/vendor/state.json")).unwrap(), state); + assert!(root.join(format!(".socket/vendor/pypi/{UUID}")).exists()); } // ── `vendor --check` wiring audit (#725) ───────────────────────────────── diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 9d46af592..9bbfae941 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -1013,16 +1013,6 @@ pub fn pipenv_lock_targets(files: &BTreeMap, candidates: &[Candi crate::patch::redirect::pipenv_lock_targets(files, &overrides) } -/// A dry-run vendored→hosted takeover the disk caller withheld from the -/// rewriters: its artifact URL and the root locks its vendored wiring -/// lives in (the wet run splices the hosted URL there, so the -/// install-policy auto-configs are previewed for those locks). -#[derive(Debug, Clone)] -pub struct TakeoverPreview { - pub artifact_url: String, - pub locks: Vec, -} - /// The host-dependent inputs of [`rewrite`]. pub struct RewriteOptions<'a> { pub dry_run: bool, @@ -1193,14 +1183,13 @@ pub fn candidate_presence_needles(dep: &DepOverride) -> Vec { /// /// `python_metadata` maps a wheel's artifact URL to its fetched METADATA; /// `withheld_from_vlt` are the uuids the vlt preflight kept out of the vlt -/// rewrite; `takeover_previews` are the disk dry run's withheld takeovers. +/// rewrite. pub async fn rewrite( view: &ProjectView<'_>, read: CandidateFiles, candidates: &[Candidate], python_metadata: BTreeMap, withheld_from_vlt: &BTreeSet, - takeover_previews: &[TakeoverPreview], options: RewriteOptions<'_>, ) -> Rewritten { let CandidateFiles { @@ -1359,22 +1348,10 @@ pub async fn rewrite( )); } - let (pnpm_warnings, trust_config_write, pnpm_rerun_only, workspace_symlinked) = pnpm_trust( - view, - &files, - &rewrite, - &overrides, - takeover_previews, - &options, - ); - let (npm_warnings, npmrc_config_write) = npm_allow_remote( - view, - &files, - &rewrite, - &overrides, - takeover_previews, - &options, - ); + let (pnpm_warnings, trust_config_write, pnpm_rerun_only, workspace_symlinked) = + pnpm_trust(view, &files, &rewrite, &overrides, &options); + let (npm_warnings, npmrc_config_write) = + npm_allow_remote(view, &files, &rewrite, &overrides, &options); if let Some((text, edit)) = trust_config_write { rewrite.files.insert(PNPM_WORKSPACE_REL.to_string(), text); // Appended last, after the lock edits it serves. v5 keeps no hosted @@ -1443,7 +1420,6 @@ fn pnpm_trust( files: &BTreeMap, rewrite: &RewriteResult, overrides: &[DepOverride], - takeover_previews: &[TakeoverPreview], options: &RewriteOptions<'_>, ) -> (Vec, ConfigWrite, bool, bool) { let mut pnpm_warnings: Vec = Vec::new(); @@ -1478,26 +1454,6 @@ fn pnpm_trust( if let Some(text) = heal_root { pnpm_lock_texts.push(text); } - // A dry-run vendored→hosted takeover of a purl vendored into the root - // pnpm lock: the wet run reverts that wiring and splices the hosted URL - // into it, so the trust config is previewed against the root lock (the - // vendored text carries the same lockfileVersion). - let takeover_pnpm_urls: Vec<&str> = takeover_previews - .iter() - .filter(|t| t.locks.iter().any(|l| l == "pnpm-lock.yaml")) - .map(|t| t.artifact_url.as_str()) - .collect(); - let takeover_root: Option<&String> = if takeover_pnpm_urls.is_empty() - || heal_root.is_some() - || rewrite.files.contains_key("pnpm-lock.yaml") - { - None - } else { - files.get("pnpm-lock.yaml") - }; - if let Some(text) = takeover_root { - pnpm_lock_texts.push(text); - } if pnpm_lock_texts.is_empty() { return ( pnpm_warnings, @@ -1523,8 +1479,6 @@ fn pnpm_trust( .zip(present) .filter(|(_, present)| *present) .filter_map(|(o, _)| url_host(&o.artifact_url)) - // Dry-run takeover purls land in the root lock on the wet run. - .chain(takeover_pnpm_urls.iter().filter_map(|url| url_host(url))) .collect(); hosts.sort_unstable(); hosts.dedup(); @@ -1537,7 +1491,6 @@ fn pnpm_trust( // gets the auto-config — spliced this run, or detected // already-redirected (heal path). let root_lock_v9 = heal_root - .or(takeover_root) .and_then(|text| pnpm_lock_version_major(text)) .is_some_and(|major| major >= 9) || rewrite @@ -1708,7 +1661,6 @@ fn npm_allow_remote( files: &BTreeMap, rewrite: &RewriteResult, overrides: &[DepOverride], - takeover_previews: &[TakeoverPreview], options: &RewriteOptions<'_>, ) -> (Vec, ConfigWrite) { let mut npm_warnings: Vec = Vec::new(); @@ -1729,15 +1681,6 @@ fn npm_allow_remote( .zip(present) .filter(|(_, present)| *present) .filter_map(|(o, _)| url_host(&o.artifact_url)) - // A dry-run vendored→hosted takeover: the wet run reverts the - // vendored wiring in a root npm lock and splices the hosted URL - // there, so preview the `.npmrc` write too. - .chain( - takeover_previews - .iter() - .filter(|t| t.locks.iter().any(|l| NPM_LOCKS.contains(&l.as_str()))) - .filter_map(|t| url_host(&t.artifact_url)), - ) .collect(); hosts.sort_unstable(); hosts.dedup(); @@ -2266,7 +2209,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options, ) .await; @@ -2340,7 +2282,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options(), ) .await; @@ -2362,7 +2303,6 @@ mod tests { &cargo, BTreeMap::new(), &BTreeSet::new(), - &[], options(), ) .await; @@ -2624,7 +2564,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options, ) .await; @@ -2744,7 +2683,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options, ) .await; @@ -2919,7 +2857,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options, ) .await; @@ -3230,7 +3167,6 @@ mod tests { &candidates, BTreeMap::new(), &BTreeSet::new(), - &[], options, ) .await; diff --git a/crates/socket-patch-core/src/hosted/memory/stages.rs b/crates/socket-patch-core/src/hosted/memory/stages.rs index 2ed9b7fc9..f9f846734 100644 --- a/crates/socket-patch-core/src/hosted/memory/stages.rs +++ b/crates/socket-patch-core/src/hosted/memory/stages.rs @@ -93,19 +93,20 @@ fn refuse_takeovers( skipped: &mut Vec, pre_warnings: &mut Vec, ) { - let takeover_capable = |p: &str| { - p.starts_with("pkg:cargo/") || p.starts_with("pkg:npm/") || p.starts_with("pkg:golang/") - }; - if !candidates.iter().any(|c| takeover_capable(&c.purl)) { + use super::super::takeover; + if !takeover::any_takeover_ecosystem(candidates.iter().map(|c| c.purl.as_str())) { return; } let vendored = vendored_entries(project); let mut refused: BTreeSet = BTreeSet::new(); - for candidate in candidates.iter().filter(|c| takeover_capable(&c.purl)) { - let has_entry = vendored.as_ref().is_some_and(|s| { + for candidate in candidates.iter() { + let entry = vendored.as_ref().and_then(|s| { crate::vendor::lookup_entry(&s.entries, strip_purl_qualifiers(&candidate.purl)) - .is_some() }); + if !takeover::in_reach(&candidate.purl, entry) { + continue; + } + let has_entry = entry.is_some(); let cargo_wired = !has_entry && candidate.purl.starts_with("pkg:cargo/") && cargo_vendored_wiring(project, &candidate.dep.name, &candidate.dep.version); @@ -316,7 +317,6 @@ pub(crate) async fn rewrite( &candidates, python_metadata, &vlt_preflight.withheld_from_vlt, - &[], RewriteOptions { dry_run: options.dry_run, targets_pipenv_lock, diff --git a/crates/socket-patch-core/src/hosted/mod.rs b/crates/socket-patch-core/src/hosted/mod.rs index 8d07ec921..ae362abb4 100644 --- a/crates/socket-patch-core/src/hosted/mod.rs +++ b/crates/socket-patch-core/src/hosted/mod.rs @@ -23,4 +23,5 @@ pub mod memory; pub mod npm_manifest; pub mod render; pub mod sbt_reads; +pub mod takeover; pub mod vlt; diff --git a/crates/socket-patch-core/src/hosted/takeover.rs b/crates/socket-patch-core/src/hosted/takeover.rs new file mode 100644 index 000000000..637cb1532 --- /dev/null +++ b/crates/socket-patch-core/src/hosted/takeover.rs @@ -0,0 +1,108 @@ +//! Which hosted candidates a vendored → hosted mode takeover covers. +//! +//! One predicate for both hosted engines: the disk flow (`scan --mode +//! hosted`) reverts the vendored wiring of exactly these purls before it +//! pins them, and the in-memory engine, which performs no takeover, refuses +//! exactly these purls instead. Keeping the two lists in one place stops them +//! drifting apart. + +use crate::vendor::VendorEntry; + +/// The ecosystems whose hosted rewriters cannot pin a purl that is still +/// vendored, so a takeover must revert the vendored wiring first: +/// +/// * cargo: `--locked` builds refuse over the unused `[patch]` entry; +/// * npm: yarn classic would hijack a resolution the vendored ledger still +/// claims, and yarn berry refuses `file:` outright; +/// * golang: the vendor-owned go.mod `replace` shadows the hosted one; +/// * pypi: every Python rewriter refuses a non-registry source, including +/// the vendored one socket-patch wrote itself (#328); +/// * maven: only a Gradle build's vendored JVM entry (see +/// [`is_gradle_jvm_entry`]); a pom-only vendored entry stays. +fn takeover_ecosystem(purl: &str) -> bool { + [ + "pkg:cargo/", + "pkg:npm/", + "pkg:golang/", + "pkg:pypi/", + "pkg:maven/", + ] + .iter() + .any(|prefix| purl.starts_with(prefix)) +} + +/// Whether any of `purls` could be taken over (a cheap pre-check before the +/// vendored ledger is consulted). +pub fn any_takeover_ecosystem<'a>(mut purls: impl Iterator) -> bool { + purls.any(takeover_ecosystem) +} + +/// A vendored JVM entry wired into a Gradle build (its revert unplans the +/// vendored Gradle wiring), as opposed to a pom-only entry. +pub fn is_gradle_jvm_entry(entry: &VendorEntry) -> bool { + entry.ecosystem == crate::vendor::jvm::layout::LEDGER_ECOSYSTEM + && entry.wiring.iter().any(|w| { + w.file.ends_with(".gradle") + || w.file.ends_with(".gradle.kts") + || w.file == crate::vendor::jvm::gradle::INDEX_REL + }) +} + +/// Whether the hosted candidate `purl`, with its vendored ledger entry (if +/// any), is in a takeover's reach. A purl with no entry is in reach for the +/// ecosystems whose vendored wiring can exist without one (a cargo +/// `[patch.crates-io]` entry whose ledger was lost); the callers probe that +/// wiring themselves. +pub fn in_reach(purl: &str, entry: Option<&VendorEntry>) -> bool { + if !takeover_ecosystem(purl) { + return false; + } + !purl.starts_with("pkg:maven/") || entry.is_some_and(is_gradle_jvm_entry) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn entry(ecosystem: &str, files: &[&str]) -> VendorEntry { + let wiring: Vec = files + .iter() + .map(|f| serde_json::json!({ "file": f, "kind": "k", "action": "rewritten" })) + .collect(); + serde_json::from_value(serde_json::json!({ + "ecosystem": ecosystem, + "basePurl": "pkg:maven/g/a@1", + "uuid": "00000000-0000-4000-8000-000000000000", + "artifact": { "path": ".socket/vendor/jvm/x" }, + "wiring": wiring, + })) + .expect("a minimal ledger entry") + } + + #[test] + fn the_takeover_ecosystems_are_in_reach() { + for purl in [ + "pkg:cargo/a@1", + "pkg:npm/a@1", + "pkg:golang/a@v1", + "pkg:pypi/a@1", + ] { + assert!(in_reach(purl, None), "{purl}"); + } + for purl in ["pkg:gem/a@1", "pkg:composer/a/b@1", "pkg:nuget/a@1"] { + assert!(!in_reach(purl, None), "{purl}"); + } + } + + #[test] + fn maven_is_in_reach_only_through_a_gradle_entry() { + let purl = "pkg:maven/g/a@1"; + assert!(!in_reach(purl, None)); + assert!(!in_reach(purl, Some(&entry("jvm", &["pom.xml"])))); + assert!(in_reach(purl, Some(&entry("jvm", &["build.gradle.kts"])))); + assert!(in_reach( + purl, + Some(&entry("jvm", &[crate::vendor::jvm::gradle::INDEX_REL])) + )); + } +} diff --git a/crates/socket-patch-core/src/patch/redirect/gradle.rs b/crates/socket-patch-core/src/patch/redirect/gradle.rs index d2553bb98..32d4880f7 100644 --- a/crates/socket-patch-core/src/patch/redirect/gradle.rs +++ b/crates/socket-patch-core/src/patch/redirect/gradle.rs @@ -1054,56 +1054,6 @@ fn may_admit_above(sel: &Selector, base: &str) -> bool { } } -/// Whether the hosted planner would refuse `dep` in the build `files` -/// holds once its vendored Gradle wiring is reverted: every refusal of -/// [`rewrite_gradle_hosted`] except the vendored-index conflict, which the -/// takeover's revert clears. The vendored backend serves the original GAV -/// and leaves lock files alone, so the lock checks hold before the revert -/// too. A takeover runs this before reverting anything, so a refused purl -/// keeps its working vendored patch. `None` when there is no Gradle build. -pub fn takeover_refusal( - files: &BTreeMap, - unreadable: &BTreeSet, - dep: &DepOverride, -) -> Option { - if !gradle_build_present(files) { - return None; - } - let (group, artifact) = coords_of(dep); - let warning = |r: Refusal| RewriteWarning { - code: r.code.into(), - detail: format!( - "{}; the vendored patch of {group}:{artifact}:{} stays in place (NOT switched to \ - hosted)", - r.detail, dep.version - ), - }; - if registry_override_of_kind(dep, "maven2").is_none() { - return Some(warning(refusal( - "redirect_gradle_override_invalid", - "the hosted grant carries no maven2 repository", - ))); - } - let graph = graph_of(files); - let lock_paths = lockfile_paths(&graph, files); - let index = files - .get(HOSTED_INDEX_REL) - .map_or(Ok(Vec::new()), |t| parse_index(t)); - let project = unreadable_refusal(unreadable).or_else(|| project_refusal(files, &graph, &index)); - let rows = index.unwrap_or_default(); - plan_dep( - dep, - files, - &graph, - &lock_paths, - &rows, - &BTreeSet::new(), - project.as_ref(), - ) - .err() - .map(warning) -} - /// `(groupId, artifactId)` of a maven dep. fn coords_of(dep: &DepOverride) -> (String, String) { let ids = dep.registry_override.as_ref().map(|o| &o.identifiers); @@ -2394,103 +2344,6 @@ mod tests { ); } - /// The takeover preflight refuses what the planner would refuse once - /// the vendored wiring is gone, and ignores the vendored index itself. - #[test] - fn takeover_refusal_mirrors_the_planner_except_the_vendored_index() { - let vendored = ( - ".socket/vendor/gradle-index.tsv", - "#socket-patch-gradle-index 1\ncom.socketfixture:victim:1.10.0\tx\ty\tz\n", - ); - let s = ("settings.gradle", "rootProject.name = 'app'\n"); - let none = BTreeSet::new(); - assert!(takeover_refusal(&files(&[s, vendored]), &none, &dep()).is_none()); - assert!(takeover_refusal(&files(&[("pom.xml", "")]), &none, &dep()).is_none()); - // A build file that exists but cannot be read refuses the takeover. - let unreadable: BTreeSet = ["settings.gradle".to_string()].into(); - assert_eq!( - takeover_refusal( - &files(&[("build.gradle", ""), vendored]), - &unreadable, - &dep() - ) - .map(|w| w.code) - .as_deref(), - Some(UNREADABLE_REFUSAL_CODE) - ); - let code = |input: &[(&str, &str)], d: DepOverride| { - takeover_refusal(&files(input), &none, &d).map(|w| w.code) - }; - assert_eq!( - code( - &[ - s, - vendored, - ( - "build.gradle", - "dependencyLocking { lockFile = file('x.lockfile') }\n" - ) - ], - dep() - ) - .as_deref(), - Some("redirect_gradle_lock_location_unknown") - ); - let mut legacy = dep(); - legacy - .registry_override - .as_mut() - .unwrap() - .identifiers - .maven_suffixed_version = None; - assert_eq!( - code(&[s, vendored], legacy).as_deref(), - Some("redirect_gradle_same_gav_unsupported") - ); - let mut no_sha = dep(); - no_sha.integrity.sha256 = None; - assert_eq!( - code(&[s, vendored], no_sha).as_deref(), - Some("redirect_gradle_override_invalid") - ); - let mut no_override = dep(); - no_override.registry_override = None; - assert_eq!( - code(&[s, vendored], no_override).as_deref(), - Some("redirect_gradle_override_invalid") - ); - assert_eq!( - code( - &[ - s, - vendored, - ( - "settings-gradle.lockfile", - "com.socketfixture:victim:1.10.0=classpath\n" - ), - ], - dep() - ) - .as_deref(), - Some("redirect_gradle_settings_classpath") - ); - let w = takeover_refusal( - &files(&[ - s, - vendored, - ( - "gradle.lockfile", - "com.socketfixture:victim:1.9=runtimeClasspath\n", - ), - ]), - &none, - &dep(), - ) - .unwrap(); - assert_eq!(w.code, "redirect_gradle_lock_conflict"); - assert!(w.detail.contains("stays in place"), "{}", w.detail); - } - /// The files a restore can touch: every build's settings (a missing /// one included), every lock and the owned files. #[test] diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 7aae30da5..eaa252f91 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -3399,11 +3399,7 @@ pub fn yarn_classic_offline_mirror( /// mirror, so yarn installs the upstream bytes and fails the patched /// integrity (or, `--offline`, never fetches the patched tarball at all). /// `Ok` for a lock that is not classic (the berry rewriter owns those). -/// -/// Exposed so the vendored→hosted mode takeover can refuse BEFORE it -/// reverts a vendored yarn classic entry (vendored mode works with a -/// mirror), like [`preflight_yarn_berry_hosted`]. -pub fn preflight_yarn_classic_hosted( +fn preflight_yarn_classic_hosted( lock: &str, yarnrc: Option<&str>, npmrc: Option<&str>, @@ -3884,14 +3880,6 @@ fn yarn_berry_tarball_url_ok(url: &str) -> bool { /// files both edit (#628). `Ok` for a lock that is not berry (the classic /// rewriter owns those). /// -/// Exposed so the vendored→hosted mode takeover (`scan`/`get --mode hosted` -/// over a vendored berry purl) can refuse BEFORE it reverts the vendored -/// wiring: the vendored revert never refuses on line endings (it keeps a -/// mixed lock mixed), so without this preflight the takeover would strip the -/// live vendored patch and then this rewriter would refuse the lock, leaving the -/// package unpatched in both modes — the bun twin is -/// [`preflight_bun_hosted`]. -/// /// Line endings: yarn berry writes a NEW lockfile with the OS line ending /// (`os.EOL`: CRLF on Windows) and keeps an existing file's majority ending /// on every later write (`normalizeLineEndings` in yarnpkg-fslib @@ -3902,32 +3890,6 @@ fn yarn_berry_tarball_url_ok(url: &str) -> bool { /// compares the file with its own majority-normalized re-render and fails /// (YN0028), while a plain install rewrites every minority line — so it is /// refused untouched, `yarn install` normalizes it first. -/// The grant prerequisite for creating a new yarn berry hosted pin: a dep -/// whose grant carries no `yarnBerry10c0` cache checksum cannot be redirected -/// (berry verifies the converted cache zip, and only the service can compute -/// that checksum). -/// -/// Exposed for the vendored→hosted mode takeover, like -/// [`preflight_yarn_berry_hosted`]: vendored mode only uses the `tarball` -/// artifact, so a vendorable patch can lack the berry checksum, and the -/// takeover must keep such a package vendored instead of reverting it and -/// then skipping the redirect. -/// Keep this unconditional gate at the takeover boundary: a lock-aware -/// rewriter may retain an already complete pin's stored checksum. -pub fn preflight_yarn_berry_hosted_dep(dep: &DepOverride) -> Result<(), RewriteWarning> { - if dep.integrity.yarn_berry10c0.is_some() { - return Ok(()); - } - Err(RewriteWarning { - code: "redirect_yarn_berry_missing_checksum".into(), - detail: format!( - "{}@{} has no yarnBerry10c0 cache checksum", - full_name(dep), - dep.version - ), - }) -} - pub fn preflight_yarn_berry_hosted( lock: &str, manifest: Option<&str>, @@ -4705,12 +4667,6 @@ fn berry_catalog_selectors(yarnrc: Option<&str>, name: &str, ranges: &[&str]) -> // Binary locks use `rewrite_bun_binary`, which accepts bytes directly. // The text path uses the shared `bun_lock_text` grammar (fail-closed on // deviations). Byte-for-byte twin of the TS `rewriteBun`. -/// Check a text Bun lock before reverting any existing vendored wiring. -/// Uses the rewriter's own version, grammar and workspace compatibility rules. -pub fn preflight_bun_hosted(content: &str) -> Result<(), RewriteWarning> { - parse_bun_hosted_lock(content).map(|_| ()) -} - fn parse_bun_hosted_lock( content: &str, ) -> Result<(Vec, Vec), RewriteWarning> { @@ -10315,6 +10271,31 @@ mod tests { } } + /// An offline mirror refuses every pin (#364), so nothing is pinned and + /// there is no hosted pin for a berry install to drop: the refusal is + /// the only warning. A vendored-to-hosted takeover reports a retracted + /// purl's first warning as its cause, which must be the mirror. + #[test] + fn yarn_classic_offline_mirror_refusal_skips_the_berry_risk_warning() { + let ovr = npm_override( + "left-pad", + "1.3.0", + "http://p.test/lp.tgz", + "sha512-PATCHED==", + ); + let mut files = classic_files(Some(r#"{"name":"p"}"#)); + files.insert( + YARNRC_REL.to_string(), + "yarn-offline-mirror \"./mirror\"\n".to_string(), + ); + let mut r = RewriteResult::default(); + rewrite_yarn_classic(&files, std::slice::from_ref(&ovr), &mut r); + assert!(r.files.is_empty() && r.edits.is_empty(), "{:?}", r.edits); + assert!(r.refused_yarn_classic_uuids.contains(&ovr.patch_uuid)); + let codes: Vec<&str> = r.warnings.iter().map(|w| w.code.as_str()).collect(); + assert_eq!(codes, ["redirect_yarn_classic_offline_mirror"], "{codes:?}"); + } + /// #907: a corepack `packageManager: yarn@1…` pin makes a stray berry /// install refuse instead of migrate, so it suppresses the warning (as /// it does the vendored one), including a pin with a corepack hash. diff --git a/crates/socket-patch-core/src/patch/redirect/vlt.rs b/crates/socket-patch-core/src/patch/redirect/vlt.rs index fa50e3f4d..f584a6eb2 100644 --- a/crates/socket-patch-core/src/patch/redirect/vlt.rs +++ b/crates/socket-patch-core/src/patch/redirect/vlt.rs @@ -90,15 +90,6 @@ pub(super) fn parse_hosted_lock(text: &str) -> Result) -> Result<(), RewriteWarning> { - match files.get(VLT_LOCK) { - Some(text) => parse_hosted_lock(text).map(|_| ()), - None => Ok(()), - } -} - /// Is `id` a registry node of `name@version`, and is its segment the /// default registry? `None` for any other node. fn registry_instance( @@ -673,6 +664,14 @@ pub fn carried_pin_original(fresh: &FileEdit, old: &FileEdit) -> Option { mod tests { use super::*; + /// The rewriter's lock-level refusal alone. An absent lock passes. + fn lock_level_refusal(files: &BTreeMap) -> Result<(), RewriteWarning> { + match files.get(VLT_LOCK) { + Some(text) => parse_hosted_lock(text).map(|_| ()), + None => Ok(()), + } + } + const SHA: &str = "sha512-PATCHED=="; const URL: &str = "https://patch.socket.dev/patch/npm/t/u/left-pad-1.3.0.tgz"; const REG_SHA: &str = "sha512-REGISTRY=="; @@ -901,7 +900,7 @@ mod tests { #[test] fn lock_level_parse_refusals() { let refused = |text: &str| { - preflight_vlt_hosted(&files(&[(VLT_LOCK, text)])) + lock_level_refusal(&files(&[(VLT_LOCK, text)])) .unwrap_err() .detail }; @@ -912,10 +911,10 @@ mod tests { "{{\n \"lockfileVersion\": 1,\n \"nodes\": {{\n \"{ID}\": [\n 0,\n \"left-pad\"\n ]\n }}\n}}\n" ); assert!(refused(&pretty).contains("canonical layout")); - assert!(preflight_vlt_hosted(&files(&[])).is_ok()); - assert!(preflight_vlt_hosted(&files(&[(VLT_LOCK, "{\"nodes\": {}}")])).is_ok()); + assert!(lock_level_refusal(&files(&[])).is_ok()); + assert!(lock_level_refusal(&files(&[(VLT_LOCK, "{\"nodes\": {}}")])).is_ok()); let ok = lock_with(&[®istry_entry()]); - assert!(preflight_vlt_hosted(&files(&[(VLT_LOCK, &ok)])).is_ok()); + assert!(lock_level_refusal(&files(&[(VLT_LOCK, &ok)])).is_ok()); } #[test] @@ -994,7 +993,7 @@ mod tests { &format!("\"{peer}\": [6,\"left-pad\",\"{REG_SHA}\",\"{BR_URL}\"]"), "\"~npm~other@1.0.0\": [5,\"other\",\"sha512-O==\"]", ]); - assert!(preflight_vlt_hosted(&files(&[(VLT_LOCK, &lock)])).is_ok()); + assert!(lock_level_refusal(&files(&[(VLT_LOCK, &lock)])).is_ok()); let result = rewrite(&lock, &[dep("left-pad", "1.3.0", Some(SHA))]); assert!(codes(&result).is_empty(), "{:?}", result.warnings); diff --git a/crates/socket-patch-core/src/utils/group_commit.rs b/crates/socket-patch-core/src/utils/group_commit.rs index abef67c8d..3b483d55f 100644 --- a/crates/socket-patch-core/src/utils/group_commit.rs +++ b/crates/socket-patch-core/src/utils/group_commit.rs @@ -18,8 +18,11 @@ //! and [`GroupCommit::commit`] writes the final state once. //! //! **What is captured.** Files under the root whose relative path has no -//! `.socket` component, plus the two ledgers (`.socket/vendor/state.json`, -//! `.socket/vendor/redirect-state.json`). Everything else under `.socket/` +//! `.socket` component, the two ledgers (`.socket/vendor/state.json`, +//! `.socket/vendor/redirect-state.json`), and the small text files the JVM +//! wiring owns under `.socket/` (the Gradle index and settings script, the +//! owned `.gitattributes` files, the derived `maven-metadata.xml` files, the +//! Coursier index). Everything else under `.socket/` //! — the artifacts under `.socket/vendor//`, workspace members' //! `.socket/vendor/` mirrors, blobs, the manifest — is written straight to //! disk as before: artifacts are read back by path (zip readers, hashing, @@ -148,6 +151,10 @@ struct Overlay { /// Directories to remove once the commit is on disk, if empty then /// (see [`remove_dir_after_commit`]). dirs_after_commit: Mutex>, + /// Set by [`GroupCommit::defer_removals`]: the vendored artifacts a + /// revert deletes are queued for after the commit too (see + /// [`defer_removal`]). + defer_removals: std::sync::atomic::AtomicBool, } static ACTIVE: Mutex>> = Mutex::new(Vec::new()); @@ -183,6 +190,7 @@ fn is_captured(rel: &Path) -> bool { let spelled = rel.to_string_lossy().replace('\\', "/"); if LEDGERS.contains(&spelled.as_str()) || crate::vendor::jvm::layout::CAPTURED_FILES.contains(&spelled.as_str()) + || crate::vendor::jvm::gradle::is_derived_metadata_path(&spelled) { return true; } @@ -338,7 +346,7 @@ pub(crate) fn read_value(path: &Path) -> Option> { } /// Whether `path` exists as the run sees it; `None` when not captured. -pub(crate) fn exists(path: &Path) -> Option { +pub fn exists(path: &Path) -> Option { let (overlay, key) = resolve(path)?; let files = overlay .files @@ -430,6 +438,42 @@ pub(crate) async fn remove_after_commit(tree: &Path, prune_bound: &Path) { remove_tree_pruned(tree, prune_bound).await; } +/// Queue the deletion of `tree` (a vendored artifact a revert removes: a +/// `.socket/vendor///` unit or a workspace tarball) and the +/// pruning of its now-empty parents up to and including `prune_bound` for +/// after the commit, when an open group covering `tree` was asked to +/// [`GroupCommit::defer_removals`]. `false` (nothing queued: the caller +/// deletes now) otherwise. +/// +/// The hosted takeover stages a whole vendored revert in a group it may +/// roll back ([`GroupCommit::rollback_to`]) or never commit (a dry run, a +/// refused rewrite): the captured lock edits then never reach the disk, so +/// the artifact they still name must not be deleted either. +pub(crate) fn defer_removal(tree: &Path, prune_bound: &Path) -> bool { + if ACTIVE_COUNT.load(Ordering::Acquire) == 0 { + return false; + } + let overlay = active() + .iter() + .find(|o| { + o.defer_removals.load(Ordering::Acquire) + && (tree.starts_with(&o.root) + || o.canonical_root + .as_deref() + .is_some_and(|r| tree.starts_with(r))) + }) + .cloned(); + let Some(overlay) = overlay else { + return false; + }; + overlay + .after_commit + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .push((tree.to_path_buf(), prune_bound.to_path_buf())); + true +} + /// Remove the directory `dir` if it is empty — the `.cargo/` a deleted /// socket-created `.cargo/config.toml` leaves — once the run's commit is on /// disk. A captured removal of the file inside it only reaches the disk at @@ -465,7 +509,15 @@ pub(crate) async fn remove_dir_after_commit(dir: &Path) { /// something else fails and stops the prune; a level already gone is /// skipped. async fn remove_tree_pruned(tree: &Path, bound: &Path) { - let _ = crate::patch::copy_tree::remove_tree(tree).await; + // A deferred artifact may be a single file (a workspace tarball). + match tokio::fs::symlink_metadata(tree).await { + Ok(meta) if !meta.is_dir() => { + let _ = tokio::fs::remove_file(tree).await; + } + _ => { + let _ = crate::patch::copy_tree::remove_tree(tree).await; + } + } let mut parent = tree.parent().map(Path::to_path_buf); while let Some(dir) = parent { if !dir.starts_with(bound) { @@ -552,6 +604,13 @@ struct Change { } impl GroupCommit { + /// Also defer the vendored artifact deletions (see [`defer_removal`]) to + /// after the commit, so a staged revert can be rolled back or dropped + /// with its artifacts intact. + pub fn defer_removals(&self) { + self.overlay.defer_removals.store(true, Ordering::Release); + } + /// Start capturing the commit points under `root`. pub fn begin(root: &Path) -> Self { let overlay = Arc::new(Overlay { @@ -560,6 +619,7 @@ impl GroupCommit { files: Mutex::new(BTreeMap::new()), after_commit: Mutex::new(Vec::new()), dirs_after_commit: Mutex::new(Vec::new()), + defer_removals: std::sync::atomic::AtomicBool::new(false), }); let mut active = active(); active.push(Arc::clone(&overlay)); @@ -598,7 +658,7 @@ impl GroupCommit { /// it held before the commit. pub async fn commit_changes(mut self) -> std::io::Result> { self.close(); - let changed = self.write().await?; + let changed = self.write(true).await?; let removals = std::mem::take( &mut *self .overlay @@ -622,7 +682,19 @@ impl GroupCommit { Ok(changed) } - async fn write(&mut self) -> std::io::Result> { + /// [`Self::commit`] without the crash journal, for a run that must + /// write nothing under `.socket/` (hosted mode without a takeover): the + /// files are replaced one by one, and a failed replacement puts the + /// ones already replaced back. A crash part-way leaves the files + /// replaced so far (the pre-journal behavior); nothing is ever left + /// half-written. + pub async fn commit_unjournaled(mut self) -> std::io::Result> { + self.close(); + let changed = self.write(false).await?; + Ok(changed.into_iter().map(|c| c.rel).collect()) + } + + async fn write(&mut self, journaled: bool) -> std::io::Result> { let root = self.overlay.root.clone(); let captured = std::mem::take( &mut *self @@ -681,10 +753,40 @@ impl GroupCommit { apply_durably(&root, only).await?; return Ok(committed(changes)); } + if !journaled { + for (at, change) in changes.iter().enumerate() { + if let Err(e) = apply_durably(&root, change).await { + let _ = restore(&root, &changes[..at]).await; + return Err(e); + } + } + return Ok(committed(changes)); + } let journal = root.join(COMMIT_JOURNAL_REL); + // The outermost directory the journal's parent chain lacks: a + // hosted run (no vendored ledger) creates `.socket/vendor/` only to + // hold the journal, and prunes it again once the journal is gone. + let mut created: Option = None; if let Some(parent) = journal.parent() { + created = parent + .ancestors() + .take_while(|dir| *dir != root && std::fs::symlink_metadata(dir).is_err()) + .last() + .map(Path::to_path_buf); tokio::fs::create_dir_all(parent).await?; } + let prune_created = |journal: &Path| { + let Some(top) = created.as_deref() else { + return; + }; + let mut level = journal.parent(); + while let Some(dir) = level { + if !dir.starts_with(top) || std::fs::remove_dir(dir).is_err() { + break; + } + level = dir.parent(); + } + }; super::fs::atomic_write_bytes(&journal, &journal_bytes(&changes)?).await?; crate::utils::failpoint::hit("group_commit_journal"); for (at, change) in changes.iter().enumerate() { @@ -696,7 +798,9 @@ impl GroupCommit { // command rolls the commit forward. return match restore(&root, &changes[..at]).await { Ok(()) => { - let _ = tokio::fs::remove_file(&journal).await; + if tokio::fs::remove_file(&journal).await.is_ok() { + prune_created(&journal); + } Err(e) } Err(_) => Err(std::io::Error::new(e.kind(), CommitPending(e))), @@ -712,6 +816,7 @@ impl GroupCommit { && tokio::fs::remove_file(&journal).await.is_ok() { sync_dir(journal.parent()); + prune_created(&journal); } Ok(committed(changes)) } @@ -1226,7 +1331,11 @@ mod tests { (".socket/gradle/socket-patch.settings.gradle", true), (".socket/vendor/maven2/.gitattributes", true), (".socket/vendor/gradle/.gitattributes", true), + (".socket/gradle/.gitattributes", true), + (".socket/vendor/.gitattributes", true), + (".socket/vendor/gradle/g/a/maven-metadata.xml", true), (".socket/vendor/gradle/g/a/1/a-1.jar", false), + (".socket/vendor/gradle/g/a/1/socket-patch.vendor.json", false), (".socket/vendor/npm/u/left-pad-1.3.0.tgz", false), (".socket/manifest.json", false), ("packages/a/.socket/vendor/npm/u/a.tgz", false), @@ -1296,6 +1405,77 @@ mod tests { assert_eq!(std::fs::read(&npmrc).unwrap(), b"earlier"); } + /// A group that defers removals keeps a reverted vendored unit until + /// its commit: a rollback or a dropped group leaves the unit (and the + /// lock wiring naming it) intact, and only a commit deletes it, pruning + /// the emptied `/` and `vendor/` levels but never `.socket/`. + #[tokio::test] + async fn deferred_unit_removals_wait_for_the_commit() { + use crate::utils::socket_dir::remove_tree_and_prune; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + let socket = root.join(".socket"); + let unit = socket.join("vendor/npm/u1"); + std::fs::create_dir_all(&unit).unwrap(); + std::fs::write(unit.join("a.tgz"), b"tgz").unwrap(); + std::fs::write(socket.join("apply.lock"), b"").unwrap(); + + // Not deferring: the removal is immediate, as before. + let other = socket.join("vendor/npm/u0"); + std::fs::create_dir_all(&other).unwrap(); + let plain = GroupCommit::begin(root); + remove_tree_and_prune(&other, &socket).await.unwrap(); + assert!(!other.exists()); + drop(plain); + + for keep in [true, false] { + let group = GroupCommit::begin(root); + group.defer_removals(); + let savepoint = group.savepoint(); + remove_tree_and_prune(&unit, &socket).await.unwrap(); + assert!(unit.exists(), "nothing is deleted before the commit"); + if keep { + group.rollback_to(savepoint); + group.commit().await.unwrap(); + assert!(unit.join("a.tgz").exists(), "a rolled-back removal is forgotten"); + } else { + drop(group); + assert!(unit.join("a.tgz").exists(), "a dropped group deletes nothing"); + } + } + + let group = GroupCommit::begin(root); + group.defer_removals(); + remove_tree_and_prune(&unit, &socket).await.unwrap(); + group.commit().await.unwrap(); + assert!(!unit.exists()); + assert!(!socket.join("vendor").exists(), "the emptied levels are pruned"); + assert!(socket.join("apply.lock").exists(), "`.socket/` itself stays"); + } + + /// A journal the commit had to create `.socket/vendor/` for (a hosted + /// run with no vendored ledger) leaves no empty directory behind. + #[tokio::test] + async fn a_journal_directory_the_commit_created_is_pruned() { + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + std::fs::create_dir_all(root.join(".socket")).unwrap(); + std::fs::write(root.join("a.lock"), b"old a").unwrap(); + std::fs::write(root.join("b.lock"), b"old b").unwrap(); + let group = GroupCommit::begin(root); + super::super::fs::atomic_write_bytes(&root.join("a.lock"), b"new a") + .await + .unwrap(); + super::super::fs::atomic_write_bytes(&root.join("b.lock"), b"new b") + .await + .unwrap(); + let changed = group.commit().await.unwrap(); + assert_eq!(changed.len(), 2, "two files go through the journal"); + assert_eq!(std::fs::read(root.join("b.lock")).unwrap(), b"new b"); + assert!(!root.join(".socket/vendor").exists()); + assert!(root.join(".socket").exists()); + } + #[tokio::test] async fn reads_see_the_runs_writes_and_nothing_reaches_disk_until_commit() { let tmp = tempfile::tempdir().unwrap(); diff --git a/crates/socket-patch-core/src/utils/socket_dir.rs b/crates/socket-patch-core/src/utils/socket_dir.rs index 5de8496ca..26ede4354 100644 --- a/crates/socket-patch-core/src/utils/socket_dir.rs +++ b/crates/socket-patch-core/src/utils/socket_dir.rs @@ -82,6 +82,16 @@ pub async fn remove_file_and_prune(path: &Path, stop_dir: &Path) -> std::io::Res /// destroy that tree. See [`containment`]. pub async fn remove_tree_and_prune(dir: &Path, stop_dir: &Path) -> std::io::Result<()> { containment::ensure_unlinked(guard_root(stop_dir), dir, "delete")?; + // A staged hosted takeover deletes the unit only once its commit is on + // disk (see `group_commit::defer_removal`). The prune bound is the + // level just below `stop_dir`, so `stop_dir` itself still survives. + let bound = dir + .ancestors() + .find(|a| a.parent() == Some(stop_dir)) + .unwrap_or(dir); + if super::group_commit::defer_removal(dir, bound) { + return Ok(()); + } crate::patch::copy_tree::remove_tree(dir).await?; if let Some(parent) = dir.parent() { prune_empty_dirs(parent, stop_dir).await; diff --git a/crates/socket-patch-core/src/vendor/bun_binary.rs b/crates/socket-patch-core/src/vendor/bun_binary.rs index 76d59fc88..406dabc53 100644 --- a/crates/socket-patch-core/src/vendor/bun_binary.rs +++ b/crates/socket-patch-core/src/vendor/bun_binary.rs @@ -548,13 +548,12 @@ pub(crate) async fn revert(entry: &VendorEntry, root: &Path, opts: RevertOpts) - } if !opts.keep_artifact { for mirror in mirrors_to_remove { - if let Err(e) = tokio::fs::remove_file(&mirror).await { + if let Err(e) = remove_mirror(&mirror).await { return RevertOutcome::failed(format!( "cannot remove workspace tarball {}: {e}", mirror.display() )); } - prune_mirror_parents(&mirror).await; } // The last npm-family entry leaves `.socket/vendor/npm/` (and // `.socket/vendor/`) empty: the shared helper prunes them so a @@ -618,6 +617,24 @@ pub(super) fn validate_mirror_path(root: &Path, rel: &str) -> Result std::io::Result<()> { + let bound = path + .ancestors() + .skip(1) + .take(5) + .find(|dir| dir.file_name().is_some_and(|name| name == ".socket")) + .unwrap_or(path); + if crate::utils::group_commit::defer_removal(path, bound) { + return Ok(()); + } + tokio::fs::remove_file(path).await?; + prune_mirror_parents(path).await; + Ok(()) +} + pub(super) async fn prune_mirror_parents(path: &Path) { // Only empty directories through the workspace's .socket, never the member. let mut parent = path.parent(); diff --git a/crates/socket-patch-core/src/vendor/bun_workspace.rs b/crates/socket-patch-core/src/vendor/bun_workspace.rs index ebb20fab7..ab40ca7ac 100644 --- a/crates/socket-patch-core/src/vendor/bun_workspace.rs +++ b/crates/socket-patch-core/src/vendor/bun_workspace.rs @@ -1,7 +1,7 @@ //! Integrity and repair for Bun's member-relative binary-lock tarballs. use std::path::{Path, PathBuf}; -use super::bun_binary::{prune_mirror_parents, undo_mirrors, validate_mirror_path}; +use super::bun_binary::{undo_mirrors, validate_mirror_path}; use super::bun_lockb::BunLockb; use super::path::parse_vendor_path; use super::state::{VendorEntry, WiringAction, WiringRecord}; @@ -173,10 +173,9 @@ pub(super) async fn cleanup(root: &Path, entry: &VendorEntry, dry_run: bool) -> } if !dry_run { for path in paths { - tokio::fs::remove_file(&path) + super::bun_binary::remove_mirror(&path) .await .map_err(|e| format!("cannot remove workspace tarball: {e}"))?; - prune_mirror_parents(&path).await; } } Ok(()) diff --git a/crates/socket-patch-core/src/vendor/cargo.rs b/crates/socket-patch-core/src/vendor/cargo.rs index be6150df5..35947000f 100644 --- a/crates/socket-patch-core/src/vendor/cargo.rs +++ b/crates/socket-patch-core/src/vendor/cargo.rs @@ -2015,12 +2015,16 @@ pub async fn revert_cargo_vendor_opts( // (and the caller keeps the ledger entry), so only the deletion is // skipped. if !dry_run && !keep_artifact { + // Best-effort (NotFound is fine): the shared per-unit revert removal + // also prunes the now-empty `.socket/vendor/cargo/` and + // `.socket/vendor/` levels, and waits for the commit of a staged + // hosted takeover. let uuid_dir = project_root.join(&base_rel); - let _ = remove_tree(&uuid_dir).await; // ignore NotFound - // Best-effort: prune the now-empty `.socket/vendor/cargo/` and - // `.socket/vendor/` levels so a fully-reverted project carries no - // vendor residue. `remove_dir` fails on non-empty. - prune_empty_vendor_levels(&uuid_dir).await; + let _ = crate::utils::socket_dir::remove_tree_and_prune( + &uuid_dir, + &project_root.join(crate::constants::SOCKET_DIR), + ) + .await; } out diff --git a/crates/socket-patch-core/src/vendor/golang.rs b/crates/socket-patch-core/src/vendor/golang.rs index 1af1e96ac..666516317 100644 --- a/crates/socket-patch-core/src/vendor/golang.rs +++ b/crates/socket-patch-core/src/vendor/golang.rs @@ -678,12 +678,16 @@ pub async fn revert_go_vendor_opts( // (and the caller keeps the ledger entry), so only the deletion is // skipped. if !dry_run && !keep_artifact { + // Best-effort (NotFound is fine): the shared per-unit revert removal + // also prunes the now-empty `.socket/vendor/golang/` and + // `.socket/vendor/` levels, and waits for the commit of a staged + // hosted takeover. let uuid_dir = project_root.join(&base_rel); - let _ = remove_tree(&uuid_dir).await; // ignore NotFound - // Best-effort: prune the now-empty `.socket/vendor/golang/` and - // `.socket/vendor/` levels so a fully-reverted project carries no - // vendor residue. `remove_dir` fails on non-empty. - prune_empty_vendor_levels(&uuid_dir).await; + let _ = crate::utils::socket_dir::remove_tree_and_prune( + &uuid_dir, + &project_root.join(crate::constants::SOCKET_DIR), + ) + .await; } if entry.took_over_go_patches { diff --git a/crates/socket-patch-core/src/vendor/jvm/apply.rs b/crates/socket-patch-core/src/vendor/jvm/apply.rs index 69ddbb929..29c4a1715 100644 --- a/crates/socket-patch-core/src/vendor/jvm/apply.rs +++ b/crates/socket-patch-core/src/vendor/jvm/apply.rs @@ -657,7 +657,7 @@ pub async fn revert(root: &Path, entry: &VendorEntry, opts: RevertOpts) -> Rever Some(bytes) => write_bytes(&path, bytes).await, None => { removed.push(rel.clone()); - remove_file(&path).await + remove_staged(rel, &path).await } }; if let Err(e) = res { @@ -670,7 +670,9 @@ pub async fn revert(root: &Path, entry: &VendorEntry, opts: RevertOpts) -> Rever // worse than a kept one. for w in present.into_iter().filter(|_| !kept) { let res = match reader.resolve(&w.file) { - Ok(path) => remove_file(&path).await.map_err(|e| e.to_string()), + Ok(path) => remove_staged(&w.file, &path) + .await + .map_err(|e| e.to_string()), Err(e) => Err(e), }; if let Err(e) = res { @@ -688,6 +690,19 @@ pub async fn revert(root: &Path, entry: &VendorEntry, opts: RevertOpts) -> Rever } } +/// Remove the project file `rel` (at `path`) for a revert. A captured file +/// goes through the overlay; an uncaptured one (a vendored tree file) is, +/// inside a group that defers removals (the hosted takeover's staged +/// revert), deleted only once that group commits: a dry run, a retracted +/// takeover or a failed commit leaves the artifact the restored wiring +/// still names. +async fn remove_staged(rel: &str, path: &Path) -> std::io::Result<()> { + if !group_commit::captures(rel) && group_commit::defer_removal(path, path) { + return Ok(()); + } + remove_file(path).await +} + /// The committed JVM layout only a JVM ledger entry ([`is_jvm_entry`]) can /// own (see [`layout::LEDGER_OWNED_PATHS`]). pub use super::layout::LEDGER_OWNED_PATHS; diff --git a/crates/socket-patch-core/src/vendor/jvm/gradle.rs b/crates/socket-patch-core/src/vendor/jvm/gradle.rs index 434b97d1f..a59158f48 100644 --- a/crates/socket-patch-core/src/vendor/jvm/gradle.rs +++ b/crates/socket-patch-core/src/vendor/jvm/gradle.rs @@ -3683,6 +3683,88 @@ mod tests { } } + /// The hosted takeover stages a vendored revert in a group that defers + /// artifact removals, then either drops the group (`--dry-run`), rolls + /// the revert back (a planner-refused, retracted takeover) or commits. + /// Only the commit may change the disk: the tree files, the derived + /// `maven-metadata.xml` (rewritten while a sibling version stays, + /// deleted with the last one) and the owned `.gitattributes` files all + /// stay byte-identical until then, and the commit lands exactly the + /// plain revert's result. + #[tokio::test] + async fn a_staged_revert_changes_nothing_until_its_group_commits() { + use crate::utils::group_commit::GroupCommit; + #[derive(Clone, Copy, Debug)] + enum End { + Drop, + Rollback, + Commit, + } + let sibling = JvmPatch { + version: "2.11.0", + uuid: UUID_B, + ..patch() + }; + let shapes: [&[(&str, &str)]; 2] = [ + &[("settings.gradle", "rootProject.name = 'x'\n")], + &[("settings.gradle", "plugins {\n id 'x' version '1'\n}\n")], + ]; + for files in shapes { + for with_sibling in [false, true] { + for end in [End::Drop, End::Rollback, End::Commit] { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path(); + testing::populate(root, files); + let mut ledger = BTreeMap::new(); + testing::vendor(root, Shape::Gradle, &patch(), &mut ledger) + .await + .unwrap(); + if with_sibling { + testing::vendor(root, Shape::Gradle, &sibling, &mut ledger) + .await + .unwrap(); + } + let (before, before_dirs) = (testing::snapshot(root), testing::dirs(root)); + + // What the plain (unstaged) revert leaves, on a copy. + let expected_dir = tempfile::tempdir().unwrap(); + for (rel, bytes) in &before { + let path = expected_dir.path().join(rel); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(path, bytes).unwrap(); + } + let mut expected_ledger = ledger.clone(); + let out = + testing::revert(expected_dir.path(), &patch(), &mut expected_ledger).await; + assert!(out.success, "{out:?}"); + let expected = testing::snapshot(expected_dir.path()); + + let group = GroupCommit::begin(root); + group.defer_removals(); + let savepoint = group.savepoint(); + let out = testing::revert(root, &patch(), &mut ledger).await; + assert!(out.success && !out.kept_artifact, "{out:?}"); + let ctx = format!("{files:?} sibling={with_sibling} {end:?}"); + assert_eq!(testing::snapshot(root), before, "staged: {ctx}"); + match end { + End::Drop => drop(group), + End::Rollback => { + group.rollback_to(savepoint); + group.commit().await.unwrap(); + } + End::Commit => { + group.commit().await.unwrap(); + assert_eq!(testing::snapshot(root), expected, "{ctx}"); + continue; + } + } + assert_eq!(testing::snapshot(root), before, "{ctx}"); + assert_eq!(testing::dirs(root), before_dirs, "{ctx}"); + } + } + } + } + #[test] fn an_apply_line_inside_a_comment_does_not_count() { let line = vendored_line(false, ""); diff --git a/crates/socket-patch-core/src/vendor/jvm/layout.rs b/crates/socket-patch-core/src/vendor/jvm/layout.rs index f98b43a9b..701d0ad0a 100644 --- a/crates/socket-patch-core/src/vendor/jvm/layout.rs +++ b/crates/socket-patch-core/src/vendor/jvm/layout.rs @@ -163,6 +163,8 @@ pub const CAPTURED_FILES: &[&str] = &[ super::gradle::SCRIPT_REL, super::maven_reactor::GITATTRIBUTES_REL, super::gradle::GITATTRIBUTES_REL, + super::gradle::SCRIPT_GITATTRIBUTES_REL, + super::gradle::VENDOR_GITATTRIBUTES_REL, super::coursier_tree::INDEX_REL, super::coursier_tree::GITIGNORE_REL, super::coursier_tree::GITATTRIBUTES_REL, diff --git a/docs/testing/yarn-berry-compatibility.md b/docs/testing/yarn-berry-compatibility.md index 6b821e06f..2edf74e27 100644 --- a/docs/testing/yarn-berry-compatibility.md +++ b/docs/testing/yarn-berry-compatibility.md @@ -119,7 +119,7 @@ What socket-patch does with those files: | leading BOM | kept | kept, both files | | mixed CRLF / LF, or a bare CR | refused untouched: `redirect_yarn_berry_mixed_line_endings` | refused before any write: `vendor_yarn_berry_mixed_line_endings` | | revert (`rollback`, `remove`, takeovers) | upstream entries reconstructed from registry metadata; mixed endings refuse as drift | byte-exact; a lock mixed after vendoring gets the restored entry in the terminator of the entry it replaces | -| mode takeover into this mode | the berry gates (line endings, `cacheKey`, `compressionLevel`) run BEFORE the vendored wiring is reverted; a refused purl stays vendored, byte-identical | the backend's project gates (both files' line endings, `cacheKey`, `compressionLevel`) run BEFORE the hosted redirect is reverted; a refused purl stays hosted, byte-identical | +| mode takeover into this mode | the vendored revert is staged and the hosted rewrite planned against it; a purl the berry rewriter refuses (line endings, `cacheKey`, `compressionLevel`, no berry checksum) is retracted and stays vendored, byte-identical | the backend's project gates (both files' line endings, `cacheKey`, `compressionLevel`) run BEFORE the hosted redirect is reverted; a refused purl stays hosted, byte-identical | Every reader — manifest-less `vex`, the lockfile inventory, the npm flavor sniff, `repair` — splits CRLF lines like LF ones and skips a leading BOM.