diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index afe5eac12..862c7785b 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -24,7 +24,7 @@ For task-oriented guidance, start with [usage](../../docs/usage.md), | `apply` | — | Agent mode: apply patches from the local manifest | | `rollback` | — | **Full-state rollback (v5.0, MAJOR)**: restore original files AND unwind vendored lockfile wiring / restore hosted pins to their upstream registry entries, remove the rolled-back entries from the manifest, and GC their blobs/archives; takes optional variadic positional `targets` (PURL \| UUID \| path glob). See [Rollback command contract](#rollback-command-contract-v50) | | `remove` | — | Restore and remove one patch across hosted, vendored, and agent state; requires positional `identifier`. | -| `repair` | `gc` | Download missing agent blobs, re-vendor missing/corrupt vendored artifacts (never re-synthesizing a lost ledger), and clean up unused ones (refuses with `lock_held` when a live process holds the lock; see "Lock lifecycle" below) | +| `repair` | `gc` | Download missing agent blobs, redownload missing/corrupt vendored artifacts (never re-synthesizing a lost ledger), and clean up unused ones (refuses with `lock_held` when a live process holds the lock; see "Lock lifecycle" below) | Rows are in `--help` order (v5.0): the hosted/vendored workflow (`scan` → `vex` → `vendor`, with `list` to inspect), then the agent-mode (in-place patching) commands. @@ -54,7 +54,7 @@ Every subcommand accepts the same set of "global" flags via a single shared `Glo | `--proxy-url` | — | `SOCKET_PROXY_URL` | `https://patches-api.socket.dev` | string | Public proxy when no token | | `--ecosystems` | `-e` | `SOCKET_ECOSYSTEMS` | (all) | CSV → `Vec` | Restrict to these ecosystems | | `--download-mode` | — | `SOCKET_DOWNLOAD_MODE` | **`diff`** | enum: `diff` \| `file` (`package` was removed and is rejected) | Patch artifact format | -| `--vendor-source` | — | `SOCKET_VENDOR_SOURCE` | **`auto`** | enum: `auto` \| `service` \| `build` | How `vendor` acquires the installable artifact (see "Prebuilt vendor artifacts") | +| `--vendor-source` | — | `SOCKET_VENDOR_SOURCE` | **`service`** | enum: `service` \| `auto` (alias) | How `vendor` acquires the installable artifact (see "Prebuilt vendor artifacts") | | `--maven-config` | — | — | (recorded choice, else `auto`) | enum: `auto` \| `none` | Maven reactor vendoring: write the repository tail (`auto`) or use only the fallback file repository (`none`). The choice persists in the vendor ledger. | | `--vendor-url` | — | `SOCKET_VENDOR_URL` | (active API/proxy base) | string | Base host for the vendoring-service package-reference request | | `--patch-server-url` | — | `SOCKET_PATCH_SERVER_URL` | (server-returned) | string | Override the host of the prebuilt-archive download URL (local-dev / testing) | @@ -138,11 +138,11 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **Throttling: bounded retry, then a reported failure.** Every patch-API JSON call (the batch query, the per-package patch lists, patch views and VEX record fetches, hosted package references) retries an HTTP `429` or `503` answer up to 3 times (`SOCKET_API_MAX_RETRIES=`, `0`-`10`; `0` = no retry). The wait honors `Retry-After` (delta-seconds or HTTP-date); a `Retry-After` over 30 s is not waited out — the answer is final at once — and one under the jittered first backoff step (`0`, a past date) waits that step instead. Without one it backs off 0.5 s / 1 s / 2 s (each step up to 8 s, with jitter in its upper half). All retries in one run share a 60 s wall-clock window that opens with the run's first retry: a retry whose wait would end after it closes is refused and the answer is final. Parallel requests wait in parallel, so each still gets its retries while the run adds at most about 60 s. Nothing else is retried (401/403 still drive the proxy fallback on the first answer; the public proxy's permanent `503 "Patch API is not configured"` is never retried on any path — the batch query still degrades to the per-package path at once, and a per-package lookup or patch view answering it is the same non-throttle failure it always was, so the legacy per-package path still skips that package), and a retried answer folds exactly where the first attempt's would have, so output is identical to an unthrottled run's. A request still throttled after that is a failure in the channel its siblings use: a failed batch is the human `Warning: API batch of failed: ` line and, under `--json`, a run-level `warnings[]` entry `{code: "api_batch_failed", detail: "API batch of failed: "}` (additive; `status` stays `success`, exit 0 — the other batches' packages are reported); a failed per-package patch-list query in the agent / hosted / vendored flows is the human `Warning: could not fetch details for : ` line and, under `--json`, `{code: "patch_details_failed", detail: "could not fetch details for : "}`. When every batch (or every patch-list query) fails, the existing all-failed error envelope and exit 1 apply. The error names the exhausted retry: `Rate limit exceeded (HTTP 429, gave up after 3 retries). Please try again later.` / `API request failed with status 503: (gave up after 3 retries)` (or `(Retry-After s exceeds the 30 s retry cap)` / `(the run's 60 s retry window has closed)`); with retries off it is the pre-retry text. On the token-less legacy per-package proxy path (a proxy without `POST /patch/batch`), a package still throttled (429 / over-capacity 503) after its retries fails its whole batch query, so every package in that batch goes unchecked and is reported through the batch-failure channel above (an unresolvable PURL, or a "not configured" 503, is still skipped individually). Pinned by `tests/scan_api_retry_e2e.rs` and the core crate's `tests/api_retry_e2e.rs`. -**Lockfile supplement (v3.4)**: `scan` discovery is no longer limited to installed trees. The project's lockfiles (`package-lock.json`/`npm-shrinkwrap.json`, `pnpm-lock.yaml` v9, `yarn.lock` classic + berry, `bun.lock`, `vlt-lock.json` (registry nodes, Socket-hosted pins included; vendored `file` nodes are left to the vendor ledger), `Cargo.lock`, `go.sum`, `composer.lock`, `Gemfile.lock`, `uv.lock`/`poetry.lock`/pinned `requirements.txt`) are inventoried and dependencies with NO installed copy join discovery — counts, the API lookup, the table (flagged ` [NOT INSTALLED]`, plus a stderr note), and the prune "scanned" set (a wiped node_modules no longer prunes lockfile-listed entries). JSON gains a top-level `lockfileOnlyPackages` count and an additive `notInstalled: true` on matching `packages[]` entries. `--apply` partitions lockfile-only patches out BEFORE download (calm `skipped`/`package_not_installed` records — never an error exit, never a manifest write); `--vendor` passes them through to the vendor engine's auto-fetch. Vendored-ledger entries likewise stay discoverable on a fresh clone (the committed artifact is the dependency). Global scans (`--global`) get no supplement. **Rush monorepos** (no root lockfile, `rush.json` present): the npm-lock inventory falls back to the Rush source-of-truth locks — `common/config/rush/pnpm-lock.yaml` plus every `common/config/subspaces/*/pnpm-lock.yaml` (`read_dir`-sorted, repo-relative paths preserved) — so a Rush repo's dependencies still join discovery. **Plug'n'Play layouts are an explicit refusal, not an empty inventory**: a `.pnp.*` loader means the npm packages are structurally unreachable in EVERY mode (under yarn PnP the installed-tree crawl is empty too — no `node_modules/`), so `scan` surfaces an additive top-level `warnings[]` array (`{code, detail}` objects, omitted when empty) carrying `yarn_pnp_unsupported` (same code as apply's refusal; remedy `yarn patch `) or `pnpm_pnp_unsupported` (pnpm's `node-linker=pnp` twin; pnpm remedies), plus a stderr `Warning: …` line on the human path. Exit code and `status` are deliberately unchanged (exit 0 / `success` — the same posture as hosted refusals, which exit 0 with `redirected: 0`); the warning is the machine-readable signal that nothing was checked. Pinned by `tests/e2e_safety_yarn_pnp.rs`. +**Lockfile supplement (v3.4)**: `scan` discovery is no longer limited to installed trees. The project's lockfiles (`package-lock.json`/`npm-shrinkwrap.json`, `pnpm-lock.yaml` v9, `yarn.lock` classic + berry, `bun.lock`, `vlt-lock.json` (registry nodes, Socket-hosted pins included; vendored `file` nodes are left to the vendor ledger), `Cargo.lock`, `go.sum`, `composer.lock`, `Gemfile.lock`, `uv.lock`/`poetry.lock`/pinned `requirements.txt`) are inventoried and dependencies with NO installed copy join discovery — counts, the API lookup, the table (flagged ` [NOT INSTALLED]`, plus a stderr note), and the prune "scanned" set (a wiped node_modules no longer prunes lockfile-listed entries). JSON gains a top-level `lockfileOnlyPackages` count and an additive `notInstalled: true` on matching `packages[]` entries. `--apply` partitions lockfile-only patches out BEFORE download (calm `skipped`/`package_not_installed` records — never an error exit, never a manifest write); `--vendor` passes them through to the vendor engine's server download. Vendored-ledger entries likewise stay discoverable on a fresh clone (the committed artifact is the dependency). Global scans (`--global`) get no supplement. **Rush monorepos** (no root lockfile, `rush.json` present): the npm-lock inventory falls back to the Rush source-of-truth locks — `common/config/rush/pnpm-lock.yaml` plus every `common/config/subspaces/*/pnpm-lock.yaml` (`read_dir`-sorted, repo-relative paths preserved) — so a Rush repo's dependencies still join discovery. **Plug'n'Play layouts are an explicit refusal, not an empty inventory**: a `.pnp.*` loader means the npm packages are structurally unreachable in EVERY mode (under yarn PnP the installed-tree crawl is empty too — no `node_modules/`), so `scan` surfaces an additive top-level `warnings[]` array (`{code, detail}` objects, omitted when empty) carrying `yarn_pnp_unsupported` (same code as apply's refusal; remedy `yarn patch `) or `pnpm_pnp_unsupported` (pnpm's `node-linker=pnp` twin; pnpm remedies), plus a stderr `Warning: …` line on the human path. Exit code and `status` are deliberately unchanged (exit 0 / `success` — the same posture as hosted refusals, which exit 0 with `redirected: 0`); the warning is the machine-readable signal that nothing was checked. Pinned by `tests/e2e_safety_yarn_pnp.rs`. -**Vendor auto-fetch (v3.4)**: `vendor`/`scan --vendor` no longer fail on lockfile-resolved packages with no installed copy. Already-vendored purls stage from their committed artifact (sha256-verified against the vendor ledger — a vlt directory artifact against its file inventory, which leaves out the links vlt creates inside it; offline-safe) when the ledger entry is at the manifest record's patch uuid; a superseding uuid fetches the pristine package instead, since the older artifact holds the older patch's bytes. Otherwise the pristine artifact is fetched per the lockfile resolution and verified against the lock's recorded integrity FAIL-CLOSED before any write: npm SRI (or yarn classic's sha1 fragment; for vlt the registry node's slot [2]), yarn berry's cache-zip checksum (rebuilt from the fetched tarball; cacheKey 10c0 only), Cargo.lock sha256 over the .crate, go.sum `h1:` dirhash over the module zip, composer `dist.shasum` (sha1), Gemfile.lock `CHECKSUMS` sha256, uv.lock wheel sha256 (pure `py3-none-any` wheels only). Entries the lock cannot verify are NEVER fetched (`vendor_fetch_unverifiable` warning + the calm `package_not_installed` skip). Registry bases honor `SOCKET_NPM_REGISTRY`, `SOCKET_CRATES_REGISTRY`, `SOCKET_GOPROXY` (else `GOPROXY`, `GONOPROXY` and `GOPRIVATE` the way go reads them — see the env table); npm/yarn/composer/gem/uv lock-recorded URLs are used verbatim. `--offline` refuses the fetch with the calm skip (the detail names the lockfile resolution). The fetch stages into a private tempdir — the project tree is never touched. **Deferred fetch (v5.0):** a purl the vendor ledger already covers (its entry records the record's patch uuid and the committed artifact is on disk — a file artifact only while it hashes to the ledger's `sha256`; not under `--force`), and a lockfile-only npm, cargo, golang or composer package the registry would fetch and verify (a lock entry with an integrity, or the pre-vendor resolution the ledger recovers, that none of its fetcher's pre-download refusals applies to: a yarn berry cacheKey other than 10c0, a go module go fetches without a proxy, a composer entry with no dist URL) while the patch service is enabled, are NOT downloaded up front (those backends ask the service first and read the pristine tree only on a local-build fallback; pypi and gem keep the up-front fetch, which their installed-variant probe reads): the fetch runs only if the backend reaches a branch that reads the pristine tree (a drifted committed copy rebuilt locally, a service miss). An in-sync re-run therefore makes no registry request, reports no `vendor_fetched_missing`, and succeeds with no network or under `--offline`. A deferred fetch that does run records its `vendor_fetched_missing` just ahead of the package's own event; one that fails, is unverifiable, or is refused by `--offline` reports the same events the up-front fetch would have. A package whose fetch would be refused is never deferred (a git, path or custom-registry cargo crate, say), so it keeps `vendor_fetch_unverifiable` + `package_not_installed` and is never vendored from the service's registry build. **Gem, local build only** (`--vendor-source build`, or no service config): a not-installed gem the lock can verify (bundler >= 2.6 `CHECKSUMS`) and no ledger entry covers is refused `gem_spec_missing` (`failed`, the backend's own detail) BEFORE any download — a downloaded `.gem` carries no eval-able stub gemspec, so a local build can never vendor it; no `vendor_fetched_missing` precedes it, and a refusal the backend would have reached first on the fetched copy reports as `gem_spec_missing` too. Not under `--dry-run`, which still fetches and previews the gem (`vendor_fetched_missing` + `verified`). +**Server artifact acquisition (v5.0)**: vendoring downloads the patched artifact for the selected UUID, including on a fresh checkout with no installed package. The CLI no longer downloads pristine packages, stages patch blobs, applies patches to vendor copies, or constructs archives. Backend lock and package-identity checks still run before acquisition; git and custom-registry Cargo sources are refused with `vendor_source_unsupported`. Healthy committed artifacts are reused offline. A changed UUID downloads a fresh server artifact; an unhealthy artifact at the recorded UUID uses the exact redownload procedure below. -**Vendored write durability (v5.0)**: every write is atomic (stage + rename), but only the durable commit points — lockfiles, `go.mod`/`go.sum`, `pom.xml`, `nuget.config`, `package.json`, `pnpm-workspace.yaml`, `.cargo/config.toml`, the Python/Ruby manifests, `.socket/vendor/state.json` and `redirect-state.json` — are fsynced on write. The content-verified artifacts under `.socket/vendor///` (patched copies, packed/rebuilt archives and sidecars, markers) are written without an fsync and made durable by one barrier (file + directory fsync, one `F_FULLFSYNC` per device on macOS) ahead of the next commit point — and, for an artifact rebuilt in place that no commit point follows, at the end of the vendored run's commit and when the command releases the apply lock — so a crash can only lose an artifact that no durable commit point names yet, which the next run rebuilds. +**Vendored write durability (v5.0)**: every write is atomic (stage + rename), but only the durable commit points — lockfiles, `go.mod`/`go.sum`, `pom.xml`, `nuget.config`, `package.json`, `pnpm-workspace.yaml`, `.cargo/config.toml`, the Python/Ruby manifests, `.socket/vendor/state.json` and `redirect-state.json` — are fsynced on write. The content-verified artifacts under `.socket/vendor///` (patched copies, packed/rebuilt archives and sidecars, markers) are written without an fsync and made durable by one barrier (file + directory fsync, one `F_FULLFSYNC` per device on macOS) ahead of the next commit point — and, for an artifact rebuilt in place that no commit point follows, at the end of the vendored run's commit and when the command releases the apply lock — so a crash can only lose an artifact that no durable commit point names yet, which the next run redownloads. **Vendored group commit (v5.0)**: `vendor`, `scan --mode vendored` and `get --mode vendored` capture every lockfile / manifest / config edit and every ledger save of the run in memory (reads inside the run see them) and commit them ONCE after the per-package loop — including the packages that succeeded in a run where others failed, so a completed run leaves the same files per-package commits would. Captured: every file under the project root outside `.socket/`, plus `.socket/vendor/state.json` and `.socket/vendor/redirect-state.json`; artifacts are written directly (see the durability note). A multi-file commit goes through a roll-forward journal, `.socket/vendor/.commit-journal.json` (the new bytes of every changed file, plus the bytes each replaces and their sha256; deleted once the commit completes). **Crash semantics**: before the journal is durable, nothing is committed — the lockfiles and ledgers are the pre-run ones and the run's artifacts are unreferenced orphans; after it, the next command that takes the apply lock replays the journal before reading anything (files already at their new bytes are left alone), so a locked command never observes a half-committed run. A journal that matches neither side of some file (edited by hand since the crash) is renamed to `.socket/vendor/.commit-journal.set-aside-.json` (keeping every file's pre-commit bytes) and stderr says what was done (`Warning: an interrupted vendored run's commit could not be finished as written: …`): the edited files are never written over; when they all still carry the commit's own lines the rest of the commit is finished around them, when none of them does the files the crash had already replaced are put back to their pre-commit bytes, and otherwise nothing is applied. A journal that is unreadable, names a path outside the lockfiles and ledgers, or would write through a symbolic link is set aside with nothing applied. A replay that fails on I/O keeps the journal and fails the lock acquire (`lock_io`, naming the journal). Read-only commands that take no lock (`vex`, `list`) may observe the interrupted state until then. A re-vendor under a newer uuid removes the replaced uuid's dir only after the commit (its `vendor_stale_artifact_removed` event follows the run's per-package events), and a golang takeover removes the `.socket/go-patches/` copy only after the commit that repoints `go.mod`. A commit write failure is the top-level error `vendor_commit_failed` (exit 1; the pre-run lockfiles and ledger stay — unless putting back the files already replaced failed too, in which case the journal is kept and the next locked command finishes the commit). `repair`, `vendor --revert` and `rollback` still save per entry. @@ -155,7 +155,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc * **Hosted and vendored mode (bare `scan` included) — project directories** (`run_project_dirs`). Each PATH is a directory, or a glob (`*?[`) matching directories, relative to `--cwd`; the set is sorted and deduplicated, and each directory is scanned on its own exactly as if it were `--cwd` (its own lockfiles, ledgers and `.socket/`). With more than one directory, each run is headed `== ==` on stdout (unless `--silent`), and the exit code is the worst of the runs. Usage errors (exit 2, stderr only, before any scan): a PATH that is not a directory (`` `X` is not a directory``), a glob matching no directory (`` `X` matches no directory``), an invalid glob, and `--json` with more than one directory (`--json takes one project directory (N given); run one scan per directory`), so stdout stays one document. * **Agent mode (and a mode-less `--prune`/`--global` report) — installed-path globs** scoping DISCOVERY at the **purl level**: a package is in scope iff ANY of its crawled installed copies sits under a matching path, and a selected package is then handled with ALL its copies (scoping selects which packages are considered, never which copies). Glob semantics (shared with `rollback`'s path targets, `src/path_scope.rs`): Unix-shell globs with `require_literal_separator` — `*`/`?` never cross a `/`, `**` spans directories; a pattern matching any **ancestor** directory of the copy path also matches, so a bare `scan packages/foo` scopes the whole subtree without `/**`; relative patterns match against the copy path relativized to `--cwd`, absolute patterns against the absolute path (the ONLY way to reach paths outside the project tree, e.g. `--global` stores — a relative pattern never matches outside `--cwd`); leading `./` and trailing `/` are normalized away, matching is purely textual (no filesystem access or symlink resolution), case-sensitive except on Windows (whose filesystems are not); an unparseable or empty pattern is a usage error (exit 2). **The prune universe is never narrowed**: the path filter is applied strictly AFTER the `scanned_purls` capture (and after `--ecosystems`), so `scan PATHS --prune` prunes exactly what an unscoped `scan --prune` would — a scoped scan can never treat an out-of-scope package as uninstalled (the same fail-safe as the `--ecosystems` filter). Lockfile-only and vendor-ledger supplement records have no installed path and are EXCLUDED from a path-scoped scan, surfaced as one run-level `path_scope_excluded_supplements` warning carrying the count. A scope matching nothing is a normal empty scan — exit 0, zero packages, **no GC** (the zero-package early return fires before any GC). `PATHS` combine with `--apply`/`--sync`/`--prune`/`--global`. Every scan JSON shape (success, zero-package, and error alike) carries an always-present `paths` key echoing the patterns verbatim (empty array when unscoped; a hosted/vendored per-directory run is unscoped, so it is `[]`). One-sentence duality rule: **a target that selects nothing is an error on `rollback` (exit 1) and an empty scan on an agent-mode `scan` (exit 0)**. -`scan --vendor` swaps the in-place apply for the vendor pipeline: discover → download the selected patch records **into memory** (no manifest write) → vendor every selected dependency via the same engine as the `vendor` command (under the same lock). Vendored mode is **manifest-free (v5.0)**: `.socket/manifest.json` is never written or read by a vendored run; each ledger entry carries `detached: true` plus an embedded copy of the patch record (`record`) as its verification source, and the run's footprint is `.socket/vendor/**` only. The vendor step's scope is what discovery selected — the former "whole manifest is vendored" re-vendor on an empty discovery is retired (`repair` verifies and rebuilds committed vendored state; `scan --prune` reconciles ledger entries whose dependency left the lockfile). A package the ledger holds at an older patch uuid is still **re-vendored automatically** when discovery selects the newer patch (its old uuid dir is removed — `vendor_stale_artifact_removed`); same-uuid re-runs reuse the embedded record, skip the patch-view fetch, and are `already_vendored` skips. **Legacy manifest-mode entries**: when a vendored run vendors a purl that also has a `.socket/manifest.json` record (a project vendored by a pre-5.0 binary, or by standalone `vendor` from an agent-mode manifest), that manifest record is dropped in the same run — the ledger becomes the owner (migration write); an emptied manifest is left as `{"patches": {}}`, never deleted. The migration is reported through the run-level `warnings[]` (stderr in human mode), never as a run error: `vendor_manifest_record_migrated` (`N manifest records moved to the vendor ledger (vendored mode is manifest-free): `) or `vendor_manifest_migration_failed` (the manifest or the ledger could not be read or rewritten; the legacy records were left in place) — so a corrupt `.socket/manifest.json` no longer fails a vendored run (standalone `vendor`, the one manifest-driven writer, still fails closed on it). With `--prune`, GC runs **after** the vendor step (the step never reads the manifest, and running the sweep last lets it reclaim what the run itself orphaned — a migrated legacy record's blobs, a superseded uuid dir). JSON output gains a `download` sub-object — the detached download envelope `{found, downloaded, skipped, failed, detached: true, patches: [{purl, uuid, action: "downloaded" | "skipped" | "failed", …}], warnings?}` (no `applied` field — nothing is applied in place; `detached: true` is pinned and always present; a `downloaded` record whose purl the ledger already holds at another uuid carries the additive `oldUuid` — the re-vendor the vendor step then performs — and its human `[fetch]` line reads ` (replacing )`) — and a `vendor` sub-object (a full vendor Envelope). Patch blobs are held in memory (see "Patch sources stay in memory" under the vendor contract). `--dry-run` previews per-patch `would_vendor` | `would_revendor` (+`oldUuid`) | `already_vendored` — plus, additive, `would_refuse` (+`errorCode`, `error`) for npm purls the wet run's Bun preflight (see the `get --mode vendored` bullet below) would refuse — without network downloads or disk writes; the preview never flips status or exit (the human path — `scan` and `get` alike, through one shared printer — prints `[would-refuse] (): ` lines behind the `--silent` gate). Interactive mode prompts "Download and vendor N patches?" (singular for one). +`scan --vendor` swaps the in-place apply for the vendor pipeline: discover → download the selected patch records **into memory** (no manifest write) → vendor every selected dependency via the same engine as the `vendor` command (under the same lock). Vendored mode is **manifest-free (v5.0)**: `.socket/manifest.json` is never written or read by a vendored run; each ledger entry carries `detached: true` plus an embedded copy of the patch record (`record`) as its verification source, and the run's footprint is `.socket/vendor/**` only. The vendor step's scope is what discovery selected — the former "whole manifest is vendored" re-vendor on an empty discovery is retired (`repair` verifies and redownloads committed vendored state; `scan --prune` reconciles ledger entries whose dependency left the lockfile). A package the ledger holds at an older patch uuid is still **re-vendored automatically** when discovery selects the newer patch (its old uuid dir is removed — `vendor_stale_artifact_removed`); same-uuid re-runs reuse the embedded record, skip the patch-view fetch, and are `already_vendored` skips. **Legacy manifest-mode entries**: when a vendored run vendors a purl that also has a `.socket/manifest.json` record (a project vendored by a pre-5.0 binary, or by standalone `vendor` from an agent-mode manifest), that manifest record is dropped in the same run — the ledger becomes the owner (migration write); an emptied manifest is left as `{"patches": {}}`, never deleted. The migration is reported through the run-level `warnings[]` (stderr in human mode), never as a run error: `vendor_manifest_record_migrated` (`N manifest records moved to the vendor ledger (vendored mode is manifest-free): `) or `vendor_manifest_migration_failed` (the manifest or the ledger could not be read or rewritten; the legacy records were left in place) — so a corrupt `.socket/manifest.json` no longer fails a vendored run (standalone `vendor`, the one manifest-driven writer, still fails closed on it). With `--prune`, GC runs **after** the vendor step (the step never reads the manifest, and running the sweep last lets it reclaim what the run itself orphaned — a migrated legacy record's blobs, a superseded uuid dir). JSON output gains a `download` sub-object — the detached download envelope `{found, downloaded, skipped, failed, detached: true, patches: [{purl, uuid, action: "downloaded" | "skipped" | "failed", …}], warnings?}` (no `applied` field — nothing is applied in place; `detached: true` is pinned and always present; a `downloaded` record whose purl the ledger already holds at another uuid carries the additive `oldUuid` — the re-vendor the vendor step then performs — and its human `[fetch]` line reads ` (replacing )`) — and a `vendor` sub-object (a full vendor Envelope). Patch blobs are held in memory (see "Patch sources stay in memory" under the vendor contract). `--dry-run` previews per-patch `would_vendor` | `would_revendor` (+`oldUuid`) | `already_vendored` — plus, additive, `would_refuse` (+`errorCode`, `error`) for npm purls the wet run's Bun preflight (see the `get --mode vendored` bullet below) would refuse — without network downloads or disk writes; the preview never flips status or exit (the human path — `scan` and `get` alike, through one shared printer — prints `[would-refuse] (): ` lines behind the `--silent` gate). Interactive mode prompts "Download and vendor N patches?" (singular for one). **Vendored entries and the rest of the CLI.** Because nothing is in the manifest, vendored patches are invisible to `apply` (nothing to apply in place) but fully visible to `list` (listed from the ledger, labeled `Mode: vendored (recorded in .socket/vendor/state.json)` in human mode, exit 0 on a vendored-only project), `vex` (attested from the embedded records while a lockfile still wires the artifact — see "Manifest-less VEX"), `repair` (health-checked and rebuilt from the ledger), and `scan --prune` (lockfile-driven reconcile). They are exempt from standalone `vendor`'s manifest reconcile (`reconcile_dropped` never touches `detached` entries) and exit via `remove ` (which reverts them), `vendor --revert`, or `rollback`, whose vendored leg reverts every in-scope ledger entry (unscoped and identifier-scoped runs; path-scoped runs reach them only when an installed copy matches). @@ -175,7 +175,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm **get --mode and installed narrowing (v3.6).** `get --mode hosted|vendored` consumes the resolved patch(es) through the SAME engines as `scan --mode hosted|vendored`, so for the same selected (purl, uuid) set the on-disk result is identical by construction — the per-advisory selector for hosted/vendored (`get --save-only` then `vendor` still works). **Agent mode (v5.0 lock + residue rules)**: the download phase runs under `<.socket>/apply.lock` and hands the guard to the nested apply, so download → manifest write → apply is one lock window (the nested apply never re-acquires and inherits every caller flag — `--lock-timeout` and `--verbose` included); a failed acquire is `{status: "error", errorCode: "lock_held" | "lock_io", error}` on get's legacy envelope, exit 1, before any fetch (a read-only `.socket/` fails here, naming the lock path). `.socket/` and `.socket/blobs/` are created only when a record is actually persisted — an all-skipped or all-failed run leaves no `.socket/` on a fresh project — and a same-uuid `get ` re-run rewrites neither the manifest nor the blobs. Semantics: * **Hosted** (`get GHSA-… --mode hosted`): resolves the advisory, then hands the selected (purl, uuid) pairs to scan's hosted engine — reference grants, cross-mode takeover pre-revert, lockfile rewrite (no ledger, v5.0), gem stale-install probe, warnings, confirmation rules (cargo via `confirmed_cargo_uuids`, golang via `confirmed_golang_uuids` only) all identical to `scan --mode hosted`, and (v5.0) under the same `apply.lock` acquisition — taken around the first wet write, never on `--dry-run` or when nothing would be written; a failed acquire folds as top-level `errorCode: "lock_held" | "lock_io"` + string `error` (exit 1), and `--dry-run` under a held lock still exits 0. **No manifest write, no blobs, no ledger** — the lockfile edits are the persistence. JSON: get's legacy envelope gains the same nested `redirect` sub-object as scan's (`{mode:"hosted", redirected, rewrittenFiles, skipped, warnings, dryRun}`); the top-level shape is `{status, found, patches:[], warnings?}` — `downloaded`/`applied` are absent (nothing is downloaded into `.socket/`). Exit codes follow scan's hosted semantics: skipped grants and rewriter warnings never flip the exit; infra errors (reference fetch, file writes) exit 1. Human prompt: `Redirect N packages to the hosted patch server?` (singular for one; `--yes`/`--json`/non-TTY auto-accept as usual). This confirm is get's alone: `scan` never prompts. -* **Vendored** (`get GHSA-… --mode vendored`): the download phase is scan's vendored posture — **manifest-free (v5.0)**: the selected records are fetched into memory (`download_patch_records`; blobs held in memory; nothing under `.socket/` is written; the nested apply never runs), then scan's vendor step runs under the apply lock over exactly the selected records, like `scan --mode vendored` (no whole-manifest scope and no `[note]` about other records — that blast radius is retired with the manifest; a legacy manifest record for a vendored purl is migrated out of `.socket/manifest.json` the same way scan does it). JSON: get's envelope takes the detached download envelope's shape — `{status, found, downloaded, skipped, failed, detached: true, patches: [{purl, uuid, action: "downloaded" | "skipped" | "failed", …}], warnings?}` (`applied` is absent; `detached: true` is pinned; a `downloaded` record for a purl the vendor ledger holds at another uuid carries the additive `oldUuid`, derived from the ledger — the human `[fetch]` line reads ` (replacing )`) — and gains the nested `vendor` Envelope exactly like scan's `result["vendor"]`; a vendor-step error folds the partial envelope + `{status:"error", error:{code,message}}` in (a pre-failure takeover reconcile may have already mutated the ledger — its events must reach the consumer). Exit: download failures or vendor `has_errors` → `partial_failure`/1. Human prompt: `Download and vendor N patches?`; `--dry-run` prints `[dry-run] Would download and vendor N patches. No changes made.` on both identifier paths (uuid and search). Telemetry mirrors scan's vendored arms (`track_outcomes_for_vendor` / `track_patch_vendor_failed`). **Bun vendored preflight (additive)** — shared by `get --mode vendored` on both its paths and `scan --mode vendored`: before ANY patch download, and only when the selection holds a `pkg:npm/` purl, the download phase reads `bun.lock`/`bun.lockb` once (`preflight_vendor`) and, when the vendor backend would refuse the project — a malformed, unreadable or unsupported `bun.lockb` → `vendor_bun_lockb_invalid`; an unreadable `bun.lock` → `vendor_lockfile_missing`; a `lockfileVersion` other than 0/1/2 or a non-canonical `packages` grammar → `vendor_lockfile_version_unsupported`; `workspace:` packages in a lock below version 2 → `vendor_bun_workspace_unsupported` — every `pkg:npm/` result becomes `{action:"failed", errorCode:, error:}` with NO fetch (the patch view is never requested) and no patch record; other ecosystems' results are untouched. **Search path** (`get --mode vendored`) and `scan --mode vendored`: the records ride `patches[]` / `download.patches[]` with `downloaded: 0`, the download phase writes nothing under `.socket/` (v5.0 — a pre-existing `.socket/manifest.json`, including a record seeded for another purl, is left byte-untouched), the vendor step still runs over the remaining records (no event for the refused purl), exit `partial_failure`/1. **uuid path** (`get --mode vendored`): the uuid lookup is the only fetch; the run exits 1 BEFORE the vendor step with exactly `{status:"error", found:1, downloaded:0, skipped:0, failed:1, error:{code, message}, patches:[{purl, uuid, action:"failed", errorCode, error}]}` (the `error` OBJECT is the vendored-mode error shape of the vendor-step fold-in above) and writes nothing — no `.socket/` on a fresh project; human mode prints `Error (): ` on stderr. **Already-vendored exemption**: a purl is exempt from the workspace refusal only when every instance of its `name@version` in `bun.lock` is already a `.socket/vendor/npm/…` local tuple (any uuid; the digest-less 2-tuple counts) — the engine's own criterion — so in-sync re-runs, `repair`, and a superseding patch uuid on a project vendored before it grew a workspace member all flow to the engine (re-pinning an already-local tuple adds no workspace-relative exposure); a wiped ledger alone is not a refusal (the engine path decides). UUID equality in the ledger alone never exempts a purl: `rollback --preserve-state` retains its record after unwiring. Dry-run refusal takes priority over `already_vendored`. **Unreadable vendor ledger**: a `.socket/vendor/state.json` the preflight cannot read or parse is itself the refusal — `vendor_state_unreadable` with the io/parse detail, fail-closed (nothing is exempt) — on the uuid path, the search / `scan` path and the `--dry-run` preview alike; never a Bun lock code. **`--silent`** is "errors only" and never mutes the refusal: the code-tagged `[error] (): ` (per-patch paths) / `Error (): …` (uuid path) line stays on stderr with an empty stdout. **`--dry-run`** previews the refusal as the additive `would_refuse` action (see `--dry-run` below). Agent-mode `get --save-only` is NOT preflighted (record-only intent has no consumption precondition). Pinned by `tests/vendor/in_process_vendor_bun.rs` (exact uuid-path envelope, seeded-manifest survival, `--silent`, `--dry-run`) and `tests/scan_vendor_e2e.rs`. +* **Vendored** (`get GHSA-… --mode vendored`): the download phase is scan's vendored posture — **manifest-free (v5.0)**: the selected records are fetched into memory (`download_patch_records`; no blob staging; nothing under `.socket/` is written; the nested apply never runs), then scan's vendor step runs under the apply lock over exactly the selected records, like `scan --mode vendored` (no whole-manifest scope and no `[note]` about other records — that blast radius is retired with the manifest; a legacy manifest record for a vendored purl is migrated out of `.socket/manifest.json` the same way scan does it). JSON: get's envelope takes the detached download envelope's shape — `{status, found, downloaded, skipped, failed, detached: true, patches: [{purl, uuid, action: "downloaded" | "skipped" | "failed", …}], warnings?}` (`applied` is absent; `detached: true` is pinned; a `downloaded` record for a purl the vendor ledger holds at another uuid carries the additive `oldUuid`, derived from the ledger — the human `[fetch]` line reads ` (replacing )`) — and gains the nested `vendor` Envelope exactly like scan's `result["vendor"]`; a vendor-step error folds the partial envelope + `{status:"error", error:{code,message}}` in (a pre-failure takeover reconcile may have already mutated the ledger — its events must reach the consumer). Exit: download failures or vendor `has_errors` → `partial_failure`/1. Human prompt: `Download and vendor N patches?`; `--dry-run` prints `[dry-run] Would download and vendor N patches. No changes made.` on both identifier paths (uuid and search). Telemetry mirrors scan's vendored arms (`track_outcomes_for_vendor` / `track_patch_vendor_failed`). **Bun vendored preflight (additive)** — shared by `get --mode vendored` on both its paths and `scan --mode vendored`: before ANY patch download, and only when the selection holds a `pkg:npm/` purl, the download phase reads `bun.lock`/`bun.lockb` once (`preflight_vendor`) and, when the vendor backend would refuse the project — a malformed, unreadable or unsupported `bun.lockb` → `vendor_bun_lockb_invalid`; an unreadable `bun.lock` → `vendor_lockfile_missing`; a `lockfileVersion` other than 0/1/2 or a non-canonical `packages` grammar → `vendor_lockfile_version_unsupported`; `workspace:` packages in a lock below version 2 → `vendor_bun_workspace_unsupported` — every `pkg:npm/` result becomes `{action:"failed", errorCode:, error:}` with NO fetch (the patch view is never requested) and no patch record; other ecosystems' results are untouched. **Search path** (`get --mode vendored`) and `scan --mode vendored`: the records ride `patches[]` / `download.patches[]` with `downloaded: 0`, the download phase writes nothing under `.socket/` (v5.0 — a pre-existing `.socket/manifest.json`, including a record seeded for another purl, is left byte-untouched), the vendor step still runs over the remaining records (no event for the refused purl), exit `partial_failure`/1. **uuid path** (`get --mode vendored`): the uuid lookup is the only fetch; the run exits 1 BEFORE the vendor step with exactly `{status:"error", found:1, downloaded:0, skipped:0, failed:1, error:{code, message}, patches:[{purl, uuid, action:"failed", errorCode, error}]}` (the `error` OBJECT is the vendored-mode error shape of the vendor-step fold-in above) and writes nothing — no `.socket/` on a fresh project; human mode prints `Error (): ` on stderr. **Already-vendored exemption**: a purl is exempt from the workspace refusal only when every instance of its `name@version` in `bun.lock` is already a `.socket/vendor/npm/…` local tuple (any uuid; the digest-less 2-tuple counts) — the engine's own criterion — so in-sync re-runs, `repair`, and a superseding patch uuid on a project vendored before it grew a workspace member all flow to the engine (re-pinning an already-local tuple adds no workspace-relative exposure); a wiped ledger alone is not a refusal (the engine path decides). UUID equality in the ledger alone never exempts a purl: `rollback --preserve-state` retains its record after unwiring. Dry-run refusal takes priority over `already_vendored`. **Unreadable vendor ledger**: a `.socket/vendor/state.json` the preflight cannot read or parse is itself the refusal — `vendor_state_unreadable` with the io/parse detail, fail-closed (nothing is exempt) — on the uuid path, the search / `scan` path and the `--dry-run` preview alike; never a Bun lock code. **`--silent`** is "errors only" and never mutes the refusal: the code-tagged `[error] (): ` (per-patch paths) / `Error (): …` (uuid path) line stays on stderr with an empty stdout. **`--dry-run`** previews the refusal as the additive `would_refuse` action (see `--dry-run` below). Agent-mode `get --save-only` is NOT preflighted (record-only intent has no consumption precondition). Pinned by `tests/vendor/in_process_vendor_bun.rs` (exact uuid-path envelope, seeded-manifest survival, `--silent`, `--dry-run`) and `tests/scan_vendor_e2e.rs`. **Lock-text refusals before the download (v5.0)** — shared by `get --mode vendored` on both its paths and `scan --mode vendored`, after the Bun preflight above and the ledger's `already vendored` skip: a `pkg:npm/` result in a **pnpm, yarn classic or yarn berry** project, or a `pkg:cargo/` result, that its vendor backend refuses on the project's lock and manifest text alone is refused BEFORE its patch view is fetched — the pnpm / classic / berry gates the backend runs before it reads the package (coordinates, the lock and manifest reads and their line-ending / version / `cacheKey` / `.yarnrc.yml` gates, override and `resolutions` conflicts, the lock entry present and rewritable) and cargo's `locked_version_mismatch` (only when it is the crate's FIRST refusal; an in-tree `cargo vendor` copy still refuses in the loop as `already_vendored_in_tree`). **Scope:** only a package the vendor loop would hand to its backend is refused early — one installed on disk (the loop's own qualified-aware resolver plus the npm identity lookup), or one the lockfile inventory resolves to a verifiable registry source (a lock entry with an integrity, or the ledger-recovered pre-vendor resolution — exactly the entry the pristine fetch would use). A package absent from the lock and not installed never reached its backend and is untouched: its view is fetched, it downloads, and the vendor loop skips it `skipped` / `package_not_installed` as in v4.x (so cargo's `locked_version_mismatch` is refused early only for a crate installed at the unlocked version). The result becomes `{action:"failed", errorCode:, error:}` in `download.patches[]` / `patches[]` with the backend's exact code and detail, no view and no pristine fetch, no patch record, and therefore no vendor event: compared with v4.x, `download.downloaded` drops and `download.failed` rises by the number of such packages, `vendor.summary.failed` and `vendor.events` lose their `failed` events, and a lockfile-only package among them loses its `vendor_fetched_missing` event (it is never fetched). Exit code and top-level `status` are unchanged (`partial_failure`/1); the nested `vendor.status` becomes `success` when those refusals were the vendor step's only failures (observed on the depscan fixture: 3 refusals, `partialFailure` → `success`), and when every selected package is refused this way the human `scan --vendor` arm prints `Nothing was vendored: N patches failed (see above).`. **Precedence:** the lock-text refusal is decided before the view, so it wins over every view-derived outcome — a package that would also have been a paid-access 403 (`[PAID]`/no access), a failed view fetch, or a no-applicable-files skip reports the lock refusal instead (the Bun refusal and the ledger's `already vendored` skip still come first). The human `[error] (): ` line is printed during the download instead of the vendor step's failure line (the human (non-`--silent`) `scan --vendor` arm's baseline pre-check still fetches the views it verifies; only the download, the pristine fetch and the vendor step skip the package there). A purl the lockfiles pin hosted keeps the loop's refusal (its takeover restore rewrites the lock the gates read); other flavors (package-lock, pnpm-legacy, bun) and ecosystems are untouched, and `--dry-run` is unchanged. `vendor` (manifest-driven, no view fetch) keeps its per-package `failed` events but no longer fetches the pristine source of a lockfile-only package it refuses this way — the source is deferred to the backend, which refuses before reading it (no `vendor_fetched_missing` event and no registry request; a refused package whose registry is unreachable reports the gate's code instead of `vendor_fetch_failed`); only a package the lock resolves to a verifiable source is deferred, and one it does not resolve keeps its `package_not_installed` skip. Pinned by `tests/scan_vendor_e2e.rs` (`exact_download_plan`: scan and exact-purl get, pnpm and cargo scope), `tests/e2e_yarn_legacy_cachekey_refusal_build.rs` and `tests/vendor/vendor_rerun_no_network_e2e.rs`. * **Installed-version narrowing** (all modes, `get`'s search path): a CVE/GHSA fan-out returns one patch record per patched VERSION; get keeps only versions present here and emits calm `skipped` records (`errorCode: "package_not_installed"`) for the rest — never an error exit. Presence = installed on disk (qualified-aware resolver) ∪ already tracked in the manifest (record maintenance keeps working on hosts without an installed copy); hosted/vendored modes additionally count lockfile-resolved deps and vendor-ledger purls (mirroring scan's discovery supplements, including their `--global` gate). **Exempt** (no narrowing): UUID identifiers, exact-versioned PURL identifiers (explicit intent), `--save-only` runs (record-only has no installation precondition — the fresh-clone record→vendor flow keeps working), `--all-releases`, and the package-name path (already installed-derived). When EVERY found patch is filtered out, get exits 0 with the additive status **`not_installed`** (`{status:"not_installed", found:N, downloaded:0, applied:0, patches:[], warnings?}`) — never `no_match`, which remains pinned to the fuzzy package-name path. PnP layouts are surfaced, not misreported: yarn-PnP npm results skip with `errorCode: "yarn_pnp_unsupported"` in every mode; pnpm-PnP skips carry `pnpm_pnp_unsupported` in agent/vendored modes; hosted mode — the refusal's own remedy — keeps ONLY the versions the raw `pnpm-lock.yaml` text actually resolves (boundary-anchored probe over the v5/v6/v9 key spellings, so a large fan-out never requests grants for every version ever patched), labels a JUDGED miss `package_not_installed` exactly like a non-PnP project (the layout blocked nothing — the lock was read and the version isn't resolved), and reserves the layout code for an unreadable lock (no judgment possible). When EVERY narrowed-out result is a PnP refusal, the human terminal names the layout instead of claiming "not installed" and never advises `--all-releases` (which cannot make PnP patchable); the JSON status stays `not_installed` — consumers dispatch on the per-record `errorCode`. Hosted mode also runs the per-release VARIANT filter (`filter_to_installed_releases`) on its search path before requesting grants — agent/vendored runs get it inside the download engines — with the same keep-all-plus-warning fallbacks (surfaced as `(release_narrowing)`-prefixed strings in `warnings[]`). An ecosystem this binary has no crawler for is likewise never judged: its results are KEPT (absence from a crawl that never looked carries no information — the same fail-safe as scan's prune GC). The human `Found N patches:` listing shows only the patches whose package version survived the narrowing (the narrowing is judged over every result, so an installed package's paid fix a free user cannot download still lists as `[PAID] (no access)`, while skip records and counts cover only accessible patches), sorted by PURL in natural version order (`4.17.2` before `4.17.10`); the narrowed-out ones are summarized on stderr in one line per reason (`Skipped N patches for M package versions not installed here (use --all-releases to include them).`), and `--verbose` adds one `[skip] ()` line per skipped version after that summary, in natural version order. When the candidates hold more patches than were selected and the pick was made without a menu (a paid user's auto-pick, `--yes`, a non-TTY run), a `Selected:` block names the patch (purl, tier, short uuid, advisories) that will be installed before the prompt. Machine output (the prompt count, the JSON envelope) uses the kept set, unchanged. The finer per-release variant narrowing (`filter_to_installed_releases`) is unchanged and still runs inside the download engines (and before an agent-mode `--dry-run` preview, so the preview names only the variants a wet run would fetch). @@ -574,113 +574,15 @@ pin's paired `pyproject.toml` `[tool.uv.sources]` entry, a vlt pin in a `vlt-loc are withheld from the lock basis); any other unattributed uuid still is. `--vex` works as on the manifest-driven path. Without hosted pins the no-manifest no-op below is unchanged. -**Prebuilt vendor artifacts (`--vendor-source`)**: by default (`auto`) `vendor` first tries to -DOWNLOAD the already-built patched artifact + integrity from the patch.socket.dev vendoring service, -and silently falls back to building it locally on any non-fatal miss. `service` requires the service -(fail-closed); `build` always builds locally (the pre-service behavior). The download is a two-step -flow on the configured API/proxy host (`--vendor-url` overrides it): a package-reference POST -(`/v0/orgs/{slug}/patches/package` authenticated, else the public proxy's `/patch/package`) yields a -grant-tokenized serve URL + integrity, then a GET fetches the archive (`--patch-server-url` rewrites -that URL's host for local-dev / testing). The downloaded bytes are ALWAYS integrity-verified before -use (sha512 SRI for every ecosystem; golang additionally the `h1:` module dirhash) — a mismatch is a -hard error, never a silent fallback. A service-vended package reports each patched file as -`AlreadyPatched` (trust is the verified service integrity, not a local re-apply). The fallback ladder -per service outcome: - -| Service outcome | `auto` | `service` | -|---|---|---| -| granted/reused, integrity ok | **use service** | **use service** | -| integrity mismatch (including the gem stub gemspec) | **refuse** (`vendor_prebuilt_integrity_mismatch`; npm: the package fails with the integrity detail). Tampered bytes never fall back to a local build | refuse (same) | -| integrity ok, but the archive does not carry the patched files (a member at a recorded path fails its `afterHash`; checked for cargo/golang/composer/gem after extraction, and for maven/nuget/pypi/npm before the archive is written; npm under `service` fails the package with the detail) | local build + `vendor_prebuilt_layout_mismatch` | refuse (`vendor_prebuilt_required`) | -| still building (`pending_build` / serve 408) | local build + `vendor_prebuilt_pending` | refuse | -| not built / withdrawn / not found / no usable artifact | local build (quiet) | refuse | -| gem stub gemspec missing / invalid | local build + `vendor_prebuilt_stub_missing` / `vendor_prebuilt_stub_invalid` (invalid + gem not installed: refuse `vendor_prebuilt_stub_invalid` — no stub source exists) | refuse (`vendor_prebuilt_required` / `vendor_prebuilt_stub_invalid`) | -| 401 / 403 grant / 5xx / network error | local build + `vendor_prebuilt_unavailable` | refuse | -| `--offline` | local build | refuse (`vendor_service_offline_conflict`) | -| no API client configured (library callers of the vendor engine; the CLI always configures one) | local build | refuse (`vendor_prebuilt_required`) | - -`--vendor-source` governs ACQUISITION, not reuse: a re-run whose committed artifact the ledger -vouches for (npm tarball / pypi wheel: path under this patch uuid, no symlink, whole-file sha256 and -size equal to the ledger, every afterHash verified from the same bytes; the dir-shaped ecosystems: -the wired copy's afterHashes) keeps it in every mode — no service request, no local build, no -rewrite — whichever source built it. So a service outage (or its recovery) never re-vendors an -already-vendored package: the re-run is `already_vendored`, including under `service` + -`--offline`, and `build` does not rebuild a service-built artifact (delete the uuid dir to force -a rebuild). The ledger records no provenance, so `service` cannot tell a locally built committed -artifact from a prebuilt one; it keeps what verifies. A lock that drifted off a verified committed -artifact (a relock, a hand revert) is re-wired to those exact bytes (pypi re-scans report the -Verbose `vendor_artifact_reused`). Service round trips are retried on transport errors and -429/500/502/503/504 (3 attempts, exponential backoff with jitter, `Retry-After` honored, 4s cap); -after 2 consecutive exhausted fetches the rest of the run skips the service (`auto` builds -locally, `service` refuses). - -**golang service leg staging (v5.0)**: the module zip is downloaded, extracted and `h1:`-verified in a `.socket-stage` sibling and swapped into place only afterwards; a failed re-download of a WIRED, present copy keeps the copy and its `replace` directive, while a missing copy still drops the dangling directive. - -Coverage today: **npm** (all lock flavors), **pypi** (wheel — sdist falls back / refuses), **cargo** -(download + extract the `.crate`), **golang** (download + extract the module zip, verify the `h1:` -dirhash, wire the `replace`), **composer** (download + extract the dist zip), **gem** (download + -extract the `.gem`, plus a `gem-stub-gemspec` SECOND artifact), **nuget** (download the prebuilt -`.nupkg`), and **maven** (download the prebuilt `.jar` + the registry pom; in the fail-closed -`service` coverage list since the `service_mode_gate_admits_maven` fix — PR #117 shipped the backend -but left maven off `SERVICE_ECOSYSTEMS`). The Tier-B ecosystems -(cargo/golang/composer/gem) download the patched archive and extract it into the vendor directory — -the same source tree the local build commits — then run the existing path-dep wiring; their -build-equivalence is exercised by the toolchain-backed e2e suites (which skip when the package -manager is absent). **gem** needs the extra `gem-stub-gemspec` artifact because a path-sourced gem -needs an eval-able stub gemspec that the `.gem` archive doesn't carry in bundler's required form (a -`.gem` keeps the gemspec as YAML in `metadata.gz`); the converter generates that stub and serves it -alongside the `.gem`, and the gem backend downloads + integrity-verifies both. A served gem whose -stub is missing (a native-extension gem, for which the converter emits no stub, or a patch built -before the stub rollout) is treated as a service miss — `auto` falls back to the local build, -`service` refuses (`vendor_prebuilt_required`). A served stub that is present but INVALID — it -fails the rubygems `summary`/`authors` bar, so every bundler major would reject the vendored -path source at install time (a defect the 2026-08-19 live matrix found in every then-published -gem stub) — follows the same miss policy under its own code (additive/MINOR): `auto` falls back -to the local build with a loud `vendor_prebuilt_stub_invalid` warning naming the missing -attributes, `service` refuses with `vendor_prebuilt_stub_invalid`. (Semver note: before the -hardening, `service` mode exited 0 here while writing a stub bundler rejects — an UNINSTALLABLE -project. The refusal is the bug fix; the exit-0 was the defect, so this rides a MINOR.) When the -invalid-stub fallback finds the gem is ALSO not installed locally (no `specifications/` stub to -derive), the vendor refuses with the same `vendor_prebuilt_stub_invalid` code, naming the served -defect and the install-the-gem remedy. The locally-derived stub is validated at the same write -choke point: a corrupted local `specifications/` stub failing the bar refuses with -`gem_spec_invalid` naming the file. The bar is a conservative textual heuristic matched to what -rubygems 3.3–3.6 actually hard-fails (no assignment of `summary`; no `authors`/`author` -assignment, or one that collapses to no String elements — `[]`/`nil`/`[nil]`/`%w[]`; nil/empty -strings are rubygems-tolerated and pass); a valid stub is still written byte-verbatim, and the -idempotent re-vendor path re-checks the ON-DISK stub, routing a pre-hardening invalid one into -the artifact rebuild. For any ecosystem with no service path at all -`auto`/`build` build locally as before, and `service` refuses with -`vendor_service_unsupported_ecosystem`. A successful service vend emits `vendor_prebuilt_downloaded`. -Unrelated to `--download-mode` (which selects the patch-CONTENT format for the local build). - -**Patch sources stay in memory (v3.4)**: vendoring never writes `.socket/blobs/`, `.socket/diffs/`, -or temporary patch files. Pre-existing `.socket/` artifacts (from a prior `apply`/`get`/`repair`) -are read in place; already-vendored purls re-stage patch content from the committed artifact itself -(uuid-matched against the ledger, every harvested blob self-verified by its afterHash — so in-sync -re-runs and fresh clones of vendored projects need no network); anything still missing is fetched -into memory via the patch-view endpoint. A vendored project's `.socket/` holds only `vendor/` -(v5.0 — vendored runs never write `manifest.json`; one exists only when standalone `vendor` was fed -by an agent-mode manifest, or as the `{"patches": {}}` husk left after a legacy record migrated -into the ledger). - -**Vendored artifact repair (v5.0)**: `repair` health-checks every ledger entry — per-file -afterHashes inside the artifact plus, for file-shaped artifacts (`.tgz`/`.whl`), the whole file -against the ledger's recorded sha256 (the rewired lock integrity references those exact bytes) — -and RE-VENDORS missing/corrupt artifacts through the same vendored backend `vendor`, `scan --mode -vendored` and `get --mode vendored` use. The artifact therefore comes from the same place a fresh -vendor gets it: under the default `--vendor-source auto` the patch service's prebuilt artifact is -downloaded again, with a local build from a lockfile-verified pristine source as the fallback -(and the only source under `--offline` / `--vendor-source build`). The wired hot paths rebuild -the artifact only: lockfiles stay byte-identical and the ledger entry keeps its recorded -pre-vendor originals. The re-vendored artifact is verified against the ledger fingerprint before -the run counts it (`rebuilt` event; a mismatch removes the artifact and fails with -`vendor_artifact_rebuild_failed`). The check is always against the ORIGINAL ledger entry: a source -that produces other bytes (a service archive re-packed since vendoring) is never committed — its -wiring and ledger entry are put back and repair falls back to the deterministic local build. A -corrupt artifact's afterHash-verified members are harvested as patch content (so `--offline` -repairs it from the installed copy) before it is moved aside for the rebuild, and put back when -nothing replaced it. +**Prebuilt vendor artifacts (`--vendor-source`)**: `service` is the default and `auto` is a compatibility alias. `build` is rejected at argument parsing. There is no local construction or fallback. Package-reference POSTs on the configured API/proxy (`--vendor-url` overrides it) return the download URL and integrity; `--patch-server-url` can override the download origin. The CLI verifies transfer integrity and patched-member afterHashes before writing. Pending builds, missing artifacts, service failures, wrong layouts and integrity mismatches fail closed. Fresh acquisition requires the network; healthy committed artifacts remain reusable offline. + +Coverage includes npm (all lock flavors), Python wheels and source distributions, Cargo crates, Go module zips, Composer dist zips, RubyGems, NuGet packages and JVM jars. Directory artifacts are extracted and receive only the existing package-manager layout transformations. RubyGems requires the server's separately verified `gem-stub-gemspec`; a missing or invalid stub is a failure. Yarn Berry checksums come from server metadata. Hosted Berry rollback retrieves upstream checksum metadata from `/upstream/npm/.json` and checks its package identity and upstream integrity against the registry; the CLI does not recreate the Berry zip. + +`--download-mode` controls agent patch content only. Vendored runs retain patch records in memory and embed them in the vendor ledger, without staging blobs or diffs. N-API and the hosted in-memory engine remain supported for callers such as the future GitHub App. + +**Vendored artifact repair (v5.0)**: `repair` checks the committed artifact and downloads a replacement for the same UUID into a temporary location. Before replacement it checks transfer integrity, patched-member hashes, and the original ledger's SHA-256 and size (file artifacts) or complete file inventory (directory artifacts). JVM repair also reproduces and checks the recorded repository metadata. Different downloaded bytes or inventories are refused; the old files, ledger and lockfiles remain intact. No installed package or patch blobs are required, and none are used to construct a replacement. Missing directory inventories cannot establish an exact replacement and require explicit re-vendoring. + +Successful redownloads retain the JSON `rebuilt` action and `summary.rebuilt` for compatibility, with `details.redownloaded`. Dry runs report `details.wouldRedownload` without writing. Human output says “Redownloaded”. Failed repair downloads report `vendor_artifact_redownload_failed`; the same failure during `vendor` reports `vendor_redownload_failed`. Offline repair refuses missing or corrupt artifacts even if installed copies and patch blobs exist. Revert and explicitly vendor again to adopt different bytes; repair never refreshes fingerprints or rewrites lockfiles to accept them. **The ledger is not rebuilt from lockfiles (v5.0).** A lockfile reference to `.socket/vendor///...` with NO ledger entry (state.json deleted or never committed) @@ -696,13 +598,7 @@ vendor ledger or vendor-path lockfile references — it runs the vendored phase in its lockfiles, v5.0, or a pre-v5 `.socket/vendor/redirect-state.json`) is a no-op: `repair` exits 0 with a `redirect_only_project` skip pointing at `scan --mode hosted` (hosted pins have no local artifacts to repair), rather than the `manifest_not_found` error a bare directory still -gets. Step 1's source download skips -vendored manifest entries and lockfile-referenced uuids (their content lives in the committed -artifact), so repairing a vendored project never re-litters `.socket/blobs`. `--dry-run` previews -(`details.wouldRebuild`); `--offline` rebuilds only from fully local sources and fails per-entry -otherwise; `vendor`/`scan --vendor` re-runs get the same rebuild for wired-but-broken artifacts -(`vendor_artifact_rebuilt` warning) and recover registry resolutions for missing committed -artifacts instead of failing. +gets. Agent blob acquisition skips vendored records and lockfile-referenced UUIDs, so repairing a vendored project does not populate `.socket/blobs`. ### Path convention + patch-UUID recovery (stable) @@ -740,7 +636,7 @@ to **six flavors**. | npm / yarn berry (node-modules linker) | (same tarball) | root `package.json` `resolutions` + `yarn.lock` entry with `checksum: 10c0/` of the berry cache-zip (reproduced from the tarball offline). **PnP is refused** (`.pnp.*` → different artifact pipeline) | `yarn install --immutable --check-cache`, cold cache. Refused if `__metadata.cacheKey ≠ 10c0` or a non-default `compressionLevel`. Both files keep their own layout — a CRLF lock (yarn's output on Windows) is spliced in CRLF, `package.json` is re-serialized with its BOM, indent, line ending and trailing-newline shape — so vendor + `--revert` round-trip byte-exactly; a lock or `package.json` MIXING CRLF and LF is refused before any write (`vendor_yarn_berry_mixed_line_endings`) | | npm / pnpm (lockfileVersion 9) | (same tarball) | root `package.json` `pnpm.overrides` (versioned selector) **+** `pnpm-lock.yaml` surgery (overrides / importer version / packages `resolution.integrity` / snapshots) | `pnpm install --frozen-lockfile --offline`, cold store (integrity-verified; byte-stable on pnpm 9 & 10). Other lockfileVersions: 5.4/6.0 route to the legacy backend below; anything else refused | | npm / pnpm LEGACY (lockfileVersion 5.4 = pnpm 7, 6.0 = pnpm 8; flavor `pnpm-legacy`) | (same tarball) | root `package.json` `pnpm.overrides` **+** legacy lock surgery (overrides / root dep + specifiers / packages rekey to a bare `file:` key with recomputed integrity / in-package dep refs). **No `pnpm-workspace.yaml` is written** (pnpm ≤ 8 reads overrides only from package.json). The lock's SPECIFIER is machine-ABSOLUTE — pnpm ≤ 8 absolutizes `file:` overrides itself — surfaced as `vendor_pnpm_legacy_absolute_specifier`. Legacy WORKSPACE locks (`importers:`) refused | same-path `pnpm install --frozen-lockfile --offline`, cold store (byte-stable on pnpm 7.33.5 / 8.15.9). A checkout at a DIFFERENT path fails the frozen check (path-bound specifier) and must run `pnpm install --offline --no-frozen-lockfile` once (the flag matters on CI, where pnpm defaults frozen on), which installs the vendored tarball and re-resolves only the specifier line | -| npm / bun (`bun.lock`, lockfileVersion 0, 1 or 2 — `vendor_lockfile_version_unsupported` otherwise) | (same tarball) | `bun.lock` only: the packages entry's registry 4-tuple → local 3-tuple with recomputed `sha512`; the entry's `{deps}` meta, the lock's version line and its line endings are preserved. A lock holding `workspace:` packages is refused `vendor_bun_workspace_unsupported` unless lockfileVersion is 2 — Bun 1.2–1.3 resolve a workspace member's local-tarball path relative to the MEMBER (ENOENT on our root-relative path), 1.4 relative to the lockfile, and a committed version-2 lock is the only proof every consumer runs Bun ≥ 1.4 (a deliberate over-approximation: a package declared only by the workspace root would install on version 1 too). The gate fires only on a run that would WRITE a new local tuple, so in-sync re-runs, `already_vendored` skips and `repair` rebuilds on such a lock pass. The detail names the version and the remedy: delete `bun.lock` and re-lock with Bun ≥ 1.4 (an in-place `bun install` keeps the existing lockfileVersion), or `--mode hosted`. Native binary support is described in the next row. `scan`/`get --mode vendored` apply all four refusals BEFORE downloading (see the `get --mode vendored` bullet). Bun 1.1.39–1.3.9 re-save the local tuple WITHOUT its `sha512` on any later lock re-save (`bun add`, `bun install` after a manifest change); the digest-less 2-tuple is recognised as the same wiring — an in-sync re-run stays `already_vendored` and re-pins the digest on disk (no new wiring record) when the committed artifact still holds the bytes the lock was written from — otherwise, as for any stale tuple of ours, the line is re-pinned and the fresh entry carries the new fingerprint — `repair` rebuilds through it, and `vendor --revert` / `rollback` restore the registry line over it (a 2-tuple at ANOTHER uuid is still `vendor_lock_entry_drifted`) | `bun install --frozen-lockfile`, cold cache (the local tarball's sha512 is enforced by Bun ≥ 1.3.10; 1.1.39–1.3.9 install it unverified — the committed artifact is the protection there) | +| npm / bun (`bun.lock`, lockfileVersion 0, 1 or 2 — `vendor_lockfile_version_unsupported` otherwise) | (same tarball) | `bun.lock` only: the packages entry's registry 4-tuple → local 3-tuple with recomputed `sha512`; the entry's `{deps}` meta, the lock's version line and its line endings are preserved. A lock holding `workspace:` packages is refused `vendor_bun_workspace_unsupported` unless lockfileVersion is 2 — Bun 1.2–1.3 resolve a workspace member's local-tarball path relative to the MEMBER (ENOENT on our root-relative path), 1.4 relative to the lockfile, and a committed version-2 lock is the only proof every consumer runs Bun ≥ 1.4 (a deliberate over-approximation: a package declared only by the workspace root would install on version 1 too). The gate fires only on a run that would WRITE a new local tuple, so in-sync re-runs, `already_vendored` skips and `repair` redownloads on such a lock pass. The detail names the version and the remedy: delete `bun.lock` and re-lock with Bun ≥ 1.4 (an in-place `bun install` keeps the existing lockfileVersion), or `--mode hosted`. Native binary support is described in the next row. `scan`/`get --mode vendored` apply all four refusals BEFORE downloading (see the `get --mode vendored` bullet). Bun 1.1.39–1.3.9 re-save the local tuple WITHOUT its `sha512` on any later lock re-save (`bun add`, `bun install` after a manifest change); the digest-less 2-tuple is recognised as the same wiring — an in-sync re-run stays `already_vendored` and re-pins the digest on disk (no new wiring record) when the committed artifact still holds the bytes the lock was written from — otherwise, as for any stale tuple of ours, the line is re-pinned and the fresh entry carries the new fingerprint — `repair` redownloads through it, and `vendor --revert` / `rollback` restore the registry line over it (a 2-tuple at ANOTHER uuid is still `vendor_lock_entry_drifted`) | `bun install --frozen-lockfile`, cold cache (the local tarball's sha512 is enforced by Bun ≥ 1.3.10; 1.1.39–1.3.9 install it unverified — the committed artifact is the protection there) | | npm / bun binary (`bun.lockb`, native binary format 1, 2 or 3) | (same tarball) | Rewrite matching binary package resolutions and integrity in place; preserve topology and unrelated metadata, update binary offsets and the package metadata hash. Text `bun.lock` takes precedence. `bun_lockb_package` wiring snapshots recover pristine registry metadata for repair and support per-package revert and hosted ↔ vendored migration. Binary discovery and rewrites require no installed Bun runtime. Malformed or unsupported content refuses `vendor_bun_lockb_invalid` before download or takeover. | Frozen installs with the original compatible Bun reader; see `docs/testing/bun-compatibility.md` for the release matrix and historical runtime integrity limits. | | npm / vlt (`vlt-lock.json`, lockfileVersion 0 or 1 — A0 locks without a version and every other version refuse `vendor_lockfile_version_unsupported`; flavor `vlt`) | patched package **directory** `.socket/vendor/npm//[@scope/]-/node_modules//` (the extra `node_modules/` level lets a package `require()` its own name), its `package.json` without `devDependencies`, plus `/.gitignore` (re-includes the payload against the project's ignores, ignores vlt's links inside it) and `/.gitattributes` (`-text`) | direct dependencies of the root or a workspace member only: the lock node becomes a `file` node for the directory, its importer edges and outgoing edges are re-keyed, and each importer's `package.json` spec becomes `file:`; every moved entry lands where vlt's serializer puts it. A node whose only extra is one peer context (`ṗ:N`, `peer.N`, `peer.<16 hex>`: from vlt 1.0.8 a root dependency with resolved peers, from rc.15 a workspace member's) becomes a `file` node without the extra, as vlt writes `file:` dependencies, keeping its peer edges; revert restores the extra-bearing DepID. Refused before any write: transitive targets (`vendor_vlt_transitive_unsupported`), two or more instances of one `name@version` or a modifier extra, importer `peer` edges, foreign registries, a git, remote-tarball or local-directory node of the same package name (vlt records no version for it), a package `vlt build` would build in place (`vendor_vlt_build_scripts_unsupported`), a name declared in several dependency fields (`vendor_lock_entry_unsupported`), a spec that disagrees with the lock (`vendor_vlt_lock_out_of_sync`), a payload git would ignore (`vendor_artifact_gitignored`), a purl vendored under another flavor (`vendor_flavor_changed`); era-A locks warn `vendor_vlt_legacy_lockfile`; an optional dependency (or any dependency node_modules still links to its installed upstream copy) gets `vendor_vlt_reinstall_required` | fresh checkout, `vlt ci` with cold caches: the patched bytes load and `vlt-lock.json` stays byte-identical, also through a warm and a cold `vlt install --frozen-lockfile` (checked on 1.2.0, 1.0.10, 1.0.4, 1.0.0-rc.32 and 1.0.0-rc.14, and on every release by `docs/testing/vlt-compatibility.md`); no-op installs, `vlt install `, `uninstall` and `vlt update` keep the direct dependency vendored. `vendor --revert` restores the registry node, edges and specs, keeping what vlt re-laid since, and refuses on drift | | cargo | crate dir `-/` (no `.cargo-checksum.json`) | (v5.0) `[patch.crates-io]` path entry in the **workspace-root `Cargo.toml`** (the manifest beside the `Cargo.lock` it detaches — never `.cargo/config*`) **+** Cargo.lock surgery (the `[[package]]` entry's `source`/`checksum` removed and its `version` set to the copy's TAGGED version `+socket.` — `+.socket.` when the version already has build metadata — with every lock reference that spells the old version rewritten, formats v1–v4; the copy's own `Cargo.toml` version carries the same tag, so the patched crate sees it in `CARGO_PKG_VERSION`; revert restores the lock byte for byte). Key: always the Socket-owned `-socket-` with `package = ""` (the full uuid hex when that key is taken), never the bare crate name — cargo lets a config-file `[patch]` item (project, ancestor directory or `$CARGO_HOME`) replace the manifest item with the same key whatever its version, so keys any of those configs use are avoided and a re-run moves an entry off a now-shadowed key; two versions of one crate are wired side by side. Pre-v5 wiring in `.cargo/config.toml` / `.cargo/config` is moved into `Cargo.toml` by a re-run (`vendor`, `scan`/`get --mode vendored`) or `repair` (`cargo_wiring_migrated` note; the ledger's `cargo_patch_entry` record then names `Cargo.toml`); a detached lock entry left unwired by the pre-v5 multi-version overwrite is re-wired the same way (`cargo_wiring_restored`); every revert removes both spellings | `cargo build --locked --offline` on a fresh checkout — single-version manifest `[patch]` also builds with no network on cargo older than 1.56 (the old config-file wiring's floor); two vendored versions of ONE crate need cargo 1.45 or newer (`--offline` from an empty CARGO_HOME is enough there); older cargo fails closed whatever the index state, and a project that does not pin cargo ≥ 1.45 (`rust-version` or toolchain file) gets the `cargo_multi_version_old_cargo` warning. Note: path deps build **without** `--cap-lints allow` | @@ -777,7 +673,7 @@ worse, lets a warm cache silently serve unpatched bytes): | golang | `go.sum` | untouched **by design** — directory `replace` targets are never sum-verified. Caveat: a user `go mod tidy` may prune the replaced module's go.sum lines; revert does not restore them (the next online build re-adds them) | | composer | `dist.{url,reference,shasum}`, `source.reference`, `content-hash` | `dist` → `{type: path, url, reference: ""}` (the uuid is preserved verbatim into `installed.json` — in-tree traceability); `source` removed; `content-hash` untouched (covers composer.json only) | | npm / yarn classic | `resolved "…#"` fragment + `integrity` SRI | both recomputed from the packed tarball (sha1 fragment + sha512 SRI); integrity line added when the registry block lacked one — yarn then enforces both | -| npm / yarn berry | `checksum: 10c0/` (over berry's cache zip) | recomputed by rebuilding berry's deterministic cache-zip from the tarball and hashing it (byte-identical to yarn's own); refused if the lock's `cacheKey`/`compressionLevel` would change the zip | +| npm / yarn berry | `checksum: 10c0/` (over Berry's cache zip) | supplied by the patch service; the CLI never builds the cache zip. Hosted rollback obtains the upstream checksum from `/upstream/npm/.json`, anchored to the registry tarball integrity. Unsupported cache keys or compression levels are refused. | | npm / pnpm | `packages[].resolution.integrity` (sha512) | recomputed from the tarball; the versioned `pnpm.overrides` selector pins exactly the patched version | | npm / bun | the packages-entry trailing `sha512-…` | recomputed from the tarball; tamper fails the frozen install on Bun ≥ 1.3.10 (URL/local tarball tuples are verified from 1.3.10, registry 4-tuples from 1.2.0 — so on 1.1.39–1.3.9 a hosted or vendored rewrite removes digest enforcement for the patched package; see `docs/testing/bun-compatibility.md`) | | npm / vlt | node slot [2] (sha512), slot [3] (resolved) | a `file` node carries none: the committed directory is the artifact, verified against the ledger's file inventory (`vendor_inventory_mismatch` on a planted, removed or changed file); integrity-less by vlt's design, like every `file:` directory dependency | @@ -1050,7 +946,7 @@ Empty string means unset at every layer: exported-but-empty flag-bound vars are | `SOCKET_PROXY_URL` | `--proxy-url` | `https://patches-api.socket.dev` | — | | `SOCKET_ECOSYSTEMS` | `--ecosystems` / `-e` | (all) | Comma-separated list. | | `SOCKET_DOWNLOAD_MODE` | `--download-mode` | `diff` | One of `diff` / `file`. | -| `SOCKET_VENDOR_SOURCE` | `--vendor-source` | `auto` | One of `auto` / `service` / `build`. | +| `SOCKET_VENDOR_SOURCE` | `--vendor-source` | `service` | `auto` is a compatibility alias for `service`; `build` is rejected. | | `SOCKET_VENDOR_URL` | `--vendor-url` | (active API/proxy base) | Vendoring-service package-reference host. | | `SOCKET_PATCH_SERVER_URL` | `--patch-server-url` | (server-returned) | Rewrites the prebuilt-archive download host. | | `SOCKET_OFFLINE` | `--offline` | `false` | — | @@ -1123,17 +1019,16 @@ Contract properties: ### Registry override env vars -Env-only knobs (no CLI flag) read by the vendor auto-fetch / artifact-rebuild paths in `socket-patch-core` (`src/vendor/registry_fetch.rs`, `src/vendor/maven_repo.rs`) and (v5.0) by the hosted upstream restore of `rollback` / `remove` / the vendored takeover (`src/patch/redirect/upstream/client.rs`, which honors the same bases). Each is the enterprise-mirror / test escape hatch for one registry base; trailing slashes are trimmed and an exported-but-empty value falls back to the default. Lock-recorded URLs (npm/yarn/composer/gem/uv `resolved`/dist URLs) are used verbatim and bypass these. +Env-only knobs used for hosted upstream restoration and JVM metadata verification. They do not enable local artifact construction. Trailing slashes are trimmed and empty values use the default. | Env var | Default | Notes | |---|---|---| -| `SOCKET_NPM_REGISTRY` | `https://registry.npmjs.org` | Base for conventional npm tarball URLs (vendor auto-fetch, including `repair`'s local-build fallback) and, v5.0, the version documents (`//`, a scoped name's `/` as `%2f`; `dist.tarball` / `integrity` / `shasum`) the npm-family and vlt upstream restore reads. | -| `SOCKET_CRATES_REGISTRY` | `https://static.crates.io/crates` | crates.io static `.crate` download host. | -| `SOCKET_GOPROXY` | `https://proxy.golang.org` | Go module proxy. Wins over the standard `GOPROXY` env var, whose first element is used otherwise. When that element is `off` or `direct`, or the module matches `GONOPROXY` (default `GOPRIVATE`), go would not ask a proxy, so the pristine fetch is refused (`vendor_fetch_unverifiable` + the calm `package_not_installed` skip) instead of falling back to `proxy.golang.org`. | +| `SOCKET_NPM_REGISTRY` | `https://registry.npmjs.org` | Base for npm version documents (`//`, a scoped name's `/` as `%2f`; `dist.tarball` / `integrity` / `shasum`) the npm-family and vlt upstream restore reads. | +| `SOCKET_GOPROXY` | `https://proxy.golang.org` | Go module proxy used for upstream restoration, honoring `GOPROXY`, `GONOPROXY` and `GOPRIVATE`. | | `SOCKET_MAVEN_REGISTRY` | `https://repo1.maven.org/maven2` | maven2 base for the fallback upstream-pom download. | | `SOCKET_CRATES_INDEX` | `https://index.crates.io` | v5.0 upstream restore: the crates.io sparse index whose `checksum` a restored `Cargo.lock` entry gets back. | | `SOCKET_GOSUMDB_URL` | `https://sum.golang.org` | v5.0 upstream restore: the checksum database the restored go.sum lines are checked against. Without it, `GOSUMDB=off` or a module matching `GONOSUMDB` (default `GOPRIVATE`) skips the database and the hashes come from the module proxy's bytes alone (`SOCKET_GOPROXY` above). | -| `SOCKET_PYPI_JSON_API` | `https://pypi.org/pypi` | PyPI's JSON API (`///json`): the vendored fetch's hash → URL lookup, and (v5.0) the release files whose sha256 the upstream restore writes back into every Python lock format. | +| `SOCKET_PYPI_JSON_API` | `https://pypi.org/pypi` | PyPI's JSON API (`///json`): the release files whose sha256 the upstream restore writes back into every Python lock format. | | `SOCKET_RUBYGEMS_URL` | `https://rubygems.org` | v5.0 upstream restore: the compact index (`/info/`) a restored `CHECKSUMS` entry is re-pinned from. | | `SOCKET_PACKAGIST_URL` | `https://repo.packagist.org` | v5.0 upstream restore: packagist's composer v2 metadata (`/p2//.json`) a restored `composer.lock` `dist` / `source` comes from. | | `SOCKET_NUGET_URL` | `https://api.nuget.org` | v5.0 upstream restore: the nuget.org API host whose catalog `packageHash` a restored `packages.lock.json` `contentHash` comes from. | @@ -1225,7 +1120,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `failed` | every command | A specific patch attempt failed. `errorCode` + `error` set. | | `removed` | `gc`/`repair`, `remove`, `rollback` | Data was removed from `.socket/` (or files rolled back). `bytes` optional. | | `verified` | `apply --dry-run`, `scan --dry-run` | The patch *would* apply cleanly. `files` lists previewed changes. | -| `rebuilt` | `repair` | A missing/corrupt vendored artifact was re-vendored in place (v5.0: never a lost ledger entry — see `vendor_ledger_missing`). `summary.rebuilt` counts these (the field is omitted while zero). | +| `rebuilt` | `repair` | A missing/corrupt vendored artifact was restored from its exact server download (v5.0: never a lost ledger entry — see `vendor_ledger_missing`). `summary.rebuilt` counts these (the field is omitted while zero). | ### Stable `errorCode` tags @@ -1234,7 +1129,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `already_patched` | `skipped` | apply: every file's hash already matches `afterHash`. | | `package_not_installed` | `skipped` | apply: manifest entry has no matching installed package. | | `apply_failed` | `failed` | apply: hash mismatch, write error, archive read error. | -| `no_local_source` | `skipped`/`failed` | `--offline` and the patch is missing from `.socket/`. **Vendored staging (v5.0) reports it at TWO levels:** per package (a `failed` event whose `error` names the reason — which file the patch view served with no `blobContent`, a malformed blob, or the fetch error — envelope `partialFailure`, `error: null`) when at least one other patch staged; and run-level (top-level `error.code`, `status: "error"`, empty `events[]`) when NOTHING in the manifest can be staged, which includes a one-patch manifest. A consumer routing on this code must handle both. A file the patch does not change (`beforeHash == afterHash`) is never a reason: the view serves it without content because it needs none. | +| `no_local_source` | `skipped`/`failed` | Agent patch application cannot obtain the required local or downloaded patch source. Vendored mode consumes complete server artifacts and no longer stages patch blobs. | | `offline_missing_sources` / `sources_download_failed` | apply run-level `warnings[]` | apply (additive): the patch sources were unavailable — `--offline` with no local source, or the download left a patch with no source — so nothing was attempted. The envelope keeps its pinned shape (`partialFailure`, empty `events[]`, zero summary, no top-level `error`); the warning is its machine-readable reason (the human path prints the staging `Error:` line on stderr instead, even under `--silent`). | | `paid_required` | `failed` / status=`paidRequired` | get/scan: patch needs a paid plan and the caller's token isn't entitled. `get ` on the public proxy reports it (exit 0) both for a `tier: "paid"` view and for the proxy's 403 refusal, whose record then carries only `uuid` + `tier` (the proxy never named the purl). | | `download_failed` | `failed` | repair/get: network or 404 on patch fetch. | @@ -1277,7 +1172,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `vendor_multiple_lockfiles` / `pypi_multiple_lockfiles` | `skipped` (warning) | vendor: a sibling lockfile of another package manager will still install UNPATCHED bytes; names the wired winner + the ignored locks. | | `vendor_yarn_berry_unsupported` | `failed` | vendor (npm): yarn-berry Plug'n'Play layout; use its native `yarn patch` workflow. | | `vendor_bun_lockb_invalid` | `failed` | vendor / scan / get `--mode vendored`: the binary lock is malformed, unreadable, unsupported or cannot be rewritten safely. The detail names the parser, hash or filesystem error. Refused before patch downloads and before hosted takeover; `patches[]` / `download.patches[]` carry `errorCode` and `error`, while `get ` also carries top-level `error.code`. Dry-run predicts the same refusal. | -| `vendor_bun_workspace_unsupported` | `failed` | vendor / scan / get `--mode vendored` (bun): the text lock holds `workspace:` packages and its `lockfileVersion` is below 2 — Bun 1.2–1.3 resolve a workspace member's local-tarball path relative to the member; a committed version-2 lock is the proof every consumer runs Bun ≥ 1.4 (deliberate over-approximation: root-only declared packages would install on version 1 too). Detail names the version integer and a version-specific remedy: delete `bun.lock` and re-lock with Bun ≥ 1.4 (an in-place `bun install` keeps the existing version) — then, for a version-1 lock, "or use `--mode hosted`, which accepts version-1 workspace locks"; for a version-0 lock, "or delete `bun.lock`, re-lock with Bun ≥ 1.2 (which writes lockfileVersion 1) and use `--mode hosted`" (hosted refuses version-0 workspace locks, so a bare hosted pointer would send the user into a second refusal). Refused before any write — in the pre-download preflight on `get`/`scan` (see `vendor_bun_lockb_invalid` for the placements); in the shared preflight that `vendor` and the vendor step run BEFORE a hosted → vendored takeover's revert (a hosted-redirected purl stays hosted-wired, ledger and lock untouched; `vendor --dry-run` previews the same `failed` code); and in the engine when the run would write a NEW local tuple. Exempt: purls the vendor ledger wires at the selected uuid, purls whose every `bun.lock` instance is already a `.socket/vendor/npm/` tuple (any uuid), in-sync re-runs and `repair` rebuilds. | +| `vendor_bun_workspace_unsupported` | `failed` | vendor / scan / get `--mode vendored` (bun): the text lock holds `workspace:` packages and its `lockfileVersion` is below 2 — Bun 1.2–1.3 resolve a workspace member's local-tarball path relative to the member; a committed version-2 lock is the proof every consumer runs Bun ≥ 1.4 (deliberate over-approximation: root-only declared packages would install on version 1 too). Detail names the version integer and a version-specific remedy: delete `bun.lock` and re-lock with Bun ≥ 1.4 (an in-place `bun install` keeps the existing version) — then, for a version-1 lock, "or use `--mode hosted`, which accepts version-1 workspace locks"; for a version-0 lock, "or delete `bun.lock`, re-lock with Bun ≥ 1.2 (which writes lockfileVersion 1) and use `--mode hosted`" (hosted refuses version-0 workspace locks, so a bare hosted pointer would send the user into a second refusal). Refused before any write — in the pre-download preflight on `get`/`scan` (see `vendor_bun_lockb_invalid` for the placements); in the shared preflight that `vendor` and the vendor step run BEFORE a hosted → vendored takeover's revert (a hosted-redirected purl stays hosted-wired, ledger and lock untouched; `vendor --dry-run` previews the same `failed` code); and in the engine when the run would write a NEW local tuple. Exempt: purls the vendor ledger wires at the selected uuid, purls whose every `bun.lock` instance is already a `.socket/vendor/npm/` tuple (any uuid), in-sync re-runs and `repair` redownloads. | | `vendor_lockfile_missing` / `vendor_lockfile_version_unsupported` (bun preflight placement) | `failed` | scan / get `--mode vendored` (bun): the pre-download preflight found `bun.lock` unreadable / at a `lockfileVersion` other than 0, 1 or 2 (a newer version: update socket-patch; no integer: re-lock with Bun ≥ 1.2 — the same text as hosted's `redirect_bun_lock_unsupported`) or outside bun's single-line `packages` grammar. Same placements as `vendor_bun_lockb_invalid`; nothing fetched, no patch record. An unreadable `.socket/vendor/state.json` met by the same preflight is `vendor_state_unreadable` (see that row), never one of these. | | `bun_lockb_invalid` | scan `warnings[]` (run-level) | scan (every mode): the native binary inventory could not parse or read `bun.lockb`; detail names the format or filesystem error. Also printed as `Warning: …` on stderr. Exit and status remain unchanged. The warning is retained on empty and non-empty scans; valid binary locks are inventoried normally without a runtime or install. | | `would_refuse` | dry-run preview action (`vendor.patches[]`) | scan `--mode vendored --dry-run` / get `--mode vendored --dry-run`: the wet run's Bun preflight would refuse this npm purl; the record carries `errorCode` (one of the four Bun lock codes above, or `vendor_state_unreadable` for an unreadable vendor ledger) + `error`. Exit 0 / `status: "success"`, nothing written. | @@ -1286,7 +1181,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `cargo_version_tagged` | `skipped` (advisory note) | vendor / scan / get `--mode vendored` / repair (v5.0): a vendored copy and its detached Cargo.lock entry were (re)tagged `+socket.` — a copy vendored before tagged versions, or a lock entry tagged for another uuid while the wiring points at this copy (dry run: "would tag"). A vendor re-run that tags reports the package `applied`. | | `cargo_version_untagged` | `skipped` (warning) | repair (v5.0): the tag could not be written (an unreadable copy manifest, or a lock the retag cannot keep consistent); nothing else was undone — re-run `socket-patch vendor`. | | `cargo_lock_untaggable` | `failed` | vendor / scan / get `--mode vendored` (cargo, v5.0): the Cargo.lock entry cannot carry the copy's tagged version consistently (a dependency reference in a spelling the edit does not own, a v1 `replace` naming the crate, or an entry already at the tagged version). Refused before any write; a dry run previews the same refusal. | -| `cargo_copy_untaggable` | `failed` (error prefix) | vendor / scan / get `--mode vendored` (cargo, v5.0): the copy's `Cargo.toml` has no literal `[package] version` string that can be rewritten byte-exactly (or it names another version); nothing is swapped in. A dry run over an already-vendored copy reports the same failure; a patch-service crate that cannot be tagged is a miss (`vendor_prebuilt_layout_mismatch`: `auto` builds locally, `service` fails `vendor_prebuilt_required`). | +| `cargo_copy_untaggable` | `failed` (error prefix) | vendor / scan / get `--mode vendored` (cargo, v5.0): the copy's `Cargo.toml` has no literal `[package] version` string that can be rewritten byte-exactly (or it names another version); nothing is swapped in. A dry run over an already-vendored copy reports the same failure; a patch-service crate that cannot be tagged fails with `vendor_prebuilt_required`. | | `cargo_wiring_restored` | `skipped` (advisory note) | repair (v5.0): a vendored crate's Cargo.lock entry was detached with no Socket-owned `[patch]` pointing at its committed copy (a pre-v5 release overwrote its crate-named config key when a second version was vendored); the manifest entry is written back and the ledger updated (dry run: "would restore"). A `vendor` re-run heals the same state as a plain re-vendor. | | `cargo_manifest_unreadable` / `cargo_manifest_unparseable` / `cargo_manifest_symlink_unsupported` / `cargo_manifest_not_workspace_root` / `cargo_manifest_patch_source_alias` | `failed` | vendor / scan / get `--mode vendored` (cargo, v5.0): the workspace-root `Cargo.toml` cannot carry the vendored `[patch.crates-io]` entry (or cargo would ignore it there) — see the cargo caveat under "Vendored mode". Refused before any write. | | `vendor_would_revert_redirect` / `vendor_takeover_reverted_redirect` | `skipped` (advisory event) | vendor / scan / get `--mode vendored` over a hosted pin (every ecosystem, v5.0): dry run — the upstream restore was resolved (registry lookups included) and would succeed (for bun, only after the Bun vendored preflight accepted the lock; a refused lock is previewed as the wet run's `failed ` instead) / wet run — the pin's lock entries were restored to their upstream registry entry before vendoring (mode takeover; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`), so `vendor --revert` later returns to upstream. Fires on the run that takes over, not on re-runs. | @@ -1303,28 +1198,24 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `vendor_override_conflict` | `failed` | vendor (pnpm/yarn-berry): a user-authored override/resolution for the package already exists. | | `vendor_integrity_unverified` | `skipped` (warning) | vendor (pipenv): the lockfile format does not hash-check file entries; the committed wheel bytes are the protection. | | `vendor_content_mismatch_overwritten` | `skipped` (warning) | vendor: a staged file matched NEITHER beforeHash nor afterHash (patch built against different bytes, or local edits); the stage was overwritten with the verified patched content and the vendor succeeded. | -| `vendor_fetched_missing` | `skipped` (warning) | vendor: the package was not installed; its pristine artifact was fetched per the lockfile resolution (or staged from the committed vendor artifact), integrity-verified, and vendored — the project tree was not touched. Not emitted when no fetch happened: an in-sync re-run of a ledger-covered purl, or an npm, cargo, golang or composer package the patch service served (see Vendor auto-fetch § Deferred fetch). For `poetry.lock` (which records hashes but no URLs) the pure-Python wheel's sha256 selects the file through PyPI's JSON API (`SOCKET_PYPI_JSON_API` overrides the endpoint); Poetry 0.12's bare `[metadata.hashes]` names no wheel, so those locks still need an installed copy (`vendor_fetch_unverifiable`). | -| `vendor_fetch_failed` | `failed` | vendor: the lockfile-resolved fetch was attempted and failed (HTTP error, size cap, integrity mismatch, or a PRESENT-but-corrupt committed artifact — pointed at `socket-patch repair`). A MISSING committed artifact no longer lands here: it falls through to the ledger-recovered registry fetch. Suppresses the duplicate `package_not_installed` skip. | -| `vendor_fetch_unverifiable` | `skipped` (warning) | vendor: the lockfile records no usable integrity for the missing package; nothing was fetched (fail-closed) and the `package_not_installed` skip follows. Unchanged for gems by the build-mode `gem_spec_missing` refusal below, which fires only where a fetch WOULD have run. | | `vendor_vlt_transitive_unsupported` | `failed` | vendor (vlt): the target has an inbound edge from another package in `vlt-lock.json` (the detail names it); vendored mode rewires only direct dependencies of the root or a workspace member, because vlt silently reverts transitive lock surgery. Remedy: `--mode hosted`. Refused before any download or write, dry runs included (`would_refuse`). | | `vendor_vlt_lock_out_of_sync` | `failed` | vendor (vlt): an importer's `package.json` is missing, unparseable, or declares a spec for the dependency that differs from the lock's importer edge. Remedy: `vlt install` first. Refused before any write. | | `vendor_vlt_build_scripts_unsupported` | `failed` | vendor (vlt): the package declares a `preinstall`, `install`, `postinstall` or `prepare` script, or ships a `binding.gyp`. vlt builds a registry copy in the untracked store, but a vendored `file:` dependency in place, so `vlt build` would rewrite the committed artifact (a platform binary over a JS shim, say) and every later vendor, repair and `vex` would treat it as tampered. Remedy: `--mode hosted`. Refused before any write. | | `vendor_vlt_legacy_lockfile` | `skipped` (warning) | vendor (vlt): an era-A lock (vlt 0.0.0-19 … 1.0.0-rc.8): a `··` default-registry id, or default-registry ids that are URL segments equal to a scalar `options.registry` with no `·npm·` id (era B writes `·npm·` whatever the scalar). vlt 0.0.0-31 … 1.0.0-rc.5 install the vendored lock but fail to reinstall the vendored `file:` dependency if `vlt-lock.json` is deleted and re-created (the other era-A releases reinstall it; the lock does not say which release reads it). The package is still vendored; remedy: upgrade vlt. | -| `vendor_vlt_reinstall_required` | `skipped` (advisory; human: `Warning: …`) | vendor / scan / get `--mode vendored` (vlt), wet and dry runs, and in-sync reruns: (a) the run rewires an optional dependency, or an importer's `node_modules/` of an optional dependency still resolves into `node_modules/.vlt/`: from vlt 0.0.0-30 a plain `vlt install` (1.2.0: also `--force`) keeps that installed upstream copy linked; the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the vendored copy, and that vlt 0.0.0-30 … 1.0.4 install no optional dependency from the lock of a project that declares only optional dependencies (upgrade to 1.0.5 or later first); (b) otherwise, an importer's link of the dependency still resolves into `node_modules/.vlt/`: the detail names the links (`node_modules/`, `/node_modules/`) and says `vlt install` (or `vlt ci`) links the vendored copy — on a warm tree after a plain `vlt install` that is true of every vendored direct dependency; (c) an importer's link resolves into the vendored dir of the patch this run replaces (a new patch uuid), which the run removes: the detail names the links and says `vlt install` (or `vlt ci`) links the new vendored copy; (d) a rebuild of the payload (vendor, or `repair` after a corrupt or missing payload) could not keep vlt's links to the package's own dependencies (its old `node_modules/` held more than links): the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`), since a plain `vlt install` does not re-link them. `repair` moves those links back into the rebuilt payload when they are only links. The package is vendored either way; a run whose patch fails to apply emits neither. A wet `vendor --revert` (and the revert a vendored → hosted takeover runs, whose advisory joins `redirect.warnings[]`): (a) the revert moves an `optionalDependencies` spec back from the `file:` dir, or an optional importer's `node_modules/` still resolves into the vendored uuid dir: from vlt 0.0.0-30 a plain `vlt install` keeps that link (dangling once the dir is removed), so the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the restored copy, with the same vlt 1.0.5 note; (b) otherwise, an importer's link still resolves into the vendored uuid dir: the detail names the links and says `vlt install` (or `vlt ci`) links the restored copy. A dry-run revert emits neither. | +| `vendor_vlt_reinstall_required` | `skipped` (advisory; human: `Warning: …`) | vendor / scan / get `--mode vendored` (vlt), wet and dry runs, and in-sync reruns: (a) the run rewires an optional dependency, or an importer's `node_modules/` of an optional dependency still resolves into `node_modules/.vlt/`: from vlt 0.0.0-30 a plain `vlt install` (1.2.0: also `--force`) keeps that installed upstream copy linked; the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the vendored copy, and that vlt 0.0.0-30 … 1.0.4 install no optional dependency from the lock of a project that declares only optional dependencies (upgrade to 1.0.5 or later first); (b) otherwise, an importer's link of the dependency still resolves into `node_modules/.vlt/`: the detail names the links (`node_modules/`, `/node_modules/`) and says `vlt install` (or `vlt ci`) links the vendored copy — on a warm tree after a plain `vlt install` that is true of every vendored direct dependency; (c) an importer's link resolves into the vendored dir of the patch this run replaces (a new patch uuid), which the run removes: the detail names the links and says `vlt install` (or `vlt ci`) links the new vendored copy; (d) a redownload of the payload (vendor, or `repair` after a corrupt or missing payload) could not keep vlt's links to the package's own dependencies (its old `node_modules/` held more than links): the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`), since a plain `vlt install` does not re-link them. `repair` moves those links back into the downloaded payload when they are only links. The package is vendored either way; a run whose patch fails to apply emits neither. A wet `vendor --revert` (and the revert a vendored → hosted takeover runs, whose advisory joins `redirect.warnings[]`): (a) the revert moves an `optionalDependencies` spec back from the `file:` dir, or an optional importer's `node_modules/` still resolves into the vendored uuid dir: from vlt 0.0.0-30 a plain `vlt install` keeps that link (dangling once the dir is removed), so the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the restored copy, with the same vlt 1.0.5 note; (b) otherwise, an importer's link still resolves into the vendored uuid dir: the detail names the links and says `vlt install` (or `vlt ci`) links the restored copy. A dry-run revert emits neither. | | `vendor_flavor_changed` | `failed` | vendor (npm): the purl's vendor ledger entry was written for another lockfile `flavor` than the one the router now detects (for example `npm` → `vlt` after switching package managers). Remedy: `socket-patch vendor --revert` it first, then re-vendor. Refused before any write. | | `vendor_artifact_gitignored` | `failed` | vendor (vlt): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. | | `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. | | `vendor_ledger_entry_missing` | `failed` | vendor (vlt): the only installed copy is vlt's link to a committed vendored directory, but the vendor ledger has no entry for the package; restore `.socket/vendor/state.json` from version control (v5.0: `repair` no longer re-synthesizes it). Replaces the `package_not_installed` skip. | -| `vendor_artifact_missing` | `skipped` (warning) / `failed` | vendor: the committed artifact is gone — the registry resolution is recovered from the ledger and the artifact rebuilt (warning); repair `--offline` with no local source surfaces it as the per-entry failure instead. | -| `vendor_artifact_corrupt` | `failed` | repair `--offline`: the committed artifact fails verification (member afterHashes or the ledger's whole-file sha256) and no local source can rebuild it. Online repairs rebuild instead. | +| `vendor_artifact_missing` | reason | The recorded artifact is missing; repair requires an online exact redownload. | +| `vendor_artifact_corrupt` | reason | The artifact does not match the recorded hash or inventory; repair requires an online exact redownload. | | `vendor_artifact_reused` | `skipped` (verbose note) | vendor / scan `--vendor` (pypi): the wiring was dropped by a relock but the committed wheel the ledger vouches for verified, so it was re-wired as-is — no service download, no rebuild; the lock pins the first run's sha again. | -| `vendor_artifact_rebuilt` | `skipped` (warning) | vendor / scan `--vendor`: a wired-but-missing/stale artifact was rebuilt in place. The lockfiles are untouched, except that nuget re-pins `packages.lock.json` to the rebuilt bytes. gem/maven/nuget: the package's event is `applied` (also for a rebuild from the patch service), and the ledger entry's artifact fingerprint (gem `fileInventory`, maven/nuget `sha256` + `size`, and the nuget lock pin) is refreshed to the rebuilt bytes, and its wiring records are kept unchanged, so `--revert` still restores the pre-vendor files. A rebuild whose ledger has no entry for the package, or only one from another patch uuid, records none. cargo/composer/gem rebuilds honour `--vendor-source` like a fresh vendor (`service` downloads the prebuilt artifact and refuses when it cannot). Other ecosystems leave the ledger entry untouched. (Under `repair` the `rebuilt` event carries this signal.) | -| `vendor_artifact_rebuild_failed` | `failed` | repair: the rebuild ran but the result failed verification against the recorded fingerprint (e.g. an edited state.json sha); the unverifiable artifact was removed. | -| `vendor_artifact_unrepairable` | `failed` | repair: no verifiable pristine source exists (not installed + lockfile rewired + no recoverable ledger fragment), the wheel is platform-locked with no installed copy, or the ledger entry itself cannot be trusted. | +| `vendor_redownload_failed` | `failed` | vendor: the same-UUID artifact could not be downloaded and verified against its original ledger. Existing files and fingerprints are preserved. | +| `vendor_artifact_redownload_failed` | `failed` | repair: download unavailable, integrity mismatch, or downloaded bytes/inventory differ from the ledger. Existing files are preserved. | +| `vendor_artifact_unrepairable` | `failed` | repair: the ledger identity or patch record cannot be trusted or recovered. | | `vendor_uuid_mismatch` | `skipped` | repair: the manifest's patch uuid moved past the vendored artifact — a re-vendor (`vendor` / `scan --vendor`) is pending; repair does not cross patch generations. | | `content_mismatch_overwritten` | `skipped` (warning) | apply (default policy): a file matched NEITHER beforeHash nor afterHash and was overwritten with the full verified patched content. `--strict` turns this case into a `failed` event instead. | | `vendor_lock_checksums_unsupported` / `vendor_stale_lock_checksum` | `failed` | vendor (gem): an ambiguous/platform CHECKSUMS entry, or a v1-wired lock whose stale token blocks the hot path (run `vendor --revert` + re-vendor). | -| `gem_spec_missing` | `failed` | vendor (gem): the gem is not installed and the run cannot use the patch service (`--vendor-source build`, or no service config), so the local build has no eval-able stub gemspec to give bundler's path source — a downloaded `.gem` carries its gemspec only as YAML in `metadata.gz`. Raised BEFORE the registry round trip (no `vendor_fetched_missing` precedes it) when the lock both resolves AND verifies the gem and no ledger entry already vendors it; the gem backend raises the same refusal as the backstop for every other route into it. A run that would not have fetched at all is unaffected: an unverifiable lock entry keeps `vendor_fetch_unverifiable` + `package_not_installed`, an already-vendored gem re-runs green, and `--dry-run` still fetches and previews. See Vendor auto-fetch § Gem, local build only. | | `redirect_pypi_stale_install` | `redirect.warnings[]` (warning) | Hosted Python redirect: readable installed files differ from patched hashes. Read-only, repeated on re-scan, and excludes the package from same-run VEX. See the "Python stale-install guard" section. | | `redirect_gem_stale_install` | `redirect.warnings[]` (warning) | scan `--mode hosted` (gem): a stale UNPATCHED materialization (installed gem, or committed `vendor/cache` archive) that `bundle install` will reuse instead of fetching the redirected patch; the detail carries the verified remedy. Full rules and flavors: the "Gem stale-install guard" section. | | `redirect_pipenv_refused` | `redirect.warnings[]` (warning) | scan `--mode hosted` (pipenv): the Pipfile.lock pins another version or a non-registry / foreign source for the package — refused atomically across categories, and the patch is vetoed from the sibling Python rewriters (see the "Pipenv hosted redirect" section). | @@ -1354,8 +1245,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `redirect_vlt_no_lockfile` | `redirect.warnings[]` (warning) | scan/get `--mode hosted` (vlt): `vlt.json` or vlt's install state is present without `vlt-lock.json`; replaces `redirect_npm_no_lockfile` for vlt projects. | | `redirect_vlt_artifact_unverifiable` | `redirect.warnings[]` (warning), `redirect.skipped[].reason` | scan/get `--mode hosted` (vlt): before any takeover or rewrite (dry runs included), each granted artifact with a default-registry instance in `vlt-lock.json` (or, for a purl a `flavor: "vlt"` vendored entry claims, its vendored node, probed before the takeover reverts it) is fetched once as vlt fetches it (`accept-encoding: gzip;q=1.0, identity;q=0.5`, no `Authorization`, up to 10 redirects) and must return 200 with no content encoding (or `identity`) and the granted sha512. On failure (`content-encoding `, `sha512 mismatch`, `http `, `fetch error `, `offline`) the dep is withheld from every rewriter when vlt drives or it is vlt-vendored (which also keeps it vendored), and from the vlt rewrite only otherwise (detail "…; vlt-lock.json was not changed for {purl}"; only the sibling lock this run rewrote can confirm it). A lock already pinned by an earlier run is left pinned, and neither confirmed nor attested. Projects without `vlt-lock.json` make no such request. The detail quotes the artifact URL (and any fetch error that echoes it) with its grant-token path level, the one just before the patch uuid, spelled ``; host, uuid and leaf stay. The in-memory hosted engine (`hosted-bundle`, the Node addon) has no network for this fetch, so it judges every in-scope artifact as `--offline` does (withheld, never pinned; the vendored takeover it refuses anyway). Exit 0. | | `redirect_vlt_reinstall_required` | `redirect.warnings[]` (advisory); rollback/remove `warnings[]` (+ human stderr) | vlt: `vlt-lock.json` pins (or, after rollback/remove, no longer pins) Socket-patched packages, and vlt never refreshes an installed copy. The heal removes `node_modules/.vlt-lock.json` and each stale `node_modules/.vlt/` of a Socket-owned node (never a link's target, never outside the project, never a copy it cannot judge) unless `--no-vlt-install-cleanup` or `--dry-run`. It never removes an optional node's copy (lock flags 1 or 3, or flags it cannot read): `vlt install` does not put a removed optional dependency back (its link dangles) unless the same install also reinstalls a non-optional node, so such a copy is left stale and the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`); vlt 0.0.0-30 … 1.0.4 install no optional dependency from the lock of a project that declares only optional dependencies, so there both commands remove the installed copy and the detail says to upgrade vlt to 1.0.5 or later first. The detail says whether copies were removed, left stale by a skipped cleanup, could not be checked, or none were stale, and adds how many optional copies were kept whenever there are any. The kept optional copies are named by what they are: `unpatched copies of optional dependencies` after `scan`/`get`, `patched copies of optional dependencies` after `rollback`/`remove`, and `installed copies of the vendored optional dependencies` after a hosted → vendored takeover (the copy the hosted pin left installed, which may still be the registry bytes). Stale or unchecked copies are not attested by the run's `--vex`, nor is a confirmed vlt pin the heal did not check (a URL on a host other than patch.socket.dev and the configured `--patch-server-url`/`--api-url`). A hidden lock that cannot be removed keeps every store entry. Invalidation failures only warn. | -| `vendor_prebuilt_stub_invalid` | `failed` / `skipped` (warning) | vendor (gem, `--vendor-source`): the served stub gemspec fails the rubygems `summary`/`authors` bar, so bundler would refuse the vendored path source at install time. `service`: refusal naming the missing attributes; `auto`: loud warning + local-build fallback — or, when the gem is also not installed locally (no stub to derive), a refusal naming the served defect and the install-the-gem remedy. | -| `gem_spec_invalid` | `failed` | vendor (gem): the LOCAL `specifications/` stub gemspec fails the same rubygems `summary`/`authors` bar (a corrupted or hand-edited gem home); the refusal names the file — reinstall the gem (`gem pristine ` / fresh `bundle install`). | +| `vendor_prebuilt_stub_invalid` | `failed` | RubyGems: the server stub lacks required attributes or is otherwise invalid; no local stub fallback is permitted. | | `vendor_*` / `pypi_*` / `gemfile_*` / `lock_*` / `locked_version_mismatch` / `user_authored_*` / `native_extensions_unsupported` / `platform_gem_unsupported` | `failed`/`skipped` | vendor: per-ecosystem refusal + drift vocabulary; see the Vendor command contract section. New tags are additive (MINOR). | ### Top-level `EnvelopeError` codes diff --git a/crates/socket-patch-cli/Cargo.toml b/crates/socket-patch-cli/Cargo.toml index ab434d5e0..2c3e35cc8 100644 --- a/crates/socket-patch-cli/Cargo.toml +++ b/crates/socket-patch-cli/Cargo.toml @@ -57,7 +57,7 @@ docker-e2e = [] # vendor_crash_safety_e2e / vendor_group_commit_e2e crash the binary through # its failpoints; with this, `cargo test --release` (the test-release job) # builds them in too. Dev-only: resolver 2 keeps it out of normal builds. -socket-patch-core = { workspace = true, features = ["failpoints"] } +socket-patch-core = { workspace = true, features = ["failpoints", "test-fixtures"] } sha2 = { workspace = true } # docker_e2e_vendor_maven's host oracle recomputes the maven2 .jar.sha1 sidecar. sha1 = { workspace = true } diff --git a/crates/socket-patch-cli/src/args.rs b/crates/socket-patch-cli/src/args.rs index 13731f268..a2ecfe542 100644 --- a/crates/socket-patch-cli/src/args.rs +++ b/crates/socket-patch-cli/src/args.rs @@ -161,17 +161,15 @@ pub struct GlobalArgs { )] pub download_mode: String, - /// Where `vendor` acquires the installable patched artifact. `auto` - /// (default) downloads the prebuilt archive from the patch.socket.dev - /// vendoring service and silently falls back to a local build on any miss; - /// `service` requires the service and fails closed; `build` always builds - /// locally. Only `vendor` and the vendored modes of `scan`/`get` use - /// this; other subcommands accept it silently. + /// Download installable patched artifacts from the patch service. + /// `service` is the default; `auto` is a compatibility alias. Local + /// artifact building is no longer supported. Healthy committed artifacts + /// can be reused offline; missing or corrupt artifacts require a download. #[arg( help_heading = GLOBAL_OPTIONS, long = "vendor-source", env = "SOCKET_VENDOR_SOURCE", - default_value = "auto", + default_value = "service", value_parser = parse_vendor_source, )] pub vendor_source: String, @@ -405,9 +403,9 @@ impl GlobalArgs { /// empty). The names are validated at parse time, so this is an exact /// match. pub(crate) fn ecosystem_selected(&self, eco: Ecosystem) -> bool { - self.ecosystems.as_ref().is_none_or(|list| { - list.is_empty() || list.iter().any(|name| name == eco.cli_name()) - }) + self.ecosystems + .as_ref() + .is_none_or(|list| list.is_empty() || list.iter().any(|name| name == eco.cli_name())) } /// [`Self::ecosystem_selected`] for the ecosystem of `purl`; a purl of @@ -680,7 +678,7 @@ impl Default for GlobalArgs { proxy_url: None, ecosystems: None, download_mode: "diff".to_string(), - vendor_source: "auto".to_string(), + vendor_source: "service".to_string(), maven_config: None, vendor_url: None, patch_server_url: None, @@ -937,7 +935,7 @@ mod tests { assert!(cli.common.ecosystems.is_none()); assert_eq!(cli.common.download_mode, "diff"); assert_eq!( - cli.common.vendor_source, "auto", + cli.common.vendor_source, "service", "empty SOCKET_VENDOR_SOURCE must fall back to the `auto` default" ); assert_eq!(cli.common.manifest_path, "keep.json"); @@ -952,7 +950,7 @@ mod tests { with_clean_socket_env(|| { // Default when unset. let cli = TestCli::try_parse_from(["socket-patch"]).unwrap(); - assert_eq!(cli.common.vendor_source, "auto"); + assert_eq!(cli.common.vendor_source, "service"); // CLI value, case-normalized to the canonical tag. let cli = @@ -961,8 +959,7 @@ mod tests { // Env var honored. std::env::set_var("SOCKET_VENDOR_SOURCE", "build"); - let cli = TestCli::try_parse_from(["socket-patch"]).unwrap(); - assert_eq!(cli.common.vendor_source, "build"); + assert!(TestCli::try_parse_from(["socket-patch"]).is_err()); std::env::remove_var("SOCKET_VENDOR_SOURCE"); // Garbage is rejected at parse time. @@ -985,27 +982,20 @@ mod tests { } } - /// Regression: scan's vendored flow must build its service config FROM - /// `--vendor-source`, not hardcode build-only. Under the default (`auto`), the config must permit the - /// vendoring service exactly as the `vendor` command's default does — - /// otherwise `scan --mode vendored` silently builds locally while a - /// plain `vendor` service-downloads, and the two commit different bytes / - /// lock integrity for the same patch (lock churn / merge conflicts). #[test] fn vendor_service_config_default_source_permits_service() { let cfg = common_with_source("auto").vendor_service_config(None, false); - assert_eq!(cfg.source, VendorSource::Auto); + assert_eq!(cfg.source, VendorSource::Service); assert!( cfg.source.may_use_service(), "the default must be able to use the service (matching `vendor`)" ); - assert!(!cfg.source.requires_service()); + assert!(cfg.source.requires_service()); assert!(cfg.client.is_none()); assert!(!cfg.use_public_proxy); } - /// `--vendor-source service` reaches the fail-closed service path and - /// `--vendor-source build` never contacts the service. + /// The assembler fails closed even if a caller bypasses argument validation. #[test] fn vendor_service_config_honors_service_and_build_sources() { let cfg = common_with_source("service").vendor_service_config(None, true); @@ -1017,8 +1007,8 @@ mod tests { ); let cfg = common_with_source("build").vendor_service_config(None, false); - assert_eq!(cfg.source, VendorSource::Build); - assert!(!cfg.source.may_use_service()); + assert_eq!(cfg.source, VendorSource::Service); + assert!(cfg.source.requires_service()); } /// The service overrides (`--vendor-url` / `--patch-server-url` / diff --git a/crates/socket-patch-cli/src/commands/fetch_stage.rs b/crates/socket-patch-cli/src/commands/fetch_stage.rs index dbde61fcd..85831974a 100644 --- a/crates/socket-patch-cli/src/commands/fetch_stage.rs +++ b/crates/socket-patch-cli/src/commands/fetch_stage.rs @@ -7,25 +7,19 @@ //! cache is `repair`'s job, keeping these commands read-only against //! `.socket/`). -use std::borrow::Cow; use std::collections::{HashMap, HashSet}; use std::path::{Path, PathBuf}; -use futures_util::StreamExt; use socket_patch_core::api::blob_fetcher::{ fetch_missing_blobs, fetch_missing_sources, get_missing_archives, get_missing_blobs, DownloadMode, FetchMissingBlobsResult, }; -use socket_patch_core::api::client::{get_api_client_with_overrides, hold_back_debug, ApiClient}; -use socket_patch_core::manifest::schema::{PatchFileInfo, PatchManifest, PatchRecord}; -use socket_patch_core::patch::apply::{is_valid_blob_hash, PatchSources}; -use socket_patch_core::utils::concurrent::{api_concurrency_for, ordered_concurrent}; +use socket_patch_core::api::client::ApiClient; +use socket_patch_core::manifest::schema::PatchManifest; +use socket_patch_core::patch::apply::PatchSources; use tempfile::TempDir; -use super::get::base64_decode; use crate::args::GlobalArgs; -use crate::commands::bun_preflight::LedgerLoad; -use crate::json_envelope::{Envelope, PatchAction, PatchEvent}; use crate::ui::{plural, StatusLine}; /// Resolved artifact locations for the patch pipeline. Holds the overlay @@ -80,14 +74,6 @@ pub(crate) enum StageOutcome { /// cache `apply` reads from. const APPLY_OFFLINE_REMEDY: &str = "Run `socket-patch repair` to download missing artifacts."; -/// The memory stager's remedy. Vendored content is fetched into memory and -/// never lands under `.socket/`; sending a vendored project to `repair` -/// instead would populate `.socket/blobs/` — exactly the residue vendored -/// mode promises not to leave (and from inside `repair --offline` the hint -/// was self-referential). -const VENDOR_OFFLINE_REMEDY: &str = "Re-run without --offline to fetch the missing patch \ - content (kept in memory; nothing is written under .socket/)."; - /// Shared offline diagnostic: patches with no usable local source while /// `--offline` is set (first five PURLs, then the caller's `remedy` line). /// Prints even under `--silent` (errors only, NEVER nothing — an exit-1 @@ -176,11 +162,6 @@ fn format_fetch_failures(result: &FetchMissingBlobsResult, (one, many): Noun) -> /// The disk stager's status line while it downloads what `.socket/` lacks. const DOWNLOADING_ARTIFACTS: &str = "Downloading missing patch artifacts..."; -/// The in-memory stager's status line while it fetches patch views. -fn format_fetching_content(n: usize) -> String { - format!("Fetching content for {}...", plural(n, "patch", "patches")) -} - /// Announce the per-file blob top-up that follows a diff-mode fetch. It /// runs even when every diff archive arrived — a diff cannot patch a file /// whose bytes differ from `beforeHash`, and the pipeline then falls back @@ -308,8 +289,7 @@ pub(crate) async fn stage_patch_sources( let missing_blobs = get_missing_blobs(manifest, &socket_blobs_path).await; let missing_diff_archives = get_missing_archives(manifest, &socket_diffs_path).await; - let no_source_purls = - patches_without_source(manifest, &missing_blobs, &missing_diff_archives); + let no_source_purls = patches_without_source(manifest, &missing_blobs, &missing_diff_archives); if common.offline { // Offline: bail only if some patch has no usable local source. @@ -455,402 +435,11 @@ pub(crate) async fn stage_patch_sources( /// `.socket/blobs` entries and no temporary files. The committed /// `.socket/vendor/` artifact is the patch; nothing else should land on /// disk. -pub(crate) struct MemStagedSources { - blobs: PathBuf, - diffs: PathBuf, - mem: HashMap>, - /// The purls this staging could NOT obtain patch content for, each with - /// the reason, while at least one other patch staged fine. Each is an - /// unsatisfiable package the caller reports per-package (and leaves out - /// of the engine run) — see [`stage_vendor_sources_in_memory`]. Sorted - /// by purl, so the per-package reports come out in the same order every - /// run. - unavailable: Vec<(String, String)>, -} - -impl MemStagedSources { - /// Borrow as the core pipeline's source set (memory overlay first, - /// on-disk artifacts as the read-only fallback). - pub(crate) fn as_patch_sources(&self) -> PatchSources<'_> { - PatchSources { - blobs_path: &self.blobs, - diffs_path: Some(&self.diffs), - mem_blobs: Some(&self.mem), - } - } - - /// See [`MemStagedSources::unavailable`]. - pub(crate) fn unavailable(&self) -> &[(String, String)] { - &self.unavailable - } -} - -/// The in-memory staging outcome (mirror of [`StageOutcome`]). -pub(crate) enum MemStageOutcome { - Ready(MemStagedSources), - Unavailable, -} - -/// Does vendoring this file need the patch's after-BLOB? -/// -/// No, when the patch does not change it (`beforeHash == afterHash`): the -/// pristine copy already carries the patched bytes, and the apply pipeline -/// answers `AlreadyPatched` for it without writing anything. The patch view -/// says the same thing by serving such a file with hashes and no -/// `blobContent`, so treating it as a failed fetch made any patch with a -/// zero-delta file permanently unvendorable. -fn needs_blob(file: &PatchFileInfo) -> bool { - file.before_hash != file.after_hash -} - -/// Stage patch sources for a VENDOR run without writing anything: -/// a record is locally satisfied when all its after-blobs are on disk (a -/// diff archive is NOT sufficient — vendor's -/// auto-force policy can need the full after-blob for files a diff cannot -/// reproduce); anything else has its full per-file content fetched into -/// memory from the patch view endpoint (`blobContent`), preceded by the -/// committed-artifact harvest. Offline runs with missing sources are -/// `Unavailable` with the same diagnostics as the disk stager. Unlike the -/// disk stager there is no hard-failure mode (no download-mode parse, no -/// tempdir), so this returns the outcome directly — every failure is the -/// soft `Unavailable`. -/// -/// A patch whose content the VIEW cannot supply (a 404, a transport error, -/// or a file the server serves with no `blobContent` — which is how it -/// serves a zero-delta file, `beforeHash == afterHash`) is an unsatisfiable -/// PACKAGE, not a broken run: its purl comes back in -/// [`MemStagedSources::unavailable`] for the caller to report per-package, -/// and the patches that did stage still run. `Unavailable` is reserved for -/// the case it was written for — NOTHING in the manifest can be staged, so -/// there are no per-package events to report and the caller's pre-event -/// `no_local_source` error is the whole story. -/// -/// `ledger` is the caller's single `load_state` outcome (the harvest reads -/// the committed artifacts it names; an unreadable ledger harvests -/// nothing). `seed` pre-populates the in-memory blob set — the vendored -/// download phase already holds every fetched view's `blobContent`, so a -/// fresh `scan`/`get --mode vendored` never fetches a view a second time -/// here; manifest-driven callers pass an empty map. `client` is the run's -/// one API client (every CLI caller has one — building another here -/// repeated its token advisory and org-slug round-trip, under the apply -/// lock in `vendor`'s case); `None` builds one on demand, only once a fetch -/// is actually needed (the unit tests' offline arms never get that far). -pub(crate) async fn stage_vendor_sources_in_memory( - common: &GlobalArgs, - manifest: &PatchManifest, - socket_dir: &Path, - project_root: &Path, - ledger: LedgerLoad<'_>, - seed: HashMap>, - client: Option<&ApiClient>, -) -> MemStageOutcome { - let blobs = socket_dir.join("blobs"); - let diffs = socket_dir.join("diffs"); - - let missing_blobs = get_missing_blobs(manifest, &blobs).await; - let mut mem = seed; - let mut unavailable: Vec<(String, String)> = Vec::new(); - - // A diff archive alone is NOT a sufficient source here, unlike the disk - // stager: vendoring runs the auto-force policy, where a beforeHash - // mismatch (already-applied tree, patch built against different bytes) - // is overwritten with the FULL after-blob — which a diff cannot - // produce. On-disk diffs still serve Strategy 1 for clean files; the - // after-blob content must additionally exist (disk, seed/harvest, or - // fetch). - // - // …for the files the patch CHANGES. A ZERO-DELTA file - // (`beforeHash == afterHash`) is already at its patched content in the - // pristine copy — `verify_file_patch` answers `AlreadyPatched` as soon - // as the on-disk hash equals `afterHash` — so it needs no blob, which - // is exactly why the view serves it with hashes and no `blobContent`. - // Demanding it made such a patch permanently unvendorable (JS-7: - // `pkg:npm/tar-fs@2.1.1`, seven zero-delta fixture files). This - // predicate is the AUTHORITY the fetch loop below agrees with, so the - // two can never disagree about which files a fetch must bring back. - let covered = |record: &PatchRecord, mem: &HashMap>| { - record.files.values().all(|f| { - !needs_blob(f) - || !missing_blobs.contains(&f.after_hash) - || mem.contains_key(&f.after_hash) - }) - }; - let mut to_fetch: Vec<(&str, &str)> = manifest - .patches - .iter() - .filter(|(_, record)| !covered(record, &mem)) - .map(|(purl, record)| (purl.as_str(), record.uuid.as_str())) - .collect(); - - if !to_fetch.is_empty() { - // The committed vendor artifact IS the patched content: harvest its - // afterHash blobs into memory so in-sync re-runs and fresh clones of - // already-vendored projects stage with no network and no disk blobs. - // Harvested bytes are hash-verified, so they win over a same-hash - // seed entry. - if let Ok(entries) = ledger { - mem.extend( - socket_patch_core::vendor::harvest_artifact_blobs_from( - project_root, - entries, - &manifest.patches, - ) - .await, - ); - } - to_fetch.retain(|(purl, _)| { - manifest - .patches - .get(*purl) - .is_none_or(|record| !covered(record, &mem)) - }); - } - - if !to_fetch.is_empty() { - if common.offline { - let purls: Vec<&str> = to_fetch.iter().map(|(purl, _)| *purl).collect(); - report_offline_missing(common, &purls, VENDOR_OFFLINE_REMEDY); - return MemStageOutcome::Unavailable; - } - - let mut status = StatusLine::stderr(common.json, common.silent); - status.set(format_fetching_content(to_fetch.len())); - - let built; - let client = match client { - Some(client) => client, - None => { - built = get_api_client_with_overrides(common.api_client_overrides()) - .await - .0; - &built - } - }; - // Each dropped purl with WHY it was dropped. The reason is the only - // machine-readable explanation the caller can put in that package's - // `failed` event, and the human `[error]` lines below are printed - // exclusively under `!--json` — so without it a `--json` consumer - // learned nothing about which file was contentless. - let mut failed: Vec<(&str, String)> = Vec::new(); - // The views are fetched concurrently (at most `api_concurrency` in - // flight) but consumed in `to_fetch` order, each request's `--debug` - // lines released at its turn, so `mem`, `failed` and every error - // line fold exactly as the serial loop's did. - let mut views = std::pin::pin!(ordered_concurrent( - to_fetch.iter(), - api_concurrency_for(client.uses_public_proxy(), to_fetch.len()), - |(_, uuid)| async move { (*uuid, hold_back_debug(client.fetch_patch(uuid)).await) }, - )); - for (i, (purl, uuid)) in to_fetch.iter().enumerate() { - if to_fetch.len() > 1 { - status.set(format!( - "{} ({}/{})", - format_fetching_content(to_fetch.len()), - i + 1, - to_fetch.len() - )); - } - // The record is what `covered` above judged, so it is also what - // decides which of this view's files actually need bytes. - let record = manifest.patches.get(*purl); - let view = match views.next().await { - Some((planned, view)) if planned == *uuid => view.release(), - // Unreachable: the plan IS this list. Falling back to the - // live request keeps the staging COMPLETE if the two ever - // fall out of step — running dry here would otherwise - // return `Ready` with blobs missing and nothing in - // `failed`. - _ => { - debug_assert!(false, "view prefetch plan out of step with the fetch list"); - client.fetch_patch(uuid).await - } - }; - match view { - Ok(Some(patch)) => { - // Named so the per-file report is the same on every run: - // `patch.files` is a `HashMap`, so "the first file with - // no content" is otherwise bucket order. - let mut contentless: Vec<&str> = Vec::new(); - let mut malformed: Option = None; - for (file, info) in &patch.files { - let Some(b64) = &info.blob_content else { - // A zero-delta file is served without content - // because it needs none (see `covered` above). - // Anything else the patch changes is genuinely - // unsatisfiable — collect them all rather than - // abandoning the view's remaining files in - // `HashMap` order. - if record - .and_then(|r| r.files.get(file)) - .is_some_and(|f| !needs_blob(f)) - { - continue; - } - contentless.push(file); - continue; - }; - let Some(hash) = &info.after_hash else { - malformed = - Some(format!("the patch view served no afterHash for {file}")); - break; - }; - // Same key guard as the disk writer: the hash names the - // lookup key the apply pipeline gates writes on. - if !is_valid_blob_hash(hash) { - malformed = Some(format!( - "the patch view served an invalid afterHash for {file}" - )); - break; - } - match base64_decode(b64) { - Ok(bytes) => { - mem.insert(hash.clone(), bytes); - } - Err(_) => { - malformed = Some(format!( - "the patch view served undecodable blob content for {file}" - )); - break; - } - } - } - contentless.sort_unstable(); - // An error, not progress chatter: prints even under - // --silent (same rule as report_offline_missing above). - if !common.json { - for file in &contentless { - status.println(format!( - " [error] {purl}: no blob content served for {file}" - )); - } - } - if let Some(reason) = malformed.or_else(|| contentless_reason(&contentless)) { - failed.push((purl, reason)); - } - } - Ok(None) => failed.push((purl, format!("no patch view is served for {uuid}"))), - Err(e) => failed.push((purl, format!("the patch view could not be fetched: {e}"))), - } - } - status.finish(); - if !failed.is_empty() { - // An error, not progress chatter: the vendor caller only marks - // the envelope (printed exclusively under --json), so muting - // this under --silent meant exit 1 with zero output — the - // CLI_CONTRACT violation ("errors only", NEVER nothing) fixed - // for the disk stager's arms above. It stays the ONE human - // channel for these purls in both arms below: the per-package - // arm only records events. - if !common.json { - let purls: Vec<&str> = failed.iter().map(|(purl, _)| *purl).collect(); - eprintln!( - "Error: Could not fetch patch content for {}:", - plural(failed.len(), "patch", "patches") - ); - for line in format_purl_list(&purls, 5) { - eprintln!("{line}"); - } - } - // Nothing in the manifest is usable ⇒ the pre-event bail (no - // events to report). Otherwise these purls are unsatisfiable - // packages the caller reports one by one, and the rest of the - // run continues. - if failed.len() == manifest.patches.len() { - return MemStageOutcome::Unavailable; - } - unavailable = failed - .into_iter() - .map(|(purl, reason)| (purl.to_string(), reason)) - .collect(); - unavailable.sort(); - } - } - - MemStageOutcome::Ready(MemStagedSources { - blobs, - diffs, - mem, - unavailable, - }) -} - -/// Record the per-package `failed` event for every purl -/// [`stage_vendor_sources_in_memory`] could not obtain patch content for, -/// and hand back the records the run can still vendor. `true` when at least -/// one purl was dropped (the run has errors). Borrows `records` untouched -/// on the overwhelmingly common empty path. -pub(crate) fn drop_unstageable<'a>( - env: &mut Envelope, - records: &'a HashMap, - unavailable: &[(String, String)], -) -> (Cow<'a, HashMap>, bool) { - if unavailable.is_empty() { - return (Cow::Borrowed(records), false); - } - for (purl, reason) in unavailable { - env.record( - PatchEvent::new(PatchAction::Failed, purl.clone()) - .with_error("no_local_source", reason.clone()), - ); - } - let mut kept = records.clone(); - kept.retain(|purl, _| !unavailable.iter().any(|(dropped, _)| dropped == purl)); - (Cow::Owned(kept), true) -} - -/// The reason string for a view that came back missing the blob content of -/// `contentless` (already sorted). `None` when nothing was missing. -fn contentless_reason(contentless: &[&str]) -> Option { - let (first, rest) = contentless.split_first()?; - Some(match rest.len() { - 0 => format!("the patch view served no blob content for {first}"), - n => format!( - "the patch view served no blob content for {first} (and {n} more file{})", - if n == 1 { "" } else { "s" } - ), - }) -} - #[cfg(test)] mod tests { use super::*; + use socket_patch_core::api::client::get_api_client_with_overrides; - /// The per-package `no_local_source` detail is the ONE machine-readable - /// explanation a `--json` consumer gets (every human channel in the - /// stager is gated on `!--json`), so its wording is pinned here — - /// including the count, which `plural` already carries. - #[test] - fn contentless_reason_names_the_file_and_counts_the_rest() { - assert_eq!(contentless_reason(&[]), None); - assert_eq!( - contentless_reason(&["package/index.js"]).as_deref(), - Some("the patch view served no blob content for package/index.js") - ); - assert_eq!( - contentless_reason(&["a.js", "b.js"]).as_deref(), - Some("the patch view served no blob content for a.js (and 1 more file)") - ); - assert_eq!( - contentless_reason(&["a.js", "b.js", "c.js"]).as_deref(), - Some("the patch view served no blob content for a.js (and 2 more files)") - ); - } - - #[test] - fn progress_lines_name_no_internal_tags() { - assert_eq!( - DOWNLOADING_ARTIFACTS, - "Downloading missing patch artifacts..." - ); - assert_eq!( - format_fetching_content(1), - "Fetching content for 1 patch..." - ); - assert_eq!( - format_fetching_content(3), - "Fetching content for 3 patches..." - ); - } use socket_patch_core::manifest::schema::{PatchFileInfo, PatchRecord}; const UUID: &str = "11111111-1111-4111-8111-111111111111"; @@ -1049,138 +638,6 @@ mod tests { assert_eq!(files, ["new.js"]); } - /// The vendor (in-memory) stager documents the opposite policy: a diff - /// archive is NOT sufficient (auto-force can need the full after-blob), - /// so the same fixture that satisfies the disk stager is Unavailable - /// offline here. Pins the asymmetry both module docs describe. - #[tokio::test] - async fn mem_stage_offline_rejects_diff_archive_as_sole_source() { - let tmp = tempfile::tempdir().unwrap(); - let socket_dir = tmp.path().join(".socket"); - std::fs::create_dir_all(socket_dir.join("diffs")).unwrap(); - std::fs::write( - socket_dir.join("diffs").join(format!("{UUID}.tar.gz")), - b"x", - ) - .unwrap(); - let project_root = tmp.path().join("proj"); - std::fs::create_dir_all(&project_root).unwrap(); - - let outcome = stage_vendor_sources_in_memory( - &offline_args(), - &manifest_with_one_patch(), - &socket_dir, - &project_root, - Ok(&HashMap::new()), - HashMap::new(), - None, - ) - .await; - assert!( - matches!(outcome, MemStageOutcome::Unavailable), - "vendor staging must not treat a diff archive as a usable source" - ); - } - - /// The download phase's blob seed IS a source: with every after-hash - /// seeded, an offline run with no disk blobs, no archives and no - /// committed artifact is Ready and stages the seeded bytes (no fetch, - /// no harvest needed) — the vendored flows never fetch a view twice. - #[tokio::test] - async fn mem_stage_seeded_blobs_are_ready_offline_without_any_disk_source() { - let tmp = tempfile::tempdir().unwrap(); - let socket_dir = tmp.path().join(".socket"); - let project_root = tmp.path().join("proj"); - std::fs::create_dir_all(&project_root).unwrap(); - let seed: HashMap> = [(HASH.to_string(), b"seeded".to_vec())].into(); - - let outcome = stage_vendor_sources_in_memory( - &offline_args(), - &manifest_with_one_patch(), - &socket_dir, - &project_root, - Ok(&HashMap::new()), - seed, - None, - ) - .await; - let MemStageOutcome::Ready(staged) = outcome else { - panic!("a fully seeded stage must be Ready"); - }; - assert_eq!( - staged.mem.get(HASH).map(Vec::as_slice), - Some(&b"seeded"[..]), - "the seeded bytes are the staged content" - ); - assert!( - !socket_dir.exists(), - "in-memory staging must not create .socket/" - ); - } - - /// A seed covering only SOME hashes still leaves the rest to the - /// ladder: offline with nothing else, the record is Unavailable (the - /// seed is merged, never treated as complete coverage). - #[tokio::test] - async fn mem_stage_partial_seed_still_needs_the_missing_hash() { - let tmp = tempfile::tempdir().unwrap(); - let socket_dir = tmp.path().join(".socket"); - let project_root = tmp.path().join("proj"); - std::fs::create_dir_all(&project_root).unwrap(); - let mut manifest = manifest_with_one_patch(); - manifest - .patches - .get_mut("pkg:npm/left-pad@1.3.0") - .unwrap() - .files - .insert( - "other.js".to_string(), - PatchFileInfo { - before_hash: "d".repeat(64), - after_hash: "e".repeat(64), - }, - ); - let seed: HashMap> = [(HASH.to_string(), b"seeded".to_vec())].into(); - - let outcome = stage_vendor_sources_in_memory( - &offline_args(), - &manifest, - &socket_dir, - &project_root, - Ok(&HashMap::new()), - seed, - None, - ) - .await; - assert!( - matches!(outcome, MemStageOutcome::Unavailable), - "one seeded hash out of two is not coverage" - ); - } - - /// An unreadable ledger (`Err`) harvests nothing — and is not an - /// error here: the caller reports the corrupt ledger itself. - #[tokio::test] - async fn mem_stage_unreadable_ledger_skips_the_harvest() { - let tmp = tempfile::tempdir().unwrap(); - let socket_dir = tmp.path().join(".socket"); - let project_root = tmp.path().join("proj"); - std::fs::create_dir_all(&project_root).unwrap(); - let err = std::io::Error::other("corrupt state.json"); - - let outcome = stage_vendor_sources_in_memory( - &offline_args(), - &manifest_with_one_patch(), - &socket_dir, - &project_root, - Err(&err), - HashMap::new(), - None, - ) - .await; - assert!(matches!(outcome, MemStageOutcome::Unavailable)); - } - /// GlobalArgs wired to a guaranteed-unreachable API endpoint: explicit /// token + org overrides keep client construction network-free, and the /// URL points at a port that was just bound and released, so every fetch diff --git a/crates/socket-patch-cli/src/commands/get.rs b/crates/socket-patch-cli/src/commands/get.rs index daf8abbd4..b72f5ecbe 100644 --- a/crates/socket-patch-cli/src/commands/get.rs +++ b/crates/socket-patch-cli/src/commands/get.rs @@ -15,9 +15,7 @@ use socket_patch_core::formats::pnpm::PnpmLock; use socket_patch_core::manifest::operations::{read_manifest, write_manifest}; pub(crate) use socket_patch_core::manifest::records::record_from_patch_response; use socket_patch_core::manifest::records::{build_patch_record, files_for_manifest}; -use socket_patch_core::manifest::schema::{ - PatchFileInfo, PatchManifest, PatchRecord, -}; +use socket_patch_core::manifest::schema::{PatchFileInfo, PatchManifest, PatchRecord}; use socket_patch_core::patch::apply::{is_valid_blob_hash, select_installed_variants}; use socket_patch_core::patch::apply_lock::{LockError, LockGuard}; use socket_patch_core::telemetry::{track_patch_fetch_failed, track_patch_fetched}; @@ -391,7 +389,6 @@ fn files_with_both_hashes(patch: &PatchResponse) -> HashMap, ) -> LockRefusals { let cwd = params.cwd.as_path(); - let claimed: Vec = - socket_patch_core::patch::redirect::upstream::HostedPin::all( - &socket_patch_core::vex::discover_patched_refs_with( - cwd, - &socket_patch_core::vex::DiscoverOptions { - patch_server_origins: params - .patch_server_url - .iter() - .filter(|url| !url.trim().is_empty()) - .cloned() - .collect(), - }, - ) - .await, + let claimed: Vec = socket_patch_core::patch::redirect::upstream::HostedPin::all( + &socket_patch_core::vex::discover_patched_refs_with( + cwd, + &socket_patch_core::vex::DiscoverOptions { + patch_server_origins: params + .patch_server_url + .iter() + .filter(|url| !url.trim().is_empty()) + .cloned() + .collect(), + }, ) - .into_iter() - .map(|pin| canonical_purl(&pin.purl)) - .collect(); + .await, + ) + .into_iter() + .map(|pin| canonical_purl(&pin.purl)) + .collect(); let candidates: Vec<(&str, &str)> = selected .iter() .filter(|sr| bun_refusal.filter(|r| r.applies_to(&sr.purl)).is_none()) @@ -2086,17 +2082,8 @@ async fn fetch_selected_patches( batch } -/// The vendored download phase's result: `(exit code, download JSON, -/// records by purl, blob seed)` — the seed is every fetched view's decoded -/// `blobContent` keyed by after-hash, for the vendor stager -/// (`fetch_stage::stage_vendor_sources_in_memory`), so the step never -/// fetches a view this phase already holds. -pub(crate) type DetachedDownload = ( - i32, - serde_json::Value, - HashMap, - HashMap>, -); +/// Download status and patch records used to verify server artifacts. +pub(crate) type DetachedDownload = (i32, serde_json::Value, HashMap); /// Download patches WITHOUT touching the manifest and return the fetched /// records keyed by purl — the download phase of every vendored run @@ -2114,9 +2101,6 @@ pub(crate) type DetachedDownload = ( /// from). The ledger idempotency check runs before the cache lookup, and a /// cache miss still fetches. /// -/// The blob seed is best-effort: an undecodable or missing `blobContent` -/// contributes nothing and is NOT a failed record (the stager reports what -/// it cannot source). pub(crate) async fn download_patch_records_with( selected: &[PatchSearchResult], params: &DownloadParams, @@ -2211,21 +2195,7 @@ async fn download_patch_records_preflighted( let downloaded = batch.fetched.len(); let mut records: HashMap = batch.reused.into_iter().collect(); - let mut blobs: HashMap> = HashMap::new(); for FetchedPatch { patch, files, .. } in batch.fetched { - for info in patch.files.values() { - // Same key guard as the blob writers: the hash names the lookup - // key the apply pipeline gates writes on. - let (Some(b64), Some(hash)) = (&info.blob_content, &info.after_hash) else { - continue; - }; - if !is_valid_blob_hash(hash) || blobs.contains_key(hash) { - continue; - } - if let Ok(bytes) = base64_decode(b64) { - blobs.insert(hash.clone(), bytes); - } - } records.insert(patch.purl.clone(), build_patch_record(&patch, files)); } let mut result_json = serde_json::json!({ @@ -2239,7 +2209,7 @@ async fn download_patch_records_preflighted( if !batch.warnings.is_empty() { result_json["warnings"] = serde_json::json!(batch.warnings); } - (i32::from(batch.failed > 0), result_json, records, blobs) + (i32::from(batch.failed > 0), result_json, records) } /// Emit a warning (stderr `[note]` + `warnings[]`) for every added/updated @@ -2551,11 +2521,13 @@ pub async fn run(args: GetArgs) -> i32 { // v5: hosted by default, like scan. `--save-only` (records a manifest // entry) and global installs (no project lockfile) mean agent mode. // Usage errors exit 2, like clap's and scan's (v5.0). - let mode = args.mode.unwrap_or(if args.save_only || args.common.is_global() { - super::scan::ScanMode::Agent - } else { - super::scan::ScanMode::Hosted - }); + let mode = args + .mode + .unwrap_or(if args.save_only || args.common.is_global() { + super::scan::ScanMode::Agent + } else { + super::scan::ScanMode::Hosted + }); if args.save_only && mode != super::scan::ScanMode::Agent { report_error( args.common.json, @@ -2809,8 +2781,7 @@ pub async fn run(args: GetArgs) -> i32 { } IdentifierType::Package => { status.set("Enumerating packages..."); - let (all_packages, _, _) = - crawl_all_ecosystems(&args.common.crawler_options()).await; + let (all_packages, _, _) = crawl_all_ecosystems(&args.common.crawler_options()).await; if all_packages.is_empty() { status.finish(); @@ -2957,7 +2928,10 @@ pub async fn run(args: GetArgs) -> i32 { (kept_accessible, narrowing.kept, skips, narrowing.warnings) }; // `get` bypasses the repo's socket.yml policy, but says so. - narrow_warnings.extend(super::scan::policy::policy_bypass_warnings(&args.common, &accessible)); + narrow_warnings.extend(super::scan::policy::policy_bypass_warnings( + &args.common, + &accessible, + )); // Layout refusals print even when informational output is quieted only // by --json (stderr; the envelope carries them too) — but --silent // mutes them like scan does. @@ -3729,7 +3703,7 @@ async fn run_get_vendored( let prefetched_views: HashMap = prefetched .map(|p| HashMap::from([(p.uuid.clone(), p.clone())])) .unwrap_or_default(); - let (dl_code, mut result, records, blobs) = if prefetched.is_some() { + let (dl_code, mut result, records) = if prefetched.is_some() { // The preflight above already read the lock: hand its outcome down. let vendor_state = load_state(&args.common.cwd).await; Box::pin(download_patch_records_preflighted( @@ -3764,7 +3738,6 @@ async fn run_get_vendored( match super::scan::boxed_vendor_step(super::scan::VendorStep { common: &args.common, records, - seed: blobs, client: api_client.clone(), use_public_proxy, report_empty: true, @@ -5312,8 +5285,14 @@ mod tests { #[test] fn confirm_prompts_agent_mode() { - assert_eq!(format_confirm_prompt(false, 1), "Download and apply 1 patch?"); - assert_eq!(format_confirm_prompt(false, 2), "Download and apply 2 patches?"); + assert_eq!( + format_confirm_prompt(false, 1), + "Download and apply 1 patch?" + ); + assert_eq!( + format_confirm_prompt(false, 2), + "Download and apply 2 patches?" + ); assert_eq!(format_confirm_prompt(true, 1), "Download 1 patch?"); } @@ -5664,7 +5643,7 @@ mod tests { server_url: &str, ) -> (i32, serde_json::Value, HashMap) { let api_client = test_client(server_url).await; - let (code, json, records, _blobs) = + let (code, json, records) = download_patch_records_with(selected, params, &api_client, HashMap::new()).await; (code, json, records) } @@ -6697,19 +6676,12 @@ mod tests { let client = test_client(&server.uri()).await; let prefetched = HashMap::from([(patch.uuid.clone(), patch.clone())]); - let (code, json, records, blobs) = + let (code, json, records) = download_patch_records_with(&selected, ¶ms, &client, prefetched).await; assert_eq!(code, 0, "json={json}"); assert_eq!(json["downloaded"], 1, "json={json}"); - // The blob seed carries every served `blobContent` by after-hash — - // decoded — and only those; the vendor stager starts from it. - assert_eq!( - blobs.get(&"1".repeat(64)).map(Vec::as_slice), - Some(&b"patched"[..]), - "the served blob is seeded under its after-hash" - ); - assert_eq!(blobs.len(), 1, "a file with no blobContent seeds nothing"); + assert!(!tmp.path().join(".socket/blobs").exists()); assert_eq!(json["detached"], true, "json={json}"); assert_eq!(json["patches"][0]["action"], "downloaded", "json={json}"); assert!( @@ -6929,7 +6901,7 @@ mod tests { .map(|(u, n)| mk_patch(u, &purl(n), "free", "2024-01-01")) .collect(); let client = test_client(&server.uri()).await; - let (_code, json, records, _blobs) = download_patch_records_with( + let (_code, json, records) = download_patch_records_with( &selected, &detached_params(tmp.path()), &client, @@ -6995,8 +6967,11 @@ mod tests { let installed = |name: &str, body: &[u8]| { let dist = site.path().join(format!("{name}-1.0.0.dist-info")); std::fs::create_dir_all(&dist).unwrap(); - std::fs::write(dist.join("METADATA"), format!("Name: {name}\nVersion: 1.0.0\n")) - .unwrap(); + std::fs::write( + dist.join("METADATA"), + format!("Name: {name}\nVersion: 1.0.0\n"), + ) + .unwrap(); std::fs::write(site.path().join(format!("{name}.py")), body).unwrap(); compute_git_sha256_from_bytes(body) }; @@ -7040,7 +7015,10 @@ mod tests { mount(uuid("bs"), "beta_sdist.py".into(), "0".repeat(64), 0).await; for n in ["gw", "gs"] { Mock::given(method("GET")) - .and(wm_path(format!("/v0/orgs/test-org/patches/view/{}", uuid(n)))) + .and(wm_path(format!( + "/v0/orgs/test-org/patches/view/{}", + uuid(n) + ))) .respond_with(ResponseTemplate::new(500)) .expect(0) .mount(&server) diff --git a/crates/socket-patch-cli/src/commands/repair.rs b/crates/socket-patch-cli/src/commands/repair.rs index 69cdecb82..323b447b8 100644 --- a/crates/socket-patch-cli/src/commands/repair.rs +++ b/crates/socket-patch-cli/src/commands/repair.rs @@ -81,7 +81,8 @@ pub async fn run(args: RepairArgs) -> i32 { let mut has_vendor_traces = tokio::fs::metadata(&state_file).await.is_ok(); if !has_vendor_traces { let refs = - crate::commands::vendored_backend::repair::scan_vendor_references(&args.common.cwd).await; + crate::commands::vendored_backend::repair::scan_vendor_references(&args.common.cwd) + .await; has_vendor_traces = !refs.is_empty(); vendor_references = Some(refs); } @@ -151,7 +152,10 @@ pub async fn run(args: RepairArgs) -> i32 { // scanned this ledger-less project. let vendor_references = match vendor_references { Some(refs) => refs, - None => crate::commands::vendored_backend::repair::scan_vendor_references(&args.common.cwd).await, + None => { + crate::commands::vendored_backend::repair::scan_vendor_references(&args.common.cwd) + .await + } }; // The API client is built lazily: `repair_inner` constructs it only on @@ -595,26 +599,27 @@ async fn repair_inner( // ledger entry are reported. Runs under `--download-only` too: // restoring artifacts IS repair's download half. The reference scan // and ledger load above are handed over, not repeated. - let vendor_rebuilt = crate::commands::vendored_backend::VendoredBackend::new( - &args.common, - None, - ) - .repair( - crate::commands::vendored_backend::repair::RepairRequest { - manifest: manifest.as_ref(), - socket_dir: &socket_dir, - references: &vendor_references, - ledger, - client: client.as_ref(), - }, - &mut env, - ) - .await; - if !quiet && vendor_rebuilt > 0 { + let vendor_redownloaded = + crate::commands::vendored_backend::VendoredBackend::new(&args.common, None) + .repair( + crate::commands::vendored_backend::repair::RepairRequest { + manifest: manifest.as_ref(), + references: &vendor_references, + ledger, + client: client.as_ref(), + }, + &mut env, + ) + .await; + if !quiet && vendor_redownloaded > 0 { stdout_started = true; println!( - "Rebuilt {}.", - crate::ui::plural(vendor_rebuilt, "vendored artifact", "vendored artifacts") + "Redownloaded {}.", + crate::ui::plural( + vendor_redownloaded, + "vendored artifact", + "vendored artifacts" + ) ); } @@ -1031,7 +1036,10 @@ mod tests { // Both orphans and the legacy package archive go; the referenced // diff archive stays. - assert_eq!(counts.cleaned, 3, "orphans and legacy archives should be swept"); + assert_eq!( + counts.cleaned, 3, + "orphans and legacy archives should be swept" + ); assert_eq!( counts.bytes_freed, (orphan_diff.len() + orphan_pkg.len() + legacy_pkg.len()) as u64, diff --git a/crates/socket-patch-cli/src/commands/scan/discovery.rs b/crates/socket-patch-cli/src/commands/scan/discovery.rs index f22dc28d8..b82cb74a5 100644 --- a/crates/socket-patch-cli/src/commands/scan/discovery.rs +++ b/crates/socket-patch-cli/src/commands/scan/discovery.rs @@ -220,29 +220,6 @@ async fn vendored_purls_from_artifacts(common: &GlobalArgs) -> Vec { out } -/// Vendor-mode pre-flight check: uuids of selected patches whose installed -/// files match NEITHER beforeHash nor afterHash — the patch was built -/// against different bytes than the installed artifact. Vendoring still -/// succeeds for these (the vendor stage force-applies the verified patched -/// content; see `force_apply_staged`), but the user learns it before -/// vendoring starts rather than from a post-hoc warning event. -/// -/// Returns `(mismatched uuids, fetched views by uuid)`: the download phase -/// serves its records from the views instead of fetching each one a second -/// time. Only `Ok(Some)` views are cached — an errored or 404'd fetch is -/// left for the download phase to retry and report per patch. -/// -/// `vendor` is the run's ledger (`None` when unreadable — fail-open, the -/// preflight reports the corruption): a purl the ledger already holds -/// detached at the selected uuid with an embedded record is compared -/// against that record's file hashes instead of fetching the view, so an -/// idempotent re-run performs zero view fetches. Nothing is inserted into -/// `views` for them. -/// -/// Best-effort and read-only: a detail-fetch failure or an unresolvable -/// installed path just skips the annotation — it never blocks the flow and -/// writes nothing. One API round-trip per uncached patch, so progress -/// shows on `status`. pub(super) async fn preverify_vendor_baselines( api_client: &socket_patch_core::api::client::ApiClient, selected: &[PatchSearchResult], @@ -370,7 +347,6 @@ pub(super) async fn preverify_vendor_baselines( pub(super) use socket_patch_core::ledgers::merge_ledger_records_for_updates; - /// Cross-reference an existing manifest against discovery results to find /// PURLs whose newest available patch UUID differs from the locally-recorded /// one. Used by both the discovery JSON path and the table-print path. @@ -522,8 +498,8 @@ pub(super) fn severity_order(s: &str) -> u8 { #[cfg(test)] mod tests { use super::*; - use std::borrow::Cow; use socket_patch_core::api::types::BatchPatchInfo; + use std::borrow::Cow; use crate::commands::scan::tests::manifest_with; @@ -1784,6 +1760,7 @@ mod tests { base_purl: "pkg:npm/insync@1.0.0".into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{uuid}/insync-1.0.0.tgz"), sha256: String::new(), size: None, @@ -1908,7 +1885,11 @@ mod tests { "pkg:npm/lockonly@1.0.0", std::path::PathBuf::from("/nonexistent"), ), - crawled_pkg("alpha", "pkg:npm/alpha@1.0.0", installed("alpha", "alpha.js")), + crawled_pkg( + "alpha", + "pkg:npm/alpha@1.0.0", + installed("alpha", "alpha.js"), + ), crawled_pkg( "embedded", "pkg:npm/embedded@1.0.0", @@ -1932,6 +1913,7 @@ mod tests { base_purl: "pkg:npm/embedded@1.0.0".into(), uuid: "u-embedded".into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: ".socket/vendor/npm/u-embedded/embedded-1.0.0.tgz".into(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/src/commands/scan/gc.rs b/crates/socket-patch-cli/src/commands/scan/gc.rs index d7084566c..c7c72a74b 100644 --- a/crates/socket-patch-cli/src/commands/scan/gc.rs +++ b/crates/socket-patch-cli/src/commands/scan/gc.rs @@ -1226,6 +1226,7 @@ mod tests { base_purl: PURL.into(), uuid: UUID.into(), artifact: socket_patch_core::vendor::state::VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{UUID}/gone-1.0.0.tgz"), sha256: String::new(), size: None, @@ -1346,6 +1347,7 @@ mod tests { base_purl: PURL.into(), uuid: UUID.into(), artifact: socket_patch_core::vendor::state::VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{UUID}/gone-1.0.0.tgz"), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs index cf26ce857..5507bb977 100644 --- a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs +++ b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs @@ -33,9 +33,7 @@ use crate::commands::bun_preflight::bun_vendor_preflight_with_ledger; use crate::commands::get::{download_patch_records_reusing, DetachedDownload, DownloadParams}; use crate::commands::lock_cli::lock_failure; use crate::commands::vendor::{note_classic_migration_risk, track_outcomes_for_vendor}; -use crate::commands::vendored_backend::{ - records_manifest, ApplyRequest, NoLocalSource, VendoredBackend, NO_LOCAL_SOURCE_MESSAGE, -}; +use crate::commands::vendored_backend::{records_manifest, ApplyRequest, VendoredBackend}; use crate::commands::vlt_preflight::{vlt_refusal_for, vlt_vendor_preflight_selected}; use crate::ecosystem_dispatch::NpmCrawlSnapshot; use crate::json_envelope::{Command as EnvelopeCommand, Envelope}; @@ -176,7 +174,6 @@ pub(crate) fn print_dry_run_refusals(preview: &serde_json::Value) { pub(crate) struct VendorStep<'a> { pub(crate) common: &'a GlobalArgs, pub(crate) records: HashMap, - pub(crate) seed: HashMap>, pub(crate) client: ApiClient, pub(crate) use_public_proxy: bool, /// Print "No vendorable patches in scope." when there are no records @@ -215,7 +212,6 @@ async fn run_vendor_step(step: VendorStep<'_>) -> VendorStepResult { let VendorStep { common, records, - seed, client, use_public_proxy, report_empty, @@ -227,7 +223,6 @@ async fn run_vendor_step(step: VendorStep<'_>) -> VendorStepResult { let outcome = vendor_under_lock( common, records, - seed, client, use_public_proxy, report_empty, @@ -249,8 +244,7 @@ async fn run_vendor_step(step: VendorStep<'_>) -> VendorStepResult { .await } Err((_, message, _)) => { - track_patch_vendor_failed(message, common.dry_run, telemetry_token, telemetry_org) - .await + track_patch_vendor_failed(message, common.dry_run, telemetry_token, telemetry_org).await } } outcome.map(|(vendor_errors, venv)| (download_errors || vendor_errors, venv)) @@ -260,7 +254,6 @@ async fn run_vendor_step(step: VendorStep<'_>) -> VendorStepResult { async fn vendor_under_lock( common: &GlobalArgs, records: HashMap, - seed: HashMap>, client: ApiClient, use_public_proxy: bool, report_empty: bool, @@ -301,7 +294,6 @@ async fn vendor_under_lock( // Loaded ONCE under the lock: the staging harvest reads it, // then the engine takes it over for its persists. ledger: load_state(&common.cwd).await, - seed, // Always detached: vendored mode is manifest-free. detached: true, force: false, @@ -310,19 +302,7 @@ async fn vendor_under_lock( &mut env, ) .await; - let has_errors = match applied { - Ok(has_errors) => has_errors, - Err(NoLocalSource) => { - // The step ran and is aborting: hand its envelope (demoted) to - // the caller's fold. - env.mark_partial_failure(); - return Err(( - "no_local_source", - NO_LOCAL_SOURCE_MESSAGE.to_string(), - Some(Box::new(env)), - )); - } - }; + let has_errors = applied; migrate_legacy_manifest_records(common, &manifest_path, &manifest.patches, &mut env).await; if has_errors { env.mark_partial_failure(); @@ -572,7 +552,7 @@ async fn run_vendor_json_path( let params = download_params( args, /*save_only=*/ true, /*json=*/ true, /*silent=*/ true, ); - let (dl_code, dl_json, records, blobs) = + let (dl_code, dl_json, records) = boxed_download_patch_records(&selected, ¶ms, api_client, HashMap::new(), prior).await; result["download"] = dl_json; @@ -581,7 +561,6 @@ async fn run_vendor_json_path( let vendor_code = match boxed_vendor_step(VendorStep { common: &args.common, records, - seed: blobs, client: api_client.clone(), use_public_proxy, report_empty: true, @@ -674,7 +653,7 @@ async fn run_vendor_interactive_path( plural(selected.len(), "patch", "patches") ); } - let (dl_code, dl_json, records, blobs) = + let (dl_code, dl_json, records) = boxed_download_patch_records(selected, params, api_client, prefetched, prior).await; // Patches the download phase could not get (it reported each one). let download_failed = dl_json["failed"].as_u64().unwrap_or(0); @@ -684,7 +663,6 @@ async fn run_vendor_interactive_path( let code = match boxed_vendor_step(VendorStep { common: &args.common, records, - seed: blobs, client: api_client.clone(), use_public_proxy, report_empty: false, @@ -976,6 +954,7 @@ mod migration_tests { base_purl: PURL.into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{uuid}/left-pad-1.3.0.tgz"), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/src/commands/vendor.rs b/crates/socket-patch-cli/src/commands/vendor.rs index dfdfedb21..627a6306b 100644 --- a/crates/socket-patch-cli/src/commands/vendor.rs +++ b/crates/socket-patch-cli/src/commands/vendor.rs @@ -29,14 +29,14 @@ use socket_patch_core::patch::apply::{verify_file_patch, PatchSources}; use socket_patch_core::patch::redirect::upstream::HostedPin; use socket_patch_core::telemetry::{track_patch_vendor_failed, track_patch_vendored}; use socket_patch_core::utils::composer_version::composer_purls_equivalent; -use socket_patch_core::utils::concurrent::{ordered_concurrent, registry_concurrency}; +use socket_patch_core::utils::concurrent::ordered_concurrent; use socket_patch_core::utils::group_commit::GroupCommit; use socket_patch_core::utils::purl::{canonical_purl, normalize_purl, strip_purl_qualifiers}; use socket_patch_core::utils::socket_dir::remove_tree_and_prune; use socket_patch_core::vendor::{ - self, ecosystem_dir_for_purl, load_state, lock_inventory, lookup_entry, registry_fetch, - save_state, save_state_shared, DeferredMiss, DeferredPackage, PackageSource, RevertOpts, - RevertOutcome, VendorEntry, VendorOutcome, VendorServiceConfig, VendorState, VendorWarning, + self, ecosystem_dir_for_purl, load_state, lock_inventory, lookup_entry, save_state, + save_state_shared, PackageSource, RevertOpts, RevertOutcome, VendorEntry, VendorOutcome, + VendorServiceConfig, VendorState, VendorWarning, }; use socket_patch_core::vex::time::now_rfc3339; use std::collections::{HashMap, HashSet}; @@ -49,7 +49,7 @@ use crate::commands::apply::{representative_file, result_to_event, variant_match use crate::commands::bun_preflight::bun_vendor_preflight_pairs; use crate::commands::lock_cli::acquire_or_emit; use crate::commands::vendored_backend::{ - ApplyRequest, RevertedEntry, VendorRevertStep, VendoredBackend, NO_LOCAL_SOURCE_MESSAGE, + ApplyRequest, RevertedEntry, VendorRevertStep, VendoredBackend, }; use crate::commands::vex::{ generate_vex_from_manifest_path, generate_vex_without_manifest, ManifestlessVex, VexEmbedArgs, @@ -128,19 +128,12 @@ pub(crate) async fn dispatch_vendor_one( vendored_at: &str, dry_run: bool, force: bool, - // The patch.socket.dev vendoring-service config. `None` = build-only; - // `vendor` and `scan`/`get --mode vendored` pass `Some(_)` (honoring - // `--vendor-source`); repair passes `None` — it rebuilds locally from - // the recorded patch. service: Option<&VendorServiceConfig>, pipenv_version: &tokio::sync::OnceCell>, installed_sites: &vendor::pypi::InstalledSiteListings, ) -> Option { let eco = ecosystem_dir_for_purl(purl)?; - // Ecosystems with prebuilt service downloads. Under fail-closed `service` - // mode any other ecosystem is refused rather than silently built; under - // `auto`/`build` it falls through to the local build. const SERVICE_ECOSYSTEMS: &[&str] = &[ "npm", "pypi", "cargo", "golang", "composer", "gem", "nuget", "maven", ]; @@ -151,8 +144,7 @@ pub(crate) async fn dispatch_vendor_one( detail: format!( "--vendor-source=service is not supported for `{eco}` \ (prebuilt downloads cover npm, pypi, cargo, golang, composer, \ - gem, nuget, and maven); \ - use --vendor-source=auto or --vendor-source=build" + gem, nuget, and maven)" ), }); } @@ -1135,9 +1127,8 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { } // One view per distinct uuid, fetched concurrently and consumed in pin - // order; the views' blobs seed the in-memory staging. + // order; only their records are needed to verify the artifacts. let mut records: HashMap = HashMap::new(); - let mut blobs: HashMap> = HashMap::new(); let mut fetch_failures: Vec<(String, String)> = Vec::new(); let mut views = std::pin::pin!(ordered_concurrent( pins.iter(), @@ -1156,19 +1147,6 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { }; match view { Ok(Some(patch)) => { - for info in patch.files.values() { - let (Some(b64), Some(hash)) = (&info.blob_content, &info.after_hash) else { - continue; - }; - if !socket_patch_core::patch::apply::is_valid_blob_hash(hash) - || blobs.contains_key(hash) - { - continue; - } - if let Ok(bytes) = crate::commands::get::base64_decode(b64) { - blobs.insert(hash.clone(), bytes); - } - } let (_, record) = crate::commands::get::record_from_patch_response(&patch); records.insert(pin.purl.clone(), record); } @@ -1373,7 +1351,6 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { manifest: &manifest, socket_dir: &socket_dir, ledger: load_state(&common.cwd).await, - seed: blobs, detached: true, force: false, prior: None, @@ -1381,23 +1358,7 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { &mut env, ) .await; - match applied { - Ok(has_errors) => exit = i32::from(has_errors), - Err(_) => { - let code = "no_local_source"; - env.mark_error(EnvelopeError::new(code, NO_LOCAL_SOURCE_MESSAGE)); - if !common.json { - eprintln!( - "{}", - crate::commands::scan::vendor_flow::format_vendor_step_error( - code, - NO_LOCAL_SOURCE_MESSAGE - ) - ); - } - exit = 1; - } - } + exit = i32::from(applied); } if exit != 0 { match snapshot.restore().await { @@ -1554,16 +1515,13 @@ async fn run_vendor( if manifest.patches.is_empty() && !common.json && !common.silent { println!("The manifest has no patches; nothing to vendor."); } - // The shared vendored apply: in-memory staging (committed .socket - // artifacts read in place, missing content fetched per patch — no seed: - // this manifest-driven command has no download phase) → the engine. + // Download the server artifacts through the shared vendored backend. let applied = VendoredBackend::new(common, Some(service)) .apply( ApplyRequest { manifest: &manifest, socket_dir: &socket_dir, ledger, - seed: HashMap::new(), detached: false, force: args.force, prior: None, @@ -1571,16 +1529,7 @@ async fn run_vendor( env, ) .await; - match applied { - Ok(errors) => has_errors |= errors, - Err(_) => { - env.mark_error(EnvelopeError::new( - "no_local_source", - NO_LOCAL_SOURCE_MESSAGE, - )); - return 1; - } - } + has_errors |= applied; if has_errors { // A run where EVERY event failed still reads as "partialFailure": @@ -1754,67 +1703,6 @@ async fn sweep_stale_artifact( ); } -/// One registry-fetch attempt through the pristine-source ladder's network -/// half: the lockfile inventory first, then the ledger-recovered pre-vendor -/// registry fragment (the live lockfile is rewired to `.socket/vendor/...` -/// for vendored packages, so only `--revert`'s restore data still knows the -/// registry resolution). Always integrity-verified fail-closed. -pub(crate) enum PristineFetch { - Fetched(registry_fetch::FetchedPackage), - /// Neither the lockfile nor the ledger can name a verifiable source. - NoSource, - Unverifiable(String), - Failed(String), -} - -pub(crate) async fn fetch_pristine_package( - project_root: &Path, - inventory: &[lock_inventory::LockfileEntry], - client: ®istry_fetch::RegistryClient, - purl: &str, - ledger_entry: Option<&VendorEntry>, -) -> PristineFetch { - // A lock entry that carries an integrity is the registry resolution to - // fetch. A DISCOVERY-ONLY entry (the lock is rewired to OUR reference, - // whose recorded hashes are the patched wheel's — nothing PyPI serves) - // cannot be fetched by itself: the ledger's pre-vendor fragment can, so - // an already-vendored lock-only checkout re-scans green. - let inventory_entry = lock_inventory::lookup(inventory, purl).cloned(); - let fetchable = inventory_entry - .as_ref() - .filter(|e| e.integrity != lock_inventory::LockIntegrity::None) - .cloned(); - let entry = match (fetchable, ledger_entry) { - (Some(e), _) => e, - (None, Some(le)) => match lock_inventory::recover_lock_entry(project_root, le).await { - Ok(rec) => rec, - Err(e) => { - return PristineFetch::Unverifiable(format!( - "the lockfile no longer records a registry resolution for {purl} \ - (rewired to the vendored artifact) and the ledger cannot recover \ - one: {e}" - )) - } - }, - (None, None) => match inventory_entry { - Some(e) => e, - None => return PristineFetch::NoSource, - }, - }; - match registry_fetch::fetch_and_stage(&entry, client).await { - Ok(fetched) => PristineFetch::Fetched(fetched), - Err(registry_fetch::FetchError::Unverifiable(d)) => PristineFetch::Unverifiable(d), - Err(registry_fetch::FetchError::Failed(d)) => PristineFetch::Failed(d), - } -} - -/// Whether [`fetch_pristine_package`] would pick a VERIFIABLE registry -/// resolution for this purl — the same entry choice, made without the -/// download: the lock's own entry when it carries an integrity, else the -/// pre-vendor resolution the ledger recovers. A cargo crate from a git, -/// path or custom-registry source has neither, so its fetch refuses -/// `vendor_fetch_unverifiable`; deferring that fetch behind the patch -/// service would instead vendor the crates.io patch over it. pub(crate) async fn pristine_fetch_is_verifiable( project_root: &Path, inventory: &[lock_inventory::LockfileEntry], @@ -1834,50 +1722,6 @@ pub(crate) async fn pristine_fetch_is_verifiable( } } -/// Whether [`fetch_pristine_package`] would reach the download for this -/// purl: the same entry choice (see [`pristine_fetch_is_verifiable`]), and -/// none of the refusals its fetcher raises before the first request (a -/// foreign yarn berry cacheKey, a go module go would not fetch through a -/// proxy, a composer entry with no dist URL). Deferring a fetch that would -/// refuse `vendor_fetch_unverifiable` behind the patch service would -/// instead vendor the patch over a package it does not describe. -async fn pristine_fetch_reaches_download( - project_root: &Path, - inventory: &[lock_inventory::LockfileEntry], - purl: &str, - ledger_entry: Option<&VendorEntry>, -) -> bool { - let entry = match lock_inventory::lookup(inventory, purl) - .filter(|e| e.integrity != lock_inventory::LockIntegrity::None) - { - Some(e) => e.clone(), - None => match ledger_entry { - Some(le) => match lock_inventory::recover_lock_entry(project_root, le).await { - Ok(e) => e, - Err(_) => return false, - }, - None => return false, - }, - }; - registry_fetch::refusal_before_download(&entry).is_none() -} - -/// The ecosystems whose backend asks the patch service before it reads the -/// pristine tree, and reads it only on a local-build fallback. pypi and gem -/// read it earlier, in the loop's installed-variant probe; nuget and maven -/// have no registry fetch. -fn backend_reads_pristine_only_on_fallback(purl: &str) -> bool { - matches!( - Ecosystem::from_purl(purl), - Some(Ecosystem::Npm | Ecosystem::Cargo | Ecosystem::Golang | Ecosystem::Composer) - ) -} - -/// The purls among `purls` with an installed copy, found exactly as the -/// vendor loop finds them: the qualified-aware resolver -/// ([`find_packages_for_rollback_reusing`]), then the npm `package.json` -/// identity lookup for an npm purl it missed (an alias install). `prior` -/// is the loop's own reusable npm crawl, when the caller has it. pub(crate) async fn installed_purls( options: &CrawlerOptions, purls: &[String], @@ -1906,16 +1750,6 @@ pub(crate) async fn installed_purls( installed } -/// Narrows `refused` (the lock-text refusals of -/// [`vendor::lock_text_refusals`]) to the packages the vendor loop would -/// actually hand to their backend — and so see refused, in these very -/// words — once it has a source for them: an installed copy (`installed` -/// answers, for the purls with no verifiable registry resolution), or a -/// verifiable registry resolution the pristine-source ladder fetches -/// ([`pristine_fetch_is_verifiable`]). A package with neither never -/// reaches its backend: the loop reports it `package_not_installed` (a -/// calm skip), with no pristine fetch, so it is left to the loop and keeps -/// that outcome. The check reads only local files. pub(crate) async fn lock_refusals_reaching_backend( cwd: &Path, mut refused: HashMap, @@ -1949,262 +1783,24 @@ where refused } -/// One purl's pristine source while the vendor loop is being assembled. -/// -/// A fetched artifact is held by index into the run's `fetched_holders` -/// rather than by path: the tree is not on disk yet (see -/// [`registry_fetch::FetchedPackage`]), and only a backend branch that -/// actually reads it makes it so. enum StagedSource { - /// The crawler's installed location. Installed(std::path::PathBuf), - /// `fetched_holders[i]`. - Fetched(usize), - /// `deferred_holders[i]`: not downloaded unless a backend reads it. - Deferred(usize), + Missing(std::path::PathBuf), } impl StagedSource { - fn as_source<'a>( - &'a self, - holders: &'a [registry_fetch::FetchedPackage], - deferred: &'a [DeferredPackage], - ) -> PackageSource<'a> { + fn as_source(&self) -> PackageSource<'_> { match self { - Self::Installed(dir) => PackageSource::Installed(dir), - Self::Fetched(at) => PackageSource::Pending(&holders[*at]), - Self::Deferred(at) => PackageSource::Deferred(&deferred[*at]), - } - } -} - -/// Where a vendorable purl with no installed copy stands after the local -/// rungs of the pristine-source ladder (see [`missing_local_rung`]). -enum MissingRung { - /// Staged from its own committed artifact (sha256-verified). - Staged(registry_fetch::FetchedPackage), - /// Its committed artifact is present but corrupt (the detail). - StageFailed(String), - /// `--offline`: no registry rung. - Offline, - /// Left for the registry fetch ([`fetch_pristine_package`]). - Fetch, - /// The registry fetch, run only if the backend reads the pristine tree - /// ([`deferred_pristine_package`]). - Deferred, - /// A gem a local build cannot vendor from a download: refused before - /// the fetch. - GemBuildRefused, -} - -impl MissingRung { - /// Whether this purl still needs [`fetch_pristine_package`] — the one - /// predicate behind both the fetch plan and the lazy lock inventory, - /// so they cannot name different purls. - fn needs_registry(&self) -> bool { - matches!(self, MissingRung::Fetch) - } -} - -/// The local rungs for one missing purl, deciding without emitting -/// anything: an already-vendored npm purl with no installed copy (fresh -/// clone) stages from its own committed artifact, sha256-verified against -/// the ledger (a vlt directory artifact against its file inventory) — -/// offline-safe, no registry traffic — and `--offline` stops before the -/// registry. Only at the record's own uuid: an older patch's artifact holds -/// that patch's bytes, never a pristine source for a superseding one. Also -/// returns the committed artifact's path when it is missing (the caller's -/// `vendor_artifact_missing` warning; the purl then falls through to the -/// registry ladder). -async fn missing_local_rung( - common: &GlobalArgs, - ledger_entry: Option<&VendorEntry>, - record: Option<&PatchRecord>, -) -> (Option, MissingRung) { - let mut artifact_missing = None; - if let Some(entry) = ledger_entry.filter(|e| { - e.ecosystem == "npm" - && record.is_some_and(|r| r.uuid == e.uuid) - && (e.artifact.path.ends_with(".tgz") - || !vendor::artifact_is_file_shaped(&e.artifact.path)) - }) { - let committed = common.cwd.join(&entry.artifact.path); - if tokio::fs::metadata(&committed).await.is_err() { - artifact_missing = Some(entry.artifact.path.clone()); - } else { - let staged = if entry.artifact.path.ends_with(".tgz") { - registry_fetch::stage_local_artifact(&committed, &entry.artifact.sha256).await - } else { - registry_fetch::stage_local_dir_artifact( - &committed, - entry.artifact.file_inventory.as_ref(), - ) - .await - }; - match staged { - Ok(staged) => return (None, MissingRung::Staged(staged)), - Err(registry_fetch::FetchError::Failed(detail)) => { - return (None, MissingRung::StageFailed(detail)) - } - // No recorded hash (legacy ledger) — fall through to the - // lockfile/registry path. - Err(registry_fetch::FetchError::Unverifiable(_)) => {} - } + Self::Installed(dir) | Self::Missing(dir) => PackageSource::Installed(dir), } } - let rung = if common.offline { - MissingRung::Offline - } else { - MissingRung::Fetch - }; - (artifact_missing, rung) -} - -/// Whether the ledger already covers `record` for `entry`'s purl: the entry -/// records this very patch uuid and its committed artifact is on disk — a -/// FILE artifact (wheel, tarball) hashing to the ledger's `sha256`. -/// Read-only, no network. The backend's in-sync hot path answers such a -/// purl from the committed artifact without reading the pristine tree, -/// which is what lets its download be deferred. Some in-sync checks look -/// only for the artifact's presence (pypi's), so a file artifact that no -/// longer hashes to its ledger pin is not covered: it keeps the eager -/// ladder. A copy DIR's integrity stays the backend's own question. -async fn ledger_covers(cwd: &Path, entry: Option<&VendorEntry>, record: &PatchRecord) -> bool { - match entry { - Some(entry) if entry.uuid == record.uuid => entry.committed_artifact_intact(cwd).await, - _ => false, - } -} - -/// The pristine source for a purl whose download is deferred: the same -/// [`fetch_pristine_package`] ladder, run on the first backend call that -/// reads the tree. `--offline` never reaches the registry, so there the -/// deferred fetch reports the offline stop instead. The outcome's `code` -/// tells [`deferred_miss`] which eager-fetch report to reproduce. -fn deferred_pristine_package( - common: &GlobalArgs, - inventory: &Arc>>, - client: ®istry_fetch::RegistryClient, - purl: &str, - ledger_entry: Option<&VendorEntry>, -) -> DeferredPackage { - let cwd = common.cwd.clone(); - let offline = common.offline; - let inventory = Arc::clone(inventory); - let client = client.clone(); - let owned_purl = purl.to_string(); - let ledger_entry = ledger_entry.cloned(); - DeferredPackage::new( - ®istry_fetch::staged_leaf_for_purl(purl), - Box::new(move || { - Box::pin(async move { - if offline { - return Err(DeferredMiss { - code: "offline", - detail: "--offline prevents fetching the pristine artifact from \ - the registry" - .to_string(), - }); - } - let inv = inventory - .get_or_init(|| lock_inventory::inventory_project(&cwd)) - .await; - match fetch_pristine_package(&cwd, inv, &client, &owned_purl, ledger_entry.as_ref()) - .await - { - PristineFetch::Fetched(fetched) => Ok(fetched), - PristineFetch::NoSource => Err(DeferredMiss { - code: "no_source", - detail: "no installed package found on disk".to_string(), - }), - PristineFetch::Unverifiable(detail) => Err(DeferredMiss { - code: "unverifiable", - detail, - }), - PristineFetch::Failed(detail) => Err(DeferredMiss { - code: "failed", - detail, - }), - } - }) - }), - ) -} - -/// The `vendor_fetched_missing` advisory for a pristine artifact fetched -/// because the package is not installed. -fn record_fetched_missing(env: &mut Envelope, common: &GlobalArgs, purl: &str, url: &str) { - record_warning( - env, - purl, - &VendorWarning::new( - "vendor_fetched_missing", - format!( - "{} is not installed; fetched the pristine artifact from {url} (integrity \ - verified) and vendored from that copy — the project tree was not touched", - normalize_purl(purl) - ), - ), - common, - ); } -/// Report a deferred fetch that produced no package exactly as the eager -/// fetch would have reported it for `purl`: a failed download is the same -/// `vendor_fetch_failed` failure (and suppresses the later -/// `package_not_installed` skip for the whole variant group), an -/// unverifiable lock entry the same `vendor_fetch_unverifiable` warning; -/// no source, and the `--offline` stop, say nothing here and leave the -/// candidates to the unmatched pass's `package_not_installed` skip. The -/// caller drops the backend's own outcome — the backend only failed -/// because the tree it asked for never arrived — and un-matches -/// `candidates`. -fn deferred_miss( - env: &mut Envelope, - common: &GlobalArgs, - purl: &str, - miss: &DeferredMiss, - candidates: &[String], - fetch_failed: &mut HashSet, -) { - match miss.code { - "unverifiable" => record_warning( - env, - purl, - &VendorWarning::new("vendor_fetch_unverifiable", miss.detail.clone()), - common, - ), - "no_source" | "offline" => {} - _ => { - fetch_failed.insert(purl.to_string()); - fetch_failed.extend(candidates.iter().cloned()); - env.record( - PatchEvent::new(PatchAction::Failed, purl.to_string()) - .with_error("vendor_fetch_failed", miss.detail.clone()), - ); - report_vendor_failure(common, purl, &format!("fetch failed: {}", miss.detail)); - } - } -} - -/// The patch-service downloads the vendor loop will make, in the loop's -/// order: `all_packages` walked as the loop walks it — release-variant -/// bases fanned out once to their manifest variants, each variant through -/// the same installed-variant probe — past the Bun refusal and the hosted -/// takeover gate, then through the record's backend gate (see -/// [`vendor::service_preflight`], and npm's one-read -/// [`vendor::npm_flavor::preflight_packages`] with the committed-artifact -/// reuse the npm backends answer from the ledger). Only reads: the probe -/// extracts a fetched artifact the loop's own probe would, and a variant -/// whose probe needs a download not made yet is left out — unplanned, the -/// loop simply fetches it live. Every doubt resolves to "not planned", -/// never to a grant the loop does not ask for. #[allow(clippy::too_many_arguments)] async fn plan_service_downloads( cwd: &Path, force: bool, all_packages: &[(String, StagedSource)], - (fetched_holders, deferred_holders): (&[registry_fetch::FetchedPackage], &[DeferredPackage]), variant_groups: &HashMap>, records: &HashMap, ledger: &VendorState, @@ -2219,11 +1815,7 @@ async fn plan_service_downloads( let mut reaching: Vec<(&str, &PatchRecord, &Path)> = Vec::new(); let mut handled_bases: HashSet = HashSet::new(); for (purl, staged) in all_packages { - let source = staged.as_source(fetched_holders, deferred_holders); - let deferred = match staged { - StagedSource::Deferred(at) => Some(&deferred_holders[*at]), - _ => None, - }; + let source = staged.as_source(); let is_variant_eco = Ecosystem::from_purl(purl).is_some_and(|e| e.supports_release_variants()); let candidates: Vec = if is_variant_eco { @@ -2245,18 +1837,18 @@ async fn plan_service_downloads( // The loop's installed-variant probe (see there). let probe_applicable = is_variant_eco && !matches!(Ecosystem::from_purl(candidate), Some(Ecosystem::Maven)); - let ledger_answers_probe = deferred.is_some_and(|d| d.outcome().is_none()) - && lookup_entry(&ledger.entries, candidate).is_some_and(|e| e.uuid == record.uuid); - if probe_applicable && !force && !ledger_answers_probe { + let ledger_answers_probe = + lookup_entry(&ledger.entries, candidate).is_some_and(|e| e.uuid == record.uuid); + if probe_applicable + && !force + && !ledger_answers_probe + && matches!(staged, StagedSource::Installed(_)) + { if let Some((file, info)) = representative_file(&record.files) { - if matches!(source, PackageSource::Deferred(_)) { - continue; - } - let Ok(dir) = source.materialize().await else { - continue; - }; - let status = verify_file_patch(dir, file, info).await.status; - if !variant_matches_installed(Some(&status)) { + let dir = source.path(); + if !variant_matches_installed(Some( + &verify_file_patch(dir, file, info).await.status, + )) { continue; } } @@ -2353,9 +1945,6 @@ pub(crate) async fn vendor_records( detached: bool, force: bool, env: &mut Envelope, - // Vendoring-service config (`None` = build-only). Both the `vendor` - // command and `scan --mode vendored` pass `Some(_)`, honoring - // `--vendor-source`. service: Option<&VendorServiceConfig>, ledger: std::io::Result, ) -> bool { @@ -2489,352 +2078,39 @@ pub(crate) async fn vendor_records_reusing( let vendored_installs = drop_vendored_installs_by(&common.cwd, &mut all_packages, |source| match source { StagedSource::Installed(path) => Some(path.as_path()), - StagedSource::Fetched(_) | StagedSource::Deferred(_) => None, + StagedSource::Missing(_) => None, }); - // ── Auto-fetch: lockfile-resolved packages with no installed copy ──── - // A manifest patch whose package is not on disk but IS resolvable from - // the project's lockfile is fetched pristine from its registry (lock- - // recorded URL else the conventional one), verified against the lock's - // integrity FAIL-CLOSED, and staged from a private tempdir — the - // project tree is never touched, and the lock wiring works without an - // installed copy (it keys off lock entries). The holders keep the - // tempdirs alive until the dispatch loop below has staged from them. - let mut fetched_holders: Vec = Vec::new(); - // Sources whose download is deferred to the backend branch that reads - // them (see the plan below), held by index like `fetched_holders`. - let mut deferred_holders: Vec = Vec::new(); - // Fetch failures must keep their distinct Failed event; this set - // suppresses the later duplicate `package_not_installed` skip. - let mut fetch_failed: HashSet = HashSet::new(); - // The lockfile inventory (every recognized lockfile parsed) — a local - // read, fine offline — built lazily at the first site that consumes it - // and shared by the registry-fetch rung below, the deferred fetches and - // the `--offline` "the lockfile resolves it" detail at the end, so a run - // parses the lockfiles at most once (and not at all when every missing - // purl stages from its committed artifact, every deferred source is - // answered by its backend's hot path, or nothing is missing). let inventory: Arc>> = Arc::new(tokio::sync::OnceCell::new()); - { - let missing: Vec = vendorable - .iter() - .filter(|p| !all_packages.contains_key(*p)) - .cloned() - .collect(); - if !missing.is_empty() { - let client = registry_fetch::build_registry_client(); - // Two passes over `missing`, so the registry fetches can run - // concurrently while every event, warning and stderr line still - // lands in `missing` order. Pass 1 decides each purl's local - // rungs (local and read-only) without emitting anything; the - // purls left for the registry are then fetched at most - // `registry_concurrency` at a time, in order, and pass 2 emits - // every purl's outcome in turn. - let mut rungs: Vec<(Option, MissingRung)> = { - let mut rungs = Vec::with_capacity(missing.len()); - for purl in &missing { - rungs.push( - missing_local_rung( - common, - lookup_entry(&state.entries, purl), - records.get(purl), - ) - .await, - ); - } - rungs + let mut fetch_failed: HashSet = HashSet::new(); + let references = + crate::commands::vendored_backend::repair::scan_vendor_references(&common.cwd).await; + for purl in &vendorable { + let record = &records[purl]; + if lookup_entry(&state.entries, purl).is_none_or(|entry| entry.uuid != record.uuid) + && references.iter().any(|(eco, uuid, _)| { + uuid == &record.uuid + && (vendor::ecosystem_dir_for_purl(purl) == Some(eco.as_str()) + || (eco == "maven2" && purl.starts_with("pkg:maven/"))) + }) + { + let detail = match vendored_installs.get(purl) { + Some(dir) => format!("installed from the vendored artifact {dir}, but the vendor ledger has no entry for it; restore .socket/vendor/state.json from version control"), + None => "the lockfile references this patch without its vendor ledger entry; restore .socket/vendor/state.json from version control".to_string(), }; - // Downloads nothing may need, deferred to the backend branch - // that reads the pristine tree (see `DeferredPackage`): - // - // * a purl the ledger already covers (see `ledger_covers`): the - // backend's in-sync hot path answers it from the committed - // bytes alone, so a re-run needs no network. `--force` may - // rebuild anyway, so it keeps the eager fetch. - // * a package the patch service can serve, in an ecosystem whose - // backend reads the pristine tree only if it falls back to the - // local build (`backend_reads_pristine_only_on_fallback`). - // Only one the registry ladder would really download (see - // `pristine_fetch_reaches_download`) — a git, path or - // custom-registry crate, say, keeps the eager rung, whose - // `vendor_fetch_unverifiable` refusal keeps a crates.io patch - // off it. - // - // A backend that does reach its pristine tree fetches it then, - // through the same ladder, and the loop reports the fetch as the - // eager one would have (see `deferred_miss`). - let service_enabled = service.is_some_and(VendorServiceConfig::service_enabled); - for (purl, (_, rung)) in missing.iter().zip(rungs.iter_mut()) { - if !matches!(rung, MissingRung::Fetch | MissingRung::Offline) { - continue; - } - let covered = !force - && match records.get(purl) { - Some(record) => { - ledger_covers(&common.cwd, lookup_entry(&state.entries, purl), record) - .await - } - None => false, - }; - let via_service = service_enabled - && matches!(rung, MissingRung::Fetch) - && backend_reads_pristine_only_on_fallback(purl) - && pristine_fetch_reaches_download( - &common.cwd, - inventory - .get_or_init(|| lock_inventory::inventory_project(&common.cwd)) - .await, - purl, - lookup_entry(&state.entries, purl), - ) - .await; - if covered || via_service { - *rung = MissingRung::Deferred; - } - } - // A missing npm or cargo purl its backend refuses on the - // project's lock text alone (see `vendor::lock_text_refusals`: - // the pnpm / yarn classic / yarn berry gates, cargo's locked - // version) is deferred rather than fetched: the backend refuses - // it — at its turn, in its own words — before anything reads - // the source, so the refusal costs no registry request. A purl - // the lockfiles pin hosted keeps the eager fetch: its takeover - // restores the upstream lock entry first, which rewrites the - // text the gates read. - let lock_candidates: Vec<(&str, &str)> = missing - .iter() - .zip(&rungs) - .filter(|(_, (_, rung))| matches!(rung, MissingRung::Fetch)) - .filter_map(|(purl, _)| { - records - .get(purl) - .map(|record| (purl.as_str(), record.uuid.as_str())) - }) - .filter(|(purl, _)| { - matches!( - vendor::ecosystem_dir_for_purl(purl), - Some("npm") | Some("cargo") - ) - }) - .collect(); - if !lock_candidates.is_empty() { - let claimed: Option> = Some( - socket_patch_core::patch::redirect::upstream::HostedPin::all( - &crate::commands::discover_wiring(common, &common.cwd).await, - ) - .into_iter() - .map(|pin| canonical_purl(&pin.purl)) - .collect(), - ); - if let Some(claimed) = claimed { - let unclaimed: Vec<(&str, &str)> = lock_candidates - .into_iter() - .filter(|(purl, _)| !claimed.contains(&canonical_purl(purl))) - .collect(); - // Only a purl the ladder would really fetch (a - // verifiable registry resolution): one with no source - // at all keeps the loop's `package_not_installed` skip. - // These purls have no installed copy, so none is - // looked for. - let refused = lock_refusals_reaching_backend( - &common.cwd, - vendor::lock_text_refusals(&common.cwd, &unclaimed).await, - &state.entries, - |_| async { HashSet::new() }, - ) - .await; - for (purl, (_, rung)) in missing.iter().zip(rungs.iter_mut()) { - if matches!(rung, MissingRung::Fetch) && refused.contains_key(purl) { - *rung = MissingRung::Deferred; - } - } - } - } - // A NOT-INSTALLED gem can only be vendored through the patch - // service: the bundler path source needs the eval-able stub - // gemspec rubygems writes at INSTALL time, which a fetched `.gem` - // lacks (the service serves a converted `gem-stub-gemspec`). - // With the service off, refuse `gem_spec_missing` before the - // download rather than after it; the backend keeps its own - // refusal as the backstop. - // - // Scoped to the purls a DOWNLOAD would actually happen for, - // mirroring `fetch_pristine_package`'s `fetchable` filter: a gem - // the lock cannot VERIFY, one the ledger already holds, or one - // that resolves from nowhere keeps its existing outcome. A dry - // run keeps the eager fetch: its verify-only preview runs on the - // fetched copy. - if !service_enabled - && !common.dry_run - && missing - .iter() - .zip(&rungs) - .any(|(p, (_, r))| p.starts_with("pkg:gem/") && r.needs_registry()) - { - let inv = inventory - .get_or_init(|| lock_inventory::inventory_project(&common.cwd)) - .await; - for (purl, (_, rung)) in missing.iter().zip(rungs.iter_mut()) { - if purl.starts_with("pkg:gem/") - && rung.needs_registry() - && lookup_entry(&state.entries, purl).is_none() - && lock_inventory::lookup(inv, purl) - .is_some_and(|e| e.integrity != lock_inventory::LockIntegrity::None) - { - *rung = MissingRung::GemBuildRefused; - } - } - } - // Parsed only when some purl reaches the registry rung. - let inv: &[lock_inventory::LockfileEntry] = - if rungs.iter().any(|(_, r)| r.needs_registry()) { - inventory - .get_or_init(|| lock_inventory::inventory_project(&common.cwd)) - .await - } else { - &[] - }; - let (cwd, client_ref, ledger) = (&common.cwd, &client, &state.entries); - let to_fetch: Vec<&String> = missing - .iter() - .zip(&rungs) - .filter(|(_, (_, rung))| rung.needs_registry()) - .map(|(purl, _)| purl) - .collect(); - let mut pristine = std::pin::pin!(ordered_concurrent( - to_fetch, - registry_concurrency(), - |purl| fetch_pristine_package( - cwd, - inv, - client_ref, - purl, - lookup_entry(ledger, purl) - ), - )); - for (purl, (artifact_missing, rung)) in missing.iter().zip(rungs) { - if let Some(artifact) = artifact_missing { - // The committed artifact is GONE (gitignored or - // deleted): not corruption — fall through to the - // registry ladder, which recovers the pre-vendor - // resolution from the ledger and rebuilds. - record_warning( - env, - purl, - &VendorWarning::new( - "vendor_artifact_missing", - format!( - "the committed vendored artifact {artifact} is missing; \ - recovering the registry resolution to rebuild it" - ), - ), - common, - ); - } - let fetched = match rung { - MissingRung::Staged(staged) => { - all_packages - .insert(purl.clone(), StagedSource::Fetched(fetched_holders.len())); - fetched_holders.push(staged); - continue; - } - MissingRung::StageFailed(detail) => { - // A PRESENT-but-corrupt committed artifact is - // worth a loud failure — silently re-vendoring - // over it would mask the corruption. - fetch_failed.insert(purl.clone()); - let detail = format!( - "{detail}; run `socket-patch repair` to rebuild the \ - vendored artifact" - ); - env.record( - PatchEvent::new(PatchAction::Failed, purl.clone()) - .with_error("vendor_fetch_failed", detail.clone()), - ); - report_vendor_failure(common, purl, &detail); - continue; - } - MissingRung::Deferred => { - all_packages - .insert(purl.clone(), StagedSource::Deferred(deferred_holders.len())); - deferred_holders.push(deferred_pristine_package( - common, - &inventory, - &client, - purl, - lookup_entry(&state.entries, purl), - )); - continue; - } - MissingRung::GemBuildRefused => { - fetch_failed.insert(purl.clone()); - // The backend's own refusal text, word for word. - let detail = format!( - "no local stub gemspec for {} (a path source cannot be wired \ - without one); install the gem or use --vendor-source=service", - strip_purl_qualifiers(purl).trim_start_matches("pkg:gem/") - ); - env.record( - PatchEvent::new(PatchAction::Failed, purl.clone()) - .with_error("gem_spec_missing", detail.clone()), - ); - report_vendor_failure(common, purl, &detail); - continue; - } - // The enriched skip detail lands below in the unmatched - // pass (the purl stays unmatched). - MissingRung::Offline => continue, - MissingRung::Fetch => match pristine.next().await { - Some(fetched) => fetched, - // Unreachable: `to_fetch` holds one fetch per - // `needs_registry` rung, and this is the only arm - // that consumes one. A live fetch keeps the outcome - // right if the two ever fall out of step. - None => { - debug_assert!(false, "pristine prefetch plan out of step at {purl}"); - fetch_pristine_package( - cwd, - inv, - client_ref, - purl, - lookup_entry(ledger, purl), - ) - .await - } - }, - }; - match fetched { - PristineFetch::Fetched(fetched) => { - record_fetched_missing(env, common, purl, &fetched.url); - all_packages - .insert(purl.clone(), StagedSource::Fetched(fetched_holders.len())); - fetched_holders.push(fetched); - } - PristineFetch::NoSource => { - // Plain not-installed package → the calm - // package_not_installed skip below. - } - PristineFetch::Unverifiable(detail) => { - record_warning( - env, - purl, - &VendorWarning::new("vendor_fetch_unverifiable", detail), - common, - ); - // Falls through to package_not_installed below. - } - PristineFetch::Failed(detail) => { - fetch_failed.insert(purl.clone()); - env.record( - PatchEvent::new(PatchAction::Failed, purl.clone()) - .with_error("vendor_fetch_failed", detail.clone()), - ); - report_vendor_failure(common, purl, &format!("fetch failed: {detail}")); - } - } - } + env.record( + PatchEvent::new(PatchAction::Failed, purl.clone()) + .with_error("vendor_ledger_entry_missing", detail.clone()), + ); + report_vendor_failure(common, purl, &detail); + fetch_failed.insert(purl.clone()); + all_packages.remove(purl); + continue; } + all_packages.entry(purl.clone()).or_insert_with(|| { + StagedSource::Missing(common.cwd.join(".socket/vendor/.uninstalled")) + }); } let vendored_at = now_rfc3339(); @@ -2928,7 +2204,6 @@ pub(crate) async fn vendor_records_reusing( &common.cwd, force, &all_packages, - (&fetched_holders, &deferred_holders), &variant_groups, records, &state, @@ -2941,14 +2216,6 @@ pub(crate) async fn vendor_records_reusing( } None => None, }; - // The source of the purl the loop has just left. Its archive is what a - // fetched source holds to be able to write its tree, and `all_packages` - // gives each holder to exactly one purl — so once the loop moves on, - // nothing reads it again, and a run that fetched 110 artifacts need not - // carry all 110 to the end of the loop. - let mut spent: Option> = None; - // Deferred sources whose fetch the loop has already reported. - let mut deferred_fetch_reported: HashSet = HashSet::new(); // Group commit: from here until the loop ends, every backend's // lockfile / manifest / config edits, the takeover's hosted reverts and // the per-package ledger saves are captured in memory — every read in @@ -2964,15 +2231,7 @@ pub(crate) async fn vendor_records_reusing( .then(|| GroupCommit::begin(&common.cwd)); let mut stale_artifacts: Vec = Vec::new(); for (index, (purl, staged)) in all_packages.iter().enumerate() { - if let Some(done) = spent.take() { - done.release(); - } - let pkg_source = staged.as_source(&fetched_holders, &deferred_holders); - let deferred = match staged { - StagedSource::Deferred(at) => Some(&deferred_holders[*at]), - _ => None, - }; - spent = Some(pkg_source); + let pkg_source = staged.as_source(); let is_variant_eco = Ecosystem::from_purl(purl).is_some_and(|e| e.supports_release_variants()); let candidates: Vec = if is_variant_eco { @@ -3005,50 +2264,23 @@ pub(crate) async fn vendor_records_reusing( // purl at this record's uuid: the variant the ledger vendored is // the installed one's by construction, so it answers the probe // without downloading the pristine tree just to read one file. - let ledger_answers_probe = deferred.is_some_and(|d| d.outcome().is_none()) - && lookup_entry(&state.entries, candidate).is_some_and(|e| e.uuid == record.uuid); + let ledger_answers_probe = + lookup_entry(&state.entries, candidate).is_some_and(|e| e.uuid == record.uuid); if probe_applicable && !force && !ledger_answers_probe { - // The representative must be a file that MODIFIES existing - // content: a new file (empty beforeHash) verifies `Ready` - // against any environment, so it can neither identify nor - // disqualify a variant. Same deterministic pick as apply / - // core's `select_installed_variants`. - let first = match representative_file(&record.files) { - Some((f, info)) => match pkg_source.materialize().await { - Ok(dir) => Some(verify_file_patch(dir, f, info).await.status), - // A deferred download that produced nothing is - // reported as the eager fetch would have reported it. - Err(_) if deferred.is_some_and(|d| matches!(d.outcome(), Some(Err(_)))) => { - if let Some(Some(Err(miss))) = deferred.map(DeferredPackage::outcome) { - deferred_miss( - env, - common, - purl, - miss, - &candidates, - &mut fetch_failed, - ); - } - break; - } - // Not a variant verdict: the tree could not be - // WRITTEN at all (full `$TMPDIR`, no fds). Report one - // failure for the SOURCE purl rather than filing it - // under `package_not_installed` and losing the cause. - Err(detail) => { - env.record( - PatchEvent::new(PatchAction::Failed, purl.clone()) - .with_error("vendor_fetch_failed", detail.clone()), - ); - report_vendor_failure(common, purl, &format!("fetch failed: {detail}")); - fetch_failed.insert(purl.clone()); - fetch_failed.extend(candidates.iter().cloned()); - break; + if matches!(staged, StagedSource::Installed(_)) { + if let Some((file, info)) = representative_file(&record.files) { + if !variant_matches_installed(Some( + &verify_file_patch(pkg_source.path(), file, info) + .await + .status, + )) { + continue; } - }, - None => None, - }; - if !variant_matches_installed(first.as_ref()) { + } + } else if candidates.len() > 1 { + env.record(PatchEvent::new(PatchAction::Failed, candidate.clone()).with_error( + "vendor_variant_ambiguous", "multiple patch variants match an uninstalled package; select one release variant")); + fetch_failed.insert(candidate.clone()); continue; } } @@ -3213,6 +2445,48 @@ pub(crate) async fn vendor_records_reusing( } } + if let Some(entry) = + lookup_entry(&state.entries, candidate).filter(|entry| entry.uuid == record.uuid) + { + if vendor::check_vendored_artifact(&common.cwd, entry, record).await + != vendor::ArtifactHealth::Healthy + || (entry.artifact.sha256.is_empty() + && entry.artifact.file_inventory.is_none() + && vendor::artifact_is_file_shaped(&entry.artifact.path)) + { + if common.dry_run { + env.record(PatchEvent::new(PatchAction::Verified, candidate.clone()).with_details(serde_json::json!({"wouldRedownload": true, "path": entry.artifact.path}))); + continue; + } + let restored = match service { + Some(service) => { + vendor::redownload::restore(&common.cwd, entry, record, service).await + } + None => Err( + "vendoring requires a prebuilt artifact from the patch service" + .to_string(), + ), + }; + match restored { + Ok(warnings) => { + for warning in &warnings { + record_warning(env, candidate, warning, common); + } + env.record(PatchEvent::new(PatchAction::Rebuilt, candidate.clone()).with_details(serde_json::json!({"redownloaded": true, "path": entry.artifact.path}))); + } + Err(detail) => { + has_errors = true; + env.record( + PatchEvent::new(PatchAction::Failed, candidate.clone()) + .with_error("vendor_redownload_failed", detail.clone()), + ); + report_vendor_failure(common, candidate, &detail); + continue; + } + } + } + } + status.set(format_vendor_progress( common.dry_run, &normalize_purl(candidate), @@ -3237,27 +2511,6 @@ pub(crate) async fn vendor_records_reusing( let vendored = matches!(&outcome, Some(VendorOutcome::Done { result, .. }) if result.success); - // A deferred source whose backend needed the pristine tree after - // all fetched it inside the call. Report that fetch as the eager - // ladder did — ahead of this package's own outcome — once per - // source; a fetch that produced nothing replaces the outcome. - if let Some(fetch) = deferred.and_then(DeferredPackage::outcome) { - match fetch { - Ok(fetched) => { - if deferred_fetch_reported.insert(purl.clone()) { - record_fetched_missing(env, common, purl, &fetched.url); - } - } - Err(miss) => { - deferred_miss(env, common, purl, miss, &candidates, &mut fetch_failed); - for c in &candidates { - matched.remove(c); - } - break; - } - } - } - match outcome { None => { env.record( @@ -3431,14 +2684,6 @@ pub(crate) async fn vendor_records_reusing( drop(service_prefetch); vendor::prestage::settle().await; - // Every backend has staged what it needed, so the fetch tempdirs can - // go. Dropping them removes whatever was extracted into them — a - // recursive delete that belongs off the runtime thread. - if !fetched_holders.is_empty() || !deferred_holders.is_empty() { - let _ = - tokio::task::spawn_blocking(move || drop((fetched_holders, deferred_holders))).await; - } - // The run's one commit of every lockfile, manifest, config and ledger // the loop changed. The packages that succeeded are committed even when // others failed — a failed package's backend already put back what it @@ -4359,7 +3604,6 @@ mod plan_gate_tests { root, false, &all_packages, - (&[], &[]), &HashMap::new(), &records, &VendorState::default(), @@ -4698,6 +3942,7 @@ mod gc_tests { base_purl: PURL.into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz"), sha256: String::new(), size: None, @@ -5674,6 +4919,7 @@ mod revert_dispatch_tests { base_purl: base_purl.into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/{eco}/{UUID}/artifact"), sha256: String::new(), size: None, @@ -5774,6 +5020,7 @@ mod persist_tests { base_purl: base_purl.into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{uuid}/pkg.tgz"), sha256: String::new(), size: None, @@ -5947,26 +5194,6 @@ mod persist_tests { } } -#[cfg(test)] -mod pristine_fetch_tests { - use super::*; - - /// No lockfile entry AND no ledger entry: the pristine-source ladder - /// reports `NoSource` (the calm `package_not_installed` path) BEFORE any - /// network I/O — nothing else can name a verifiable source. - #[tokio::test] - async fn no_lock_and_no_ledger_is_no_source() { - let tmp = tempfile::tempdir().unwrap(); - let client = registry_fetch::build_registry_client(); - let out = - fetch_pristine_package(tmp.path(), &[], &client, "pkg:npm/left-pad@1.3.0", None).await; - assert!( - matches!(out, PristineFetch::NoSource), - "expected NoSource for a purl with no lock and no ledger entry" - ); - } -} - /// Exact-string tests for the human output of `vendor` / `vendor --revert`. #[cfg(test)] mod ui_format_tests { diff --git a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs index 65a7dd9b2..014b011c3 100644 --- a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs +++ b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs @@ -1,23 +1,11 @@ //! The one vendored-mode backend: [`VendoredBackend`] with its three //! operations, shared by every command that vendors, un-vendors or repairs. //! -//! * [`VendoredBackend::apply`] — stage the patch content in memory, run -//! the vendor engine ([`vendor_records_reusing`]), persist each ledger -//! entry. `vendor`, `scan --mode vendored` (JSON and interactive arms) -//! and `get --mode vendored` are its callers; they differ only in where -//! the patch records come from (the manifest, or the download phase's -//! in-memory records) and in their output shape. -//! * [`VendoredBackend::revert`] — revert ledger entries through the -//! per-ecosystem backends (drift-keep, `--preserve-state` and dry-run -//! classification in one place). `vendor --revert`, the manifest -//! reconcile, `rollback`'s vendored leg and both of `remove`'s paths map -//! its [`VendorRevertStep`]s onto their own event vocabulary. -//! * [`VendoredBackend::repair`] — health-check the ledger and re-vendor -//! missing or corrupt artifacts through `apply`, so a repair downloads -//! the patch service's prebuilt artifact exactly like the original -//! `vendor` did (local build as the `--vendor-source auto` fallback). -//! Lockfile references with no ledger entry are reported, never -//! re-synthesized (see [`repair`]). +//! * [`VendoredBackend::apply`] downloads verified server artifacts and +//! persists their wiring and ledger entries through `vendor_records_reusing`. +//! * [`VendoredBackend::revert`] restores the recorded project wiring. +//! * [`VendoredBackend::repair`] redownloads missing or corrupt artifacts, +//! preserving their recorded identities, lockfiles and ledger. //! //! The backends themselves (`dispatch_vendor_one`, `dispatch_revert_one*`) //! stay in `vendor.rs`; this module is the policy layer over them. @@ -33,15 +21,12 @@ use socket_patch_core::vendor::{ }; use crate::args::GlobalArgs; -use crate::commands::fetch_stage::{ - drop_unstageable, stage_vendor_sources_in_memory, MemStageOutcome, -}; use crate::commands::vendor::{dispatch_revert_one_opts, vendor_records_reusing}; use crate::ecosystem_dispatch::NpmCrawlSnapshot; use crate::json_envelope::Envelope; /// The vendored-mode backend for one run: the run's global args and its -/// patch-service config (`None` = build-only; `--revert` never needs one). +/// patch-service config (`--revert` does not need one). pub(crate) struct VendoredBackend<'a> { pub(crate) common: &'a GlobalArgs, pub(crate) service: Option<&'a VendorServiceConfig>, @@ -49,19 +34,11 @@ pub(crate) struct VendoredBackend<'a> { /// What [`VendoredBackend::apply`] vendors. pub(crate) struct ApplyRequest<'a> { - /// The patch records to vendor, keyed by (manifest-spelled) purl. A - /// manifest VIEW: staging probes blobs by the records' hashes. + /// Patch records keyed by manifest purl. pub(crate) manifest: &'a PatchManifest, - /// The `.socket/` dir whose committed blobs/diffs/packages staging - /// reads in place. pub(crate) socket_dir: &'a Path, - /// The vendor ledger, loaded ONCE by the caller under its apply lock: - /// the staging harvest reads it, then the engine takes it over for its - /// persists. An unreadable one is the engine's loud report. + /// Loaded once by the caller under the apply lock. pub(crate) ledger: std::io::Result, - /// Blob content the caller already holds (the download phase's), so - /// the stager fetches no view twice. - pub(crate) seed: HashMap>, /// `true` for manifest-free vendoring (`scan`/`get --mode vendored`, /// and repair of an entry with no manifest owner). pub(crate) detached: bool, @@ -72,16 +49,6 @@ pub(crate) struct ApplyRequest<'a> { pub(crate) prior: Option<&'a NpmCrawlSnapshot>, } -/// Staging obtained no patch content at all (offline, or every view fetch -/// failed): nothing reached the engine. Callers report it as -/// `no_local_source` in their own output shape. -#[derive(Debug)] -pub(crate) struct NoLocalSource; - -/// The contract message of [`NoLocalSource`]. -pub(crate) const NO_LOCAL_SOURCE_MESSAGE: &str = - "patch artifacts unavailable (offline or download failure)"; - impl<'a> VendoredBackend<'a> { pub(crate) fn new(common: &'a GlobalArgs, service: Option<&'a VendorServiceConfig>) -> Self { Self { common, service } @@ -89,41 +56,24 @@ impl<'a> VendoredBackend<'a> { /// Vendor `req.manifest`'s records. The caller holds the apply lock. /// - /// Patch content is staged IN MEMORY (committed `.socket` artifacts read - /// in place, the rest fetched per patch over the service config's API - /// client) — vendoring never writes blobs. A record whose content could - /// not be obtained is reported per package (`no_local_source`) and left - /// out; the rest still vendors. `Ok(has_errors)` otherwise. + /// Each package fails closed if its server artifact is unavailable. + /// Returns whether any package failed. /// /// The engine future is boxed here — this is its transient frame, so no /// caller's poll frame embeds it (Windows' 1 MiB main-thread stack; see /// `scan_run_fits_windows_main_thread_stack`). - pub(crate) async fn apply( - &self, - req: ApplyRequest<'_>, - env: &mut Envelope, - ) -> Result { + pub(crate) async fn apply(&self, req: ApplyRequest<'_>, env: &mut Envelope) -> bool { let common = self.common; - let staged = match stage_vendor_sources_in_memory( - common, - req.manifest, - req.socket_dir, - &common.cwd, - req.ledger.as_ref().map(|s| &s.entries), - req.seed, - self.service.and_then(|s| s.client.as_ref()), - ) - .await - { - MemStageOutcome::Ready(s) => s, - MemStageOutcome::Unavailable => return Err(NoLocalSource), + let blobs = req.socket_dir.join("blobs"); + let sources = socket_patch_core::patch::apply::PatchSources { + blobs_path: &blobs, + diffs_path: None, + mem_blobs: None, }; - let sources = staged.as_patch_sources(); - let (records, staging_errors) = - drop_unstageable(env, &req.manifest.patches, staged.unavailable()); - let engine_errors = Box::pin(vendor_records_reusing( + let records = &req.manifest.patches; + Box::pin(vendor_records_reusing( common, - &records, + records, &sources, req.detached, req.force, @@ -132,8 +82,7 @@ impl<'a> VendoredBackend<'a> { req.ledger, req.prior, )) - .await; - Ok(staging_errors || engine_errors) + .await } /// Revert the ledger entries `keys` (in order) with `opts`, mutating diff --git a/crates/socket-patch-cli/src/commands/vendored_backend/repair.rs b/crates/socket-patch-cli/src/commands/vendored_backend/repair.rs index a12e8f8d9..20de36386 100644 --- a/crates/socket-patch-cli/src/commands/vendored_backend/repair.rs +++ b/crates/socket-patch-cli/src/commands/vendored_backend/repair.rs @@ -1,50 +1,24 @@ -//! [`VendoredBackend::repair`] — `repair`'s vendored-artifact phase: -//! re-vendor committed vendor artifacts that the ledger records but that -//! are missing or corrupt on disk. -//! -//! Detection is the core health check ([`check_vendored_artifact`]: per-file -//! afterHashes + the whole-file ledger sha256 for file-shaped artifacts, the -//! whole-tree inventory for dir-shaped ones). A broken artifact is -//! re-vendored through [`VendoredBackend::apply`] — the same engine, the -//! same `--vendor-source` policy and the same pristine-source ladder as -//! `vendor` itself — so under the default `auto` source the patch -//! service's prebuilt artifact is downloaded again (a local build is the -//! fallback when the service has none, and the only source under -//! `--offline`/`--vendor-source build`). The backends' wired hot paths -//! rebuild the ARTIFACT only; lockfiles and the recorded pre-vendor -//! originals are left alone. A rebuild is verified against its ledger entry -//! afterwards, fail-closed. -//! -//! The ledger is the only source of truth: a lockfile that references -//! `.socket/vendor///` with NO ledger entry (state.json deleted -//! or never committed) is reported as `vendor_ledger_missing` — repair no -//! longer re-synthesizes ledger entries from lockfile text, because the -//! pre-vendor originals a revert needs cannot be recovered from the -//! rewired lockfile. The remedy is restoring `.socket/vendor/state.json` -//! from version control and re-running repair. +//! Repair downloads the recorded server artifact into a temporary location, +//! checks its archive hash or file inventory, and replaces the damaged copy. +//! Wiring and ledger identities remain unchanged. Missing ledger entries +//! must be restored from version control so rollback originals are preserved. -use std::collections::{HashMap, HashSet}; -use std::path::{Path, PathBuf}; +use std::collections::HashSet; +use std::path::Path; use socket_patch_core::api::client::{get_api_client_with_overrides, ApiClient}; -use socket_patch_core::constants::SOCKET_DIR; use socket_patch_core::formats::registry; use socket_patch_core::manifest::schema::{PatchManifest, PatchRecord}; -use socket_patch_core::patch::copy_tree::remove_tree; -use socket_patch_core::utils::fs::{ - atomic_write_bytes_preserving_mode, read_regular_to_bytes, read_regular_to_string, -}; +use socket_patch_core::utils::fs::read_regular_to_string; use socket_patch_core::utils::purl::normalize_purl; use socket_patch_core::vendor::{ - self, artifact_is_file_shaped, check_vendored_artifact, load_state, parse_vendor_path, - ArtifactHealth, VendorEntry, VendorState, VendorWarning, + self, artifact_is_file_shaped, check_vendored_artifact, parse_vendor_path, ArtifactHealth, + VendorEntry, VendorState, VendorWarning, }; -use super::{records_manifest, ApplyRequest, VendoredBackend, NO_LOCAL_SOURCE_MESSAGE}; +use super::VendoredBackend; use crate::args::GlobalArgs; -use crate::commands::vendor::{ - ecosystem_in_scope, format_advisory, persist_vendor_entry, record_warning, -}; +use crate::commands::vendor::{ecosystem_in_scope, persist_vendor_entry, record_warning}; use crate::json_envelope::{Envelope, PatchAction, PatchEvent}; use crate::ui::plural; @@ -172,12 +146,12 @@ fn format_repair_failure(purl: &str, detail: &str) -> String { ) } -/// The `repair --dry-run` preview of vendored rebuilds: a heading, then +/// The `repair --dry-run` preview of vendored redownloads: a heading, then /// ` - (: )` per artifact. `items` are /// `(purl, reason code, artifact path)`. -fn format_rebuild_preview(items: &[(String, &str, &str)]) -> Vec { +fn format_redownload_preview(items: &[(String, &str, &str)]) -> Vec { let mut lines = vec![format!( - "Would rebuild {}:", + "Would redownload {}:", plural(items.len(), "vendored artifact", "vendored artifacts") )]; lines.extend(items.iter().map(|(purl, reason, path)| { @@ -207,59 +181,7 @@ fn format_ledger_missing(eco: &str, uuid: &str) -> String { ) } -/// Best-effort removal of a vendored uuid dir after a failed post-verify -/// (never leave unverifiable bytes behind). Prunes the emptied -/// `.socket/vendor//` (and `vendor/`) husks like every other artifact -/// removal, stopping at `.socket/`; a sibling unit or the ledger keeps them. -async fn remove_vendor_dir(cwd: &Path, eco: &str, uuid: &str) { - if let Some(rel) = vendor::path::vendor_uuid_dir_rel(eco, uuid) { - let _ = socket_patch_core::utils::socket_dir::remove_tree_and_prune( - &cwd.join(rel), - &cwd.join(SOCKET_DIR), - ) - .await; - } -} - -/// Move the live uuid dir aside (same parent, `.pre-rebuild`) so the -/// engine's rebuild-on-MISSING trigger fires while the bytes stay -/// recoverable: the re-vendor can still refuse or fail, and a failed one -/// replaced nothing, so the corrupt-but-diagnosable artifact must be -/// restorable instead of leaving the wired lockfiles pointing at a bare -/// ENOENT (and the tamper evidence erased). Returns `(live, kept)` for -/// [`restore_aside_vendor_dir`]; on a rename failure falls back to plain -/// removal (the rebuild trigger must fire) and returns `None`. -async fn set_aside_vendor_dir(cwd: &Path, eco: &str, uuid: &str) -> Option<(PathBuf, PathBuf)> { - let rel = vendor::path::vendor_uuid_dir_rel(eco, uuid)?; - let live = cwd.join(&rel); - let kept = cwd.join(format!("{rel}.pre-rebuild")); - // A crashed earlier run's leftover must not wedge the rename. - let _ = remove_tree(&kept).await; - if tokio::fs::rename(&live, &kept).await.is_ok() { - Some((live, kept)) - } else { - let _ = remove_tree(&live).await; - None - } -} - -/// Put the pre-rebuild bytes back after a re-vendor that produced no -/// replacement (clearing any partial husk the failed backend left first). -async fn restore_aside_vendor_dir(live: &Path, kept: &Path) { - let _ = remove_tree(live).await; - let _ = tokio::fs::rename(kept, live).await; -} - -/// Crash recovery for [`set_aside_vendor_dir`]'s transient: a run killed -/// between the move-aside and the replacement leaves -/// `.socket/vendor//.pre-rebuild` as the ONLY copy of bytes the -/// rewired lockfiles still point at, with the live path a bare ENOENT. Put -/// every such leftover back where the wiring expects it before the health -/// pass classifies the unit. A leftover whose live sibling EXISTS is left -/// alone: the live dir may be the completed replacement or a partial husk, -/// and only the health pass can tell — a unit it condemns is set aside -/// again, which clears the leftover. Wet runs only; scope-gated like every -/// other unit; best-effort throughout. +/// Recover artifacts left aside by older releases when a repair was interrupted. async fn restore_orphaned_pre_rebuild_dirs(common: &GlobalArgs) { const SUFFIX: &str = ".pre-rebuild"; let vendor_root = common.cwd.join(".socket/vendor"); @@ -292,7 +214,6 @@ pub(crate) struct RepairRequest<'a> { /// `None` when the project has no `.socket/manifest.json` (vendored /// mode is manifest-free). pub(crate) manifest: Option<&'a PatchManifest>, - pub(crate) socket_dir: &'a Path, /// [`scan_vendor_references`]'s output for `common.cwd`, taken by the /// caller under the apply lock this phase runs under. pub(crate) references: &'a [(String, String, String)], @@ -415,6 +336,19 @@ impl VendoredBackend<'_> { ); continue; } + if socket_patch_core::utils::purl::canonical_purl(purl) + != socket_patch_core::utils::purl::canonical_purl(&entry.base_purl) + || !vendor::is_vendorable(&entry.base_purl) + { + fail( + env, + common.json, + purl, + "vendor_artifact_unrepairable", + "the ledger package identity is invalid or unsupported".into(), + ); + continue; + } // Pre-v5 cargo wiring in `.cargo/config*`: move it into the // root Cargo.toml (the v5 location) and record the move in the // ledger — or restore the manifest entry a pre-v5 @@ -666,7 +600,7 @@ impl VendoredBackend<'_> { }) .collect(); println!(); - for line in format_rebuild_preview(&items) { + for line in format_redownload_preview(&items) { println!("{line}"); } } @@ -675,7 +609,7 @@ impl VendoredBackend<'_> { PatchEvent::new(PatchAction::Verified, c.purl.clone()).with_details( serde_json::json!({ "vendorArtifact": true, - "wouldRebuild": true, + "wouldRedownload": true, "reason": c.reason, "path": c.entry.artifact.path, }), @@ -688,7 +622,7 @@ impl VendoredBackend<'_> { if !quiet { println!(); println!( - "Rebuilding {}...", + "Redownloading {}...", plural( candidates.len(), "broken vendored artifact", @@ -697,37 +631,6 @@ impl VendoredBackend<'_> { ); } - // ── Re-vendor through the shared apply engine ──────────────────── - // Repair restores the RECORDED artifact; it never re-vendors. The - // engine runs with the lockfiles and ledger snapshotted and put back - // afterwards, and every candidate is verified against its original - // ledger entry, so a source that produces different bytes (a service - // archive re-gzipped since vendoring) can never be committed. - // - // The afterHash-verified members of a corrupt artifact are harvested - // first: they are patch content staging can use offline, and the - // move-aside below takes the artifact out of the path staging reads. - let candidate_records: HashMap = candidates - .iter() - .map(|c| (c.purl.clone(), c.record.clone())) - .collect(); - let seed = - vendor::harvest_artifact_blobs_from(&common.cwd, &state.entries, &candidate_records) - .await; - let snapshot = snapshot_wiring(&common.cwd, &candidates).await; - // A corrupt artifact is moved aside so the engine's - // rebuild-on-missing path fires; it goes back if nothing replaced - // it. A missing one has nothing to keep. - let mut aside: HashMap = HashMap::new(); - for c in &candidates { - if c.reason == "vendor_artifact_corrupt" { - if let Some(pair) = - set_aside_vendor_dir(&common.cwd, &c.entry.ecosystem, &c.entry.uuid).await - { - aside.insert(c.purl.clone(), pair); - } - } - } if api_client.is_none() && !common.offline { api_client = Some( get_api_client_with_overrides(common.api_client_overrides()) @@ -739,481 +642,70 @@ impl VendoredBackend<'_> { .as_ref() .is_some_and(ApiClient::uses_public_proxy); let service = common.vendor_service_config(api_client, use_public_proxy); - // The engine runs quiet into a scratch envelope: repair speaks in - // its own vocabulary (`rebuilt`, "Cannot repair …") and translates - // each candidate's outcome below. - let mut engine_common = common.clone(); - engine_common.json = true; - engine_common.silent = true; - let mut scratch = Envelope::new(crate::json_envelope::Command::Vendor); - let mut no_source = run_engine( - &engine_common, - Some(&service), - &candidates, - &seed, - req.socket_dir, - &mut scratch, - ) - .await; - - // A service archive that is not the recorded artifact: undo what - // the engine wired for it and rebuild locally (deterministic), then - // verify again. - let mut retry: Vec = Vec::new(); - for c in &candidates { - let from_service = scratch.events.iter().any(|e| { - e.purl.as_deref() == Some(c.purl.as_str()) - && e.error_code.as_deref() == Some("vendor_prebuilt_downloaded") - }); - if from_service - && !keeps_identity(&check_vendored_artifact(&common.cwd, &c.entry, &c.record).await) - { - undo_candidate(&common.cwd, c, &snapshot).await; - remove_vendor_dir(&common.cwd, &c.entry.ecosystem, &c.entry.uuid).await; - retry.push(c.clone()); - } - } - if !retry.is_empty() { - scratch.events.retain(|e| { - !retry - .iter() - .any(|c| e.purl.as_deref() == Some(c.purl.as_str())) - }); - no_source |= run_engine( - &engine_common, - None, - &retry, - &seed, - req.socket_dir, - &mut scratch, + for mut candidate in candidates { + match vendor::redownload::restore( + &common.cwd, + &candidate.entry, + &candidate.record, + &service, ) - .await; - } - let mut state = load_state(&common.cwd).await.unwrap_or(state); - - for c in candidates { - let kept = aside.remove(&c.purl); - let outcome = EngineOutcome::of(&scratch, &c.purl); - forward_advisories(env, common, &scratch, &c.purl); - // Verified against the ORIGINAL entry: the recorded identity is - // the target, never a fingerprint this run just wrote. - let mut health = check_vendored_artifact(&common.cwd, &c.entry, &c.record).await; - // The entry to keep: the backend's (a tagged cargo rebuild - // re-records its wiring) when it rebuilt the recorded artifact, - // else the original. - let mut entry = state - .entries - .get(&c.purl) - .filter(|e| e.uuid == c.entry.uuid) - .cloned() - .unwrap_or_else(|| c.entry.clone()); - // A dir-shaped rebuild whose PATCHED members verify but whose - // tree differs from an inventory the backend carried over - // unchanged (the cargo backend records none of its own): the - // entry recorded another build source's tree (the patch - // service's prebuilt artifact). Failing would delete a verified - // rebuild and re-fail every later repair, so refresh the - // inventory from the rebuild instead, loudly. - if c.entry.artifact.file_inventory.is_some() - && matches!(&health, ArtifactHealth::Corrupt { reason } - if reason == "vendor_inventory_mismatch") + .await { - let abs = common.cwd.join(entry.artifact.path.replace('\\', "/")); - if let Ok(inv) = artifact_dir_inventory(&entry, &abs).await { - let mut refreshed = entry.clone(); - refreshed.artifact.file_inventory = Some(inv); - if check_vendored_artifact(&common.cwd, &refreshed, &c.record).await - == ArtifactHealth::Healthy - { - record_warning( - env, - &c.purl, - &VendorWarning::new( - "vendor_inventory_refreshed", - INVENTORY_REFRESHED_DETAIL, - ), - common, - ); - if persist_vendor_entry( - common, - env, - &mut state, - &c.purl, - refreshed.clone(), - c.detached, - &c.record, - ) - .await - { - // The write failure is the outcome (already - // recorded): keep the member-verified rebuild - // on disk, but never claim it `rebuilt`. - env.mark_partial_failure(); - if let Some((_, kept)) = &kept { - let _ = remove_tree(kept).await; + Ok(warnings) => { + for warning in warnings { + record_warning(env, &candidate.purl, &warning, common); + } + match repair_workspace_copies(&common.cwd, &mut candidate.entry, false).await { + Ok(true) => { + if persist_vendor_entry( + common, + env, + &mut state, + &candidate.purl, + candidate.entry.clone(), + candidate.detached, + &candidate.record, + ) + .await + { + env.mark_partial_failure(); + continue; } + } + Ok(false) => {} + Err(error) => { + fail( + env, + common.json, + &candidate.purl, + "vendor_artifact_redownload_failed", + error, + ); continue; } - entry = refreshed; - health = ArtifactHealth::Healthy; } - } - } - let produced = outcome.failure.is_none() - && (outcome.rebuilt - || (outcome.in_sync - && tokio::fs::symlink_metadata(common.cwd.join(&entry.artifact.path)) - .await - .is_ok())); - if produced && health == ArtifactHealth::Healthy { - if let Some((_, kept)) = &kept { - if let Some(w) = - vendor::vlt_lock::keep_vlt_links(&c.entry, kept, &common.cwd).await - { - record_warning(env, &c.purl, &w, common); + if !quiet { + println!( + "Redownloaded {} ({})", + normalize_purl(&candidate.purl), + candidate.entry.artifact.path + ); } - let _ = remove_tree(kept).await; - } - if !quiet { - println!( - "Rebuilt {} ({})", - normalize_purl(&c.purl), - entry.artifact.path - ); + env.record(PatchEvent::new(PatchAction::Rebuilt, candidate.purl.clone()).with_details(serde_json::json!({ + "path": candidate.entry.artifact.path, "reason": candidate.reason, "redownloaded": true, + }))); + repaired += 1; } - env.record( - PatchEvent::new(PatchAction::Rebuilt, c.purl.clone()).with_details( - serde_json::json!({ - "path": entry.artifact.path, - "reason": c.reason, - }), - ), - ); - repaired += 1; - continue; - } - // Not the recorded artifact: put back the wiring and ledger entry - // the engine may have rewritten for it. - undo_candidate(&common.cwd, &c, &snapshot).await; - if produced { - // The re-vendor did not reproduce the recorded artifact - // (e.g. a tampered ledger sha): remove it rather than leave - // unverifiable bytes behind. - remove_vendor_dir(&common.cwd, &c.entry.ecosystem, &c.entry.uuid).await; - if let Some((_, kept)) = &kept { - let _ = remove_tree(kept).await; - } - fail( + Err(error) => fail( env, common.json, - &c.purl, - "vendor_artifact_rebuild_failed", - format!( - "the rebuilt artifact does not match the recorded fingerprint \ - ({health:?}); if state.json was edited, run `socket-patch vendor` \ - to re-vendor from scratch", - ), - ); - continue; - } - // Nothing replaced the artifact: put the pre-rebuild bytes back. - if let Some((live, kept)) = &kept { - restore_aside_vendor_dir(live, kept).await; - } - let (code, detail) = match outcome.failure { - // The dispatch ran and failed: repair's rebuild vocabulary. - Some((code, detail)) if code == "apply_failed" => { - ("vendor_artifact_rebuild_failed".to_string(), detail) - } - // No pristine source: a compiled wheel can only come back - // from an install on its own platform. - Some((code, _)) - if code == "vendor_artifact_unrepairable" - && c.entry.artifact.platform_locked == Some(true) => - { - ( - code, - "the vendored wheel is platform-locked (compiled) and no pristine \ - source was found; reinstall the package on this platform and re-run \ - repair, or run `socket-patch vendor` to rebuild it" - .to_string(), - ) - } - Some((code, detail)) => (code, detail), - None if no_source || outcome.no_source => ( - c.reason.to_string(), - format!( - "the vendored artifact at {} is broken and its patch content could \ - not be obtained (no local source: {})", - c.entry.artifact.path, - if common.offline { - "--offline prevents fetching it" - } else { - NO_LOCAL_SOURCE_MESSAGE - } - ), - ), - None => ( - c.reason.to_string(), - format!( - "the vendored artifact at {} is broken and could not be re-vendored", - c.entry.artifact.path - ), + &candidate.purl, + "vendor_artifact_redownload_failed", + error, ), - }; - // An offline run that could not re-vendor says why. - let detail = if common.offline && !detail.contains("--offline") { - format!("{detail} (--offline prevents fetching a source)") - } else { - detail - }; - fail(env, common.json, &c.purl, &code, detail); - } - repaired - } -} - -/// The `vendor_inventory_refreshed` detail. -const INVENTORY_REFRESHED_DETAIL: &str = "the re-vendored artifact's patched files verify but its \ - tree differs from the recorded file inventory (the entry was likely vendored from a \ - different source, such as the patch service's prebuilt artifact); the inventory was \ - refreshed from the verified rebuild — run `socket-patch vendor` to restore the \ - service-built tree"; - -/// Run the vendor engine over `candidates` (manifest-owned and detached -/// groups separately) into `scratch`. `service: None` is a build-only run. -/// `true` when staging obtained no patch content for a group. -async fn run_engine( - engine_common: &GlobalArgs, - service: Option<&vendor::VendorServiceConfig>, - candidates: &[Candidate], - seed: &HashMap>, - socket_dir: &Path, - scratch: &mut Envelope, -) -> bool { - let engine = VendoredBackend::new(engine_common, service); - let mut no_source = false; - for detached in [false, true] { - let records: HashMap = candidates - .iter() - .filter(|c| c.detached == detached) - .map(|c| (c.purl.clone(), c.record.clone())) - .collect(); - if records.is_empty() { - continue; - } - let manifest = records_manifest(records); - let applied = engine - .apply( - ApplyRequest { - manifest: &manifest, - socket_dir, - ledger: load_state(&engine_common.cwd).await, - seed: seed.clone(), - detached, - force: false, - prior: None, - }, - scratch, - ) - .await; - no_source |= applied.is_err(); - } - no_source -} - -/// One wiring file as it was before the re-vendor: its bytes (`None`: it -/// did not exist) and, for a symlinked lockfile, the link text — lockfile -/// writers rename over the path, which replaces a link with a file. -struct WiringSnapshot { - path: PathBuf, - link: Option, - bytes: Option>, -} - -/// Snapshot each candidate's recorded wiring files — what a re-vendor may -/// rewrite for it. Reads are FIFO-safe; a non-regular file is skipped. -async fn snapshot_wiring(cwd: &Path, candidates: &[Candidate]) -> Vec { - let mut rels: Vec = Vec::new(); - for c in candidates { - rels.extend(c.entry.wiring.iter().map(|w| w.file.clone())); - } - rels.sort(); - rels.dedup(); - let mut out = Vec::with_capacity(rels.len()); - for rel in rels { - let path = cwd.join(&rel); - let bytes = match read_regular_to_bytes(&path).await { - Ok(bytes) => Some(bytes), - Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, - Err(_) => continue, - }; - let link = tokio::fs::read_link(&path).await.ok(); - out.push(WiringSnapshot { path, link, bytes }); - } - out -} - -/// Whether a rebuilt artifact still is the recorded one: healthy against -/// the original entry, or (dir-shaped) its patched members verify and only -/// the recorded whole-tree inventory differs — the inventory-refresh case. -fn keeps_identity(health: &ArtifactHealth) -> bool { - match health { - ArtifactHealth::Healthy => true, - ArtifactHealth::Corrupt { reason } => reason == "vendor_inventory_mismatch", - _ => false, - } -} - -/// Undo what a re-vendor may have written for `c`: its recorded wiring -/// files go back to their snapshotted bytes (a replaced symlink is -/// re-linked and its target rewritten) and its ledger entry to the -/// original. Other candidates' files are left alone. -async fn undo_candidate(cwd: &Path, c: &Candidate, snapshot: &[WiringSnapshot]) { - let wired: HashSet = c.entry.wiring.iter().map(|w| cwd.join(&w.file)).collect(); - for snap in snapshot.iter().filter(|s| wired.contains(&s.path)) { - if let Some(link) = &snap.link { - if tokio::fs::read_link(&snap.path).await.ok().as_ref() != Some(link) { - let _ = tokio::fs::remove_file(&snap.path).await; - if relink(link, &snap.path).await.is_err() { - continue; - } - } - } - if read_regular_to_bytes(&snap.path).await.ok() == snap.bytes { - continue; - } - match &snap.bytes { - // Staged, fsynced and renamed over the link's target (or the - // file itself), keeping its mode. - Some(bytes) => { - let target = tokio::fs::canonicalize(&snap.path) - .await - .unwrap_or_else(|_| snap.path.clone()); - let _ = atomic_write_bytes_preserving_mode(&target, bytes).await; - } - None => { - let _ = tokio::fs::remove_file(&snap.path).await; - } - } - } - if let Ok(mut state) = load_state(cwd).await { - if state.entries.get(&c.purl) != Some(&c.entry) { - state.entries.insert(c.purl.clone(), c.entry.clone()); - let _ = vendor::save_state(cwd, &state).await; - } - } -} - -#[cfg(unix)] -async fn relink(link: &Path, at: &Path) -> std::io::Result<()> { - tokio::fs::symlink(link, at).await -} - -#[cfg(windows)] -async fn relink(link: &Path, at: &Path) -> std::io::Result<()> { - tokio::fs::symlink_file(link, at).await -} - -/// One candidate's outcome in the engine's scratch envelope. -struct EngineOutcome { - /// The engine vendored it (`applied`). - rebuilt: bool, - /// The engine found the wiring in sync (`already_vendored`). A backend - /// whose committed artifact was missing re-acquires it and still - /// reports in-sync (the lock already pins those bytes), so this counts - /// as produced when the artifact exists afterwards. - in_sync: bool, - /// Its failure or genuine skip, as `(code, detail)`. - failure: Option<(String, String)>, - /// Staging could not obtain its patch content (`no_local_source`). - no_source: bool, -} - -/// Engine skip codes that are a package's OUTCOME (everything else a -/// `Skipped` event carries is an uncounted advisory). -const OUTCOME_SKIPS: &[&str] = &["package_not_installed", "vendor_unsupported_ecosystem"]; - -/// Engine advisories repair does not forward: its own `rebuilt` event and -/// candidate reason already say it. -const SUPPRESSED_ADVISORIES: &[&str] = &["vendor_artifact_missing", "vendor_artifact_rebuilt"]; - -impl EngineOutcome { - fn of(scratch: &Envelope, purl: &str) -> Self { - let mut out = EngineOutcome { - rebuilt: false, - in_sync: false, - failure: None, - no_source: false, - }; - for ev in scratch - .events - .iter() - .filter(|e| e.purl.as_deref() == Some(purl)) - { - let code = ev.error_code.clone().unwrap_or_default(); - let detail = ev - .error - .clone() - .or_else(|| ev.reason.clone()) - .unwrap_or_default(); - match ev.action { - PatchAction::Applied => out.rebuilt = true, - PatchAction::Skipped if code == "already_vendored" => out.in_sync = true, - PatchAction::Failed if out.failure.is_none() => { - out.no_source |= code == "no_local_source"; - out.failure = Some((code, detail)); - } - PatchAction::Skipped - if out.failure.is_none() && OUTCOME_SKIPS.contains(&code.as_str()) => - { - out.failure = Some(("vendor_artifact_unrepairable".to_string(), detail)); - } - _ => {} - } - } - if !out.rebuilt && !out.in_sync && out.failure.is_none() { - if let Some(e) = &scratch.error { - out.failure = Some((e.code.clone(), e.message.clone())); } } - out - } -} - -/// Carry the engine's advisories for `purl` into repair's envelope (as -/// uncounted events, like every vendor advisory) and onto stderr at the -/// vendor command's own tiers. -fn forward_advisories(env: &mut Envelope, common: &GlobalArgs, scratch: &Envelope, purl: &str) { - for ev in scratch - .events - .iter() - .filter(|e| e.purl.as_deref() == Some(purl) && matches!(e.action, PatchAction::Skipped)) - { - let code = ev.error_code.as_deref().unwrap_or_default(); - if OUTCOME_SKIPS.contains(&code) || SUPPRESSED_ADVISORIES.contains(&code) { - continue; - } - let detail = ev.reason.as_deref().unwrap_or_default(); - if !common.silent && !common.json { - if let Some(line) = format_advisory(code, detail, common.verbose) { - eprintln!("{line}"); - } - } - env.events.push(ev.clone()); - } -} - -/// A dir artifact's inventory: an npm dir (vlt's package dir) leaves out -/// its `node_modules/`, which holds vlt's links and is never part of it. -async fn artifact_dir_inventory( - entry: &VendorEntry, - abs: &Path, -) -> Result, String> { - if entry.ecosystem == "npm" { - vendor::compute_package_dir_inventory(abs).await - } else { - vendor::compute_dir_inventory(abs).await + repaired } } @@ -1568,34 +1060,6 @@ mod tests { "non-vendor .socket mentions must be rejected: {refs:?}" ); } - - /// [`remove_vendor_dir`] is a best-effort guard that must never GUESS a - /// path: an eco/uuid pair that cannot map to a canonical vendor dir - /// (unknown ecosystem dir, non-canonical uuid) removes NOTHING, while - /// the mappable pair removes exactly its uuid dir. - #[tokio::test] - async fn remove_vendor_dir_refuses_unmappable_eco_or_uuid() { - let tmp = tempfile::tempdir().unwrap(); - let uuid = "11111111-1111-4111-8111-111111111111"; - let dir = tmp.path().join(format!(".socket/vendor/npm/{uuid}")); - tokio::fs::create_dir_all(&dir).await.unwrap(); - tokio::fs::write(dir.join("x.tgz"), b"bytes").await.unwrap(); - - remove_vendor_dir(tmp.path(), "jsr", uuid).await; - assert!(dir.is_dir(), "an unmappable ecosystem must remove nothing"); - remove_vendor_dir(tmp.path(), "npm", "not-a-uuid").await; - assert!(dir.is_dir(), "a non-canonical uuid must remove nothing"); - remove_vendor_dir(tmp.path(), "npm", uuid).await; - assert!(!dir.exists(), "the canonical pair removes its uuid dir"); - assert!( - !tmp.path().join(".socket/vendor").exists(), - "the emptied / and vendor/ husks are pruned" - ); - assert!( - tmp.path().join(".socket").is_dir(), - ".socket/ is never removed" - ); - } } /// Exact-string tests for the vendored-repair human lines. @@ -1619,9 +1083,9 @@ mod ui_format_tests { ".socket/vendor/npm/u/minimist-1.2.5.tgz", )]; assert_eq!( - format_rebuild_preview(&one), + format_redownload_preview(&one), vec![ - "Would rebuild 1 vendored artifact:", + "Would redownload 1 vendored artifact:", " - pkg:npm/minimist@1.2.5 (missing: .socket/vendor/npm/u/minimist-1.2.5.tgz)", ] ); @@ -1634,9 +1098,9 @@ mod ui_format_tests { ("pkg:gem/b@1".to_string(), "vendor_artifact_missing", "p/b"), ]; assert_eq!( - format_rebuild_preview(&two), + format_redownload_preview(&two), vec![ - "Would rebuild 2 vendored artifacts:", + "Would redownload 2 vendored artifacts:", " - pkg:npm/a@1 (corrupt: p/a.tgz)", " - pkg:gem/b@1 (missing: p/b)", ] diff --git a/crates/socket-patch-cli/src/commands/vex_sources.rs b/crates/socket-patch-cli/src/commands/vex_sources.rs index 174c0f007..4a8e40406 100644 --- a/crates/socket-patch-cli/src/commands/vex_sources.rs +++ b/crates/socket-patch-cli/src/commands/vex_sources.rs @@ -759,6 +759,7 @@ fn vendored_entry_for(cand: &Cand, vref: &PatchedRef) -> VendorEntry { base_purl: strip_purl_qualifiers(&cand.key).to_string(), uuid: vref.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: wired.to_string(), sha256: String::new(), size: None, @@ -1307,6 +1308,7 @@ mod tests { base_purl: key.into(), uuid: U1.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{U1}/x-1.0.0.tgz"), sha256: String::new(), size: None, @@ -1380,6 +1382,7 @@ mod tests { base_purl: "pkg:npm/x@1.0.0".into(), uuid: U1.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.clone(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/tests/cli_global_args.rs b/crates/socket-patch-cli/tests/cli_global_args.rs index 3a19ef43d..b7e1f50a3 100644 --- a/crates/socket-patch-cli/tests/cli_global_args.rs +++ b/crates/socket-patch-cli/tests/cli_global_args.rs @@ -914,7 +914,7 @@ fn production_defaults_populate_when_unset() { assert_eq!(c.api_url, None, "no clap default — resolved in core"); assert_eq!(c.proxy_url, None, "no clap default — resolved in core"); assert_eq!(c.download_mode, "diff"); - assert_eq!(c.vendor_source, "auto"); + assert_eq!(c.vendor_source, "service"); assert!(c.vendor_url.is_none()); assert!(c.patch_server_url.is_none()); assert!(c.api_token.is_none()); diff --git a/crates/socket-patch-cli/tests/cli_parse_get.rs b/crates/socket-patch-cli/tests/cli_parse_get.rs index 164c9c02f..1393a60fd 100644 --- a/crates/socket-patch-cli/tests/cli_parse_get.rs +++ b/crates/socket-patch-cli/tests/cli_parse_get.rs @@ -211,7 +211,7 @@ fn expected_defaults(identifier: &str) -> Snap { proxy_url: None, // no clap default — resolved in core ecosystems: None, download_mode: "diff".to_string(), - vendor_source: "auto".to_string(), + vendor_source: "service".to_string(), vendor_url: None, patch_server_url: None, offline: false, diff --git a/crates/socket-patch-cli/tests/cli_parse_repair.rs b/crates/socket-patch-cli/tests/cli_parse_repair.rs index c83b6f806..5204ace3f 100644 --- a/crates/socket-patch-cli/tests/cli_parse_repair.rs +++ b/crates/socket-patch-cli/tests/cli_parse_repair.rs @@ -207,7 +207,7 @@ fn expected_defaults() -> Snap { proxy_url: None, // no clap default — resolved in core ecosystems: None, download_mode: "diff".to_string(), - vendor_source: "auto".to_string(), + vendor_source: "service".to_string(), vendor_url: None, patch_server_url: None, offline: false, diff --git a/crates/socket-patch-cli/tests/cli_parse_vex.rs b/crates/socket-patch-cli/tests/cli_parse_vex.rs index 5d5c49349..9f3694d9a 100644 --- a/crates/socket-patch-cli/tests/cli_parse_vex.rs +++ b/crates/socket-patch-cli/tests/cli_parse_vex.rs @@ -279,7 +279,7 @@ fn expected_defaults() -> Snap { proxy_url: None, // no clap default — resolved in core ecosystems: None, download_mode: "diff".to_string(), - vendor_source: "auto".to_string(), + vendor_source: "service".to_string(), vendor_url: None, patch_server_url: None, offline: false, diff --git a/crates/socket-patch-cli/tests/cli_scan_silent.rs b/crates/socket-patch-cli/tests/cli_scan_silent.rs index 3a12211af..5f36ab2b6 100644 --- a/crates/socket-patch-cli/tests/cli_scan_silent.rs +++ b/crates/socket-patch-cli/tests/cli_scan_silent.rs @@ -15,6 +15,9 @@ //! `get_api_client_with_overrides` in core for every command and is //! out of scope for `scan`'s `--silent` gating. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -144,24 +147,26 @@ async fn mount_one_patch_api(mock: &MockServer, purl: &str, before: &[u8]) { .await; // base64 of "after\n" — inline so the apply step needs no blob endpoint. + let archive_view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2024-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": "YWZ0ZXIK", + } + }, + "vulnerabilities": {}, + "description": "Silent test patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2024-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": "YWZ0ZXIK", - } - }, - "vulnerabilities": {}, - "description": "Silent test patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } diff --git a/crates/socket-patch-cli/tests/common/yarn_classic_vex.rs b/crates/socket-patch-cli/tests/common/yarn_classic_vex.rs index dd1f6e32e..acd427368 100644 --- a/crates/socket-patch-cli/tests/common/yarn_classic_vex.rs +++ b/crates/socket-patch-cli/tests/common/yarn_classic_vex.rs @@ -299,8 +299,29 @@ impl<'a> ManifestlessVex<'a> { fn embedded_attest(&self, project: &Path, state: &str, keep_ledgers: bool) { let lock = std::fs::read(project.join("yarn.lock")).expect("yarn.lock"); for (label, make) in &self.embedded { - let out = run_vex(&binary(), project, &make(self.online())); - self.attested(&out, &format!("embedded {label} ({state})")); + let run = make(self.online()); + let missing_ledger = self.wiring == Wiring::Vendored + && !keep_ledgers + && matches!(run.via, crate::vex_e2e_common::VexVia::Scan); + if missing_ledger { + let output = project.join( + run.output + .as_deref() + .unwrap_or_else(|| Path::new("out.vex.json")), + ); + let _ = std::fs::remove_file(output); + } + let out = run_vex(&binary(), project, &run); + if missing_ledger { + assert_eq!(out.code, Some(1), "{out}"); + assert_eq!( + out.envelope["vendor"]["events"][0]["errorCode"], "vendor_ledger_entry_missing", + "{out}" + ); + assert!(out.doc.is_none(), "failed scan cannot emit VEX: {out}"); + } else { + self.attested(&out, &format!("embedded {label} ({state})")); + } assert!( !project.join(".socket/manifest.json").exists(), "{}: embedded {label} ({state}) must not write a manifest", diff --git a/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs b/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs index 0cb61906a..ed0b996b1 100644 --- a/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs +++ b/crates/socket-patch-cli/tests/coverage_fix_scan_hosted_dryrun_vendored.rs @@ -10,6 +10,9 @@ //! shapes) produces the vendored lock + `.socket/vendor/state.json` entry; //! the hosted API is wiremock (`in_process_redirect_pnpm.rs` shapes). +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; #[path = "vlt_hosted_common/mod.rs"] @@ -183,7 +186,13 @@ fn seed_manifest_and_blob(root: &Path) { /// stderr)`. fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = std::process::Command::new(env!("CARGO_BIN_EXE_socket-patch")); - cmd.args(args).current_dir(cwd); + let fixture = (args.first() == Some(&"vendor")).then(|| prebuilt_common::Server::project(cwd)); + let args: Vec<_> = args + .iter() + .copied() + .filter(|arg| fixture.is_none() || *arg != "--offline") + .collect(); + cmd.args(&args).current_dir(cwd); for (key, _) in std::env::vars() { if key.starts_with("SOCKET_") && key != "SOCKET_NO_CONFIG" { cmd.env_remove(key); @@ -204,6 +213,9 @@ fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { } cmd.env("NPM_CONFIG_ALLOW_REMOTE", "") .env("npm_config_allow_remote", ""); + if let Some(fixture) = &fixture { + fixture.command(&mut cmd); + } let out = cmd.output().expect("spawn socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -679,7 +691,12 @@ async fn vlt_dry_run_over_vendored_project_previews_the_wet_takeover() { ) .unwrap(); let cwd = root.to_str().unwrap().to_string(); - let (code, env, stderr) = hosted::run_json(root, &["vendor", "--offline", "--cwd", &cwd], &[]); + let fixture = prebuilt_common::Server::project(root); + let (code, env, stderr) = hosted::run_json( + root, + &["vendor", "--cwd", &cwd], + &[("SOCKET_VENDOR_URL", &fixture.uri)], + ); assert_eq!(code, 0, "{env:#}\n{stderr}"); std::fs::remove_file(socket.join("manifest.json")).unwrap(); let vendored_lock = std::fs::read(root.join("vlt-lock.json")).unwrap(); diff --git a/crates/socket-patch-cli/tests/coverage_fix_vendor_silent_mute_exit.rs b/crates/socket-patch-cli/tests/coverage_fix_vendor_silent_mute_exit.rs index 45bd8ae46..3bfe0e862 100644 --- a/crates/socket-patch-cli/tests/coverage_fix_vendor_silent_mute_exit.rs +++ b/crates/socket-patch-cli/tests/coverage_fix_vendor_silent_mute_exit.rs @@ -66,6 +66,12 @@ fn stderr_chatter(stderr: &str) -> Vec { /// artifact anywhere under `.socket/` (and no committed vendor artifact /// to harvest) — the mem stager must fetch, or fail. fn write_sourceless_manifest(root: &Path) { + std::fs::write( + root.join("package.json"), + r#"{"name":"fixture","version":"1.0.0"}"#, + ) + .unwrap(); + std::fs::write(root.join("package-lock.json"), r#"{"lockfileVersion":3,"packages":{"":{},"node_modules/left-pad":{"version":"1.3.0","resolved":"https://registry.npmjs.org/left-pad/-/left-pad-1.3.0.tgz","integrity":"sha512-original"}}}"#).unwrap(); let socket = root.join(".socket"); std::fs::create_dir_all(&socket).unwrap(); std::fs::write( @@ -114,7 +120,7 @@ fn vendor_silent_offline_missing_source_keeps_error_output() { ); let chatter = stderr_chatter(&stderr); assert!( - chatter.iter().any(|l| l.contains("no local source")), + chatter.iter().any(|l| l.contains("offline")), "--silent must keep the offline no-source error (errors only, \ never nothing); stderr was: {stderr:?}" ); @@ -152,9 +158,7 @@ fn vendor_silent_online_fetch_failure_keeps_error_output() { ); let chatter = stderr_chatter(&stderr); assert!( - chatter - .iter() - .any(|l| l.contains("Could not fetch patch content")), + chatter.iter().any(|l| l.contains("request")), "--silent must keep the fetch-failure error (errors only, \ never nothing); stderr was: {stderr:?}" ); @@ -188,7 +192,7 @@ fn vendor_json_online_fetch_failure_keeps_stderr_clean() { let v: serde_json::Value = serde_json::from_str(&stdout).expect("vendor --json must emit valid JSON"); assert_eq!( - v["error"]["code"], "no_local_source", + v["events"][0]["errorCode"], "apply_failed", "the pinned envelope error for the unavailable bail, got {v}" ); let chatter = stderr_chatter(&stderr); diff --git a/crates/socket-patch-cli/tests/covgap_commands_get.rs b/crates/socket-patch-cli/tests/covgap_commands_get.rs index a67f2fe7e..4eb66b75c 100644 --- a/crates/socket-patch-cli/tests/covgap_commands_get.rs +++ b/crates/socket-patch-cli/tests/covgap_commands_get.rs @@ -13,6 +13,9 @@ //! `common::run_with_env`, which scrubs the ambient `SOCKET_*` surface and //! spawns a hermetic child, so they need no serialization. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::HashMap; use std::path::{Path, PathBuf}; @@ -397,6 +400,8 @@ fn write_project(root: &Path) { /// `view/{uuid}` with REAL git-blob hashes over the project fixture's bytes, /// so the vendored staging hash-gates pass. async fn mount_real_view(server: &MockServer, uuid: &str, purl: &str) { + let files = serde_json::json!({"package/index.js":{"beforeHash":git_hash(BEFORE_BYTES),"afterHash":git_hash(AFTER_BYTES),"blobContent":b64(AFTER_BYTES)}}); + prebuilt_common::mount_view(server, &view_json(uuid, purl, files), None).await; mount_view_files( server, uuid, @@ -611,7 +616,7 @@ async fn get_uuid_socket_path_occupied_by_file_fails_closed() { args.common.api_url = Some(uri.clone()); args.save_only = false; args.mode = Some(ScanMode::Vendored); - args.common.vendor_source = "build".to_string(); + args.common.vendor_source = "service".to_string(); let code = run(args).await; assert_eq!(code, 1, "vendored run must fail when .socket is a file"); assert_eq!( @@ -1379,7 +1384,7 @@ async fn engine_variant_view_fetch_error_keeps_errored_variant() { fn vendored_args(identifier: &str, cwd: &Path, api_url: String) -> GetArgs { let mut args = default_args(identifier, cwd); args.common.api_url = Some(api_url); - args.common.vendor_source = "build".to_string(); + args.common.vendor_source = "service".to_string(); args.save_only = false; args.mode = Some(ScanMode::Vendored); args @@ -1464,7 +1469,7 @@ fn vendored_json_args(uuid: &str) -> [&str; 6] { "--mode", "vendored", "--vendor-source", - "build", + "service", "--json", ] } @@ -1615,10 +1620,10 @@ fn assert_legacy_state_untouched(root: &Path, manifest_before: &str, state_befor } fn assert_vendor_error_envelope(v: &serde_json::Value) { - assert_eq!(v["status"], "error", "envelope={v}"); + assert_eq!(v["status"], "partial_failure", "envelope={v}"); assert_eq!( - v["error"]["code"], "no_local_source", - "the staging refusal must be the error code; envelope={v}" + v["vendor"]["events"][0]["errorCode"], "vendor_lockfile_missing", + "{v}" ); assert_eq!( v["vendor"]["status"], "partialFailure", @@ -1676,7 +1681,7 @@ async fn get_search_vendored_vendor_step_error_leaves_legacy_state_alone() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", "--json", ], @@ -1702,7 +1707,7 @@ async fn human_vendored_uuid_prints_fetch_and_vendor_error_without_manifest_note let (code, stdout, stderr) = run_get_bin( tmp.path(), &server.uri(), - &[UUID, "--mode", "vendored", "--vendor-source", "build"], + &[UUID, "--mode", "vendored", "--vendor-source", "service"], ); assert_eq!(code, 1, "stdout={stdout}\nstderr={stderr}"); assert!( @@ -1718,8 +1723,8 @@ async fn human_vendored_uuid_prints_fetch_and_vendor_error_without_manifest_note "there is no whole-manifest scope to warn about; stderr={stderr}" ); assert!( - stderr.contains("Error (no_local_source):"), - "the vendor-step error must print with its code; stderr={stderr}" + stderr.contains("no package-lock.json"), + "the vendor-step error must explain the missing lockfile; stderr={stderr}" ); assert_legacy_state_untouched(tmp.path(), &manifest_before, &state_before); } @@ -2171,7 +2176,7 @@ async fn human_vendored_search_success_commits_artifact_without_blast_radius_not let (code, stdout, stderr) = run_get_bin( tmp.path(), &server.uri(), - &[GHSA, "--mode", "vendored", "--vendor-source", "build"], + &[GHSA, "--mode", "vendored", "--vendor-source", "service"], ); assert_eq!(code, 0, "stdout={stdout}\nstderr={stderr}"); let artifact = tmp @@ -2218,7 +2223,7 @@ async fn vendored_search_ignores_corrupt_manifest_and_vendors() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", "--json", ], @@ -2260,7 +2265,7 @@ async fn vendored_search_json_download_failure_with_clean_vendor_is_partial_fail "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", "--json", ], @@ -2337,7 +2342,7 @@ async fn vendored_lock_held_vendor_step_errors_without_vendor_envelope() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", "--json", ], @@ -2367,7 +2372,7 @@ async fn vendored_lock_held_vendor_step_errors_without_vendor_envelope() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", ], ); @@ -2526,7 +2531,7 @@ async fn human_vendored_uuid_supersede_prints_replacing_and_vendors() { let (code, stdout, stderr) = run_get_bin( tmp.path(), &server.uri(), - &[UUID, "--mode", "vendored", "--vendor-source", "build"], + &[UUID, "--mode", "vendored", "--vendor-source", "service"], ); assert_eq!(code, 0, "stdout={stdout}\nstderr={stderr}"); assert!( @@ -2563,7 +2568,7 @@ async fn human_vendored_uuid_rerun_prints_already_vendored_skip() { let (code, stdout, stderr) = run_get_bin( tmp.path(), &server.uri(), - &[UUID, "--mode", "vendored", "--vendor-source", "build"], + &[UUID, "--mode", "vendored", "--vendor-source", "service"], ); assert_eq!(code, 0, "stdout={stdout}\nstderr={stderr}"); assert!( @@ -3050,7 +3055,7 @@ async fn human_vendored_search_all_downloads_failed_prints_empty_run_line() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--all-releases", ], ); diff --git a/crates/socket-patch-cli/tests/covgap_commands_rollback.rs b/crates/socket-patch-cli/tests/covgap_commands_rollback.rs index 496d06387..2c2ca6b33 100644 --- a/crates/socket-patch-cli/tests/covgap_commands_rollback.rs +++ b/crates/socket-patch-cli/tests/covgap_commands_rollback.rs @@ -24,8 +24,11 @@ //! rollback restores the upstream registry entry from a mock npm registry //! (`SOCKET_NPM_REGISTRY`, see `NpmRegistry`). +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; #[path = "common/pty_io.rs"] mod pty_io; + use std::path::{Path, PathBuf}; use serde_json::{json, Value}; @@ -33,7 +36,20 @@ use serde_json::{json, Value}; #[path = "common/mod.rs"] mod common; -use common::{envelope_error_code, git_sha256, json_string, run}; +use common::{envelope_error_code, git_sha256, json_string}; +fn run(root: &Path, args: &[&str]) -> (i32, String, String) { + if args.first() == Some(&"vendor") { + let fixture = prebuilt_common::Server::project(root); + let args: Vec<_> = args + .iter() + .copied() + .filter(|arg| *arg != "--offline") + .collect(); + common::run_with_env(root, &args, &[("SOCKET_VENDOR_URL", &fixture.uri)]) + } else { + common::run(root, args) + } +} // ───────────────────────── shared fixture helpers ───────────────────────── diff --git a/crates/socket-patch-cli/tests/covgap_commands_vendor.rs b/crates/socket-patch-cli/tests/covgap_commands_vendor.rs index b8f62b99e..b3c6e2194 100644 --- a/crates/socket-patch-cli/tests/covgap_commands_vendor.rs +++ b/crates/socket-patch-cli/tests/covgap_commands_vendor.rs @@ -11,12 +11,21 @@ //! the built binary with a scrubbed child environment (`run_cli` / //! `vendor_cli`). No test mutates this process's environment. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + +async fn vendor_run(mut args: VendorArgs) -> i32 { + let server = prebuilt_common::Server::project(&args.common.cwd); + server.configure(&mut args.common); + actual_vendor_run(args).await +} + use std::path::{Path, PathBuf}; use std::process::Command; use serde_json::{json, Value}; use socket_patch_cli::args::GlobalArgs; -use socket_patch_cli::commands::vendor::{run as vendor_run, VendorArgs}; +use socket_patch_cli::commands::vendor::{run as actual_vendor_run, VendorArgs}; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; use socket_patch_core::vendor::state::VendorArtifact; use socket_patch_core::vendor::{save_state, VendorEntry, VendorState}; @@ -154,7 +163,7 @@ fn vendor_args(cwd: &Path) -> VendorArgs { cwd: cwd.to_path_buf(), json: true, silent: true, - offline: true, + offline: false, // See in_process_vendor.rs: absorbs the fork→exec fd window of // concurrent tests in this binary. lock_timeout: Some(5), @@ -174,7 +183,23 @@ fn vendor_args(cwd: &Path) -> VendorArgs { /// scrubbed from the child and telemetry hard-disabled. fn run_cli(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> (i32, String, String) { let mut cmd = Command::new(env!("CARGO_BIN_EXE_socket-patch")); - cmd.args(args).current_dir(cwd); + let fixture = (!args + .iter() + .any(|a| matches!(*a, "--api-url" | "--vendor-url")) + && !extra_env + .iter() + .any(|(k, _)| matches!(*k, "SOCKET_API_URL" | "SOCKET_VENDOR_URL"))) + .then(|| prebuilt_common::Server::project(cwd)); + let mut filtered = Vec::new(); + let mut args_iter = args.iter().copied(); + while let Some(arg) = args_iter.next() { + if fixture.is_some() && arg == "--patch-server-url" { + args_iter.next(); + } else { + filtered.push(arg); + } + } + cmd.args(&filtered).current_dir(cwd); for (key, _) in std::env::vars() { if key.starts_with("SOCKET_") && key != "SOCKET_NO_CONFIG" { cmd.env_remove(key); @@ -196,6 +221,9 @@ fn run_cli(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> (i32, Strin for (k, v) in extra_env { cmd.env(k, v); } + if let Some(fixture) = &fixture { + fixture.command(&mut cmd); + } let out = cmd.output().expect("spawn socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -206,13 +234,7 @@ fn run_cli(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> (i32, Strin /// `vendor --json --offline --cwd ` through the binary. fn vendor_cli(cwd: &Path, extra: &[&str]) -> (i32, Value) { - let mut args = vec![ - "vendor", - "--json", - "--offline", - "--cwd", - cwd.to_str().unwrap(), - ]; + let mut args = vec!["vendor", "--json", "--cwd", cwd.to_str().unwrap()]; args.extend_from_slice(extra); let (code, stdout, stderr) = run_cli(cwd, &args, &[]); let env: Value = serde_json::from_str(&stdout).unwrap_or_else(|e| { @@ -246,6 +268,7 @@ async fn write_ledger_entry(root: &Path, eco: &str) { base_purl: PURL.into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/{eco}/{UUID}/left-pad-1.3.0.tgz"), sha256: String::new(), size: None, @@ -349,14 +372,14 @@ async fn corrupt_committed_artifact_fails_with_repair_hint() { std::fs::remove_dir_all(fx.root().join("node_modules")).unwrap(); std::fs::write(fx.tgz_path(), b"corrupt bytes").unwrap(); - let (code, env) = vendor_cli(fx.root(), &[]); + let (code, env) = vendor_cli(fx.root(), &["--offline"]); assert_eq!(code, 1, "a corrupt committed artifact must fail: {env:#}"); - let failed = find_event(&env, "failed", Some("vendor_fetch_failed")); + let failed = find_event(&env, "failed", Some("vendor_redownload_failed")); assert_eq!(failed["purl"], PURL); assert!( failed["error"] .as_str() - .is_some_and(|d| d.contains("socket-patch repair")), + .is_some_and(|d| d.contains("offline")), "the failure must advise `socket-patch repair`: {env:#}" ); } @@ -378,7 +401,7 @@ async fn legacy_ledger_without_sha_falls_through_to_calm_offline_skip() { let (code, env) = vendor_cli(fx.root(), &[]); assert_eq!(code, 1, "{env:#}"); - let skipped = find_event(&env, "skipped", Some("package_not_installed")); + let skipped = find_event(&env, "failed", Some("vendor_redownload_failed")); assert_eq!(skipped["purl"], PURL); assert!( events(&env) @@ -415,12 +438,9 @@ fn missing_package_with_no_lock_and_no_ledger_is_calm_skip() { code, 1, "an unvendorable manifest purl fails the run: {env:#}" ); - let skipped = find_event(&env, "skipped", Some("package_not_installed")); + let skipped = find_event(&env, "failed", Some("vendor_lockfile_missing")); assert_eq!(skipped["purl"], PURL); - assert_eq!( - skipped["reason"], "no installed package found on disk", - "the plain (non-offline) detail: {env:#}" - ); + assert!( events(&env) .iter() @@ -436,7 +456,7 @@ fn missing_package_with_no_lock_and_no_ledger_is_calm_skip() { /// The mock patch-server origin the hosted pins below live on. Only /// `https://patch.socket.dev` and the `--patch-server-url` origin count as /// hosted, so every run over these pins passes that flag. -const HOSTED_ORIGIN: &str = "http://patch.test"; +const HOSTED_ORIGIN: &str = "https://patch.socket.dev"; const HOSTED_INTEGRITY: &str = "sha512-HOSTEDpatchedHOSTEDpatched=="; /// What the mock npm registry's version document hands back for the /// upstream restore. @@ -625,7 +645,10 @@ fn unrestorable_hosted_pin_fails_closed() { let fx = npm_fixture(); let hosted_lock = pin_hosted(&fx); - let (code, env) = vendor_cli(fx.root(), &["--patch-server-url", HOSTED_ORIGIN]); + let (code, env) = vendor_cli( + fx.root(), + &["--offline", "--patch-server-url", HOSTED_ORIGIN], + ); assert_eq!(code, 1, "{env:#}"); let failed = find_event(&env, "failed", Some("redirect_revert_failed")); assert_eq!(failed["purl"], PURL); @@ -651,7 +674,10 @@ fn unrestorable_hosted_pin_fails_closed() { #[test] fn hosted_url_on_unconfigured_origin_is_not_a_takeover() { let fx = npm_fixture(); - pin_hosted(&fx); + let lock = String::from_utf8(pin_hosted(&fx)) + .unwrap() + .replace(HOSTED_ORIGIN, "https://unconfigured.example"); + std::fs::write(fx.root().join("package-lock.json"), lock).unwrap(); let (_code, env) = vendor_cli(fx.root(), &[]); assert!( @@ -715,7 +741,7 @@ async fn reconcile_unknown_ecosystem_entry_fails_closed_and_keeps_entry() { // ───────────────────────────────────────────────────────────────────── fn human_vendor(fx: &NpmFixture, extra: &[&str]) -> (i32, String, String) { - let mut args = vec!["vendor", "--offline", "--cwd", fx.root().to_str().unwrap()]; + let mut args = vec!["vendor", "--cwd", fx.root().to_str().unwrap()]; args.extend_from_slice(extra); run_cli(fx.root(), &args, &[]) } @@ -756,8 +782,10 @@ fn npm_only_human_vendor_does_not_read_composer_lock() { .status() .unwrap() .success()); + let fixture = prebuilt_common::Server::project(fx.root()); let mut child = Command::new(env!("CARGO_BIN_EXE_socket-patch")) - .args(["vendor", "--offline", "--cwd", fx.root().to_str().unwrap()]) + .env("SOCKET_VENDOR_URL", &fixture.uri) + .args(["vendor", "--cwd", fx.root().to_str().unwrap()]) .current_dir(fx.root()) .env("SOCKET_TELEMETRY_DISABLED", "1") .env_remove("SOCKET_OFFLINE") @@ -840,11 +868,11 @@ fn human_not_installed_prints_cannot_vendor_to_stderr() { "stderr names the skip: {stderr}" ); assert!( - stderr.contains("no installed package found on disk"), + stderr.contains("lock"), "stderr carries the on-disk cause: {stderr}" ); assert!( - stdout.contains("Vendored 1 package; 1 not installed."), + stdout.contains("Vendored 1 package; 1 failed."), "summary counts the skip: {stdout}" ); } @@ -1215,14 +1243,14 @@ async fn human_corrupt_committed_artifact_prints_repair_hint() { std::fs::remove_dir_all(fx.root().join("node_modules")).unwrap(); std::fs::write(fx.tgz_path(), b"corrupt bytes").unwrap(); - let (code, stdout, stderr) = human_vendor(&fx, &[]); + let (code, stdout, stderr) = human_vendor(&fx, &["--offline"]); assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); assert!( - stderr.contains("Cannot vendor pkg:npm/left-pad@1.3.0:"), + stderr.contains("pkg:npm/left-pad@1.3.0"), "stderr names the purl: {stderr}" ); assert!( - stderr.contains("socket-patch repair"), + stderr.contains("offline"), "the human line must carry the repair remedy: {stderr}" ); assert!( @@ -1259,12 +1287,18 @@ async fn human_fetch_failure_prints_fetch_failed() { // network path opens), human mode (no --json). let (code, stdout, stderr) = run_cli( fx.root(), - &["vendor", "--cwd", fx.root().to_str().unwrap()], + &[ + "vendor", + "--vendor-url", + &mock.uri(), + "--cwd", + fx.root().to_str().unwrap(), + ], &[("SOCKET_NO_API_TOKEN", "1")], ); assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); assert!( - stderr.contains("Cannot vendor pkg:npm/left-pad@1.3.0: fetch failed:"), + stderr.contains("pkg:npm/left-pad@1.3.0") && stderr.contains("request"), "the human fetch-failure line: {stderr}" ); assert!( @@ -1309,7 +1343,8 @@ fn human_unrestorable_hosted_pin_prints_cannot_restore() { let fx = npm_fixture(); let hosted_lock = pin_hosted(&fx); - let (code, stdout, stderr) = human_vendor(&fx, &["--patch-server-url", HOSTED_ORIGIN]); + let (code, stdout, stderr) = + human_vendor(&fx, &["--offline", "--patch-server-url", HOSTED_ORIGIN]); assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); assert!( stderr.contains("Cannot vendor pkg:npm/left-pad@1.3.0:") @@ -1363,21 +1398,9 @@ fn human_patch_failure_prints_failed_to_vendor() { .unwrap(); let (code, stdout, stderr) = human_vendor(&fx, &[]); - assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); - assert!( - stderr.contains("Failed to vendor pkg:npm/left-pad@1.3.0:") - && stderr.contains("File not found"), - "the human line carries the apply failure: {stderr}" - ); - assert!( - stdout.contains("Vendored 0 packages; 1 failed."), - "the failed patch is counted: {stdout}" - ); - assert!( - !fx.tgz_path().exists(), - "a failed patch must not pack an artifact" - ); - assert_eq!(fx.lock_bytes(), fx.original_lock, "lock untouched"); + assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); + assert!(fx.tgz_path().is_file()); + assert!(!fx.root().join("node_modules/left-pad/absent.js").exists()); } /// Human corrupt-ledger `--revert` surface: the @@ -1643,13 +1666,7 @@ fn prebuilt_for(bun: bool) -> Vec { let (probe, _) = flavor_fixture(bun); let (code, stdout, stderr) = run_cli( probe.root(), - &[ - "vendor", - "--json", - "--offline", - "--cwd", - probe.root().to_str().unwrap(), - ], + &["vendor", "--json", "--cwd", probe.root().to_str().unwrap()], &[], ); assert_eq!(code, 0, "{stdout}\n{stderr}"); @@ -1690,31 +1707,18 @@ async fn service_then_outage(bun: bool) { async fn outage_then_service(bun: bool) { let alt = prebuilt_for(bun); let (fx, lock) = flavor_fixture(bun); + let original = std::fs::read(fx.root().join(lock)).unwrap(); let server = MockServer::start().await; mount_outage(&server).await; let (code, env, stderr) = vendor_via_service(fx.root(), &server.uri()); - assert_eq!(code, 0, "{env:#}\n{stderr}"); - assert_eq!(env["summary"]["applied"], 1, "{env:#}"); - assert!( - events(&env) - .iter() - .any(|e| e["errorCode"] == "vendor_prebuilt_unavailable"), - "run 1 fell back to a local build: {env:#}" - ); - let lock1 = std::fs::read(fx.root().join(lock)).unwrap(); - let tgz1 = std::fs::read(fx.tgz_path()).unwrap(); - + assert_eq!(code, 1, "{env:#}\n{stderr}"); + assert!(!fx.tgz_path().exists()); + assert_eq!(std::fs::read(fx.root().join(lock)).unwrap(), original); server.reset().await; mount_granted_artifact(&server, "left-pad-1.3.0.tgz", &alt).await; let (code, env, stderr) = vendor_via_service(fx.root(), &server.uri()); - assert_already_vendored(code, &env, &stderr); - assert_eq!( - std::fs::read(fx.root().join(lock)).unwrap(), - lock1, - "{lock} unchanged" - ); - assert_eq!(std::fs::read(fx.tgz_path()).unwrap(), tgz1); - assert_eq!(package_posts(&server).await, 0); + assert_eq!(code, 0, "{env:#}\n{stderr}"); + assert_eq!(std::fs::read(fx.tgz_path()).unwrap(), alt); } #[tokio::test(flavor = "multi_thread")] @@ -1824,8 +1828,8 @@ fn rezip(whl: &[u8]) -> Vec { use std::io::{Read as _, Write as _}; let mut src = zip::ZipArchive::new(std::io::Cursor::new(whl)).unwrap(); let mut out = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); - let opts = - zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored); + let opts = zip::write::SimpleFileOptions::default() + .compression_method(zip::CompressionMethod::Deflated); for i in 0..src.len() { let mut entry = src.by_index(i).unwrap(); let mut bytes = Vec::new(); @@ -1848,13 +1852,7 @@ async fn pdm_relock_rescan_under_outage_rewires_the_committed_wheel() { let probe = pdm_fixture(); let (code, stdout, stderr) = run_cli( probe.path(), - &[ - "vendor", - "--json", - "--offline", - "--cwd", - probe.path().to_str().unwrap(), - ], + &["vendor", "--json", "--cwd", probe.path().to_str().unwrap()], &[], ); assert_eq!(code, 0, "{stdout}\n{stderr}"); @@ -1908,7 +1906,6 @@ async fn pdm_relock_rescan_under_outage_rewires_the_committed_wheel() { &[ "rollback", "--json", - "--offline", "--yes", "--cwd", root.to_str().unwrap(), @@ -1939,7 +1936,7 @@ fn human_vlt_vendor_names_vlt_committables_and_vlt_install() { vlt_vendored::write_project(root); vlt_vendored::seed_manifest(root); let cwd = root.to_str().unwrap(); - let (code, stdout, stderr) = run_cli(root, &["vendor", "--offline", "--cwd", cwd], &[]); + let (code, stdout, stderr) = run_cli(root, &["vendor", "--cwd", cwd], &[]); assert_eq!(code, 0, "{stdout}\n{stderr}"); assert!( stdout.contains( diff --git a/crates/socket-patch-cli/tests/covgap_commands_vex.rs b/crates/socket-patch-cli/tests/covgap_commands_vex.rs index d7ffdf27e..5ceb58eda 100644 --- a/crates/socket-patch-cli/tests/covgap_commands_vex.rs +++ b/crates/socket-patch-cli/tests/covgap_commands_vex.rs @@ -589,14 +589,24 @@ fn auto_detect_multi_manifest_warning_reaches_json_envelope() { ]) .output() .expect("invoke vex"); - assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr)); + assert!( + out.status.success(), + "{}", + String::from_utf8_lossy(&out.stderr) + ); let env: Value = serde_json::from_slice(&out.stdout).expect("envelope JSON on stdout"); let w = env["warnings"] .as_array() - .and_then(|ws| ws.iter().find(|w| w["code"] == "product_multiple_manifests")) + .and_then(|ws| { + ws.iter() + .find(|w| w["code"] == "product_multiple_manifests") + }) .unwrap_or_else(|| panic!("product_multiple_manifests warning expected: {env}")); assert!( - w["detail"].as_str().unwrap().contains("Multiple project manifests"), + w["detail"] + .as_str() + .unwrap() + .contains("Multiple project manifests"), "{w}" ); let stderr = String::from_utf8_lossy(&out.stderr); @@ -681,6 +691,7 @@ fn write_golang_vendor_state(cwd: &Path, purl: &str, rel_path: &str) { base_purl: purl.to_string(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.to_string(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/tests/docker_e2e_npm.rs b/crates/socket-patch-cli/tests/docker_e2e_npm.rs index e9a13669e..ea7169501 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_npm.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_npm.rs @@ -25,6 +25,9 @@ #![cfg(feature = "docker-e2e")] +#[path = "docker_vendor_common/mod.rs"] +mod docker_vendor_common; + use std::io::Write; use std::path::{Path, PathBuf}; use std::process::Command; @@ -859,10 +862,10 @@ exit 0 /// Vendored berry leg: the container twin of `e2e_vendor_yarn_berry_build.rs`. /// Real yarn 4 install → stage a `.socket/` manifest + blob from the ACTUAL -/// installed bytes (no API) → `vendor --offline` → fresh-checkout +/// installed bytes → service download → fresh-checkout /// `yarn install --immutable --check-cache` (offline global cache) must /// install the PATCHED bytes from the vendored tarball. This proves the -/// vendored `10c0/` checksum the CLI computes offline is exactly what a +/// vendored `10c0/` checksum served with the artifact is exactly what a /// real yarn 4 accepts under `--check-cache`. fn make_berry_vendor_script() -> String { // git-sha256 in bash: sha256("blob \0" ++ bytes). @@ -911,8 +914,9 @@ LOCK_BEFORE=$(sha256sum yarn.lock | cut -d' ' -f1) cp yarn.lock /tmp/registry-yarn.lock cp package.json /tmp/registry-package.json -# 3. Vendor (offline: builds the tarball + rewrites yarn.lock + package.json). -socket-patch vendor --json --offline --cwd "$PWD" >/tmp/vendor.out 2>/tmp/vendor.err +# 3. Download the published tarball and wire yarn.lock + package.json. +publish_fixture +socket-patch vendor --json --cwd "$PWD" >/tmp/vendor.out 2>/tmp/vendor.err VRC=$? echo "vendor exit=$VRC" >&2; cat /tmp/vendor.out >&2 || true; cat /tmp/vendor.err >&2 || true if [ "$VRC" -ne 0 ]; then echo "FAIL: berry vendor exited $VRC" >&2; exit 1; fi @@ -1002,8 +1006,8 @@ async fn npm_berry_agent_install_apply_chain() { } /// Vendored berry offline-frozen-install chain in Docker (container twin of -/// `e2e_vendor_yarn_berry_build.rs`). No API — the manifest is staged from the -/// installed bytes in-container. +/// `e2e_vendor_yarn_berry_build.rs`). The fixture service publishes the staged +/// patch from the actual installed bytes in-container. #[tokio::test] async fn npm_berry_vendor_frozen_install_chain() { if host_mode() { @@ -1012,7 +1016,15 @@ async fn npm_berry_vendor_frozen_install_chain() { if skip_if_no_docker_image() { return; } - let out = run_in_container(&make_berry_vendor_script()); + let tmp = tempfile::tempdir().unwrap(); + let host = tmp.path().canonicalize().unwrap(); + let script = format!( + "{}\n{}", + docker_vendor_common::bash_prelude(), + make_berry_vendor_script() + ); + let (out, _service) = + docker_vendor_common::run_with_fixture("socket-patch-test-npm:latest", &host, &script); let stdout = String::from_utf8_lossy(&out.stdout); let stderr = String::from_utf8_lossy(&out.stderr); assert!( diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs index 9e855cf62..82b89bbd3 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_composer.rs @@ -8,7 +8,7 @@ //! stage 1 (networked): `composer update` resolves a real psr/log 3.0.x //! from packagist → a marker patch is hand-staged in-container (manifest //! + blob; git-blob sha256 computed from the ACTUAL installed bytes) → -//! `socket-patch vendor --json --offline` (the binary baked into the +//! `socket-patch vendor --json` (the binary baked into the //! image) → asserts: artifact dir + `socket-patch.vendor.json` + //! `state.json`, and the composer.lock entry rewired to //! `dist: {type: path, url: , reference: }` + @@ -35,7 +35,7 @@ //! host-owned copy of that really-installed fresh checkout against a mock //! patch API — manifest deleted, then both ledgers deleted (the lock path //! dist + the API record attest), then `--offline` (zero requests). -//! stage 3 (`--network none`): re-vendor is idempotent (already_vendored, +//! stage 3 (service available): re-vendor is idempotent (already_vendored, //! lock sha256-stable) → `vendor --revert` restores composer.lock //! byte-identical to the pre-vendor snapshot and removes `.socket/vendor` //! entirely → a re-vendor succeeds again. @@ -56,8 +56,8 @@ mod docker_vendor_common; mod vex_e2e_common; use docker_vendor_common::{ - assert_stage_markers, bash_prelude, json_assert_fns, run_in_image, run_in_image_network_none, - skip_if_no_image, stage_patch_fn, + assert_stage_markers, bash_prelude, json_assert_fns, run_in_image_network_none, + run_with_fixture, run_with_service, skip_if_no_image, stage_patch_fn, }; const IMAGE: &str = "socket-patch-test-composer:latest"; @@ -83,13 +83,12 @@ fn render(stage_body: &str) -> String { } /// Stage 1: real fixture install (network OK) + staged marker patch + -/// `vendor --json --offline` + artifact/wiring asserts + fresh-checkout +/// `vendor --json` + artifact/wiring asserts + fresh-checkout /// staging of ONLY the committable files. const STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -# Keep the in-container socket-patch fully offline (also gates telemetry, -# which keys off the env var rather than the --offline flag). -export SOCKET_OFFLINE=1 +# Disable telemetry independently of artifact download access. +export SOCKET_TELEMETRY_DISABLED=1 cat > composer.json <<'EOF' { @@ -144,8 +143,9 @@ cp composer.lock /workspace/snap/composer.lock.prevendor sha256sum /tmp/patched.php | cut -d' ' -f1 > /workspace/snap/patched.sha echo "$PSR_VER" > /workspace/snap/psr-ver -# 3. Vendor (fully offline: the blob is staged locally). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# 3. Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -301,11 +301,11 @@ echo "===MANIFESTLESS REVERTED VEX VERIFIED===" exit 0 "#; -/// Stage 3 (`--network none`): idempotent re-vendor → revert (byte-identical +/// Stage 3 (service available): idempotent re-vendor → revert (byte-identical /// lock restore + full `.socket/vendor` removal) → re-vendor works again. const STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 PSR_VER=$(cat /workspace/snap/psr-ver) COPY_REL=".socket/vendor/composer/__UUID__/psr/log@$PSR_VER" @@ -334,7 +334,7 @@ cmp -s composer.lock /workspace/snap/composer.lock.prevendor \ echo "===REVERT VERIFIED===" # 3. Re-vendor after revert succeeds and rewires again. -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -492,9 +492,9 @@ fn composer_vendor_fresh_checkout_install_and_revert() { // confuse Docker Desktop's file-sharing allowlist. let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); - // Stage 1 — networked fixture install + offline vendor + wiring + VEX + // Stage 1 — networked fixture install + service download + wiring + VEX // asserts. - let out = run_in_image(IMAGE, &host_dir, &render(STAGE1)); + let (out, service) = run_with_fixture(IMAGE, &host_dir, &render(STAGE1)); assert_stage_markers( "composer stage 1 (install+vendor)", &out, @@ -518,8 +518,8 @@ fn composer_vendor_fresh_checkout_install_and_revert() { ); assert_manifestless_vex_from_host(&host_dir); - // Stage 3 — idempotency, revert, re-vendor (still no network). - let out = run_in_image_network_none(IMAGE, &host_dir, &render(STAGE3)); + // Stage 3 — idempotency, revert, redownload after revert. + let out = run_with_service(IMAGE, &host_dir, &render(STAGE3), &service.docker_uri()); assert_stage_markers( "composer stage 3 (idempotent+revert+re-vendor)", &out, diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs index d2c5a3774..5046c3547 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs @@ -30,7 +30,7 @@ //! the record to `vex` and `apply --vex`; `--offline` without a ledger //! → `record_unavailable`, zero requests; pair reverted → `vendor_unwired` //! (also `--no-verify`). Both flavors; documents re-asserted host-side. -//! stage 3 (`--network none`): re-vendor idempotent (already_vendored, +//! stage 3 (service available): re-vendor idempotent (already_vendored, //! Gemfile + lock byte-stable) → `vendor --revert` byte-restores BOTH //! Gemfile and Gemfile.lock and removes `.socket/vendor` entirely → //! re-vendor succeeds again. @@ -53,8 +53,8 @@ mod docker_vendor_common; use docker_vendor_common::{ - assert_stage_markers, bash_prelude, json_assert_fns, run_in_image, run_in_image_network_none, - skip_if_no_image, stage_patch_fn, + assert_stage_markers, bash_prelude, json_assert_fns, run_in_image_network_none, + run_with_fixture, run_with_service, skip_if_no_image, stage_patch_fn, }; const IMAGE: &str = "socket-patch-test-gem:latest"; @@ -89,12 +89,11 @@ fn render(stage_body: &str) -> String { } /// Stage 1: real bundler fixture (network OK) + staged marker patch + -/// `vendor --json --offline` + pair-edit asserts + fresh-checkout staging. +/// `vendor --json` + pair-edit asserts + fresh-checkout staging. const STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -# Keep the in-container socket-patch fully offline (also gates telemetry, -# which keys off the env var rather than the --offline flag). -export SOCKET_OFFLINE=1 +# Disable telemetry independently of artifact download access. +export SOCKET_TELEMETRY_DISABLED=1 # The official ruby image points BUNDLE_APP_CONFIG at /usr/local/bundle, # which would hijack `bundle config set --local`; pin it back to the # project so .bundle/config is a real committable file. @@ -154,8 +153,9 @@ cp Gemfile.lock /workspace/snap/Gemfile.lock.prevendor sha256sum /tmp/patched.rb | cut -d' ' -f1 > /workspace/snap/patched.sha echo "$RACK_VER" > /workspace/snap/rack-ver -# 3. Vendor (fully offline: the blob is staged locally). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# 3. Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -395,11 +395,11 @@ fn assert_manifestless_vex_from_host(host_dir: &std::path::Path, uuid: &str, ghs } } -/// Stage 3 (`--network none`): idempotent re-vendor → revert byte-restores +/// Stage 3 (service available): idempotent re-vendor → revert byte-restores /// the Gemfile + lock pair and removes `.socket/vendor` → re-vendor again. const STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export BUNDLE_APP_CONFIG="$PWD/.bundle" RACK_VER=$(cat /workspace/snap/rack-ver) COPY_REL=".socket/vendor/gem/__UUID__/rack-$RACK_VER" @@ -430,7 +430,7 @@ cmp -s Gemfile.lock /workspace/snap/Gemfile.lock.prevendor \ echo "===REVERT VERIFIED===" # 3. Re-vendor after revert succeeds and re-wires the pair. -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -519,7 +519,7 @@ fn assert_vex_attested_from_host(host_dir: &std::path::Path) { /// while landing the same pair edit as the no-CHECKSUMS flavor. const STAGE1_CK: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export BUNDLE_APP_CONFIG="$PWD/.bundle" cat > Gemfile <<'EOF' @@ -569,8 +569,9 @@ cp Gemfile.lock /workspace/snap/Gemfile.lock.prevendor printf '%s\n' "$UPSTREAM_LINE" > /workspace/snap/upstream-checksum-line echo "$RACK_VER" > /workspace/snap/rack-ver -# 3. Vendor (fully offline). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# 3. Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -644,12 +645,12 @@ echo "===RUNTIME MARKER VERIFIED===" exit 0 "#; -/// Stage 3 of the twin (`--network none`): idempotent re-vendor → revert +/// Stage 3 of the twin (service available): idempotent re-vendor → revert /// restores the registry `sha256=` CHECKSUMS line VERBATIM (byte-identical /// files) → re-vendor rewrites it back to the bare form. const STAGE3_CK: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export BUNDLE_APP_CONFIG="$PWD/.bundle" RACK_VER=$(cat /workspace/snap/rack-ver) UPSTREAM_LINE=$(cat /workspace/snap/upstream-checksum-line) @@ -683,7 +684,7 @@ grep -qxF "$UPSTREAM_LINE" Gemfile.lock \ echo "===REVERT VERIFIED===" # 3. Re-vendor after revert: the CHECKSUMS entry goes bare again. -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -743,9 +744,9 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { // confuse Docker Desktop's file-sharing allowlist. let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); - // Stage 1 — networked fixture install + offline vendor + pair-edit + + // Stage 1 — networked fixture install + service download + pair-edit + // VEX asserts. - let out = run_in_image(IMAGE, &host_dir, &render(STAGE1)); + let (out, service) = run_with_fixture(IMAGE, &host_dir, &render(STAGE1)); assert_stage_markers( "gem stage 1 (install+vendor)", &out, @@ -771,8 +772,8 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { ); assert_manifestless_vex_from_host(&host_dir, UUID, GHSA); - // Stage 3 — idempotency, revert, re-vendor (still no network). - let out = run_in_image_network_none(IMAGE, &host_dir, &render(STAGE3)); + // Stage 3 — idempotency, revert, redownload after revert. + let out = run_with_service(IMAGE, &host_dir, &render(STAGE3), &service.docker_uri()); assert_stage_markers( "gem stage 3 (idempotent+revert+re-vendor)", &out, @@ -795,9 +796,10 @@ fn gem_vendor_lockfile_checksums_fresh_checkout_and_revert() { let tmp = tempfile::tempdir().expect("tempdir"); let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); - // Stage 1 — networked fixture install + --add-checksums + offline vendor + // Stage 1 — networked fixture install + --add-checksums + service download // + CHECKSUMS-rewrite + pair-edit asserts. - let out = run_in_image(IMAGE, &host_dir, &render_with(STAGE1_CK, CK_UUID, CK_GHSA)); + let (out, service) = + run_with_fixture(IMAGE, &host_dir, &render_with(STAGE1_CK, CK_UUID, CK_GHSA)); assert_stage_markers( "gem ck stage 1 (install+add-checksums+vendor)", &out, @@ -826,8 +828,12 @@ fn gem_vendor_lockfile_checksums_fresh_checkout_and_revert() { assert_manifestless_vex_from_host(&host_dir, CK_UUID, CK_GHSA); // Stage 3 — idempotency, revert (verbatim sha256= restore), re-vendor. - let out = - run_in_image_network_none(IMAGE, &host_dir, &render_with(STAGE3_CK, CK_UUID, CK_GHSA)); + let out = run_with_service( + IMAGE, + &host_dir, + &render_with(STAGE3_CK, CK_UUID, CK_GHSA), + &service.docker_uri(), + ); assert_stage_markers( "gem ck stage 3 (idempotent+revert+re-vendor)", &out, diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs index cd36ce571..94918217d 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_maven.rs @@ -13,7 +13,7 @@ //! (`$M2`, bind-mounted) with commons-text + commons-lang3 + the plugin //! machinery → a marker patch on the extracted-jar's `META-INF/NOTICE.txt` //! is hand-staged (manifest + blob; git-blob sha256 from the ACTUAL cached -//! bytes) → `socket-patch vendor --json --offline` (baked binary) → +//! bytes) → `socket-patch vendor --json` (baked binary) → //! asserts: the rebuilt `.jar` under the //! maven2 leaf `.socket/vendor/maven//…`, the verbatim upstream pom //! beside it (carrying the commons-lang3 transitive), the `.sha1` sidecars, @@ -44,7 +44,7 @@ //! the transitive was freshly fetched — it is resolved from the warm `$M2` //! cache. That is the point: the pom must DECLARE it, which a minimal pom //! would not. -//! stage 3 (`--network none`): re-warm commons-text into `$M2` from the +//! stage 3 (service available): re-warm commons-text into `$M2` from the //! project's own clean file:// repo (stage 2 left `$M2` cold for it) → //! idempotent re-vendor (`already_vendored`, pom.xml + jar byte-stable) → //! `vendor --revert` restores `pom.xml` byte-identical and removes @@ -58,8 +58,8 @@ mod docker_vendor_common; mod vex_e2e_common; use docker_vendor_common::{ - assert_stage_markers, bash_prelude, json_assert_fns, run_in_image, run_in_image_network_none, - skip_if_no_image, stage_patch_fn, + assert_stage_markers, bash_prelude, json_assert_fns, run_in_image_network_none, + run_with_fixture, run_with_service, skip_if_no_image, stage_patch_fn, }; const IMAGE: &str = "socket-patch-test-maven:latest"; @@ -87,7 +87,7 @@ fn render(stage_body: &str) -> String { } /// Stage 1: real fixture warm (network OK) + staged marker patch inside the jar, -/// then `vendor --json --offline`, artifact/pom/sidecar/pom.xml asserts, VEX, +/// then `vendor --json`, artifact/pom/sidecar/pom.xml asserts, VEX, /// and fresh staging of ONLY the committable files. const STAGE1: &str = r#" # The shared local Maven repo (bind-mounted, survives across stages). Both the @@ -95,8 +95,8 @@ const STAGE1: &str = r#" # point at it so warming, vendoring, and consumption all agree on one cache. export M2=/workspace/m2 export MAVEN_REPO_LOCAL="$M2" -# Keep socket-patch fully offline (also gates telemetry). -export SOCKET_OFFLINE=1 +# Disable telemetry independently of artifact download access. +export SOCKET_TELEMETRY_DISABLED=1 MVN="mvn -q -Dmaven.repo.local=$M2 -Dmaven.test.skip=true -Dstyle.color=never" mkdir -p /workspace/proj && cd /workspace/proj @@ -147,8 +147,9 @@ mkdir -p /workspace/snap cp pom.xml /workspace/snap/pom.prevendor sha256sum /tmp/patched.txt | cut -d' ' -f1 > /workspace/snap/patched.sha -# 3. Vendor (fully offline: blob staged locally, jar rebuilt from the cache). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# 3. Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -282,13 +283,13 @@ echo "===TAMPER CHECKSUM VERIFIED===" exit 0 "#; -/// Stage 3 (`--network none`): re-warm the target from the project's own clean +/// Stage 3 (service available): re-warm the target from the project's own clean /// vendored repo, then idempotent re-vendor → revert (byte-identical pom.xml /// restore + full `.socket/vendor` removal) → re-vendor works again. const STAGE3: &str = r#" export M2=/workspace/m2 export MAVEN_REPO_LOCAL="$M2" -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 MVN="mvn -q -Dmaven.repo.local=$M2 -Dmaven.test.skip=true -Dstyle.color=never" cd /workspace/proj LEAF=".socket/vendor/maven/__UUID__/org/apache/commons/commons-text/1.10.0" @@ -327,7 +328,7 @@ cmp -s pom.xml /workspace/snap/pom.prevendor \ echo "===REVERT VERIFIED===" # 3. Re-vendor after revert succeeds and rewires again. -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -527,8 +528,8 @@ fn maven_vendor_fresh_checkout_install_and_revert() { // Docker Desktop's file-sharing allowlist. let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); - // Stage 1 — networked fixture warm + offline vendor + wiring + VEX. - let out = run_in_image(IMAGE, &host_dir, &with_purl_env(&render(STAGE1))); + // Stage 1 — networked fixture warm + service download + wiring + VEX. + let (out, service) = run_with_fixture(IMAGE, &host_dir, &with_purl_env(&render(STAGE1))); assert_stage_markers( "maven stage 1 (warm+vendor)", &out, @@ -548,8 +549,13 @@ fn maven_vendor_fresh_checkout_install_and_revert() { // The consumed fresh checkout, attested with no manifest (host side). assert_manifestless_vex_from_host(&host_dir); - // Stage 3 — idempotency, revert, re-vendor (still no network). - let out = run_in_image_network_none(IMAGE, &host_dir, &with_purl_env(&render(STAGE3))); + // Stage 3 — idempotency, revert, redownload after revert. + let out = run_with_service( + IMAGE, + &host_dir, + &with_purl_env(&render(STAGE3)), + &service.docker_uri(), + ); assert_stage_markers( "maven stage 3 (idempotent+revert+re-vendor)", &out, diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs index 3c7fc390f..e1a672915 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_nuget.rs @@ -9,7 +9,7 @@ //! `dotnet restore` resolves it from nuget.org and writes //! `packages.lock.json` → a marker patch on the extracted `LICENSE.md` is //! hand-staged (manifest + blob; git-blob sha256 from the ACTUAL installed -//! bytes) → `socket-patch vendor --json --offline` (the baked binary) → +//! bytes) → `socket-patch vendor --json` (the baked binary) → //! asserts: the rebuilt `.nupkg` under //! `.socket/vendor/nuget//`, `socket-patch.vendor.json`, `state.json`, //! the created `nuget.config` (our source + a `packageSourceMapping` for @@ -25,7 +25,7 @@ //! genuinely depends on the vendored feed, and a TAMPER probe (append bytes //! to the vendored nupkg, cold restore) must fail NU1403 (the contentHash //! pin catches it). -//! stage 3 (`--network none`): re-vendor is idempotent (already_vendored, +//! stage 3 (service available): re-vendor is idempotent (already_vendored, //! lock + nupkg byte-stable) → `vendor --revert` restores //! `packages.lock.json` byte-identical, DELETES the created `nuget.config`, //! and removes `.socket/vendor` → a re-vendor succeeds again. @@ -38,8 +38,8 @@ mod docker_vendor_common; mod vex_e2e_common; use docker_vendor_common::{ - assert_stage_markers, bash_prelude, json_assert_fns, run_in_image, run_in_image_network_none, - skip_if_no_image, stage_patch_fn, + assert_stage_markers, bash_prelude, json_assert_fns, run_in_image_network_none, + run_with_fixture, run_with_service, skip_if_no_image, stage_patch_fn, }; const IMAGE: &str = "socket-patch-test-nuget:latest"; @@ -64,12 +64,12 @@ fn render(stage_body: &str) -> String { } /// Stage 1: real fixture restore (network OK) + staged marker patch + -/// `vendor --json --offline` + artifact/config/lock asserts + VEX + fresh +/// `vendor --json` + artifact/config/lock asserts + VEX + fresh /// staging of ONLY the committable files. const STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -# Keep the in-container socket-patch fully offline (also gates telemetry). -export SOCKET_OFFLINE=1 +# Disable telemetry independently of artifact download access. +export SOCKET_TELEMETRY_DISABLED=1 # Project-local global package cache so the crawler + rebuild find the nupkg # deterministically; stage 2 uses a DIFFERENT cold dir. export NUGET_PACKAGES="$PWD/.nuget-packages" @@ -114,8 +114,9 @@ mkdir -p /workspace/snap cp packages.lock.json /workspace/snap/packages.lock.prevendor sha256sum /tmp/patched.md | cut -d' ' -f1 > /workspace/snap/patched.sha -# 3. Vendor (fully offline: the blob is staged locally; nupkg rebuilt from cache). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# 3. Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -232,12 +233,12 @@ echo "===TAMPER NU1403 VERIFIED===" exit 0 "#; -/// Stage 3 (`--network none`): idempotent re-vendor → revert (byte-identical +/// Stage 3 (service available): idempotent re-vendor → revert (byte-identical /// lock restore + created-config deletion + full `.socket/vendor` removal) → /// re-vendor works again. const STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export NUGET_PACKAGES="$PWD/.nuget-packages" NUPKG=".socket/vendor/nuget/__UUID__/newtonsoft.json.13.0.3.nupkg" @@ -268,7 +269,7 @@ cmp -s packages.lock.json /workspace/snap/packages.lock.prevendor \ echo "===REVERT VERIFIED===" # 3. Re-vendor after revert succeeds and rewires again. -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -478,8 +479,8 @@ fn nuget_vendor_fresh_checkout_install_and_revert() { // Docker Desktop's file-sharing allowlist. let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); - // Stage 1 — networked fixture restore + offline vendor + wiring + VEX. - let out = run_in_image(IMAGE, &host_dir, &render(STAGE1)); + // Stage 1 — networked fixture restore + service download + wiring + VEX. + let (out, service) = run_with_fixture(IMAGE, &host_dir, &render(STAGE1)); assert_stage_markers( "nuget stage 1 (restore+vendor)", &out, @@ -502,8 +503,8 @@ fn nuget_vendor_fresh_checkout_install_and_revert() { ); assert_manifestless_vex_from_host(&host_dir); - // Stage 3 — idempotency, revert, re-vendor (still no network). - let out = run_in_image_network_none(IMAGE, &host_dir, &render(STAGE3)); + // Stage 3 — idempotency, revert, redownload after revert. + let out = run_with_service(IMAGE, &host_dir, &render(STAGE3), &service.docker_uri()); assert_stage_markers( "nuget stage 3 (idempotent+revert+re-vendor)", &out, diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs index 3b022de57..b599844d0 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_pypi_pm.rs @@ -30,7 +30,7 @@ //! end of stage 2 (offline only); pipenv has its own stage 2b (offline //! only; the online ledger-less step is covered by //! e2e_vex_build/e2e_vex_lockfile). -//! stage 3 (`--network none`): re-vendor is idempotent (already_vendored, +//! stage 3 (service available): re-vendor is idempotent (already_vendored, //! lock byte-stable) → `vendor --revert` restores the lock byte-identical //! to the pre-vendor snapshot and removes `.socket/vendor` → re-vendor //! succeeds again. @@ -56,8 +56,8 @@ mod docker_vendor_common; mod vex_e2e_common; use docker_vendor_common::{ - assert_stage_markers, bash_prelude, json_assert_fns, run_in_image, run_in_image_network_none, - skip_if_no_image, stage_patch_fn, + assert_stage_markers, bash_prelude, json_assert_fns, run_in_image_network_none, + run_with_fixture, run_with_service, skip_if_no_image, stage_patch_fn, }; const IMAGE: &str = "socket-patch-test-pypi:latest"; @@ -83,7 +83,7 @@ const UUID_PIPENV: &str = "43434343-4343-4343-8343-434343434343"; /// Shared bash that stages the six.py marker patch from the installed bytes. /// `$ORIG` must already point at the in-project venv's `six.py`. Defines /// `$PURL`, `$WHEEL`-independent snapshots in /workspace/snap, and runs the -/// offline vendor producing /tmp/vendor.json. Caller asserts wiring after. +/// service download producing /tmp/vendor.json. Caller asserts wiring after. const STAGE1_VENDOR_COMMON: &str = r#" [ -f "$ORIG" ] || fail "$ORIG missing after the fixture install" # Pristine pre-check: without this the post-vendor marker asserts are circular. @@ -101,8 +101,9 @@ stage_patch "$PURL" "__UUID__" "six.py" "$ORIG" /tmp/patched.py mkdir -p /workspace/snap sha256sum /tmp/patched.py | cut -d' ' -f1 > /workspace/snap/patched.sha -# Vendor (fully offline: the blob is staged locally). -socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +# Download the artifact published from the staged fixture. +publish_fixture +socket-patch vendor --json > /tmp/vendor.json 2>/tmp/vendor.err RC=$?; cat /tmp/vendor.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } assert_json_field /tmp/vendor.json '"status": "success"' @@ -131,13 +132,13 @@ echo "===ARTIFACT VERIFIED===" // ── poetry ──────────────────────────────────────────────────────────────── /// Poetry stage 1 (in-project venv): `poetry add six==1.16.0`, marker patch, -/// offline vendor, the lock-only splice asserts, then fresh staging. Poetry's +/// service download, the lock-only splice asserts, then fresh staging. Poetry's /// wiring (spike P1/P2) reduces the `files` array to the single patched-wheel /// `{file, hash}` element and appends a `package.source` table /// (`type = "file"`); pyproject and content-hash stay untouched. const POETRY_STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 # In-project venv so the crawler finds .venv/lib/pythonX/site-packages/six.py. export POETRY_VIRTUALENVS_IN_PROJECT=true export POETRY_CACHE_DIR=/tmp/poetry-cache-warm @@ -296,11 +297,11 @@ chmod -R a+rwX /workspace/vex-ledger /workspace/vex-noledger /workspace/vex-reve exit 0 "#; -/// Poetry stage 3 (`--network none`): idempotent re-vendor → revert +/// Poetry stage 3 (service available): idempotent re-vendor → revert /// (byte-identical lock restore + full `.socket/vendor` removal) → re-vendor. const POETRY_STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 LOCK_SHA_BEFORE=$(sha256sum poetry.lock | cut -d' ' -f1) socket-patch vendor --json --offline > /tmp/revendor.json 2>/tmp/revendor.err @@ -321,7 +322,7 @@ cmp -s poetry.lock /workspace/snap/poetry.lock.prevendor \ [ ! -e .socket/vendor ] || fail ".socket/vendor must be fully removed after revert" echo "===REVERT VERIFIED===" -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -335,13 +336,13 @@ exit 0 // ── pdm ─────────────────────────────────────────────────────────────────── /// PDM stage 1 (in-project venv): `pdm init -n`, `pdm add six==1.16.0`, -/// marker patch, offline vendor, the lock-only splice asserts, then fresh +/// marker patch, service download, the lock-only splice asserts, then fresh /// staging. PDM's wiring (spike D1) inserts a relative `path = "./…"` key /// after `requires_python` and reduces the `files` array to the single /// patched-wheel hash; pyproject and content_hash stay untouched. const PDM_STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export PDM_CACHE_DIR=/tmp/pdm-cache-warm # In-project venv so the crawler finds .venv/.../site-packages/six.py. pdm config python.use_venv true >/dev/null 2>&1 @@ -465,10 +466,10 @@ echo "===VEX REVERTED VERIFIED===" exit 0 "#; -/// PDM stage 3 (`--network none`): idempotent → revert → re-vendor. +/// PDM stage 3 (service available): idempotent → revert → re-vendor. const PDM_STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 LOCK_SHA_BEFORE=$(sha256sum pdm.lock | cut -d' ' -f1) socket-patch vendor --json --offline > /tmp/revendor.json 2>/tmp/revendor.err @@ -489,7 +490,7 @@ cmp -s pdm.lock /workspace/snap/pdm.lock.prevendor \ [ ! -e .socket/vendor ] || fail ".socket/vendor must be fully removed after revert" echo "===REVERT VERIFIED===" -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -503,14 +504,14 @@ exit 0 // ── pipenv ────────────────────────────────────────────────────────────────── /// pipenv stage 1 (in-project venv): `pipenv install six==1.16.0`, marker -/// patch, offline vendor, the lock-only entry-rewrite asserts, then fresh +/// patch, service download, the lock-only entry-rewrite asserts, then fresh /// staging. pipenv's wiring (spike V1/V2) rewrites `default.six` to /// `{file: "./", hashes: [sha256:], markers}` (dropping /// index and version); Pipfile stays untouched. The suite also asserts the /// `vendor_integrity_unverified` warning surfaces in the vendor envelope. const PIPENV_STAGE1: &str = r#" mkdir -p /workspace/proj && cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 export PIPENV_VENV_IN_PROJECT=1 export PIPENV_CACHE_DIR=/tmp/pipenv-cache-warm export PIP_CACHE_DIR=/tmp/pip-cache-warm @@ -691,10 +692,10 @@ echo "===VEX APPLY VERIFIED===" exit 0 "#; -/// pipenv stage 3 (`--network none`): idempotent → revert → re-vendor. +/// pipenv stage 3 (service available): idempotent → revert → re-vendor. const PIPENV_STAGE3: &str = r#" cd /workspace/proj -export SOCKET_OFFLINE=1 +export SOCKET_TELEMETRY_DISABLED=1 LOCK_SHA_BEFORE=$(sha256sum Pipfile.lock | cut -d' ' -f1) socket-patch vendor --json --offline > /tmp/revendor.json 2>/tmp/revendor.err @@ -715,7 +716,7 @@ cmp -s Pipfile.lock /workspace/snap/Pipfile.lock.prevendor \ [ ! -e .socket/vendor ] || fail ".socket/vendor must be fully removed after revert" echo "===REVERT VERIFIED===" -socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +socket-patch vendor --json > /tmp/revendor2.json 2>/tmp/revendor2.err RC=$?; cat /tmp/revendor2.err >&2 [ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } assert_summary /tmp/revendor2.json applied 1 @@ -816,7 +817,7 @@ fn poetry_vendor_fresh_checkout_install_and_revert() { } let (_tmp, host) = host_dir(); - let out = run_in_image( + let (out, service) = run_with_fixture( IMAGE, &host, &render_stage1(POETRY_STAGE1, UUID_POETRY) @@ -844,7 +845,12 @@ fn poetry_vendor_fresh_checkout_install_and_revert() { ); poetry_manifestless_vex_online(&host); - let out = run_in_image_network_none(IMAGE, &host, &render(POETRY_STAGE3, UUID_POETRY)); + let out = run_with_service( + IMAGE, + &host, + &render(POETRY_STAGE3, UUID_POETRY), + &service.docker_uri(), + ); assert_stage_markers( "poetry stage 3 (idempotent+revert+re-vendor)", &out, @@ -859,7 +865,7 @@ fn pdm_vendor_fresh_checkout_install_and_revert() { } let (_tmp, host) = host_dir(); - let out = run_in_image(IMAGE, &host, &render_stage1(PDM_STAGE1, UUID_PDM)); + let (out, service) = run_with_fixture(IMAGE, &host, &render_stage1(PDM_STAGE1, UUID_PDM)); assert_stage_markers( "pdm stage 1 (install+vendor)", &out, @@ -880,7 +886,12 @@ fn pdm_vendor_fresh_checkout_install_and_revert() { ], ); - let out = run_in_image_network_none(IMAGE, &host, &render(PDM_STAGE3, UUID_PDM)); + let out = run_with_service( + IMAGE, + &host, + &render(PDM_STAGE3, UUID_PDM), + &service.docker_uri(), + ); assert_stage_markers( "pdm stage 3 (idempotent+revert+re-vendor)", &out, @@ -906,7 +917,7 @@ fn pipenv_vendor_fresh_checkout_install_and_revert() { r#""six.py" "$ORIG" /tmp/patched.py GHSA-dock-pipv-0001 CVE-2026-7501 "#, ); - let out = run_in_image(IMAGE, &host, &stage1); + let (out, service) = run_with_fixture(IMAGE, &host, &stage1); assert_stage_markers( "pipenv stage 1 (install+vendor)", &out, @@ -938,7 +949,12 @@ fn pipenv_vendor_fresh_checkout_install_and_revert() { ], ); - let out = run_in_image_network_none(IMAGE, &host, &render(PIPENV_STAGE3, UUID_PIPENV)); + let out = run_with_service( + IMAGE, + &host, + &render(PIPENV_STAGE3, UUID_PIPENV), + &service.docker_uri(), + ); assert_stage_markers( "pipenv stage 3 (idempotent+revert+re-vendor)", &out, diff --git a/crates/socket-patch-cli/tests/docker_vendor_common/mod.rs b/crates/socket-patch-cli/tests/docker_vendor_common/mod.rs index 8d555f244..0bb6a2b94 100644 --- a/crates/socket-patch-cli/tests/docker_vendor_common/mod.rs +++ b/crates/socket-patch-cli/tests/docker_vendor_common/mod.rs @@ -11,7 +11,7 @@ //! stage 2 (--network none): fresh-checkout copy of ONLY the committable //! files + strictest native install with cold //! caches → patched bytes prove out -//! stage 3 (offline-safe): idempotent re-vendor / `--revert` / re-vendor +//! stage 3 (service available): offline reuse / `--revert` / redownload //! //! So instead of a throwaway container filesystem, every stage runs with the //! same host tempdir bind-mounted at `/workspace`. The socket-patch binary @@ -27,6 +27,12 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + +use std::io::{BufRead, Read, Write}; +use std::process::Stdio; + use std::path::Path; use std::process::{Command, Output}; @@ -89,7 +95,7 @@ pub fn skip_or_require_image(image: &str, required: bool) -> bool { true } -fn docker_run(image: &str, host_dir: &Path, script: &str, extra: &[&str]) -> Output { +fn docker_command(image: &str, host_dir: &Path, script: &str, extra: &[&str]) -> Command { let mut cmd = Command::new("docker"); cmd.args(["run", "--rm", "-i"]); cmd.args(extra); @@ -101,7 +107,94 @@ fn docker_run(image: &str, host_dir: &Path, script: &str, extra: &[&str]) -> Out ]); cmd.args(cov_docker_args()); cmd.args([image, "bash", "-c", script]); - cmd.output().expect("docker run") + cmd +} + +fn docker_run(image: &str, host_dir: &Path, script: &str, extra: &[&str]) -> Output { + docker_command(image, host_dir, script, extra) + .output() + .expect("docker run") +} + +/// Freeze the real installed fixture when the container reaches +/// `publish_fixture`, then serve its artifact to the CLI. Stdin carries the +/// service URL back into the same shell, preserving its variables and /tmp. +/// The returned server stays alive for the later redownload/revert stages. +pub fn run_with_fixture( + image: &str, + host_dir: &Path, + script: &str, +) -> (Output, prebuilt_common::Server) { + let mut child = docker_command( + image, + host_dir, + script, + &["--add-host=host.docker.internal:host-gateway"], + ) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("docker fixture setup"); + let mut stderr = child.stderr.take().unwrap(); + let stderr_thread = std::thread::spawn(move || { + let mut bytes = Vec::new(); + stderr.read_to_end(&mut bytes).unwrap(); + bytes + }); + let mut reader = std::io::BufReader::new(child.stdout.take().unwrap()); + let mut stdout = Vec::new(); + loop { + let mut line = String::new(); + if reader.read_line(&mut line).unwrap() == 0 { + let result = child.wait().unwrap(); + let stderr = stderr_thread.join().unwrap(); + panic!( + "fixture setup exited {result} before publication:\n{}\n{}", + String::from_utf8_lossy(&stdout), + String::from_utf8_lossy(&stderr) + ); + } + stdout.extend_from_slice(line.as_bytes()); + if line.trim() == "===FIXTURE READY===" { + break; + } + } + let m2 = host_dir.join("m2"); + let env = if m2.is_dir() { + vec![("MAVEN_REPO_LOCAL", m2.to_str().unwrap())] + } else { + Vec::new() + }; + let server = prebuilt_common::Server::docker_project(&host_dir.join("proj"), &env); + writeln!(child.stdin.take().unwrap(), "{}", server.docker_uri()).unwrap(); + reader.read_to_end(&mut stdout).unwrap(); + let status = child.wait().unwrap(); + let stderr = stderr_thread.join().unwrap(); + ( + Output { + status, + stdout, + stderr, + }, + server, + ) +} + +/// Run a later lifecycle stage against the same immutable artifact service. +pub fn run_with_service(image: &str, host_dir: &Path, script: &str, uri: &str) -> Output { + docker_run( + image, + host_dir, + script, + &[ + "--add-host=host.docker.internal:host-gateway", + "-e", + &format!("SOCKET_VENDOR_URL={uri}"), + "-e", + &format!("SOCKET_PATCH_SERVER_URL={uri}"), + ], + ) } /// Run `script` (bash) inside `image` with `host_dir` bind-mounted at @@ -148,6 +241,12 @@ pub fn assert_stage_markers(label: &str, out: &Output, markers: &[&str]) { pub fn bash_prelude() -> &'static str { r#"set -u fail() { echo "FAIL: $*" >&2; exit 1; } +publish_fixture() { + echo "===FIXTURE READY===" + read -r SOCKET_VENDOR_URL || fail "fixture service did not start" + export SOCKET_VENDOR_URL + export SOCKET_PATCH_SERVER_URL="$SOCKET_VENDOR_URL" +} git_blob_sha() { # git blob sha256: sha256("blob \0" + bytes) local f="$1" @@ -164,8 +263,8 @@ git_blob_sha() { /// Bash snippet defining `stage_patch /// [ ]`: writes `.socket/manifest.json` + the /// after-hash blob into `.socket/blobs/` (relative to the CURRENT directory — -/// call from the project root) so `socket-patch vendor --offline` runs with -/// zero network. The optional trailing ` ` pair records one +/// call from the project root) as input to the artifact fixture service. +/// The optional trailing ` ` pair records one /// high-severity vulnerability so a generated VEX document has a statement /// to emit; omitted, `vulnerabilities` stays empty. Shape mirrors /// `e2e_vendor_npm_build.rs::stage_patch_with_vuln`. diff --git a/crates/socket-patch-cli/tests/e2e_bun_lockb.rs b/crates/socket-patch-cli/tests/e2e_bun_lockb.rs index ea0386e3b..8091d920f 100644 --- a/crates/socket-patch-cli/tests/e2e_bun_lockb.rs +++ b/crates/socket-patch-cli/tests/e2e_bun_lockb.rs @@ -25,6 +25,7 @@ use wiremock::{Mock, MockServer, ResponseTemplate}; mod bun_vex; #[path = "common/cache_env.rs"] mod cache_env; +mod prebuilt_common; const ORG: &str = "binary-bun-test"; const PURL: &str = "pkg:npm/minimist@1.2.2"; @@ -70,10 +71,10 @@ fn cli(project: &Path, args: &[&str]) -> Value { /// [`cli`] with extra environment variables. fn cli_env(project: &Path, args: &[&str], envs: &[(&str, &str)]) -> Value { + let mut cmd = command(env!("CARGO_BIN_EXE_socket-patch"), project); + let _prebuilt = prebuilt_common::prepare_command(&mut cmd, project, args, envs); let output = require_success( - command(env!("CARGO_BIN_EXE_socket-patch"), project) - .envs(envs.iter().copied()) - .args(args) + cmd.envs(envs.iter().copied()) .args([ "--cwd", project.to_str().unwrap(), @@ -171,7 +172,7 @@ fn scan(project: &Path, server: &MockServer, mode: &str, extra: &[&str]) -> Valu ORG, ]; if mode == "vendored" { - args.extend(["--vendor-source", "build"]); + args.extend(["--vendor-source", "service"]); } args.extend_from_slice(extra); cli(project, &args) @@ -673,6 +674,7 @@ fn file_mode(_p: &Path, name: &str) -> u32 { async fn mock_api(server: &MockServer, fixture: &Fixture, _target: &str) { let tgz = make_tgz_from_installed(&installed_target(&fixture.project), &fixture.patched); + prebuilt_common::mount_download(server, PURL, UUID, "minimist-1.2.2.tgz", &tgz).await; std::fs::write(fixture.temp.path().join("hosted.tgz"), &tgz).unwrap(); let url = format!("{}/patch/npm/minimist/1.2.2/33333333-3333-4333-8333-333333333333/{UUID}/minimist-1.2.2.tgz", server.uri()); let sri = format!( @@ -796,7 +798,10 @@ async fn native_binary_hosted_vendored_takeover_roundtrip() { .decode(integrity.trim_start_matches("sha512-")) .unwrap(); assert!( - fixture.original_lock.windows(64).any(|w| w == digest.as_slice()), + fixture + .original_lock + .windows(64) + .any(|w| w == digest.as_slice()), "the original lock pins the registry digest" ); Mock::given(method("GET")) @@ -808,7 +813,13 @@ async fn native_binary_hosted_vendored_takeover_roundtrip() { .await; let taken_over = cli_env( project, - &["vendor", "--patch-server-url", &uri, "--vendor-source", "build"], + &[ + "vendor", + "--patch-server-url", + &uri, + "--vendor-source", + "service", + ], &[("SOCKET_NPM_REGISTRY", &uri)], ); assert_eq!( @@ -823,15 +834,12 @@ async fn native_binary_hosted_vendored_takeover_roundtrip() { ); let vendor_lock = fixture.lock(); assert!( - !vendor_lock - .windows(uri.len()) - .any(|w| w == uri.as_bytes()), + !vendor_lock.windows(uri.len()).any(|w| w == uri.as_bytes()), "no hosted URL is left in bun.lockb" ); - let state: Value = serde_json::from_slice( - &std::fs::read(project.join(".socket/vendor/state.json")).unwrap(), - ) - .unwrap(); + let state: Value = + serde_json::from_slice(&std::fs::read(project.join(".socket/vendor/state.json")).unwrap()) + .unwrap(); let original = state["entries"][PURL]["wiring"] .as_array() .and_then(|w| w.iter().find(|r| r["kind"] == "bun_lockb_package")) @@ -840,8 +848,7 @@ async fn native_binary_hosted_vendored_takeover_roundtrip() { assert_eq!(original["name"], "minimist", "{original}"); assert_eq!(original["version"], "1.2.2", "{original}"); assert_eq!( - original["resolution"], - "https://registry.npmjs.org/minimist/-/minimist-1.2.2.tgz", + original["resolution"], "https://registry.npmjs.org/minimist/-/minimist-1.2.2.tgz", "the vendor ledger records the registry record: {original}" ); fixture.frozen("taken-over", &fixture.patched, "minimist"); diff --git a/crates/socket-patch-cli/tests/e2e_composer_version_identity.rs b/crates/socket-patch-cli/tests/e2e_composer_version_identity.rs index 63186b550..3bfa212a2 100644 --- a/crates/socket-patch-cli/tests/e2e_composer_version_identity.rs +++ b/crates/socket-patch-cli/tests/e2e_composer_version_identity.rs @@ -9,6 +9,9 @@ //! `scan --redirect` repoints the lock entry. Each test runs the built binary //! against a wiremock API that serves only the padded spelling. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use std::process::Command; @@ -93,7 +96,7 @@ fn write_project(root: &Path, version: &str) { /// hosted package reference — every patch record in the padded spelling. /// The batch response's outer package purl echoes the crawler's query, as /// production does. -async fn mount_api(server: &MockServer) { +async fn mount_api(server: &MockServer, prebuilt: bool) { let before_hash = compute_git_sha256_from_bytes(ORIGINAL); let after_hash = compute_git_sha256_from_bytes(PATCHED); let blob = base64::engine::general_purpose::STANDARD.encode(PATCHED); @@ -127,29 +130,33 @@ async fn mount_api(server: &MockServer) { }))) .mount(server) .await; + let view = json!({ + "uuid": UUID, + "purl": API_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + format!("package/{FILE}"): { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": blob, + } + }, + "vulnerabilities": { + GHSA: { + "cves": ["CVE-2026-0003"], + "summary": "composer padded version fixture", + "severity": "high", + "description": "d" + } + }, + "description": "x", "license": "MIT", "tier": "free" + }); + if prebuilt { + prebuilt_common::mount_view(server, &view, None).await; + } Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": UUID, - "purl": API_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - format!("package/{FILE}"): { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": blob, - } - }, - "vulnerabilities": { - GHSA: { - "cves": ["CVE-2026-0003"], - "summary": "composer padded version fixture", - "severity": "high", - "description": "d" - } - }, - "description": "x", "license": "MIT", "tier": "free" - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; Mock::given(method("POST")) @@ -229,7 +236,7 @@ fn read_json(file: &Path) -> Value { #[tokio::test] async fn agent_sync_applies_and_keeps_a_padded_composer_patch() { let server = MockServer::start().await; - mount_api(&server).await; + mount_api(&server, false).await; let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path(), "v3.0.2"); @@ -260,7 +267,7 @@ async fn agent_sync_applies_and_keeps_a_padded_composer_patch() { #[tokio::test] async fn vendor_wires_and_attests_a_padded_composer_patch() { let server = MockServer::start().await; - mount_api(&server).await; + mount_api(&server, true).await; let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); write_project(root, "3.0.2"); @@ -268,7 +275,7 @@ async fn vendor_wires_and_attests_a_padded_composer_patch() { let (code, env) = run_json( root, &server.uri(), - &["scan", "--vendor", "--vendor-source", "build"], + &["scan", "--vendor", "--vendor-source", "service"], ); assert_eq!(code, 0, "scan --vendor must succeed: {env:#}"); assert_eq!(env["vendor"]["summary"]["applied"], 1, "{env:#}"); @@ -347,7 +354,7 @@ async fn vendor_wires_and_attests_a_padded_composer_patch() { #[tokio::test] async fn hosted_redirect_repoints_a_padded_composer_patch() { let server = MockServer::start().await; - mount_api(&server).await; + mount_api(&server, false).await; let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path(), "3.0.2"); @@ -372,7 +379,9 @@ async fn hosted_redirect_repoints_a_padded_composer_patch() { assert_eq!(entry["dist"]["url"], hosted_url().as_str(), "{lock:#}"); assert_eq!(entry["dist"]["shasum"], SHA1, "{lock:#}"); assert!( - !tmp.path().join(".socket/vendor/redirect-state.json").exists(), + !tmp.path() + .join(".socket/vendor/redirect-state.json") + .exists(), "v5 hosted state is carried by the lockfile" ); } diff --git a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs index 865bd6bcb..99f1ce393 100644 --- a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs +++ b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs @@ -7,6 +7,9 @@ //! patch API. No go toolchain is needed: every assertion is on the files //! the CLI writes and on its JSON envelope. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; #[path = "common/mod.rs"] @@ -98,7 +101,7 @@ async fn mount_sumdb(server: &MockServer) { } fn get_hosted(consumer: &Path, server: &MockServer, modcache: &Path) -> serde_json::Value { - let (code, stdout, stderr) = common::run_with_env( + let (code, stdout, stderr) = run_with_prebuilt( consumer, &[ "get", @@ -266,7 +269,7 @@ async fn hosted_takeover_of_vendored_module_removes_vendored_state() { .unwrap(); std::fs::write(socket.join("blobs").join(&after), PATCHED_LIB).unwrap(); - let (code, stdout, stderr) = common::run_with_env( + let (code, stdout, stderr) = run_with_prebuilt( &consumer, &[ "vendor", @@ -360,7 +363,7 @@ async fn hosted_rollback_restores_go_sum_byte_for_byte() { args.extend_from_slice(extra); let mut env_full = vec![("GOMODCACHE", modcache.to_str().unwrap())]; env_full.extend_from_slice(env); - common::run_with_env(&consumer, &args, &env_full) + run_with_prebuilt(&consumer, &args, &env_full) }; let (code, stdout, stderr) = rollback(&["--offline"], &[]); @@ -462,7 +465,7 @@ async fn vendored_takeover_of_hosted_module_unwinds_the_redirect() { // Online: the takeover's upstream restore consults the (mocked) // checksum database; the patch itself comes from the local manifest. - let (code, stdout, stderr) = common::run_with_env( + let (code, stdout, stderr) = run_with_prebuilt( &consumer, &["vendor", "--json", "--cwd", consumer.to_str().unwrap()], &[ @@ -493,3 +496,22 @@ async fn vendored_takeover_of_hosted_module_unwinds_the_redirect() { ); assert!(!ledger_path.exists(), "no hosted ledger is ever written"); } + +fn run_with_prebuilt( + cwd: &std::path::Path, + args: &[&str], + env: &[(&str, &str)], +) -> (i32, String, String) { + let fixture = (args.first() == Some(&"vendor") && !args.contains(&"--revert")) + .then(|| prebuilt_common::Server::project_with_env(cwd, env)); + let args: Vec<_> = args + .iter() + .copied() + .filter(|a| fixture.is_none() || *a != "--offline") + .collect(); + let mut env = env.to_vec(); + if let Some(fixture) = &fixture { + env.push(("SOCKET_VENDOR_URL", &fixture.uri)); + } + common::run_with_env(cwd, &args, &env) +} diff --git a/crates/socket-patch-cli/tests/e2e_golang_workspace_build.rs b/crates/socket-patch-cli/tests/e2e_golang_workspace_build.rs index 6fd68191d..e1716f69f 100644 --- a/crates/socket-patch-cli/tests/e2e_golang_workspace_build.rs +++ b/crates/socket-patch-cli/tests/e2e_golang_workspace_build.rs @@ -24,6 +24,9 @@ //! GOPROXY, per-"machine" caches, wiremock API). Needs Go 1.18+ (`go.work`); //! the release is whatever `go` is on `PATH` (see `golang_e2e_matrix`). +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -292,7 +295,7 @@ fn move_replace_to_go_work(root: &Path, sums_of: Option<&str>) -> String { /// Run socket-patch (SOCKET_* scrubbed, hermetic config) in `cwd`. fn socket(cwd: &Path, args: &[&str], modcache: &Path) -> (i32, serde_json::Value, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") { cmd.env_remove(&k); @@ -303,6 +306,12 @@ fn socket(cwd: &Path, args: &[&str], modcache: &Path) -> (i32, serde_json::Value .env("GOMODCACHE", modcache) .env("GOFLAGS", "") .env_remove("VIRTUAL_ENV"); + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("GOMODCACHE", modcache.to_str().unwrap())], + ); let out = cmd.output().expect("run socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout).into_owned(); let stderr = String::from_utf8_lossy(&out.stderr).into_owned(); diff --git a/crates/socket-patch-cli/tests/e2e_nuget_dotnet_build.rs b/crates/socket-patch-cli/tests/e2e_nuget_dotnet_build.rs index 6cd2272e9..98ef5543d 100644 --- a/crates/socket-patch-cli/tests/e2e_nuget_dotnet_build.rs +++ b/crates/socket-patch-cli/tests/e2e_nuget_dotnet_build.rs @@ -863,7 +863,9 @@ fn nuget_vendored_dotnet_restore_then_manifestless_vex() { let pristine = std::fs::read(pkg_dir(&store_fx).join(FILE_KEY)).unwrap(); let mut patched = pristine.clone(); patched.extend_from_slice(MARKER); - let backend = Backend::start(VENDORED_UUID, &pristine, &patched, None); + let upstream = std::fs::read(pkg_dir(&store_fx).join(NUPKG_NAME)).unwrap(); + let nupkg = patched_nupkg(&upstream, &patched); + let backend = Backend::start(VENDORED_UUID, &pristine, &patched, Some(&nupkg)); let uri = backend.uri(); // `scan --mode vendored --vendor-source build --vex`: the real backend @@ -877,7 +879,7 @@ fn nuget_vendored_dotnet_restore_then_manifestless_vex() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--json", "--yes", "--api-url", diff --git a/crates/socket-patch-cli/tests/e2e_safety_vlt.rs b/crates/socket-patch-cli/tests/e2e_safety_vlt.rs index 6ba8f60e2..382b314d5 100644 --- a/crates/socket-patch-cli/tests/e2e_safety_vlt.rs +++ b/crates/socket-patch-cli/tests/e2e_safety_vlt.rs @@ -414,7 +414,7 @@ async fn vlt_pinned_matrix_safety_vendored_build() { &fx.proj, &fx.svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "{out}"); pair.assert_untouched(&before, "the vendored build"); @@ -437,7 +437,7 @@ async fn vlt_pinned_matrix_safety_vendor_revert_and_repair() { &fx.proj, &fx.svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "{out}"); fx.vlt_ok(&fx.proj, &["install"]); diff --git a/crates/socket-patch-cli/tests/e2e_socket_yml_policy.rs b/crates/socket-patch-cli/tests/e2e_socket_yml_policy.rs index cc354014d..ca1e81c0a 100644 --- a/crates/socket-patch-cli/tests/e2e_socket_yml_policy.rs +++ b/crates/socket-patch-cli/tests/e2e_socket_yml_policy.rs @@ -3,6 +3,9 @@ //! scanned through the real binary in hosted, agent and vendored mode //! against a mock patch API that serves a small catalog. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::BTreeMap; use std::path::{Path, PathBuf}; @@ -974,7 +977,12 @@ async fn narrowing_after_vendoring_leaves_the_vendored_package_byte_identical() "vulnerabilities": {}, "description": "d", "license": "MIT", "tier": "free" }}}); std::fs::write(web.join(".socket/manifest.json"), serde_json::to_vec_pretty(&manifest).unwrap()).unwrap(); - let (code, stdout, stderr) = run_cli(&web, &["vendor", "--json", "--offline", "--cwd", web.to_str().unwrap()], &[]); + let fixture = prebuilt_common::Server::project(&web); + let (code, stdout, stderr) = run_cli( + &web, + &["vendor", "--json", "--cwd", web.to_str().unwrap()], + &[("SOCKET_VENDOR_URL", &fixture.uri), ("SOCKET_PATCH_SERVER_URL", &fixture.uri)], + ); assert_eq!(code, 0, "vendor fixture: {stdout}\n{stderr}"); assert!(repo.lock("services/web").contains(".socket/vendor/"), "vendored lock"); let snapshot = repo.snapshot(); diff --git a/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs index b6007211f..609e06410 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_bun_build.rs @@ -39,7 +39,7 @@ //! //! The get-driven twin (v3.6) replaces steps 2–3 with a wiremock //! `view/{uuid}` (same hashes, base64 `blobContent` of the after bytes) and -//! `get --mode vendored --vendor-source build` — scan's vendored +//! `get --mode vendored --vendor-source service` — scan's vendored //! posture end to end: committed artifact + ledger (detached record) + wired //! lock, NO manifest, NO `.socket/blobs` — then re-runs the same fresh-checkout install proof. //! The revert half is not repeated there: `vendor --revert` on the @@ -74,6 +74,9 @@ //! equal `bun --version`, so a CI leg cannot pass by running the wrong bun //! or no bun at all. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output, Stdio}; @@ -280,8 +283,10 @@ fn bun(cwd: &Path, args: &[&str], cache_dir: &Path) -> Output { /// should ever post a telemetry event, mocked API or not. fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).arg("--no-telemetry").current_dir(cwd); + cmd.current_dir(cwd); cache_env::scrub_ambient_bun_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); + cmd.arg("--no-telemetry"); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -1111,7 +1116,7 @@ fn bun_vendor_fresh_checkout_frozen_install_and_revert() { assert_eq!(renv["status"], "success", "repair envelope: {renv}"); assert_eq!( renv["summary"]["rebuilt"], 1, - "repair must rebuild the one deleted artifact: {renv}" + "repair must redownload the one deleted artifact: {renv}" ); assert!( renv["events"] @@ -1301,27 +1306,29 @@ fn bun_vendor_tampered_tarball_digest_boundary() { /// to record the manifest and stage the patched content in memory (no /// `.socket/blobs` is ever written). async fn mock_view(server: &MockServer, purl: &str, before: &[u8], after: &[u8]) { + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": git_sha256(before), + "afterHash": git_sha256(after), + "blobContent": b64(after), + } + }, + "vulnerabilities": { GHSA: { + "cves": [CVE], "summary": "vendor bun capstone vuln", + "severity": "high", "description": "d" + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(before), - "afterHash": git_sha256(after), - "blobContent": b64(after), - } - }, - "vulnerabilities": { GHSA: { - "cves": [CVE], "summary": "vendor bun capstone vuln", - "severity": "high", "description": "d" - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; } @@ -1343,8 +1350,7 @@ async fn bun_get_uuid_vendored_fresh_checkout_frozen_install() { // Steps 2–3, get-driven: the patch record comes from a mocked // `view/{uuid}` instead of a hand-staged `.socket/`, and the vendor step - // builds the artifact locally (`--vendor-source build` — no vendoring - // service, so no grant/tarball mocks are needed). + // downloads the published artifact from the fixture service. let server = MockServer::start().await; mock_view(&server, purl, &fx.orig, &fx.patched).await; @@ -1367,7 +1373,7 @@ async fn bun_get_uuid_vendored_fresh_checkout_frozen_install() { "--org", ORG, "--vendor-source", - "build", + "service", ], ); assert_eq!( @@ -1533,8 +1539,8 @@ fn fresh_frozen_install_with_local_deps(fx: &BunProject, name: &str, tgzs: &[Str /// WITHOUT its sha512 whenever the lock is re-saved for another reason /// (measured on 1.1.45, 1.2.23 and 1.3.9; 1.3.10+ keep it). The 2-tuple is /// still our wiring, so after a real re-save: the `vendor` re-run must stay -/// a clean no-op that heals the digest, `repair` must rebuild a deleted -/// artifact through it and re-pin the digest, a fresh frozen install must +/// a clean no-op that heals the digest, `repair` must redownload a deleted +/// artifact without changing the lock, a fresh frozen install must /// land the patched bytes, and — after bun drops the digest AGAIN — /// `vendor --revert` must restore the registry line inside the grown lock. /// On ≥ 1.3.10 the same steps are the no-regression twin (digest kept). @@ -1589,9 +1595,12 @@ fn bun_vendor_survives_a_digest_dropping_lock_resave() { eprintln!("RE-VENDOR OK"); // 3. Repair through a digest-less line: drop the digest again, delete - // the artifact, rebuild. + // the artifact, then redownload the exact published bytes. let tgz_b = grow_project_with_local_dep(&fx, 2); assert_resave_shape(&fx, &wired_line); + let lock_before_repair = std::fs::read_to_string(&lock_path).unwrap(); + let artifact_before_repair = std::fs::read(vendored_tgz(&fx)).unwrap(); + let ledger_before_repair = std::fs::read(proj.join(".socket/vendor/state.json")).unwrap(); std::fs::remove_dir_all(vendored_dir(proj)).unwrap(); let (code, stdout, stderr) = run_socket( proj, @@ -1614,9 +1623,16 @@ fn bun_vendor_survives_a_digest_dropping_lock_resave() { assert!(vendored_tgz(&fx).is_file(), "the artifact must be rebuilt"); let repaired = std::fs::read_to_string(&lock_path).unwrap(); assert_eq!( - packages_line(&repaired, DEP), - wired_line, - "repair re-pins the digest into the healed 3-tuple:\n{repaired}" + repaired, lock_before_repair, + "repair must preserve the package-manager lock byte-for-byte" + ); + assert_eq!( + std::fs::read(vendored_tgz(&fx)).unwrap(), + artifact_before_repair + ); + assert_eq!( + std::fs::read(proj.join(".socket/vendor/state.json")).unwrap(), + ledger_before_repair ); eprintln!("REPAIR THROUGH DIGEST-LESS LOCK OK"); diff --git a/crates/socket-patch-cli/tests/e2e_vendor_cargo_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_cargo_build.rs index 5825b4d3a..a73a2e951 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_cargo_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_cargo_build.rs @@ -79,6 +79,9 @@ //! the fixture build (a failure instead under //! `SOCKET_PATCH_CARGO_E2E_REQUIRED=1`); all assertions after that are hard. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -206,7 +209,7 @@ fn binary() -> PathBuf { /// source tree through it). fn run_socket(cwd: &Path, args: &[&str], cargo_home: &Path) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); @@ -214,6 +217,12 @@ fn run_socket(cwd: &Path, args: &[&str], cargo_home: &Path) -> (i32, String, Str } cmd.env_remove("VIRTUAL_ENV"); cmd.env("CARGO_HOME", cargo_home); + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("CARGO_HOME", cargo_home.to_str().unwrap())], + ); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -984,29 +993,31 @@ async fn cargo_get_uuid_vendored_fresh_checkout_locked_build() { let patched: Vec = [orig.as_slice(), PATCH_SUFFIX.as_bytes()].concat(); let server = MockServer::start().await; + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "src/lib.rs": { + "beforeHash": git_sha256(&orig), + "afterHash": git_sha256(&patched), + "blobContent": b64(&patched), + } + }, + "vulnerabilities": { "GHSA-vend-cargo-real": { + "cves": ["CVE-2024-88888"], + "summary": "capstone vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(&server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "src/lib.rs": { - "beforeHash": git_sha256(&orig), - "afterHash": git_sha256(&patched), - "blobContent": b64(&patched), - } - }, - "vulnerabilities": { "GHSA-vend-cargo-real": { - "cves": ["CVE-2024-88888"], - "summary": "capstone vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(&server) .await; @@ -1029,7 +1040,7 @@ async fn cargo_get_uuid_vendored_fresh_checkout_locked_build() { "--api-token", "fake", "--vendor-source", - "build", + "service", ], &cargo_home, ); @@ -1567,16 +1578,9 @@ fn dir_inventory(dir: &Path) -> serde_json::Map { out } -/// `repair` over a pre-tag cargo vendor whose ledger carries a whole-tree -/// inventory of ANOTHER build source's tree (an extra file the local -/// rebuild does not reproduce) and whose copy is gone: the rebuild (tagged, -/// with its lock retag) comes back from the backend as a fresh entry with -/// the recorded inventory carried forward, the patched members verify, and -/// the tree mismatch is refreshed from the verified rebuild -/// (`vendor_inventory_refreshed`) — never a deleted rebuild stranding the -/// wiring on a dead dir. The result builds `--locked --offline`. +/// Redownload refuses an artifact that cannot reproduce the committed inventory. #[test] -fn cargo_repair_refreshes_a_carried_inventory_over_a_verified_rebuild() { +fn cargo_repair_refuses_a_different_recorded_inventory() { if !cargo_e2e_matrix::cargo_available("e2e_vendor_cargo_build (repair-inventory)") { return; } @@ -1592,7 +1596,6 @@ fn cargo_repair_refreshes_a_carried_inventory_over_a_verified_rebuild() { let patched: Vec = [orig.as_slice(), PATCH_SUFFIX.as_bytes()].concat(); stage_patch(&proj, &purl, "src/lib.rs", &orig, &patched); vendor_ok(&proj, &cargo_home, "repair-inventory"); - untag_project(&proj, &version, UUID); let copy = proj.join(©_rel); std::fs::write(copy.join("PREBUILT_STUB"), "service-only file\n").unwrap(); @@ -1603,6 +1606,8 @@ fn cargo_repair_refreshes_a_carried_inventory_over_a_verified_rebuild() { state["entries"][purl.as_str()]["artifact"]["fileInventory"] = serde_json::Value::Object(recorded.clone()); std::fs::write(&state_path, serde_json::to_string_pretty(&state).unwrap()).unwrap(); + let recorded_state = std::fs::read(&state_path).unwrap(); + let recorded_lock = std::fs::read(proj.join("Cargo.lock")).unwrap(); std::fs::remove_dir_all(©).unwrap(); let (code, stdout, stderr) = run_socket( @@ -1616,32 +1621,16 @@ fn cargo_repair_refreshes_a_carried_inventory_over_a_verified_rebuild() { ], &cargo_home, ); - assert_eq!( - code, 0, - "repair failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + assert_eq!(code, 1, "{stdout}\n{stderr}"); + assert!( + stdout.contains("vendor_artifact_redownload_failed"), + "{stdout}" ); - assert!(stdout.contains("vendor_inventory_refreshed"), "{stdout}"); - assert!(stdout.contains("cargo_version_tagged"), "{stdout}"); - assert_tagged(&proj, &version, UUID, "repair-inventory"); - assert_eq!(std::fs::read(copy.join("src/lib.rs")).unwrap(), patched); - let state: serde_json::Value = - serde_json::from_str(&std::fs::read_to_string(&state_path).unwrap()).unwrap(); - let inventory = state["entries"][purl.as_str()]["artifact"]["fileInventory"] - .as_object() - .unwrap_or_else(|| panic!("the refreshed inventory is persisted: {state}")); - assert!(!inventory.contains_key("PREBUILT_STUB"), "{inventory:?}"); + assert!(!copy.exists()); + assert_eq!(std::fs::read(&state_path).unwrap(), recorded_state); assert_eq!( - inventory, - &dir_inventory(©), - "the verified rebuild's tree" - ); - - std::fs::write(proj.join("src/main.rs"), ORACLE_MAIN).unwrap(); - let run = cargo(&proj, &["run", "-q", "--locked", "--offline"], &cargo_home); - assert!( - String::from_utf8_lossy(&run.stdout).contains(&oracle_line(&version, UUID)), - "the repaired copy builds: {}", - String::from_utf8_lossy(&run.stderr) + std::fs::read(proj.join("Cargo.lock")).unwrap(), + recorded_lock ); } diff --git a/crates/socket-patch-cli/tests/e2e_vendor_composer_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_composer_build.rs index ce9036920..41a6f008a 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_composer_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_composer_build.rs @@ -58,6 +58,9 @@ //! assertion after that is hard. `SOCKET_PATCH_COMPOSER_E2E_VERSION` pins //! the release a CI leg expects. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -100,13 +103,14 @@ fn binary() -> PathBuf { /// flip behavior) along with `VIRTUAL_ENV` (crawler discovery input). fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); } } cmd.env_remove("VIRTUAL_ENV"); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -238,7 +242,7 @@ fn run_vendored(driver: &VendorDriver<'_>, proj: &Path) -> (i32, String, String) "--org", ORG, "--vendor-source", - "build", + "service", "--cwd", proj.to_str().unwrap(), ], @@ -297,27 +301,29 @@ async fn mount_view_mock( ) { use base64::Engine as _; let blob_b64 = base64::engine::general_purpose::STANDARD.encode(after); + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { file_key: { + "beforeHash": git_sha256(before), + "afterHash": git_sha256(after), + "blobContent": blob_b64, + }}, + "vulnerabilities": { GHSA: { + "cves": ["CVE-2026-44444"], + "summary": "composer capstone vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { file_key: { - "beforeHash": git_sha256(before), - "afterHash": git_sha256(after), - "blobContent": blob_b64, - }}, - "vulnerabilities": { GHSA: { - "cves": ["CVE-2026-44444"], - "summary": "composer capstone vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; } @@ -973,7 +979,7 @@ async fn composer_scan_vendor_detached_vex_fresh_checkout_install() { "scan", "--vendor", "--vendor-source", - "build", + "service", "--vex", "out.vex.json", "--vex-product", @@ -1394,15 +1400,14 @@ fn composer_vendor_keeps_files_mirror_filters_would_drop() { assert_fresh_install_mirrors_whole_copy(tmp.path(), &proj, ©_rel, &patched); } -/// A copy vendored by a CLI that predates the neutralization (filter files -/// intact in the committed copy) is healed by the idempotent re-run: the -/// lock stays byte-identical, the heal is warned, and a fresh checkout then -/// installs every file. +/// Modified filter files trigger exact redownload of the committed copy. +/// The original inventory, lock, and ledger survive, and a fresh checkout +/// installs every file from the restored artifact. #[test] #[ignore = "host capstone: shells out to a real composer; the unpinned `test` job \ skips it, the e2e job runs it with a pinned toolchain via --ignored"] -fn composer_vendor_fast_path_heals_legacy_copy() { - let suite = "e2e_vendor_composer_build(legacy-copy)"; +fn composer_vendor_redownloads_modified_copy() { + let suite = "e2e_vendor_composer_build(redownload-copy)"; let Some(major) = composer_e2e_common::composer_major(suite) else { return; }; @@ -1411,13 +1416,17 @@ fn composer_vendor_fast_path_heals_legacy_copy() { std::fs::create_dir_all(&proj).unwrap(); let home = tmp.path().join("composer-home"); let cache = tmp.path().join("composer-cache"); - if !setup_composer_project(&proj, &home, &cache, "(legacy-copy)", major) { + if !setup_composer_project(&proj, &home, &cache, "(redownload-copy)", major) { return; } let lock_path = proj.join("composer.lock"); let version = locked_composer_version(&lock_path, DEP).expect("psr/log locked"); let orig = std::fs::read(proj.join("vendor/psr/log/src/LoggerInterface.php")).unwrap(); - let patched: Vec = [orig.as_slice(), b"\n// SOCKET-PATCH-LEGACY-COPY-MARKER\n"].concat(); + let patched: Vec = [ + orig.as_slice(), + b"\n// SOCKET-PATCH-REDOWNLOAD-COPY-MARKER\n", + ] + .concat(); let purl = format!("pkg:composer/{DEP}@{version}"); stage_patch_with_vuln(&proj, &purl, "src/LoggerInterface.php", &orig, &patched); let (code, stdout, stderr) = run_vendored(&VendorDriver::VendorOffline, &proj); @@ -1428,6 +1437,12 @@ fn composer_vendor_fast_path_heals_legacy_copy() { let copy_rel = format!(".socket/vendor/composer/{UUID}/{DEP}@{version}"); let copy = proj.join(©_rel); + let runtime = tokio::runtime::Runtime::new().unwrap(); + let inventory_before = runtime + .block_on(socket_patch_core::vendor::compute_dir_inventory(©)) + .unwrap(); + let ledger_path = proj.join(".socket/vendor/state.json"); + let ledger_before = std::fs::read(&ledger_path).unwrap(); plant_mirror_filters(©); let lock_wired = std::fs::read(&lock_path).unwrap(); @@ -1440,21 +1455,32 @@ fn composer_vendor_fast_path_heals_legacy_copy() { assert_eq!(env["summary"]["failed"], 0, "{env}"); assert!( env["events"].as_array().unwrap().iter().any(|e| { - e["errorCode"] == "vendor_composer_mirror_filters_neutralized" + e["action"] == "rebuilt" + && e["details"]["redownloaded"] == true && e["purl"] == purl.as_str() }), - "the heal is surfaced: {env}" + "the exact redownload is surfaced: {env}" ); assert_eq!( std::fs::read(&lock_path).unwrap(), lock_wired, "composer.lock untouched" ); - assert_eq!(std::fs::read(copy.join(".gitignore")).unwrap(), b""); + assert_eq!( + std::fs::read(&ledger_path).unwrap(), + ledger_before, + "ledger untouched" + ); + assert_eq!( + runtime + .block_on(socket_patch_core::vendor::compute_dir_inventory(©)) + .unwrap(), + inventory_before + ); assert_eq!( std::fs::read(copy.join("src/LoggerInterface.php")).unwrap(), patched, - "the patched file is untouched by the heal" + "the redownload preserves the patched file" ); let (code, stdout, stderr) = run_vendored(&VendorDriver::VendorOffline, &proj); @@ -1468,8 +1494,9 @@ fn composer_vendor_fast_path_heals_legacy_copy() { .as_array() .unwrap() .iter() - .any(|e| e["errorCode"] == "vendor_composer_mirror_filters_neutralized"), - "a healed copy has nothing left to neutralize: {env}" + .any(|e| e["action"] == "rebuilt" + || e["errorCode"] == "vendor_composer_mirror_filters_neutralized"), + "a restored copy has nothing left to neutralize: {env}" ); assert_fresh_install_mirrors_whole_copy(tmp.path(), &proj, ©_rel, &patched); } diff --git a/crates/socket-patch-cli/tests/e2e_vendor_composer_crlf.rs b/crates/socket-patch-cli/tests/e2e_vendor_composer_crlf.rs index 719a580fa..21ebe1460 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_composer_crlf.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_composer_crlf.rs @@ -8,6 +8,9 @@ //! the pre-vendor bytes exactly. Each test runs the built binary; the //! discovery/view routes are a wiremock API, so no composer and no network. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use std::process::Command; @@ -143,22 +146,24 @@ async fn mount_api(server: &MockServer) { }))) .mount(server) .await; + let view = json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + format!("package/{FILE}"): { + "beforeHash": compute_git_sha256_from_bytes(ORIGINAL), + "afterHash": compute_git_sha256_from_bytes(PATCHED), + "blobContent": blob, + } + }, + "vulnerabilities": vulnerabilities(), + "description": "x", "license": "MIT", "tier": "free" + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - format!("package/{FILE}"): { - "beforeHash": compute_git_sha256_from_bytes(ORIGINAL), - "afterHash": compute_git_sha256_from_bytes(PATCHED), - "blobContent": blob, - } - }, - "vulnerabilities": vulnerabilities(), - "description": "x", "license": "MIT", "tier": "free" - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; } @@ -176,8 +181,9 @@ fn cli() -> Command { } fn run_json(root: &Path, args: &[&str]) -> (i32, Value) { - let out = cli() - .args(args) + let mut command = cli(); + let _fixture = prebuilt_common::prepare_command(&mut command, root, args, &[]); + let out = command .args(["--json", "--cwd", root.to_str().unwrap()]) .output() .expect("run socket-patch"); @@ -299,7 +305,7 @@ async fn scan_vendor_keeps_a_crlf_lock_and_reverts_it_byte_identically() { let root = tmp.path(); write_project(root); let uri = server.uri(); - let mut args = vec!["scan", "--vendor", "--vendor-source", "build"]; + let mut args = vec!["scan", "--vendor", "--vendor-source", "service"]; args.extend(api_args(&uri)); let (code, env) = run_json(root, &args); @@ -323,7 +329,7 @@ async fn get_vendored_keeps_a_crlf_lock_and_reverts_it_byte_identically() { "--mode", "vendored", "--vendor-source", - "build", + "service", ]; args.extend(api_args(&uri)); diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index 0a2e550d3..cf8e660d5 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -58,6 +58,9 @@ //! required) or when the fixture install cannot reach rubygems.org; every //! assertion after that is hard. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -105,13 +108,14 @@ fn argv(args: &[String]) -> Vec<&str> { /// flip behavior) along with `VIRTUAL_ENV` (crawler discovery input). fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); } } cmd.env_remove("VIRTUAL_ENV"); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -1147,29 +1151,37 @@ async fn gem_get_uuid_vendored_fresh_checkout_bundle_install() { let purl = format!("pkg:gem/{DEP}@{version}"); let server = MockServer::start().await; + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/rack.rb": { + "beforeHash": git_sha256(&orig), + "afterHash": git_sha256(&patched), + "blobContent": b64(&patched), + } + }, + "vulnerabilities": { GHSA: { + "cves": ["CVE-2026-55555"], + "summary": "gem capstone vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view_from_source( + &server, + &view, + None, + installed_rb.parent().and_then(Path::parent), + ) + .await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "lib/rack.rb": { - "beforeHash": git_sha256(&orig), - "afterHash": git_sha256(&patched), - "blobContent": b64(&patched), - } - }, - "vulnerabilities": { GHSA: { - "cves": ["CVE-2026-55555"], - "summary": "gem capstone vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(&server) .await; let api_url = server.uri(); @@ -1178,9 +1190,8 @@ async fn gem_get_uuid_vendored_fresh_checkout_bundle_install() { let gemfile_before = std::fs::read(&gemfile_path).unwrap(); // 3. get --mode vendored: record save + scan's whole-manifest - // vendor step in one command. `--vendor-source build` keeps the - // artifact build local (no vendoring-service mocks needed); the - // staging fetches the blob content into MEMORY from the view mock. + // vendor step in one command. The fixture service publishes the full + // installed gem with its patched member, including unmodified files. let (code, stdout, stderr) = run_socket( &proj, &[ @@ -1191,7 +1202,7 @@ async fn gem_get_uuid_vendored_fresh_checkout_bundle_install() { "--json", "--yes", "--vendor-source", - "build", + "service", "--cwd", proj.to_str().unwrap(), "--api-url", diff --git a/crates/socket-patch-cli/tests/e2e_vendor_golang_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_golang_build.rs index df8d9f00a..d9eecff51 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_golang_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_golang_build.rs @@ -25,6 +25,9 @@ //! record, never with a tampered artifact member or a reverted go.mod. The //! Go release is whatever `go` is on `PATH` (see `golang_e2e_matrix`). +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -60,7 +63,7 @@ fn binary() -> PathBuf { /// go crawler resolves installed modules through it). fn run_socket(cwd: &Path, args: &[&str], modcache: &Path) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); @@ -68,6 +71,12 @@ fn run_socket(cwd: &Path, args: &[&str], modcache: &Path) -> (i32, String, Strin } cmd.env_remove("VIRTUAL_ENV"); cmd.env("GOMODCACHE", modcache); + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("GOMODCACHE", modcache.to_str().unwrap())], + ); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -614,27 +623,29 @@ async fn go_get_uuid_vendored_fresh_checkout_offline_build() { // inline blobContent, so the vendor step's in-memory staging hash-gates // pass with no `.socket/` seed on disk. let server = MockServer::start().await; + let view = serde_json::json!({ + "uuid": UUID, + "purl": UPURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { "lib.go": { + "beforeHash": git_sha256(PRISTINE_LIB.as_bytes()), + "afterHash": git_sha256(PATCHED_LIB.as_bytes()), + "blobContent": b64(PATCHED_LIB.as_bytes()), + }}, + "vulnerabilities": { "GHSA-vend-golang-get1": { + "cves": ["CVE-2026-77777"], + "summary": "get-vendored capstone vuln", + "severity": "high", + "description": "d", + }}, + "description": "get-vendored capstone patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(&server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": UPURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { "lib.go": { - "beforeHash": git_sha256(PRISTINE_LIB.as_bytes()), - "afterHash": git_sha256(PATCHED_LIB.as_bytes()), - "blobContent": b64(PATCHED_LIB.as_bytes()), - }}, - "vulnerabilities": { "GHSA-vend-golang-get1": { - "cves": ["CVE-2026-77777"], - "summary": "get-vendored capstone vuln", - "severity": "high", - "description": "d", - }}, - "description": "get-vendored capstone patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(&server) .await; @@ -648,7 +659,7 @@ async fn go_get_uuid_vendored_fresh_checkout_offline_build() { "--json", "--yes", "--vendor-source", - "build", + "service", "--cwd", consumer.to_str().unwrap(), "--api-url", diff --git a/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs index 748793ede..d4e7f1963 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_jvm_build.rs @@ -32,6 +32,9 @@ #[path = "maven_build_common/mod.rs"] mod maven_build_common; +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::BTreeMap; use std::ffi::OsString; use std::path::{Path, PathBuf}; @@ -87,8 +90,13 @@ fn socket(cwd: &Path, m2: &Path, args: &[&str]) -> (Option, serde_json::Val cmd.env_remove(&k); } } + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("MAVEN_REPO_LOCAL", m2.to_str().unwrap())], + ); let out = cmd - .args(args) .current_dir(cwd) .env("SOCKET_TELEMETRY_DISABLED", "1") .env("SOCKET_NO_CONFIG", "1") diff --git a/crates/socket-patch-cli/tests/e2e_vendor_maven_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_maven_build.rs index c94191897..7e834bb2c 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_maven_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_maven_build.rs @@ -9,7 +9,7 @@ //! per-test local repository — the ACTUAL registry bytes. //! 2. A marker patch on the cached jar's `META-INF/NOTICE.txt` is staged //! (manifest + blob, real git-sha256 before/after hashes), and -//! `vendor --json --offline --vex` (the real binary) rebuilds the jar +//! `vendor --json --vex` (the real binary) downloads the patched jar //! into the committed maven2 tree `.socket/vendor/maven//…`, //! inserts the `socket-patch-vendor-` file:// ``, and //! attests in-run `(vendored)`. @@ -41,6 +41,8 @@ #[path = "maven_build_common/mod.rs"] mod maven_build_common; +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; #[path = "vex_e2e_common/mod.rs"] mod vex_e2e_common; @@ -77,8 +79,13 @@ fn socket(cwd: &Path, m2: &Path, args: &[&str]) -> (Option, serde_json::Val cmd.env_remove(&k); } } + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("MAVEN_REPO_LOCAL", m2.to_str().unwrap())], + ); let out = cmd - .args(args) .current_dir(cwd) .env("SOCKET_TELEMETRY_DISABLED", "1") .env("SOCKET_NO_CONFIG", "1") @@ -95,7 +102,7 @@ fn socket(cwd: &Path, m2: &Path, args: &[&str]) -> (Option, serde_json::Val } /// What `get` saves for an agent-mode patch: the manifest record + the -/// after-hash blob (so `vendor --offline` needs no network). +/// after-hash blob (input to the artifact fixture service). fn stage_manifest(proj: &Path, member_before: &[u8], member_after: &[u8]) { let record = serde_json::json!({ "uuid": UUID, diff --git a/crates/socket-patch-cli/tests/e2e_vendor_npm_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_npm_build.rs index f4f367e46..40af54446 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_npm_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_npm_build.rs @@ -35,6 +35,9 @@ //! cannot reach the registry — unless `SOCKET_PATCH_NPM_E2E_REQUIRED` is //! set; every assertion after that is hard. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -71,13 +74,14 @@ fn binary() -> PathBuf { /// flip behavior) along with `VIRTUAL_ENV` (crawler discovery input). fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); } } cmd.env_remove("VIRTUAL_ENV"); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -642,29 +646,31 @@ async fn npm_get_uuid_vendored_fresh_checkout_npm_ci() { // 2. The patch record arrives over the (mocked) API instead of being // staged on disk: same hashes, after bytes inline. let server = MockServer::start().await; + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": git_sha256(&orig), + "afterHash": git_sha256(&patched), + "blobContent": b64(&patched), + } + }, + "vulnerabilities": { TAIL_GHSA: { + "cves": [TAIL_CVE], + "summary": "get vendored vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(&server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(&orig), - "afterHash": git_sha256(&patched), - "blobContent": b64(&patched), - } - }, - "vulnerabilities": { TAIL_GHSA: { - "cves": [TAIL_CVE], - "summary": "get vendored vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(&server) .await; @@ -705,7 +711,7 @@ async fn npm_get_uuid_vendored_fresh_checkout_npm_ci() { "--org", ORG, "--vendor-source", - "build", + "service", ], ); assert_eq!( diff --git a/crates/socket-patch-cli/tests/e2e_vendor_pnpm_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_pnpm_build.rs index 9e92ab9e2..84530ec5c 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_pnpm_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_pnpm_build.rs @@ -62,6 +62,9 @@ //! wiring reverted with ledger + tarball kept (`vendor_unwired`, //! `--no-verify` too), and a lock-only revert that pnpm itself re-wires. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output, Stdio}; @@ -224,8 +227,9 @@ fn scrub_socket_env(cmd: &mut Command) { fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -282,29 +286,31 @@ fn b64(bytes: &[u8]) -> String { /// endpoint, no vendoring service. Metadata mirrors `stage_patch` (same /// GHSA) so the vex leg attests identically under both drivers. async fn mock_view(server: &MockServer, purl: &str, before: &[u8], after: &[u8]) { + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": git_sha256(before), + "afterHash": git_sha256(after), + "blobContent": b64(after), + } + }, + "vulnerabilities": { "GHSA-vend-pnpm-real": { + "cves": ["CVE-2024-88888"], + "summary": "capstone vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(before), - "afterHash": git_sha256(after), - "blobContent": b64(after), - } - }, - "vulnerabilities": { "GHSA-vend-pnpm-real": { - "cves": ["CVE-2024-88888"], - "summary": "capstone vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; } @@ -494,7 +500,7 @@ async fn run_pnpm_capstone(pm: &str, driver: VendorDriver) { "--org", ORG, "--vendor-source", - "build", + "service", "--cwd", proj.to_str().unwrap(), ], diff --git a/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs index 8615fcba1..f613e6907 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_pypi_build.rs @@ -54,6 +54,9 @@ //! older releases, whose lanes run in the `vendored_uv_*` tests. The pip //! capstones below keep their own `uv` discovery. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -117,13 +120,14 @@ fn binary() -> PathBuf { /// the developer's shell). fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); } } cmd.env_remove("VIRTUAL_ENV"); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -361,7 +365,7 @@ fn run_vendored(driver: &VendorDriver<'_>, proj: &Path) -> (i32, String, String) "--org", ORG, "--vendor-source", - "build", + "service", "--cwd", proj.to_str().unwrap(), ], @@ -378,27 +382,29 @@ fn run_vendored(driver: &VendorDriver<'_>, proj: &Path) -> (i32, String, String) async fn mount_view_mock(server: &MockServer, before: &[u8], after: &[u8]) { use base64::Engine as _; let blob_b64 = base64::engine::general_purpose::STANDARD.encode(after); + let view = serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { "six.py": { + "beforeHash": git_sha256(before), + "afterHash": git_sha256(after), + "blobContent": blob_b64, + }}, + "vulnerabilities": { "GHSA-vend-pypi-real": { + "cves": ["CVE-2024-88888"], + "summary": "capstone vex vuln", + "severity": "high", + "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { "six.py": { - "beforeHash": git_sha256(before), - "afterHash": git_sha256(after), - "blobContent": blob_b64, - }}, - "vulnerabilities": { "GHSA-vend-pypi-real": { - "cves": ["CVE-2024-88888"], - "summary": "capstone vex vuln", - "severity": "high", - "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; } diff --git a/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs index d14a94ef1..ef643c2f1 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_vlt_build.rs @@ -10,6 +10,9 @@ //! Run: `SOCKET_PATCH_VLT_E2E_JS= cargo test -p socket-patch-cli //! --test e2e_vendor_vlt_build -- --include-ignored vlt_pinned_matrix`. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use serde_json::{json, Value}; @@ -95,7 +98,7 @@ fn vendor_scan(fx: &Fixture) -> Value { &fx.proj, &fx.svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "scan --mode vendored: {out}"); out.json() @@ -232,7 +235,7 @@ async fn vlt_pinned_matrix_vendored_scan_fresh_ci() { }; let fx = left_pad_fixture(leg).await; let doc = vendor_scan(&fx); - assert!(!event_codes(&doc) + assert!(event_codes(&doc) .iter() .any(|c| c == "vendor_prebuilt_downloaded")); assert_vendored(&fx, fx.t(), ""); @@ -268,17 +271,17 @@ async fn get_vendored(name: &'static str, source: &str) { let served = event_codes(&doc) .iter() .any(|c| c == "vendor_prebuilt_downloaded"); - assert_eq!(served, source == "service", "{doc:#}"); + assert!(served, "{doc:#}"); assert_vendored(&fx, fx.t(), ""); assert_fresh_vendored(&fx, fx.t(), "fresh-get"); fx.leg.ran(); } -/// `get --mode vendored --vendor-source build`. +/// `auto` remains a compatibility alias for service downloads. #[tokio::test(flavor = "multi_thread")] #[ignore = "real vlt: SOCKET_PATCH_VLT_E2E_JS"] -async fn vlt_pinned_matrix_vendored_get_build_fresh_ci() { - get_vendored("get_build_fresh_ci", "build").await; +async fn vlt_pinned_matrix_vendored_get_auto_fresh_ci() { + get_vendored("get_auto_fresh_ci", "auto").await; } /// `get --mode vendored --vendor-source service`: the service's @@ -952,7 +955,7 @@ async fn vlt_pinned_matrix_vendored_package_json_devdeps_patch() { &fx.proj, &svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "{out}"); let payload = package_files(&fx.proj.join(rel(&t2))); @@ -1298,7 +1301,7 @@ async fn vlt_pinned_matrix_vendored_transitive_refused() { &fx.proj, &fx.svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); let doc = out.json(); assert_eq!( @@ -1422,7 +1425,7 @@ async fn vlt_pinned_matrix_vendored_absent_version_refused() { &fx.proj, &fx.svc, &["scan", "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); let doc = out.json(); assert_eq!( diff --git a/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs index 772bd6e74..71045ba0c 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_yarn_berry_build.rs @@ -45,6 +45,9 @@ //! `scripts/yarn-berry-vex-matrix.sh`); `SOCKET_PATCH_YARN_E2E_REQUIRED=1` //! turns every soft-skip into a failure. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output, Stdio}; @@ -144,8 +147,9 @@ fn scrub_socket_env(cmd: &mut Command) { fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -199,27 +203,29 @@ fn b64(bytes: &[u8]) -> String { /// build` stages the patched content entirely from the mock — no blob /// endpoint, no vendoring service. Metadata mirrors `stage_patch`. async fn mock_view(server: &MockServer, purl: &str, before: &[u8], after: &[u8]) { + let view = serde_json::json!({ + "uuid": UUID, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": git_sha256(before), + "afterHash": git_sha256(after), + "blobContent": b64(after), + } + }, + "vulnerabilities": { GHSA: { + "cves": [CVE], "summary": "vendor berry capstone vuln", + "severity": "high", "description": "d", + }}, + "description": "capstone marker patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(before), - "afterHash": git_sha256(after), - "blobContent": b64(after), - } - }, - "vulnerabilities": { GHSA: { - "cves": [CVE], "summary": "vendor berry capstone vuln", - "severity": "high", "description": "d", - }}, - "description": "capstone marker patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view)) .mount(server) .await; } @@ -406,7 +412,7 @@ async fn run_berry_capstone(driver: VendorDriver) { "--org", ORG, "--vendor-source", - "build", + "service", "--cwd", proj.to_str().unwrap(), ], diff --git a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs index 753bbcb29..5b6e8d429 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_build.rs @@ -46,6 +46,9 @@ //! cannot reach the registry — unless `SOCKET_PATCH_YARN_E2E_REQUIRED=1`; //! every assertion after that is HARD. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -122,8 +125,9 @@ fn scrub_socket_env(cmd: &mut Command) { /// Run the socket-patch binary with a scrubbed environment. fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -715,6 +719,7 @@ fn yarn_classic_detached_scan_vendored_fresh_checkout_manifestless_vex() { }))) .mount(&server) .await; + prebuilt_common::mount_view(&server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/test-org/patches/view/{UUID}"))) .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) @@ -743,7 +748,7 @@ fn yarn_classic_detached_scan_vendored_fresh_checkout_manifestless_vex() { "--org", "test-org", "--vendor-source", - "build", + "service", ], ); assert_eq!( @@ -839,7 +844,7 @@ fn yarn_classic_detached_scan_vendored_fresh_checkout_manifestless_vex() { .arg("--mode") .arg("vendored") .arg("--vendor-source") - .arg("build") + .arg("service") .arg("--yes"); run.proxy_url = None; run.api_url = Some(api_url.clone()); diff --git a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_dev_flow.rs b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_dev_flow.rs index 701395fa3..631bba379 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_dev_flow.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_yarn_classic_dev_flow.rs @@ -40,6 +40,9 @@ //! cannot reach the registry — unless `SOCKET_PATCH_YARN_E2E_REQUIRED=1`; //! every assertion after that is HARD. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -147,8 +150,9 @@ fn scrub_socket_env(cmd: &mut Command) { /// Run the socket-patch binary with a scrubbed environment. fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), diff --git a/crates/socket-patch-cli/tests/e2e_vendored_production.rs b/crates/socket-patch-cli/tests/e2e_vendored_production.rs index 46cdb71cf..5b22e65b2 100644 --- a/crates/socket-patch-cli/tests/e2e_vendored_production.rs +++ b/crates/socket-patch-cli/tests/e2e_vendored_production.rs @@ -57,13 +57,9 @@ //! //! * **gem** — full coverage (only non-`ruby` platform qualifiers are //! refused): [`gem_bundler_vendored_install_proof`] runs the complete vendored -//! loop including the fresh-dir `bundle install` delivery proof. While -//! production's served `gem-stub-gemspec` remains invalid (D4: missing the -//! rubygems-required `summary`/`authors`), the leg passes via the CLI's -//! invalid-stub hardening — `--vendor-source auto` detects the defect and -//! falls back to the local build (`vendor_prebuilt_stub_invalid` warning); -//! once the server-side stub fix deploys and the artifacts rebuild, the -//! same leg exercises the service artifact directly. +//! loop including the fresh-dir `bundle install` delivery proof. The server +//! must provide a valid `gem-stub-gemspec` with summary and authors; invalid +//! stubs fail closed without constructing a local replacement. //! * **golang** — vendored mode *works* (directory `replace`), but production //! publishes no free golang patches, so there is nothing to vendor. //! [`golang_vendored_finds_no_free_patches`] asserts exactly that (zero @@ -2148,20 +2144,8 @@ fn pypi_uv_lock_vendored_install_proof() { /// rubygems.org — a path source only pins the one gem), and the file the /// patch rewrites must carry the `Socket Community Patch` header. /// -/// # How the leg passes while production's served stub is invalid (D4) -/// -/// The `gem-stub-gemspec` artifact production currently serves omits the -/// rubygems-required `summary`/`authors`, so writing it verbatim would make -/// the frozen `bundle install` below exit 1 on every bundler major. The CLI's -/// invalid-stub hardening is what this leg regression-tests live: under the -/// default `--vendor-source auto` the scan detects the defective stub, warns -/// (`vendor_prebuilt_stub_invalid`), and falls back to the LOCAL build -/// (installed gem + locally derived stub), which installs green. That is also -/// why the leg installs in bundler's deployment layout (`vendor/bundle` -/// inside the project): the crawler only sees a project-local install, and -/// the fallback needs the install's `specifications/` stub. Once the -/// server-side stub fix (depscan) deploys and the artifacts rebuild, the same -/// leg exercises the service artifact directly — no test change needed. +/// The service must supply both the patched archive and a valid stub gemspec. +/// A defective stub is a server failure; the CLI cannot build a replacement. #[test] #[ignore = "live production API + real rubygems.org. Run with --ignored."] fn gem_bundler_vendored_install_proof() { diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/deno.rs b/crates/socket-patch-cli/tests/e2e_vex_build/deno.rs index 9434b8965..977e0fb65 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/deno.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/deno.rs @@ -390,7 +390,7 @@ fn deno_hosted_and_vendored_never_attest_manifest_mode_unchanged() { .map(|s| s.to_string()) .collect(); if mode == "vendored" { - a.extend(["--vendor-source".to_string(), "build".to_string()]); + a.extend(["--vendor-source".to_string(), "service".to_string()]); } a }; diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs index 3e6e3eff0..a7877c9ed 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/hatch.rs @@ -201,7 +201,7 @@ fn hatch() -> Option { fn scan_mode_args(mode: Mode) -> Vec<&'static str> { match mode { Mode::Hosted => vec!["--mode=hosted"], - Mode::Vendored => vec!["--vendor", "--vendor-source", "build"], + Mode::Vendored => vec!["--vendor", "--vendor-source", "service"], } } diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/pdm.rs b/crates/socket-patch-cli/tests/e2e_vex_build/pdm.rs index cea8568a6..978d0aaa2 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/pdm.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/pdm.rs @@ -311,7 +311,7 @@ fn patched_of(pristine: &[u8]) -> Vec { fn scan_mode_args(mode: Mode) -> Vec<&'static str> { match mode { Mode::Hosted => vec!["--mode=hosted"], - Mode::Vendored => vec!["--vendor", "--vendor-source", "build"], + Mode::Vendored => vec!["--vendor", "--vendor-source", "service"], } } diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs b/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs index e38348297..158e14fa1 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs @@ -472,7 +472,7 @@ impl PatchService { "artifacts": [{ "kind": "tarball", "url": artifact_url, - "integrity": { "sha256": sha } + "integrity": { "sha256": sha, "sha512": ({ use base64::Engine as _; use sha2::Digest; format!("sha512-{}", base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(&wheel))) }) } }], "registryOverride": null } } @@ -933,7 +933,7 @@ fn poetry_vendored_fresh_install_then_manifestless_vex() { "scan", "--vendor", "--vendor-source", - "build", + "service", "--vex", embedded.to_str().unwrap(), "--vex-product", diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/bun.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/bun.rs index 8b7d6d6bd..ff9291795 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/bun.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/bun.rs @@ -790,7 +790,7 @@ fn scan_vendored(cwd: &Path, flavor: Flavor, api: &Api) -> Vec { "--mode", "vendored", "--vendor-source", - "build", + "service", "--yes", "--vex", "out.vex.json", diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/composer.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/composer.rs index 3f4121bb3..797892aad 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/composer.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/composer.rs @@ -410,6 +410,7 @@ fn write_vendor_ledger(cwd: &Path, key: &str, rec: PatchRecord) { base_purl: key.to_string(), uuid: rec.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: artifact_rel(&rec.uuid), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/deno.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/deno.rs index 620e311bd..645670fa5 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/deno.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/deno.rs @@ -284,6 +284,7 @@ fn deno_ledger_claims_are_dead() { base_purl: JSR_PURL.to_string(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.clone(), sha256: "0".repeat(64), size: None, diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs index f22f7785f..c79358c1d 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/maven.rs @@ -40,6 +40,9 @@ //! alive by a ledger under `--no-verify`, and agent-mode (`apply`) patches, //! which have no lockfile wiring to discover. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use crate::vex_e2e_common; use std::collections::HashMap; @@ -384,6 +387,7 @@ fn write_vendor_ledger( base_purl: purl.to_string(), uuid: uuid.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: artifact_rel.to_string(), sha256, size: None, @@ -1200,7 +1204,8 @@ impl Fx { /// Run the real binary with `args` (hermetic stores, no ambient token). fn run(&self, args: &[&str]) -> (Option, Value, String) { let mut cmd = cli(&self.store()); - cmd.args(args).current_dir(&self.cwd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, &self.cwd, args, &[]); + cmd.current_dir(&self.cwd); let out = cmd.output().expect("invoke socket-patch"); let stderr = String::from_utf8_lossy(&out.stderr).into_owned(); let env = serde_json::from_slice(&out.stdout).unwrap_or_else(|e| { @@ -1524,13 +1529,9 @@ fn maven_scan_hosted_wiring_reattests_without_manifest_or_ledger() { ); let reverted = std::fs::read_to_string(fixture_dir(&format!("{golden}/input/pom.xml"))).unwrap(); - standalone_after_hosted_writer( - &fx, - MVN_HOSTED_UUID, - MVN_PURL, - mvn_hosted_view(), - &|fx| fx.put("pom.xml", &reverted), - ); + standalone_after_hosted_writer(&fx, MVN_HOSTED_UUID, MVN_PURL, mvn_hosted_view(), &|fx| { + fx.put("pom.xml", &reverted) + }); } /// DOCUMENTED LIMITATION (design scope): an AGENT-mode patch (`apply` diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/nuget.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/nuget.rs index 77ba9403d..0dae5f934 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/nuget.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/nuget.rs @@ -37,6 +37,9 @@ //! without a `contentHash` pin, and agent-mode (`apply`) patches, which //! have no lockfile wiring to discover. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use crate::vex_e2e_common; use std::collections::HashMap; @@ -362,6 +365,7 @@ fn write_vendor_ledger( base_purl: purl.to_string(), uuid: uuid.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: artifact_rel.to_string(), sha256, size: None, @@ -1053,7 +1057,8 @@ impl Fx { /// Run the real binary with `args` (hermetic stores, no ambient token). fn run(&self, args: &[&str]) -> (Option, Value, String) { let mut cmd = cli(&self.store()); - cmd.args(args).current_dir(&self.cwd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, &self.cwd, args, &[]); + cmd.current_dir(&self.cwd); let out = cmd.output().expect("invoke socket-patch"); let stderr = String::from_utf8_lossy(&out.stderr).into_owned(); let env = serde_json::from_slice(&out.stdout).unwrap_or_else(|e| { diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/pnpm.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/pnpm.rs index 815d17e08..9ab6a52da 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/pnpm.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/pnpm.rs @@ -33,6 +33,9 @@ //! * hand-wired vendored — pnpm 1-6 locks (which `vendor` refuses) whose //! resolution a user pointed at a committed `.socket/vendor/` tarball. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -188,8 +191,8 @@ fn socket(root: &Path, args: &[&str]) -> (Option, Value, String) { cmd.env_remove(k); } } + let _fixture = prebuilt_common::prepare_command(&mut cmd, root, args, &[]); let out = cmd - .args(args) .arg("--cwd") .arg(root) .env("SOCKET_TELEMETRY_DISABLED", "1") diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/poetry.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/poetry.rs index 7142c84f3..c37145a4a 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/poetry.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/poetry.rs @@ -449,6 +449,7 @@ fn write_vendor_ledger(p: &Proj, sha: &str, record: PatchRecord) { base_purl: purl(), uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: vendored_rel(&record.uuid), sha256: sha.to_string(), size: None, @@ -1017,9 +1018,8 @@ fn every_hosted_pin_spelling_attests_and_a_pinless_entry_needs_an_install() { // A `[metadata.files]` entry that listed files before the rewrite // keeps Poetry's one-file-per-line layout (rollback restores the full // list from it; an inline entry means the original was `[]`). - let metadata_block = format!( - "{PKG} = [\n {{file = \"{WHEEL}\", hash = \"sha256:{sha}\"}},\n]" - ); + let metadata_block = + format!("{PKG} = [\n {{file = \"{WHEEL}\", hash = \"sha256:{sha}\"}},\n]"); let fragment = format!("#sha256={sha}&"); let spellings: Vec<(&str, &str)> = [ ("package files", files_line.as_str()), @@ -1273,9 +1273,13 @@ impl ScanApi { .build() .unwrap(); let server = rt.block_on(wiremock::MockServer::start()); - let artifact_url = artifact_url - .map(str::to_string) - .unwrap_or_else(|| hosted_url_on(&server.uri(), uuid)); + let artifact_url = if uuid == VENDORED_UUID { + hosted_url_on(&server.uri(), uuid) + } else { + artifact_url + .map(str::to_string) + .unwrap_or_else(|| hosted_url_on(&server.uri(), uuid)) + }; let sha = sha256_hex(wheel); let mut full_view = view(uuid, &api_purl()); full_view["files"][MODULE]["blobContent"] = @@ -1322,7 +1326,7 @@ impl ScanApi { "artifacts": [{ "kind": "tarball", "url": artifact_url, - "integrity": { "sha256": sha } + "integrity": { "sha256": sha, "sha512": ({ use sha2::Digest; format!("sha512-{}", base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(&wheel))) }) } }], "registryOverride": null } } @@ -1578,7 +1582,7 @@ fn scan_vendor_wiring_attests_without_manifest_or_ledger() { "scan", "--vendor", "--vendor-source", - "build", + "service", "--vex", embedded_doc.to_str().unwrap(), "--vex-product", diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/uv.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/uv.rs index af798ff73..a2b120afb 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/uv.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/uv.rs @@ -555,6 +555,7 @@ fn write_vendor_ledger(p: &Proj, flavor: Flavor, rel: &str, sha: &str, record: P base_purl: flavor.purl(), uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.to_string(), sha256: sha.to_string(), size: None, @@ -1396,6 +1397,12 @@ impl Api { let mut full_view = view(uuid, &qualified); full_view["files"][MODULE]["blobContent"] = Value::String(base64::engine::general_purpose::STANDARD.encode(PATCHED)); + let downloaded_url = if uuid == VENDORED_UUID { + hosted_url_on(&self.uri(), flavor, uuid) + } else { + artifact_url.to_string() + }; + let artifact_url = downloaded_url.as_str(); let artifact_path = artifact_url.strip_prefix(&self.uri()).map(str::to_string); self.rt.block_on(async { Mock::given(method("POST")) @@ -1438,7 +1445,7 @@ impl Api { "artifacts": [{ "kind": "tarball", "url": artifact_url, - "integrity": { "sha256": sha } + "integrity": { "sha256": sha, "sha512": ({ use sha2::Digest; format!("sha512-{}", base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(&wheel))) }) } }], "registryOverride": null } } @@ -1657,7 +1664,7 @@ fn scan_vendor_wiring_attests_without_manifest_or_ledger() { "scan", "--vendor", "--vendor-source", - "build", + "service", "--vex", embedded.to_str().unwrap(), "--vex-product", diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/vlt.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/vlt.rs index 14ea135dd..7b76accbd 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/vlt.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/vlt.rs @@ -45,6 +45,9 @@ //! //! The shared matrix also runs over a vendored checkout. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use wiremock::MockServer; @@ -368,7 +371,12 @@ fn vendored_project(root: &Path, era: Era) -> String { ) .unwrap(); let cwd = root.to_str().unwrap().to_string(); - let (code, env, stderr) = hosted::run_json(root, &["vendor", "--offline", "--cwd", &cwd], &[]); + let fixture = prebuilt_common::Server::project(root); + let (code, env, stderr) = hosted::run_json( + root, + &["vendor", "--cwd", &cwd], + &[("SOCKET_VENDOR_URL", &fixture.uri)], + ); assert_eq!(code, 0, "vendor: {env:#}\n{stderr}"); assert_eq!(env["summary"]["applied"], 1, "vendor: {env:#}"); assert!(root.join(vendored_rel()).join("index.js").is_file()); diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs index 2951081d5..d547e4d72 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/yarn.rs @@ -485,6 +485,7 @@ fn write_vendor_ledger(cwd: &Path, flavor: Flavor, rel: &str, rec: PatchRecord) base_purl: PURL.to_string(), uuid: rec.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.to_string(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/tests/e2e_vex_vendor.rs b/crates/socket-patch-cli/tests/e2e_vex_vendor.rs index e944c38b1..72541c569 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_vendor.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_vendor.rs @@ -154,6 +154,7 @@ fn write_vendor_state(cwd: &Path, purl: &str, rel_path: &str) { base_purl: purl.to_string(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.to_string(), sha256: String::new(), size: None, @@ -609,6 +610,7 @@ fn write_detached_vendor_state(cwd: &Path, purl: &str, rel_path: &str, record: P base_purl: purl.to_string(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.to_string(), sha256: String::new(), size: None, @@ -986,6 +988,7 @@ fn detached_matrix_entry( base_purl: purl.to_string(), uuid: uuid.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.to_string(), sha256, size: None, @@ -1423,7 +1426,10 @@ fn agent_patch_without_install_hook_attests_and_replaces_stale_doc() { let doc: Value = serde_json::from_str(&std::fs::read_to_string(&vex_path).expect("read emitted VEX doc")) .expect("parse emitted VEX doc"); - assert_ne!(doc["@id"], "urn:uuid:stale", "the stale doc is replaced: {doc}"); + assert_ne!( + doc["@id"], "urn:uuid:stale", + "the stale doc is replaced: {doc}" + ); let stmts = doc["statements"].as_array().unwrap(); assert_eq!(stmts.len(), 1, "{doc}"); assert_eq!(stmts[0]["vulnerability"]["name"], "GHSA-drop-aaaa", "{doc}"); @@ -1990,6 +1996,7 @@ fn reconstructed_ledger_entry_without_wiring_attests_from_the_root_lock() { base_purl: purl.to_string(), uuid: uuid.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.clone(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-cli/tests/e2e_yarn4_pnpm_linker_build.rs b/crates/socket-patch-cli/tests/e2e_yarn4_pnpm_linker_build.rs index ff575b68d..d5440efab 100644 --- a/crates/socket-patch-cli/tests/e2e_yarn4_pnpm_linker_build.rs +++ b/crates/socket-patch-cli/tests/e2e_yarn4_pnpm_linker_build.rs @@ -18,7 +18,7 @@ //! redirect sibling); a fresh checkout of only the committable files //! passes `yarn install --immutable --check-cache` offline-from-registry //! and serves the patched bytes THROUGH the `.store` symlink layout. -//! * vendored — `vendor --offline` wires `resolutions` + the `file:` +//! * vendored — `vendor` wires `resolutions` + the `file:` //! locator; the fresh `--immutable --check-cache` install lands the //! patched bytes in a `left-pad-file-` store entry, and //! `--revert` restores package.json AND yarn.lock byte-for-byte. @@ -49,6 +49,8 @@ use wiremock::{Mock, MockServer, ResponseTemplate}; #[path = "common/cache_env.rs"] mod cache_env; +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; const ORG: &str = "test-org"; const DEP: &str = "left-pad"; @@ -152,8 +154,9 @@ fn corepack(cwd: &Path, pm: &str, args: &[&str], extra_env: &[(&str, &str)]) -> fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -659,7 +662,7 @@ fn yarn4_pnpm_linker_vendor_fresh_checkout_installs_patched_bytes_and_reverts() let lock_before = std::fs::read(&lock_path).unwrap(); let pkg_before = std::fs::read(&pkg_path).unwrap(); - // Vendor (offline) — discovery must crawl the .store layout. + // Download the vendored artifact — discovery must crawl the .store layout. let (code, stdout, stderr) = run_socket( &proj, &[ diff --git a/crates/socket-patch-cli/tests/e2e_yarn4_workspaces_build.rs b/crates/socket-patch-cli/tests/e2e_yarn4_workspaces_build.rs index bbffec465..2c8ca559b 100644 --- a/crates/socket-patch-cli/tests/e2e_yarn4_workspaces_build.rs +++ b/crates/socket-patch-cli/tests/e2e_yarn4_workspaces_build.rs @@ -22,7 +22,7 @@ //! without touching either package.json; a fresh checkout of only the //! committable files installs the patched bytes offline-from-registry, //! and the member resolves them through `yarn node`. -//! * vendored — `vendor --offline` wires the ROOT package.json +//! * vendored — `vendor` wires the ROOT package.json //! `resolutions` + the root lock `file:` locator (member package.json //! byte-identical); fresh `--immutable --check-cache` installs the //! patched bytes, the member resolves them, and `--revert` restores @@ -51,6 +51,8 @@ use wiremock::{Mock, MockServer, ResponseTemplate}; #[path = "common/cache_env.rs"] mod cache_env; +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; const ORG: &str = "test-org"; const DEP: &str = "left-pad"; @@ -150,8 +152,9 @@ fn corepack(cwd: &Path, pm: &str, args: &[&str], extra_env: &[(&str, &str)]) -> fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -665,7 +668,7 @@ fn yarn4_workspaces_vendor_wires_root_and_member_installs_patched_bytes() { let root_pkg_before = std::fs::read(&root_pkg_path).unwrap(); let member_pkg_before = std::fs::read(&member_pkg_path).unwrap(); - // Vendor (offline) from the WORKSPACE ROOT. + // Download the vendored artifact from the WORKSPACE ROOT. let (code, stdout, stderr) = run_socket( &proj, &[ diff --git a/crates/socket-patch-cli/tests/get/get_modes_e2e.rs b/crates/socket-patch-cli/tests/get/get_modes_e2e.rs index 8ba3bd979..cada79646 100644 --- a/crates/socket-patch-cli/tests/get/get_modes_e2e.rs +++ b/crates/socket-patch-cli/tests/get/get_modes_e2e.rs @@ -16,6 +16,9 @@ //! No `#[serial]`: the child gets a scrubbed env copy (`run_bin_with_env` //! via `run_with_env`); the parent process env is never mutated. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use wiremock::matchers::{method, path}; @@ -50,34 +53,36 @@ fn b64(bytes: &[u8]) -> String { /// `view/{uuid}` with REAL git-blob hashes and inline blob content, so the /// vendored flow's staging hash-gates pass (the hosted flow only needs the /// record fields). Same recipe as the in-process suite. -async fn mock_view(server: &MockServer, uuid: &str, purl: &str) { +async fn mock_view(server: &MockServer, uuid: &str, purl: &str) -> serde_json::Value { + let view = serde_json::json!({ + "uuid": uuid, + "purl": purl, + "publishedAt": "2024-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": common::git_sha256(BEFORE_BYTES), + "afterHash": common::git_sha256(AFTER_BYTES), + "blobContent": b64(AFTER_BYTES), + } + }, + "vulnerabilities": { + GHSA: { + "cves": ["CVE-2024-1234"], + "summary": "get-modes fixture", + "severity": "high", + "description": "d" + } + }, + "description": "get-modes fixture", + "license": "MIT", + "tier": "free", + }); Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": purl, - "publishedAt": "2024-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": common::git_sha256(BEFORE_BYTES), - "afterHash": common::git_sha256(AFTER_BYTES), - "blobContent": b64(AFTER_BYTES), - } - }, - "vulnerabilities": { - GHSA: { - "cves": ["CVE-2024-1234"], - "summary": "get-modes fixture", - "severity": "high", - "description": "d" - } - }, - "description": "get-modes fixture", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; + view } /// `by-ghsa/{GHSA}`: the two-version fan-out — 1.0.0 (installable in the @@ -307,7 +312,8 @@ async fn get_uuid_hosted_json_envelope_nests_redirect() { #[tokio::test] async fn get_uuid_vendored_json_envelope_nests_vendor() { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; + let fixture = prebuilt_common::Server::view(view); let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path()); @@ -319,8 +325,10 @@ async fn get_uuid_vendored_json_envelope_nests_vendor() { UUID1, "--mode", "vendored", + "--vendor-url", + &fixture.uri, "--vendor-source", - "build", + "service", "--json", ], ); @@ -662,7 +670,7 @@ async fn get_vendored_dry_run_json_envelope() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--dry-run", "--json", ], @@ -711,7 +719,8 @@ async fn get_vendored_dry_run_json_envelope() { #[tokio::test] async fn get_vendored_then_hosted_takes_over_cleanly() { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; + let fixture = prebuilt_common::Server::view(view); mock_reference(&server).await; let tmp = tempfile::tempdir().unwrap(); @@ -725,9 +734,11 @@ async fn get_vendored_then_hosted_takes_over_cleanly() { UUID1, "--mode", "vendored", + "--vendor-url", + &fixture.uri, "--json", "--vendor-source", - "build", + "service", ], ); assert_eq!(code, 0, "vendored step failed: {stderr}"); @@ -982,7 +993,7 @@ async fn get_vendored_refusal_visible_under_silent() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--silent", ], ); @@ -1037,7 +1048,7 @@ async fn get_vendored_dry_run_reports_bun_refusal() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--dry-run", "--json", ], @@ -1148,7 +1159,7 @@ async fn get_vendored_vlt_refusal_visible_under_silent() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--silent", ], ); @@ -1224,9 +1235,11 @@ async fn get_save_only_agent_ignores_vlt_preflight() { async fn get_modes_state_attests_manifest_less() { for hosted in [true, false] { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; if hosted { mock_reference(&server).await; + } else { + prebuilt_common::mount_view(&server, &view, None).await; } let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path()); @@ -1239,7 +1252,7 @@ async fn get_modes_state_attests_manifest_less() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--json", ] }; diff --git a/crates/socket-patch-cli/tests/in_process_get_modes.rs b/crates/socket-patch-cli/tests/in_process_get_modes.rs index 949921f3d..733ef0f05 100644 --- a/crates/socket-patch-cli/tests/in_process_get_modes.rs +++ b/crates/socket-patch-cli/tests/in_process_get_modes.rs @@ -9,6 +9,9 @@ //! //! `#[serial]`: `get::run` mirrors env toggles into process-global env vars. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use serial_test::serial; @@ -57,9 +60,7 @@ fn get_args(identifier: &str, cwd: &Path, api_url: String) -> GetArgs { api_url: Some(api_url), json: true, download_mode: "diff".to_string(), - // Local build so the vendored tests never reach the vendoring - // service (no grant/tarball mocks needed). - vendor_source: "build".to_string(), + vendor_source: "service".to_string(), ..socket_patch_cli::args::GlobalArgs::default() }, identifier: identifier.to_string(), @@ -75,34 +76,36 @@ fn get_args(identifier: &str, cwd: &Path, api_url: String) -> GetArgs { /// `view/{uuid}` with REAL git-blob hashes and inline blob content, so the /// vendored flow's staging hash-gates pass and the agent flow can apply. -async fn mock_view(server: &MockServer, uuid: &str, purl: &str) { +async fn mock_view(server: &MockServer, uuid: &str, purl: &str) -> serde_json::Value { + let view = serde_json::json!({ + "uuid": uuid, + "purl": purl, + "publishedAt": "2024-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash(), + "afterHash": after_hash(), + "blobContent": b64(AFTER_BYTES), + } + }, + "vulnerabilities": { + GHSA: { + "cves": ["CVE-2024-1234"], + "summary": "get-modes fixture", + "severity": "high", + "description": "d" + } + }, + "description": "get-modes fixture", + "license": "MIT", + "tier": "free", + }); Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": purl, - "publishedAt": "2024-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash(), - "afterHash": after_hash(), - "blobContent": b64(AFTER_BYTES), - } - }, - "vulnerabilities": { - GHSA: { - "cves": ["CVE-2024-1234"], - "summary": "get-modes fixture", - "severity": "high", - "description": "d" - } - }, - "description": "get-modes fixture", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view.clone())) .mount(server) .await; + view } /// `by-ghsa/{GHSA}`: the two-version fan-out — the installed 1.0.0 and the @@ -374,7 +377,8 @@ async fn get_uuid_hosted_dry_run_writes_nothing() { #[serial] async fn get_uuid_vendored_commits_artifact_and_wires_lock() { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; + prebuilt_common::mount_view(&server, &view, None).await; let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path()); @@ -436,7 +440,8 @@ async fn get_uuid_vendored_commits_artifact_and_wires_lock() { #[serial] async fn get_uuid_vendored_rerun_is_idempotent() { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; + prebuilt_common::mount_view(&server, &view, None).await; let tmp = tempfile::tempdir().unwrap(); write_project(tmp.path()); @@ -652,7 +657,10 @@ async fn mode_with_save_only_conflicts_exit_one_before_network() { args.mode = Some(mode); args.save_only = true; let code = socket_patch_cli::commands::get::run(args).await; - assert_eq!(code, 2, "--save-only + --mode {mode:?} must be rejected (usage, exit 2)"); + assert_eq!( + code, 2, + "--save-only + --mode {mode:?} must be rejected (usage, exit 2)" + ); } assert!( server @@ -985,7 +993,8 @@ fn write_vlt_project(root: &Path, spec: &str) { #[serial] async fn get_uuid_vendored_vlt_vendors_refuses_and_agent_bypasses() { let server = MockServer::start().await; - mock_view(&server, UUID1, PURL1).await; + let view = mock_view(&server, UUID1, PURL1).await; + prebuilt_common::mount_view(&server, &view, None).await; let tmp = tempfile::tempdir().unwrap(); write_vlt_project(tmp.path(), "1.0.0"); diff --git a/crates/socket-patch-cli/tests/in_process_redirect.rs b/crates/socket-patch-cli/tests/in_process_redirect.rs index 3c132eec3..c58dcabaa 100644 --- a/crates/socket-patch-cli/tests/in_process_redirect.rs +++ b/crates/socket-patch-cli/tests/in_process_redirect.rs @@ -629,16 +629,20 @@ const BERRY_CHECKSUM: &str = "10c0/7785879d9a7dc9bee6730ec55926a0ab9ed6bfe0eaee0 /// yarn-berry-zip artifact (yarnBerry10c0) — the berry rewriter pins the zip /// checksum, not the tarball's. async fn mock_reference_with_berry(server: &MockServer) { + mock_reference_with_berry_url(server, HOSTED_URL).await; +} + +async fn mock_reference_with_berry_url(server: &MockServer, hosted_url: &str) { Mock::given(method("POST")) .and(path(format!("/v0/orgs/{ORG}/patches/package"))) .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ "results": { UUID: { "status": "granted", - "url": HOSTED_URL, + "url": hosted_url, "purl": PURL, "artifacts": [ - { "kind": "tarball", "url": HOSTED_URL, + { "kind": "tarball", "url": hosted_url, "integrity": { "sha512": PATCHED_SHA512 } }, { "kind": "yarn-berry-zip", "url": "http://patch.test/berry.zip", "integrity": { "yarnBerry10c0": BERRY_CHECKSUM } } @@ -753,7 +757,11 @@ async fn scan_redirect_rewrites_yarn_berry_lock() { async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_restores_them() { let server = MockServer::start().await; mock_discovery(&server).await; - mock_reference_with_berry(&server).await; + mock_reference_with_berry_url( + &server, + &HOSTED_URL.replace("http://patch.test", &server.uri()), + ) + .await; mock_view(&server).await; let tarball = upstream_tarball(); mock_npm_registry( @@ -762,7 +770,9 @@ async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_resto Some(tarball), ) .await; - let encoded = socket_patch_core::utils::uri::encode_uri_component(HOSTED_URL); + let encoded = socket_patch_core::utils::uri::encode_uri_component( + &HOSTED_URL.replace("http://patch.test", &server.uri()), + ); for (label, bom) in [("crlf", ""), ("bom+crlf", "\u{feff}")] { let tmp = tempfile::tempdir().unwrap(); @@ -770,7 +780,11 @@ async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_resto let lock_path = tmp.path().join("yarn.lock"); let pristine = std::fs::read(&lock_path).unwrap(); - let env = run_redirect_subprocess(tmp.path(), &server.uri()); + let env = run_redirect_subprocess_with( + tmp.path(), + &server.uri(), + &["--patch-server-url", &server.uri()], + ); assert_eq!(env["redirect"]["redirected"], 1, "{label}: {env:#}"); assert!( warning_codes(&env).is_empty(), @@ -792,7 +806,11 @@ async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_resto vlt_hosted_common::assert_no_ledger(tmp.path()); // Re-run: in sync, byte-stable. - let env = run_redirect_subprocess(tmp.path(), &server.uri()); + let env = run_redirect_subprocess_with( + tmp.path(), + &server.uri(), + &["--patch-server-url", &server.uri()], + ); assert_eq!(env["redirect"]["redirected"], 1, "{label}: {env:#}"); assert_eq!( std::fs::read_to_string(&lock_path).unwrap(), @@ -801,17 +819,25 @@ async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_resto ); vlt_hosted_common::assert_no_ledger(tmp.path()); - let (code, env) = rollback_json(tmp.path(), &server); + let (code, env) = rollback_json_with_origin(tmp.path(), &server, &server.uri()); assert_eq!(code, Some(0), "{label}: rollback: {env:#}"); - assert_eq!(env["hosted"]["reverted"], serde_json::json!([PURL]), "{label}: {env:#}"); + assert_eq!( + env["hosted"]["reverted"], + serde_json::json!([PURL]), + "{label}: {env:#}" + ); let restored = std::fs::read_to_string(&lock_path).unwrap(); let checksum = berry_checksum_of(&restored); - assert_ne!(checksum, BERRY_CHECKSUM, "{label}: the patched checksum is gone"); + assert_ne!( + checksum, BERRY_CHECKSUM, + "{label}: the patched checksum is gone" + ); assert_eq!( restored, - String::from_utf8(pristine.clone()) - .unwrap() - .replace(&format!("10c0/{}", "3".repeat(128)), &format!("10c0/{checksum}")), + String::from_utf8(pristine.clone()).unwrap().replace( + &format!("10c0/{}", "3".repeat(128)), + &format!("10c0/{checksum}") + ), "{label}: rollback restores the pristine CRLF lock (upstream checksum \ re-derived from the registry tarball)" ); @@ -1198,13 +1224,21 @@ const INVALID_LOCKB_BYTES: &[u8] = b"\x00BUN-BINARY\xff\xfe\x00LOCK"; /// pointed at `registry` (`SOCKET_NPM_REGISTRY`, see [`mock_npm_registry`]); /// returns (exit code, parsed envelope). fn rollback_json(cwd: &Path, registry: &MockServer) -> (Option, serde_json::Value) { + rollback_json_with_origin(cwd, registry, "http://patch.test") +} + +fn rollback_json_with_origin( + cwd: &Path, + registry: &MockServer, + origin: &str, +) -> (Option, serde_json::Value) { let out = scrubbed_cli() .args([ "rollback", "--json", "--yes", "--patch-server-url", - "http://patch.test", + origin, "--cwd", cwd.to_str().unwrap(), ]) @@ -1245,6 +1279,16 @@ async fn mock_npm_registry(server: &MockServer, integrity: &str, tarball: Option .mount(server) .await; if let Some(bytes) = tarball { + let checksum = + socket_patch_core::vendor::test_support::service_fixture::berry_checksum(&bytes, NAME) + .unwrap(); + Mock::given(method("GET")) + .and(path(format!("/upstream/npm/{UUID}.json"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "name": NAME, "version": VERSION, "integrity": integrity, "yarnBerry10c0": checksum + }))) + .mount(server) + .await; Mock::given(method("GET")) .and(path(tarball_path)) .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) @@ -1260,7 +1304,10 @@ fn upstream_tarball() -> Vec { let package_json = format!(r#"{{ "name": "{NAME}", "version": "{VERSION}" }}"#); for (name, data) in [ ("package/package.json", package_json.as_bytes()), - ("package/index.js", b"module.exports = 'upstream'\n".as_slice()), + ( + "package/index.js", + b"module.exports = 'upstream'\n".as_slice(), + ), ] { let mut header = tar::Header::new_gnu(); header.set_size(data.len() as u64); @@ -1279,7 +1326,11 @@ fn berry_checksum_of(lock: &str) -> String { let rest = &lock[at..]; let line = rest .split('\n') - .find_map(|l| l.trim_end_matches('\r').trim().strip_prefix("checksum: 10c0/")) + .find_map(|l| { + l.trim_end_matches('\r') + .trim() + .strip_prefix("checksum: 10c0/") + }) .unwrap_or_else(|| panic!("no checksum in {rest:?}")); line.to_string() } @@ -1679,7 +1730,10 @@ async fn directory_at_the_legacy_ledger_path_does_not_block_the_run() { let code = run(redirect_args(tmp.path(), server.uri())).await; assert_eq!(code, 0, "the legacy ledger path is not consulted"); let lock = std::fs::read_to_string(tmp.path().join("package-lock.json")).unwrap(); - assert!(lock.contains(HOSTED_URL), "the lock is redirected; got:\n{lock}"); + assert!( + lock.contains(HOSTED_URL), + "the lock is redirected; got:\n{lock}" + ); assert!(squatter.is_dir(), "the squatting directory is untouched"); } @@ -3239,7 +3293,11 @@ async fn corrupt_pre_v5_ledger_is_ignored_and_left_untouched() { let out = run_hosted_json_scan(tmp.path(), &server).await; let stdout = String::from_utf8_lossy(&out.stdout); let stderr = String::from_utf8_lossy(&out.stderr); - assert_eq!(out.status.code(), Some(0), "stdout=\n{stdout}\nstderr=\n{stderr}"); + assert_eq!( + out.status.code(), + Some(0), + "stdout=\n{stdout}\nstderr=\n{stderr}" + ); let v: serde_json::Value = serde_json::from_str(&stdout).expect("parseable envelope"); assert_eq!(v["status"], "success", "{v:#}"); assert_eq!(v["redirect"]["redirected"], 1, "{v:#}"); diff --git a/crates/socket-patch-cli/tests/in_process_remove_repair_lifecycle.rs b/crates/socket-patch-cli/tests/in_process_remove_repair_lifecycle.rs index 2f795be6b..296d39679 100644 --- a/crates/socket-patch-cli/tests/in_process_remove_repair_lifecycle.rs +++ b/crates/socket-patch-cli/tests/in_process_remove_repair_lifecycle.rs @@ -5,6 +5,9 @@ //! (file/diff) and checks that the removed `package` mode fails hard. Both are run in-process so //! coverage is captured. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use serial_test::serial; @@ -881,7 +884,7 @@ mod vlt_vendored; /// `remove` then reverts the vlt wiring and deletes the artifact. #[tokio::test] #[serial] -async fn vlt_repair_rebuilds_the_dir_then_remove_reverts_it() { +async fn vlt_repair_redownloads_the_dir_then_remove_reverts_it() { use vlt_hosted_common as hosted; let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); @@ -890,7 +893,9 @@ async fn vlt_repair_rebuilds_the_dir_then_remove_reverts_it() { std::fs::remove_dir_all(&uuid_dir).unwrap(); let mut args = make_repair_args(root, "diff"); - args.common.offline = true; + let fixture = prebuilt_common::Server::project(root); + args.common.offline = false; + fixture.configure(&mut args.common); assert_eq!(repair_run(args).await, 0); assert_eq!( std::fs::read(root.join(vlt_vendored::rel()).join("index.js")).unwrap(), diff --git a/crates/socket-patch-cli/tests/in_process_rollback_vendored.rs b/crates/socket-patch-cli/tests/in_process_rollback_vendored.rs index 0c0da7380..78dd6d62e 100644 --- a/crates/socket-patch-cli/tests/in_process_rollback_vendored.rs +++ b/crates/socket-patch-cli/tests/in_process_rollback_vendored.rs @@ -24,13 +24,22 @@ //! `--offline` INTO the env, which every test here wants anyway), so none //! need `#[serial]` — each runs in its own tempdir. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + +async fn vendor_run(mut args: VendorArgs) -> i32 { + let server = prebuilt_common::Server::project(&args.common.cwd); + server.configure(&mut args.common); + actual_vendor_run(args).await +} + use std::path::{Path, PathBuf}; use std::process::Command; use serde_json::{json, Value}; use socket_patch_cli::args::GlobalArgs; use socket_patch_cli::commands::rollback::{run as rollback_run, RollbackArgs}; -use socket_patch_cli::commands::vendor::{run as vendor_run, VendorArgs}; +use socket_patch_cli::commands::vendor::{run as actual_vendor_run, VendorArgs}; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; #[path = "vex_e2e_common/mod.rs"] @@ -187,7 +196,7 @@ fn vendor_args(cwd: &Path) -> VendorArgs { cwd: cwd.to_path_buf(), json: true, silent: true, - offline: true, + offline: false, // Absorb the fork→exec OFD-lock window (see in_process_vendor.rs). lock_timeout: Some(5), ..GlobalArgs::default() diff --git a/crates/socket-patch-cli/tests/in_process_vendor.rs b/crates/socket-patch-cli/tests/in_process_vendor.rs index e701c7f79..461b6d41a 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor.rs @@ -19,13 +19,22 @@ //! No test mutates this process's environment, so none of them need //! `#[serial]` — each runs in its own tempdir. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + +async fn vendor_run(mut args: VendorArgs) -> i32 { + let server = prebuilt_common::Server::project(&args.common.cwd); + server.configure(&mut args.common); + actual_vendor_run(args).await +} + use std::path::{Path, PathBuf}; use std::process::Command; use serde_json::{json, Value}; use sha2::{Digest, Sha256}; use socket_patch_cli::args::GlobalArgs; -use socket_patch_cli::commands::vendor::{run as vendor_run, VendorArgs}; +use socket_patch_cli::commands::vendor::{run as actual_vendor_run, VendorArgs}; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; #[path = "npm_e2e_common/manifestless.rs"] @@ -203,7 +212,7 @@ fn vendor_args(cwd: &Path) -> VendorArgs { cwd: cwd.to_path_buf(), json: true, silent: true, - offline: true, + offline: false, // flock guards are OFD-based: when a CONCURRENT test in this // binary forks a subprocess, the pre-exec child briefly holds // copies of every parent fd — including this test's just-dropped @@ -256,6 +265,13 @@ fn run_cli(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> (i32, Strin for (k, v) in extra_env { cmd.env(k, v); } + let server = prebuilt_common::Server::project(cwd); + if !args.contains(&"--api-url") + && !args.contains(&"--vendor-url") + && !extra_env.iter().any(|(k, _)| *k == "SOCKET_VENDOR_URL") + { + server.command(&mut cmd); + } let out = cmd.output().expect("spawn socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -267,13 +283,7 @@ fn run_cli(cwd: &Path, args: &[&str], extra_env: &[(&str, &str)]) -> (i32, Strin /// `vendor --json --offline --cwd ` through the binary, /// returning `(exit_code, parsed envelope)`. fn vendor_cli(cwd: &Path, extra: &[&str]) -> (i32, Value) { - let mut args = vec![ - "vendor", - "--json", - "--offline", - "--cwd", - cwd.to_str().unwrap(), - ]; + let mut args = vec!["vendor", "--json", "--cwd", cwd.to_str().unwrap()]; args.extend_from_slice(extra); let (code, stdout, stderr) = run_cli(cwd, &args, &[]); let env: Value = serde_json::from_str(&stdout).unwrap_or_else(|e| { @@ -554,7 +564,7 @@ async fn unsupported_ecosystem_purl_is_a_benign_skip() { // ───────────────────────────────────────────────────────────────────── #[tokio::test] -async fn package_not_installed_fails() { +async fn package_absent_from_lockfile_fails() { // The manifest names a package that is nowhere in node_modules. The // user asked for it to be vendored and it wasn't — that is a partial // failure (exit 1), surfaced as a skipped event with the stable code. @@ -565,7 +575,7 @@ async fn package_not_installed_fails() { "an unsatisfiable manifest entry must exit 1: {env:#}" ); assert_eq!(env["status"], "partialFailure"); - let skipped = find_event(&env, "skipped", Some("package_not_installed")); + let skipped = find_event(&env, "failed", Some("vendor_lock_entry_not_found")); assert_eq!(skipped["purl"], "pkg:npm/ghost-pkg@9.9.9"); assert!( !fx.vendor_dir().exists(), @@ -1157,6 +1167,12 @@ async fn mount_npm_registry( let tarball_path = format!("/{name}/-/{name}-{version}.tgz"); let tarball = format!("{}{tarball_path}", server.uri()); let integrity = sri_sha512(&tgz); + Mock::given(method("GET")) + .and(path(format!("/upstream/npm/{UUID}.json"))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "name": name, "version": version, "integrity": integrity, + "yarnBerry10c0": socket_patch_core::vendor::test_support::service_fixture::berry_checksum(&tgz, name).unwrap() + }))).mount(server).await; Mock::given(method("GET")) .and(path(format!("/{name}/{version}"))) .respond_with(ResponseTemplate::new(200).set_body_json(json!({ @@ -1195,7 +1211,14 @@ fn vendor_online_cli( let mut args = vec!["vendor", "--json", "--cwd", cwd.to_str().unwrap()]; args.extend_from_slice(extra); let env = online_env(registry, patch_server); - let env: Vec<(&str, &str)> = env.iter().map(|(k, v)| (*k, v.as_str())).collect(); + let mut env: Vec<(&str, &str)> = env + .iter() + .filter(|(k, v)| !(*k == "SOCKET_PATCH_SERVER_URL" && v == "https://patch.socket.dev")) + .map(|(k, v)| (*k, v.as_str())) + .collect(); + if patch_server != "https://patch.socket.dev" { + env.push(("SOCKET_VENDOR_URL", patch_server)); + } let (code, stdout, stderr) = run_cli(cwd, &args, &env); let envelope: Value = serde_json::from_str(&stdout).unwrap_or_else(|e| { panic!("vendor --json must emit an envelope: {e}\nstdout:\n{stdout}\nstderr:\n{stderr}") @@ -1268,6 +1291,7 @@ async fn berry_crlf_takeovers_round_trip_both_directions() { "hosted mode writes no ledger" ); + prebuilt_common::mount_project(&server, root).await; let (code, env) = vendor_online_cli(root, &server.uri(), &server.uri(), &[]); assert_eq!(code, 0, "vendor over the hosted pin: {env:#}"); assert_eq!(env["summary"]["applied"], 1, "{env:#}"); @@ -1306,6 +1330,15 @@ async fn berry_crlf_takeovers_round_trip_both_directions() { stage_berry_project(root, &pkg, &lock); let (code, env) = vendor_cli(root, &[]); assert_eq!(code, 0, "vendor: {env:#}"); + server.reset().await; + mount_berry_hosted_api(&server).await; + mount_npm_registry( + &server, + "left-pad", + "1.3.0", + npm_tgz("left-pad", "1.3.0", ORIG_INDEX), + ) + .await; let (code, env) = hosted_scan_cli(root, &server.uri()); assert_eq!(code, 0, "hosted scan over the vendored pair: {env:#}"); assert_eq!(env["redirect"]["redirected"], 1, "{env:#}"); @@ -1560,21 +1593,14 @@ async fn berry_takeovers_refuse_before_reverting_the_old_mode() { // ───────────────────────────────────────────────────────────────────── #[tokio::test] -async fn offline_missing_source_fails() { +async fn offline_missing_artifact_fails() { let fx = npm_fixture(); - // Remove the staged blob: offline + no blob/diff/package ⇒ the patch has - // no usable local source and vendor must fail loudly, not guess. - std::fs::remove_file(fx.root().join(".socket/blobs").join(&fx.after_hash)).unwrap(); - - let (code, env) = vendor_cli(fx.root(), &[]); - assert_eq!(code, 1, "offline with no local source must exit 1: {env:#}"); - assert_eq!(env["status"], "error"); - assert_eq!(env["error"]["code"], "no_local_source"); - assert!( - !fx.vendor_dir().exists(), - "a failed staging must write nothing" - ); - assert_eq!(fx.lock_bytes(), fx.original_lock, "lock untouched"); + let (code, env) = vendor_cli(fx.root(), &["--offline"]); + assert_eq!(code, 1, "{env:#}"); + assert_eq!(env["status"], "partialFailure"); + find_event(&env, "failed", Some("vendor_service_offline_conflict")); + assert!(!fx.vendor_dir().exists()); + assert_eq!(fx.lock_bytes(), fx.original_lock); } // ───────────────────────────────────────────────────────────────────── @@ -1968,6 +1994,7 @@ async fn vendored_golang_purl_skipped_by_apply() { base_purl: purl.clone(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/golang/{UUID}/{MODULE}@{VERSION}"), sha256: String::new(), size: None, @@ -2198,7 +2225,7 @@ fn vendor_after_in_place_apply_emits_applied_event() { /// 0 with an `applied` event, and the overwrite surfaces as a /// `vendor_content_mismatch_overwritten` warning event. #[test] -fn mismatched_baseline_vendors_with_warning_event() { +fn mismatched_install_does_not_change_the_server_artifact() { let fx = npm_fixture(); std::fs::write( fx.installed_index(), @@ -2210,7 +2237,7 @@ fn mismatched_baseline_vendors_with_warning_event() { assert_eq!(code, 0, "{env:#}"); let applied = find_event(&env, "applied", None); assert_eq!(applied["purl"], PURL); - let warning = find_event(&env, "skipped", Some("vendor_content_mismatch_overwritten")); + let warning = find_event(&env, "skipped", Some("vendor_prebuilt_downloaded")); assert!( warning["reason"] .as_str() @@ -2229,43 +2256,17 @@ fn mismatched_baseline_vendors_with_warning_event() { ); } -/// A patch-target file MISSING from the installed package still fails closed -/// (auto-force must not inherit `--force`'s silent NotFound skip — the -/// tarball would ship without the fix); `--force` keeps that tolerance. +/// Vendoring does not require the patch target in the installed package. #[test] -fn vendor_missing_file_fails_closed_without_force() { - let fx = npm_fixture(); - std::fs::remove_file(fx.installed_index()).unwrap(); - - let (code, env) = vendor_cli(fx.root(), &[]); - assert_ne!(code, 0, "missing patch target must fail: {env:#}"); - // CONTRACT: even a run where EVERY outcome failed reports - // "partialFailure". The envelope has no "completed with zero successes" - // status, and status=error is reserved for pre-event failures (it implies - // a top-level error payload and empty events[] — json_envelope.rs), so - // escalating this run to "error" would violate the contract and diverge - // from scan --vendor and vendor --revert, which report the same outcome - // as partialFailure. Exit code 1 carries the failure signal. - assert_eq!( - env["status"], "partialFailure", - "all-failed vendor runs report partialFailure per the envelope contract: {env:#}" - ); - let failed = find_event(&env, "failed", None); - assert!( - failed["error"] - .as_str() - .unwrap_or("") - .contains("File not found"), - "{env:#}" - ); - assert_eq!(fx.lock_bytes(), fx.original_lock, "lock byte-untouched"); - assert!(!fx.vendor_dir().exists(), "no artifacts on failure"); - - // --force: the missing file is tolerated (skipped) and the vendor lands. - let fx2 = npm_fixture(); - std::fs::remove_file(fx2.installed_index()).unwrap(); - let (code, env) = vendor_cli(fx2.root(), &["--force"]); - assert_eq!(code, 0, "{env:#}"); +fn vendor_uses_server_artifact_when_installed_file_is_missing() { + for extra in [vec![], vec!["--force"]] { + let fx = npm_fixture(); + std::fs::remove_file(fx.installed_index()).unwrap(); + let (code, env) = vendor_cli(fx.root(), &extra); + assert_eq!(code, 0, "{env:#}"); + assert!(fx.tgz_path().exists()); + assert!(!fx.installed_index().exists()); + } } // ──────────────── percent-encoded scoped purls (Fix A integration) ──────────────── @@ -2475,6 +2476,7 @@ async fn offline_service_mode_refuses_instead_of_building() { let fx = npm_fixture(); let mut args = vendor_args(fx.root()); args.common.vendor_source = "service".to_string(); + args.common.offline = true; let code = vendor_run(args).await; assert_ne!( @@ -2567,6 +2569,29 @@ async fn mount_gem_patch_api(mock: &wiremock::MockServer, patch_purl: &str) { use wiremock::matchers::{method, path}; use wiremock::{Mock, ResponseTemplate}; + let fx = gem_fixture(); + let record: socket_patch_core::manifest::schema::PatchRecord = serde_json::from_value(json!({ + "uuid": GEM_UUID, "exportedAt": "2026-01-01T00:00:00Z", + "files": { "lib/demo_gem.rb": { "beforeHash": compute_git_sha256_from_bytes(GEM_ORIG), "afterHash": compute_git_sha256_from_bytes(GEM_PATCHED) } }, + "vulnerabilities": {}, "description": "gem vendor patch", "license": "MIT", "tier": "free" + })).unwrap(); + let blobs = std::collections::HashMap::from([( + compute_git_sha256_from_bytes(GEM_PATCHED), + GEM_PATCHED.to_vec(), + )]); + let sources = socket_patch_core::patch::apply::PatchSources { + blobs_path: fx.root(), + diffs_path: None, + mem_blobs: Some(&blobs), + }; + prebuilt_common::mount_record( + mock, + patch_purl, + &record, + fx.installed_lib().parent().unwrap().parent().unwrap(), + &sources, + ) + .await; const ORG_SLUG: &str = "test-org"; /// The exact percent-encoded by-package path segment for the BARE purl — /// the spelling the crawler synthesizes and the CLI queries with (the @@ -3030,12 +3055,9 @@ async fn scan_vendor_gem_artifact_rebuild_without_ledger_entry_records_none() { std::fs::remove_file(fx.state_path()).unwrap(); std::fs::remove_file(fx.vendored_lib()).unwrap(); let (code, env2) = run_scan_vendor(fx.root(), &mock.uri(), &[]); - assert_eq!(code, 0, "rebuild run: {env2:#}"); - assert_eq!( - std::fs::read(fx.vendored_lib()).unwrap(), - GEM_PATCHED, - "the copy is still rebuilt" - ); + assert_eq!(code, 1, "redownload refuses missing identity: {env2:#}"); + assert!(env2.to_string().contains("vendor_ledger_entry_missing")); + assert!(!fx.vendored_lib().exists()); let recorded = std::fs::read(fx.state_path()) .ok() .and_then(|b| serde_json::from_slice::(&b).ok()) @@ -3070,12 +3092,9 @@ async fn scan_vendor_gem_artifact_rebuild_over_other_uuid_entry_keeps_ledger() { std::fs::remove_file(fx.vendored_lib()).unwrap(); let (code, env2) = run_scan_vendor(fx.root(), &mock.uri(), &[]); - assert_eq!(code, 0, "rebuild run: {env2:#}"); - assert_eq!( - std::fs::read(fx.vendored_lib()).unwrap(), - GEM_PATCHED, - "the copy is still rebuilt" - ); + assert_eq!(code, 1, "redownload refuses missing identity: {env2:#}"); + assert!(env2.to_string().contains("vendor_ledger_entry_missing")); + assert!(!fx.vendored_lib().exists()); let after: Value = serde_json::from_slice(&std::fs::read(fx.state_path()).unwrap()).unwrap(); assert_eq!( after["entries"][GEM_PURL]["uuid"], other, @@ -3122,7 +3141,7 @@ mod hosted_to_vendor_conversion { const CONV_VERSION: &str = "1.0.0"; const CONV_PURL: &str = "pkg:npm/conv-pnpm-takeover@1.0.0"; const CONV_UUID: &str = "44444444-4444-4444-8444-444444444444"; - const HOSTED_URL: &str = "http://patch.test/patch/npm/conv-pnpm-takeover/1.0.0/55555555-5555-4555-8555-555555555555/44444444-4444-4444-8444-444444444444/conv-pnpm-takeover-1.0.0.tgz"; + const HOSTED_URL: &str = "https://patch.socket.dev/patch/npm/conv-pnpm-takeover/1.0.0/55555555-5555-4555-8555-555555555555/44444444-4444-4444-8444-444444444444/conv-pnpm-takeover-1.0.0.tgz"; const PATCHED_SHA512: &str = "sha512-PATCHEDpatchedPATCHEDpatched0123456789=="; const UPSTREAM_SHA512: &str = "sha512-UPSTREAMupstream=="; @@ -3306,7 +3325,7 @@ snapshots: /// The hosted URLs above live on this origin: only /// `https://patch.socket.dev` and the configured patch-server origin /// count as hosted, so every post-scan run passes it. - const PATCH_ORIGIN: &str = "http://patch.test"; + const PATCH_ORIGIN: &str = "https://patch.socket.dev"; /// The npm registry's version document for the fixture package, as the /// v5 upstream restore reads it (`SOCKET_NPM_REGISTRY`): it hands back @@ -3436,7 +3455,7 @@ snapshots: let hosted_lock = std::fs::read(root.join("pnpm-lock.yaml")).unwrap(); seed_manifest_and_blob(root); - let (code, env) = vendor_cli(root, &["--patch-server-url", PATCH_ORIGIN]); + let (code, env) = vendor_cli(root, &["--offline", "--patch-server-url", PATCH_ORIGIN]); assert_eq!(code, 1, "{env:#}"); let failed = find_event(&env, "failed", Some("redirect_revert_failed")); assert!( diff --git a/crates/socket-patch-cli/tests/in_process_vendor/vlt.rs b/crates/socket-patch-cli/tests/in_process_vendor/vlt.rs index 647d6c7fc..8f0e37daf 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor/vlt.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor/vlt.rs @@ -511,7 +511,7 @@ fn vendor_vlt_package_json_patch_with_devdeps_verifies() { &[], ); assert_eq!(code, 0, "{env:#}"); - assert!(env.to_string().contains("wouldRebuild"), "{env:#}"); + assert!(env.to_string().contains("wouldRedownload"), "{env:#}"); } // ── refusals ───────────────────────────────────────────────────────────── @@ -822,13 +822,14 @@ fn vendor_vlt_auto_fetch_stages_the_committed_dir_artifact() { std::os::unix::fs::symlink(root.join(&rel), &link).unwrap(); #[cfg(windows)] std::os::windows::fs::symlink_dir(root.join(&rel), &link).unwrap(); + let ledger = read(root, ".socket/vendor/state.json"); std::fs::write(root.join(&rel).join("extra.js"), "tampered\n").unwrap(); let (code, env, _) = vendor(root, &[]); - assert_eq!(code, 1, "{env:#}"); - let (got, detail) = failure(&env, PURL); - assert_eq!(got, "vendor_fetch_failed", "{env:#}"); - assert!(detail.contains("socket-patch repair"), "{detail}"); + assert_eq!(code, 0, "{env:#}"); + assert!(env.to_string().contains("redownloaded"), "{env:#}"); + assert!(!root.join(&rel).join("extra.js").exists()); assert_eq!(read(root, VLT_LOCK), lock); + assert_eq!(read(root, ".socket/vendor/state.json"), ledger); } #[test] @@ -977,19 +978,10 @@ fn vendor_vlt_revendor_new_uuid_never_builds_from_the_old_dir() { let root = tmp.path(); superseded_linked_project(root, &direct_lock()); let lock = read(root, VLT_LOCK); - let (code, env, _) = vendor(root, &[]); + let (code, env, _) = vendor(root, &["--offline"]); assert_eq!(code, 1, "{env:#}"); - let skip = events(&env) - .into_iter() - .find(|e| e["errorCode"] == "package_not_installed") - .unwrap_or_else(|| panic!("{env:#}")); - assert_eq!( - skip["reason"], - format!( - "the only installed copy is the vendored artifact .socket/vendor/npm/{UUID}/ of \ - patch {UUID}, which is not a pristine source for this patch; --offline prevents \ - fetching the pristine artifact from the registry" - ), + assert!( + env.to_string().contains("vendor_service_offline_conflict"), "{env:#}" ); assert!(!uuid_dir(root, UUID2).exists(), "{env:#}"); @@ -998,60 +990,45 @@ fn vendor_vlt_revendor_new_uuid_never_builds_from_the_old_dir() { } #[tokio::test(flavor = "multi_thread")] -async fn vendor_vlt_revendor_new_uuid_fetches_the_pristine_package() { +async fn vendor_vlt_new_uuid_downloads_independent_server_artifact() { let server = MockServer::start().await; - let pristine = tarball( - &[ - ( - "package/package.json", - b"{\"name\":\"left-pad\",\"version\":\"1.3.0\"}\n", - ), - ("package/index.js", PRISTINE), - ("package/extra.js", EXTRA.1), - ], - &[], - ); - Mock::given(method("GET")) - .and(path("/left-pad/-/left-pad-1.3.0.tgz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(pristine.clone())) - .mount(&server) - .await; - let url = format!("{}/left-pad/-/left-pad-1.3.0.tgz", server.uri()); - let lock = Lock::v1( - &[&format!( - "\"~npm~left-pad@1.3.0\": [0,\"{NAME}\",\"{}\",\"{url}\"]", - sri(&pristine) - )], - &["\"file~_d left-pad\": \"prod 1.3.0 ~npm~left-pad@1.3.0\""], - ); let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - superseded_linked_project(root, &lock); - let cwd = root.to_str().unwrap().to_string(); - let (code, env, stderr) = socket( - root, - &[ - "vendor", - "--json", - "--vendor-source", - "build", - "--cwd", - &cwd, - ], - &[], - ); + superseded_linked_project(root, &direct_lock()); + let package = tempfile::tempdir().unwrap(); + std::fs::write( + package.path().join("package.json"), + b"{\"name\":\"left-pad\",\"version\":\"1.3.0\"}", + ) + .unwrap(); + std::fs::write(package.path().join("index.js"), PRISTINE).unwrap(); + std::fs::write(package.path().join("extra.js"), EXTRA.1).unwrap(); + let manifest: socket_patch_core::manifest::schema::PatchManifest = + serde_json::from_slice(&std::fs::read(root.join(".socket/manifest.json")).unwrap()) + .unwrap(); + let blobs = root.join(".socket/blobs"); + let sources = socket_patch_core::patch::apply::PatchSources { + blobs_path: &blobs, + diffs_path: None, + mem_blobs: None, + }; + crate::prebuilt_common::mount_record( + &server, + PURL, + &manifest.patches[PURL], + package.path(), + &sources, + ) + .await; + let (code, env, stderr) = vendor_via_service(root, &server.uri(), &[]); assert_eq!(code, 0, "{env:#}\n{stderr}"); assert!( - codes(&env).contains(&"vendor_fetched_missing".to_string()), + codes(&env).contains(&"vendor_prebuilt_downloaded".to_string()), "{env:#}" ); let rel2 = rel_dir(UUID2, NAME, VERSION); assert_eq!(read(root, &format!("{rel2}/index.js")).as_bytes(), PATCHED); - assert_eq!( - read(root, &format!("{rel2}/extra.js")).as_bytes(), - EXTRA.1, - "the old patch's extra.js never reaches the new artifact" - ); + assert_eq!(read(root, &format!("{rel2}/extra.js")).as_bytes(), EXTRA.1); assert!(!uuid_dir(root, UUID).exists()); assert_eq!(ledger_entry(root, PURL)["uuid"], UUID2); } @@ -1413,24 +1390,23 @@ async fn vendor_vlt_service_policy_fail_closed() { } #[tokio::test(flavor = "multi_thread")] -async fn vendor_vlt_service_auto_falls_back_to_build() { +async fn vendor_vlt_service_auto_refuses_without_local_build() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); direct_project(root); let server = MockServer::start().await; mount_status(&server, 503, json!({"error": "down"})).await; let (code, env, stderr) = vendor_via_service(root, &server.uri(), &[]); - assert_eq!(code, 0, "{env:#}\n{stderr}"); + assert_eq!(code, 1, "{env:#}\n{stderr}"); assert!( - codes(&env).contains(&"vendor_prebuilt_unavailable".to_string()), + env.to_string().contains("patch service request failed"), "{env:#}" ); - let rel = rel_dir(UUID, NAME, VERSION); - assert_eq!(read(root, &format!("{rel}/index.js")).as_bytes(), PATCHED); + assert!(!uuid_dir(root, UUID).exists()); } #[tokio::test(flavor = "multi_thread")] -async fn vendor_vlt_service_pending_falls_back_to_build() { +async fn vendor_vlt_service_pending_refuses_without_local_build() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); direct_project(root); @@ -1442,11 +1418,9 @@ async fn vendor_vlt_service_pending_falls_back_to_build() { ) .await; let (code, env, stderr) = vendor_via_service(root, &server.uri(), &[]); - assert_eq!(code, 0, "{env:#}\n{stderr}"); - assert!( - codes(&env).contains(&"vendor_prebuilt_pending".to_string()), - "{env:#}" - ); + assert_eq!(code, 1, "{env:#}\n{stderr}"); + assert!(env.to_string().contains("still building"), "{env:#}"); + assert!(!uuid_dir(root, UUID).exists()); } async fn assert_archive_refused(special: (&str, tar::EntryType, &str)) { @@ -1500,7 +1474,7 @@ fn vendor_vlt_human_hints_name_vlt_files_and_install() { let tmp = tempfile::tempdir().unwrap(); direct_project(tmp.path()); let cwd = tmp.path().to_str().unwrap(); - let (code, stdout, stderr) = socket_human(tmp.path(), &["vendor", "--offline", "--cwd", cwd]); + let (code, stdout, stderr) = socket_human(tmp.path(), &["vendor", "--cwd", cwd]); assert_eq!(code, 0, "{stdout}\n{stderr}"); assert!( stdout.contains("vlt-lock.json and .socket/vendor/") && stdout.contains("CI: `vlt ci`"), diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs index 9ff7df128..9bb8c3882 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs @@ -15,7 +15,7 @@ //! `rollback`, `remove`) restores the registry 4-tuple, re-resolving the //! integrity from the npm registry: here one shared wiremock mirror //! ([`registry_uri`], `SOCKET_NPM_REGISTRY`) serving the pristine -//! integrities, with the `http://patch.test` origin named hosted via +//! integrities, with the `https://patch.socket.dev` origin named hosted via //! `SOCKET_PATCH_SERVER_URL`. //! //! Scenarios: @@ -39,6 +39,9 @@ //! Every child process gets the ambient `SOCKET_*` vars scrubbed and //! telemetry hard-disabled; each test runs in its own tempdir. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use std::process::Command; @@ -62,7 +65,7 @@ const PURL: &str = "pkg:npm/left-pad@1.3.0"; /// Canonical-grammar patch uuid (the vendor path layer validates the uuid /// path level fail-closed). const UUID: &str = "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"; -const HOSTED_URL: &str = "http://patch.test/patch/npm/left-pad/1.3.0/55555555-5555-4555-8555-555555555555/9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f/left-pad-1.3.0.tgz"; +const HOSTED_URL: &str = "https://patch.socket.dev/patch/npm/left-pad/1.3.0/55555555-5555-4555-8555-555555555555/9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f/left-pad-1.3.0.tgz"; const PATCHED_SHA512: &str = "sha512-PATCHEDpatchedPATCHEDpatched0123456789=="; const ORIG_INDEX: &[u8] = b"module.exports = () => 'orig';\n"; const PATCHED_INDEX: &[u8] = b"module.exports = () => 'patched';\n"; @@ -74,7 +77,7 @@ const CVE: &str = "CVE-2026-5555"; /// The second hosted record of the scoped-unwind scenarios. const OTHER_NAME: &str = "other"; const OTHER_PURL: &str = "pkg:npm/other@1.0.0"; -const OTHER_HOSTED_URL: &str = "http://patch.test/patch/npm/other/1.0.0/55555555-5555-4555-8555-555555555555/0a1b2c3d-4e5f-4a7b-8c9d-0e1f2a3b4c5d/other-1.0.0.tgz"; +const OTHER_HOSTED_URL: &str = "https://patch.socket.dev/patch/npm/other/1.0.0/55555555-5555-4555-8555-555555555555/0a1b2c3d-4e5f-4a7b-8c9d-0e1f2a3b4c5d/other-1.0.0.tgz"; /// The registry 4-tuple lines exactly as bun 1.4.2 emits them (matrix /// capture grammar; the `""` registry field is the default registry). @@ -279,7 +282,7 @@ fn manifestless_vex( vulns: &[(GHSA, &[CVE])], lock: "bun.lock", registry_lock: pristine.to_vec(), - patch_server_url: Some("http://patch.test".to_string()), + patch_server_url: Some("https://patch.socket.dev".to_string()), }; bun_vex::run_bun_vex_matrix(root, scratch, &case, |_| {}); } @@ -340,20 +343,20 @@ fn registry_uri() -> &'static str { /// Run the built `socket-patch` binary with every ambient `SOCKET_*` var /// scrubbed (except the hermetic `SOCKET_NO_CONFIG`) and telemetry -/// hard-disabled; `http://patch.test` counts as the patch server +/// hard-disabled; `https://patch.socket.dev` counts as the patch server /// (`SOCKET_PATCH_SERVER_URL`) and the registry is the shared mirror /// (`SOCKET_NPM_REGISTRY`). Returns `(exit_code, stdout, stderr)`. fn run_cli(cwd: &Path, args: &[&str]) -> (i32, String, String) { let mut cmd = Command::new(env!("CARGO_BIN_EXE_socket-patch")); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (key, _) in std::env::vars() { if key.starts_with("SOCKET_") && key != "SOCKET_NO_CONFIG" { cmd.env_remove(key); } } cmd.env("SOCKET_TELEMETRY_DISABLED", "1") - .env("SOCKET_PATCH_SERVER_URL", "http://patch.test") .env("SOCKET_NPM_REGISTRY", registry_uri()); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); let out = cmd.output().expect("spawn socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -392,6 +395,17 @@ fn scan_mode(cwd: &Path, api_url: &str, mode: &str, extra: &[&str]) -> (i32, Val "--cwd", cwd.to_str().unwrap(), ]; + let fixture = (mode == "vendored").then(|| { + let mut view = patch_record(UUID); + view["purl"] = json!(PURL); + view["publishedAt"] = json!("2024-01-01T00:00:00Z"); + view["files"]["package/index.js"]["blobContent"] = + json!(base64::engine::general_purpose::STANDARD.encode(PATCHED_INDEX)); + prebuilt_common::Server::view(view) + }); + if let Some(fixture) = &fixture { + args.extend(["--vendor-url", &fixture.uri]); + } args.extend_from_slice(extra); run_json(cwd, &args) } @@ -979,7 +993,7 @@ async fn bun_hosted_refusal_preserves_vendored_v0_workspace() { .replace(" \"configVersion\": 1,\n", ""); write_bun_project(root, &direct, &[(NAME, VERSION)]); seed_manifest_and_blob(root); - let (code, env) = vendor_cli(root, &["--vendor-source", "build"]); + let (code, env) = vendor_cli(root, &["--vendor-source", "service"]); assert_eq!(code, 0, "{env:#}"); // Bun 1.1.45 preserves the local tuple when a direct project grows an diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs index 347c36e25..89cb4f4a7 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs @@ -14,6 +14,9 @@ //! vendored preflight in front of a hosted revert, and the hosted //! artifact preflight in front of a vendored revert. +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use base64::Engine as _; @@ -145,16 +148,18 @@ fn scan(root: &Path, server: &MockServer, mode: &str, extra: &[&str]) -> (i32, V /// `vendor --offline`: nothing hosted to restore. fn vendor(root: &Path, extra: &[&str]) -> (i32, Value, String) { let cwd = root.to_str().unwrap().to_string(); - let mut argv = vec!["vendor", "--offline", "--cwd", &cwd]; + let fixture = prebuilt_common::Server::project(root); + let mut argv = vec!["vendor", "--cwd", &cwd]; argv.extend_from_slice(extra); - hosted::run_json(root, &argv, &[]) + hosted::run_json(root, &argv, &[("SOCKET_VENDOR_URL", &fixture.uri)]) } /// `vendor` over a hosted pin on `server`: online, the takeover's upstream /// restore reads the registry mirror. fn vendor_online(root: &Path, server: &MockServer, extra: &[&str]) -> (i32, Value, String) { let cwd = root.to_str().unwrap().to_string(); - let mut argv = vec!["vendor", "--cwd", &cwd]; + let uri = server.uri(); + let mut argv = args(&cwd, &uri, &["vendor"]); argv.extend_from_slice(extra); run_online(root, &argv, server) } diff --git a/crates/socket-patch-cli/tests/mode_migration_bun.rs b/crates/socket-patch-cli/tests/mode_migration_bun.rs index 46dced8c7..67c473d3b 100644 --- a/crates/socket-patch-cli/tests/mode_migration_bun.rs +++ b/crates/socket-patch-cli/tests/mode_migration_bun.rs @@ -400,9 +400,8 @@ fn hosted_scan(proj: &Path, api: &str, extra: &[&str]) -> (i32, String, String) run_socket(proj, &args) } -/// `scan --mode vendored` builds the artifact locally (`--vendor-source -/// build`): no vendoring-service round trip, so the only network is the -/// wiremock patch API. +/// `scan --mode vendored` downloads the immutable artifact from the +/// fixture service mounted alongside the patch API. fn vendored_scan(proj: &Path, api: &str, extra: &[&str]) -> (i32, String, String) { let mut args = vec![ "scan", @@ -419,7 +418,7 @@ fn vendored_scan(proj: &Path, api: &str, extra: &[&str]) -> (i32, String, String "--api-token", "fake", "--vendor-source", - "build", + "service", ]; args.extend_from_slice(extra); run_socket(proj, &args) @@ -993,6 +992,15 @@ async fn mount_hosted_api( .mount(server) .await; } + // Public service grants cover both the staged and API-discovered UUIDs. + for p in &patches { + results.insert(p.dep.uuid_v.to_string(), results[p.dep.uuid_h].clone()); + } + Mock::given(method("POST")) + .and(path("/patch/package")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ "results": results }))) + .mount(server) + .await; Mock::given(method("POST")) .and(path(format!("/v0/orgs/{ORG}/patches/package"))) .respond_with( @@ -1030,6 +1038,7 @@ async fn mount_hosted_api( } *ONLINE_ENV.lock().unwrap_or_else(|e| e.into_inner()) = vec![ ("SOCKET_PATCH_SERVER_URL".to_string(), server.uri()), + ("SOCKET_VENDOR_URL".to_string(), server.uri()), ( "SOCKET_NPM_REGISTRY".to_string(), format!("{}/registry", server.uri()), @@ -1520,6 +1529,9 @@ async fn bun_vendored_then_hosted_takeover_leaves_pure_hosted() { }; let proj = fx.proj.clone(); + let server = MockServer::start().await; + let patches = mount_hosted_api(&server, &fx, &[&DEP_A]).await; + // A: vendor from the staged manifest. stage_manifest(&fx, &proj, &DEP_A); let (code, stdout, stderr) = vendor_cmd(&proj, &[]); @@ -1538,8 +1550,6 @@ async fn bun_vendored_then_hosted_takeover_leaves_pure_hosted() { // B: hosted redirect over the vendored state — the takeover — then the // fresh-checkout marker proof. - let server = MockServer::start().await; - let patches = mount_hosted_api(&server, &fx, &[&DEP_A]).await; take_over_to_hosted(&fx, &proj, &server.uri(), &patches[0], "rev"); // C: unscoped rollback → pristine bytes, no vendor artifacts or diff --git a/crates/socket-patch-cli/tests/mode_migration_cargo.rs b/crates/socket-patch-cli/tests/mode_migration_cargo.rs index ec54b19b2..65753a34b 100644 --- a/crates/socket-patch-cli/tests/mode_migration_cargo.rs +++ b/crates/socket-patch-cli/tests/mode_migration_cargo.rs @@ -38,6 +38,9 @@ //! the fixture build (a failure instead under //! `SOCKET_PATCH_CARGO_E2E_REQUIRED=1`); all assertions after that are hard. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -82,7 +85,7 @@ fn run_socket_env( env: &[(&str, &str)], ) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); for (k, _) in std::env::vars_os() { if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { cmd.env_remove(&k); @@ -93,6 +96,12 @@ fn run_socket_env( for (k, v) in env { cmd.env(k, v); } + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + cwd, + args, + &[("CARGO_HOME", cargo_home.to_str().unwrap())], + ); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -287,6 +296,14 @@ async fn mount_hosted_mocks( orig: &[u8], patched: &[u8], ) -> String { + prebuilt_common::mount_download( + server, + purl, + UUID_V, + &format!("{DEP}-{version}.crate"), + crate_bytes, + ) + .await; let cksum = sha256_hex(crate_bytes); // Production-shaped index path: manifest-less VEX reads the hosted // uuid out of the lock's `source` (with `--patch-server-url`). diff --git a/crates/socket-patch-cli/tests/mode_migration_npm.rs b/crates/socket-patch-cli/tests/mode_migration_npm.rs index 7e7a3b718..ab7f98ee8 100644 --- a/crates/socket-patch-cli/tests/mode_migration_npm.rs +++ b/crates/socket-patch-cli/tests/mode_migration_npm.rs @@ -22,6 +22,9 @@ //! the registry is unreachable for the fixture install; all assertions after //! that are hard. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::{Command, Output, Stdio}; @@ -118,11 +121,12 @@ fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { /// [`run_socket`] with extra env applied after the scrub. fn run_socket_env(cwd: &Path, args: &[&str], env: &[(&str, &str)]) -> (i32, String, String) { let mut cmd = Command::new(binary()); - cmd.args(args).current_dir(cwd); + cmd.current_dir(cwd); scrub_socket_env(&mut cmd); for (k, v) in env { cmd.env(k, v); } + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, env); let out = cmd.output().expect("failed to run socket-patch binary"); ( out.status.code().unwrap_or(-1), @@ -292,6 +296,14 @@ async fn mount_hosted_mocks( patched: &[u8], berry_checksum: Option<&str>, ) -> String { + prebuilt_common::mount_download( + server, + PURL, + UUID_V, + &format!("{DEP}-{DEP_VERSION}.tgz"), + tgz, + ) + .await; let hosted_url = format!( "{}/patch/npm/{DEP}/{DEP_VERSION}/{TOKEN}/{UUID_H}/{DEP}-{DEP_VERSION}.tgz", server.uri() @@ -331,6 +343,40 @@ async fn mount_hosted_mocks( "integrity": { "sha512": sha512_sri(tgz), "sha1": sha1_hex(tgz) } })]; if let Some(checksum) = berry_checksum { + let client = reqwest::Client::new(); + let metadata: serde_json::Value = client + .get(format!("https://registry.npmjs.org/{DEP}/{DEP_VERSION}")) + .send() + .await + .unwrap() + .error_for_status() + .unwrap() + .json() + .await + .unwrap(); + let upstream = client + .get(metadata["dist"]["tarball"].as_str().unwrap()) + .send() + .await + .unwrap() + .error_for_status() + .unwrap() + .bytes() + .await + .unwrap(); + let upstream_checksum = + socket_patch_core::vendor::test_support::service_fixture::berry_checksum( + &upstream, DEP, + ) + .unwrap(); + Mock::given(method("GET")) + .and(path(format!("/upstream/npm/{UUID_H}.json"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "name": DEP, "version": DEP_VERSION, + "integrity": sha512_sri(&upstream), "yarnBerry10c0": upstream_checksum + }))) + .mount(server) + .await; artifacts.push(serde_json::json!({ "kind": "yarn-berry-zip", "url": hosted_url, "integrity": { "yarnBerry10c0": checksum } diff --git a/crates/socket-patch-cli/tests/mode_migration_vlt.rs b/crates/socket-patch-cli/tests/mode_migration_vlt.rs index 64349b266..ce9c6341d 100644 --- a/crates/socket-patch-cli/tests/mode_migration_vlt.rs +++ b/crates/socket-patch-cli/tests/mode_migration_vlt.rs @@ -143,7 +143,7 @@ fn remove_upstream(fx: &Fixture, dir: &Path, purl: &str) -> SocketOut { } fn vendored_scan(fx: &Fixture, dir: &Path, extra: &[&str]) -> SocketOut { - let mut args = vec!["--vendor-source", "build"]; + let mut args = vec!["--vendor-source", "service"]; args.extend_from_slice(extra); api_upstream(fx, dir, &["scan", "--mode", "vendored"], &args) } @@ -486,7 +486,7 @@ async fn vlt_pinned_matrix_migration_rollback_from_mixed() { &fx, &fx.proj, &["get", &a.uuid, "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "{out}"); assert_pure_vendored(&fx.proj, &a); @@ -636,7 +636,7 @@ async fn vlt_pinned_matrix_migration_agent_rollback_after_takeovers() { &fx, &dir, &["get", &b.uuid, "--mode", "vendored"], - &["--vendor-source", "build"], + &["--vendor-source", "service"], ); assert_eq!(out.code, 0, "{out}"); for t in [&a, &b] { @@ -753,14 +753,14 @@ async fn vlt_pinned_matrix_migration_flavor_changed() { ); assert_ok(&npm, "npm install"); stage_manifest(&dir, &[fx.t()]); - let out = vendor_cmd(&dir, &[]); + let out = vendor_upstream(&fx, &dir, &[]); assert_eq!(out.code, 0, "{out}"); let ledger = std::fs::read_to_string(dir.join(".socket/vendor/state.json")).unwrap(); assert!(ledger.contains("\"npm\""), "{ledger}"); write_vlt_json(&dir, fx.leg.version(), &fx.reg.url(), &VltJson::default()); std::fs::remove_dir_all(dir.join("node_modules")).unwrap(); fx.vlt_ok(&dir, &["install"]); - let out = vendor_cmd(&dir, &["--force"]); + let out = vendor_upstream(&fx, &dir, &["--force"]); assert!(has_code(&out.json(), "vendor_flavor_changed"), "{out}"); fx.leg.ran(); } diff --git a/crates/socket-patch-cli/tests/prebuilt_common/mod.rs b/crates/socket-patch-cli/tests/prebuilt_common/mod.rs new file mode 100644 index 000000000..6ef93fa0d --- /dev/null +++ b/crates/socket-patch-cli/tests/prebuilt_common/mod.rs @@ -0,0 +1,637 @@ +//! Prebuilt artifacts for CLI lifecycle tests. The bytes are fixed on first +//! publication of a fixture UUID; later repairs cannot rebuild from local edits. +#![allow(dead_code)] +use std::collections::HashMap; +use std::path::{Path, PathBuf}; +use std::sync::{mpsc, Arc, Mutex, OnceLock}; + +use base64::Engine as _; +use sha2::{Digest, Sha512}; +use socket_patch_core::manifest::schema::{PatchManifest, PatchRecord}; +use socket_patch_core::patch::apply::PatchSources; +use socket_patch_core::vendor::test_support::service_fixture::{ + archive, berry_checksum, Secondary, +}; +use wiremock::matchers::{method, path, path_regex}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +type Package = (String, Vec, Secondary); +static PUBLISHED: OnceLock>> = OnceLock::new(); +static MAVEN_METADATA: OnceLock>>>> = + OnceLock::new(); + +pub struct Server { + pub uri: String, + stop: Option>, + thread: Option>, +} +impl Drop for Server { + fn drop(&mut self) { + self.stop.take(); + if let Some(thread) = self.thread.take() { + thread.join().unwrap(); + } + } +} +impl Server { + pub fn view(view: serde_json::Value) -> Self { + let (ready_tx, ready_rx) = mpsc::channel(); + let (stop, stopped) = mpsc::channel(); + let thread = std::thread::spawn(move || { + let runtime = tokio::runtime::Builder::new_multi_thread() + .worker_threads(1) + .enable_all() + .build() + .unwrap(); + runtime.block_on(async move { + let server = MockServer::start().await; + mount_view(&server, &view, None).await; + ready_tx.send(server.uri()).unwrap(); + let _ = stopped.recv(); + drop(server); + }); + }); + Self { + uri: ready_rx.recv().unwrap(), + stop: Some(stop), + thread: Some(thread), + } + } + + pub fn project(root: &Path) -> Self { + Self::project_with_env(root, &[]) + } + + pub fn project_with_env(root: &Path, env: &[(&str, &str)]) -> Self { + Self::project_with_bind(root, env, false) + } + + pub fn docker_project(root: &Path, env: &[(&str, &str)]) -> Self { + Self::project_with_bind(root, env, true) + } + + pub fn docker_uri(&self) -> String { + format!( + "http://host.docker.internal:{}", + self.uri.rsplit(':').next().unwrap() + ) + } + + fn project_with_bind(root: &Path, env: &[(&str, &str)], docker: bool) -> Self { + let root = root.to_path_buf(); + let extra: Vec<_> = env + .iter() + .filter(|(k, _)| { + matches!( + *k, + "CARGO_HOME" + | "GOMODCACHE" + | "MAVEN_REPO_LOCAL" + | "NUGET_PACKAGES" + | "GEM_HOME" + | "VIRTUAL_ENV" + | "BUNDLE_PATH" + ) + }) + .map(|(_, v)| PathBuf::from(v)) + .collect(); + let (ready_tx, ready_rx) = mpsc::channel(); + let (stop, stopped) = mpsc::channel(); + let thread = std::thread::spawn(move || { + let runtime = tokio::runtime::Builder::new_multi_thread() + .worker_threads(1) + .enable_all() + .build() + .unwrap(); + runtime.block_on(async move { + let address = if docker { "0.0.0.0:0" } else { "127.0.0.1:0" }; + let listener = std::net::TcpListener::bind(address).unwrap(); + let server = MockServer::builder().listener(listener).start().await; + mount_project_with_roots(&server, &root, extra).await; + ready_tx.send(server.uri()).unwrap(); + // The HTTP worker runs on the runtime's worker thread. + let _ = stopped.recv(); + drop(server); + }); + }); + Self { + uri: ready_rx.recv().expect("fixture server started"), + stop: Some(stop), + thread: Some(thread), + } + } + + pub fn configure(&self, common: &mut socket_patch_cli::args::GlobalArgs) { + common.vendor_url = Some(self.uri.clone()); + } + + pub fn command(&self, command: &mut std::process::Command) { + command + .env("SOCKET_VENDOR_URL", &self.uri) + .env("SOCKET_MAVEN_REGISTRY", &self.uri); + } +} + +pub async fn mount_project(server: &MockServer, root: &Path) { + mount_project_with_roots(server, root, Vec::new()).await; +} + +async fn mount_project_with_roots(server: &MockServer, root: &Path, extra: Vec) { + let mut records = std::fs::read(root.join(".socket/manifest.json")) + .ok() + .and_then(|b| serde_json::from_slice::(&b).ok()) + .map(|m| m.patches) + .unwrap_or_default(); + if let Ok(state) = socket_patch_core::vendor::load_state(&root).await { + for entry in state.entries.values() { + if let Some(record) = &entry.record { + records + .entry(entry.base_purl.clone()) + .or_insert_with(|| record.clone()); + } + } + } + let mut results = serde_json::Map::new(); + let blobs = root.join(".socket/blobs"); + let sources = PatchSources { + blobs_path: &blobs, + diffs_path: None, + mem_blobs: None, + }; + let mut roots = vec![root.to_path_buf()]; + let store = root.with_file_name("store"); + if store.is_dir() { + roots.push(store); + } + let repositories = extra.clone(); + roots.extend(extra); + let paths = list_dirs(&roots); + let mut metadata = MAVEN_METADATA + .get_or_init(Default::default) + .lock() + .unwrap() + .get(root) + .cloned() + .unwrap_or_default(); + for repository in repositories { + for directory in list_dirs(&[repository.clone()]) { + for entry in std::fs::read_dir(directory).into_iter().flatten().flatten() { + let path = entry.path(); + if path.is_file() + && matches!( + path.extension().and_then(|e| e.to_str()), + Some("pom" | "module" | "jar") + ) + { + if let Ok(relative) = path.strip_prefix(&repository) { + metadata + .entry(format!( + "/{}", + relative.to_string_lossy().replace('\\', "/") + )) + .or_insert_with(|| std::fs::read(path).unwrap()); + } + } + } + } + } + MAVEN_METADATA + .get() + .unwrap() + .lock() + .unwrap() + .insert(root.to_path_buf(), metadata.clone()); + for (route, bytes) in metadata { + for (algorithm, digest) in [ + ("sha512", hex::encode(Sha512::digest(&bytes))), + ("sha1", hex::encode(sha1::Sha1::digest(&bytes))), + ] { + Mock::given(method("GET")) + .and(path(format!("{route}.{algorithm}"))) + .respond_with(ResponseTemplate::new(200).set_body_string(digest)) + .with_priority(1) + .mount(server) + .await; + } + Mock::given(method("GET")) + .and(path(route)) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .with_priority(1) + .mount(server) + .await; + } + for (purl, record) in records { + if purl.starts_with("pkg:jsr/") { + continue; + } + let key = (root.to_path_buf(), record.uuid.clone()); + let cached = PUBLISHED + .get_or_init(Default::default) + .lock() + .unwrap() + .get(&key) + .cloned(); + let package = match cached { + Some(package) => package, + None => { + let source = source_dir(&root, &paths, &purl); + let fallback = (!source.is_dir()).then(|| package_layout(&purl)); + let source = fallback.as_ref().map(|tmp| tmp.path()).unwrap_or(&source); + let Ok(package) = archive(&purl, &source, &record, &sources).await else { + continue; + }; + PUBLISHED + .get() + .unwrap() + .lock() + .unwrap() + .insert(key, package.clone()); + package + } + }; + mount_package(&server, &purl, &record, package, &mut results).await; + if purl.starts_with("pkg:maven/") { + let source = source_dir(&root, &paths, &purl); + let gav = purl + .trim_start_matches("pkg:maven/") + .split('?') + .next() + .unwrap(); + if let Some((ga, v)) = gav.rsplit_once('@') { + if let Some((g, a)) = ga.split_once('/') { + for ext in ["pom", "module", "jar"] { + if let Ok(bytes) = std::fs::read(source.join(format!("{a}-{v}.{ext}"))) { + let route = format!("/{}/{a}/{v}/{a}-{v}.{ext}", g.replace('.', "/")); + Mock::given(method("GET")) + .and(path(format!("{route}.sha512"))) + .respond_with( + ResponseTemplate::new(200) + .set_body_string(hex::encode(Sha512::digest(&bytes))), + ) + .mount(server) + .await; + Mock::given(method("GET")) + .and(path(route)) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(server) + .await; + } + } + } + } + } + } + Mock::given(method("POST")) + .and(path_regex(r"/(patch|patches)/package$")) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({"results":results})), + ) + .with_priority(1) + .mount(server) + .await; +} + +pub async fn mount_record( + server: &MockServer, + purl: &str, + record: &PatchRecord, + dir: &Path, + sources: &PatchSources<'_>, +) { + let package = archive(purl, dir, record, sources).await.unwrap(); + let mut results = serde_json::Map::new(); + mount_package(server, purl, record, package, &mut results).await; + Mock::given(method("POST")) + .and(path_regex(r"/(patch|patches)/package$")) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({"results":results})), + ) + .with_priority(1) + .mount(server) + .await; +} + +async fn mount_package( + server: &MockServer, + purl: &str, + record: &PatchRecord, + (leaf, bytes, secondary): Package, + results: &mut serde_json::Map, +) { + let route = format!("/artifacts/{}/{leaf}", record.uuid); + let url = format!("{}{route}", server.uri()); + let mut artifacts = + vec![serde_json::json!({"kind":"tarball","url":url,"integrity":{"sha512":sri(&bytes)}})]; + if let Some(npm) = purl.strip_prefix("pkg:npm/") { + if let Some((name, _)) = npm.rsplit_once('@') { + if let Some(sum) = berry_checksum(&bytes, name) { + artifacts.push( + serde_json::json!({"kind":"yarn-berry-zip","integrity":{"yarnBerry10c0":sum}}), + ); + } + } + } + for (kind, leaf, data) in secondary { + let route = format!("/artifacts/{}/{leaf}", record.uuid); + artifacts.push(serde_json::json!({"kind":kind,"url":format!("{}{route}",server.uri()),"integrity":{"sha512":sri(&data)}})); + Mock::given(method("GET")) + .and(path(route)) + .respond_with(ResponseTemplate::new(200).set_body_bytes(data)) + .mount(server) + .await; + } + results.insert( + record.uuid.clone(), + serde_json::json!({"status":"granted","purl":purl,"url":url,"artifacts":artifacts}), + ); + Mock::given(method("GET")) + .and(path(route)) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(server) + .await; +} + +fn sri(bytes: &[u8]) -> String { + format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(Sha512::digest(bytes)) + ) +} + +fn list_dirs(roots: &[PathBuf]) -> Vec { + let mut found = Vec::new(); + let mut pending = roots.to_vec(); + while let Some(dir) = pending.pop() { + if found.len() > 10000 { + break; + } + if let Ok(entries) = std::fs::read_dir(&dir) { + for entry in entries.flatten() { + if entry.file_type().is_ok_and(|t| t.is_dir()) + && !matches!( + entry.file_name().to_str(), + Some(".socket" | ".git" | "target") + ) + { + pending.push(entry.path()); + } + } + } + found.push(dir); + } + found.sort(); + found +} + +fn package_layout(purl: &str) -> tempfile::TempDir { + let tmp = tempfile::tempdir().unwrap(); + let Some((_, coords)) = purl.split('?').next().unwrap().split_once('/') else { + return tmp; + }; + let Some((name, version)) = coords.rsplit_once('@') else { + return tmp; + }; + let name = name.replace("%40", "@"); + let version = version.replace("%2B", "+").replace("%2b", "+"); + if purl.starts_with("pkg:npm/") { + std::fs::write( + tmp.path().join("package.json"), + serde_json::to_vec(&serde_json::json!({"name": name, "version": version})).unwrap(), + ) + .unwrap(); + } else if purl.starts_with("pkg:cargo/") { + std::fs::write( + tmp.path().join("Cargo.toml"), + format!("[package]\nname = {name:?}\nversion = {version:?}\n"), + ) + .unwrap(); + } else if purl.starts_with("pkg:golang/") { + std::fs::write(tmp.path().join("go.mod"), format!("module {name}\n")).unwrap(); + } else if purl.starts_with("pkg:pypi/") { + let info = tmp + .path() + .join(format!("{}-{version}.dist-info", name.replace('-', "_"))); + std::fs::create_dir_all(&info).unwrap(); + std::fs::write( + info.join("WHEEL"), + "Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py3-none-any\n", + ) + .unwrap(); + std::fs::write( + info.join("METADATA"), + format!("Metadata-Version: 2.1\nName: {name}\nVersion: {version}\n\n"), + ) + .unwrap(); + } + tmp +} + +fn source_dir(root: &Path, paths: &[PathBuf], purl: &str) -> PathBuf { + let raw = purl.split('?').next().unwrap(); + let Some((kind, nv)) = raw.trim_start_matches("pkg:").split_once('/') else { + return root.join("missing"); + }; + let Some((name, version)) = nv.rsplit_once('@') else { + return root.join("missing"); + }; + let decoded = name + .replace("%40", "@") + .replace("%2F", "/") + .replace("%2f", "/"); + let name = decoded.as_str(); + let version = version.replace("%2B", "+").replace("%2b", "+"); + if kind == "npm" { + let direct = root.join("node_modules").join(name); + let version_matches = std::fs::read(direct.join("package.json")) + .ok() + .and_then(|bytes| serde_json::from_slice::(&bytes).ok()) + .is_none_or(|package| package["version"].as_str().is_none_or(|v| v == version)); + if direct.is_dir() && version_matches { + return direct; + } + } + for dir in paths { + let leaf = dir.file_name().and_then(|v| v.to_str()).unwrap_or(""); + let matches = match kind { + "npm" | "composer" => std::fs::read(dir.join(if kind == "npm" { + "package.json" + } else { + "composer.json" + })) + .ok() + .and_then(|b| serde_json::from_slice::(&b).ok()) + .is_some_and(|p| { + p["name"].as_str() == Some(name) + && (kind != "npm" || p["version"].as_str().is_none_or(|v| v == version)) + }), + "pypi" => { + leaf == "site-packages" + && dir + .join(format!("{}-{}.dist-info", name.replace('-', "_"), version)) + .is_dir() + } + "cargo" | "gem" => leaf == format!("{name}-{version}"), + "golang" => dir + .to_string_lossy() + .ends_with(&format!("{name}@{version}")), + "maven" => dir + .join(format!( + "{}-{version}.jar", + name.rsplit('/').next().unwrap() + )) + .is_file(), + "nuget" => dir + .join(format!("{}.{}.nupkg", name.to_lowercase(), version)) + .is_file(), + _ => false, + }; + if matches { + return dir.clone(); + } + } + root.join("missing") +} + +pub async fn mount_view(server: &MockServer, view: &serde_json::Value, gemspec: Option<&[u8]>) { + mount_view_from_source(server, view, gemspec, None).await; +} + +pub async fn mount_view_from_source( + server: &MockServer, + view: &serde_json::Value, + gemspec: Option<&[u8]>, + installed: Option<&Path>, +) { + let purl = view["purl"].as_str().unwrap(); + let mut manifest_record = view.clone(); + manifest_record["exportedAt"] = view["publishedAt"].clone(); + let record: PatchRecord = serde_json::from_value(manifest_record).unwrap(); + let mut blobs = HashMap::new(); + for info in view["files"].as_object().unwrap().values() { + if let Some(encoded) = info["blobContent"].as_str() { + blobs.insert( + info["afterHash"].as_str().unwrap().to_string(), + base64::engine::general_purpose::STANDARD + .decode(encoded) + .unwrap(), + ); + } + } + let tmp = tempfile::tempdir().unwrap(); + let source = tmp.path().join("gems/package"); + std::fs::create_dir_all(&source).unwrap(); + let package = purl.split('?').next().unwrap().split_once('/').unwrap().1; + let (name, version) = package.rsplit_once('@').unwrap(); + let name = name.replace("%40", "@"); + let layout = package_layout(purl); + copy_tree(installed.unwrap_or(layout.path()), &source); + if let Some(gemspec) = gemspec { + std::fs::create_dir_all(tmp.path().join("specifications")).unwrap(); + std::fs::write( + tmp.path() + .join("specifications") + .join(format!("{name}-{version}.gemspec")), + gemspec, + ) + .unwrap(); + } + let sources = PatchSources { + blobs_path: tmp.path(), + diffs_path: None, + mem_blobs: Some(&blobs), + }; + let package = archive(purl, &source, &record, &sources).await.unwrap(); + let mut results = serde_json::Map::new(); + mount_package(server, purl, &record, package, &mut results).await; + type Grants = Arc>>; + static GRANTS: OnceLock>> = OnceLock::new(); + let grants = GRANTS + .get_or_init(Default::default) + .lock() + .unwrap() + .entry(server.uri()) + .or_default() + .clone(); + grants.lock().unwrap().extend(results); + Mock::given(method("POST")) + .and(path_regex(r"/(patch|patches)/package$")) + .respond_with(move |request: &wiremock::Request| { + let body: serde_json::Value = serde_json::from_slice(&request.body).unwrap(); + let all = grants.lock().unwrap(); + let results: serde_json::Map = body["uuids"] + .as_array() + .unwrap() + .iter() + .filter_map(|v| { + let uuid = v.as_str()?; + all.get(uuid).map(|grant| (uuid.to_string(), grant.clone())) + }) + .collect(); + ResponseTemplate::new(200).set_body_json(serde_json::json!({"results":results})) + }) + .with_priority(1) + .mount(server) + .await; +} + +/// Download fixtures for lifecycle setup. Package-manager offline checks are +/// separate commands and retain their original flags. +pub fn prepare_command( + command: &mut std::process::Command, + root: &Path, + args: &[&str], + env: &[(&str, &str)], +) -> Option { + let download = + matches!(args.first(), Some(&"vendor") | Some(&"repair")) && !args.contains(&"--revert"); + if download { + if let Some(i) = args.iter().position(|a| *a == "--patch-server-url") { + command.args(args).args(["--vendor-url", args[i + 1]]); + return None; + } + let fixture = Server::project_with_env(root, env); + command.args(args.iter().copied().filter(|a| *a != "--offline")); + fixture.command(command); + Some(fixture) + } else { + command.args(args); + None + } +} + +fn copy_tree(from: &Path, to: &Path) { + std::fs::create_dir_all(to).unwrap(); + for entry in std::fs::read_dir(from).unwrap() { + let entry = entry.unwrap(); + let dest = to.join(entry.file_name()); + if entry.file_type().unwrap().is_dir() { + copy_tree(&entry.path(), &dest); + } else { + std::fs::copy(entry.path(), dest).unwrap(); + } + } +} + +pub async fn mount_download(server: &MockServer, purl: &str, uuid: &str, leaf: &str, bytes: &[u8]) { + let record: PatchRecord = serde_json::from_value(serde_json::json!({ + "uuid": uuid, "exportedAt": "2026-01-01T00:00:00Z", "files": {}, + "vulnerabilities": {}, "description": "fixture", "license": "MIT", "tier": "free" + })) + .unwrap(); + let mut grants = serde_json::Map::new(); + mount_package( + server, + purl, + &record, + (leaf.into(), bytes.to_vec(), Vec::new()), + &mut grants, + ) + .await; + Mock::given(method("POST")) + .and(path("/patch/package")) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({ "results": grants })), + ) + .mount(server) + .await; +} diff --git a/crates/socket-patch-cli/tests/repair/coverage_fix_repair_vendor_predelete.rs b/crates/socket-patch-cli/tests/repair/coverage_fix_repair_vendor_predelete.rs index 762fafe6d..72362c0a2 100644 --- a/crates/socket-patch-cli/tests/repair/coverage_fix_repair_vendor_predelete.rs +++ b/crates/socket-patch-cli/tests/repair/coverage_fix_repair_vendor_predelete.rs @@ -113,31 +113,33 @@ async fn mount_gem_patch_api(mock: &MockServer) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/padlock.rb": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-dddd-eeee-ffff": { + "cves": ["CVE-2026-0002"], + "summary": "gem test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Gem vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, Some(GEMSPEC_STUB)).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{GEM_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": GEM_UUID, - "purl": GEM_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "lib/padlock.rb": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-dddd-eeee-ffff": { - "cves": ["CVE-2026-0002"], - "summary": "gem test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Gem vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -226,6 +228,7 @@ async fn repair_keeps_corrupt_forensic_bytes_when_rebuild_dispatch_refuses() { ))) .unwrap(); + mock.reset().await; let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); let v = parse_env(&stdout); diff --git a/crates/socket-patch-cli/tests/repair/covgap_commands_repair.rs b/crates/socket-patch-cli/tests/repair/covgap_commands_repair.rs index b960ce6f3..c00520fd4 100644 --- a/crates/socket-patch-cli/tests/repair/covgap_commands_repair.rs +++ b/crates/socket-patch-cli/tests/repair/covgap_commands_repair.rs @@ -13,7 +13,7 @@ //! skip event, exit stays 0, loop continues to the packages pass), //! * an unremovable `apply.lock` (read-only `.socket`) stays non-fatal //! and silent (exit stays 0), -//! * the loud "Rebuilt N vendored artifacts." summary after the +//! * the loud "Redownloaded N vendored artifacts." summary after the //! vendored-repair phase. //! //! Everything runs offline or against a wiremock server — no real hosts. @@ -157,7 +157,7 @@ fn item_lines(s: &str) -> Vec<&str> { fn repair_manifest_not_found_human_mode_prints_to_stderr() { let tmp = tempfile::tempdir().expect("tempdir"); let out = socket_cmd(tmp.path()) - .args(["repair", "--offline"]) + .args(["repair"]) .output() .expect("run socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout); @@ -192,7 +192,7 @@ fn repair_failed_human_mode_prints_error_to_stderr() { std::fs::write(socket.join("manifest.json"), "{ not valid json").unwrap(); let out = socket_cmd(tmp.path()) - .args(["repair", "--offline"]) + .args(["repair"]) .output() .expect("run socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout); @@ -372,7 +372,7 @@ fn repair_removes_orphan_archives_human_mode_prints_relabeled_summary() { write_archive(&socket, "packages", ORPHAN_PKG, b"orphan pkg bytes"); let out = socket_cmd(tmp.path()) - .args(["repair", "--offline"]) + .args(["repair"]) .output() .expect("run socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout); @@ -586,7 +586,7 @@ fn repair_exits_zero_and_stays_quiet_when_lock_file_unremovable() { } // --------------------------------------------------------------------------- -// Loud vendored-repair summary — "Rebuilt N vendored artifacts." +// Loud vendored-repair summary — "Redownloaded N vendored artifacts." // --------------------------------------------------------------------------- const UUID: &str = "11111111-1111-4111-8111-111111111111"; @@ -677,31 +677,33 @@ async fn mount_patch_api(mock: &MockServer) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], + "summary": "test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], - "summary": "test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -732,13 +734,13 @@ fn run_cli(root: &Path, mock_uri: &str, argv: &[&str], json: bool) -> (i32, Stri ) } -/// The loud "Rebuilt N vendored artifacts." summary after the +/// The loud "Redownloaded N vendored artifacts." summary after the /// vendored-repair phase — uncovered only because the sibling e2e suite /// drives every repair through `--json`. Reuses the hermetic /// offline-rebuild fixture (installed copy + seeded after-blob), so the /// repair itself makes zero network requests. #[tokio::test] -async fn repair_offline_rebuild_human_mode_prints_rebuilt_summary() { +async fn repair_redownload_human_mode_prints_summary() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -767,16 +769,17 @@ async fn repair_offline_rebuild_human_mode_prints_rebuilt_summary() { std::fs::write(blobs.join(git_sha256(AFTER)), AFTER).unwrap(); let before_reqs = mock.received_requests().await.unwrap().len(); - let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair", "--offline"], false); + let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"], false); assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); assert!( - stdout.contains("Rebuilt 1 vendored artifact."), + stdout.contains("Redownloaded 1 vendored artifact."), "the loud run must print the vendored-rebuild summary; stdout=\n{stdout}" ); assert!(tgz.is_file(), "the tarball was rebuilt offline"); let after_reqs = mock.received_requests().await.unwrap().len(); assert_eq!( - before_reqs, after_reqs, - "--offline repair must make no network requests" + after_reqs - before_reqs, + 2, + "repair grants and downloads the recorded server artifact" ); } diff --git a/crates/socket-patch-cli/tests/repair/covgap_commands_repair_vendor.rs b/crates/socket-patch-cli/tests/repair/covgap_commands_repair_vendor.rs index bf4baf458..c5ebab59f 100644 --- a/crates/socket-patch-cli/tests/repair/covgap_commands_repair_vendor.rs +++ b/crates/socket-patch-cli/tests/repair/covgap_commands_repair_vendor.rs @@ -235,31 +235,33 @@ async fn mount_npm_routes(mock: &MockServer) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], + "summary": "test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(&mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], - "summary": "test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -288,31 +290,33 @@ async fn mount_gem_routes(mock: &MockServer, after: &[u8]) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/padlock.rb": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": b64(after), + } + }, + "vulnerabilities": { + "GHSA-dddd-eeee-ffff": { + "cves": ["CVE-2026-0002"], + "summary": "gem test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Gem vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(&mock, &archive_view, Some(GEMSPEC_STUB)).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{GEM_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": GEM_UUID, - "purl": GEM_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "lib/padlock.rb": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": b64(after), - } - }, - "vulnerabilities": { - "GHSA-dddd-eeee-ffff": { - "cves": ["CVE-2026-0002"], - "summary": "gem test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Gem vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -378,7 +382,8 @@ fn run_cli_with( cmd.env_remove(name); } } - cmd.env("SOCKET_TELEMETRY_DISABLED", "1"); + cmd.env("SOCKET_TELEMETRY_DISABLED", "1") + .env("SOCKET_API_CONCURRENCY", "1"); for (k, v) in envs { cmd.env(k, v); } @@ -708,11 +713,8 @@ async fn repair_uses_the_embedded_record_without_manifest() { assert!( events_of(&v).iter().any(|e| e["action"] == "failed" && e["purl"] == PURL - && e["errorCode"] == "vendor_artifact_missing" - && e["error"] - .as_str() - .unwrap_or("") - .contains("no local source")), + && e["errorCode"] == "vendor_artifact_redownload_failed" + && e["error"].as_str().unwrap_or("").contains("--offline")), "envelope={v}" ); assert!( @@ -919,7 +921,7 @@ async fn repair_rebuilds_via_ledger_recovered_registry_fetch() { /// non-soft candidate fails with `vendor_fetch_failed` and nothing is /// invented on disk. #[tokio::test] -async fn repair_fails_when_ledger_recovered_fetch_fails() { +async fn repair_redownload_ignores_old_registry_failure() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let integrity = sri_of(&pristine_tgz()); @@ -939,16 +941,21 @@ async fn repair_fails_when_ledger_recovered_fetch_fails() { std::fs::remove_file(&tgz).unwrap(); std::fs::remove_dir_all(tmp.path().join("node_modules")).unwrap(); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); assert!( - events_of(&v).iter().any(|e| e["action"] == "failed" - && e["purl"] == PURL - && e["errorCode"] == "vendor_fetch_failed"), - "envelope={v}" + events_of(&v) + .iter() + .any(|e| e["purl"] == PURL && e["details"]["redownloaded"] == true), + "{v}" + ); + assert!(tgz.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); - assert!(!tgz.exists(), "a failed fetch must not invent an artifact"); } // ─────────────────── rebuild-loop failure arms ─────────────────── @@ -957,7 +964,7 @@ async fn repair_fails_when_ledger_recovered_fetch_fails() { /// tarball): the backend's refusal code (`vendor_lockfile_missing`) /// surfaces verbatim as the per-entry failure. #[tokio::test] -async fn repair_rebuild_refused_when_lockfile_missing() { +async fn repair_redownload_leaves_missing_lockfile_missing() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -971,24 +978,30 @@ async fn repair_rebuild_refused_when_lockfile_missing() { std::fs::remove_file(&tgz).unwrap(); std::fs::remove_file(tmp.path().join("package-lock.json")).unwrap(); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); assert!( - events_of(&v).iter().any(|e| e["action"] == "failed" - && e["purl"] == PURL - && e["errorCode"] == "vendor_lockfile_missing"), - "the backend refusal code surfaces verbatim: {v}" + events_of(&v) + .iter() + .any(|e| e["purl"] == PURL && e["details"]["redownloaded"] == true), + "{v}" + ); + assert!(tgz.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); - assert!(!tgz.exists(), "a refused dispatch replaced nothing"); + assert!(!tmp.path().join("package-lock.json").exists()); } /// The dispatch RUNS but fails (`result.success == false`): the installed /// copy's patched file is gone, so the backend's fail-closed /// missing-patch-file pre-check fails the rebuild — -/// `vendor_artifact_rebuild_failed` with the underlying cause. +/// `vendor_artifact_redownload_failed` with the underlying cause. #[tokio::test] -async fn repair_rebuild_fails_when_installed_patch_file_missing() { +async fn repair_redownload_does_not_require_installed_file() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -1003,17 +1016,21 @@ async fn repair_rebuild_fails_when_installed_patch_file_missing() { // Keep package.json so the crawler still resolves the install. std::fs::remove_file(tmp.path().join("node_modules/left-pad/index.js")).unwrap(); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); assert!( - events_of(&v).iter().any(|e| e["action"] == "failed" - && e["purl"] == PURL - && e["errorCode"] == "vendor_artifact_rebuild_failed" - && e["error"].as_str().unwrap_or("").contains("File not found")), - "envelope={v}" + events_of(&v) + .iter() + .any(|e| e["purl"] == PURL && e["details"]["redownloaded"] == true), + "{v}" + ); + assert!(tgz.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); - assert!(!tgz.exists(), "a failed dispatch replaced nothing"); } /// Human twin: the failed rebuild exits 1, so the run must close on @@ -1032,6 +1049,7 @@ async fn human_repair_rebuild_failure_does_not_claim_complete() { std::fs::remove_file(&tgz).unwrap(); std::fs::remove_file(tmp.path().join("node_modules/left-pad/index.js")).unwrap(); + mock.reset().await; let (code, stdout, stderr) = run_cli_human(tmp.path(), &mock.uri(), &["repair"]); assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); assert!( @@ -1179,7 +1197,7 @@ async fn repair_restores_crashed_set_aside_leftover_before_classifying() { /// gone, no CHECKSUMS sha recorded to verify a registry fetch against): /// `vendor_artifact_unrepairable` naming the missing source. #[tokio::test] -async fn repair_gem_unverifiable_reason_is_surfaced() { +async fn repair_redownloads_gem_without_registry_checksum() { let mock = MockServer::start().await; mount_gem_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -1189,23 +1207,21 @@ async fn repair_gem_unverifiable_reason_is_surfaced() { std::fs::remove_dir_all(©).unwrap(); std::fs::remove_dir_all(tmp.path().join("vendor/bundle")).unwrap(); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); - let failed = events_of(&v) - .into_iter() - .find(|e| e["action"] == "failed" && e["purl"] == GEM_PURL) - .unwrap_or_else(|| panic!("expected a failed event: {v}")); - assert_eq!( - failed["errorCode"], "vendor_artifact_unrepairable", - "{failed}" - ); - let detail = failed["error"].as_str().unwrap_or(""); assert!( - detail.contains("no installed package found on disk"), - "the missing source is named: {failed}" + events_of(&v) + .iter() + .any(|e| e["purl"] == GEM_PURL && e["details"]["redownloaded"] == true), + "{v}" + ); + assert!(copy.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); - assert!(!copy.exists(), "no artifact is invented without a source"); } // ─────────────── backend warning forwarding during rebuilds ─────────────── @@ -1216,7 +1232,7 @@ async fn repair_gem_unverifiable_reason_is_surfaced() { /// hash — so the rebuild fails loudly (no artifact invented), exactly like /// a `vendor` re-run over the same tree. #[tokio::test] -async fn repair_refuses_a_drifted_variant_install() { +async fn repair_redownload_ignores_drifted_installed_variant() { let mock = MockServer::start().await; mount_gem_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -1233,16 +1249,21 @@ async fn repair_refuses_a_drifted_variant_install() { ) .unwrap(); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); assert!( - events_of(&v).iter().any(|e| e["action"] == "failed" - && e["purl"] == GEM_PURL - && e["errorCode"] == "vendor_artifact_unrepairable"), - "envelope={v}" + events_of(&v) + .iter() + .any(|e| e["purl"] == GEM_PURL && e["details"]["redownloaded"] == true), + "{v}" + ); + assert!(copy.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); - assert!(!copy.exists(), "no artifact is invented from a drifted copy"); } // ─────────────────────── --ecosystems scoping ─────────────────────── @@ -1291,8 +1312,8 @@ async fn repair_ecosystems_scope_skips_out_of_scope_entries() { // ───────────────────────── human output ───────────────────────── -/// The human (non-`--json`) output lines: the "Rebuilding N…" header and -/// per-purl "Rebuilt …" line on stdout for a successful rebuild, and the +/// The human (non-`--json`) output lines: the "Redownloading N…" header and +/// per-purl "Redownloaded …" line on stdout for a successful rebuild, and the /// "Cannot repair vendored artifact for …" stderr line for a failure. #[tokio::test] async fn repair_human_output_lines() { @@ -1310,11 +1331,11 @@ async fn repair_human_output_lines() { let (code, stdout, stderr) = run_cli_human(tmp.path(), &mock.uri(), &["repair"]); assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); assert!( - stdout.contains("Rebuilding 1 broken vendored artifact..."), + stdout.contains("Redownloading 1 broken vendored artifact..."), "the rebuild header is printed: {stdout}" ); assert!( - stdout.contains(&format!("Rebuilt {PURL}")), + stdout.contains(&format!("Redownloaded {PURL}")), "the per-purl rebuilt line is printed: {stdout}" ); assert!(tgz.is_file(), "the human-mode repair still rebuilds"); @@ -1344,7 +1365,7 @@ async fn repair_human_output_lines() { /// npm lock also exercises the wired-integrity probe's None fall-through /// (the unverified-registry rung must NOT fire without a trust anchor). #[tokio::test] -async fn repair_platform_locked_detail_when_no_pristine_source() { +async fn repair_redownload_preserves_platform_locked_artifact() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -1363,27 +1384,20 @@ async fn repair_platform_locked_detail_when_no_pristine_source() { state["entries"][PURL]["artifact"]["platformLocked"] = serde_json::json!(true); write_state(tmp.path(), &state); + let ledger_before = std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 0, "{stdout} {stderr}"); let v = parse_env(&stdout); - let failed = events_of(&v) - .into_iter() - .find(|e| { - e["action"] == "failed" - && e["purl"] == PURL - && e["errorCode"] == "vendor_artifact_unrepairable" - }) - .unwrap_or_else(|| panic!("expected an unrepairable failure: {v}")); assert!( - failed["error"] - .as_str() - .unwrap_or("") - .contains("platform-locked (compiled)"), - "the platform-locked advice must win the detail selection: {failed}" + events_of(&v) + .iter() + .any(|e| e["purl"] == PURL && e["details"]["redownloaded"] == true), + "{v}" ); - assert!( - !tgz.exists(), - "no artifact is invented without a pristine source" + assert!(tgz.exists()); + assert_eq!( + std::fs::read(tmp.path().join(".socket/vendor/state.json")).unwrap(), + ledger_before ); } @@ -1424,7 +1438,7 @@ async fn repair_tampered_base_purl_surfaces_precise_unverifiable_detail() { failed["error"] .as_str() .unwrap_or("") - .contains("no installed package found on disk"), + .contains("ledger package identity"), "the missing source is named: {failed}" ); assert!( @@ -1500,24 +1514,25 @@ async fn repair_no_backend_for_purl_restores_set_aside_bytes() { mount_blob(&mock).await; // The in-memory stager recovers the jsr record's content through the // patch view endpoint (the corrupt artifact yields no harvest). + let archive_view = serde_json::json!({ + "uuid": JSR_UUID, + "purl": JSR_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": git_sha256(BEFORE), + "afterHash": git_sha256(AFTER), + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": {}, + "description": "jsr vendor patch", + "license": "MIT", + "tier": "free", + }); Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{JSR_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": JSR_UUID, - "purl": JSR_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(BEFORE), - "afterHash": git_sha256(AFTER), - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": {}, - "description": "jsr vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(&mock) .await; let tmp = tempfile::tempdir().unwrap(); @@ -1570,7 +1585,7 @@ async fn repair_no_backend_for_purl_restores_set_aside_bytes() { && e["error"] .as_str() .unwrap_or("") - .contains("vendoring is not supported for this ecosystem")), + .contains("ledger package identity is invalid or unsupported")), "envelope={v}" ); assert_eq!( @@ -1605,7 +1620,7 @@ async fn repair_no_backend_for_purl_restores_set_aside_bytes() { && e["error"] .as_str() .unwrap_or("") - .contains("vendoring is not supported for this ecosystem")), + .contains("ledger package identity is invalid or unsupported")), "envelope={v}" ); assert!( @@ -1712,7 +1727,7 @@ fn writable_vendor_dir(root: &Path) { /// `vendor_inventory_refreshed` advisory and never claims `rebuilt`. #[cfg(unix)] #[tokio::test] -async fn repair_inventory_refresh_persist_failure_stays_loud() { +async fn repair_refuses_changed_inventory_with_readonly_ledger() { if is_root() { eprintln!("skipped: read-only-dir contraption is inert as root"); return; @@ -1738,27 +1753,12 @@ async fn repair_inventory_refresh_persist_failure_stays_loud() { assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); let v = parse_env(&stdout); assert!( - events_of(&v).iter().any(|e| e["action"] == "skipped" - && e["purl"] == GEM_PURL - && e["errorCode"] == "vendor_inventory_refreshed"), - "the refresh advisory still rides the envelope: {v}" - ); - assert!( - events_of(&v).iter().any(|e| e["action"] == "failed" - && e["purl"] == GEM_PURL - && e["errorCode"] == "vendor_state_write_failed"), - "envelope={v}" - ); - assert!( - !events_of(&v) + events_of(&v) .iter() - .any(|e| e["action"] == "rebuilt" && e["purl"] == GEM_PURL), - "an unpersisted refresh must not be claimed rebuilt: {v}" - ); - assert_eq!( - std::fs::read(copy.join("lib/padlock.rb")).unwrap(), - AFTER, - "the member-verified rebuild is kept on disk" + .any(|e| e["action"] == "failed" + && e["errorCode"] == "vendor_artifact_redownload_failed"), + "{v}" ); + assert!(!copy.exists()); + assert_eq!(read_state(tmp.path()), state); } - diff --git a/crates/socket-patch-cli/tests/repair/main.rs b/crates/socket-patch-cli/tests/repair/main.rs index 861a4d1dd..41f066184 100644 --- a/crates/socket-patch-cli/tests/repair/main.rs +++ b/crates/socket-patch-cli/tests/repair/main.rs @@ -24,3 +24,6 @@ mod covgap_commands_repair_vendor; mod repair_invariants; mod repair_vendor_e2e; mod repair_vendor_flavors_e2e; + +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; diff --git a/crates/socket-patch-cli/tests/repair/repair_vendor_e2e.rs b/crates/socket-patch-cli/tests/repair/repair_vendor_e2e.rs index a12a741d9..a3effd3e5 100644 --- a/crates/socket-patch-cli/tests/repair/repair_vendor_e2e.rs +++ b/crates/socket-patch-cli/tests/repair/repair_vendor_e2e.rs @@ -156,31 +156,33 @@ async fn mount_patch_api(mock: &MockServer) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], + "summary": "test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], - "summary": "test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -287,7 +289,7 @@ async fn repair_rebuilds_deleted_vendored_tarball() { /// 2. `repair --offline` rebuilds from purely local sources (installed copy /// + seeded blob) with zero network. #[tokio::test] -async fn repair_offline_rebuilds_from_local_sources() { +async fn repair_offline_refuses_even_with_installed_tree_and_blobs() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -306,10 +308,10 @@ async fn repair_offline_rebuilds_from_local_sources() { let before_reqs = mock.received_requests().await.unwrap().len(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair", "--offline"]); - assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); let v = parse_env(&stdout); - assert_eq!(v["summary"]["rebuilt"], 1, "envelope={v}"); - assert!(tgz.is_file(), "tarball rebuilt offline"); + assert_eq!(v["summary"]["failed"], 1, "envelope={v}"); + assert!(!tgz.exists(), "no local rebuild"); let after_reqs = mock.received_requests().await.unwrap().len(); assert_eq!( before_reqs, after_reqs, @@ -410,25 +412,10 @@ async fn repair_keeps_corrupt_artifact_when_no_rebuild_source_exists() { "arm 2: an unrebuildable corrupt artifact must not be destroyed" ); - // Heal: with the installed copy restored, the same repair rebuilds the - // recorded bytes — retention never wedges the corrupt→rebuild path. - let pkg = tmp.path().join("node_modules/left-pad"); - std::fs::create_dir_all(&pkg).unwrap(); - std::fs::write( - pkg.join("package.json"), - br#"{"name":"left-pad","version":"1.3.0"}"#, - ) - .unwrap(); - std::fs::write(pkg.join("index.js"), BEFORE).unwrap(); - let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair", "--offline"]); + let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); - let v = parse_env(&stdout); - assert_eq!(v["summary"]["rebuilt"], 1, "envelope={v}"); - assert_eq!( - std::fs::read(&tgz).unwrap(), - tgz_bytes, - "rebuild restores the recorded bytes once a source exists" - ); + assert_eq!(std::fs::read(&tgz).unwrap(), tgz_bytes); + assert!(!tmp.path().join("node_modules").exists()); } /// 4. A tampered ledger sha can never be satisfied: the rebuild is removed @@ -457,12 +444,13 @@ async fn repair_fails_closed_on_tampered_ledger_sha() { assert!( events_of(&env) .iter() - .any(|e| e["action"] == "failed" && e["errorCode"] == "vendor_artifact_rebuild_failed"), + .any(|e| e["action"] == "failed" + && e["errorCode"] == "vendor_artifact_redownload_failed"), "envelope={env}" ); assert!( - !tgz.exists(), - "an unverifiable rebuild must not be left on disk" + tgz.is_file(), + "a failed redownload must preserve the original artifact" ); } @@ -595,7 +583,10 @@ async fn repair_reports_missing_ledger_instead_of_reconstructing() { } let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 1, "wholesale={wholesale} stdout={stdout} stderr={stderr}"); + assert_eq!( + code, 1, + "wholesale={wholesale} stdout={stdout} stderr={stderr}" + ); let v = parse_env(&stdout); let missing: Vec<_> = events_of(&v) .into_iter() @@ -632,7 +623,11 @@ async fn repair_reports_missing_ledger_instead_of_reconstructing() { assert!(!tgz.exists(), "nothing is rebuilt without a ledger"); continue; } - assert_eq!(std::fs::read(&tgz).unwrap(), tgz_bytes, "artifact untouched"); + assert_eq!( + std::fs::read(&tgz).unwrap(), + tgz_bytes, + "artifact untouched" + ); // Restoring the ledger from version control is the recovery. std::fs::write(&state_path, &state1).unwrap(); @@ -665,7 +660,7 @@ async fn repair_dry_run_previews_rebuild() { let v = parse_env(&stdout); assert!( events_of(&v).iter().any(|e| e["action"] == "verified" - && e["details"]["wouldRebuild"] == true + && e["details"]["wouldRedownload"] == true && e["purl"] == PURL), "envelope={v}" ); @@ -770,31 +765,33 @@ async fn mount_gem_patch_api(mock: &MockServer) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/padlock.rb": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-dddd-eeee-ffff": { + "cves": ["CVE-2026-0002"], + "summary": "gem test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Gem vendor patch", + "license": "MIT", + "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, Some(GEMSPEC_STUB)).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{GEM_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": GEM_UUID, - "purl": GEM_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "lib/padlock.rb": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-dddd-eeee-ffff": { - "cves": ["CVE-2026-0002"], - "summary": "gem test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Gem vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -1020,10 +1017,10 @@ async fn repair_gem_dir_tamper_matrix_and_vex_refusal() { /// rebuild must refresh the stale inventory — loudly, with the /// provenance named — instead of deleting the rebuild and stranding the /// wired pair on a dead dir. RED without the refresh: exit 1 -/// vendor_artifact_rebuild_failed, artifact gone, and every subsequent +/// vendor_artifact_redownload_failed, artifact gone, and every subsequent /// repair loops the same failure. #[tokio::test] -async fn repair_refreshes_stale_inventory_from_service_provenance() { +async fn repair_refuses_changed_service_inventory() { const SERVICE_STUB: &[u8] = b"# converter-generated stub\nGem::Specification.new do |s|\n s.name = \"padlock\"\n s.version = \"1.2.0\"\n s.summary = \"repair fixture\"\n s.authors = [\"socket-patch e2e\"]\n s.require_paths = [\"lib\"]\nend\n"; let mock = MockServer::start().await; mount_gem_patch_api(&mock).await; @@ -1053,54 +1050,12 @@ async fn repair_refreshes_stale_inventory_from_service_provenance() { std::fs::remove_dir_all(©).unwrap(); - // THE pin: the local rebuild's stub differs from the recorded - // inventory; repair keeps the member-verified rebuild and refreshes - // the inventory rather than deleting it and failing forever. + let ledger_before = std::fs::read(&state_path).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); - let v = parse_env(&stdout); - assert!( - events_of(&v) - .iter() - .any(|e| e["action"] == "rebuilt" && e["purl"] == GEM_PURL), - "envelope={v}" - ); - assert!( - events_of(&v).iter().any(|e| e["action"] == "skipped" - && e["errorCode"] == "vendor_inventory_refreshed" - && e["purl"] == GEM_PURL), - "the provenance switch must be surfaced: {v}" - ); - assert_eq!( - std::fs::read(copy.join("padlock.gemspec")).unwrap(), - GEMSPEC_STUB, - "the local rebuild's stub is kept" - ); - assert_eq!( - std::fs::read(copy.join("lib/padlock.rb")).unwrap(), - AFTER, - "patched member intact" - ); - let state: serde_json::Value = - serde_json::from_str(&std::fs::read_to_string(&state_path).unwrap()).unwrap(); - assert_eq!( - state["entries"][GEM_PURL]["artifact"]["fileInventory"]["padlock.gemspec"], - serde_json::json!(sha256_hex(GEMSPEC_STUB)), - "inventory refreshed from the verified rebuild: {state}" - ); - - // The loop is dead: the next repair is clean. - let (code, stdout, _) = run_cli(tmp.path(), &mock.uri(), &["repair"]); - assert_eq!(code, 0, "{stdout}"); - let v = parse_env(&stdout); - assert!( - v["summary"]["rebuilt"].is_null() || v["summary"]["rebuilt"] == 0, - "no repair loop: {v}" - ); - assert!( - !events_of(&v).iter().any(|e| e["action"] == "failed"), - "no repair loop: {v}" - ); + assert_eq!(code, 1, "{stdout} {stderr}"); + assert!(stdout.contains("vendor_inventory_mismatch")); + assert!(!copy.exists()); + assert_eq!(std::fs::read(&state_path).unwrap(), ledger_before); } /// G3b. Backward tolerance: a pre-inventory ledger entry (fileInventory @@ -1265,7 +1220,7 @@ use crate::vlt_hosted_common; use crate::vlt_vendored; /// `repair --dry-run` over a deleted vlt directory artifact previews the -/// rebuild (`wouldRebuild`, the dir path) and writes nothing; the wet run +/// rebuild (`wouldRedownload`, the dir path) and writes nothing; the wet run /// rebuilds it offline from the installed copy. #[test] fn repair_previews_then_rebuilds_a_deleted_vlt_dir() { @@ -1284,11 +1239,16 @@ fn repair_previews_then_rebuilds_a_deleted_vlt_dir() { assert_eq!(code, 0, "{v:#}\n{stderr}"); let text = v.to_string(); assert!( - text.contains("wouldRebuild") && text.contains(&vlt_vendored::rel()), + text.contains("wouldRedownload") && text.contains(&vlt_vendored::rel()), "{v:#}" ); assert!(!uuid_dir.exists()); - let (code, v, stderr) = hosted::run_json(root, &["repair", "--offline", "--cwd", &cwd], &[]); + let fixture = crate::prebuilt_common::Server::project(root); + let (code, v, stderr) = hosted::run_json( + root, + &["repair", "--cwd", &cwd], + &[("SOCKET_VENDOR_URL", &fixture.uri)], + ); assert_eq!(code, 0, "{v:#}\n{stderr}"); assert_eq!( std::fs::read(root.join(vlt_vendored::rel()).join("index.js")).unwrap(), @@ -1320,7 +1280,9 @@ fn retar(tgz: &[u8], mtime: u32, edit: impl FnOnce(&mut Vec<(String, Vec)>)) header.set_size(bytes.len() as u64); header.set_mode(0o644); header.set_cksum(); - builder.append_data(&mut header, path, bytes.as_slice()).unwrap(); + builder + .append_data(&mut header, path, bytes.as_slice()) + .unwrap(); } builder.into_inner().unwrap().finish().unwrap() } @@ -1330,7 +1292,7 @@ fn retar(tgz: &[u8], mtime: u32, edit: impl FnOnce(&mut Vec<(String, Vec)>)) /// patched bytes BEFORE setting the artifact aside, rebuilds from the /// pristine installed copy, and restores the byte-exact recorded archive. #[tokio::test] -async fn repair_offline_harvests_a_corrupt_artifacts_valid_members() { +async fn repair_offline_never_repacks_verified_members() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); @@ -1351,10 +1313,14 @@ async fn repair_offline_harvests_a_corrupt_artifacts_valid_members() { std::fs::write(&tgz, &corrupt).unwrap(); let (code, stdout, stderr) = run_cli(tmp.path(), &mock.uri(), &["repair", "--offline"]); - assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); + assert_eq!(code, 1, "stdout={stdout} stderr={stderr}"); let v = parse_env(&stdout); - assert_eq!(v["summary"]["rebuilt"], 1, "envelope={v}"); - assert_eq!(std::fs::read(&tgz).unwrap(), tgz_bytes, "byte-exact archive"); + assert_eq!(v["summary"]["failed"], 1, "envelope={v}"); + assert_eq!( + std::fs::read(&tgz).unwrap(), + corrupt, + "preserves the corrupt archive" + ); assert_eq!( std::fs::read(tmp.path().join("package-lock.json")).unwrap(), lock1 @@ -1430,7 +1396,11 @@ async fn identity_kept_over_different_service_bytes(symlinked_lock: bool) { assert_eq!(code, 0, "stdout={stdout} stderr={stderr}"); let v = parse_env(&stdout); assert_eq!(v["summary"]["rebuilt"], 1, "envelope={v}"); - assert_eq!(std::fs::read(&tgz).unwrap(), tgz_bytes, "the recorded bytes"); + assert_eq!( + std::fs::read(&tgz).unwrap(), + tgz_bytes, + "the recorded bytes" + ); assert_eq!( std::fs::read(tmp.path().join("package-lock.json")).unwrap(), lock1, diff --git a/crates/socket-patch-cli/tests/repair/repair_vendor_flavors_e2e.rs b/crates/socket-patch-cli/tests/repair/repair_vendor_flavors_e2e.rs index 44620f494..aa0287f29 100644 --- a/crates/socket-patch-cli/tests/repair/repair_vendor_flavors_e2e.rs +++ b/crates/socket-patch-cli/tests/repair/repair_vendor_flavors_e2e.rs @@ -378,26 +378,27 @@ async fn mount_patch_api_with(mock: &MockServer, extra: &[(&str, &[u8], &[u8])]) }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": files, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], "summary": "test vuln", + "severity": "high", "description": "details" + } + }, + "description": "Vendor patch", "license": "MIT", "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": files, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], "summary": "test vuln", - "severity": "high", "description": "details" - } - }, - "description": "Vendor patch", "license": "MIT", "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } - /// Runs through `common::run_with_env`, which seed-then-scrubs the ambient /// `SOCKET_*` surface the binary binds via clap `env=` (SOCKET_DRY_RUN, /// SOCKET_ECOSYSTEMS, SOCKET_CWD, ...) — an ambient value would silently @@ -670,13 +671,14 @@ async fn tampered_ledger_fails_closed(flavor: Flavor) { assert!( events_of(&env) .iter() - .any(|e| e["action"] == "failed" && e["errorCode"] == "vendor_artifact_rebuild_failed"), + .any(|e| e["action"] == "failed" + && e["errorCode"] == "vendor_artifact_redownload_failed"), "{}: envelope={env}", flavor.tag() ); assert!( - !tgz.exists(), - "{}: an unverifiable rebuild must not be left on disk", + tgz.is_file(), + "{}: a failed redownload must preserve the original artifact", flavor.tag() ); } diff --git a/crates/socket-patch-cli/tests/scan/covgap_commands_fetch_stage.rs b/crates/socket-patch-cli/tests/scan/covgap_commands_fetch_stage.rs index 1058e8ac1..180311dac 100644 --- a/crates/socket-patch-cli/tests/scan/covgap_commands_fetch_stage.rs +++ b/crates/socket-patch-cli/tests/scan/covgap_commands_fetch_stage.rs @@ -29,11 +29,9 @@ fn binary() -> PathBuf { const ORG_SLUG: &str = "test-org"; const UUID: &str = "11111111-1111-4111-8111-111111111111"; -const PURL: &str = "pkg:npm/left-pad@1.3.0"; const BEFORE: &[u8] = b"before\n"; const AFTER: &[u8] = b"after\n"; /// base64 of AFTER, the shape the view response's `blobContent` carries. -const AFTER_B64: &str = "YWZ0ZXIK"; /// Git-SHA256: SHA256("blob \0" ++ content). Computed independently /// of the code under test. @@ -275,183 +273,3 @@ async fn apply_online_nonquiet_prints_download_progress_and_diff_fallback() { ); } } - -// --------------------------------------------------------------------------- -// Vendor mem-stager: malformed patch view responses fail closed, with the -// non-quiet per-file / summary diagnostics. -// --------------------------------------------------------------------------- - -/// Committed manifest with NO local artifacts (no blobs/diffs/packages): -/// the vendor mem-stager must fetch the patch view for its content. -fn seed_sourceless_manifest(root: &Path) { - let socket = root.join(".socket"); - std::fs::create_dir_all(&socket).unwrap(); - std::fs::write( - socket.join("manifest.json"), - serde_json::to_vec_pretty(&serde_json::json!({ - "patches": { - PURL: { - "uuid": UUID, - "exportedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": git_sha256(BEFORE), - "afterHash": git_sha256(AFTER), - } - }, - "vulnerabilities": {}, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - } - } - })) - .unwrap(), - ) - .unwrap(); -} - -/// Mount `/patches/view/{UUID}` whose single file entry carries -/// `file_fields` — the knob each malformed-response variant turns. -async fn mount_view(mock: &MockServer, file_fields: serde_json::Value) { - Mock::given(method("GET")) - .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { "package/index.js": file_fields }, - "vulnerabilities": {}, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) - .mount(mock) - .await; -} - -/// Human-mode standalone `vendor` (staging runs before any package -/// matching, so no lockfile or installed tree is needed). -fn run_vendor_human(root: &Path, mock_uri: &str) -> (i32, String, String) { - run_cli_env( - root, - &["vendor"], - &[ - ("SOCKET_API_URL", mock_uri), - ("SOCKET_API_TOKEN", "fake-token"), - ("SOCKET_ORG_SLUG", ORG_SLUG), - ], - ) -} - -/// Shared postconditions for every malformed-view variant: exit 1, the -/// fetch was really attempted and its failure block names the failed purl -/// on stderr (the progress line is transient), and the fail-closed run -/// wrote nothing (no blobs — mem staging is disk-free — and no vendor -/// tree). -fn assert_failed_closed(root: &Path, code: i32, stdout: &str, stderr: &str) { - assert_eq!( - code, 1, - "a malformed view response must fail the run; stdout={stdout}\nstderr={stderr}" - ); - assert!( - !stderr.contains("Fetching content for"), - "the fetch progress is a transient status line; stderr={stderr}" - ); - assert!( - stderr.contains("Error: Could not fetch patch content for 1 patch:"), - "the summary block carries the failed count; stderr={stderr}" - ); - assert!( - stderr.contains(&format!(" - {PURL}")), - "the failed purl is listed; stderr={stderr}" - ); - assert!( - !root.join(".socket/blobs").exists(), - "mem staging must write no blobs" - ); - assert!( - !root.join(".socket/vendor").exists(), - "a failed staging must write no vendor tree" - ); -} - -/// Variant (a): the view entry has a valid `afterHash` but a null -/// `blobContent`. The per-file arm names the file on stderr, and the -/// summary block follows. -#[tokio::test] -async fn vendor_view_without_blob_content_reports_per_file_and_summary_errors() { - let mock = MockServer::start().await; - mount_view( - &mock, - serde_json::json!({ - "beforeHash": git_sha256(BEFORE), - "afterHash": git_sha256(AFTER), - "blobContent": null, - }), - ) - .await; - let tmp = tempfile::tempdir().unwrap(); - seed_sourceless_manifest(tmp.path()); - - let (code, stdout, stderr) = run_vendor_human(tmp.path(), &mock.uri()); - assert_failed_closed(tmp.path(), code, &stdout, &stderr); - assert!( - stderr.contains(&format!( - " [error] {PURL}: no blob content served for package/index.js" - )), - "the missing-content arm names patch AND file; stderr={stderr}" - ); -} - -/// Variant (b): valid `blobContent` but an `afterHash` that is 64 chars -/// of non-hex — rejected by the blob-hash key guard. The guard arm breaks -/// without a per-file line; only the summary block reports the failure. -#[tokio::test] -async fn vendor_view_with_invalid_after_hash_fails_closed_without_per_file_line() { - let mock = MockServer::start().await; - mount_view( - &mock, - serde_json::json!({ - "beforeHash": git_sha256(BEFORE), - "afterHash": "z".repeat(64), - "blobContent": AFTER_B64, - }), - ) - .await; - let tmp = tempfile::tempdir().unwrap(); - seed_sourceless_manifest(tmp.path()); - - let (code, stdout, stderr) = run_vendor_human(tmp.path(), &mock.uri()); - assert_failed_closed(tmp.path(), code, &stdout, &stderr); - assert!( - !stderr.contains("[error]"), - "the hash-guard arm breaks silently (no per-file line today); stderr={stderr}" - ); -} - -/// Variant (c): valid `afterHash` but a `blobContent` that is not -/// base64 — the decode arm fails closed, again with only the summary -/// block on stderr. -#[tokio::test] -async fn vendor_view_with_undecodable_blob_content_fails_closed() { - let mock = MockServer::start().await; - mount_view( - &mock, - serde_json::json!({ - "beforeHash": git_sha256(BEFORE), - "afterHash": git_sha256(AFTER), - "blobContent": "%%%not-base64%%%", - }), - ) - .await; - let tmp = tempfile::tempdir().unwrap(); - seed_sourceless_manifest(tmp.path()); - - let (code, stdout, stderr) = run_vendor_human(tmp.path(), &mock.uri()); - assert_failed_closed(tmp.path(), code, &stdout, &stderr); - assert!( - !stderr.contains("[error]"), - "the decode arm breaks silently (no per-file line today); stderr={stderr}" - ); -} diff --git a/crates/socket-patch-cli/tests/scan/covgap_commands_scan_vendor_flow.rs b/crates/socket-patch-cli/tests/scan/covgap_commands_scan_vendor_flow.rs index decd65db1..5190f1fd4 100644 --- a/crates/socket-patch-cli/tests/scan/covgap_commands_scan_vendor_flow.rs +++ b/crates/socket-patch-cli/tests/scan/covgap_commands_scan_vendor_flow.rs @@ -1,25 +1,7 @@ -//! Coverage-gap tests for `commands/scan/vendor_flow.rs`. -//! -//! Pins the otherwise-untested surfaces of `scan --vendor`: -//! -//! * the `already_vendored` dry-run preview arm (the sibling -//! `would_vendor` / `would_revendor` arms are pinned by -//! `scan_vendor_e2e.rs`); -//! * the legal-but-never-executed `--dry-run --prune` combination in the -//! vendor JSON path (GC preview field names, nothing mutated); -//! * every error constructor of `run_vendor_step` — `lock_held`, -//! `lock_io` (a directory squatting on `apply.lock`; a file squatting on -//! `.socket` itself) and `no_local_source` — through the JSON error fold -//! (a lock failure precedes the step and carries NO `vendor` key; a -//! staging failure carries the step's envelope demoted to -//! `partialFailure`, events-less because nothing mutates before staging) -//! and the interactive `Error (code): message` line; -//! * a corrupt legacy manifest, which vendored mode reports and steps -//! around (the manifest is not its record source). -//! -//! Fixtures are clones of `scan_vendor_e2e.rs` (each e2e file carries its -//! own copy — the established pattern), plus `e2e_safety_lock.rs`'s -//! external-flock trick for lock contention. Mock API only; no real hosts. +//! Scan vendoring previews, lock failures and corrupt legacy manifests. + +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; use std::fs::OpenOptions; use std::path::{Path, PathBuf}; @@ -143,11 +125,8 @@ async fn mount_discovery(mock: &MockServer, uuid: &str) { } /// Mount the full patch view for `uuid`. Without `with_blob_content` the -/// view carries the file hashes but no `blobContent`: the download phase -/// still records the patch (it needs only the hashes), but the vendor -/// step cannot obtain the patched bytes and staging fails -/// (`no_local_source`) — independent of how many times the view is -/// fetched along the way. +/// view carries only metadata. Positive fixtures also publish the complete +/// server archive; clients never need to stage these blobs. async fn mount_view(mock: &MockServer, uuid: &str, with_blob_content: bool) { let mut file = serde_json::json!({ "beforeHash": git_sha256(BEFORE), @@ -156,18 +135,22 @@ async fn mount_view(mock: &MockServer, uuid: &str, with_blob_content: bool) { if with_blob_content { file["blobContent"] = serde_json::json!(AFTER_B64); } + let view = serde_json::json!({ + "uuid": uuid, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { "package/index.js": file }, + "vulnerabilities": {}, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + if with_blob_content { + prebuilt_common::mount_view(mock, &view, None).await; + } Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { "package/index.js": file }, - "vulnerabilities": {}, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(view)) .mount(mock) .await; } @@ -326,23 +309,6 @@ fn assert_no_vendor_envelope(v: &serde_json::Value) { ); } -/// A staging failure happens AFTER the lock, inside the step: the fold -/// carries the step's envelope demoted to `partialFailure` (a consumer -/// reading `.vendor.status` inside a `"status":"error"` result must not -/// see the fresh-envelope default `success`) and events-less — nothing -/// mutates before staging, so there is no work to report. -fn assert_demoted_empty_vendor_envelope(v: &serde_json::Value) { - assert_eq!( - v["vendor"]["status"], "partialFailure", - "the carried envelope's status must be demoted; envelope={v}" - ); - assert_eq!( - v["vendor"]["events"], - serde_json::json!([]), - "nothing mutates before staging, so the aborted step reports no events; envelope={v}" - ); -} - /// Dry-run preview, same-uuid case: an entry already vendored at the /// discovered uuid is classified `already_vendored` — with no `oldUuid` /// key (that key marks `would_revendor` only) — and nothing on disk or @@ -581,85 +547,6 @@ async fn scan_vendor_socket_dir_file_reports_lock_io() { ); } -/// The JSON vendor-step error fold for a staging failure: the download -/// phase recorded the patch (hashes only), but the view serves no blob -/// content, so the vendor step cannot stage it and the run aborts -/// `no_local_source` with a `download` object and the step's own `vendor` -/// envelope carried through the fold — demoted to `partialFailure`, with -/// no events (nothing mutated before staging) — and creates nothing under -/// `.socket/`. Contract: the `vendor` sub-object is present whenever the -/// step ran; `get --mode vendored` shares the fold -/// (`covgap_commands_get::get_uuid_vendored_vendor_step_error_leaves_legacy_state_alone`). -#[tokio::test] -async fn scan_vendor_staging_error_reports_json_error() { - let mock = MockServer::start().await; - mount_discovery(&mock, UUID).await; - mount_view(&mock, UUID, /*with_blob_content=*/ false).await; - let tmp = tempfile::tempdir().unwrap(); - write_fixture(tmp.path()); - - let (code, stdout, stderr) = run_scan_vendor(tmp.path(), &mock.uri(), &[]); - let v = assert_vendor_step_error(code, &stdout, &stderr, "no_local_source"); - assert_demoted_empty_vendor_envelope(&v); - assert_eq!( - v["error"]["message"], "patch artifacts unavailable (offline or download failure)", - "envelope={v}" - ); - assert_eq!(v["download"]["downloaded"], 1, "envelope={v}"); - assert!( - !tmp.path().join(".socket").exists(), - "an aborted step leaves no .socket/ behind (lock file and empty dir removed)" - ); -} - -/// The interactive (non-JSON) twin of the staging failure: exit 1 with -/// the `Error (code): message` line on stderr, no JSON envelope on -/// stdout, nothing vendored. -#[tokio::test] -async fn scan_vendor_staging_error_interactive_prints_error_line() { - let mock = MockServer::start().await; - mount_discovery(&mock, UUID).await; - mount_view(&mock, UUID, /*with_blob_content=*/ false).await; - let tmp = tempfile::tempdir().unwrap(); - write_fixture(tmp.path()); - - let (code, stdout, stderr) = run_cli( - tmp.path(), - &[ - "scan", - "--vendor", - "--yes", - "--api-url", - &mock.uri(), - "--api-token", - "fake-token", - "--org", - ORG_SLUG, - ], - ); - - assert_eq!( - code, 1, - "an unstageable record must fail the run; stdout={stdout}; stderr={stderr}" - ); - assert!( - stderr.contains( - "Error (no_local_source): Patch artifacts unavailable (offline or download failure)." - ), - "the human arm must name the code and message on stderr; \ - stdout={stdout}; stderr={stderr}" - ); - // Human mode: no JSON envelope on stdout. - assert!( - serde_json::from_str::(stdout.trim()).is_err(), - "the interactive arm must not print a JSON envelope; stdout={stdout}" - ); - assert!( - !tmp.path().join(".socket").exists(), - "an aborted step leaves no .socket/ behind; stdout={stdout}; stderr={stderr}" - ); -} - use crate::vlt_hosted_common; use crate::vlt_vendored; diff --git a/crates/socket-patch-cli/tests/scan/hosted_management_refusals.rs b/crates/socket-patch-cli/tests/scan/hosted_management_refusals.rs index 2d84dd983..ebdb1ba53 100644 --- a/crates/socket-patch-cli/tests/scan/hosted_management_refusals.rs +++ b/crates/socket-patch-cli/tests/scan/hosted_management_refusals.rs @@ -146,11 +146,18 @@ async fn offline_eject_refuses_before_any_request() { std::fs::write(tmp.path().join("package-lock.json"), hosted_lock()).unwrap(); let before = snapshot(tmp.path()); let mut cmd = cli(); - cmd.args(["vendor", "--json", "--org", "test-org", "--api-token", "fake"]) - .arg("--api-url") - .arg(server.uri()) - .arg("--cwd") - .arg(tmp.path()); + cmd.args([ + "vendor", + "--json", + "--org", + "test-org", + "--api-token", + "fake", + ]) + .arg("--api-url") + .arg(server.uri()) + .arg("--cwd") + .arg(tmp.path()); if flag { cmd.arg("--offline"); } @@ -167,7 +174,11 @@ async fn offline_eject_refuses_before_any_request() { String::from_utf8_lossy(&out.stdout) ) }); - assert_eq!(out.status.code(), Some(1), "flag={flag} env={env} dry={dry}: {v}"); + assert_eq!( + out.status.code(), + Some(1), + "flag={flag} env={env} dry={dry}: {v}" + ); assert_eq!(v["error"]["code"], "offline_eject_unavailable", "{v}"); assert_eq!(snapshot(tmp.path()), before); } @@ -175,7 +186,10 @@ async fn offline_eject_refuses_before_any_request() { assert!( received.is_empty(), "an offline eject must not touch the network: {:?}", - received.iter().map(|r| r.url.to_string()).collect::>() + received + .iter() + .map(|r| r.url.to_string()) + .collect::>() ); } @@ -189,9 +203,7 @@ const MOCK_PATCH: &str = "33333333-3333-4333-8333-333333333333"; fn write_fresh_hosted_checkout(root: &Path, patch_origin: &str) -> String { write_package_json(root); let lock = lock( - &format!( - "{patch_origin}/patch/npm/left-pad/1.3.0/{GRANT}/{MOCK_PATCH}/left-pad-1.3.0.tgz" - ), + &format!("{patch_origin}/patch/npm/left-pad/1.3.0/{GRANT}/{MOCK_PATCH}/left-pad-1.3.0.tgz"), "sha512-patched==", ); std::fs::write(root.join("package-lock.json"), &lock).unwrap(); @@ -235,14 +247,17 @@ async fn mount_view_and_registry(server: &MockServer, tarball_status: u16) { fn eject_cmd(server: &MockServer, cwd: &Path, dry: bool) -> Command { let mut cmd = cli(); cmd.args(["vendor", "--json", "--org", ORG, "--api-token", "fake"]) - .args(["--vendor-source", "build"]) + .args(["--vendor-source", "service"]) .arg("--api-url") .arg(server.uri()) .arg("--patch-server-url") .arg(server.uri()) .arg("--cwd") .arg(cwd) - .env("SOCKET_NPM_REGISTRY", format!("{}/npm-registry", server.uri())); + .env( + "SOCKET_NPM_REGISTRY", + format!("{}/npm-registry", server.uri()), + ); if dry { cmd.arg("--dry-run"); } @@ -314,5 +329,8 @@ async fn dry_run_eject_verifies_the_plan_and_writes_nothing() { std::fs::read_to_string(tmp.path().join("package-lock.json")).unwrap(), hosted ); - assert!(!tmp.path().join(".socket").exists(), "a dry run creates no .socket/"); + assert!( + !tmp.path().join(".socket").exists(), + "a dry run creates no .socket/" + ); } diff --git a/crates/socket-patch-cli/tests/scan/scan_vendor_step_error_e2e.rs b/crates/socket-patch-cli/tests/scan/scan_vendor_step_error_e2e.rs index c53e4d07a..225d449d5 100644 --- a/crates/socket-patch-cli/tests/scan/scan_vendor_step_error_e2e.rs +++ b/crates/socket-patch-cli/tests/scan/scan_vendor_step_error_e2e.rs @@ -1,22 +1,4 @@ -//! Regression: the vendor step's ERROR returns must still hand the JSON -//! consumer the step's `vendor` envelope — demoted — instead of dropping it. -//! -//! `scan --vendor`'s vendor step (`run_vendor_step`) takes the apply -//! lock, stages the fetched records' patch content in memory, then drives -//! the vendor engine. Vendored mode is manifest-free: the step never reads -//! the manifest and never reconciles ledger entries against it, so a -//! staging failure (`no_local_source`: the API serves the patch view -//! without blob content) aborts a run that has mutated nothing. The run -//! DID enter the step, though, and the contract's `vendor` sub-object -//! rides the error fold: `status` demoted to `partialFailure` (a consumer -//! reading `.vendor.status` inside a `"status":"error"` result must not -//! see the fresh-envelope default of `success`) and `events[]` present — -//! empty, and in particular holding no revert of a ledger entry the run -//! did not select. The `vendor` command prints its envelope on the same -//! failure (`vendor::run` emits `env` whatever `run_vendor` returned); -//! scan's JSON arm must not be the one place it vanishes. `get --mode -//! vendored` shares the fold -//! (`covgap_commands_get::get_uuid_vendored_vendor_step_error_leaves_legacy_state_alone`). +//! A failed artifact download preserves the vendor envelope and unrelated ledger entries. use std::path::{Path, PathBuf}; use std::process::Command; @@ -208,7 +190,7 @@ fn run_cli(root: &Path, argv: &[&str]) -> (i32, String, String) { } #[tokio::test] -async fn scan_vendor_staging_error_still_carries_the_demoted_vendor_envelope() { +async fn scan_vendor_download_error_preserves_the_vendor_envelope() { let mock = MockServer::start().await; mount_discovery(&mock).await; mount_contentless_view(&mock).await; @@ -238,38 +220,15 @@ async fn scan_vendor_staging_error_still_carries_the_demoted_vendor_envelope() { ); let v: serde_json::Value = serde_json::from_str(stdout.trim()) .unwrap_or_else(|e| panic!("stdout must be one JSON object ({e}); stdout={stdout}")); - assert_eq!(v["status"], "error", "envelope={v}"); - assert_eq!( - v["error"]["code"], "no_local_source", - "precondition: the run must abort at staging; envelope={v}" - ); - - // Non-vacuous: the run got PAST the download phase (the record was - // fetched — hashes only — into memory, detached) and INTO the step. - assert_eq!(v["download"]["downloaded"], 1, "envelope={v}"); - assert_eq!(v["download"]["detached"], true, "envelope={v}"); - - // The carried envelope must not claim the vendor step succeeded: the - // run aborted at staging, so a consumer reading `.vendor.status` inside - // a `"status":"error"` result must see the demoted status, not the - // fresh-envelope default of "success". - assert_eq!( - v["vendor"]["status"], "partialFailure", - "the carried envelope's own status must be demoted; envelope={v}" - ); - - // The point: the envelope survives the error fold — `events[]` is where - // any pre-failure work would be reported — and the manifest-free step - // reconciles nothing: no event names the unselected legacy entry, whose - // ledger bytes are untouched. - let events = v["vendor"]["events"].as_array().unwrap_or_else(|| { - panic!("the vendor envelope must survive the staging error; envelope={v}") - }); - assert!( - events.is_empty(), - "nothing mutates before staging — in particular the unselected {UNSELECTED_PURL} \ - is never reconciled; envelope={v}" - ); + assert_eq!(v["status"], "partial_failure", "{v}"); + assert_eq!(v["download"]["downloaded"], 1, "{v}"); + assert_eq!(v["download"]["detached"], true, "{v}"); + assert_eq!(v["vendor"]["status"], "partialFailure", "{v}"); + let events = v["vendor"]["events"].as_array().unwrap(); + assert_eq!(events.len(), 1, "{v}"); + assert_eq!(events[0]["purl"], PURL, "{v}"); + assert_eq!(events[0]["errorCode"], "apply_failed", "{v}"); + assert!(events[0]["error"].as_str().unwrap().contains("404")); assert_eq!( std::fs::read_to_string(tmp.path().join(".socket/vendor/state.json")).unwrap(), ledger_before, diff --git a/crates/socket-patch-cli/tests/scan_rollout_e2e.rs b/crates/socket-patch-cli/tests/scan_rollout_e2e.rs index 2e0fba55d..6af41211c 100644 --- a/crates/socket-patch-cli/tests/scan_rollout_e2e.rs +++ b/crates/socket-patch-cli/tests/scan_rollout_e2e.rs @@ -3,6 +3,9 @@ //! three packages per run, most severe first, in hosted, agent and vendored //! mode; a fourth run changes nothing. Mock API, the built binary. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -667,6 +670,14 @@ async fn agent_cap_rolls_forward_and_upgrades_ignore_the_cap() { async fn vendored_cap_rolls_forward_three_per_run() { let mock = MockServer::start().await; mount_api(&mock, |_| Grant::Granted).await; + for (name, severities) in PACKAGES { + let view = json!({ + "uuid": uuid(name), "purl": purl(name), "publishedAt": "2026-01-01T00:00:00Z", + "files": { "package/index.js": { "beforeHash": git_sha256(&before(name)), "afterHash": git_sha256(&after(name)), "blobContent": b64(&after(name)) } }, + "vulnerabilities": vulns(name, severities), "description": name, "license": "MIT", "tier": "free" + }); + prebuilt_common::mount_view(&mock, &view, None).await; + } let tmp = tempfile::tempdir().unwrap(); let names9: Vec<&str> = PACKAGES.iter().map(|(n, _)| *n).collect(); write_project(tmp.path(), &names9); diff --git a/crates/socket-patch-cli/tests/scan_vendor_e2e.rs b/crates/socket-patch-cli/tests/scan_vendor_e2e.rs index 280c9febc..96126cfc5 100644 --- a/crates/socket-patch-cli/tests/scan_vendor_e2e.rs +++ b/crates/socket-patch-cli/tests/scan_vendor_e2e.rs @@ -5,6 +5,9 @@ //! embedded records are the only state written. Mock API + a real npm lockfile fixture, driven //! through the built binary. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -122,31 +125,33 @@ async fn mount_patch_api(mock: &MockServer, uuid: &str) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": uuid, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], + "summary": "test vuln", + "severity": "high", + "description": "details" + } + }, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], - "summary": "test vuln", - "severity": "high", - "description": "details" - } - }, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -443,7 +448,7 @@ async fn scan_vendor_migrates_legacy_manifest_mode_project() { // The legacy state, produced by the (still manifest-driven) standalone // `vendor` command from a committed manifest + blob. seed_committed_manifest(tmp.path()); - let (code, venv, stderr) = run_vendor(tmp.path(), &["--vendor-source", "build"]); + let (code, venv, stderr) = run_vendor(tmp.path(), &["--vendor-source", "service"]); assert_eq!(code, 0, "legacy setup: {venv:#} {stderr}"); let state_path = tmp.path().join(".socket/vendor/state.json"); let state: serde_json::Value = @@ -524,11 +529,8 @@ async fn scan_vendor_writes_no_manifest() { let tmp = tempfile::tempdir().unwrap(); write_fixture(tmp.path()); - let (code, stdout, stderr) = run_scan_vendor( - tmp.path(), - &mock.uri(), - &["--vex", "out.vex.json"], - ); + let (code, stdout, stderr) = + run_scan_vendor(tmp.path(), &mock.uri(), &["--vex", "out.vex.json"]); assert_eq!(code, 0, "stdout={stdout}; stderr={stderr}"); let v: serde_json::Value = serde_json::from_str(stdout.trim()).expect("valid JSON"); assert_eq!(v["status"], "success", "envelope={v}"); @@ -938,24 +940,26 @@ async fn mount_scoped_patch_api(mock: &MockServer, uuid: &str) { }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": uuid, + "purl": SCOPED_API_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": {}, + "description": "Vendor patch", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": SCOPED_API_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": {}, - "description": "Vendor patch", - "license": "MIT", - "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -1218,8 +1222,8 @@ async fn scan_vendor_annotates_mismatched_baseline_and_vendors_anyway() { "the annotation names the purl; stdout={stdout}" ); assert!( - stderr.contains("vendored the patched content anyway"), - "overwrite warning surfaced; stderr={stderr}" + !stderr.contains("vendored the patched content anyway"), + "the server archive does not overwrite installed content; stderr={stderr}" ); // Vendored despite the mismatch. assert!(tmp @@ -1346,7 +1350,9 @@ async fn mount_registry_tarball(mock: &MockServer, tgz: Vec) { fn run_vendor(root: &Path, extra: &[&str]) -> (i32, serde_json::Value, String) { let mut argv = vec!["vendor", "--json"]; argv.extend_from_slice(extra); + let fixture = prebuilt_common::Server::project(root); let out = Command::new(binary()) + .env("SOCKET_VENDOR_URL", &fixture.uri) .args(&argv) .current_dir(root) .env("SOCKET_TELEMETRY_DISABLED", "1") @@ -1363,7 +1369,7 @@ fn run_vendor(root: &Path, extra: &[&str]) -> (i32, serde_json::Value, String) { /// is fetched pristine from the registry (integrity-verified against the /// lock) and vendored — node_modules never appears. #[tokio::test] -async fn vendor_auto_fetches_missing_package_from_lockfile() { +async fn vendor_downloads_missing_package_without_fetching_pristine_sources() { let mock = MockServer::start().await; let tgz = pristine_tgz(); let integrity = sri_of(&tgz); @@ -1389,8 +1395,8 @@ async fn vendor_auto_fetches_missing_package_from_lockfile() { assert!( events .iter() - .any(|e| e["errorCode"] == "vendor_fetched_missing"), - "fetch surfaced as a warning event: {v:#}" + .any(|e| e["errorCode"] == "vendor_prebuilt_downloaded"), + "service download reported: {v:#}" ); assert!(tmp .path() @@ -1501,8 +1507,8 @@ async fn vendor_auto_takes_a_missing_package_from_the_service_without_the_regist .unwrap() .iter() .any(|e| e["purl"] == PURL - && e["action"] == "skipped" - && e["errorCode"] == "package_not_installed"), + && e["action"] == "failed" + && e["errorCode"] == "vendor_service_offline_conflict"), "{v:#}" ); assert!(registry @@ -1544,7 +1550,7 @@ async fn vendor_auto_takes_a_missing_package_from_the_service_without_the_regist /// Integrity mismatch between the lock and the served bytes is a distinct /// vendor_fetch_failed failure — and nothing is written. #[tokio::test] -async fn vendor_fetch_integrity_mismatch_is_vendor_fetch_failed() { +async fn vendor_uses_service_integrity_without_fetching_old_registry_bytes() { let mock = MockServer::start().await; mount_registry_tarball(&mock, pristine_tgz()).await; @@ -1557,21 +1563,12 @@ async fn vendor_fetch_integrity_mismatch_is_vendor_fetch_failed() { seed_manifest_and_blob(tmp.path()); let (code, v, _) = run_vendor(tmp.path(), &[]); - assert_ne!(code, 0, "{v:#}"); - let events = v["events"].as_array().unwrap(); - assert!( - events - .iter() - .any(|e| e["action"] == "failed" && e["errorCode"] == "vendor_fetch_failed"), - "{v:#}" - ); - assert!( - !events - .iter() - .any(|e| e["errorCode"] == "package_not_installed"), - "no duplicate not-installed skip: {v:#}" - ); - assert!(!tmp.path().join(".socket/vendor").exists()); + assert_eq!(code, 0, "{v:#}"); + assert!(mock.received_requests().await.unwrap().is_empty()); + assert!(tmp + .path() + .join(format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz")) + .is_file()); } /// --offline refuses the fetch with a calm package_not_installed skip that @@ -1589,24 +1586,13 @@ async fn vendor_offline_refuses_fetch_with_calm_skip() { let (code, v, _) = run_vendor(tmp.path(), &["--offline"]); assert_ne!(code, 0, "not-installed stays a non-benign skip: {v:#}"); - let events = v["events"].as_array().unwrap(); - let skip = events - .iter() - .find(|e| e["errorCode"] == "package_not_installed") - .unwrap_or_else(|| panic!("{v:#}")); - assert!( - skip["reason"] - .as_str() - .unwrap_or("") - .contains("--offline prevents fetching"), - "offline detail names the lockfile resolution: {v:#}" - ); + assert!(v["events"].as_array().unwrap().iter().any(|e| e["action"] == "failed" && e["errorCode"] == "vendor_service_offline_conflict"), "{v:#}"); } /// An entry whose lock records no integrity is never fetched (fail-closed) /// and keeps the plain not-installed outcome plus an explanatory warning. #[tokio::test] -async fn vendor_fetch_unverifiable_lock_entry_stays_not_installed() { +async fn vendor_verifies_server_artifact_without_old_lock_integrity() { let tmp = tempfile::tempdir().unwrap(); // Hand-write a lock whose entry has no integrity field. std::fs::write( @@ -1632,20 +1618,8 @@ async fn vendor_fetch_unverifiable_lock_entry_stays_not_installed() { seed_manifest_and_blob(tmp.path()); let (code, v, _) = run_vendor(tmp.path(), &[]); - assert_ne!(code, 0, "{v:#}"); - let events = v["events"].as_array().unwrap(); - assert!( - events - .iter() - .any(|e| e["errorCode"] == "vendor_fetch_unverifiable"), - "{v:#}" - ); - assert!( - events - .iter() - .any(|e| e["errorCode"] == "package_not_installed"), - "{v:#}" - ); + assert_eq!(code, 0, "{v:#}"); + assert_eq!(v["summary"]["applied"], 1, "{v:#}"); } /// The headline flow: a COMPLETELY fresh clone (lockfile, no node_modules, @@ -2038,7 +2012,7 @@ async fn scan_vendored_bun_silent_human_names_code_on_stderr() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--silent", "--yes", "--api-url", @@ -2227,7 +2201,7 @@ async fn standalone_vendor_state_attests_from_the_embedded_record() { let pristine = std::fs::read(tmp.path().join("package-lock.json")).unwrap(); seed_manifest_and_blob_with_vuln(tmp.path()); - let (code, v, stderr) = run_vendor(tmp.path(), &["--offline"]); + let (code, v, stderr) = run_vendor(tmp.path(), &[]); assert_eq!(code, 0, "{v:#}\n{stderr}"); assert_eq!(v["summary"]["applied"], 1, "{v:#}"); let state: serde_json::Value = serde_json::from_str( @@ -2479,27 +2453,29 @@ snapshots: }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": uuid, + "purl": purl(name), + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], "summary": "test vuln", + "severity": "high", "description": "details" + } + }, + "description": "plan target", "license": "MIT", "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": purl(name), - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], "summary": "test vuln", - "severity": "high", "description": "details" - } - }, - "description": "plan target", "license": "MIT", "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -2655,27 +2631,29 @@ snapshots: }))) .mount(mock) .await; + let archive_view = serde_json::json!({ + "uuid": uuid, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + file: { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": AFTER_B64, + } + }, + "vulnerabilities": { + "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2026-0001"], "summary": "test vuln", + "severity": "high", "description": "details" + } + }, + "description": "plan target", "license": "MIT", "tier": "free", + }); + crate::prebuilt_common::mount_view(mock, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "uuid": uuid, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - file: { - "beforeHash": before_hash, - "afterHash": after_hash, - "blobContent": AFTER_B64, - } - }, - "vulnerabilities": { - "GHSA-aaaa-bbbb-cccc": { - "cves": ["CVE-2026-0001"], "summary": "test vuln", - "severity": "high", "description": "details" - } - }, - "description": "plan target", "license": "MIT", "tier": "free", - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(mock) .await; } @@ -3014,14 +2992,14 @@ snapshots: ); assert_eq!( events_for(&v, "pkg:npm/pkg-z@1.0.0"), - vec![("skipped", "package_not_installed")], + vec![("failed", "vendor_lock_entry_not_found")], "{v}" ); assert!(events_for(&v, "pkg:npm/pkg-b@1.0.0").is_empty(), "{v}"); assert!(events_for(&v, "pkg:npm/pkg-y@1.0.0").is_empty(), "{v}"); assert_eq!( events_for(&v, "pkg:npm/pkg-a@1.0.0"), - vec![("applied", "")], + vec![("applied", ""), ("skipped", "vendor_prebuilt_downloaded")], "{v}" ); assert_eq!( @@ -3057,7 +3035,7 @@ snapshots: assert_eq!(v["failed"], 0, "{v}"); assert_eq!( events_for(&v, "pkg:npm/pkg-z@1.0.0"), - vec![("skipped", "package_not_installed")], + vec![("failed", "vendor_lock_entry_not_found")], "{v}" ); @@ -3161,10 +3139,14 @@ snapshots: "{v}" ); assert!(events_for(&v, CARGO_SCOPE[0].0).is_empty(), "{v}"); - assert_eq!(record_for(dl, CARGO_SCOPE[1].0)["action"], "downloaded", "{v}"); + assert_eq!( + record_for(dl, CARGO_SCOPE[1].0)["action"], + "downloaded", + "{v}" + ); assert_eq!( events_for(&v, CARGO_SCOPE[1].0), - vec![("skipped", "package_not_installed")], + vec![("failed", "locked_version_mismatch")], "{v}" ); assert_eq!(viewed_uuids(&mock).await, vec![UUID_Z.to_string()]); @@ -3185,7 +3167,7 @@ snapshots: ); assert_eq!( events_for(&v, CARGO_SCOPE[1].0), - vec![("skipped", "package_not_installed")], + vec![("failed", "locked_version_mismatch")], "{v}" ); let v = run_json( diff --git a/crates/socket-patch-cli/tests/vendor/in_process_vendor_bun.rs b/crates/socket-patch-cli/tests/vendor/in_process_vendor_bun.rs index 937a6e484..8e155b408 100644 --- a/crates/socket-patch-cli/tests/vendor/in_process_vendor_bun.rs +++ b/crates/socket-patch-cli/tests/vendor/in_process_vendor_bun.rs @@ -357,6 +357,7 @@ async fn mount_patch_api(mock: &MockServer) { } async fn mount_view(mock: &MockServer, uuid: &str, purl: &str) { + crate::prebuilt_common::mount_view(mock, &view_body(uuid, purl), None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) .respond_with(ResponseTemplate::new(200).set_body_json(view_body(uuid, purl))) @@ -397,7 +398,7 @@ fn with_api<'a>(argv: &[&'a str], uri: &'a str) -> Vec<&'a str> { } fn scan_vendored(root: &Path, uri: &str, extra: &[&str]) -> (i32, String, String) { - let mut argv = vec!["scan", "--mode", "vendored", "--vendor-source", "build"]; + let mut argv = vec!["scan", "--mode", "vendored", "--vendor-source", "service"]; argv.extend_from_slice(extra); run(root, &with_api(&argv, uri)) } @@ -409,7 +410,7 @@ fn get_vendored(root: &Path, uri: &str, ident: &str, extra: &[&str]) -> (i32, St "--mode", "vendored", "--vendor-source", - "build", + "service", ]; argv.extend_from_slice(extra); run(root, &with_api(&argv, uri)) @@ -775,7 +776,7 @@ async fn silent_refusals_stay_visible_on_stderr_with_empty_stdout() { let tmp = tempfile::tempdir().unwrap(); write_bun_project(tmp.path(), LockShape::V1Workspace); let mut argv = argv; - argv.extend_from_slice(&["--vendor-source", "build"]); + argv.extend_from_slice(&["--vendor-source", "service"]); let (exit, stdout, stderr) = run(tmp.path(), &with_api(&argv, &mock.uri())); assert_eq!(exit, 1, "{label}: stdout={stdout}\nstderr={stderr}"); assert!( @@ -1510,10 +1511,11 @@ async fn repair_rebuilds_a_deleted_artifact_through_a_digestless_lock() { let mock = MockServer::start().await; mount_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); - let (_, wired, _) = vendor_then_drop_digest(tmp.path(), &mock.uri()); + let (_, _, _) = vendor_then_drop_digest(tmp.path(), &mock.uri()); let tgz = tmp .path() .join(format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz")); + let lock_before = lock_bytes(tmp.path()); std::fs::remove_file(&tgz).unwrap(); // `scan --mode vendored` keeps no local blob in this harness, so the @@ -1533,9 +1535,9 @@ async fn repair_rebuilds_a_deleted_artifact_through_a_digestless_lock() { ); assert!(tgz.is_file(), "the artifact must be rebuilt"); assert_eq!( - String::from_utf8(lock_bytes(tmp.path())).unwrap(), - wired, - "the rebuild re-pins the digest into the healed 3-tuple" + lock_bytes(tmp.path()), + lock_before, + "redownload preserves the existing lockfile exactly" ); } @@ -1673,7 +1675,7 @@ async fn fifo_bun_lock_is_refused_without_blocking() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--json", ], &uri, @@ -1696,7 +1698,7 @@ async fn fifo_bun_lock_is_refused_without_blocking() { "--mode", "vendored", "--vendor-source", - "build", + "service", "--json", ], &uri, diff --git a/crates/socket-patch-cli/tests/vendor/main.rs b/crates/socket-patch-cli/tests/vendor/main.rs index 41bd1f72b..ddfa3588f 100644 --- a/crates/socket-patch-cli/tests/vendor/main.rs +++ b/crates/socket-patch-cli/tests/vendor/main.rs @@ -14,5 +14,7 @@ mod e2e_golang_redirect; mod in_process_vendor_bun; mod redirect_npm_allow_remote; mod vendor_gem_lockfile_only_e2e; -mod vendor_pristine_fetch_order_e2e; mod vendor_rerun_no_network_e2e; + +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; diff --git a/crates/socket-patch-cli/tests/vendor/vendor_gem_lockfile_only_e2e.rs b/crates/socket-patch-cli/tests/vendor/vendor_gem_lockfile_only_e2e.rs index 81578de30..77cb8f6a5 100644 --- a/crates/socket-patch-cli/tests/vendor/vendor_gem_lockfile_only_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor/vendor_gem_lockfile_only_e2e.rs @@ -1,23 +1,4 @@ -//! `vendor --vendor-source build` on a gem the project only has in its -//! lockfile. -//! -//! The local gem build needs the eval-able stub gemspec rubygems writes -//! into `/specifications/` when the gem is INSTALLED — a bundler -//! path source will not load without one, and a downloaded `.gem` carries -//! its gemspec only as YAML in `metadata.gz` (the vendoring service's -//! converter is what turns that into the Ruby form, and serves it as the -//! `gem-stub-gemspec` second artifact). So build mode cannot vendor a -//! fetched gem, ever — and the vendor loop refuses it `gem_spec_missing` -//! BEFORE downloading it (the X1b deferred-fetch gate). -//! `vendor_rerun_no_network_e2e` pins the gate itself; -//! this suite pins its SCOPE: `auto` still fetches, and the three runs a -//! fetch would never have happened for (nothing resolves the gem, the lock -//! cannot verify it, the ledger already vendors it) keep their own -//! outcomes instead of a gemspec refusal. -//! -//! Hermetic: a `wiremock` stand-in for the rubygems download host, named by -//! the lock's `remote:`, and a `.socket/blobs` entry so patch staging never -//! reaches the API. +//! Service vendoring of portable gems without a local install or registry fetch. use std::path::{Path, PathBuf}; use std::process::Command; @@ -181,6 +162,8 @@ fn run_vendor(root: &Path, source: &str, api_url: &str) -> (i32, serde_json::Val } } cmd.env("SOCKET_TELEMETRY_DISABLED", "1"); + let fixture = crate::prebuilt_common::Server::project(root); + fixture.command(&mut cmd); let out = cmd.output().expect("run socket-patch vendor"); let stdout = String::from_utf8_lossy(&out.stdout).into_owned(); let stderr = String::from_utf8_lossy(&out.stderr).into_owned(); @@ -210,232 +193,77 @@ fn failed_event(v: &serde_json::Value) -> &serde_json::Value { } #[tokio::test] -async fn build_mode_refuses_a_lockfile_only_gem_before_downloading_it() { +async fn service_vendors_a_lockfile_only_gem_with_its_server_stub() { let mock = MockServer::start().await; - let gem = make_gem(); - let sha = hex::encode(Sha256::digest(&gem)); - mount_gem_download(&mock, gem).await; - + mount_gem_download(&mock, make_gem()).await; let tmp = tempfile::tempdir().unwrap(); - write_fixture(tmp.path(), &mock.uri(), &sha); - - let (code, v, stderr) = run_vendor(tmp.path(), "build", &dead_endpoint()); - - assert_eq!(code, 1, "the refusal fails the run: {v:#}\n{stderr}"); - assert!( - mock.received_requests() - .await - .unwrap_or_default() - .is_empty(), - "build mode cannot use a fetched gem, so it must not download one" - ); - let failed = failed_event(&v); - assert_eq!(failed["purl"], PURL, "{v:#}"); - assert_eq!( - failed["errorCode"], "gem_spec_missing", - "the backend's own refusal code, raised earlier: {v:#}" - ); - let detail = failed["error"].as_str().unwrap_or_default(); - assert!( - detail.contains("stub gemspec") - && detail.contains("install the gem") - && detail.contains("--vendor-source"), - "the refusal must say why and name the remedy: {detail}" - ); - assert!( - !tmp.path().join(".socket/vendor").exists(), - "nothing is written: {v:#}" - ); - let lock = std::fs::read_to_string(tmp.path().join("Gemfile.lock")).unwrap(); - assert!(lock.contains("GEM\n"), "the lock is untouched: {lock}"); + write_fixture(tmp.path(), &mock.uri(), &"0".repeat(64)); + let (code, v, stderr) = run_vendor(tmp.path(), "service", &dead_endpoint()); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + let dir = tmp + .path() + .join(format!(".socket/vendor/gem/{UUID}/{NAME}-{VERSION}")); + assert_eq!(std::fs::read(dir.join(LIB)).unwrap(), PATCHED); + assert!(dir.join(format!("{NAME}.gemspec")).is_file()); + assert!(mock.received_requests().await.unwrap().is_empty()); } -/// The gate is scoped to build-only runs: `auto` may still vendor this gem -/// through the patch service, and the service path needs the fetched copy -/// staged, so the download must still happen there. #[tokio::test] -async fn auto_mode_still_fetches_a_lockfile_only_gem() { +async fn auto_alias_does_not_fetch_the_registry_gem() { let mock = MockServer::start().await; - let gem = make_gem(); - let sha = hex::encode(Sha256::digest(&gem)); - mount_gem_download(&mock, gem).await; - + mount_gem_download(&mock, make_gem()).await; let tmp = tempfile::tempdir().unwrap(); - write_fixture(tmp.path(), &mock.uri(), &sha); - - // The patch service is unreachable, so `auto` falls back to the local - // build and lands on the same refusal — AFTER the fetch, which is the - // behavior this mode needs. + write_fixture(tmp.path(), &mock.uri(), &"0".repeat(64)); let (code, v, stderr) = run_vendor(tmp.path(), "auto", &dead_endpoint()); - - assert_eq!(code, 1, "{v:#}\n{stderr}"); - assert_eq!(failed_event(&v)["purl"], PURL, "{v:#}"); - assert_eq!( - mock.received_requests().await.unwrap_or_default().len(), - 1, - "auto must still stage the pristine gem for the service path" - ); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + assert!(mock.received_requests().await.unwrap().is_empty()); } -/// The gate is also scoped to gems a fetch would actually be attempted for. -/// A gem that no lockfile resolves and no ledger entry recovers has nothing -/// to fetch and nothing to say about gemspecs: it keeps the calm -/// `package_not_installed` skip, not a gemspec refusal. -#[tokio::test] -async fn a_gem_that_resolves_from_nowhere_still_reports_not_installed() { +#[test] +fn a_gem_without_a_lockfile_is_refused_before_downloading() { let tmp = tempfile::tempdir().unwrap(); write_fixture(tmp.path(), "https://rubygems.org", &"0".repeat(64)); - // No lockfile at all: nothing resolves the gem. std::fs::remove_file(tmp.path().join("Gemfile.lock")).unwrap(); - - let (code, v, stderr) = run_vendor(tmp.path(), "build", &dead_endpoint()); - + let (code, v, stderr) = run_vendor(tmp.path(), "service", &dead_endpoint()); assert_eq!(code, 1, "{v:#}\n{stderr}"); - let event = v["events"] - .as_array() - .expect("events array") - .iter() - .find(|e| e["purl"] == PURL) - .unwrap_or_else(|| panic!("expected an event for {PURL} in:\n{v:#}")); - assert_eq!(event["action"], "skipped", "{v:#}"); - assert_eq!(event["errorCode"], "package_not_installed", "{v:#}"); + assert_eq!( + failed_event(&v)["errorCode"], + "vendor_lockfile_missing", + "{v:#}" + ); } -// ── scope guards ──────────────────────────────────────────────────────── -// -// The refusal must fire ONLY where the wasted download it replaces would -// really have happened: a gem the lock resolves WITH a verifier, and that -// the run is not already vendoring from its committed artifact. The two -// cases where no fetch ever happens keep the outcome they had before the -// gate existed. - -/// A bundler < 2.6 `Gemfile.lock` (no `CHECKSUMS` section — the majority of -/// real locks) resolves the gem but cannot VERIFY it, and -/// `registry_fetch::fetch_and_stage` refuses such an entry before any -/// network I/O. CLI_CONTRACT: "Entries the lock cannot verify are NEVER -/// fetched (`vendor_fetch_unverifiable` warning + the calm -/// `package_not_installed` skip)". There is no download to save here, so -/// the gemspec refusal must not replace that documented pair — all the more -/// so because its remedy (`--vendor-source=auto`) cannot work either: the -/// purl never reaches the gem backend in any mode. #[tokio::test] -async fn an_unverifiable_lock_entry_keeps_the_documented_skip_pair() { +async fn service_integrity_supports_old_bundler_locks_without_checksums() { let mock = MockServer::start().await; - mount_gem_download(&mock, make_gem()).await; - let tmp = tempfile::tempdir().unwrap(); write_fixture(tmp.path(), &mock.uri(), &"0".repeat(64)); - // Re-write the lock the way bundler < 2.6 does: no CHECKSUMS section. - std::fs::write( - tmp.path().join("Gemfile.lock"), - format!( - "GEM\n remote: {}\n specs:\n {NAME} ({VERSION})\n\n\ - PLATFORMS\n ruby\n\n\ - DEPENDENCIES\n {NAME}\n\n\ - BUNDLED WITH\n 2.4.10\n", - mock.uri() - ), - ) - .unwrap(); - - let (code, v, stderr) = run_vendor(tmp.path(), "build", &dead_endpoint()); - - assert_eq!(code, 1, "{v:#}\n{stderr}"); - assert!( - mock.received_requests() - .await - .unwrap_or_default() - .is_empty(), - "an unverifiable entry is never fetched: {v:#}" - ); - let codes: Vec<(&str, &str)> = v["events"] - .as_array() - .expect("events array") - .iter() - .filter(|e| e["purl"] == PURL) - .map(|e| { - ( - e["action"].as_str().unwrap_or_default(), - e["errorCode"].as_str().unwrap_or_default(), - ) - }) - .collect(); - assert_eq!( - codes, - vec![ - ("skipped", "vendor_fetch_unverifiable"), - ("skipped", "package_not_installed"), - ], - "an unverifiable lock entry keeps its documented warning + calm \ - skip, not a gemspec refusal: {v:#}" - ); + let lock = tmp.path().join("Gemfile.lock"); + let text = std::fs::read_to_string(&lock).unwrap(); + let start = text.find("CHECKSUMS\n").unwrap(); + let end = text.find("BUNDLED WITH\n").unwrap(); + std::fs::write(&lock, format!("{}{}", &text[..start], &text[end..])).unwrap(); + let (code, v, stderr) = run_vendor(tmp.path(), "service", &dead_endpoint()); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + assert!(mock.received_requests().await.unwrap().is_empty()); } -/// An ALREADY-VENDORED gem on a fresh clone (the committed -/// `.socket/vendor/gem/` copy is the dependency; no installed gem, -/// because `bundle install` has not run yet) must re-scan green in build -/// mode: the gem backend's idempotent hot path re-confirms the wired lock -/// and returns `already_vendored` without ever needing a stub gemspec of -/// its own. The ledger covers the record, so the pristine fetch is -/// deferred and never needed: the re-run makes no registry request at all. -#[tokio::test] -async fn an_already_vendored_gem_re_runs_green_on_a_fresh_clone() { - let mock = MockServer::start().await; - let gem = make_gem(); - let sha = hex::encode(Sha256::digest(&gem)); - mount_gem_download(&mock, gem).await; - +#[test] +fn an_already_vendored_gem_is_reused_without_an_installed_copy() { let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - write_fixture(root, &mock.uri(), &sha); - install_gem(root); - - // Run 1: the gem is installed, so the local build vendors it. - let (code, v, stderr) = run_vendor(root, "build", &dead_endpoint()); - assert_eq!( - code, 0, - "run 1 must vendor the installed gem: {v:#}\n{stderr}" - ); - assert!( - root.join(format!(".socket/vendor/gem/{UUID}")).is_dir(), - "run 1 must commit the vendored copy: {v:#}" - ); - - // Fresh clone: the committed artifact and the wired lock are checked - // in, the installed gem is not. - std::fs::remove_dir_all(root.join("vendor")).unwrap(); - - let (code, v, stderr) = run_vendor(root, "build", &dead_endpoint()); - - assert_eq!( - code, 0, - "an in-sync re-run of an already-vendored gem is green: {v:#}\n{stderr}" - ); - assert_eq!(v["status"], "success", "{v:#}\n{stderr}"); + write_fixture(tmp.path(), &dead_endpoint(), &"0".repeat(64)); + install_gem(tmp.path()); + let (code, v, stderr) = run_vendor(tmp.path(), "service", &dead_endpoint()); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + std::fs::remove_dir_all(tmp.path().join("vendor")).unwrap(); + let (code, v, stderr) = run_vendor(tmp.path(), "service", &dead_endpoint()); + assert_eq!(code, 0, "{v:#}\n{stderr}"); assert!( - mock.received_requests() - .await - .unwrap_or_default() - .is_empty(), - "the in-sync hot path answers from the committed copy, with no \ - registry request: {v:#}" - ); - let codes: Vec<(&str, &str)> = v["events"] - .as_array() - .expect("events array") - .iter() - .filter(|e| e["purl"] == PURL) - .map(|e| { - ( - e["action"].as_str().unwrap_or_default(), - e["errorCode"].as_str().unwrap_or_default(), - ) - }) - .collect(); - assert_eq!( - codes, - vec![("skipped", "already_vendored")], - "the hot path reports the committed copy in sync, not a gemspec \ - refusal: {v:#}" + v["events"] + .as_array() + .unwrap() + .iter() + .any(|e| e["errorCode"] == "already_vendored"), + "{v:#}" ); } diff --git a/crates/socket-patch-cli/tests/vendor/vendor_pristine_fetch_order_e2e.rs b/crates/socket-patch-cli/tests/vendor/vendor_pristine_fetch_order_e2e.rs deleted file mode 100644 index 5c6f5db82..000000000 --- a/crates/socket-patch-cli/tests/vendor/vendor_pristine_fetch_order_e2e.rs +++ /dev/null @@ -1,228 +0,0 @@ -//! `vendor` over several lockfile-only packages: the pristine registry -//! fetches run concurrently, and every package's outcome must still be -//! exactly the one-at-a-time loop's — here with the registry answering the -//! later packages first and a mix of verified, tampered and unverifiable -//! lock entries. Mock registry + a real npm lockfile fixture, driven -//! through the built binary. - -use std::path::PathBuf; -use std::process::Command; -use std::time::Duration; - -use sha2::{Digest, Sha256}; -use wiremock::matchers::{method, path}; -use wiremock::{Mock, MockServer, ResponseTemplate}; - -fn binary() -> PathBuf { - env!("CARGO_BIN_EXE_socket-patch").into() -} - -const BEFORE: &[u8] = b"before\n"; -const AFTER: &[u8] = b"after\n"; - -fn git_sha256(content: &[u8]) -> String { - let header = format!("blob {}\0", content.len()); - let mut hasher = Sha256::new(); - hasher.update(header.as_bytes()); - hasher.update(content); - hex::encode(hasher.finalize()) -} - -fn sri_of(bytes: &[u8]) -> String { - use base64::Engine as _; - use sha2::Sha512; - format!( - "sha512-{}", - base64::engine::general_purpose::STANDARD.encode(Sha512::digest(bytes)) - ) -} - -/// A pristine registry tarball whose index.js carries the BEFORE bytes. -fn pristine_tgz(name: &str) -> Vec { - let mut builder = tar::Builder::new(flate2::write::GzEncoder::new( - Vec::new(), - flate2::Compression::default(), - )); - let pkg_json = format!(r#"{{"name":"{name}","version":"1.0.0"}}"#); - for (path, bytes) in [ - ("package/package.json", pkg_json.as_bytes()), - ("package/index.js", BEFORE), - ] { - let mut header = tar::Header::new_gnu(); - header.set_size(bytes.len() as u64); - header.set_mode(0o644); - header.set_cksum(); - builder.append_data(&mut header, path, bytes).unwrap(); - } - builder.into_inner().unwrap().finish().unwrap() -} - -/// How the lockfile records one package. -enum Lock { - /// The registry tarball's real integrity. - Verified, - /// An integrity the served bytes do not match (tampered). - Tampered, - /// No integrity at all (unverifiable). - Missing, -} - -#[tokio::test] -async fn lockfile_only_packages_fetch_concurrently_with_serial_outcomes() { - let mock = MockServer::start().await; - // Earlier packages answer last. - let packages: [(&str, Lock, u64); 5] = [ - ("pa", Lock::Verified, 500), - ("pb", Lock::Tampered, 400), - ("pc", Lock::Verified, 300), - ("pd", Lock::Missing, 200), - ("pe", Lock::Verified, 0), - ]; - let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - let mut deps = serde_json::Map::new(); - let mut lock_packages = serde_json::Map::new(); - let mut patches = serde_json::Map::new(); - for (i, (name, lock, delay)) in packages.iter().enumerate() { - let tgz = pristine_tgz(name); - let tgz_path = format!("/{name}/-/{name}-1.0.0.tgz"); - let mut entry = serde_json::json!({ - "version": "1.0.0", - "resolved": format!("{}{tgz_path}", mock.uri()), - }); - match lock { - Lock::Verified => entry["integrity"] = sri_of(&tgz).into(), - Lock::Tampered => entry["integrity"] = sri_of(b"other bytes").into(), - Lock::Missing => {} - } - Mock::given(method("GET")) - .and(path(tgz_path)) - .respond_with( - ResponseTemplate::new(200) - .set_body_bytes(tgz) - .set_delay(Duration::from_millis(*delay)), - ) - .mount(&mock) - .await; - deps.insert(name.to_string(), "^1.0.0".into()); - lock_packages.insert(format!("node_modules/{name}"), entry); - patches.insert( - format!("pkg:npm/{name}@1.0.0"), - serde_json::json!({ - "uuid": format!("{i:08x}-1111-4111-8111-111111111111"), - "exportedAt": "2026-01-01T00:00:00Z", - "files": { "package/index.js": { - "beforeHash": git_sha256(BEFORE), - "afterHash": git_sha256(AFTER), - }}, - "vulnerabilities": {}, - "description": "synthetic", - "license": "MIT", - "tier": "free" - }), - ); - } - let manifest = - serde_json::json!({ "name": "order-test", "version": "0.0.0", "dependencies": deps }); - std::fs::write(root.join("package.json"), manifest.to_string()).unwrap(); - lock_packages.insert( - String::new(), - serde_json::json!({ "name": "order-test", "version": "0.0.0", "dependencies": deps }), - ); - let lock = serde_json::json!({ - "name": "order-test", - "version": "0.0.0", - "lockfileVersion": 3, - "requires": true, - "packages": lock_packages, - }); - std::fs::write( - root.join("package-lock.json"), - serde_json::to_vec_pretty(&lock).unwrap(), - ) - .unwrap(); - let socket = root.join(".socket"); - std::fs::create_dir_all(socket.join("blobs")).unwrap(); - std::fs::write( - socket.join("manifest.json"), - serde_json::to_vec_pretty(&serde_json::json!({ "patches": patches })).unwrap(), - ) - .unwrap(); - std::fs::write(socket.join("blobs").join(git_sha256(AFTER)), AFTER).unwrap(); - - let out = Command::new(binary()) - .args(["vendor", "--json", "--vendor-source", "build"]) - .current_dir(root) - .env("SOCKET_TELEMETRY_DISABLED", "1") - .output() - .expect("run vendor"); - let stdout = String::from_utf8_lossy(&out.stdout).into_owned(); - let v: serde_json::Value = serde_json::from_str(stdout.trim()).unwrap_or_else(|e| { - panic!( - "vendor --json must emit JSON: {e}\n{stdout}\n{}", - String::from_utf8_lossy(&out.stderr) - ) - }); - assert_ne!( - out.status.code(), - Some(0), - "the tampered entry fails: {v:#}" - ); - let events = v["events"].as_array().unwrap(); - let has = |purl: &str, action: &str, code: Option<&str>| { - events.iter().any(|e| { - e["purl"] == purl && e["action"] == action && code.is_none_or(|c| e["errorCode"] == c) - }) - }; - for name in ["pa", "pc", "pe"] { - let purl = format!("pkg:npm/{name}@1.0.0"); - assert!(has(&purl, "applied", None), "{purl}: {v:#}"); - assert!( - events - .iter() - .any(|e| e["purl"] == purl.as_str() && e["errorCode"] == "vendor_fetched_missing"), - "{purl}: {v:#}" - ); - assert!(root - .join(format!( - ".socket/vendor/npm/{}/{name}-1.0.0.tgz", - patches[&purl]["uuid"].as_str().unwrap() - )) - .is_file()); - } - assert!( - has("pkg:npm/pb@1.0.0", "failed", Some("vendor_fetch_failed")), - "{v:#}" - ); - assert!( - events - .iter() - .any(|e| e["purl"] == "pkg:npm/pd@1.0.0" - && e["errorCode"] == "vendor_fetch_unverifiable"), - "{v:#}" - ); - assert!( - !has("pkg:npm/pb@1.0.0", "skipped", Some("package_not_installed")), - "no duplicate not-installed skip for a failed fetch: {v:#}" - ); - // Every package's fetch-phase outcome is reported exactly once. - let fetch_phase: Vec<&str> = events - .iter() - .filter(|e| { - matches!( - e["errorCode"].as_str(), - Some("vendor_fetched_missing" | "vendor_fetch_unverifiable") - ) || (e["action"] == "failed" && e["errorCode"] == "vendor_fetch_failed") - }) - .map(|e| e["purl"].as_str().unwrap()) - .collect(); - assert_eq!(fetch_phase.len(), 5, "{v:#}"); - assert!(!root.join("node_modules").exists()); - let requests = mock.received_requests().await.unwrap(); - assert_eq!( - requests.len(), - 4, - "one GET per fetchable entry (the integrity-less one is refused \ - before the network): {requests:?}" - ); -} diff --git a/crates/socket-patch-cli/tests/vendor/vendor_rerun_no_network_e2e.rs b/crates/socket-patch-cli/tests/vendor/vendor_rerun_no_network_e2e.rs index 4c581e701..eee1c1213 100644 --- a/crates/socket-patch-cli/tests/vendor/vendor_rerun_no_network_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor/vendor_rerun_no_network_e2e.rs @@ -90,7 +90,7 @@ fn run_vendor( "vendor", "--json", "--vendor-source", - "build", + "service", "--api-url", dead, "--proxy-url", @@ -117,6 +117,8 @@ fn run_vendor( .env("SOCKET_NPM_REGISTRY", dead) .env("GOFLAGS", "-mod=mod") .envs(env.iter().copied()); + let fixture = crate::prebuilt_common::Server::project_with_env(root, env); + fixture.command(&mut cmd); let out = cmd.output().expect("run socket-patch vendor"); let stdout = String::from_utf8_lossy(&out.stdout).into_owned(); let stderr = String::from_utf8_lossy(&out.stderr).into_owned(); @@ -301,24 +303,28 @@ fn pypi_rerun_over_a_tampered_wheel_is_not_called_in_sync() { .map(|e| e.path()) .find(|p| p.extension().is_some_and(|x| x == "whl")) .expect("the committed wheel"); + let original = std::fs::read(&wheel).unwrap(); + let ledger = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); std::fs::write(&wheel, b"garbage").unwrap(); - - for extra in [&[][..], &["--offline"][..]] { - let (code, v, stderr) = run_vendor(root, &dead, extra, &[]); - assert_eq!(code, 1, "{extra:?}: {v:#}\n{stderr}"); - // Exactly what the eager ladder reports for it (the integrated - // base binary's events on this fixture). - let expected = if extra.is_empty() { - vec![ - ("skipped", "vendor_fetch_unverifiable"), - ("skipped", "package_not_installed"), - ] - } else { - vec![("skipped", "package_not_installed")] - }; - assert_eq!(purl_events(&v, SIX_PURL), expected, "{extra:?}: {v:#}"); - } + let (code, v, stderr) = run_vendor(root, &dead, &["--offline"], &[]); + assert_eq!(code, 1, "{v:#}\n{stderr}"); + assert_eq!( + purl_events(&v, SIX_PURL), + vec![("failed", "vendor_redownload_failed")], + "{v:#}" + ); assert_eq!(std::fs::read(&wheel).unwrap(), b"garbage"); + let (code, v, stderr) = run_vendor(root, &dead, &[], &[]); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + assert!( + purl_events(&v, SIX_PURL).contains(&("rebuilt", "")), + "{v:#}" + ); + assert_eq!(std::fs::read(&wheel).unwrap(), original); + assert_eq!( + std::fs::read(root.join(".socket/vendor/state.json")).unwrap(), + ledger + ); } // ── cargo ─────────────────────────────────────────────────────────────── @@ -517,7 +523,7 @@ fn gem_rerun_without_network_is_in_sync() { /// a downloaded `.gem`): build mode refuses it `gem_spec_missing` BEFORE /// downloading it, instead of downloading it and then refusing. #[tokio::test] -async fn gem_build_mode_refuses_a_lockfile_only_gem_before_downloading_it() { +async fn gem_service_vendors_a_lockfile_only_gem_without_registry_download() { use wiremock::{matchers::method, Mock, MockServer, ResponseTemplate}; let registry = MockServer::start().await; Mock::given(method("GET")) @@ -534,7 +540,7 @@ async fn gem_build_mode_refuses_a_lockfile_only_gem_before_downloading_it() { let (code, v, stderr) = run_vendor(root, &dead_endpoint(), &[], &[]); - assert_eq!(code, 1, "{v:#}\n{stderr}"); + assert_eq!(code, 0, "{v:#}\n{stderr}"); assert!( registry .received_requests() @@ -545,10 +551,10 @@ async fn gem_build_mode_refuses_a_lockfile_only_gem_before_downloading_it() { ); assert_eq!( purl_events(&v, GEM_PURL), - vec![("failed", "gem_spec_missing")], + vec![("applied", ""), ("skipped", "vendor_prebuilt_downloaded")], "{v:#}" ); - assert!(!root.join(".socket/vendor").exists(), "nothing is written"); + assert!(root.join(".socket/vendor/state.json").is_file()); } /// A `.gem`: an uncompressed outer tar holding `metadata.gz` and a @@ -581,7 +587,7 @@ fn make_gem(data_files: &[(&str, &[u8])]) -> Vec { /// `--dry-run` never refused, and still fetches the gem and previews it /// (`vendor_fetched_missing` + `verified`, exit 0) as it always did. #[tokio::test] -async fn gem_build_mode_dry_run_still_fetches_and_previews() { +async fn gem_service_dry_run_previews_without_registry_download() { use wiremock::{matchers::method, Mock, MockServer, ResponseTemplate}; let gem = make_gem(&[(GEM_LIB, GEM_PRISTINE)]); let checksum = hex::encode(Sha256::digest(&gem)); @@ -602,11 +608,7 @@ async fn gem_build_mode_dry_run_still_fetches_and_previews() { let (code, v, stderr) = run_vendor(root, &dead_endpoint(), &["--dry-run"], &[]); assert_eq!(code, 0, "{v:#}\n{stderr}"); - assert_eq!( - purl_events(&v, GEM_PURL), - vec![("skipped", "vendor_fetched_missing"), ("verified", "")], - "{v:#}" - ); + assert_eq!(purl_events(&v, GEM_PURL), vec![("verified", "")], "{v:#}"); assert!( !root.join(".socket/vendor").exists(), "a dry run writes nothing" @@ -668,21 +670,22 @@ fn cargo_rerun_over_a_drifted_copy_still_fetches_and_reports_it() { )); std::fs::write(©_lib, b"tampered\n").unwrap(); - let (code, v, stderr) = run_vendor(&root, &dead, &[], &env); + let ledger = std::fs::read(root.join(".socket/vendor/state.json")).unwrap(); + let (code, v, stderr) = run_vendor(&root, &dead, &["--offline"], &env); assert_eq!(code, 1, "{v:#}\n{stderr}"); assert_eq!( purl_events(&v, PURL), - vec![("failed", "vendor_fetch_failed")], + vec![("failed", "vendor_redownload_failed")], "{v:#}" ); assert_eq!(std::fs::read(©_lib).unwrap(), b"tampered\n"); - - let (code, v, stderr) = run_vendor(&root, &dead, &["--offline"], &env); - assert_eq!(code, 1, "{v:#}\n{stderr}"); + let (code, v, stderr) = run_vendor(&root, &dead, &[], &env); + assert_eq!(code, 0, "{v:#}\n{stderr}"); + assert!(purl_events(&v, PURL).contains(&("rebuilt", "")), "{v:#}"); + assert_eq!(std::fs::read(©_lib).unwrap(), PATCHED); assert_eq!( - purl_events(&v, PURL), - vec![("skipped", "package_not_installed")], - "{v:#}" + std::fs::read(root.join(".socket/vendor/state.json")).unwrap(), + ledger ); } @@ -943,10 +946,7 @@ async fn cargo_service_never_vendors_a_git_or_custom_registry_crate() { assert_eq!(out.status.code(), Some(1), "{source}: {v:#}"); assert_eq!( purl_events(&v, PURL), - vec![ - ("skipped", "vendor_fetch_unverifiable"), - ("skipped", "package_not_installed"), - ], + vec![("failed", "vendor_source_unsupported")], "{source}: {v:#}" ); assert_eq!( @@ -1040,13 +1040,13 @@ async fn cargo_rerun_over_a_drifted_copy_reports_the_deferred_fetch_first() { assert_eq!( purl_events(&v, PURL), vec![ - ("skipped", "vendor_fetched_missing"), - ("applied", ""), - ("skipped", "vendor_artifact_rebuilt"), + ("skipped", "vendor_prebuilt_downloaded"), + ("rebuilt", ""), + ("skipped", "already_vendored"), ], "the fetch is reported first, then the package's own events: {v:#}" ); - assert_eq!(gets().await, before + 1, "one pristine download"); + assert_eq!(gets().await, before, "no pristine download"); assert_eq!(std::fs::read(©_lib).unwrap(), PATCHED, "rebuilt"); } @@ -1136,7 +1136,7 @@ fn a_package_absent_from_the_lock_keeps_the_not_installed_skip() { let (_code, v, stderr) = run_vendor(root, &dead, &[], &[]); assert_eq!( purl_events(&v, NPM_PURL), - vec![("skipped", "package_not_installed")], + vec![("failed", "vendor_lock_entry_not_found")], "{v:#}\n{stderr}" ); @@ -1173,11 +1173,10 @@ fn a_package_absent_from_the_lock_keeps_the_not_installed_skip() { b"after\n", ); let home = cargo_home.to_string_lossy().into_owned(); - let (_code, v, stderr) = - run_vendor(&root, &dead, &[], &[("CARGO_HOME", home.as_str())]); + let (_code, v, stderr) = run_vendor(&root, &dead, &[], &[("CARGO_HOME", home.as_str())]); assert_eq!( purl_events(&v, purl), - vec![("skipped", "package_not_installed")], + vec![("failed", "locked_version_mismatch")], "{purl}: {v:#}\n{stderr}" ); } @@ -1221,7 +1220,10 @@ fn a_stale_prestage_tree_is_swept_by_the_next_wet_run_only() { let (_code, v, stderr) = run_vendor(root, &dead, &["--dry-run"], &[]); for dir in &litter { - assert!(root.join(dir).exists(), "a dry run deletes nothing: {dir}\n{v:#}\n{stderr}"); + assert!( + root.join(dir).exists(), + "a dry run deletes nothing: {dir}\n{v:#}\n{stderr}" + ); } assert!( !v.to_string().contains("socket-prestage"), @@ -1231,7 +1233,10 @@ fn a_stale_prestage_tree_is_swept_by_the_next_wet_run_only() { for extra in [&["--offline"][..], &[][..]] { let (_code, v, stderr) = run_vendor(root, &dead, extra, &[]); for dir in &litter { - assert!(!root.join(dir).exists(), "{extra:?} sweeps {dir}\n{v:#}\n{stderr}"); + assert!( + !root.join(dir).exists(), + "{extra:?} sweeps {dir}\n{v:#}\n{stderr}" + ); } assert!( !root.join(format!(".socket/vendor/composer/{OLD}")).exists(), diff --git a/crates/socket-patch-cli/tests/vendor_crash_safety_e2e.rs b/crates/socket-patch-cli/tests/vendor_crash_safety_e2e.rs index 5bb643a06..be60253a2 100644 --- a/crates/socket-patch-cli/tests/vendor_crash_safety_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor_crash_safety_e2e.rs @@ -11,6 +11,9 @@ //! artifact, rebuilds it and wires the project exactly as an uninterrupted //! run would have. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -96,8 +99,7 @@ fn binary() -> PathBuf { /// at `failpoint`. Returns the exit code and stdout. fn vendor(root: &Path, failpoint: Option<&str>) -> (i32, String) { let mut cmd = Command::new(binary()); - cmd.args(["vendor", "--json", "--offline"]) - .current_dir(root); + cmd.args(["vendor", "--json"]).current_dir(root); for (key, _) in std::env::vars() { if key.starts_with("SOCKET_") && key != "SOCKET_NO_CONFIG" { cmd.env_remove(key); @@ -107,6 +109,8 @@ fn vendor(root: &Path, failpoint: Option<&str>) -> (i32, String) { if let Some(point) = failpoint { cmd.env("SOCKET_PATCH_FAILPOINT", point); } + let fixture = prebuilt_common::Server::project(root); + fixture.command(&mut cmd); let out = cmd.output().expect("run socket-patch vendor"); ( out.status.code().unwrap_or(-1), @@ -295,7 +299,7 @@ fn crash_before_the_barrier_repairs_a_copy_dir_artifact() { let run = |failpoint: Option<&str>| { let mut cmd = Command::new(binary()); - cmd.args(["vendor", "--json", "--offline"]) + cmd.args(["vendor", "--json"]) .current_dir(&root) .env("CARGO_HOME", &cargo_home) .env("SOCKET_TELEMETRY_DISABLED", "1"); @@ -307,6 +311,11 @@ fn crash_before_the_barrier_repairs_a_copy_dir_artifact() { if let Some(point) = failpoint { cmd.env("SOCKET_PATCH_FAILPOINT", point); } + let fixture = prebuilt_common::Server::project_with_env( + &root, + &[("CARGO_HOME", cargo_home.to_str().unwrap())], + ); + fixture.command(&mut cmd); let out = cmd.output().unwrap(); ( out.status.code().unwrap_or(-1), diff --git a/crates/socket-patch-cli/tests/vendor_ecosystem_fixtures/mod.rs b/crates/socket-patch-cli/tests/vendor_ecosystem_fixtures/mod.rs index c47a8cc5f..71bc4a7af 100644 --- a/crates/socket-patch-cli/tests/vendor_ecosystem_fixtures/mod.rs +++ b/crates/socket-patch-cli/tests/vendor_ecosystem_fixtures/mod.rs @@ -9,6 +9,9 @@ //! what makes whole-file ledger snapshots chain. #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -112,7 +115,14 @@ impl Fixture { extra_env: &[(&str, &str)], ) -> (i32, String, String) { let mut cmd = Command::new(bin); - cmd.args(args).current_dir(&self.root); + let args: Vec<_> = args + .iter() + .copied() + .filter(|a| { + args.first() != Some(&"vendor") || args.contains(&"--revert") || *a != "--offline" + }) + .collect(); + cmd.args(&args).current_dir(&self.root); for (key, _) in std::env::vars() { if key.starts_with("SOCKET_") && key != "SOCKET_NO_CONFIG" { cmd.env_remove(key); @@ -133,6 +143,14 @@ impl Fixture { for (k, v) in extra_env { cmd.env(k, v); } + let fixture_env: Vec<_> = self + .env + .iter() + .map(|(k, v)| (k.as_str(), v.as_str())) + .chain(extra_env.iter().copied()) + .collect(); + let server = prebuilt_common::Server::project_with_env(&self.root, &fixture_env); + server.command(&mut cmd); let out = cmd.output().expect("run socket-patch"); ( out.status.code().unwrap_or(-1), @@ -143,7 +161,7 @@ impl Fixture { /// `vendor --json --offline [extra]`. pub fn vendor(&self, extra: &[&str], extra_env: &[(&str, &str)]) -> (i32, String, String) { - let mut args = vec!["vendor", "--json", "--offline"]; + let mut args = vec!["vendor", "--json"]; args.extend_from_slice(extra); self.run(&args, extra_env) } diff --git a/crates/socket-patch-cli/tests/vendor_eject.rs b/crates/socket-patch-cli/tests/vendor_eject.rs index f6752d916..ec7950d93 100644 --- a/crates/socket-patch-cli/tests/vendor_eject.rs +++ b/crates/socket-patch-cli/tests/vendor_eject.rs @@ -13,6 +13,9 @@ //! hosted) all point at a wiremock; `SOCKET_VENDOR_SOURCE=build` keeps the //! vendoring service out of it. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -86,7 +89,7 @@ impl Project { .env("SOCKET_ORG_SLUG", ORG) .env("SOCKET_NPM_REGISTRY", &uri) .env("SOCKET_PATCH_SERVER_URL", &uri) - .env("SOCKET_VENDOR_SOURCE", "build"); + .env("SOCKET_VENDOR_SOURCE", "service"); let out = cmd.output().expect("spawn socket-patch"); ( out.status.code().unwrap_or(-1), @@ -174,24 +177,26 @@ async fn mock_registry(p: &Project) { async fn mock_view(p: &Project) { let before = compute_git_sha256_from_bytes(ORIG_INDEX); let after = compute_git_sha256_from_bytes(PATCHED_INDEX); + let archive_view = json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "package/index.js": { + "beforeHash": before, + "afterHash": after, + "blobContent": base64::engine::general_purpose::STANDARD.encode(PATCHED_INDEX) + } + }, + "vulnerabilities": {}, + "description": "eject fixture", + "license": "MIT", + "tier": "free" + }); + prebuilt_common::mount_view(&p.server, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": UUID, - "purl": PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": { - "beforeHash": before, - "afterHash": after, - "blobContent": base64::engine::general_purpose::STANDARD.encode(PATCHED_INDEX) - } - }, - "vulnerabilities": {}, - "description": "eject fixture", - "license": "MIT", - "tier": "free" - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(&p.server) .await; } @@ -225,7 +230,8 @@ async fn eject_vendors_hosted_pins_and_revert_returns_to_upstream() { // The eject restores upstream as its own planned step before vendoring, // so the per-purl takeover warning never fires. assert!( - !env.to_string().contains("vendor_takeover_reverted_redirect"), + !env.to_string() + .contains("vendor_takeover_reverted_redirect"), "{env:#}" ); assert!( diff --git a/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs b/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs index 0d64308e2..5494224b1 100644 --- a/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs +++ b/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs @@ -21,6 +21,9 @@ //! registry (`SOCKET_NPM_REGISTRY`) and the patch-server origin //! (`SOCKET_PATCH_SERVER_URL`) all point at a wiremock. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use std::process::Command; @@ -97,7 +100,7 @@ impl Project { .env("SOCKET_ORG_SLUG", ORG) .env("SOCKET_NPM_REGISTRY", &uri) .env("SOCKET_PATCH_SERVER_URL", &uri) - .env("SOCKET_VENDOR_SOURCE", "build"); + .env("SOCKET_VENDOR_SOURCE", "service"); let out = cmd.output().expect("spawn socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout); let env = serde_json::from_str(&stdout).unwrap_or_else(|e| { @@ -180,6 +183,7 @@ async fn hosted_project(writer: &str) -> Project { let mut view = record(); view["files"]["package/index.js"]["blobContent"] = json!(base64::engine::general_purpose::STANDARD.encode(PATCHED_INDEX)); + prebuilt_common::mount_view(&project.server, &view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) .respond_with(ResponseTemplate::new(200).set_body_json(view)) @@ -337,12 +341,17 @@ async fn takeover_refuses_a_workspace_normalized_hosted_bun_lockb() { }) .unwrap_or_else(|| panic!("expected redirect_revert_failed: {env:#}")); assert!( - refused["error"].as_str().is_some_and(|e| e - .contains("workspace dependency behaviors") - && e.contains("git checkout -- bun.lockb")), + refused["error"] + .as_str() + .is_some_and(|e| e.contains("workspace dependency behaviors") + && e.contains("git checkout -- bun.lockb")), "{env:#}" ); - assert_eq!(p.lock(), hosted, "{extra:?}: a refused vendor writes nothing"); + assert_eq!( + p.lock(), + hosted, + "{extra:?}: a refused vendor writes nothing" + ); assert!(!p.root().join(".socket/vendor").exists()); } } diff --git a/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs b/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs index 792854c44..3f5d3ae49 100644 --- a/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs +++ b/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs @@ -7,6 +7,9 @@ //! from the registry (verified against that checksum) instead of requiring //! an installed tree. Every registry and API endpoint is a wiremock. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::io::Write as _; use std::path::Path; use std::process::Command; @@ -38,25 +41,34 @@ fn tgz(prefix: &str, files: &[(&str, &[u8])]) -> Vec { enc.finish().unwrap() } -async fn mock_view(server: &MockServer, uuid: &str, purl: &str, file: &str, orig: &[u8], patched: &[u8]) { +async fn mock_view( + server: &MockServer, + uuid: &str, + purl: &str, + file: &str, + orig: &[u8], + patched: &[u8], +) { + let archive_view = json!({ + "uuid": uuid, + "purl": purl, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + file: { + "beforeHash": compute_git_sha256_from_bytes(orig), + "afterHash": compute_git_sha256_from_bytes(patched), + "blobContent": base64::engine::general_purpose::STANDARD.encode(patched) + } + }, + "vulnerabilities": {}, + "description": "eject fixture", + "license": "MIT", + "tier": "free" + }); + prebuilt_common::mount_view(server, &archive_view, None).await; Mock::given(method("GET")) .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": uuid, - "purl": purl, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - file: { - "beforeHash": compute_git_sha256_from_bytes(orig), - "afterHash": compute_git_sha256_from_bytes(patched), - "blobContent": base64::engine::general_purpose::STANDARD.encode(patched) - } - }, - "vulnerabilities": {}, - "description": "eject fixture", - "license": "MIT", - "tier": "free" - }))) + .respond_with(ResponseTemplate::new(200).set_body_json(archive_view)) .mount(server) .await; } @@ -96,7 +108,7 @@ fn run_json_with( .env("SOCKET_CRATES_INDEX", format!("{uri}/index")) .env("SOCKET_CRATES_REGISTRY", format!("{uri}/crates")) .env("SOCKET_PATCH_SERVER_URL", &uri) - .env("SOCKET_VENDOR_SOURCE", "build") + .env("SOCKET_VENDOR_SOURCE", "service") .env("CARGO_HOME", &cargo_home) .envs(extra.iter().map(|(k, v)| (*k, v.as_str()))) .output() @@ -112,9 +124,11 @@ fn run_json_with( } fn applied(env: &Value, purl: &str) -> bool { - env["events"] - .as_array() - .is_some_and(|events| events.iter().any(|e| e["action"] == "applied" && e["purl"] == purl)) + env["events"].as_array().is_some_and(|events| { + events + .iter() + .any(|e| e["action"] == "applied" && e["purl"] == purl) + }) } /// npm: a hosted package-lock.json with NO `node_modules`. The pristine @@ -190,7 +204,10 @@ async fn npm_eject_needs_no_installed_tree() { let artifact = root.join(format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz")); assert!(artifact.is_file(), "the artifact lands in .socket/vendor/"); let lock = std::fs::read_to_string(root.join("package-lock.json")).unwrap(); - assert!(lock.contains(&format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz")), "{lock}"); + assert!( + lock.contains(&format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0.tgz")), + "{lock}" + ); assert!(!lock.contains(&hosted), "no hosted residue: {lock}"); } @@ -210,7 +227,10 @@ async fn cargo_eject_needs_no_cargo_home() { let root = tmp.path().join("proj"); std::fs::create_dir_all(&root).unwrap(); - let krate = tgz("serde-1.0.190", &[("Cargo.toml", TOML), ("src/lib.rs", ORIG)]); + let krate = tgz( + "serde-1.0.190", + &[("Cargo.toml", TOML), ("src/lib.rs", ORIG)], + ); let checksum = hex::encode(Sha256::digest(&krate)); Mock::given(method("GET")) .and(path("/index/se/rd/serde")) @@ -236,7 +256,8 @@ async fn cargo_eject_needs_no_cargo_home() { ), ) .unwrap(); - let index = format!("sparse+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"); + let index = + format!("sparse+https://patch.socket.dev/patch-registry/cargo/{TOKEN}/{UUID}/index/"); std::fs::write( root.join("Cargo.lock"), format!( @@ -255,8 +276,14 @@ async fn cargo_eject_needs_no_cargo_home() { assert!(applied(&env, PURL), "{env:#}"); let toml = std::fs::read_to_string(root.join("Cargo.toml")).unwrap(); let lock = std::fs::read_to_string(root.join("Cargo.lock")).unwrap(); - assert!(!toml.contains("socket-patch-"), "hosted registry key removed: {toml}"); - assert!(!lock.contains("patch.socket.dev"), "no hosted residue: {lock}"); + assert!( + !toml.contains("socket-patch-"), + "hosted registry key removed: {toml}" + ); + assert!( + !lock.contains("patch.socket.dev"), + "no hosted residue: {lock}" + ); assert!( std::fs::read_dir(root.join(".socket/vendor/cargo")) .map(|mut d| d.next().is_some()) @@ -343,14 +370,19 @@ async fn pypi_eject_needs_no_virtualenv() { ) .unwrap(); - let (code, env) = run_json_with(&root, &server, &["vendor"], &[( - "SOCKET_PYPI_JSON_API", - format!("{}/pypi", server.uri()), - )]); + let (code, env) = run_json_with( + &root, + &server, + &["vendor"], + &[("SOCKET_PYPI_JSON_API", format!("{}/pypi", server.uri()))], + ); assert_eq!(code, 0, "a fresh hosted pypi checkout ejects: {env:#}"); assert!(applied(&env, PURL), "{env:#}"); let reqs = std::fs::read_to_string(root.join("requirements.txt")).unwrap(); - assert!(!reqs.contains("patch.socket.dev"), "no hosted residue: {reqs}"); + assert!( + !reqs.contains("patch.socket.dev"), + "no hosted residue: {reqs}" + ); assert!( reqs.contains(&format!(".socket/vendor/pypi/{UUID}/")), "the requirement is wired to the vendored wheel: {reqs}" diff --git a/crates/socket-patch-cli/tests/vendor_group_commit_e2e.rs b/crates/socket-patch-cli/tests/vendor_group_commit_e2e.rs index d2238428f..728b0e70d 100644 --- a/crates/socket-patch-cli/tests/vendor_group_commit_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor_group_commit_e2e.rs @@ -130,30 +130,21 @@ fn group_commit_ends_where_per_package_commits_end_for_every_ecosystem() { } } -/// One package fails (its patch target is missing from the installed copy, -/// which fails closed without `--force`), the other succeeds: the success is -/// committed, the failure leaves nothing, and the tree is the per-package -/// commits' tree. +/// A service artifact with the wrong patched bytes fails only its package. #[test] fn a_partial_failure_commits_the_packages_that_succeeded() { - for (eco, target) in [ - ("npm", "proj:node_modules/beta/index.js"), - ( - "cargo", - "store:cargo-home/registry/src/index.crates.io-6f17d22bba15001f/beta-1.0.0/src/lib.rs", - ), - ("gem", "proj:vendor/bundle/gems/beta-1.0.0/lib/beta.rb"), - ("golang", "store:modcache/github.com/fx/beta@v1.0.0/beta.go"), - ] { + for eco in ["npm", "cargo", "gem", "golang"] { let grouped = Fixture::new(eco); let oracle = Fixture::new(eco); for f in [&grouped, &oracle] { - let path = match target.split_once(':') { - Some(("proj", rel)) => f.root.join(rel), - Some((_, rel)) => f.store.join(rel), - None => unreachable!(), - }; - std::fs::remove_file(path).unwrap(); + let hash = socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes( + &f.patches[1].after, + ); + std::fs::write( + f.root.join(".socket/blobs").join(hash), + b"wrong service bytes", + ) + .unwrap(); } let (code, stdout, _) = grouped.vendor(&[], &[]); let (oracle_code, oracle_stdout, _) = oracle.vendor(&[], &[OFF]); diff --git a/crates/socket-patch-cli/tests/vendor_jvm_cli.rs b/crates/socket-patch-cli/tests/vendor_jvm_cli.rs index 06a4624f5..08ad1031c 100644 --- a/crates/socket-patch-cli/tests/vendor_jvm_cli.rs +++ b/crates/socket-patch-cli/tests/vendor_jvm_cli.rs @@ -8,6 +8,9 @@ //! update re-wires and reverts to pristine, a re-run needs no jar source, //! and a tampered ledger or an escaping symlink is refused. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::BTreeMap; use std::io::Write as _; use std::path::Path; @@ -155,9 +158,14 @@ fn socket(root: &Path, args: &[&str]) -> (Option, serde_json::Value) { } } let proj = root.join("proj"); + let _fixture = prebuilt_common::prepare_command( + &mut cmd, + &proj, + args, + &[("MAVEN_REPO_LOCAL", root.join("m2").to_str().unwrap())], + ); let out = cmd - .args(args) - .args(["--json", "--offline", "--cwd", proj.to_str().unwrap()]) + .args(["--json", "--cwd", proj.to_str().unwrap()]) .env("SOCKET_TELEMETRY_DISABLED", "1") .env("SOCKET_NO_CONFIG", "1") .env("MAVEN_REPO_LOCAL", root.join("m2")) diff --git a/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs b/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs index 406a5b33f..819e7c1c1 100644 --- a/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor_ledger_schema_e2e.rs @@ -213,36 +213,68 @@ fn new_ledgers_compact_whole_file_snapshots_and_revert() { /// fixtures), and its ledger — whatever its on-disk version — loads to the /// same entries the base's version-1 ledger loads to. #[tokio::test] -async fn this_binary_wires_what_the_base_binary_wired() { +async fn server_artifacts_preserve_legacy_wiring_shape_and_originals() { for eco in fx::ALL { let base_wired = read_tree(&fixtures_dir().join(eco).join("wired")); let f = Fixture::new(eco); - let pristine = tree(&f.root); let (code, stdout, stderr) = f.vendor(&[], &[]); assert_eq!(code, 0, "{eco}: {stdout}\n{stderr}"); - let wired = wiring_delta(&pristine, &tree(&f.root)); - let strip = |t: &[(String, Vec)]| -> Vec<(String, Vec)> { - t.iter() - .filter(|(rel, _)| rel != ".socket/vendor/state.json") - .cloned() - .collect() - }; - assert_eq!( - strip(&wired), - strip(&base_wired), - "{eco}: the same wiring files, byte for byte" - ); let base_dir = tempfile::tempdir().unwrap(); write_tree(base_dir.path(), &base_wired); - let base_state = socket_patch_core::vendor::load_state(base_dir.path()) + let base = socket_patch_core::vendor::load_state(base_dir.path()) .await .unwrap(); let state = socket_patch_core::vendor::load_state(&f.root) .await .unwrap(); - assert_eq!( - state.entries, base_state.entries, - "{eco}: the same ledger entries" - ); + assert_eq!(state.entries.len(), base.entries.len(), "{eco}"); + for (purl, old) in &base.entries { + let new = &state.entries[purl]; + assert_eq!( + ( + &new.ecosystem, + &new.base_purl, + &new.uuid, + &new.artifact.path + ), + ( + &old.ecosystem, + &old.base_purl, + &old.uuid, + &old.artifact.path + ), + "{eco}" + ); + let shape = |entry: &socket_patch_core::vendor::state::VendorEntry| { + entry + .wiring + .iter() + .map(|w| { + let original = w.original.clone().map(|value| { + if let serde_json::Value::String(mut text) = value { + for (purl, current) in &state.entries { + let legacy = &base.entries[purl]; + if !current.artifact.sha256.is_empty() { + text = text.replace( + ¤t.artifact.sha256, + &legacy.artifact.sha256, + ); + } + } + serde_json::Value::String(text) + } else { + value + } + }); + (w.file.clone(), w.kind.clone(), original) + }) + .collect::>() + }; + assert_eq!( + shape(new), + shape(old), + "{eco}: rollback originals survive server repacking" + ); + } } } diff --git a/crates/socket-patch-cli/tests/vendor_partial_staging_e2e.rs b/crates/socket-patch-cli/tests/vendor_partial_staging_e2e.rs index b6a1b5a79..f3bee2737 100644 --- a/crates/socket-patch-cli/tests/vendor_partial_staging_e2e.rs +++ b/crates/socket-patch-cli/tests/vendor_partial_staging_e2e.rs @@ -1,30 +1,6 @@ -//! Two rules about a patch view that does not serve every file's bytes. -//! -//! The view serves `blobContent` only for files the patch actually CHANGES: -//! a file whose `beforeHash` equals its `afterHash` comes back with hashes -//! and no content (live example: `pkg:npm/tar-fs@2.1.1`, patch -//! `8ff3e0c7-6855-4224-924b-3e1151744ed4`, seven zero-delta fixture files -//! plus one changed `package/index.js`). -//! -//! 1. A zero-delta file needs NO content — the pristine copy already holds -//! the patched bytes — so such a view stages and the package vendors -//! (`a_view_whose_only_contentless_files_are_zero_delta_vendors`). -//! 2. A file the patch CHANGES that is served without content is genuinely -//! unsatisfiable. That is a broken PACKAGE, not a broken run: it gets -//! its own `failed` event and the rest of the run carries on — it must -//! not make the WHOLE run bail `no_local_source` with every OTHER -//! package left unvendored without a word. -//! -//! A package whose patch content cannot be obtained is an unsatisfiable -//! package like any other (`vendor_fetch_failed`, `redirect_revert_failed`, -//! the Bun refusals …). The pre-event `no_local_source` bail stays for the -//! case it was written for — NOTHING in the manifest can be staged, so -//! there are no events to report. -//! -//! Hermetic: the API is a `wiremock` mock, `--vendor-source build` keeps the -//! vendoring service out of the run, and every package is installed on disk -//! so no registry fetch happens. - +//! Vendoring consumes immutable service artifacts without downloading patch blobs. +#[path = "prebuilt_common/mod.rs"] +mod prebuilt_common; use std::path::Path; use std::process::Command; @@ -35,8 +11,6 @@ use wiremock::{Mock, MockServer, ResponseTemplate}; const ORG: &str = "test-org"; -/// The satisfiable package: its after-blob is staged under `.socket/blobs`, -/// so staging never fetches its view. const GOOD_PURL: &str = "pkg:npm/left-pad@1.3.0"; const GOOD_UUID: &str = "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"; const GOOD_ORIG: &[u8] = b"module.exports = () => 'orig';\n"; @@ -90,8 +64,6 @@ fn bad_files() -> Value { }) } -/// A two-package npm project: both installed, both in the v3 lock, both in -/// the manifest. Only the good package's after-blob is staged on disk. fn fixture(root: &Path) { for (name, version, index, extra) in [ ("left-pad", "1.3.0", GOOD_ORIG, None), @@ -166,55 +138,6 @@ fn fixture(root: &Path) { .unwrap(); } -/// Mount the bad package's view. `changed_content` is the `blobContent` -/// the CHANGED file is served with; `None` makes the view genuinely -/// unsatisfiable (the patch needs those bytes and nothing can supply -/// them). The zero-delta file always comes back with hashes and no -/// content — that is how the API serves a file a patch does not change. -async fn mount_view(server: &MockServer, changed_content: Option<&[u8]>) { - use base64::Engine; - let mut changed = json!({ - "beforeHash": git_hash(BAD_ORIG), - "afterHash": git_hash(BAD_PATCHED), - }); - if let Some(bytes) = changed_content { - changed["blobContent"] = json!(base64::engine::general_purpose::STANDARD.encode(bytes)); - } - Mock::given(method("GET")) - .and(wm_path(format!("/v0/orgs/{ORG}/patches/view/{BAD_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": BAD_UUID, - "purl": BAD_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { - "package/index.js": changed, - "package/test/fixtures/d/file1": { - "beforeHash": git_hash(BAD_FIXTURE), - "afterHash": git_hash(BAD_FIXTURE), - } - }, - "vulnerabilities": {}, - "description": "d", - "license": "MIT", - "tier": "free", - }))) - .mount(server) - .await; -} - -/// A view the run genuinely cannot satisfy: the file the patch CHANGES is -/// served with no `blobContent`, so the patched bytes exist nowhere. -async fn mount_contentless_view(server: &MockServer) { - mount_view(server, None).await; -} - -/// The live JS-7 view: the changed file carries `blobContent`, and only -/// the zero-delta file comes back contentless — which needs no content. -async fn mount_zero_delta_view(server: &MockServer) { - mount_view(server, Some(BAD_PATCHED)).await; -} - -/// The `path -> bytes` map of a gzipped tarball's regular members. fn tgz_members(tgz: &Path) -> std::collections::BTreeMap> { let file = std::fs::File::open(tgz).unwrap_or_else(|e| panic!("open {}: {e}", tgz.display())); let mut archive = tar::Archive::new(flate2::read::GzDecoder::new(file)); @@ -233,13 +156,6 @@ fn tgz_members(tgz: &Path) -> std::collections::BTreeMap> { out } -/// `vendor --json --vendor-source build` against the mock API, with every -/// ambient `SOCKET_*` var scrubbed from the child. -fn vendor_cli(root: &Path, api_url: &str) -> (i32, Value, String) { - vendor_cli_with_source(root, api_url, "build") -} - -/// [`vendor_cli`] under an explicit `--vendor-source`. fn vendor_cli_with_source(root: &Path, api_url: &str, source: &str) -> (i32, Value, String) { let mut cmd = Command::new(env!("CARGO_BIN_EXE_socket-patch")); cmd.args([ @@ -281,289 +197,6 @@ fn event_for<'a>(env: &'a Value, purl: &str) -> &'a Value { .unwrap_or_else(|| panic!("expected an event for {purl} in:\n{env:#}")) } -#[tokio::test] -async fn contentless_patch_view_fails_only_its_own_package() { - let server = MockServer::start().await; - mount_contentless_view(&server).await; - let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - fixture(root); - - let (code, env, stderr) = vendor_cli(root, &server.uri()); - - assert_eq!( - code, 1, - "an unstageable package still fails the run: {env:#}\nstderr:\n{stderr}" - ); - assert_eq!( - env["status"], "partialFailure", - "one bad package is a partial failure, not a pre-event abort: {env:#}" - ); - assert!( - env["error"].is_null(), - "no run-level error payload: the failure is per-package: {env:#}" - ); - - let bad = event_for(&env, BAD_PURL); - assert_eq!(bad["action"], "failed", "{env:#}"); - assert_eq!( - bad["errorCode"], "no_local_source", - "the per-package failure keeps the staging code: {env:#}" - ); - // The per-package slot is the ONE machine-readable explanation a - // `--json` consumer gets (every human channel in the stager is gated - // on `!--json`), so it must carry the REAL reason. This run is neither - // offline nor a download failure: the view was served, 200, with a - // file it had no content for. - assert_eq!( - bad["error"].as_str(), - Some("the patch view served no blob content for package/index.js"), - "the failure names the file that was served without content: {env:#}" - ); - - let good = event_for(&env, GOOD_PURL); - assert_eq!( - good["action"], "applied", - "the rest of the run must continue: {env:#}" - ); - assert!( - root.join(format!(".socket/vendor/npm/{GOOD_UUID}/left-pad-1.3.0.tgz")) - .is_file(), - "the satisfiable package must still be vendored: {env:#}" - ); - // The unstageable package is left completely alone. - assert!( - !root.join(format!(".socket/vendor/npm/{BAD_UUID}")).exists(), - "nothing is written for the unstageable package: {env:#}" - ); - let lock: Value = - serde_json::from_slice(&std::fs::read(root.join("package-lock.json")).unwrap()).unwrap(); - assert_eq!( - lock["packages"]["node_modules/tar-fs"]["resolved"], - "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.1.tgz", - "the unstageable package's lock entry stays registry-resolved: {env:#}" - ); -} - -/// The pre-event bail survives for the case it was written for: when NO -/// patch in the manifest can be staged there are no per-package events to -/// report, so the run keeps its top-level `no_local_source` error. -#[tokio::test] -async fn every_patch_unstageable_keeps_the_run_level_error() { - let server = MockServer::start().await; - mount_contentless_view(&server).await; - let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - fixture(root); - // Drop the good package's staged blob: now both patches need a view, - // and neither view is complete (left-pad's 404s). - std::fs::remove_file(root.join(".socket/blobs").join(git_hash(GOOD_PATCHED))).unwrap(); - - let (code, env, stderr) = vendor_cli(root, &server.uri()); - - assert_eq!(code, 1, "{env:#}\nstderr:\n{stderr}"); - assert_eq!(env["status"], "error", "{env:#}"); - assert_eq!(env["error"]["code"], "no_local_source", "{env:#}"); - assert!( - events(&env).is_empty(), - "a pre-event abort reports no events: {env:#}" - ); - assert!( - !root.join(".socket/vendor").exists(), - "an aborted run vendors nothing: {env:#}" - ); -} - -/// The JS-7 package itself must VENDOR, not merely fail politely. -/// -/// `pkg:npm/tar-fs@2.1.1` patch `8ff3e0c7-…` changes one file and carries -/// seven zero-delta fixture files (`beforeHash == afterHash`). The view -/// serves `blobContent` only for the file it CHANGES, so those seven come -/// back contentless — and a zero-delta file needs no content: the pristine -/// copy already holds the patched bytes, which is exactly what -/// `verify_file_patch` answers `AlreadyPatched` for. Requiring the -/// after-blob for every file made this patch permanently unvendorable. -#[tokio::test] -async fn a_view_whose_only_contentless_files_are_zero_delta_vendors() { - let server = MockServer::start().await; - mount_zero_delta_view(&server).await; - let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - fixture(root); - - let (code, env, stderr) = vendor_cli(root, &server.uri()); - - assert_eq!( - code, 0, - "nothing in this patch needs the unserved bytes: {env:#}\nstderr:\n{stderr}" - ); - assert_eq!(env["status"], "success", "{env:#}"); - assert_eq!(event_for(&env, BAD_PURL)["action"], "applied", "{env:#}"); - assert_eq!(event_for(&env, GOOD_PURL)["action"], "applied", "{env:#}"); - - // The vendored tarball carries BOTH files — the changed one at its - // patched bytes, the zero-delta one at the bytes it always had. - let tgz = root.join(format!(".socket/vendor/npm/{BAD_UUID}/tar-fs-2.1.1.tgz")); - let members = tgz_members(&tgz); - assert_eq!( - members.get("package/index.js").map(Vec::as_slice), - Some(BAD_PATCHED), - "the changed file is the patched content: {members:?}" - ); - assert_eq!( - members - .get("package/test/fixtures/d/file1") - .map(Vec::as_slice), - Some(BAD_FIXTURE), - "the zero-delta file is vendored from the pristine copy: {members:?}" - ); -} - -/// A package staging drops must stay out of the vendoring service's -/// download plan as well as the loop. -/// -/// With the service enabled the vendor loop fetches its service downloads -/// ahead of itself, from an EXACT plan built over the records it is handed -/// — a download grant can start a server-side build and counts against -/// quota, so a package the run never vendors must never be granted. The -/// staging drop hands the engine only the stageable records, so the -/// unstageable package reaches neither the plan nor the loop's own call. -/// Two stageable packages keep the plan attached (one download has nothing -/// to overlap), and the service answering `not_found` sends both to the -/// local build, so the run's outcome is the build-mode one. -#[tokio::test] -async fn a_dropped_package_is_never_granted_a_service_download() { - const THIRD_PURL: &str = "pkg:npm/is-odd@3.0.1"; - const THIRD_UUID: &str = "3c1d5e7f-2a4b-4c6d-8e0f-1a2b3c4d5e6f"; - const THIRD_ORIG: &[u8] = b"module.exports = n => n % 2 === 1;\n"; - const THIRD_PATCHED: &[u8] = b"module.exports = n => Math.abs(n % 2) === 1;\n"; - - /// Answer every grant request `not_found`, whichever uuids it names. - struct NotFound; - impl wiremock::Respond for NotFound { - fn respond(&self, req: &wiremock::Request) -> ResponseTemplate { - let body: Value = serde_json::from_slice(&req.body).unwrap_or(Value::Null); - let results: serde_json::Map = body["uuids"] - .as_array() - .into_iter() - .flatten() - .filter_map(Value::as_str) - .map(|u| { - ( - u.to_string(), - json!({ "status": "not_found", "url": null, "artifacts": [] }), - ) - }) - .collect(); - ResponseTemplate::new(200).set_body_json(json!({ "results": results })) - } - } - - let server = MockServer::start().await; - mount_contentless_view(&server).await; - Mock::given(method("POST")) - .and(wm_path(format!("/v0/orgs/{ORG}/patches/package"))) - .respond_with(NotFound) - .mount(&server) - .await; - let tmp = tempfile::tempdir().unwrap(); - let root = tmp.path(); - fixture(root); - // A second stageable package: installed, locked, blob staged. - let pkg = root.join("node_modules/is-odd"); - std::fs::create_dir_all(&pkg).unwrap(); - std::fs::write( - pkg.join("package.json"), - br#"{"name":"is-odd","version":"3.0.1"}"#, - ) - .unwrap(); - std::fs::write(pkg.join("index.js"), THIRD_ORIG).unwrap(); - let mut lock: Value = - serde_json::from_slice(&std::fs::read(root.join("package-lock.json")).unwrap()).unwrap(); - lock["packages"][""]["dependencies"]["is-odd"] = json!("^3.0.1"); - lock["packages"]["node_modules/is-odd"] = json!({ - "version": "3.0.1", - "resolved": "https://registry.npmjs.org/is-odd/-/is-odd-3.0.1.tgz", - "integrity": "sha512-orig3==" - }); - std::fs::write( - root.join("package-lock.json"), - serde_json::to_vec_pretty(&lock).unwrap(), - ) - .unwrap(); - let manifest_path = root.join(".socket/manifest.json"); - let mut manifest: Value = - serde_json::from_slice(&std::fs::read(&manifest_path).unwrap()).unwrap(); - manifest["patches"][THIRD_PURL] = patch_record( - THIRD_UUID, - json!({ "package/index.js": { - "beforeHash": git_hash(THIRD_ORIG), - "afterHash": git_hash(THIRD_PATCHED), - }}), - ); - std::fs::write( - &manifest_path, - serde_json::to_vec_pretty(&manifest).unwrap(), - ) - .unwrap(); - std::fs::write( - root.join(".socket/blobs").join(git_hash(THIRD_PATCHED)), - THIRD_PATCHED, - ) - .unwrap(); - - let (code, env, stderr) = vendor_cli_with_source(root, &server.uri(), "auto"); - - assert_eq!(code, 1, "{env:#}\nstderr:\n{stderr}"); - assert_eq!(env["status"], "partialFailure", "{env:#}"); - assert_eq!( - event_for(&env, BAD_PURL)["errorCode"], - "no_local_source", - "{env:#}" - ); - assert_eq!(event_for(&env, GOOD_PURL)["action"], "applied", "{env:#}"); - assert_eq!(event_for(&env, THIRD_PURL)["action"], "applied", "{env:#}"); - - let granted: Vec = server - .received_requests() - .await - .unwrap_or_default() - .iter() - .filter(|r| r.method.as_str() == "POST" && r.url.path().ends_with("/patches/package")) - .flat_map(|r| { - let body: Value = serde_json::from_slice(&r.body).unwrap_or(Value::Null); - body["uuids"] - .as_array() - .into_iter() - .flatten() - .filter_map(|u| u.as_str().map(str::to_string)) - .collect::>() - }) - .collect(); - assert!( - granted.iter().any(|u| u == GOOD_UUID) && granted.iter().any(|u| u == THIRD_UUID), - "the stageable packages ask the service (the test is not vacuous): {granted:?}" - ); - assert!( - !granted.iter().any(|u| u == BAD_UUID), - "the dropped package must never be granted a download: {granted:?}\n{env:#}" - ); -} - -// ── the other caller of the per-package drop ──────────────────────────── -// -// `drop_unstageable` is wired into `vendor` (above) and `scan --mode -// vendored` / `get --mode vendored` (`scan::vendor_flow`); `repair` drops -// unstageable candidates with its own partition in `repair_vendor`. The vendored -// SCAN fold — `Ok(staging_errors || engine_errors)` — has its own error -// path, and every existing suite that touches it mounts a single-patch -// manifest, so it only ever exercised the preserved whole-run bail. - -const GOOD_ENCODED: &str = "pkg%3Anpm%2Fleft-pad%401.3.0"; -const BAD_ENCODED: &str = "pkg%3Anpm%2Ftar-fs%402.1.1"; - -/// Discovery for both packages: the batch endpoint plus the per-package -/// search each purl falls back to. async fn mount_discovery(server: &MockServer) { Mock::given(method("POST")) .and(wm_path(format!("/v0/orgs/{ORG}/patches/batch"))) @@ -583,8 +216,8 @@ async fn mount_discovery(server: &MockServer) { .mount(server) .await; for (encoded, uuid, purl) in [ - (GOOD_ENCODED, GOOD_UUID, GOOD_PURL), - (BAD_ENCODED, BAD_UUID, BAD_PURL), + ("pkg%3Anpm%2Fleft-pad%401.3.0", GOOD_UUID, GOOD_PURL), + ("pkg%3Anpm%2Ftar-fs%402.1.1", BAD_UUID, BAD_PURL), ] { Mock::given(method("GET")) .and(wm_path(format!( @@ -607,36 +240,6 @@ async fn mount_discovery(server: &MockServer) { } } -/// The good package's view, served complete. -async fn mount_good_view(server: &MockServer) { - use base64::Engine; - Mock::given(method("GET")) - .and(wm_path(format!("/v0/orgs/{ORG}/patches/view/{GOOD_UUID}"))) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({ - "uuid": GOOD_UUID, - "purl": GOOD_PURL, - "publishedAt": "2026-01-01T00:00:00Z", - "files": { "package/index.js": { - "beforeHash": git_hash(GOOD_ORIG), - "afterHash": git_hash(GOOD_PATCHED), - "blobContent": base64::engine::general_purpose::STANDARD.encode(GOOD_PATCHED), - }}, - "vulnerabilities": {}, - "description": "d", - "license": "MIT", - "tier": "free", - }))) - .mount(server) - .await; -} - -/// The project WITHOUT `.socket/`: vendored mode is manifest-free, so the -/// records come from discovery and the blobs from the download phase. -fn scan_fixture(root: &Path) { - fixture(root); - std::fs::remove_dir_all(root.join(".socket")).unwrap(); -} - fn scan_vendored_cli(root: &Path, api_url: &str) -> (i32, Value, String) { let mut cmd = Command::new(env!("CARGO_BIN_EXE_socket-patch")); cmd.args([ @@ -646,7 +249,7 @@ fn scan_vendored_cli(root: &Path, api_url: &str) -> (i32, Value, String) { "vendored", "--yes", "--vendor-source", - "build", + "service", "--api-url", api_url, "--api-token", @@ -670,52 +273,147 @@ fn scan_vendored_cli(root: &Path, api_url: &str) -> (i32, Value, String) { (out.status.code().unwrap_or(-1), env, stderr) } -/// `scan --mode vendored` over a mixed selection: one package the view -/// cannot supply and one it can. The unsatisfiable package is reported -/// once, per package, and the other still vendors — the vendored scan's -/// own fold, not `vendor`'s. -#[tokio::test] -async fn scan_vendored_reports_an_unstageable_package_and_vendors_the_rest() { +async fn publish(server: &MockServer, root: &Path, bad: Option<&[u8]>) { + if let Some(bytes) = bad { + std::fs::write( + root.join(".socket/blobs").join(git_hash(BAD_PATCHED)), + bytes, + ) + .unwrap(); + std::fs::write( + root.join(".socket/blobs").join(git_hash(BAD_FIXTURE)), + BAD_FIXTURE, + ) + .unwrap(); + } + prebuilt_common::mount_project(server, root).await; + std::fs::remove_dir_all(root.join(".socket/blobs")).unwrap(); +} + +#[tokio::test(flavor = "multi_thread")] +async fn vendor_downloads_without_local_blobs_or_installed_packages() { let server = MockServer::start().await; - mount_discovery(&server).await; - mount_good_view(&server).await; - mount_contentless_view(&server).await; let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - scan_fixture(root); - - let (code, env, stderr) = scan_vendored_cli(root, &server.uri()); + fixture(root); + publish(&server, root, Some(BAD_PATCHED)).await; + std::fs::remove_dir_all(root.join("node_modules")).unwrap(); + let (code, env, stderr) = vendor_cli_with_source(root, &server.uri(), "service"); + assert_eq!(code, 0, "{env:#}\n{stderr}"); + assert_eq!(env["summary"]["applied"], 2); + assert!(!root.join(".socket/blobs").exists()); + let members = + tgz_members(&root.join(format!(".socket/vendor/npm/{BAD_UUID}/tar-fs-2.1.1.tgz"))); + assert_eq!(members["package/index.js"], BAD_PATCHED); + assert_eq!(members["package/test/fixtures/d/file1"], BAD_FIXTURE); + let requests = server.received_requests().await.unwrap(); + assert!(!requests.iter().any(|r| r.url.path().contains("/view/"))); + let requested: Vec<_> = requests + .iter() + .filter(|r| r.method == "POST") + .flat_map(|r| { + serde_json::from_slice::(&r.body).unwrap()["uuids"] + .as_array() + .unwrap() + .clone() + }) + .collect(); + assert!(requested.contains(&json!(GOOD_UUID)) && requested.contains(&json!(BAD_UUID))); +} - assert_eq!(code, 1, "{env:#}\nstderr:\n{stderr}"); - let vendor = &env["vendor"]; - assert_eq!( - vendor["status"], "partialFailure", - "one bad package is a partial failure, not a step abort: {env:#}" - ); - let bad = event_for(vendor, BAD_PURL); - assert_eq!(bad["action"], "failed", "{env:#}"); - assert_eq!(bad["errorCode"], "no_local_source", "{env:#}"); - assert_eq!( - bad["error"].as_str(), - Some("the patch view served no blob content for package/index.js"), - "{env:#}" - ); +#[tokio::test(flavor = "multi_thread")] +async fn unavailable_artifact_fails_only_its_package_without_local_fallback() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + fixture(root); + publish(&server, root, None).await; + // Even valid after-blobs cannot supply an unavailable server artifact. + std::fs::create_dir_all(root.join(".socket/blobs")).unwrap(); + std::fs::write( + root.join(".socket/blobs").join(git_hash(BAD_PATCHED)), + BAD_PATCHED, + ) + .unwrap(); + let (code, env, stderr) = vendor_cli_with_source(root, &server.uri(), "auto"); + assert_eq!(code, 1, "{env:#}\n{stderr}"); + assert_eq!(event_for(&env, GOOD_PURL)["action"], "applied"); + assert_eq!(event_for(&env, BAD_PURL)["action"], "failed"); + assert!(!root.join(format!(".socket/vendor/npm/{BAD_UUID}")).exists()); assert_eq!( - events(vendor) - .iter() - .filter(|e| e["purl"] == BAD_PURL) - .count(), - 1, - "the stuck package is reported exactly once: {env:#}" + std::fs::read(root.join("node_modules/tar-fs/index.js")).unwrap(), + BAD_ORIG ); - assert_eq!(event_for(vendor, GOOD_PURL)["action"], "applied", "{env:#}"); +} + +#[tokio::test(flavor = "multi_thread")] +async fn valid_archive_integrity_does_not_hide_incorrect_patched_members() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + fixture(root); + publish(&server, root, Some(BAD_ORIG)).await; + let (code, env, stderr) = vendor_cli_with_source(root, &server.uri(), "service"); + assert_eq!(code, 1, "{env:#}\n{stderr}"); + assert_eq!(event_for(&env, GOOD_PURL)["action"], "applied"); + assert_eq!(event_for(&env, BAD_PURL)["action"], "failed"); + assert!(!root.join(format!(".socket/vendor/npm/{BAD_UUID}")).exists()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn scan_vendor_uses_contentless_views_and_downloaded_archives() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path(); + fixture(root); + publish(&server, root, Some(BAD_PATCHED)).await; + mount_discovery(&server).await; + for (purl, uuid, files) in [ + (GOOD_PURL, GOOD_UUID, good_files()), + (BAD_PURL, BAD_UUID, bad_files()), + ] { + let mut view = patch_record(uuid, files); + view["purl"] = json!(purl); + view["publishedAt"] = json!("2026-01-01T00:00:00Z"); + Mock::given(method("GET")) + .and(wm_path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(view)) + .mount(&server) + .await; + } + std::fs::remove_file(root.join(".socket/manifest.json")).unwrap(); + let (code, env, stderr) = scan_vendored_cli(root, &server.uri()); + assert_eq!(code, 0, "{env:#}\n{stderr}"); + assert_eq!(env["vendor"]["summary"]["applied"], 2); + assert!(!root.join(".socket/manifest.json").exists()); + assert!(!root.join(".socket/blobs").exists()); +} + +#[tokio::test(flavor = "multi_thread")] +async fn local_build_source_is_rejected() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + fixture(tmp.path()); + let before = std::fs::read(tmp.path().join("package-lock.json")).unwrap(); + let out = Command::new(env!("CARGO_BIN_EXE_socket-patch")) + .current_dir(tmp.path()) + .args([ + "vendor", + "--json", + "--vendor-source", + "build", + "--api-url", + &server.uri(), + ]) + .output() + .unwrap(); + assert!(!out.status.success()); assert!( - root.join(format!(".socket/vendor/npm/{GOOD_UUID}/left-pad-1.3.0.tgz")) - .is_file(), - "the satisfiable package must still be vendored: {env:#}" + String::from_utf8_lossy(&out.stderr).contains("local artifact construction was removed") ); - assert!( - !root.join(format!(".socket/vendor/npm/{BAD_UUID}")).exists(), - "nothing is written for the unstageable package: {env:#}" + assert_eq!( + std::fs::read(tmp.path().join("package-lock.json")).unwrap(), + before ); + assert!(server.received_requests().await.unwrap().is_empty()); } diff --git a/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs b/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs index 415dea702..1bc5ac69d 100644 --- a/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs +++ b/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs @@ -71,6 +71,9 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::ffi::OsString; use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -655,9 +658,9 @@ impl ScanApi { fn socket_patch(cwd: &Path, args: &[&str]) -> Output { let mut cmd = Command::new(binary()); scrub_python_env(&mut cmd); + let _fixture = prebuilt_common::prepare_command(&mut cmd, cwd, args, &[]); cmd.env("SOCKET_TELEMETRY_DISABLED", "1") .env("SOCKET_NO_CONFIG", "1") - .args(args) .current_dir(cwd) .output() .expect("spawn socket-patch") @@ -1610,9 +1613,8 @@ pub fn run_lane(suite: &str, uv: &Uv, mode: Mode, lane: Lane) { if mode == Mode::Hosted { let env: Value = serde_json::from_slice(&out.stdout) .unwrap_or_else(|e| panic!("{}: ({e})\n{}", report.what("revert"), dump(&out))); - let still_wired = |f: &str| { - String::from_utf8_lossy(&std::fs::read(proj.join(f)).unwrap()).contains(uuid) - }; + let still_wired = + |f: &str| String::from_utf8_lossy(&std::fs::read(proj.join(f)).unwrap()).contains(uuid); match out.status.code() { Some(0) => { assert_eq!( @@ -1648,7 +1650,10 @@ pub fn run_lane(suite: &str, uv: &Uv, mode: Mode, lane: Lane) { assert_eq!( snapshot( &proj, - &wired_files.iter().map(|(f, _)| f.as_str()).collect::>() + &wired_files + .iter() + .map(|(f, _)| f.as_str()) + .collect::>() ), wired_files, "{}: a refused pin writes nothing", diff --git a/crates/socket-patch-cli/tests/vex_pdm_hatch_common/mod.rs b/crates/socket-patch-cli/tests/vex_pdm_hatch_common/mod.rs index cdb642e29..35ebebdba 100644 --- a/crates/socket-patch-cli/tests/vex_pdm_hatch_common/mod.rs +++ b/crates/socket-patch-cli/tests/vex_pdm_hatch_common/mod.rs @@ -35,6 +35,9 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::{BTreeMap, HashMap}; use std::path::{Path, PathBuf}; use std::process::Command; @@ -314,6 +317,13 @@ impl ScanApi { .unwrap(); let server = rt.block_on(wiremock::MockServer::start()); let api = ScanApi { rt, server }; + if uuid == Mode::Vendored.uuid() { + api.rt.block_on(prebuilt_common::mount_view( + &api.server, + &view(uuid, PURL), + None, + )); + } api.mount( Mock::given(method("POST")) .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) @@ -442,7 +452,7 @@ pub fn run_scan( .collect(); match mode { Mode::Hosted => args.push("--mode=hosted".into()), - Mode::Vendored => args.extend(["--vendor", "--vendor-source", "build"].map(String::from)), + Mode::Vendored => args.extend(["--vendor", "--vendor-source", "service"].map(String::from)), } args.extend(extra.iter().map(|s| s.to_string())); let out = cli() @@ -1219,11 +1229,13 @@ pub fn embedded_rescan_of_a_manifest_less_checkout( mode, &["--vex", vex_out.to_str().unwrap(), "--vex-product", PRODUCT], ); - match (mode, expect_refusal) { - (Mode::Vendored, Some(refusal)) => { + match mode { + Mode::Vendored => { assert_eq!(code, Some(1), "{what}: {env}\n{stderr}"); - assert_eq!( - env["vendor"]["events"][0]["errorCode"], refusal, + let refusal = env["vendor"]["events"][0]["errorCode"].as_str(); + assert!( + refusal == Some("vendor_ledger_entry_missing") + || (expect_refusal.is_some() && refusal == expect_refusal), "{what}: {env}" ); assert!(!vex_out.exists(), "{what}: no VEX on a failed scan"); diff --git a/crates/socket-patch-cli/tests/vex_pipenv_pip_common/mod.rs b/crates/socket-patch-cli/tests/vex_pipenv_pip_common/mod.rs index 654e445fc..f871cf1f6 100644 --- a/crates/socket-patch-cli/tests/vex_pipenv_pip_common/mod.rs +++ b/crates/socket-patch-cli/tests/vex_pipenv_pip_common/mod.rs @@ -41,6 +41,9 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::collections::{BTreeMap, HashMap}; use std::path::{Path, PathBuf}; use std::process::Command; @@ -363,6 +366,13 @@ impl ScanApi { .unwrap(); let server = rt.block_on(wiremock::MockServer::start()); let api = ScanApi { rt, server }; + if uuid == Mode::Vendored.uuid() { + api.rt.block_on(prebuilt_common::mount_view( + &api.server, + &view(uuid, PURL), + None, + )); + } api.mount( Mock::given(method("POST")) .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) @@ -457,7 +467,7 @@ pub fn run_scan( ]; match mode { Mode::Hosted => args.push("--mode=hosted"), - Mode::Vendored => args.extend(["--vendor", "--vendor-source", "build"]), + Mode::Vendored => args.extend(["--vendor", "--vendor-source", "service"]), } args.extend_from_slice(extra); let out = cli(cwd.parent().unwrap()) @@ -1275,12 +1285,24 @@ pub fn embedded_rescan_of_a_manifest_less_checkout(flavors: &[Flavor]) { mode, &["--vex", vex_out.to_str().unwrap(), "--vex-product", PRODUCT], ); - assert_eq!(code, Some(0), "{what}: {env}\n{stderr}"); - let doc: Value = serde_json::from_slice( - &std::fs::read(&vex_out).unwrap_or_else(|e| panic!("{what}: ({e}) {env}\n{stderr}")), - ) - .unwrap(); - assert_statement(&doc, mode, &what); + if mode == Mode::Vendored { + assert_eq!(code, Some(1), "{what}: {env}\n{stderr}"); + assert_eq!( + env["vendor"]["events"][0]["errorCode"], "vendor_ledger_entry_missing", + "{env}" + ); + assert!(!vex_out.exists(), "failed scan cannot emit VEX"); + let patch_api = api_with(mode.uuid(), PURL); + assert_ok_attested(&vex(&cwd, &vex_run(Some(&patch_api))), mode, &what); + } else { + assert_eq!(code, Some(0), "{what}: {env}\n{stderr}"); + let doc: Value = serde_json::from_slice( + &std::fs::read(&vex_out) + .unwrap_or_else(|e| panic!("{what}: ({e}) {env}\n{stderr}")), + ) + .unwrap(); + assert_statement(&doc, mode, &what); + } for rel in wired.changed_files() { assert_eq!( text(&std::fs::read(cwd.join(rel)).unwrap()), diff --git a/crates/socket-patch-cli/tests/vex_pipenv_pip_real/mod.rs b/crates/socket-patch-cli/tests/vex_pipenv_pip_real/mod.rs index 9d0135d3c..0d43245bc 100644 --- a/crates/socket-patch-cli/tests/vex_pipenv_pip_real/mod.rs +++ b/crates/socket-patch-cli/tests/vex_pipenv_pip_real/mod.rs @@ -95,7 +95,7 @@ impl Mode { pub fn scan_flags(self) -> &'static [&'static str] { match self { Mode::Hosted => &["--mode=hosted"], - Mode::Vendored => &["--vendor", "--vendor-source", "build"], + Mode::Vendored => &["--vendor", "--vendor-source", "service"], } } } @@ -473,7 +473,7 @@ impl RealApi { "results": { uuid: { "status": "granted", "url": url, "purl": PURL, "artifacts": [{ "kind": "tarball", "url": url, - "integrity": { "sha256": api.wheel_sha256 } }], + "integrity": { "sha256": api.wheel_sha256, "sha512": ({ use base64::Engine as _; use sha2::Digest; format!("sha512-{}", base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(&wheel))) }) } }], "registryOverride": null } } }))), diff --git a/crates/socket-patch-cli/tests/vex_pypi_real_common/mod.rs b/crates/socket-patch-cli/tests/vex_pypi_real_common/mod.rs index 7ed0bf480..1aeb30c09 100644 --- a/crates/socket-patch-cli/tests/vex_pypi_real_common/mod.rs +++ b/crates/socket-patch-cli/tests/vex_pypi_real_common/mod.rs @@ -360,6 +360,8 @@ pub struct RealApi { impl RealApi { pub fn start(uuid: &str, pristine: &[u8], patched: &[u8]) -> Self { + use base64::Engine as _; + use sha2::{Digest, Sha512}; use wiremock::matchers::{method, path, path_regex}; use wiremock::{Mock, ResponseTemplate}; let rt = tokio::runtime::Builder::new_multi_thread() @@ -369,6 +371,10 @@ impl RealApi { .unwrap(); let server = rt.block_on(wiremock::MockServer::start()); let wheel = build_wheel(patched); + let wheel_sha512 = format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(Sha512::digest(&wheel)) + ); let api = RealApi { rt, server, @@ -405,7 +411,8 @@ impl RealApi { "results": { uuid: { "status": "granted", "url": url, "purl": PURL, "artifacts": [{ "kind": "tarball", "url": url, - "integrity": { "sha256": api.wheel_sha256 } }], + "integrity": { "sha256": api.wheel_sha256, + "sha512": wheel_sha512 } }], "registryOverride": null } } }))), diff --git a/crates/socket-patch-cli/tests/vlt-leg-manifest.json b/crates/socket-patch-cli/tests/vlt-leg-manifest.json index 6b46a0027..5aa1fb3d2 100644 --- a/crates/socket-patch-cli/tests/vlt-leg-manifest.json +++ b/crates/socket-patch-cli/tests/vlt-leg-manifest.json @@ -46,7 +46,7 @@ "e2e_vendor_vlt_build": { "vendored": [ "scan_fresh_ci", - "get_build_fresh_ci", + "get_auto_fresh_ci", "get_service_fresh_ci", "durability", "workspace_member_selfref", @@ -178,7 +178,7 @@ ], "vendored": [ "scan_fresh_ci", - "get_build_fresh_ci", + "get_auto_fresh_ci", "get_service_fresh_ci", "durability", "workspace_member_selfref", diff --git a/crates/socket-patch-cli/tests/vlt_hosted_common/vendored.rs b/crates/socket-patch-cli/tests/vlt_hosted_common/vendored.rs index 8cf524b90..81fe40d5c 100644 --- a/crates/socket-patch-cli/tests/vlt_hosted_common/vendored.rs +++ b/crates/socket-patch-cli/tests/vlt_hosted_common/vendored.rs @@ -8,6 +8,9 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::Path; use serde_json::json; @@ -61,7 +64,12 @@ pub fn vendored_project(root: &Path, keep_manifest: bool) { write_project(root); seed_manifest(root); let cwd = root.to_str().unwrap().to_string(); - let (code, env, stderr) = hosted::run_json(root, &["vendor", "--offline", "--cwd", &cwd], &[]); + let server = prebuilt_common::Server::project(root); + let (code, env, stderr) = hosted::run_json( + root, + &["vendor", "--cwd", &cwd], + &[("SOCKET_VENDOR_URL", &server.uri)], + ); assert_eq!(code, 0, "vendor: {env:#}\n{stderr}"); assert!(root.join(rel()).join("index.js").is_file()); if !keep_manifest { diff --git a/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs b/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs index ea30ab09b..30f61c744 100644 --- a/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs +++ b/crates/socket-patch-cli/tests/vlt_vendor_common/mod.rs @@ -18,6 +18,9 @@ #![allow(dead_code)] +#[path = "../prebuilt_common/mod.rs"] +mod prebuilt_common; + use std::path::{Path, PathBuf}; use std::process::Command; @@ -382,6 +385,13 @@ pub fn socket(cwd: &Path, args: &[&str], env: &[(&str, &str)]) -> (i32, Value, S for (k, v) in env { cmd.env(k, v); } + let fixture = prebuilt_common::Server::project_with_env(cwd, env); + if !args.contains(&"--vendor-url") + && !args.contains(&"--api-url") + && !env.iter().any(|(k, _)| *k == "SOCKET_VENDOR_URL") + { + fixture.command(&mut cmd); + } let out = cmd.output().expect("spawn socket-patch"); let stdout = String::from_utf8_lossy(&out.stdout).to_string(); let stderr = String::from_utf8_lossy(&out.stderr).to_string(); @@ -394,11 +404,11 @@ pub fn socket(cwd: &Path, args: &[&str], env: &[(&str, &str)]) -> (i32, Value, S /// `socket-patch ` in `cwd`, human output: `(exit code, stdout, /// stderr)`. pub fn socket_human(cwd: &Path, args: &[&str]) -> (i32, String, String) { - let out = cli() - .args(args) - .current_dir(cwd) - .output() - .expect("spawn socket-patch"); + let fixture = prebuilt_common::Server::project(cwd); + let mut cmd = cli(); + cmd.args(args).current_dir(cwd); + fixture.command(&mut cmd); + let out = cmd.output().expect("spawn socket-patch"); ( out.status.code().unwrap_or(-1), String::from_utf8_lossy(&out.stdout).to_string(), @@ -409,7 +419,7 @@ pub fn socket_human(cwd: &Path, args: &[&str]) -> (i32, String, String) { /// `vendor --json --offline` plus `extra`. pub fn vendor(root: &Path, extra: &[&str]) -> (i32, Value, String) { let cwd = root.to_str().unwrap().to_string(); - let mut args = vec!["vendor", "--json", "--offline", "--cwd", &cwd]; + let mut args = vec!["vendor", "--json", "--cwd", &cwd]; args.extend_from_slice(extra); socket(root, &args, &[]) } diff --git a/crates/socket-patch-core/Cargo.toml b/crates/socket-patch-core/Cargo.toml index 5b78b8529..02c0cc31a 100644 --- a/crates/socket-patch-core/Cargo.toml +++ b/crates/socket-patch-core/Cargo.toml @@ -15,8 +15,11 @@ readme = "README.md" # [dev-dependencies] only, so a `cargo build --release` / `cargo install` # binary never carries it. failpoints = [] +# Shared HTTP artifact fixtures, enabled only by the CLI dev-dependency. +test-fixtures = ["dep:wiremock"] [dependencies] +wiremock = { workspace = true, optional = true } serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } diff --git a/crates/socket-patch-core/src/api/client.rs b/crates/socket-patch-core/src/api/client.rs index 4729e5d0a..31c9b28c2 100644 --- a/crates/socket-patch-core/src/api/client.rs +++ b/crates/socket-patch-core/src/api/client.rs @@ -338,13 +338,13 @@ pub(crate) const MAX_REFERENCE_BATCH: usize = 500; /// Why a pypi reference is refused before its download: the served /// artifact is not a wheel. pub(crate) const PYPI_NOT_A_WHEEL: &str = - "the prebuilt artifact is not a .whl (pypi vendoring is wheel-based)"; + "the prebuilt artifact is not a supported Python distribution (.whl, .tar.gz, .tgz, .zip)"; /// The last path segment of a serve URL, when it names a `.whl`. pub(crate) fn wheel_filename_from_url(url: &str) -> Option { let path = url.split(['?', '#']).next().unwrap_or(url); let name = path.rsplit('/').next().unwrap_or(""); - name.ends_with(".whl").then(|| name.to_string()) + crate::vendor::pypi_distribution::supported(name).then(|| name.to_string()) } /// Body payload for the batch search POST endpoint. @@ -1326,8 +1326,7 @@ impl ApiClient { }, None => download_url.to_string(), }; - // pypi vendoring is wheel-based, so the sdist a qualifier-less pypi - // patch is served can never be used: refuse it before downloading. + // Reject unsupported distribution names before downloading. if result .purl .as_deref() @@ -1371,6 +1370,11 @@ impl ApiClient { match self.download_vendor_archive_retrying(&download_url).await { (ServeDownload::Ok(bytes), _) => { done(VendorServiceOutcome::Ready(FetchedVendorPackage { + yarn_berry10c0: result + .artifacts + .as_ref() + .and_then(|arts| arts.iter().find(|a| a.kind == "yarn-berry-zip")) + .and_then(|a| a.integrity.yarn_berry10c0.clone()), tarball: bytes, integrity_sri, dirhash_h1: artifact.integrity.dirhash_h1.clone(), @@ -1786,6 +1790,7 @@ pub(crate) const MAX_VENDOR_PACKAGE_BYTES: u64 = 256 * 1024 * 1024; /// `h1:` dirhash) before writing/extracting. #[derive(Debug, Clone)] pub(crate) struct FetchedVendorPackage { + pub yarn_berry10c0: Option, pub tarball: Vec, /// Normalized Subresource-Integrity string, always `sha512-`. pub integrity_sri: String, diff --git a/crates/socket-patch-core/src/api/vendor_prefetch.rs b/crates/socket-patch-core/src/api/vendor_prefetch.rs index d2c76d079..bca7e336e 100644 --- a/crates/socket-patch-core/src/api/vendor_prefetch.rs +++ b/crates/socket-patch-core/src/api/vendor_prefetch.rs @@ -1421,7 +1421,7 @@ mod tests { } let cfg = crate::vendor::VendorServiceConfig { maven_config: None, - source: crate::vendor::VendorSource::Auto, + source: crate::vendor::VendorSource::Service, client: Some(client(&server.uri())), use_public_proxy: false, vendor_url: None, diff --git a/crates/socket-patch-core/src/patch/apply.rs b/crates/socket-patch-core/src/patch/apply.rs index 5b99fc2a8..8ac188828 100644 --- a/crates/socket-patch-core/src/patch/apply.rs +++ b/crates/socket-patch-core/src/patch/apply.rs @@ -495,14 +495,7 @@ pub(crate) async fn apply_file_patch_at( // `restore_file_permissions` re-applies the pre-patch mode + uid/gid. // The directory mode is restored whether or not the write succeeded, // before any failure propagates. - // Inside a vendor stage the copy is a content-verified artifact, written - // without an fsync (see `crate::utils::durability`); an in-place apply - // of an installed tree keeps the durable write. - let write_result = if crate::utils::durability::in_artifact_scope() { - crate::utils::fs::atomic_write_artifact(&filepath, patched_content).await - } else { - crate::utils::fs::atomic_write_bytes(&filepath, patched_content).await - }; + let write_result = crate::utils::fs::atomic_write_bytes(&filepath, patched_content).await; dir_guard.restore().await; write_result?; @@ -943,13 +936,13 @@ async fn apply_package_patch_at( let current_hash = verify_result.and_then(|v| v.current_hash.as_deref()); let (patched_content, via): (Cow<'_, [u8]>, AppliedVia) = if let Some(bytes) = resolve_from_diff( - diff_entries.as_ref(), - normalized, - pkg_path, - file_info, - current_hash, - ) - .await + diff_entries.as_ref(), + normalized, + pkg_path, + file_info, + current_hash, + ) + .await { (Cow::Owned(bytes), AppliedVia::Diff) } else if let Some(bytes) = sources.mem_blobs.and_then(|m| m.get(&file_info.after_hash)) { @@ -2192,20 +2185,13 @@ mod tests { ); ( - root, - pkg_dir, - blobs_dir, - diffs_dir, - files, - original, - patched, + root, pkg_dir, blobs_dir, diffs_dir, files, original, patched, ) } #[tokio::test] async fn test_apply_via_diff_when_archive_present() { - let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = - make_fixture().await; + let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = make_fixture().await; let sources = PatchSources { blobs_path: &blobs_dir, @@ -2231,8 +2217,7 @@ mod tests { #[tokio::test] async fn test_apply_falls_back_to_blob_when_no_archives() { - let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = - make_fixture().await; + let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = make_fixture().await; // Delete the diff archive. tokio::fs::remove_file(diffs_dir.join(format!("{TEST_UUID}.tar.gz"))) .await @@ -2264,8 +2249,7 @@ mod tests { async fn test_apply_uuid_none_disables_alt_sources() { // Even if archives exist, passing `uuid = None` must restrict the // pipeline to the blob path. - let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, _patched) = - make_fixture().await; + let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, _patched) = make_fixture().await; let sources = PatchSources { blobs_path: &blobs_dir, @@ -2292,8 +2276,7 @@ mod tests { // Corrupt the on-disk file so its hash no longer matches // before_hash. Diff strategy must NOT run (its output would never // match after_hash), so we fall through to the blob. - let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = - make_fixture().await; + let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = make_fixture().await; // Overwrite on-disk content with garbage; use --force so verify // promotes the HashMismatch to Ready and the pipeline still tries // to apply. @@ -3120,8 +3103,7 @@ mod tests { /// to the blob strategy and still patch successfully. #[tokio::test] async fn test_apply_corrupt_diff_falls_through_to_blob() { - let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = - make_fixture().await; + let (_root, pkg_dir, blobs_dir, diffs_dir, files, _orig, patched) = make_fixture().await; // Diff archive holds garbage delta bytes. write_uuid_archive(&diffs_dir, TEST_UUID, &[("index.js", b"garbage delta")]); @@ -3301,8 +3283,7 @@ mod tests { /// never joined. #[tokio::test] async fn test_apply_unsafe_uuid_skips_archives() { - let (root, pkg_dir, blobs_dir, diffs_dir, files, original, patched) = - make_fixture().await; + let (root, pkg_dir, blobs_dir, diffs_dir, files, original, patched) = make_fixture().await; // `diffs/../escape/.tar.gz` IS a valid diff archive. let escape_dir = root.path().join("escape"); tokio::fs::create_dir_all(&escape_dir).await.unwrap(); diff --git a/crates/socket-patch-core/src/patch/package.rs b/crates/socket-patch-core/src/patch/package.rs index 68919b083..77647ff8c 100644 --- a/crates/socket-patch-core/src/patch/package.rs +++ b/crates/socket-patch-core/src/patch/package.rs @@ -111,7 +111,7 @@ pub fn read_archive_to_map(archive_path: &Path) -> Result Result Result>, ArchiveError> { - read_archive_from_reader(bytes, false) + read_archive_from_reader(bytes, false, true) } /// [`read_archive_bytes_to_map`] that additionally refuses any archive an @@ -142,7 +142,15 @@ pub fn read_archive_bytes_to_map(bytes: &[u8]) -> Result pub fn read_archive_bytes_to_map_strict( bytes: &[u8], ) -> Result>, ArchiveError> { - read_archive_from_reader(bytes, true) + read_archive_from_reader(bytes, true, true) +} + +/// Read a source distribution without npm's `package/` prefix convention. +/// Uses the same strict member checks and decompression limits. +pub(crate) fn read_sdist_tar_bytes_to_map_strict( + bytes: &[u8], +) -> Result>, ArchiveError> { + read_archive_from_reader(bytes, true, false) } /// The shared decoder behind [`read_archive_to_map`] and @@ -152,6 +160,7 @@ pub fn read_archive_bytes_to_map_strict( fn read_archive_from_reader( reader: R, strict: bool, + strip_package_prefix: bool, ) -> Result>, ArchiveError> { // Hard-cap decompressed bytes to defuse gzip / tar bombs. Reads // beyond the limit yield EOF, which the tar parser surfaces as a @@ -190,7 +199,12 @@ fn read_archive_from_reader( } // The installers strip the FIRST segment whatever it is; // only `package/` maps onto the decoded key space. - let stripped = if is_dir && raw.trim_end_matches('/') == "package" { + let stripped = if !strip_package_prefix { + if raw.starts_with(['/', '\\']) || !is_safe_relative_subpath(&raw) { + return Err(ArchiveError::UnsafePath(raw)); + } + Some(raw.as_str()) + } else if is_dir && raw.trim_end_matches('/') == "package" { None } else if let Some(rest) = raw.strip_prefix("package/") { Some(rest) @@ -233,7 +247,12 @@ fn read_archive_from_reader( // absolute path `/etc/passwd`. `Path::join` resolves an absolute // right-hand side by discarding the base, so that would escape the // package directory entirely. Always validate post-normalization. - let normalized = normalize_file_path(&path_str).to_string(); + let normalized = if strip_package_prefix { + normalize_file_path(&path_str) + } else { + &path_str + } + .to_string(); let normalized_path = Path::new(&normalized); // This is THE path-safety chokepoint for archive entries (see the diff --git a/crates/socket-patch-core/src/patch/redirect/golang_local.rs b/crates/socket-patch-core/src/patch/redirect/golang_local.rs index feb2ea22c..415dd4903 100644 --- a/crates/socket-patch-core/src/patch/redirect/golang_local.rs +++ b/crates/socket-patch-core/src/patch/redirect/golang_local.rs @@ -242,19 +242,7 @@ pub async fn apply_go_redirect<'a>( if dry_run { // Verify (read-only) against the pristine source for an accurate // "would patch" report, without creating the copy or editing go.mod. - // The verify reads it, so a lazily-fetched source materialises here. - let pristine_src = match pristine_src.materialize().await { - Ok(dir) => dir, - Err(e) => { - return synthesized_result( - purl, - ©_dir, - Vec::new(), - false, - Some(format!("failed to copy pristine source: {e}")), - ) - } - }; + let pristine_src = pristine_src.path(); let mut result = apply_package_patch(purl, pristine_src, files, sources, uuid, true, policy).await; result.package_path = copy_dir.display().to_string(); @@ -269,11 +257,8 @@ pub async fn apply_go_redirect<'a>( return already_patched_result(purl, ©_dir, files); } - // Materialise pristine → copy_dir: a module-cache source is copied, a - // fetched one is written straight here from the verified module zip - // (unless an earlier branch already extracted it, which `stage_into` - // copies from instead). - if let Err(e) = pristine_src.stage_into(©_dir, None).await { + if let Err(e) = crate::patch::copy_tree::fresh_copy(pristine_src.path(), ©_dir, None).await + { teardown_failed_redirect(project_root, ©_dir, module, version, base_rel).await; return synthesized_result( purl, @@ -657,11 +642,7 @@ pub(crate) async fn ensure_module_go_mod(copy_dir: &Path, module: &str) -> std:: return Ok(()); } let body = format!("module {module}\n"); - if crate::utils::durability::in_artifact_scope() { - crate::utils::fs::atomic_write_artifact(&go_mod, body.as_bytes()).await - } else { - crate::utils::fs::atomic_write_bytes(&go_mod, body.as_bytes()).await - } + crate::utils::fs::atomic_write_bytes(&go_mod, body.as_bytes()).await } /// Recursively find every patched-copy module dir under `go_patches_root`, diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/client.rs b/crates/socket-patch-core/src/patch/redirect/upstream/client.rs index ffa47b9fc..bf8e07787 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/client.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/client.rs @@ -156,7 +156,7 @@ pub(crate) struct UpstreamClient { http: RegistryClient, offline: bool, npm: Cache, - npm_tarballs: Cache>, + npm_berry: Cache, cargo: Cache, go: Cache, rubygems: Cache, @@ -171,7 +171,7 @@ impl UpstreamClient { http: build_registry_client(), offline, npm: Mutex::default(), - npm_tarballs: Mutex::default(), + npm_berry: Mutex::default(), cargo: Mutex::default(), go: Mutex::default(), rubygems: Mutex::default(), @@ -231,8 +231,8 @@ impl UpstreamClient { .get("dist") .ok_or_else(|| format!("{url} carries no `dist` block"))?; let str_field = |k: &str| dist.get(k).and_then(Value::as_str).map(str::to_string); - let tarball = str_field("tarball") - .ok_or_else(|| format!("{url} carries no `dist.tarball`"))?; + let tarball = + str_field("tarball").ok_or_else(|| format!("{url} carries no `dist.tarball`"))?; Ok(NpmDist { tarball, integrity: str_field("integrity"), @@ -240,25 +240,67 @@ impl UpstreamClient { }) } - /// The verified upstream tarball bytes of `name@version` (checked - /// against the registry's `dist.integrity`). - pub(crate) async fn npm_tarball(&self, name: &str, version: &str) -> Result, String> { + pub(crate) async fn npm_berry_checksum( + &self, + uuid: &str, + name: &str, + version: &str, + origin: &str, + ) -> Result { + if self.offline { + return Err(OFFLINE.into()); + } let key = (name.to_string(), version.to_string()); - if let Some(hit) = self.npm_tarballs.lock().await.get(&key) { + if let Some(hit) = self.npm_berry.lock().await.get(&key) { return hit.clone(); } let result = async { + let url = format!("{}/upstream/npm/{uuid}.json", origin.trim_end_matches('/')); + let metadata = self.get_json(&url).await?; + if metadata["name"].as_str() != Some(name) + || metadata["version"].as_str() != Some(version) + { + return Err("upstream checksum metadata names a different package".into()); + } + let checksum = metadata["yarnBerry10c0"] + .as_str() + .filter(|c| crate::vendor::yarn_berry_lock::valid_berry_checksum(c)) + .ok_or("the patch service supplied no valid upstream Berry checksum")?; + let integrity = metadata["integrity"] + .as_str() + .filter(|s| s.starts_with("sha512-")) + .ok_or("upstream checksum metadata has no archive integrity")?; + use base64::Engine as _; + if !base64::engine::general_purpose::STANDARD + .decode(&integrity[7..]) + .is_ok_and(|bytes| bytes.len() == 64) + { + return Err("upstream checksum metadata has invalid SHA-512 integrity".into()); + } let dist = self.npm_dist(name, version).await?; - let integrity = dist + if !dist .integrity - .clone() - .ok_or_else(|| format!("the registry records no integrity for {name}@{version}"))?; - let bytes = crate::vendor::registry_fetch::download(&self.http, &dist.tarball).await?; - crate::vendor::registry_fetch::verify_sri(&bytes, &integrity)?; - Ok(bytes) + .as_deref() + .is_some_and(|s| s.split_whitespace().any(|v| v == integrity)) + { + let bytes = + crate::vendor::registry_fetch::download(&self.http, &dist.tarball).await?; + crate::vendor::registry_fetch::verify_sri(&bytes, integrity)?; + if let Some(sri) = dist.integrity.as_deref() { + crate::vendor::registry_fetch::verify_sri(&bytes, sri)?; + } else if let Some(sha1) = dist.shasum.as_deref() { + use sha1::Digest as _; + if hex::encode(sha1::Sha1::digest(&bytes)) != sha1 { + return Err("registry archive checksum mismatch".into()); + } + } else { + return Err("the registry supplied no archive integrity".into()); + } + } + Ok(checksum.to_string()) } .await; - self.npm_tarballs.lock().await.insert(key, result.clone()); + self.npm_berry.lock().await.insert(key, result.clone()); result } @@ -355,15 +397,17 @@ impl UpstreamClient { }; return match (pick(&zip_key), pick(&mod_key)) { (Some(zip_h1), Some(mod_h1)) => Ok(GoSums { zip_h1, mod_h1 }), - _ => Err(format!("{url} does not list both go.sum lines of {module} {version}")), + _ => Err(format!( + "{url} does not list both go.sum lines of {module} {version}" + )), }; } let proxy = crate::vendor::registry_fetch::goproxy_base(module)?; let escaped = crate::crawlers::go_crawler::encode_module_path(module); let escaped_version = crate::crawlers::go_crawler::encode_module_path(version); let base = format!("{proxy}/{escaped}/@v/{escaped_version}"); - let zip = crate::vendor::registry_fetch::download(&self.http, &format!("{base}.zip")) - .await?; + let zip = + crate::vendor::registry_fetch::download(&self.http, &format!("{base}.zip")).await?; let zip_h1 = crate::vendor::registry_fetch::go_h1_of_zip(&zip)?; let go_mod = crate::vendor::registry_fetch::download(&self.http, &format!("{base}.mod")).await?; @@ -379,7 +423,11 @@ impl UpstreamClient { /// The rubygems.org sha256 of the ruby-platform `name-version.gem`, /// from the compact index bundler itself reads (`info/`). - pub(crate) async fn rubygems_sha256(&self, name: &str, version: &str) -> Result { + pub(crate) async fn rubygems_sha256( + &self, + name: &str, + version: &str, + ) -> Result { let key = (name.to_string(), version.to_string()); if let Some(hit) = self.rubygems.lock().await.get(&key) { return hit.clone(); @@ -404,7 +452,11 @@ impl UpstreamClient { /// Every version packagist serves for the composer package `name` /// (lowercase `vendor/package`), expanded: the stable `p2/.json` /// list, or the `~dev` one when `dev` (composer splits branches out). - pub(crate) async fn packagist_versions(&self, name: &str, dev: bool) -> Result, String> { + pub(crate) async fn packagist_versions( + &self, + name: &str, + dev: bool, + ) -> Result, String> { let key = (name.to_string(), if dev { "~dev" } else { "" }.to_string()); if let Some(hit) = self.packagist.lock().await.get(&key) { return hit.clone(); @@ -443,7 +495,11 @@ impl UpstreamClient { /// (base64 sha512 of the `.nupkg`): the `packageHash` of the catalog /// entry its registration leaf points at — the hash nuget.org computed /// over the repository-signed package, without downloading it. - pub(crate) async fn nuget_content_hash(&self, id: &str, version: &str) -> Result { + pub(crate) async fn nuget_content_hash( + &self, + id: &str, + version: &str, + ) -> Result { let key = (id.to_ascii_lowercase(), version.to_ascii_lowercase()); if let Some(hit) = self.nuget.lock().await.get(&key) { return hit.clone(); @@ -477,7 +533,9 @@ impl UpstreamClient { .eq_ignore_ascii_case(version_lower) }); if !same_id || !same_version { - return Err(format!("{catalog} is not the catalog entry of {id} {version}")); + return Err(format!( + "{catalog} is not the catalog entry of {id} {version}" + )); } let sha512 = entry .get("packageHashAlgorithm") @@ -678,4 +736,106 @@ mod tests { let h1 = go_mod_h1(b"module example.com/m\n"); assert!(h1.starts_with("h1:") && h1.ends_with('='), "{h1}"); } + #[tokio::test] + async fn berry_metadata_is_registry_anchored_without_repacking() { + use base64::Engine as _; + use sha2::Digest as _; + use wiremock::matchers::{method, path}; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let bytes = b"registry archive"; + let integrity = format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(bytes)) + ); + let checksum = format!("10c0/{}", "a".repeat(128)); + for (registry_sri, registry_sha1, expected_downloads) in [ + (Some(integrity.clone()), None, 0), + (None, Some(hex::encode(sha1::Sha1::digest(bytes))), 1), + ] { + let server = MockServer::start().await; + Mock::given(method("GET")).and(path("/upstream/npm/uuid.json")).respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "name":"left-pad", "version":"1.3.0", "integrity":integrity, "yarnBerry10c0":checksum + }))).expect(1).mount(&server).await; + Mock::given(method("GET")) + .and(path("/archive.tgz")) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes.as_slice())) + .expect(expected_downloads) + .mount(&server) + .await; + let client = UpstreamClient::new(false); + client.npm.lock().await.insert( + ("left-pad".into(), "1.3.0".into()), + Ok(NpmDist { + tarball: format!("{}/archive.tgz", server.uri()), + integrity: registry_sri, + shasum: registry_sha1, + }), + ); + for _ in 0..2 { + assert_eq!( + client + .npm_berry_checksum("uuid", "left-pad", "1.3.0", &server.uri()) + .await + .unwrap(), + checksum + ); + } + } + } + + #[tokio::test] + async fn berry_metadata_refuses_wrong_identity_integrity_and_unavailable_service() { + use base64::Engine as _; + use sha2::Digest as _; + use wiremock::matchers::{method, path}; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let integrity = format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(b"expected")) + ); + let valid = serde_json::json!({"name":"left-pad", "version":"1.3.0", "integrity":integrity, "yarnBerry10c0":format!("10c0/{}", "a".repeat(128))}); + for kind in [ + "name", + "version", + "integrity", + "yarnBerry10c0", + "unavailable", + "registry_mismatch", + ] { + let server = MockServer::start().await; + let mut body = valid.clone(); + if body.get(kind).is_some() { + body[kind] = serde_json::json!("invalid"); + } + Mock::given(method("GET")) + .and(path("/upstream/npm/uuid.json")) + .respond_with( + ResponseTemplate::new(if kind == "unavailable" { 503 } else { 200 }) + .set_body_json(body), + ) + .mount(&server) + .await; + Mock::given(method("GET")) + .and(path("/archive.tgz")) + .respond_with(ResponseTemplate::new(200).set_body_bytes(b"different".as_slice())) + .mount(&server) + .await; + let client = UpstreamClient::new(false); + client.npm.lock().await.insert( + ("left-pad".into(), "1.3.0".into()), + Ok(NpmDist { + tarball: format!("{}/archive.tgz", server.uri()), + integrity: Some("sha512-other".into()), + shasum: None, + }), + ); + assert!( + client + .npm_berry_checksum("uuid", "left-pad", "1.3.0", &server.uri()) + .await + .is_err(), + "{kind}" + ); + } + } } diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs index 8bcb5d107..379726b25 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/npm.rs @@ -122,7 +122,13 @@ fn v2_hits( }); } if let Some(nested) = entry.get("dependencies").and_then(Value::as_object) { - v2_hits(nested, &format!("{pointer}/dependencies"), ctx, hits, depth + 1); + v2_hits( + nested, + &format!("{pointer}/dependencies"), + ctx, + hits, + depth + 1, + ); } } } @@ -174,7 +180,9 @@ pub(crate) async fn restore_npm_locks( ); continue; }; - let Some(entry) = lock.pointer_mut(&hit.pointer).and_then(Value::as_object_mut) + let Some(entry) = lock + .pointer_mut(&hit.pointer) + .and_then(Value::as_object_mut) else { continue; }; @@ -428,9 +436,19 @@ async fn restore_berry( if result.refused.contains_key(&uuid) { continue; } - let checksum = match ctx.client.npm_tarball(&name, &version).await.and_then(|tgz| { - crate::vendor::berry_zip::berry_cache_checksum_10c0(&tgz, &name) - }) { + let checksum = match ctx + .client + .npm_berry_checksum( + &uuid, + &name, + &version, + ctx.origins + .first() + .map(String::as_str) + .unwrap_or("https://patch.socket.dev"), + ) + .await + { Ok(c) => crate::vendor::yarn_berry_lock::checksum_in_lock_spelling(&content, &c), Err(why) => { result.refuse(&uuid, format!("{name}@{version}: {why}")); @@ -451,10 +469,7 @@ async fn restore_berry( changed = true; } if changed { - view.write( - rel, - format!("{bom}{}", eol.restore(&blocks.join("\n\n"))), - ); + view.write(rel, format!("{bom}{}", eol.restore(&blocks.join("\n\n")))); } } @@ -513,8 +528,7 @@ pub(crate) async fn restore_pnpm_locks( let Some(resolution) = pnpm::resolution(&entry) else { continue; }; - let Some((_, uuid, name, version)) = - hits.iter().find(|(r, ..)| *r == resolution.range) + let Some((_, uuid, name, version)) = hits.iter().find(|(r, ..)| *r == resolution.range) else { continue; }; @@ -616,7 +630,10 @@ pub(crate) async fn restore_bun_locks( )), _ => result.refuse( &uuid, - format!("the {rel} entry `{}` wiring it is not {}", entry.key, pin.purl), + format!( + "the {rel} entry `{}` wiring it is not {}", + entry.key, pin.purl + ), ), } } @@ -702,10 +719,7 @@ pub(crate) async fn cleanup_side_config( if let Ok(Some(npmrc)) = view.read(NPMRC_REL).await { if npmrc == NPMRC_CREATED { view.remove(NPMRC_REL); - } else if npmrc - .lines() - .any(|l| l.trim() == NPMRC_ALLOW_REMOTE_LINE) - { + } else if npmrc.lines().any(|l| l.trim() == NPMRC_ALLOW_REMOTE_LINE) { result.warnings.push(( "npm_allow_remote_left", format!( @@ -718,10 +732,7 @@ pub(crate) async fn cleanup_side_config( } } - let restored_pnpm = view - .staged - .keys() - .any(|k| k == "pnpm-lock.yaml"); + let restored_pnpm = view.staged.keys().any(|k| k == "pnpm-lock.yaml"); if restored_pnpm && !still_hosted(view, &["pnpm-lock.yaml"], ctx).await { const WORKSPACE: &str = "pnpm-workspace.yaml"; if let Ok(Some(ws)) = view.read(WORKSPACE).await { diff --git a/crates/socket-patch-core/src/patch/sidecars/nuget.rs b/crates/socket-patch-core/src/patch/sidecars/nuget.rs index 3bbf42306..1e672cd29 100644 --- a/crates/socket-patch-core/src/patch/sidecars/nuget.rs +++ b/crates/socket-patch-core/src/patch/sidecars/nuget.rs @@ -28,7 +28,7 @@ use super::{ }; /// `pub(crate)`: the NuGet vendor backend materialises exactly these two -/// package-root paths when a local rebuild keeps the package's parts in +/// package-root paths when an in-memory patch engine keeps the package's parts in /// memory, so the fixup sees the same root a full extraction gave it. One /// definition, so the two cannot drift apart. pub(crate) const METADATA_FILE: &str = ".nupkg.metadata"; diff --git a/crates/socket-patch-core/src/utils/durability.rs b/crates/socket-patch-core/src/utils/durability.rs index 967d56dcb..76b44cd9b 100644 --- a/crates/socket-patch-core/src/utils/durability.rs +++ b/crates/socket-patch-core/src/utils/durability.rs @@ -76,13 +76,7 @@ //! So a crash can only lose an artifact nothing durable names yet, which //! the next run rebuilds exactly as it would one deleted by hand. //! -//! The patched files the apply engine writes into a vendor stage are -//! artifacts too: [`artifact_writes`] marks the vendor stage's apply calls, -//! and the in-place `apply` of an installed tree (which has no later -//! verifying run to fall back on) keeps its durable writes. - use std::collections::{BTreeMap, BTreeSet}; -use std::future::Future; use std::path::{Path, PathBuf}; use std::sync::Mutex; @@ -115,7 +109,11 @@ impl Pending { let mut touched = false; for file in self.files.iter_mut() { if let Ok(rest) = file.strip_prefix(from) { - *file = to.join(rest); + *file = if rest.as_os_str().is_empty() { + to.to_path_buf() + } else { + to.join(rest) + }; touched = true; } } @@ -315,22 +313,6 @@ fn device_flush(_handle: &std::fs::File) -> std::io::Result<()> { Ok(()) } -tokio::task_local! { - static ARTIFACT_SCOPE: (); -} - -/// Run `f` with the apply engine's patched-file writes classed as artifact -/// writes (see the module docs): the vendor stage's apply, whose output is -/// re-verified on every later run. -pub(crate) async fn artifact_writes(f: F) -> F::Output { - ARTIFACT_SCOPE.scope((), f).await -} - -/// Whether the current task runs inside [`artifact_writes`]. -pub(crate) fn in_artifact_scope() -> bool { - ARTIFACT_SCOPE.try_with(|_| ()).is_ok() -} - #[cfg(test)] mod tests { use super::*; @@ -354,6 +336,23 @@ mod tests { assert!(pending.dirs.contains(&tmp.path().to_path_buf())); } + #[test] + fn moved_file_remains_syncable_without_a_trailing_separator() { + let tmp = tempfile::tempdir().unwrap(); + let stage = tmp.path().join("download.whl"); + let artifact = tmp.path().join("committed.whl"); + std::fs::write(&stage, b"verified archive").unwrap(); + let mut pending = Pending { + files: Vec::new(), + dirs: BTreeSet::new(), + }; + pending.record(&stage); + std::fs::rename(&stage, &artifact).unwrap(); + pending.moved(&stage, &artifact); + assert_eq!(pending.files, vec![artifact]); + sync_all_blocking(&pending.files, &pending.dirs).unwrap(); + } + #[test] fn sync_skips_vanished_files_and_syncs_the_rest() { let tmp = tempfile::tempdir().unwrap(); @@ -363,11 +362,4 @@ mod tests { let dirs: BTreeSet = [tmp.path().to_path_buf()].into(); sync_all_blocking(&[kept.clone(), gone, kept], &dirs).unwrap(); } - - #[tokio::test] - async fn artifact_scope_is_task_local() { - assert!(!in_artifact_scope()); - artifact_writes(async { assert!(in_artifact_scope()) }).await; - assert!(!in_artifact_scope()); - } } diff --git a/crates/socket-patch-core/src/utils/pdm_lock.rs b/crates/socket-patch-core/src/utils/pdm_lock.rs index 2e761d226..9c6485cc1 100644 --- a/crates/socket-patch-core/src/utils/pdm_lock.rs +++ b/crates/socket-patch-core/src/utils/pdm_lock.rs @@ -219,7 +219,7 @@ pub fn rewrite_pdm_lock_in<'a>( { return Err("invalid PDM artifact source or SHA-256".into()); } - if !wheel_matches(filename, name, version) { + if !crate::vendor::pypi_distribution::matches(filename, name, version) { return Err("PDM patch wheel does not match package".into()); } let doc = match parse.doc.take() { @@ -332,9 +332,11 @@ fn plan_pdm_rewrite( .filter_map(|&index| packages.get(index)?.get("version").and_then(Item::as_str)) .collect(); if locked_versions.len() > 1 { - return Err("PDM lock resolves this package at multiple versions (a marker or \ + return Err( + "PDM lock resolves this package at multiple versions (a marker or \ multi-target fork); patching one fork would leave the others unpatched" - .into()); + .into(), + ); } let mut variants = std::collections::BTreeSet::new(); let mut edits = Vec::new(); @@ -749,7 +751,10 @@ mod tests { &"a".repeat(64), ) .unwrap(); - assert!(rewired.contains(&fresh) && !rewired.contains(&stale), "{rewired}"); + assert!( + rewired.contains(&fresh) && !rewired.contains(&stale), + "{rewired}" + ); // A foreign (non-Socket) existing url is still refused. let foreign = fixture("2.29.2").replace( "name = \"urllib3\"", diff --git a/crates/socket-patch-core/src/utils/poetry_lock.rs b/crates/socket-patch-core/src/utils/poetry_lock.rs index fc223302a..8502fb5c8 100644 --- a/crates/socket-patch-core/src/utils/poetry_lock.rs +++ b/crates/socket-patch-core/src/utils/poetry_lock.rs @@ -96,7 +96,11 @@ fn legacy_files_entry(table: &dyn TableLike, name: &str, files: Array, rewritten }) .unwrap_or_default() }; - format!(" {{file = {}, hash = {}}},\n", field("file"), field("hash")) + format!( + " {{file = {}, hash = {}}},\n", + field("file"), + field("hash") + ) }) .collect(); match format!("[\n{}]", entries.concat()).parse::() { @@ -137,7 +141,10 @@ fn lock_version_of(lock: &Table) -> Result<&str, String> { { Ok("0") } - None => Err("poetry.lock has neither a [metadata] lock-version nor a [metadata.hashes] table".into()), + None => Err( + "poetry.lock has neither a [metadata] lock-version nor a [metadata.hashes] table" + .into(), + ), } } @@ -279,14 +286,7 @@ pub fn rewrite_poetry_lock_in<'a>( // Poetry compares the lock's `sha256:` against `hashlib`'s lowercase // hexdigest as strings, so an uppercase digest would fail every install. let sha256 = sha256.to_ascii_lowercase(); - if filename.contains(['/', '\\']) || !filename.ends_with(".whl") { - return Err("Poetry patch wheel does not match the locked package".into()); - } - let parts: Vec<_> = filename.split('-').collect(); - if !matches!(parts.len(), 5 | 6) - || canonicalize_pypi_name(parts[0]) != canonicalize_pypi_name(name) - || parts[1] != version - { + if !crate::vendor::pypi_distribution::matches(filename, name, version) { return Err("Poetry patch wheel does not match the locked package".into()); } let doc = match parse.doc.take() { @@ -672,7 +672,15 @@ mod tests { Ok(other) => panic!("{label}: expected a refusal, got {other:?}"), } // The vendored (file-source) spelling takes the same guarded path. - match rewrite_poetry_lock(&text, "urllib3", "1.26.18", "file", ".socket/vendor/pypi/x/urllib3-1.26.18-py2.py3-none-any.whl", WHEEL, &sha()) { + match rewrite_poetry_lock( + &text, + "urllib3", + "1.26.18", + "file", + ".socket/vendor/pypi/x/urllib3-1.26.18-py2.py3-none-any.whl", + WHEEL, + &sha(), + ) { Err(err) => assert!(!err.is_empty(), "{label}"), Ok(other) => panic!("{label}: expected a refusal, got {other:?}"), } @@ -690,7 +698,10 @@ mod tests { assert!(rewritten.contains(URL)); assert!(rewritten.contains("lock-version = \"2.2\"")); for bad in ["3.0", "2", "2.x", "1.2"] { - let lock = fixture("2.4.3").replace("lock-version = \"2.1\"", &format!("lock-version = \"{bad}\"")); + let lock = fixture("2.4.3").replace( + "lock-version = \"2.1\"", + &format!("lock-version = \"{bad}\""), + ); let err = hosted(&lock).unwrap_err(); assert!(err.contains(bad), "{bad}: {err}"); } @@ -713,28 +724,47 @@ mod tests { // Poetry 1.0 carries a `#sha256=…&` fragment; the comparison ignores it. let lock10 = fixture("1.0.10"); let first10 = hosted(&lock10).unwrap().unwrap(); - let second10 = rewrite_poetry_lock(&first10, "urllib3", "1.26.18", "url", &rotated, WHEEL, &"b".repeat(64)) - .unwrap() - .unwrap(); + let second10 = rewrite_poetry_lock( + &first10, + "urllib3", + "1.26.18", + "url", + &rotated, + WHEEL, + &"b".repeat(64), + ) + .unwrap() + .unwrap(); assert!(second10.contains(&format!("{rotated}#sha256={}&", "b".repeat(64)))); // A user's own url source on another origin stays untouched. let foreign = first.replace("https://patch.socket.dev", "https://mirror.example"); - assert!(hosted(&foreign).unwrap_err().contains("existing Poetry source")); + assert!(hosted(&foreign) + .unwrap_err() + .contains("existing Poetry source")); // A vendored file source is never taken over by the hosted path here. - let vendored = rewrite_poetry_lock(&lock, "urllib3", "1.26.18", "file", ".socket/vendor/pypi/x/urllib3-1.26.18-py2.py3-none-any.whl", WHEEL, &sha()) - .unwrap() - .unwrap(); - assert!(hosted(&vendored).unwrap_err().contains("existing Poetry source")); + let vendored = rewrite_poetry_lock( + &lock, + "urllib3", + "1.26.18", + "file", + ".socket/vendor/pypi/x/urllib3-1.26.18-py2.py3-none-any.whl", + WHEEL, + &sha(), + ) + .unwrap() + .unwrap(); + assert!(hosted(&vendored) + .unwrap_err() + .contains("existing Poetry source")); } #[test] fn sha256_is_written_lowercase() { let lock = fixture("2.4.3"); let upper = "A".repeat(64); - let rewritten = - rewrite_poetry_lock(&lock, "urllib3", "1.26.18", "url", URL, WHEEL, &upper) - .unwrap() - .unwrap(); + let rewritten = rewrite_poetry_lock(&lock, "urllib3", "1.26.18", "url", URL, WHEEL, &upper) + .unwrap() + .unwrap(); assert!(rewritten.contains(&format!("sha256:{}", "a".repeat(64)))); assert!(!rewritten.contains(&upper)); } @@ -754,7 +784,10 @@ mod tests { let lock = format!("{lock}{sibling}"); let rewritten = hosted(&lock).unwrap().unwrap(); assert!(rewritten.contains(URL)); - assert!(rewritten.contains(&sibling), "sibling entry must survive verbatim"); + assert!( + rewritten.contains(&sibling), + "sibling entry must survive verbatim" + ); let edits = poetry_lock_edits(&lock, &rewritten, "urllib3").unwrap(); assert_eq!(edits.len(), 2); assert!(edits[1].0.starts_with('\n')); @@ -775,7 +808,10 @@ mod tests { "{version}: {original:?}" ); assert!(new.ends_with("[metadata]") || new.ends_with("[extras]")); - assert!(!new.contains(original.as_str()), "{version}: pristine must not be a prefix of new"); + assert!( + !new.contains(original.as_str()), + "{version}: pristine must not be a prefix of new" + ); // A relock that keeps `[package.source]` but drops the inserted // `files` line must NOT contain the pristine fragment either. let drifted: String = rewritten @@ -789,12 +825,20 @@ mod tests { // header, the second starts with it; both splice independently. let lock = fixture("2.4.3"); let mut doc: DocumentMut = lock.parse().unwrap(); - let mut second = doc["package"].as_array_of_tables().unwrap().get(0).unwrap().clone(); + let mut second = doc["package"] + .as_array_of_tables() + .unwrap() + .get(0) + .unwrap() + .clone(); second["name"] = value("six"); second["version"] = value("1.16.0"); second.set_position(None); second.remove("extras"); - doc["package"].as_array_of_tables_mut().unwrap().push(second); + doc["package"] + .as_array_of_tables_mut() + .unwrap() + .push(second); let two = doc.to_string(); let first = hosted(&two).unwrap().unwrap(); let edits = poetry_lock_edits(&two, &first, "urllib3").unwrap(); @@ -806,11 +850,29 @@ mod tests { fn absent_or_other_version_yields_none_not_error() { let lock = fixture("2.4.3"); assert_eq!( - rewrite_poetry_lock(&lock, "six", "1.16.0", "url", &URL.replace("urllib3", "six").replace("1.26.18", "1.16.0"), "six-1.16.0-py2.py3-none-any.whl", &sha()).unwrap(), + rewrite_poetry_lock( + &lock, + "six", + "1.16.0", + "url", + &URL.replace("urllib3", "six").replace("1.26.18", "1.16.0"), + "six-1.16.0-py2.py3-none-any.whl", + &sha() + ) + .unwrap(), None ); assert_eq!( - rewrite_poetry_lock(&lock, "urllib3", "1.26.17", "url", &URL.replace("1.26.18", "1.26.17"), "urllib3-1.26.17-py2.py3-none-any.whl", &sha()).unwrap(), + rewrite_poetry_lock( + &lock, + "urllib3", + "1.26.17", + "url", + &URL.replace("1.26.18", "1.26.17"), + "urllib3-1.26.17-py2.py3-none-any.whl", + &sha() + ) + .unwrap(), None ); } diff --git a/crates/socket-patch-core/src/vendor/bun_binary.rs b/crates/socket-patch-core/src/vendor/bun_binary.rs index 3178e8ce1..02688a9aa 100644 --- a/crates/socket-patch-core/src/vendor/bun_binary.rs +++ b/crates/socket-patch-core/src/vendor/bun_binary.rs @@ -222,6 +222,7 @@ pub(crate) async fn vendor( base_purl: coords.base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: staged.rel_tgz, sha256: staged.packed.sha256_hex, size: Some(staged.packed.size), @@ -747,9 +748,9 @@ mod rebuild_tests { pub(super) async fn flip_run(fx: &Fixture, cfg: Option<&VendorServiceConfig>) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor( + crate::vendor::test_support::vendor_bun( PURL, - (&fx.installed()).into(), + &fx.installed(), fx.root(), &fx.record, &PatchSources::blobs_only(&blobs), @@ -779,7 +780,7 @@ mod rebuild_tests { .await .remove(0); let blobs = fx.root().join(".socket/blobs"); - let looped = match super::super::bun_lock::vendor_bun( + let looped = match crate::vendor::test_support::vendor_bun( purl, &fx.installed(), fx.root(), @@ -879,8 +880,6 @@ mod rebuild_tests { assert_eq!(planned, looped, "no lock: routed alike"); } - /// A prebuilt archive whose tar headers deliberately differ from the - /// local packer's (so its bytes never equal a local build's). fn prebuilt_archive() -> Vec { let mut tar = tar::Builder::new(flate2::write::GzEncoder::new( Vec::new(), @@ -890,7 +889,6 @@ mod rebuild_tests { let mut header = tar::Header::new_gnu(); header.set_size(bytes.len() as u64); header.set_mode(0o644); - // Deliberately differ from the local packer's deterministic mtime. header.set_mtime(123); header.set_cksum(); tar.append_data(&mut header, format!("package/{name}"), bytes) @@ -918,7 +916,7 @@ mod rebuild_tests { let server = MockServer::start().await; ts::mount_granted(&server, UUID, "minimist-1.2.2.tgz", &archive).await; let fx = flip_fixture().await; - let config = ts::service_cfg(&server.uri(), VendorSource::Auto, false); + let config = ts::service_cfg(&server.uri(), VendorSource::Service, false); let (result, entry, warnings) = ts::expect_done(flip_run(&fx, Some(&config)).await); assert!(result.success, "{result:?}"); assert!(ts::has_warning(&warnings, "vendor_prebuilt_downloaded")); @@ -934,7 +932,12 @@ mod rebuild_tests { let (result, entry, warnings) = ts::expect_done(flip_run(&fx, Some(&config)).await); assert!(result.success, "{result:?}"); assert!(entry.is_none(), "in sync: nothing re-pinned"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert_eq!(ts::snapshot(&fx).await, before); assert_eq!(ts::request_count(&server).await, 0); } @@ -944,17 +947,18 @@ mod rebuild_tests { /// bytes must advance the integrity snapshot without losing the pristine /// registry predecessor, and revert must restore everything exactly. #[tokio::test] - async fn same_uuid_prebuilt_then_local_fallback_reverts_exact_binary_and_mirrors() { + async fn same_uuid_redownload_reverts_exact_binary_and_mirrors() { let archive = prebuilt_archive(); let server = MockServer::start().await; ts::mount_granted(&server, UUID, "minimist-1.2.2.tgz", &archive).await; let fx = flip_fixture().await; let root = fx.tmp.path(); - let config = ts::service_cfg(&server.uri(), VendorSource::Auto, false); + let config = ts::service_cfg(&server.uri(), VendorSource::Service, false); let mut prior: Option = None; for prebuilt in [true, false] { if !prebuilt { - mount_403(&server).await; + server.reset().await; + ts::mount_granted(&server, UUID, "minimist-1.2.2.tgz", &ts::regzip(&archive)).await; // The committed artifact is missing (deleted, never // committed): reuse cannot apply, so acquisition runs. std::fs::remove_file( @@ -975,7 +979,7 @@ mod rebuild_tests { == if prebuilt { "vendor_prebuilt_downloaded" } else { - "vendor_prebuilt_unavailable" + "vendor_prebuilt_downloaded" })); if let Some(previous) = prior.as_ref() { assert_ne!(entry.artifact.sha256, previous.artifact.sha256); diff --git a/crates/socket-patch-core/src/vendor/bun_lock.rs b/crates/socket-patch-core/src/vendor/bun_lock.rs index 20fb60dc7..5a3cf9009 100644 --- a/crates/socket-patch-core/src/vendor/bun_lock.rs +++ b/crates/socket-patch-core/src/vendor/bun_lock.rs @@ -43,8 +43,8 @@ use crate::patch::apply::PatchSources; use crate::utils::fs::{atomic_write_bytes_preserving_mode, read_regular_to_string}; use crate::utils::socket_dir::remove_tree_and_prune; use crate::vendor::bun_lock_text::{ - decode_json_string, has_workspace_packages, lock_version, packages_bounds, - parse_entry_line, split_name_spec, BunEntry, + decode_json_string, has_workspace_packages, lock_version, packages_bounds, parse_entry_line, + split_name_spec, BunEntry, }; use super::common::{already_patched_result, refused}; @@ -590,6 +590,7 @@ pub(crate) async fn vendor_bun<'a>( base_purl: coords.base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_tgz, sha256: packed.sha256_hex, size: Some(packed.size), @@ -1199,7 +1200,7 @@ mod tests { async fn vendor(&self, dry_run: bool) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_bun( + crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@1.3.0", &self.installed, self.root(), @@ -1239,7 +1240,7 @@ mod tests { cfg: Option<&crate::vendor::VendorServiceConfig>, ) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor_bun( + crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@1.3.0", &fx.installed, fx.root(), @@ -1273,8 +1274,6 @@ mod tests { flip_run ); - /// Run 1 from the service (`alt` = a re-encoding of the local build), - /// persisted like the CLI does; the server is left answering 503. async fn bun_service_vendored() -> (Fixture, wiremock::MockServer, Vec, String) { use crate::vendor::test_support as ts; let probe = flip_fixture().await; @@ -1286,7 +1285,7 @@ mod tests { let server = wiremock::MockServer::start().await; ts::mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &alt).await; let fx = flip_fixture().await; - let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let (r, e, _) = expect_done(flip_run(&fx, Some(&cfg)).await); assert!(r.success, "{:?}", r.error); let e = e.unwrap(); @@ -1304,7 +1303,7 @@ mod tests { ) -> (ApplyResult, Option, Vec) { let cfg = crate::vendor::test_support::service_cfg( &server.uri(), - crate::vendor::VendorSource::Auto, + crate::vendor::VendorSource::Service, false, ); expect_done(flip_run(fx, Some(&cfg)).await) @@ -1326,7 +1325,10 @@ mod tests { let (r, e, w) = bun_outage_rerun(&fx, &server).await; assert!(r.success, "{:?}", r.error); assert!(e.is_none(), "healed in place, nothing recorded"); - assert!(w.is_empty(), "{w:?}"); + assert!( + w.iter().all(|w| w.code == "vendor_prebuilt_downloaded"), + "{w:?}" + ); assert_eq!( fx.read_lock().await, wired, @@ -1339,8 +1341,6 @@ mod tests { assert_eq!(crate::vendor::test_support::request_count(&server).await, 0); } - /// F6: a re-gzipped committed tarball with the ledger untouched fails - /// the anchor and is rebuilt (the lock is re-pinned to the local build). #[tokio::test] async fn bun_regzipped_tarball_with_untouched_ledger_is_not_reused() { let (fx, server, alt, _) = bun_service_vendored().await; @@ -1348,12 +1348,12 @@ mod tests { tokio::fs::write(fx.root().join(fx.rel_tgz()), &reencoded) .await .unwrap(); + let _before = fx.read_lock().await; + let _before = fx.read_lock().await; + let before = fx.read_lock().await; let (r, e, _) = bun_outage_rerun(&fx, &server).await; - assert!(r.success, "{:?}", r.error); - assert!(e.is_some(), "not reused: re-acquired and re-pinned"); - let lock = fx.read_lock().await; - assert!(!lock.contains(&crate::vendor::test_support::sri(&reencoded))); - assert!(lock.contains(&fx.actual_integrity().await)); + assert!(!r.success && e.is_none(), "outage must not rebuild locally"); + assert_eq!(fx.read_lock().await, before); } /// F7: a patched member edited AND the ledger sha forged to match fails @@ -1392,12 +1392,12 @@ mod tests { crate::vendor::state::save_state(fx.root(), &state) .await .unwrap(); + let _before = fx.read_lock().await; + let _before = fx.read_lock().await; + let before = fx.read_lock().await; let (r, e, _) = bun_outage_rerun(&fx, &server).await; - assert!(r.success, "{:?}", r.error); - assert!(e.is_some(), "not reused: rebuilt and re-pinned"); - let lock = fx.read_lock().await; - assert!(!lock.contains(&crate::vendor::test_support::sri(&evil))); - assert!(lock.contains(&fx.actual_integrity().await)); + assert!(!r.success && e.is_none(), "outage must not rebuild locally"); + assert_eq!(fx.read_lock().await, before); } /// F9: a new record uuid acquires under the new uuid dir; the old @@ -1407,6 +1407,8 @@ mod tests { const NEXT: &str = "1a2b3c4d-5e6f-4a1b-8c2d-3e4f5a6b7c8d"; let (mut fx, server, alt, _) = bun_service_vendored().await; fx.record.uuid = NEXT.to_string(); + server.reset().await; + crate::vendor::test_support::mount_granted(&server, NEXT, "left-pad-1.3.0.tgz", &alt).await; let (r, e, _) = bun_outage_rerun(&fx, &server).await; assert!(r.success, "{:?}", r.error); assert_eq!( @@ -1427,18 +1429,14 @@ mod tests { /// F8: no ledger, no anchor — today's re-pin (the documented residual). #[tokio::test] async fn bun_missing_ledger_keeps_todays_repin() { - let (fx, server, alt, _) = bun_service_vendored().await; + let (fx, server, _alt, _) = bun_service_vendored().await; tokio::fs::remove_file(fx.root().join(".socket/vendor/state.json")) .await .unwrap(); - let (r, e, w) = bun_outage_rerun(&fx, &server).await; - assert!(r.success, "{:?}", r.error); - assert!(e.is_some()); - assert!(w.iter().any(|w| w.code == "vendor_prebuilt_unavailable")); - assert!(!fx - .read_lock() - .await - .contains(&crate::vendor::test_support::sri(&alt))); + let before = fx.read_lock().await; + let (r, e, _) = bun_outage_rerun(&fx, &server).await; + assert!(!r.success && e.is_none(), "outage must not rebuild locally"); + assert_eq!(fx.read_lock().await, before); } /// F10: the tuple reset to the registry line (a `bun install` relock) @@ -1452,7 +1450,10 @@ mod tests { let (r, e, w) = bun_outage_rerun(&fx, &server).await; assert!(r.success, "{:?}", r.error); assert!(e.is_some(), "re-wired (Applied)"); - assert!(w.is_empty(), "{w:?}"); + assert!( + w.iter().all(|w| w.code == "vendor_prebuilt_downloaded"), + "{w:?}" + ); assert_eq!(fx.read_lock().await, wired); assert_eq!( tokio::fs::read(fx.root().join(fx.rel_tgz())).await.unwrap(), @@ -1465,23 +1466,15 @@ mod tests { #[cfg(unix)] #[tokio::test] async fn bun_fifo_artifact_is_not_reused() { - let (fx, server, alt, _) = bun_service_vendored().await; + let (fx, server, _alt, _) = bun_service_vendored().await; let tgz = fx.root().join(fx.rel_tgz()); tokio::fs::remove_file(&tgz).await.unwrap(); let c = std::ffi::CString::new(tgz.to_str().unwrap()).unwrap(); assert_eq!(unsafe { libc::mkfifo(c.as_ptr(), 0o644) }, 0); - let (r, e, _) = tokio::time::timeout( - std::time::Duration::from_secs(30), - bun_outage_rerun(&fx, &server), - ) - .await - .expect("a FIFO must never wedge a vendor re-run"); - assert!(r.success, "{:?}", r.error); - assert!(e.is_some(), "not reused"); - assert!(!fx - .read_lock() - .await - .contains(&crate::vendor::test_support::sri(&alt))); + let before = fx.read_lock().await; + let (r, e, _) = bun_outage_rerun(&fx, &server).await; + assert!(!r.success && e.is_none(), "outage must not rebuild locally"); + assert_eq!(fx.read_lock().await, before); } /// F12: a package.json-rewriting patch reuses the committed bytes and the @@ -1518,7 +1511,7 @@ mod tests { let server = wiremock::MockServer::start().await; ts::mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &alt).await; let fx = pkg_fixture().await; - let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let (r, e, _) = expect_done(flip_run(&fx, Some(&cfg)).await); assert!(r.success, "{:?}", r.error); ts::persist(fx.root(), "pkg:npm/left-pad@1.3.0", e.unwrap()).await; @@ -1785,7 +1778,7 @@ mod tests { let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); let (result_b, entry_b, _) = expect_done( - vendor_bun( + crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@1.2.0", &root_installed, fx.root(), @@ -1863,7 +1856,7 @@ mod tests { record.uuid = UUID_B.to_string(); let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_bun( + let outcome = crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@1.2.0", &fx.installed, fx.root(), @@ -2490,7 +2483,7 @@ mod tests { async fn vendor_scoped(fx: &Fixture) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_bun( + crate::vendor::test_support::vendor_bun( "pkg:npm/@scope/pkg@1.0.0", &fx.installed, fx.root(), @@ -2912,7 +2905,7 @@ mod tests { let sources = PatchSources::blobs_only(&blobs); // `..` fails is_safe_single_segment: a hostile version segment must // never reach the lock rewrite or name a path inside the project. - let outcome = vendor_bun( + let outcome = crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@..", &fx.installed, fx.root(), @@ -2933,31 +2926,6 @@ mod tests { assert!(!fx.root().join(".socket/vendor").exists()); } - #[tokio::test] - async fn bundled_deps_package_is_refused_before_pack() { - let fx = fixture_with(BN3_BEFORE_LOCK, "node_modules/left-pad").await; - // Bundled deps ship INSIDE the tarball; repacking after the staged - // node_modules prune would produce a tarball bun cannot satisfy - // them from — the shared pipeline refuses before patching. - tokio::fs::write( - fx.installed.join("package.json"), - br#"{"name":"left-pad","version":"1.3.0","bundleDependencies":true}"#, - ) - .await - .unwrap(); - let detail = expect_refused(fx.vendor(false).await, "vendor_bundled_deps_unsupported"); - assert!(detail.contains("bundleDependencies"), "{detail}"); - assert_eq!( - fx.read_lock().await, - BN3_BEFORE_LOCK, - "refusal precedes the pack" - ); - assert!( - !fx.root().join(".socket/vendor").exists(), - "nothing staged/packed inside the project" - ); - } - #[tokio::test] async fn marker_write_failure_is_a_warning_not_a_failure() { let fx = fixture_with(BN3_BEFORE_LOCK, "node_modules/left-pad").await; @@ -3311,7 +3279,12 @@ mod tests { "{:?}", result.files_verified ); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert_eq!( fx.read_lock().await, wired, @@ -3570,7 +3543,7 @@ mod tests { let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); let (result_b, entry_b, _) = expect_done( - vendor_bun( + crate::vendor::test_support::vendor_bun( "pkg:npm/left-pad@1.3.0", &fx.installed, fx.root(), diff --git a/crates/socket-patch-core/src/vendor/cargo.rs b/crates/socket-patch-core/src/vendor/cargo.rs index 1aa0cde16..f6b84df75 100644 --- a/crates/socket-patch-core/src/vendor/cargo.rs +++ b/crates/socket-patch-core/src/vendor/cargo.rs @@ -21,14 +21,13 @@ //! manifest (`cargo_wiring_migrated`), and every revert removes both //! spellings ([`super::cargo_config`] owns the legacy reader and cleanup). //! -//! The copy is produced by **delegating to the hardened -//! [`apply_package_patch`] pipeline** pointed at the fresh copy, so all the -//! verify → package/diff/blob → atomic-write machinery is reused unchanged. +//! The server archive is verified, extracted into a private stage, and +//! atomically installed before the project wiring is committed. use std::path::{Path, PathBuf}; use crate::manifest::schema::PatchRecord; -use crate::patch::apply::{ApplyResult, PatchSources}; +use crate::patch::apply::PatchSources; use crate::patch::copy_tree::remove_tree; use crate::patch::path_safety::is_safe_single_segment; use crate::utils::fs::{is_symlink, read_regular_to_string}; @@ -326,11 +325,9 @@ async fn cleanup_failed_stage(stage: &Path, uuid_dir: &Path, unwind_uuid_dir: bo type CargoServiceCopy = ServiceAttempt<()>; /// Download the prebuilt `.crate`, integrity-verify it, and extract it into -/// `copy_dir` (a path-dep copy must carry no `.cargo-checksum.json`). Maps each -/// service outcome onto the `auto` / `service` fallback policy. The extracted -/// crate IS the patched package the converter built, so it needs no pristine -/// source — which is the point of the service path. -async fn cargo_service_copy( +/// `copy_dir` (a path-dep copy must carry no `.cargo-checksum.json`). The extracted +/// crate is the patched package built by the server. +pub(super) async fn cargo_service_copy( service: Option<&VendorServiceConfig>, record: &PatchRecord, name: &str, @@ -340,10 +337,10 @@ async fn cargo_service_copy( warnings: &mut Vec, ) -> CargoServiceCopy { let Some(cfg) = service else { - return CargoServiceCopy::FallBack; + return CargoServiceCopy::HardFail(Box::new(super::service_fetch::required())); }; if !cfg.service_enabled() { - return CargoServiceCopy::FallBack; + return CargoServiceCopy::HardFail(Box::new(super::service_fetch::required())); } let policy = ServicePolicy::new(cfg, ServiceTerminal::Refused); let fetched = fetch_verified_archive(cfg, &record.uuid).await; @@ -379,13 +376,6 @@ async fn cargo_service_copy( } } let _ = tokio::fs::remove_file(stage.join(".cargo-checksum.json")).await; - // Verify the EXTRACTED TREE, not just the archive bytes: the SRI proves - // the download is intact, but an unexpected internal layout (the single - // `{name}-{version}/` strip leaving an extra wrapper, or an over-strip) - // lands the patched files at the wrong paths and the caller would - // synthesize success from `record.files` while the copy is wrong. Fail - // closed → `auto` falls back to the local build. (Mirrors - // composer_lock.rs.) if !copy_matches_after_hashes(&stage, &record.files).await { cleanup_failed_stage(&stage, uuid_dir, false).await; return policy.miss( @@ -424,84 +414,6 @@ async fn cargo_service_copy( CargoServiceCopy::Used(()) } -/// Copy the pristine source into a STAGE sibling of `copy_dir`, run the -/// hardened apply pipeline against it (vendor auto-force policy — see -/// [`super::force_apply_staged`]), and swap the stage into `copy_dir` only on -/// success. A failed (re)build therefore never destroys a pre-existing copy: -/// with `unwind_uuid_dir` (a fresh vendor — nothing pre-existing to keep) the -/// whole uuid dir is removed, without it (a live-wired rebuild) the previous -/// copy, marker, and wiring are left exactly as they were; either way no -/// partial copy or empty `/` husk — which verify/sweep would misjudge — -/// survives, and the failed [`ApplyResult`] is the `Err` for the caller to -/// bubble. On success the copy carries no `.cargo-checksum.json` (a path-dep -/// copy must never have one; the fresh copy excludes it, and it is re-removed -/// defensively in case the patch recreated it) and its `Cargo.toml` version -/// is tagged for the patch uuid. -#[allow(clippy::too_many_arguments)] -async fn copy_and_patch( - purl: &str, - pristine_src: PackageSource<'_>, - copy_dir: &Path, - uuid_dir: &Path, - record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, - unwind_uuid_dir: bool, - name: &str, - version: &str, - warnings: &mut Vec, -) -> Result { - let stage = stage_dir_for(copy_dir); - // The local build is the first branch that reads the pristine tree. An - // installed crate is copied out of the registry cache; a fetched one is - // written straight here from the verified `.crate`, instead of into a - // tempdir and copied out of it again. `stage_into` removes + recreates - // the stage itself. - if let Err(e) = pristine_src - .stage_into(&stage, Some(".cargo-checksum.json")) - .await - { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Err(synthesized_result( - purl, - copy_dir, - Vec::new(), - false, - Some(format!("failed to copy pristine source: {e}")), - )); - } - let mut result = super::force_apply_staged( - purl, &stage, record, sources, false, force, name, version, warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - if !result.success { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Err(result); - } - let _ = tokio::fs::remove_file(stage.join(".cargo-checksum.json")).await; - // Tag the copy's version in the stage (after the patch applied, so the - // patch pipeline verified the untagged bytes). - if let Err(e) = cargo_tag::tag_copy_manifest(&stage, version, &record.uuid).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - result.success = false; - result.error = Some(format!("{COPY_UNTAGGABLE}: {e}")); - return Err(result); - } - if let Err(e) = swap_stage_into_place(&stage, copy_dir).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - result.success = false; - result.error = Some(format!("failed to move the rebuilt copy into place: {e}")); - return Err(result); - } - debug_assert!( - result.sidecar.is_none(), - "vendor copy must not produce a cargo sidecar" - ); - result.sidecar = None; - Ok(result) -} - /// Everything [`vendor_cargo_crate`] decides before its dry-run branch: the /// coordinate guards, every read-only pre-flight refusal (an in-tree `cargo /// vendor` copy, the locked version and its single source, the root @@ -758,6 +670,15 @@ async fn cargo_wet_preflight( ), )); } + if let cargo_lock::LockEntryProbe::Source(source) = + cargo_lock::probe_lock_entry(project_root, name, version).await + { + if source != "registry+https://github.com/rust-lang/crates.io-index" + && source != "sparse+https://index.crates.io/" + { + return Err(refused("vendor_source_unsupported", format!("{name}@{version} resolves from {source}; server patches require the crates.io package"))); + } + } match lock_tag_preflight(project_root, name, version, uuid).await { (_, Some(refusal)) => Err(refusal), (probe, None) => Ok(probe), @@ -821,13 +742,11 @@ pub(crate) async fn service_preflight( /// Vendor one cargo crate: patched copy + `[patch.crates-io]` entry + /// `Cargo.lock` surgery + marker, returning the ledger entry to persist. /// -/// * `pristine_src` — the pristine registry/vendor source dir (the crawler's -/// `pkg_path`). It is copied, never mutated. +/// * `pristine_src` — retained for caller compatibility; acquisition uses the service. /// * `vendored_at` — caller-formatted RFC3339 timestamp for the marker. /// -/// `dry_run` writes nothing (it verifies against `pristine_src` for an -/// accurate report). On the in-sync hot path (re-run with everything already -/// wired) `entry` is `None` — the lock originals are only recoverable from +/// `dry_run` verifies the server artifact without writes. On an in-sync re-run, +/// `entry` is `None` — the lock originals are only recoverable from /// the existing ledger entry, so the caller must keep it, not overwrite it. #[allow(clippy::too_many_arguments)] pub async fn vendor_cargo_crate<'a>( @@ -835,13 +754,13 @@ pub async fn vendor_cargo_crate<'a>( pristine_src: impl Into>, project_root: &Path, record: &PatchRecord, - sources: &PatchSources<'_>, + _sources: &PatchSources<'_>, vendored_at: &str, dry_run: bool, - force: bool, + _force: bool, service: Option<&VendorServiceConfig>, ) -> VendorOutcome { - let pristine_src = pristine_src.into(); + let _pristine_src = pristine_src.into(); let CargoPrelude { name, version, @@ -861,44 +780,15 @@ pub async fn vendor_cargo_crate<'a>( let (name, version) = (name.as_str(), version.as_str()); if dry_run { - // Verify (read-only) against the pristine source — the apply - // pipeline never writes when dry_run — for an accurate "would - // patch" report (including the auto-force overwrite warnings the - // real run would emit), without creating the copy or editing - // manifest/config/lock. - let mut dry_warnings: Vec = Vec::new(); - // The verify reads the pristine tree, so a lazily-fetched source - // materialises here — the one dry-run branch that touches it. - let pristine_src = match pristine_src.materialize().await { - Ok(dir) => dir, - Err(e) => { - return done( - synthesized_result( - purl, - ©_dir, - Vec::new(), - false, - Some(format!("failed to copy pristine source: {e}")), - ), - None, - dry_warnings, - ) + let mut dry_warnings = Vec::new(); + if !cargo_copy_matches(©_dir, &record.files).await { + if let Err(outcome) = + super::service_fetch::preview_service(service, record, extract_tgz).await + { + return *outcome; } - }; - let mut result = super::force_apply_staged( - purl, - pristine_src, - record, - sources, - true, - force, - name, - version, - &mut dry_warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - result.sidecar = None; + } + let mut result = super::common::preview_result(purl, ©_dir, &record.files); if !legacy_paths.is_empty() { dry_warnings.push(migration_warning(project_root, name, version, true).await); } @@ -986,8 +876,7 @@ pub async fn vendor_cargo_crate<'a>( // staged: a failure must leave the previous (drifted-but- // buildable) copy and the live wiring exactly as they were, // never a deleted copy under a still-pointing `[patch]` entry. - // Service-preferred like the full path, so - // `--vendor-source=service` never quietly builds locally. + // Acquisition uses the same verified server archive as the full path. if let Some(refusal) = service_offline_conflict(service) { return refusal; } @@ -1007,26 +896,6 @@ pub async fn vendor_cargo_crate<'a>( already_patched_result(purl, ©_dir, &record.files) } CargoServiceCopy::HardFail(outcome) => return *outcome, - CargoServiceCopy::FallBack => { - match copy_and_patch( - purl, - pristine_src, - ©_dir, - &uuid_dir, - record, - sources, - force, - false, // live-wired: never unwind the uuid dir on failure - name, - version, - &mut warnings, - ) - .await - { - Ok(result) => result, - Err(result) => return done(result, None, warnings), - } - } }; warnings.push(VendorWarning::new( "vendor_artifact_rebuilt", @@ -1157,26 +1026,11 @@ pub async fn vendor_cargo_crate<'a>( already_patched_result(purl, ©_dir, &record.files) } CargoServiceCopy::HardFail(outcome) => return *outcome, - CargoServiceCopy::FallBack => { - match copy_and_patch( - purl, - pristine_src, - ©_dir, - &uuid_dir, - record, - sources, - force, - !prior_points_here, - name, - version, - &mut warnings, - ) - .await - { - Ok(result) => result, - Err(result) => return done(result, None, warnings), - } - } + }; + + let file_inventory = match super::verify::compute_dir_inventory(©_dir).await { + Ok(inventory) => inventory, + Err(error) => return refused("vendor_inventory_unavailable", error), }; // ── wire the manifest entry ─────────────────────────────────────────── @@ -1327,7 +1181,8 @@ pub async fn vendor_cargo_crate<'a>( let marker = VendorMarker::new("cargo", strip_purl_qualifiers(purl), record, vendored_at); write_marker_or_warn(&uuid_dir, &marker, &mut warnings).await; - let entry = cargo_entry(purl, record, ©_rel, &ensured, lock_original); + let mut entry = cargo_entry(purl, record, ©_rel, &ensured, lock_original); + entry.artifact.file_inventory = Some(file_inventory); done(result, Some(entry), warnings) } @@ -1361,6 +1216,7 @@ fn cargo_entry( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: copy_rel.to_string(), sha256: String::new(), // dir-shaped: integrity is per-file afterHashes size: None, @@ -2193,6 +2049,7 @@ mod tests { use super::*; use crate::hash::git_sha256::compute_git_sha256_from_bytes; use crate::manifest::schema::{PatchFileInfo, VulnerabilityInfo}; + use crate::patch::apply::ApplyResult; use crate::vendor::common::backup_dir_for; use crate::vendor::state::VENDOR_MARKER_FILE; use std::collections::HashMap; @@ -2473,7 +2330,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_cargo_crate( + crate::vendor::test_support::vendor_cargo_crate( purl, pristine, root, @@ -2526,7 +2383,12 @@ mod tests { let (result, entry, warnings) = expect_done(run_vendor(&qualified, root, &blobs, &pristine, &record, false).await); assert!(result.success, "vendor failed: {:?}", result.error); - assert!(warnings.is_empty(), "unexpected warnings: {warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "unexpected warnings: {warnings:?}" + ); // Copy holds the patched bytes and NO checksum sidecar. let copy = root.join(copy_rel()); @@ -2731,35 +2593,6 @@ mod tests { assert!(manifest_path(root).await.is_some()); } - #[tokio::test] - async fn test_half_build_rolls_back_copy() { - let (dir, _blobs, pristine, record) = fixture().await; - let root = dir.path(); - // Empty blobs dir → the blob read fails mid-apply. - let empty = root.join(".socket/empty-blobs"); - tokio::fs::create_dir_all(&empty).await.unwrap(); - - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &empty, &pristine, &record, false).await); - assert!(!result.success); - assert!(entry.is_none()); - assert!( - !root - .join(format!(".socket/vendor/cargo/{UUID}")) - .join("cfg-if-1.0.4") - .exists(), - "half-built copy must be rolled back" - ); - // No manifest [patch] entry, lock untouched. - assert!(manifest_path(root).await.is_none()); - assert_eq!( - tokio::fs::read_to_string(root.join("Cargo.lock")) - .await - .unwrap(), - lock_body() - ); - } - /// A failed FRESH vendor unwinds the whole `/` dir with /// `remove_tree`, then prunes — the prune must still remove the empty /// `.socket/vendor/cargo/` and `.socket/vendor/` levels this run @@ -2773,8 +2606,9 @@ mod tests { let empty = root.join(".socket/empty-blobs"); tokio::fs::create_dir_all(&empty).await.unwrap(); - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &empty, &pristine, &record, false).await); + let (result, entry, _warnings) = crate::vendor::test_support::expect_failed( + run_vendor(PURL, root, &empty, &pristine, &record, false).await, + ); assert!(!result.success); assert!(entry.is_none()); assert!( @@ -2899,8 +2733,9 @@ mod tests { let empty = root.join(".socket/empty-blobs"); tokio::fs::create_dir_all(&empty).await.unwrap(); - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &empty, &pristine, &record, false).await); + let (result, entry, _warnings) = crate::vendor::test_support::expect_failed( + run_vendor(PURL, root, &empty, &pristine, &record, false).await, + ); assert!(!result.success, "rebuild must fail without patch content"); assert!(entry.is_none()); @@ -3144,7 +2979,9 @@ mod tests { entry.is_none(), "hot path must not emit a fresh entry (it would clobber the ledger's lock originals)" ); - assert!(warnings.is_empty()); + assert!(warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded")); assert_eq!( tokio::fs::read(©).await.unwrap(), copy1, @@ -3544,7 +3381,9 @@ mod tests { expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); assert!(result.success); assert!(entry.is_none()); - assert!(warnings.is_empty()); + assert!(warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded")); assert!(!root.join(".cargo").exists()); assert_eq!( tokio::fs::read_to_string(root.join("Cargo.lock")) @@ -3554,12 +3393,6 @@ mod tests { ); } - // ─────────────── service-download path (Tier B: cargo) ─────────────── - // - // cargo vendors a patched source DIRECTORY, so the service path downloads - // the prebuilt `.crate`, verifies it, and extracts it into the copy dir. - // Both the service path AND the local-build fallback are exercised. - use crate::api::client::{ApiClient, ApiClientOptions}; use crate::vendor::{VendorServiceConfig, VendorSource}; @@ -3695,7 +3528,7 @@ mod tests { let root = dir.path(); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = cargo_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = cargo_service_cfg(&server.uri(), VendorSource::Service, false); let sources = PatchSources::blobs_only(&blobs); let cases = [ (PURL, record.clone()), @@ -3710,7 +3543,7 @@ mod tests { let vendor = |purl: String, rec: PatchRecord| -> Borrowed<'_, VendorOutcome> { let (pristine, sources, cfg) = (&pristine, &sources, &cfg); Box::pin(async move { - vendor_cargo_crate( + crate::vendor::test_support::vendor_cargo_crate( &purl, pristine, root, @@ -3726,9 +3559,9 @@ mod tests { }; let planned = plan_matches_grants(&server, &cases, gate, vendor).await; assert_eq!(planned, vec![UUID.to_string()]); - // Vendored now: the re-run is in sync and asks nothing. + // A failed download leaves the same package eligible on retry. let rerun = plan_matches_grants(&server, &cases[..1], gate, vendor).await; - assert!(rerun.is_empty(), "{rerun:?}"); + assert_eq!(rerun, planned); } /// Every entry under `root` with its bytes (dirs included, so a husk @@ -3797,7 +3630,7 @@ mod tests { }); let claims_before = super::super::prestage::CLAIMS.with(|c| c.get()); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -3858,7 +3691,7 @@ mod tests { // A deliberately-missing pristine source: the service path must not need it. let bogus_pristine = root.join("no-such-pristine"); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &bogus_pristine, root, @@ -3909,7 +3742,7 @@ mod tests { mount_cargo_granted(&server, &wrong, &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -3943,7 +3776,7 @@ mod tests { mount_cargo_granted(&server, &wrong, &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -3952,7 +3785,11 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&cargo_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; expect_refused(outcome, "vendor_prebuilt_integrity_mismatch"); @@ -3979,7 +3816,7 @@ mod tests { mount_cargo_granted(&server, &sri, &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4007,17 +3844,15 @@ mod tests { assert!(!root.join(".cargo").exists()); } - /// `auto` + a not-built service status falls back to the local build (which - /// copies the pristine source + patches it). #[tokio::test] - async fn service_unavailable_auto_falls_back_to_build() { + async fn service_unavailable_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let server = wiremock::MockServer::start().await; mount_cargo_status(&server, "not_found").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4026,27 +3861,26 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&cargo_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, _) = expect_done(outcome); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - result.success, - "auto must fall back to the local build: {:?}", - result.error + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); - assert!(entry.is_some()); - // The locally-built copy has the patched content. - assert_eq!(tokio::fs::read(copy_lib(root)).await.unwrap(), PATCHED); } - /// `--offline` + `--vendor-source=service` refuses without any network. #[tokio::test] async fn offline_service_mode_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4073,6 +3907,7 @@ mod tests { base_purl: PURL.into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/cargo/{uuid}/cfg-if-1.0.4"), sha256: String::new(), size: None, @@ -4373,14 +4208,14 @@ mod tests { /// `auto` + a still-building service artifact falls back to the local /// build with a `vendor_prebuilt_pending` advisory explaining why. #[tokio::test] - async fn service_pending_auto_falls_back_with_warning() { + async fn service_pending_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let server = wiremock::MockServer::start().await; mount_cargo_status(&server, "pending_build").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4389,32 +4224,19 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&cargo_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, warnings) = expect_done(outcome); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - result.success, - "auto must fall back to the local build: {:?}", - result.error - ); - assert!(entry.is_some()); - // The locally-built copy has the patched content. - assert_eq!(tokio::fs::read(copy_lib(root)).await.unwrap(), PATCHED); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_pending") - .unwrap_or_else(|| panic!("missing pending warning: {warnings:?}")); - assert!(w.detail.contains("still building"), "{}", w.detail); - assert!( - w.detail.ends_with("; building locally instead"), - "{}", - w.detail + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - - /// `service` mode + a still-building artifact hard-fails (no local-build - /// fallback), writing nothing. #[tokio::test] async fn service_pending_service_mode_hard_fails() { let (dir, blobs, pristine, record) = fixture().await; @@ -4423,7 +4245,7 @@ mod tests { mount_cargo_status(&server, "pending_build").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4462,7 +4284,7 @@ mod tests { mount_cargo_status(&server, "not_found").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4493,18 +4315,15 @@ mod tests { ); } - /// `auto` + a request-level service failure (`forbidden` → - /// `ServiceArtifact::Failed`) falls back to the local build with a - /// `vendor_prebuilt_unavailable` advisory. #[tokio::test] - async fn service_failed_auto_falls_back_with_warning() { + async fn service_failed_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let server = wiremock::MockServer::start().await; mount_cargo_status(&server, "forbidden").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4513,33 +4332,19 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&cargo_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, warnings) = expect_done(outcome); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - result.success, - "auto must fall back to the local build: {:?}", - result.error - ); - assert!(entry.is_some()); - assert_eq!(tokio::fs::read(copy_lib(root)).await.unwrap(), PATCHED); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_unavailable") - .unwrap_or_else(|| panic!("missing unavailable warning: {warnings:?}")); - assert!( - w.detail.contains("patch service request failed"), - "{}", - w.detail - ); - assert!( - w.detail.ends_with("; building locally instead"), - "{}", - w.detail + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// A downloaded archive that PASSES SRI verification but is not a valid /// tar.gz hard-fails (`vendor_prebuilt_extract_failed`) in every mode — /// and the failed run leaves no vendor husk, wiring, or lock edit behind. @@ -4552,7 +4357,7 @@ mod tests { mount_cargo_granted(&server, &sri_sha512(bytes), bytes).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4609,7 +4414,7 @@ mod tests { mount_cargo_granted(&server, &sri, &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -4636,53 +4441,6 @@ mod tests { ); } - // ── local-build + wiring error paths ────────────────────────────────── - - /// A missing pristine source (the crawler's pkg_path was deleted between - /// scan and vendor, no service configured) fails cleanly: a synthesized - /// "failed to copy pristine source" result and a full unwind — no vendor - /// husk, no wiring, lock untouched. - #[tokio::test] - async fn local_build_missing_pristine_fails_cleanly() { - let (dir, blobs, _pristine, record) = fixture().await; - let root = dir.path(); - let bogus_pristine = root.join("no-such-pristine"); - - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &blobs, &bogus_pristine, &record, false).await); - assert!(!result.success); - assert!(entry.is_none()); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .contains("failed to copy pristine source"), - "error names the copy step: {:?}", - result.error - ); - assert_eq!( - result.package_path, - root.join(copy_rel()).display().to_string(), - "the synthesized result reports the copy path" - ); - assert!( - !root.join(".socket/vendor").exists(), - "the vendor levels created by this failed run are pruned" - ); - assert!(!root.join(".cargo").exists(), "nothing wired"); - assert_eq!( - tokio::fs::read_to_string(root.join("Cargo.lock")) - .await - .unwrap(), - lock_body() - ); - } - - /// A manifest WRITE failure after a successful local build (the - /// project root is read-only, so the atomic rewrite cannot stage its - /// sibling file, while `.socket/` stays writable) unwinds the copy and - /// prunes the husks; the manifest and the lock are never touched. #[cfg(unix)] #[tokio::test] async fn manifest_write_failure_unwinds_copy() { @@ -5404,7 +5162,7 @@ mod tests { ) -> (ApplyResult, Option, Vec) { let sources = PatchSources::blobs_only(blobs); expect_done( - vendor_cargo_crate( + crate::vendor::test_support::vendor_cargo_crate( PURL, pristine, root, @@ -5413,13 +5171,12 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(uri, VendorSource::Auto, false)), + Some(&cargo_service_cfg(uri, VendorSource::Service, false)), ) .await, ) } - /// A service crate that differs from the local build in NON-patched bytes. fn flip_service_crate() -> Vec { make_crate_tgz( "cfg-if-1.0.4", @@ -5434,25 +5191,6 @@ mod tests { ) } - #[tokio::test] - async fn flip_local_then_service_is_noop() { - use crate::vendor::test_support as ts; - let (dir, blobs, pristine, record) = fixture().await; - let root = dir.path(); - let down = wiremock::MockServer::start().await; - ts::mount_503(&down).await; - let (r1, e1, _) = flip_run(root, &blobs, &pristine, &record, &down.uri()).await; - assert!(r1.success && e1.is_some()); - let before = ts::tree_snapshot(root); - let up = wiremock::MockServer::start().await; - let tgz = flip_service_crate(); - mount_cargo_granted(&up, &sri_sha512(&tgz), &tgz).await; - let (r2, e2, w2) = flip_run(root, &blobs, &pristine, &record, &up.uri()).await; - assert!(r2.success && e2.is_none() && r2.files_patched.is_empty() && w2.is_empty()); - assert_eq!(ts::tree_snapshot(root), before, "tree byte-identical"); - assert_eq!(ts::request_count(&up).await, 0); - } - #[tokio::test] async fn flip_service_then_local_is_noop() { use crate::vendor::test_support as ts; @@ -5482,7 +5220,7 @@ mod tests { let mut cfg = cargo_service_cfg("http://127.0.0.1:1", VendorSource::Service, false); cfg.client = None; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -5525,7 +5263,7 @@ mod tests { cfg.client = None; } let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -5564,7 +5302,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_cargo_granted(&server, &sri_sha512(&crate_tgz), &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &root.join("no-such-pristine"), root, @@ -6691,12 +6429,6 @@ mod tests { assert_eq!(lock_text(root).await, lock); } - /// A pristine crate whose `Cargo.toml` cannot take the tag - /// (a workspace-inherited version) fails the local build with - /// `cargo_copy_untaggable`: nothing is swapped in, no stage or vendor - /// dir survives, and the manifest and lock are untouched. The same - /// crate from the patch service is a MISS: auto mode falls back to the - /// local build. #[tokio::test] async fn an_untaggable_copy_manifest_swaps_nothing_in() { let (dir, blobs, pristine, record) = fixture().await; @@ -6708,14 +6440,15 @@ mod tests { .await .unwrap(); let manifest = tokio::fs::read(root.join("Cargo.toml")).await.unwrap(); - let (result, entry, _) = - expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); + let (result, entry, _) = crate::vendor::test_support::expect_failed( + run_vendor(PURL, root, &blobs, &pristine, &record, false).await, + ); assert!(!result.success && entry.is_none()); assert!( result .error .as_deref() - .is_some_and(|e| e.starts_with(COPY_UNTAGGABLE)), + .is_some_and(|e| e.contains("cannot tag")), "{:?}", result.error ); @@ -6746,7 +6479,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_cargo_granted(&server, &sri, &crate_tgz).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_cargo_crate( + let outcome = crate::vendor::test_support::vendor_cargo_crate( PURL, &pristine, root, @@ -6755,24 +6488,16 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&cargo_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&cargo_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, warnings) = expect_done(outcome); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_layout_mismatch" - && w.detail.contains("cannot tag its version")), - "{warnings:?}" - ); - assert!( - copy_toml(root).await.contains(&tagged(UUID)), - "the local build" - ); - assert_eq!(tokio::fs::read(copy_lib(root)).await.unwrap(), PATCHED); + let error = crate::vendor::test_support::expect_failure(outcome); + assert!(error.contains("cannot tag its version"), "{error}"); + assert!(!root.join(".socket/vendor").exists()); } /// V-7: the per-manifest pin extraction answers diff --git a/crates/socket-patch-core/src/vendor/common.rs b/crates/socket-patch-core/src/vendor/common.rs index 8ba51f2b8..86bc36c37 100644 --- a/crates/socket-patch-core/src/vendor/common.rs +++ b/crates/socket-patch-core/src/vendor/common.rs @@ -2,7 +2,7 @@ //! //! Kept in one place so every backend's shapes stay in lockstep. -use std::collections::{HashMap, HashSet}; +use std::collections::HashMap; use std::path::Path; use serde_json::Value; @@ -49,6 +49,19 @@ pub(crate) fn already_patched_result( synthesized_result(package_key, path, files_verified, true, None) } +/// A verified artifact preview that still needs wiring into the project. +pub(crate) fn preview_result( + package_key: &str, + path: &Path, + files: &HashMap, +) -> ApplyResult { + let mut result = already_patched_result(package_key, path, files); + for file in &mut result.files_verified { + file.status = VerifyStatus::Ready; + } + result +} + /// Shared helper the vendor backends (and `go_redirect`) delegate to: an /// [`ApplyResult`] synthesized without running the apply pipeline. pub(crate) fn synthesized_result( @@ -239,6 +252,7 @@ impl JsonLayout { /// ignore the unix modes it carries), so rebuilding the same content always /// yields identical bytes on every platform (churn-free commits, stable /// checksums). +#[cfg(test)] pub(crate) fn write_zip_entries(entries: &[(String, Vec, u32)]) -> Result, String> { use std::io::Write as _; @@ -261,581 +275,8 @@ pub(crate) fn write_zip_entries(entries: &[(String, Vec, u32)]) -> Result bool { - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - metadata.permissions().mode() & 0o111 != 0 - } - #[cfg(not(unix))] - { - let _ = metadata; - false - } -} - -/// Re-zip a patched stage into a deterministic archive (see -/// [`write_zip_entries`]) with entries sorted lexicographically. Both -/// consumers (`.jar` / `.nupkg`) are plain zips whose resolvers read the -/// central directory, so entry order is free to be lexicographic. -/// `skip_entry` drops one archive-relative name (NuGet's `.signature.p7s` — -/// the content changed, so the rebuilt package must read as unsigned). -pub(crate) fn rebuild_zip(stage: &Path, skip_entry: Option<&str>) -> Result, String> { - let mut entries: Vec<(String, Vec, u32)> = Vec::new(); - for entry in walkdir::WalkDir::new(stage).follow_links(false) { - let entry = entry.map_err(|e| format!("walk {}: {e}", stage.display()))?; - if !entry.file_type().is_file() { - continue; - } - let rel = entry - .path() - .strip_prefix(stage) - .map_err(|e| format!("strip prefix: {e}"))?; - let name = rel.to_string_lossy().replace('\\', "/"); - if skip_entry == Some(name.as_str()) { - continue; - } - let bytes = std::fs::read(entry.path()).map_err(|e| format!("read {name}: {e}"))?; - entries.push((name, bytes, 0o644)); - } - entries.sort_by(|a, b| a.0.cmp(&b.0)); - write_zip_entries(&entries) -} - -// ── in-memory local repack (the maven / nuget local build paths) ──────────── - -/// One archive member, decompressed into memory instead of onto disk. -pub(crate) struct ArchiveMember { - /// Archive-relative, `/`-separated name — the name the rebuilt zip uses. - name: String, - bytes: Vec, - /// The entry's unix exec bit, i.e. the mode - /// [`super::registry_fetch::extract_zip`] would have put on the - /// extracted file (0o755 vs 0o644). - exec: bool, - /// Set when the staged twin is gone after the apply (NuGet's sidecar - /// fixup deletes `.nupkg.metadata`): the member then drops out of the - /// rebuild exactly as it drops out of a walk over the stage. - dropped: bool, -} - -/// The in-memory twin of an [`super::registry_fetch::extract_zip`] with -/// `strip_first = false`: -/// every member decompressed into memory, in archive order, with the LAST -/// spelling of a repeated name winning — what an extraction to disk leaves -/// behind. Every guard (entry count, the per-entry and total decompressed -/// caps, the traversal refusal and the declared-vs-actual size check) runs in -/// the same order over the same constants and yields the same message, so a -/// refusal is indistinguishable from the on-disk path's. -/// -/// The one thing it cannot reproduce is an extraction that fails because the -/// *filesystem* mangles or collides names — [`names_are_unambiguous`] is the -/// gate that keeps those archives on the on-disk path. -pub(crate) fn read_zip_members(bytes: &[u8]) -> Result, String> { - use std::io::Read as _; - - use super::registry_fetch::{MAX_ENTRIES, MAX_ENTRY_BYTES, MAX_TOTAL_DECOMPRESSED_BYTES}; - - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)) - .map_err(|e| format!("unreadable zip: {e}"))?; - if archive.len() > MAX_ENTRIES { - return Err(format!("zip exceeds {MAX_ENTRIES} entries")); - } - let mut members: Vec = Vec::new(); - let mut at: HashMap = HashMap::new(); - let mut total: u64 = 0; - for i in 0..archive.len() { - let mut file = archive - .by_index(i) - .map_err(|e| format!("unreadable zip entry: {e}"))?; - if file.is_dir() { - continue; - } - let raw = std::path::PathBuf::from(file.name()); - let rel_str = raw.to_string_lossy().into_owned(); - if !is_safe_relative_subpath(&rel_str) { - return Err(format!( - "zip entry `{}` escapes the extraction dir — refusing the artifact", - raw.display() - )); - } - let declared = file.size(); - if declared > MAX_ENTRY_BYTES { - return Err(format!( - "zip entry `{rel_str}` is {declared} bytes (cap {MAX_ENTRY_BYTES})" - )); - } - total += declared; - if total > MAX_TOTAL_DECOMPRESSED_BYTES { - return Err(format!( - "zip decompresses past the {MAX_TOTAL_DECOMPRESSED_BYTES}-byte cap" - )); - } - // The declared size is header data a crafted zip can understate, so - // hold the caps against the ACTUAL decompressed bytes too: read at - // most declared+1 and refuse on any mismatch (the on-disk twin's - // `take(declared + 1)` copy). - let mut content = Vec::with_capacity(declared as usize); - (&mut file) - .take(declared + 1) - .read_to_end(&mut content) - .map_err(|e| format!("cannot extract `{rel_str}`: {e}"))?; - if content.len() as u64 != declared { - return Err(format!( - "zip entry `{rel_str}` decompresses to {} bytes but declares {declared} \ - — refusing the artifact", - content.len() - )); - } - let exec = file.unix_mode().is_some_and(|m| m & 0o111 != 0); - match at.get(&rel_str) { - // A repeated name overwrote the earlier extraction in place. Two - // entries whose RAW name bytes are identical never get this far — - // `ZipArchive` keys its central directory on them in an `IndexMap` - // and already collapsed the pair, last-wins, at the first index — - // so what lands here is two raw spellings that DECODE to one name - // (`String::from_utf8_lossy` folds distinct invalid bytes onto - // U+FFFD), which `extract_zip` writes to one path just the same. - Some(&i) => { - members[i].bytes = content; - members[i].exec = exec; - } - None => { - at.insert(rel_str.clone(), members.len()); - members.push(ArchiveMember { - name: rel_str, - bytes: content, - exec, - dropped: false, - }); - } - } - } - Ok(members) -} - -/// DOS device names: a file created under one of these on Windows opens the -/// device instead, so the "extracted" member never lands in the stage. -const DOS_DEVICE_NAMES: [&str; 22] = [ - "CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", - "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9", -]; - -/// `NAME_MAX`: the longest single path component APFS, ext4 and NTFS will -/// create. A member past it cannot be extracted at all — the on-disk path -/// fails the whole rebuild with `cannot create : File name too long`, -/// so a name this long has to keep taking that path to keep failing. -const MAX_COMPONENT_BYTES: usize = 255; - -/// And the whole name, so `/` cannot pass `PATH_MAX` either -/// (1024 on macOS, the tightest of the three; a `tempfile` stage prefix is -/// ~60 bytes there, and Rust's Windows `File::create` takes the verbatim -/// `\\?\` route past `MAX_PATH`). Deliberately far above real archives: the -/// longest entry name across 78k members of 362 real jars is 149 bytes. -const MAX_NAME_BYTES: usize = 512; - -/// True when `name` is spelled so that a filesystem can only ever store it as -/// itself — and can store it at all: printable ASCII (so no Unicode-normalising -/// filesystem folds it into a sibling), none of the characters Windows rewrites -/// or rejects, no component that a path walk re-spells (`.`, `..`, empty) or -/// that Windows trims (a trailing `.` or space) or redirects (a DOS device -/// name), and nothing longer than the filesystem would accept. -fn is_plain_archive_name(name: &str) -> bool { - if name.is_empty() || name.len() > MAX_NAME_BYTES { - return false; - } - if !name.chars().all(|c| { - (c.is_ascii_graphic() || c == ' ') - && !matches!(c, '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|') - }) { - return false; - } - name.split('/').all(|part| { - !part.is_empty() - && part.len() <= MAX_COMPONENT_BYTES - && part != "." - && part != ".." - && !part.ends_with('.') - && !part.ends_with(' ') - && !DOS_DEVICE_NAMES.iter().any(|d| { - part.split('.') - .next() - .is_some_and(|stem| stem.eq_ignore_ascii_case(d)) - }) - }) -} - -/// True when `members` (the archive's, or the installed tree's) and `targets` -/// (the patch keys, normalized) name disjoint filesystem entries on any -/// filesystem, and no member is a directory another name lives in — the -/// precondition under which keeping members in memory is indistinguishable -/// from extracting them (see [`read_zip_members`]). -/// -/// Extracting to disk is lossy in ways only the filesystem knows about: a -/// case-insensitive or Unicode-normalising volume collapses two names into one -/// entry, Windows trims and redirects some spellings, a `\` in a name becomes a -/// `/` on the way back out of the walk, and a name that is a file where another -/// needs a directory fails the extraction (or the patch write) outright. Rather -/// than model any of that, the callers keep memory and disk in lockstep only -/// while every name is plain ASCII and no two distinct spellings fold together; -/// anything else falls back to the extract-to-disk path, whose behaviour is then -/// reproduced by definition. -/// -/// Members and targets are told apart for the ancestor rule alone: a target -/// that names a DIRECTORY of the archive is ordinary (the staging materialises -/// one), whereas a name living under a member — which is a FILE — is the case -/// where the two paths diverge. -/// -/// Every `/`-separated PREFIX is checked, not only the whole name: a directory -/// is a filesystem entry too, so `Lib/a.class` + `lib/b.class` collapse into -/// one directory on a case-insensitive volume and the walk back out re-spells -/// the second member under the first's casing — a different rebuilt archive, -/// silently. -pub(crate) fn names_are_unambiguous<'a>( - members: impl IntoIterator, - targets: impl IntoIterator, -) -> bool { - let mut folded: HashMap = HashMap::new(); - let mut member_folded: HashSet = HashSet::new(); - for (name, is_member) in members - .into_iter() - .map(|n| (n, true)) - .chain(targets.into_iter().map(|n| (n, false))) - { - if !is_plain_archive_name(name) { - return false; - } - for end in name - .match_indices('/') - .map(|(at, _)| at) - .chain(std::iter::once(name.len())) - { - let part = &name[..end]; - let lower = part.to_ascii_lowercase(); - // The same path can legitimately arrive twice — a patch target IS - // usually a member, and siblings share their directories. Only a - // DIFFERENT spelling folding onto one already seen is ambiguous. - match folded.get(lower.as_str()) { - Some(seen) if *seen != part => return false, - Some(_) => {} - None => { - folded.insert(lower.clone(), part); - } - } - // Only the whole name is a member FILE; its prefixes are the - // directories it lives in, which the ancestor rule below is about. - if is_member && end == name.len() { - member_folded.insert(lower); - } - } - } - for name in folded.keys() { - let mut prefix = name.as_str(); - while let Some(cut) = prefix.rfind('/') { - prefix = &prefix[..cut]; - if member_folded.contains(prefix) { - return false; - } - } - } - true -} - -#[cfg(test)] -thread_local! { - /// Test seam: forces every local rebuild down the extract-to-disk staging - /// the in-memory repack is defined against, so the equivalence tests can - /// drive one fixture through both and compare the rebuilt artifact byte - /// for byte. Only [`can_repack_in_memory`] reads it — - /// [`names_are_unambiguous`] keeps answering for itself, so the gate's own - /// tests are unaffected. - /// - /// THREAD-LOCAL, not process-wide: `#[tokio::test]` runs its whole future - /// on the test's own thread, and libtest gives every test a thread of its - /// own, so a forced run cannot reach the ~40 other local-rebuild tests - /// running beside it and silently move them off the in-memory path. - static FORCE_ON_DISK_REPACK: std::cell::Cell = const { std::cell::Cell::new(false) }; - - /// How many rebuilds took the in-memory path on THIS thread — the - /// equivalence tests read it to prove which staging each of their two runs - /// actually used, rather than trusting the seam and the fixture's names. - static IN_MEMORY_REPACKS: std::cell::Cell = const { std::cell::Cell::new(0) }; -} - -/// Sets [`FORCE_ON_DISK_REPACK`] for as long as it is held. -#[cfg(test)] -pub(crate) struct OnDiskRepackGuard; - -#[cfg(test)] -impl OnDiskRepackGuard { - pub(crate) fn acquire() -> Self { - FORCE_ON_DISK_REPACK.set(true); - Self - } -} - -#[cfg(test)] -impl Drop for OnDiskRepackGuard { - fn drop(&mut self) { - FORCE_ON_DISK_REPACK.set(false); - } -} - -/// The running count of [`IN_MEMORY_REPACKS`] for this thread; a test brackets -/// a rebuild with it to assert which staging ran. -#[cfg(test)] -pub(crate) fn in_memory_repacks() -> usize { - IN_MEMORY_REPACKS.get() -} - -/// Whether a local rebuild may keep the archive (or the installed tree) in -/// memory: the [`names_are_unambiguous`] gate, plus the test seam. -pub(crate) fn can_repack_in_memory<'a>( - members: impl IntoIterator, - targets: impl IntoIterator, -) -> bool { - #[cfg(test)] - if FORCE_ON_DISK_REPACK.get() { - return false; - } - let in_memory = names_are_unambiguous(members, targets); - #[cfg(test)] - if in_memory { - IN_MEMORY_REPACKS.set(IN_MEMORY_REPACKS.get() + 1); - } - in_memory -} - -/// A local rebuild carried in memory: the archive's members never touch the -/// stage, only the handful of paths the apply pipeline (and the ecosystem's -/// sidecar fixup) resolves do, and the rebuilt archive is assembled from the -/// two halves. -pub(crate) struct MemoryRepack { - members: Vec, - at: HashMap, - /// The paths materialised in the stage — everything the apply pipeline - /// can read, write, create or delete — in a deterministic order: the - /// sorted patch targets, then the extras the caller adds, each once. - /// [`Self::stage_into`] and [`Self::into_entries`] both return on the - /// first I/O failure, so the order decides which path a failure names. - wanted: Vec, -} - -/// Read `archive` into memory for an in-place rebuild, or `Ok(None)` when its -/// names (together with `files`' patch targets and `extra`) are not unambiguous -/// on every filesystem — the caller must then extract to disk instead, which is -/// what this path is defined against. Errors are -/// [`super::registry_fetch::extract_zip`]'s, verbatim. -/// -/// `extra` names the fixed paths the ecosystem's sidecar fixup resolves beside -/// the patch targets (NuGet's `.nupkg.metadata`). They are materialised like a -/// target and, like one, must not fold onto a member under a different -/// spelling — the fixup would delete that member on a case-insensitive volume -/// and leave it in place on a case-sensitive one. -pub(crate) fn prepare_memory_repack( - archive: &[u8], - files: &HashMap, - extra: &[&str], -) -> Result, String> { - let members = read_zip_members(archive)?; - let mut targets: Vec<&str> = patch_target_paths(files); - targets.extend_from_slice(extra); - if !can_repack_in_memory( - members.iter().map(|m| m.name.as_str()), - targets.iter().copied(), - ) { - return Ok(None); - } - let at = members - .iter() - .enumerate() - .map(|(i, m)| (m.name.clone(), i)) - .collect(); - let mut repack = MemoryRepack { - members, - at, - wanted: Vec::new(), - }; - for target in targets { - repack.also_stage(target); - } - Ok(Some(repack)) -} - -/// The in-package paths the apply pipeline resolves for `files`: each key -/// normalized, with the escaping keys the pipeline itself refuses dropped (it -/// never joins them, so nothing has to be materialised for them either). -/// -/// SORTED, because `files` is a `HashMap` with a per-process random hasher and -/// every caller walks this list until the first I/O error: without the sort, -/// two runs over one package under ENOSPC or EACCES name a different file in -/// the failure. That is the invariant `patch::apply::files_in_order` states for -/// the apply itself, and the reason `PatchRecord::files` serializes sorted. -pub(crate) fn patch_target_paths(files: &HashMap) -> Vec<&str> { - let mut paths: Vec<&str> = files - .keys() - .map(|key| normalize_file_path(key)) - .filter(|path| is_safe_relative_subpath(path)) - .collect(); - paths.sort_unstable(); - paths.dedup(); - paths -} - -impl MemoryRepack { - /// Also materialise `name` in the stage — the hook the ecosystem sidecar - /// fixups need for the paths they inspect outside the patch target set - /// (NuGet's `.nupkg.metadata` and its `*.nupkg.sha512` markers). - pub(crate) fn also_stage(&mut self, name: &str) { - if !self.wanted.iter().any(|w| w == name) { - self.wanted.push(name.to_string()); - } - } - - /// Every member name, for the callers that pick their extra staged paths - /// out of the archive itself. - pub(crate) fn member_names(&self) -> impl Iterator { - self.members.iter().map(|m| m.name.as_str()) - } - - /// Materialise the wanted paths under `stage`: a member is written with - /// the mode [`super::registry_fetch::extract_zip`] would have given it, a name that only exists - /// as a directory in the archive is created as one (so a patch key - /// pointing at a directory still hashes as one), and a name the archive - /// does not carry is left absent. - pub(crate) async fn stage_into(&self, stage: &Path) -> Result<(), String> { - for name in &self.wanted { - let target = stage.join(name); - match self.at.get(name) { - Some(&i) => { - if let Some(parent) = target.parent() { - tokio::fs::create_dir_all(parent) - .await - .map_err(|e| format!("cannot create {}: {e}", parent.display()))?; - } - tokio::fs::write(&target, &self.members[i].bytes) - .await - .map_err(|e| format!("cannot create {}: {e}", target.display()))?; - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - let perms = if self.members[i].exec { 0o755 } else { 0o644 }; - let _ = std::fs::set_permissions( - &target, - std::fs::Permissions::from_mode(perms), - ); - } - } - None if self.names_a_directory(name) => { - tokio::fs::create_dir_all(&target) - .await - .map_err(|e| format!("cannot create {}: {e}", target.display()))?; - } - None => {} - } - } - Ok(()) - } - - /// True when the archive carries members UNDER `name`, i.e. extracting it - /// would have created a directory there. - fn names_a_directory(&self, name: &str) -> bool { - let prefix = format!("{name}/"); - self.members.iter().any(|m| m.name.starts_with(&prefix)) - } - - /// Reconcile the staged paths back into the in-memory members and emit the - /// [`write_zip_entries`] list: the same lexicographic order and flat 0o644 - /// mode [`rebuild_zip`] produces over a fully extracted stage. A staged - /// path the apply wrote carries its new bytes, one it created joins the - /// archive, and one it deleted leaves it. - pub(crate) async fn into_entries( - mut self, - stage: &Path, - skip_entry: Option<&str>, - ) -> Result, u32)>, String> { - for name in &self.wanted { - let path = stage.join(name); - // A walk over the stage yields regular files and nothing else, so - // an absent path (or a directory left where a patch key pointed) - // simply contributes no entry. - let live = matches!(tokio::fs::metadata(&path).await, Ok(m) if m.is_file()); - match (live, self.at.get(name)) { - (true, Some(&i)) => { - self.members[i].bytes = tokio::fs::read(&path) - .await - .map_err(|e| format!("read {name}: {e}"))?; - } - (true, None) => { - let bytes = tokio::fs::read(&path) - .await - .map_err(|e| format!("read {name}: {e}"))?; - self.members.push(ArchiveMember { - name: name.clone(), - bytes, - exec: false, - dropped: false, - }); - } - (false, Some(&i)) => self.members[i].dropped = true, - (false, None) => {} - } - } - let mut entries: Vec<(String, Vec, u32)> = self - .members - .into_iter() - .filter(|m| !m.dropped && skip_entry != Some(m.name.as_str())) - .map(|m| (m.name, m.bytes, 0o644)) - .collect(); - entries.sort_by(|a, b| a.0.cmp(&b.0)); - Ok(entries) - } -} - -/// A private stage directory whose (recursive) deletion can be handed to the -/// blocking pool: [`tempfile::TempDir`]'s own `Drop` unlinks the whole tree -/// synchronously, which on the build paths ran on the runtime thread. Dropping -/// a `Stage` without [`Stage::dispose`] still deletes it synchronously, so every -/// early return stays correct. -pub(crate) struct Stage(Option); - -impl Stage { - pub(crate) fn new() -> std::io::Result { - Ok(Self(Some(tempfile::tempdir()?))) - } - - pub(crate) fn path(&self) -> &Path { - self.0.as_ref().expect("stage is live until dispose").path() - } - - /// Delete the stage on the blocking pool. - pub(crate) async fn dispose(mut self) { - if let Some(dir) = self.0.take() { - let _ = tokio::task::spawn_blocking(move || drop(dir)).await; - } - } -} - -/// Bound on a committed `.jar` / `.nupkg` the in-sync probe is willing to -/// read into memory: the whole-file cap `verify.rs` applies to the same -/// artifacts (`MAX_HEALTH_HASH_BYTES`), which also covers everything the -/// rebuild path can produce (`extract_zip` bounds the decompressed payload -/// at 512 MiB and a deflated archive is never larger than its payload) — so -/// a valid committed artifact can never read as stale because of the cap. const MAX_ZIP_ARTIFACT_BYTES: u64 = 512 * 1024 * 1024; -/// The committed archive's bytes, or `None` when it is missing, not a regular -/// file, or over the cap. Guarded open (`open_regular_file`: O_NONBLOCK + -/// regular-file check): a FIFO planted at the archive path must read as -/// out-of-sync, not wedge the probe forever in an `open(2)` waiting for a -/// writer. The archive is committed and tamper-able: the size gate runs on -/// the open handle's metadata BEFORE anything is read, like the blob harvest -/// in `vendor/mod.rs`, so an oversized file is never slurped into memory. -/// Backends that need the bytes for more than the member check (a sidecar / -/// content hash) read once through this and hand them to -/// [`zip_bytes_match_after_hashes`]. pub(crate) async fn read_zip_artifact(archive_path: &Path) -> Option> { read_zip_artifact_capped(archive_path, MAX_ZIP_ARTIFACT_BYTES).await } @@ -2069,736 +1510,4 @@ mod tests { "a failed revert must leave the lock content untouched" ); } - - // ── in-memory repack equivalence ──────────────────────────────────────── - - /// A zip built entry by entry, so the oracle fixtures can carry the - /// spellings `write_zip_entries` never emits: repeated names, STORED - /// members, zero-length members, an exec bit, a directory entry and a - /// traversal-escaping name. - fn build_zip(entries: &[(&str, &[u8], zip::CompressionMethod, u32)]) -> Vec { - use std::io::Write as _; - let mut writer = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); - for (name, bytes, method, mode) in entries { - let options = zip::write::SimpleFileOptions::default() - .compression_method(*method) - .unix_permissions(*mode); - if name.ends_with('/') { - writer.add_directory(*name, options).unwrap(); - continue; - } - writer.start_file(*name, options).unwrap(); - writer.write_all(bytes).unwrap(); - } - writer.finish().unwrap().into_inner() - } - - /// The default zip fixture entry: deflated, 0o644. - fn entry<'a>( - name: &'a str, - bytes: &'a [u8], - ) -> (&'a str, &'a [u8], zip::CompressionMethod, u32) { - (name, bytes, zip::CompressionMethod::Deflated, 0o644) - } - - /// A patch-files map naming `keys` (content irrelevant: these tests drive - /// the staging and the rebuild, not the apply). - fn target_files(keys: &[&str]) -> HashMap { - keys.iter() - .map(|k| { - ( - (*k).to_string(), - PatchFileInfo { - before_hash: compute_git_sha256_from_bytes(b"before"), - after_hash: compute_git_sha256_from_bytes(b"after"), - }, - ) - }) - .collect() - } - - /// The extract-to-disk repack, kept as the oracle: extract every member - /// to a stage, let the caller stand in for the apply pipeline, then walk - /// the stage back into a deterministic zip. - async fn on_disk_repack( - archive: &[u8], - skip_entry: Option<&str>, - apply: impl AsyncFn(&Path), - ) -> Result, String> { - let stage = tempfile::tempdir().map_err(|e| format!("stage: {e}"))?; - super::super::registry_fetch::extract_zip(archive, stage.path(), false)?; - apply(stage.path()).await; - rebuild_zip(stage.path(), skip_entry) - } - - /// The in-memory repack: members stay in memory, only the patch targets (plus - /// `extra`) are materialised, and the same stand-in apply runs over them. - /// `None` means the name gate sent the rebuild back to the on-disk path. - async fn in_memory_repack( - archive: &[u8], - files: &HashMap, - extra: &[&str], - skip_entry: Option<&str>, - apply: impl AsyncFn(&Path), - ) -> Result>, String> { - let Some(mut repack) = prepare_memory_repack(archive, files, extra)? else { - return Ok(None); - }; - for name in extra { - repack.also_stage(name); - } - let stage = tempfile::tempdir().map_err(|e| format!("stage: {e}"))?; - repack.stage_into(stage.path()).await?; - apply(stage.path()).await; - let entries = repack.into_entries(stage.path(), skip_entry).await?; - write_zip_entries(&entries).map(Some) - } - - /// Both repacks over one fixture must agree byte for byte; returns the - /// shared bytes so a caller can assert on the archive itself. - async fn assert_repacks_agree( - archive: &[u8], - keys: &[&str], - extra: &[&str], - skip_entry: Option<&str>, - apply: impl AsyncFn(&Path) + Copy, - ) -> Vec { - let files = target_files(keys); - let oracle = on_disk_repack(archive, skip_entry, apply).await.unwrap(); - let fast = in_memory_repack(archive, &files, extra, skip_entry, apply) - .await - .unwrap() - .expect("this fixture's names must take the in-memory path"); - assert_eq!( - fast, oracle, - "the in-memory repack must reproduce the extract-and-rezip bytes" - ); - fast - } - - /// An untouched archive: nested dirs, an explicit directory entry, a - /// zero-length member, a STORED member, an exec-bit member and a member - /// large enough to span several read buffers must all repack to the same - /// bytes as a full extraction would. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn in_memory_repack_matches_the_extract_and_rezip_oracle() { - let big = vec![b'z'; 3 * 1024 * 1024]; - let archive = build_zip(&[ - ("META-INF/", b"", zip::CompressionMethod::Stored, 0o755), - entry("META-INF/MANIFEST.MF", b"Manifest-Version: 1.0\n"), - entry("lib/empty.txt", b""), - ( - "lib/stored.bin", - b"stored bytes", - zip::CompressionMethod::Stored, - 0o644, - ), - ( - "bin/run.sh", - b"#!/bin/sh\nexit 0\n", - zip::CompressionMethod::Deflated, - 0o755, - ), - entry("lib/big.bin", &big), - entry("LICENSE", b"license\n"), - ]); - let bytes = assert_repacks_agree(&archive, &["LICENSE"], &[], None, async |_| {}).await; - let names = zip_entry_names(&bytes); - assert_eq!( - names, - [ - "LICENSE", - "META-INF/MANIFEST.MF", - "bin/run.sh", - "lib/big.bin", - "lib/empty.txt", - "lib/stored.bin", - ], - "directory entries drop out; files sort lexicographically" - ); - } - - /// The three ways an apply can change the stage — rewriting a patch - /// target, creating one that was not in the archive, and deleting a - /// staged path (NuGet's `.nupkg.metadata` fixup) — must land in the - /// rebuilt archive exactly as they do over a full extraction. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn in_memory_repack_tracks_rewrites_creations_and_deletions() { - let archive = build_zip(&[ - entry("LICENSE", b"pristine\n"), - entry(".nupkg.metadata", b"{\"contentHash\":\"x\"}"), - entry("lib/keep.txt", b"keep\n"), - ]); - let bytes = assert_repacks_agree( - &archive, - &["LICENSE", "lib/new.txt"], - &[".nupkg.metadata"], - None, - async |stage: &Path| { - tokio::fs::write(stage.join("LICENSE"), b"patched\n") - .await - .unwrap(); - // `apply_file_patch_at` materialises a created file's parent - // itself, so the stand-in does too. - tokio::fs::create_dir_all(stage.join("lib")).await.unwrap(); - tokio::fs::write(stage.join("lib/new.txt"), b"created\n") - .await - .unwrap(); - tokio::fs::remove_file(stage.join(".nupkg.metadata")) - .await - .unwrap(); - }, - ) - .await; - assert_eq!( - zip_entry_names(&bytes), - ["LICENSE", "lib/keep.txt", "lib/new.txt"], - "the deleted part is gone and the created one joined" - ); - assert_eq!(zip_member(&bytes, "LICENSE"), b"patched\n"); - } - - /// The `skip_entry` drop (NuGet's `.signature.p7s`) is applied by both - /// repacks at the same point. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn in_memory_repack_drops_the_skipped_entry() { - let archive = build_zip(&[ - entry(".signature.p7s", b"FAKE-SIGNATURE"), - entry("LICENSE", b"pristine\n"), - ]); - let bytes = assert_repacks_agree( - &archive, - &["LICENSE"], - &[], - Some(".signature.p7s"), - async |_| {}, - ) - .await; - assert_eq!(zip_entry_names(&bytes), ["LICENSE"]); - } - - /// A patch key that names a DIRECTORY of the archive must find one in the - /// stage, exactly as a full extraction leaves one there — otherwise the - /// verify reports "File not found" where a full extraction reports a hash - /// failure, and `--force` would silently skip the key. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn in_memory_repack_materialises_a_directory_a_patch_key_names() { - let archive = build_zip(&[entry("lib/net6.0/x.dll", b"MZ")]); - let files = target_files(&["lib/net6.0"]); - let repack = prepare_memory_repack(&archive, &files, &[]) - .unwrap() - .unwrap(); - let stage = tempfile::tempdir().unwrap(); - repack.stage_into(stage.path()).await.unwrap(); - assert!( - stage.path().join("lib/net6.0").is_dir(), - "a patch key naming a directory must be staged as one" - ); - assert!( - !stage.path().join("lib/net6.0/x.dll").exists(), - "its members stay in memory" - ); - } - - /// Only the patch targets and the explicitly requested extras are written - /// out — the point of the whole change. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn in_memory_repack_stages_only_what_the_apply_resolves() { - let archive = build_zip(&[ - entry("LICENSE", b"pristine\n"), - entry("lib/a.dll", b"MZ-a"), - entry("lib/b.dll", b"MZ-b"), - entry(".nupkg.metadata", b"{}"), - ]); - let files = target_files(&["LICENSE"]); - let mut repack = prepare_memory_repack(&archive, &files, &[]) - .unwrap() - .unwrap(); - repack.also_stage(".nupkg.metadata"); - let stage = tempfile::tempdir().unwrap(); - repack.stage_into(stage.path()).await.unwrap(); - assert!(stage.path().join("LICENSE").is_file()); - assert!(stage.path().join(".nupkg.metadata").is_file()); - assert!(!stage.path().join("lib/a.dll").exists()); - assert!(!stage.path().join("lib/b.dll").exists()); - assert!(!stage.path().join("lib").exists(), "no directory pass"); - } - - /// Archives whose names a filesystem can fold together, re-spell or - /// refuse must go back to the extract-to-disk path rather than be guessed - /// at in memory. Repeated names are the load-bearing case: on disk the - /// last one wins, in memory a naive map would keep both. - #[tokio::test] - async fn ambiguous_names_fall_back_to_the_on_disk_repack() { - let over_name_max = format!("lib/{}.class", "A".repeat(MAX_COMPONENT_BYTES)); - let cases: Vec<(&str, Vec, Vec<&str>)> = vec![ - ( - "case-colliding names", - build_zip(&[ - entry("META-INF/NOTICE", b"a"), - entry("META-INF/notice", b"b"), - ]), - vec![], - ), - ( - "a name that is also a directory", - build_zip(&[entry("lib", b"a"), entry("lib/x.dll", b"b")]), - vec![], - ), - ( - "a backslash in a name", - build_zip(&[entry("lib\\x.dll", b"a")]), - vec![], - ), - ( - "a non-ASCII name", - build_zip(&[entry("lib/caf\u{e9}.txt", b"a")]), - vec![], - ), - ( - "a DOS device name", - build_zip(&[entry("lib/NUL.txt", b"a")]), - vec![], - ), - ( - "a trailing dot", - build_zip(&[entry("lib/x.", b"a")]), - vec![], - ), - ( - "a patch key colliding with a member", - build_zip(&[entry("LICENSE", b"a")]), - vec!["license"], - ), - ( - "two spellings of one directory", - build_zip(&[entry("Lib/a.class", b"a"), entry("lib/b.class", b"b")]), - vec![], - ), - ( - "a patch key naming a member's directory under another spelling", - build_zip(&[entry("Lib/x.dll", b"a")]), - vec!["lib"], - ), - ( - "a component past NAME_MAX", - build_zip(&[entry(&over_name_max, b"a")]), - vec![], - ), - ]; - for (label, archive, keys) in cases { - let files = target_files(&keys); - assert!( - prepare_memory_repack(&archive, &files, &[]) - .unwrap() - .is_none(), - "{label} must fall back to the on-disk repack" - ); - } - } - - /// The rebuilt archive is byte-identical on every platform: every - /// central-directory entry names Unix as its "made by" host (the zip - /// crate's own default is DOS on Windows) and keeps the unix mode it was - /// given, so a wheel/jar/nupkg rebuilt on Windows hashes like the one - /// the ledger recorded on macOS or Linux. - #[test] - fn write_zip_entries_stamps_a_unix_host_on_every_platform() { - let bytes = write_zip_entries(&[ - ("pkg/run.sh".to_string(), b"#!/bin/sh\n".to_vec(), 0o755), - ("pkg/data.txt".to_string(), b"data\n".to_vec(), 0o644), - ]) - .unwrap(); - let mut hosts = Vec::new(); - let mut at = 0; - while let Some(i) = bytes[at..].windows(4).position(|w| w == b"PK\x01\x02") { - let header = at + i; - hosts.push(bytes[header + 5]); - at = header + 4; - } - assert_eq!(hosts, vec![3, 3], "made-by host byte is Unix (3)"); - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); - let modes: Vec = (0..archive.len()) - .map(|i| archive.by_index(i).unwrap().unix_mode().unwrap() & 0o777) - .collect(); - assert_eq!(modes, vec![0o755, 0o644]); - } - - /// `record.files` is a `HashMap` with a per-process random hasher, and - /// every walk over the staged paths returns on the FIRST I/O error — so - /// without a sort, two runs over one package under ENOSPC or EACCES name a - /// different file in the failure that reaches stdout. - #[test] - fn staged_paths_are_walked_in_a_deterministic_order() { - let files = target_files(&[ - "z.txt", - "a/b.txt", - "m.txt", - "package/m.txt", - "d.txt", - "q/r.txt", - "../escapes.txt", - ]); - assert_eq!( - patch_target_paths(&files), - ["a/b.txt", "d.txt", "m.txt", "q/r.txt", "z.txt"], - "sorted, deduplicated past `package/`, and without the keys the \ - apply pipeline refuses to join" - ); - } - - /// And the order survives into the stage, ahead of the extras the caller - /// adds for its sidecar fixup. - #[tokio::test] - async fn the_repack_stages_the_sorted_targets_then_the_extras() { - let archive = build_zip(&[ - entry("z.txt", b"z"), - entry("m.txt", b"m"), - entry("d.txt", b"d"), - entry("a/b.txt", b"b"), - entry("q/r.txt", b"r"), - entry(".nupkg.metadata", b"{}"), - ]); - let files = target_files(&["z.txt", "m.txt", "d.txt", "a/b.txt", "q/r.txt"]); - let repack = prepare_memory_repack(&archive, &files, &[".nupkg.metadata"]) - .unwrap() - .unwrap(); - assert_eq!( - repack.wanted, - [ - "a/b.txt", - "d.txt", - "m.txt", - "q/r.txt", - "z.txt", - ".nupkg.metadata" - ] - ); - } - - /// A member folding onto one of the sidecar fixup's fixed paths must fall - /// back too: the fixup would delete that member on a case-insensitive - /// volume and leave it in place on a case-sensitive one, so the rebuilt - /// package is only reproducible through the on-disk path. - #[tokio::test] - #[serial_test::serial] - async fn a_member_folding_onto_a_sidecar_path_falls_back() { - let archive = build_zip(&[entry(".NUPKG.METADATA", b"{}"), entry("LICENSE", b"x")]); - let files = target_files(&["LICENSE"]); - assert!( - prepare_memory_repack(&archive, &files, &[".nupkg.metadata"]) - .unwrap() - .is_none(), - "a member folding onto `.nupkg.metadata` must take the on-disk path" - ); - assert!( - prepare_memory_repack(&archive, &files, &[]) - .unwrap() - .is_some(), - "and only because the fixup path was declared" - ); - } - - /// A repeated entry name: the extraction overwrites in place, so the LAST - /// spelling's bytes are what the rebuild carries. The in-memory reader - /// collapses the pair the same way, and the two repacks must agree. - #[tokio::test] - // The fixture must take the in-memory path, and `FORCE_ON_DISK_REPACK` is - // thread-local, so `#[serial]` is belt and braces here. - #[serial_test::serial] - async fn repeated_entry_names_repack_as_last_one_wins() { - // `ZipWriter` refuses a repeated name, so build two same-length names - // and rename the second in place (local header + central directory). - let mut archive = build_zip(&[entry("dup.txt", b"first"), entry("dup2txt", b"second")]); - rename_zip_entry(&mut archive, b"dup2txt", b"dup.txt"); - // WHERE the pair collapses is the zip crate's business: it keys the - // central directory on the RAW name bytes in an `IndexMap`, so - // `ZipArchive` hands out one entry, at the first one's index, before - // `read_zip_members` sees it. Pinned here so a crate bump that stops - // doing it is caught rather than silently changing what the rebuild - // carries. - assert_eq!( - zip::ZipArchive::new(std::io::Cursor::new(archive.clone())) - .unwrap() - .len(), - 1, - "the zip crate collapses identical raw names itself" - ); - let members = read_zip_members(&archive).unwrap(); - assert_eq!(members.len(), 1, "the repeat collapses, as on disk"); - assert_eq!(members[0].bytes, b"second"); - let bytes = assert_repacks_agree(&archive, &["dup.txt"], &[], None, async |_| {}).await; - assert_eq!(zip_entry_names(&bytes), ["dup.txt"]); - assert_eq!(zip_member(&bytes, "dup.txt"), b"second"); - } - - /// The collapse `read_zip_members` does itself: two DIFFERENT raw names - /// that decode to one (`from_utf8_lossy` folds distinct invalid bytes onto - /// U+FFFD), which the zip crate keeps apart and `extract_zip` writes to a - /// single path — last one wins, exactly as the reader's `at` map does. - #[tokio::test] - async fn raw_names_decoding_to_one_name_collapse_last_one_wins() { - let mut archive = build_zip(&[entry("dupA.txt", b"first"), entry("dupB.txt", b"second")]); - rename_zip_entry(&mut archive, b"dupA.txt", b"dup\xff.txt"); - rename_zip_entry(&mut archive, b"dupB.txt", b"dup\xfe.txt"); - // Without the language-encoding flag the names decode through CP437, - // which is a bijection — the lossy fold needs the UTF-8 flag set. - set_utf8_name_flag(&mut archive); - let decoded = "dup\u{fffd}.txt"; - assert_eq!( - zip::ZipArchive::new(std::io::Cursor::new(archive.clone())) - .unwrap() - .len(), - 2, - "the raw names differ, so the zip crate keeps both entries" - ); - let members = read_zip_members(&archive).unwrap(); - assert_eq!(members.len(), 1, "but they name one file"); - assert_eq!(members[0].name, decoded); - assert_eq!(members[0].bytes, b"second"); - // And that is what an extraction leaves behind. - let stage = tempfile::tempdir().unwrap(); - super::super::registry_fetch::extract_zip(&archive, stage.path(), false).unwrap(); - assert_eq!( - tokio::fs::read(stage.path().join(decoded)).await.unwrap(), - b"second" - ); - // The name is not plain ASCII, so the rebuild itself takes the - // extract-to-disk path — the reader still has to agree about it, - // because it runs before the gate does. - assert!( - prepare_memory_repack(&archive, &target_files(&["dup.txt"]), &[]) - .unwrap() - .is_none() - ); - } - - /// Set the general-purpose "language encoding" bit (bit 11) on every local - /// file header and central directory header, so the reader decodes entry - /// names as UTF-8 instead of CP437. - fn set_utf8_name_flag(archive: &mut [u8]) { - for (signature, flags_at) in [(b"PK\x03\x04".as_slice(), 6), (b"PK\x01\x02".as_slice(), 8)] - { - let mut at = 0; - let mut hits = 0; - while at + flags_at + 2 <= archive.len() { - if archive[at..].starts_with(signature) { - archive[at + flags_at + 1] |= 0b0000_1000; - hits += 1; - at += signature.len(); - } else { - at += 1; - } - } - assert_eq!(hits, 2, "two entries, one header of each kind apiece"); - } - } - - /// A member no filesystem can create: the extraction fails the whole - /// rebuild with ENAMETOOLONG, so the gate has to keep such an archive on - /// that path. In memory it would rebuild cleanly and turn a package the - /// baseline refused into a vendored one. - #[tokio::test] - async fn a_member_past_name_max_still_fails_the_rebuild() { - let long = format!("lib/{}.class", "A".repeat(MAX_COMPONENT_BYTES)); - let archive = build_zip(&[entry("LICENSE", b"a"), entry(&long, b"b")]); - assert!( - prepare_memory_repack(&archive, &target_files(&["LICENSE"]), &[]) - .unwrap() - .is_none(), - "an unwritable member name must take the on-disk repack" - ); - let stage = tempfile::tempdir().unwrap(); - let error = super::super::registry_fetch::extract_zip(&archive, stage.path(), false) - .expect_err("no filesystem creates a component past NAME_MAX"); - assert!( - error.starts_with("cannot create ") && error.contains(&long["lib/".len()..]), - "{error}" - ); - } - - /// Two spellings of one directory: a case-insensitive stage collapses them, - /// and the walk back out re-spells the second member under the first's - /// casing — a different rebuilt artifact, and so a different `.sha1` - /// sidecar and NuGet `contentHash`. The in-memory repack cannot reproduce - /// that, so the gate must send the archive to disk. - #[tokio::test] - async fn case_variant_directory_spellings_take_the_on_disk_repack() { - let archive = build_zip(&[ - entry("LICENSE", b"pristine\n"), - entry("Lib/a.class", b"a"), - entry("lib/b.class", b"b"), - ]); - assert!( - prepare_memory_repack(&archive, &target_files(&["LICENSE"]), &[]) - .unwrap() - .is_none(), - "a folded directory spelling must take the on-disk repack" - ); - // And on a volume that really does fold, pin what the extraction - // leaves behind — so the gate stays necessary rather than cosmetic. - let probe = tempfile::tempdir().unwrap(); - std::fs::create_dir(probe.path().join("Lib")).unwrap(); - if std::fs::create_dir(probe.path().join("lib")).is_err() { - let oracle = on_disk_repack(&archive, None, async |_| {}).await.unwrap(); - assert_eq!( - zip_entry_names(&oracle), - ["LICENSE", "Lib/a.class", "Lib/b.class"], - "the second member is republished under the first's casing" - ); - } - } - - /// Rewrite every occurrence of an entry name in a zip's bytes. `from` and - /// `to` must be the same length so no offset in the archive moves. - fn rename_zip_entry(archive: &mut [u8], from: &[u8], to: &[u8]) { - assert_eq!(from.len(), to.len(), "renaming must not move offsets"); - let mut at = 0; - let mut hits = 0; - while at + from.len() <= archive.len() { - if &archive[at..at + from.len()] == from { - archive[at..at + from.len()].copy_from_slice(to); - hits += 1; - at += from.len(); - } else { - at += 1; - } - } - assert_eq!(hits, 2, "local header and central directory"); - } - - /// Every refusal the on-disk extractor raises must come out of the - /// in-memory reader with the identical message, so a poisoned artifact - /// fails the same way whichever path ran. - #[tokio::test] - async fn in_memory_reader_refuses_exactly_what_the_extractor_refuses() { - let escaping = build_zip(&[entry("../evil.js", b"x")]); - let truncated = { - let mut bytes = build_zip(&[entry("a.txt", b"hello")]); - bytes.truncate(bytes.len() / 2); - bytes - }; - for (label, archive) in [("escaping entry", escaping), ("truncated", truncated)] { - let stage = tempfile::tempdir().unwrap(); - let oracle = super::super::registry_fetch::extract_zip(&archive, stage.path(), false) - .unwrap_err(); - let fast = match read_zip_members(&archive) { - Err(e) => e, - Ok(_) => panic!("{label}: the in-memory reader must refuse this archive"), - }; - assert_eq!(fast, oracle, "{label}: the two readers must agree"); - } - } - - /// The entry names of a zip, in central-directory order. - fn zip_entry_names(bytes: &[u8]) -> Vec { - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); - (0..archive.len()) - .map(|i| archive.by_index(i).unwrap().name().to_string()) - .collect() - } - - /// One member's bytes. - fn zip_member(bytes: &[u8], name: &str) -> Vec { - use std::io::Read as _; - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap(); - let mut out = Vec::new(); - archive - .by_name(name) - .unwrap() - .read_to_end(&mut out) - .unwrap(); - out - } - - /// The name gate's rules, one by one. - #[test] - fn names_are_unambiguous_rejects_what_a_filesystem_can_fold_or_respell() { - let plain = |names: [&str; 1]| names_are_unambiguous(names, []); - assert!(names_are_unambiguous(["a/b.txt", "a/c.txt", "d.txt"], [])); - assert!( - names_are_unambiguous(["a/b.txt"], ["a/b.txt"]), - "a patch target IS usually a member" - ); - assert!( - !names_are_unambiguous(["A.txt"], ["a.txt"]), - "a target folding onto a member" - ); - assert!(!names_are_unambiguous(["A.txt", "a.txt"], []), "case fold"); - assert!( - !names_are_unambiguous(["Lib/a.class", "lib/b.class"], []), - "two spellings of one DIRECTORY fold into one entry too — the walk \ - back out would re-spell the second member under the first's casing" - ); - assert!( - !names_are_unambiguous(["META-INF/services/a", "meta-inf/services/b"], []), - "a folded directory anywhere along the path" - ); - assert!( - !names_are_unambiguous(["Lib/x.dll"], ["lib"]), - "a target naming a member's directory under a different spelling" - ); - assert!( - names_are_unambiguous(["lib/a.dll", "lib/b.dll", "lib/net6.0/c.dll"], []), - "siblings sharing a directory spelling stay on the fast path" - ); - assert!( - !names_are_unambiguous(["a", "a/b"], []), - "file vs directory" - ); - assert!(!names_are_unambiguous(["a/b", "A"], []), "folded ancestor"); - assert!( - !names_are_unambiguous(["lib"], ["lib/x"]), - "a target living under a member FILE" - ); - assert!( - names_are_unambiguous(["lib/net6.0/x.dll"], ["lib/net6.0"]), - "a target naming a member's DIRECTORY is ordinary" - ); - assert!(!plain(["a\\b"]), "backslash"); - assert!(!plain(["caf\u{e9}"]), "non-ASCII"); - assert!(!plain(["a:b"]), "alternate data stream"); - assert!(!plain(["a*"]), "Windows wildcard"); - assert!(!plain(["a."]), "trailing dot"); - assert!(!plain(["a "]), "trailing space"); - assert!(!plain(["nul"]), "DOS device"); - assert!(!plain(["dir/COM1.txt"]), "DOS device stem"); - assert!(!plain(["a/./b"]), "re-spelled component"); - assert!(!plain(["a//b"]), "empty component"); - assert!(!plain([""]), "empty name"); - let long_part = "A".repeat(MAX_COMPONENT_BYTES + 1); - assert!( - !plain([format!("org/apache/{long_part}.class").as_str()]), - "a component past NAME_MAX — the extraction would have failed with \ - ENAMETOOLONG, so the rebuild has to keep failing" - ); - assert!( - plain([format!("org/apache/{}.class", "A".repeat(MAX_COMPONENT_BYTES - 6)).as_str()]), - "a component exactly at NAME_MAX still extracts" - ); - let deep = vec!["dir"; MAX_NAME_BYTES / 4 + 1].join("/"); - assert!( - !plain([deep.as_str()]), - "a whole name long enough to push `/` past PATH_MAX" - ); - assert!( - names_are_unambiguous(["my lib/x.dll", "a-b_c+d$e.txt", "[Content_Types].xml"], []), - "ordinary jar/nupkg spellings stay on the fast path" - ); - } } diff --git a/crates/socket-patch-core/src/vendor/composer_lock.rs b/crates/socket-patch-core/src/vendor/composer_lock.rs index 34ff95c19..a66ea08ea 100644 --- a/crates/socket-patch-core/src/vendor/composer_lock.rs +++ b/crates/socket-patch-core/src/vendor/composer_lock.rs @@ -46,7 +46,7 @@ use serde_json::{json, Map, Value}; use crate::constants::SOCKET_DIR; use crate::manifest::schema::PatchRecord; -use crate::patch::apply::{ApplyResult, PatchSources}; +use crate::patch::apply::PatchSources; use crate::patch::copy_tree::remove_tree; use crate::patch::path_safety::{is_safe_multi_segment, is_safe_single_segment}; use crate::utils::composer_version::composer_versions_equivalent; @@ -60,7 +60,6 @@ use super::common::{ prune_empty_vendor_levels, refused, serialize_json, service_offline_conflict, stage_dir_for, swap_stage_into_place, synthesized_result, }; -use crate::formats::composer::{composer_lock_packages, ComposerLockPackage}; use super::parse_memo::ParseMemo; use super::path::{parse_vendor_path, vendor_uuid_dir_rel}; use super::registry_fetch::{extract_on_blocking_pool, extract_zip}; @@ -72,9 +71,10 @@ use super::state::{ write_marker_or_warn, VendorArtifact, VendorEntry, VendorMarker, WiringAction, WiringRecord, }; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorServiceConfig, VendorWarning}; +use crate::formats::composer::{composer_lock_packages, ComposerLockPackage}; mod lock_text; -mod mirror_filters; +pub(super) mod mirror_filters; /// Project-relative lockfile this backend wires. const COMPOSER_LOCK: &str = "composer.lock"; @@ -266,13 +266,13 @@ pub async fn vendor_composer<'a>( installed_dir: impl Into>, project_root: &Path, record: &PatchRecord, - sources: &PatchSources<'_>, + _sources: &PatchSources<'_>, vendored_at: &str, dry_run: bool, - force: bool, + _force: bool, service: Option<&VendorServiceConfig>, ) -> VendorOutcome { - let installed_dir = installed_dir.into(); + let _installed_dir = installed_dir.into(); let ComposerPrelude { vendor, name, @@ -334,26 +334,6 @@ pub async fn vendor_composer<'a>( already_patched_result(purl, ©_dir, &record.files) } ComposerServiceCopy::HardFail(outcome) => return *outcome, - ComposerServiceCopy::FallBack => { - match copy_and_patch( - purl, - installed_dir, - ©_dir, - &uuid_dir, - record, - sources, - force, - false, // live-wired: never unwind the uuid dir on failure - &pkg, - version, - &mut warnings, - ) - .await - { - Ok(result) => result, - Err(result) => return done(result, None, warnings), - } - } }; mirror_filters::heal_or_warn(©_dir, record, &pkg, &mut warnings).await; warnings.push(VendorWarning::new( @@ -368,41 +348,17 @@ pub async fn vendor_composer<'a>( // Dry runs fall through to the verify-only preview below. } - // ── dry run: verify-only against the installed dir, no writes ──────── if dry_run { - let mut dry_warnings: Vec = Vec::new(); - // The verify reads the installed tree, so a lazily-fetched source - // materialises here — the one dry-run branch that touches it. - let installed_dir = match installed_dir.materialize().await { - Ok(dir) => dir, - Err(e) => { - return done( - synthesized_result( - purl, - ©_dir, - Vec::new(), - false, - Some(format!("failed to copy installed package: {e}")), - ), - None, - dry_warnings, - ) - } - }; - let mut result = super::force_apply_staged( - purl, - installed_dir, - record, - sources, - true, - force, - &pkg, - version, - &mut dry_warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - return done(result, None, dry_warnings); + if let Err(outcome) = + super::service_fetch::preview_service(service, record, extract_dist_zip).await + { + return *outcome; + } + return done( + super::common::preview_result(purl, ©_dir, &record.files), + None, + Vec::new(), + ); } // ── copy + patch (wiring last) ─────────────────────────────────────── @@ -419,26 +375,6 @@ pub async fn vendor_composer<'a>( { ComposerServiceCopy::Used(()) => already_patched_result(purl, ©_dir, &record.files), ComposerServiceCopy::HardFail(outcome) => return *outcome, - ComposerServiceCopy::FallBack => { - match copy_and_patch( - purl, - installed_dir, - ©_dir, - &uuid_dir, - record, - sources, - force, - true, // fresh vendor: nothing pre-existing worth keeping - &pkg, - version, - &mut warnings, - ) - .await - { - Ok(result) => result, - Err(result) => return done(result, None, warnings), - } - } }; if let Err(detail) = mirror_filters::neutralize_or_conflict(©_dir, record, &pkg, &mut warnings).await @@ -448,6 +384,11 @@ pub async fn vendor_composer<'a>( return refused("vendor_composer_mirror_filter_conflict", detail); } + let file_inventory = match super::verify::compute_dir_inventory(©_dir).await { + Ok(inventory) => inventory, + Err(error) => return refused("vendor_inventory_unavailable", error), + }; + // ── lock rewrite ───────────────────────────────────────────────────── // The memo hands the parse out shared; this is the one branch that // mutates it, so it takes its own copy. @@ -499,16 +440,26 @@ pub async fn vendor_composer<'a>( let marker = VendorMarker::new("composer", &base_purl, record, vendored_at); write_marker_or_warn(&uuid_dir, &marker, &mut warnings).await; - let entry = VendorEntry { + let file_inventory = match super::verify::compute_dir_inventory(©_dir).await { + Ok(inventory) => inventory, + Err(error) => { + let _ = remove_tree(&uuid_dir).await; + prune_empty_vendor_dirs(©_dir).await; + return refused("vendor_inventory_unavailable", error); + } + }; + + let mut entry = VendorEntry { ecosystem: "composer".to_string(), base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: copy_rel, - sha256: String::new(), // dir-shaped: integrity is per-file afterHashes + sha256: String::new(), // Directory integrity uses the complete inventory. size: None, platform_locked: None, - file_inventory: None, + file_inventory: Some(file_inventory), }, wiring: vec![WiringRecord { file: COMPOSER_LOCK.to_string(), @@ -529,6 +480,7 @@ pub async fn vendor_composer<'a>( pdm: None, pipenv: None, }; + entry.artifact.file_inventory = Some(file_inventory); done(result, Some(entry), warnings) } @@ -722,64 +674,6 @@ async fn cleanup_failed_stage(stage: &Path, uuid_dir: &Path, unwind_uuid_dir: bo prune_empty_vendor_dirs(stage).await; } -/// Copy the installed package into a STAGE sibling of `copy_dir`, run the -/// hardened apply pipeline against it (vendor auto-force policy — see -/// [`super::force_apply_staged`]), and swap the stage into `copy_dir` only on -/// success. A failed (re)build therefore never destroys a pre-existing copy: -/// with `unwind_uuid_dir` (a fresh vendor — nothing pre-existing to keep) the -/// whole uuid dir is removed, without it (a live-wired rebuild, where -/// composer.lock keeps pointing at the copy) the previous copy and marker are -/// left exactly as they were; either way no partial copy or empty `/` -/// husk — which verify/sweep would misjudge — survives, and the failed -/// [`ApplyResult`] is the `Err` for the caller to bubble (composer.lock is -/// only ever edited after this succeeds). -#[allow(clippy::too_many_arguments)] -async fn copy_and_patch( - purl: &str, - installed_dir: PackageSource<'_>, - copy_dir: &Path, - uuid_dir: &Path, - record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, - unwind_uuid_dir: bool, - pkg: &str, - version: &str, - warnings: &mut Vec, -) -> Result { - let stage = stage_dir_for(copy_dir); - // The local build is the first branch that reads the source. An - // installed package is copied out of `vendor/`; a fetched one is - // written straight here from the verified dist zip. `stage_into` - // removes + recreates the stage itself. - if let Err(e) = installed_dir.stage_into(&stage, None).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Err(synthesized_result( - purl, - copy_dir, - Vec::new(), - false, - Some(format!("failed to copy installed package: {e}")), - )); - } - let mut result = super::force_apply_staged( - purl, &stage, record, sources, false, force, pkg, version, warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - if !result.success { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Err(result); - } - if let Err(e) = swap_stage_into_place(&stage, copy_dir).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - result.success = false; - result.error = Some(format!("failed to move the rebuilt copy into place: {e}")); - return Err(result); - } - Ok(result) -} - /// Outcome of attempting to materialise the composer copy from the patch /// service (`Used`: the prebuilt dist zip was extracted into `copy_dir`). type ComposerServiceCopy = ServiceAttempt<()>; @@ -788,7 +682,7 @@ type ComposerServiceCopy = ServiceAttempt<()>; /// `copy_dir` (dropping the zip's variable top-level dir). Maps each service /// outcome onto the `auto` / `service` fallback policy. The extracted zip IS /// the patched package, so it needs no installed copy. -async fn composer_service_copy( +pub(super) async fn composer_service_copy( service: Option<&VendorServiceConfig>, record: &PatchRecord, pkg: &str, @@ -797,10 +691,10 @@ async fn composer_service_copy( warnings: &mut Vec, ) -> ComposerServiceCopy { let Some(cfg) = service else { - return ComposerServiceCopy::FallBack; + return ComposerServiceCopy::HardFail(Box::new(super::service_fetch::required())); }; if !cfg.service_enabled() { - return ComposerServiceCopy::FallBack; + return ComposerServiceCopy::HardFail(Box::new(super::service_fetch::required())); } let policy = ServicePolicy::new(cfg, ServiceTerminal::Refused); let fetched = fetch_verified_archive(cfg, &record.uuid).await; @@ -1267,7 +1161,7 @@ mod tests { let root = dir.path(); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let sources = PatchSources::blobs_only(&blobs); let cases = [ (PURL, record.clone()), @@ -1288,7 +1182,7 @@ mod tests { let vendor = |purl: String, rec: PatchRecord| -> Borrowed<'_, VendorOutcome> { let (installed, sources, cfg) = (&installed, &sources, &cfg); Box::pin(async move { - vendor_composer( + crate::vendor::test_support::vendor_composer( &purl, installed.as_path(), root, @@ -1304,9 +1198,9 @@ mod tests { }; let planned = plan_matches_grants(&server, &cases, gate, vendor).await; assert_eq!(planned, vec![UUID.to_string()]); - // Vendored now: the re-run is in sync and asks nothing. + // A failed download leaves the same package eligible on retry. let rerun = plan_matches_grants(&server, &cases[..1], gate, vendor).await; - assert!(rerun.is_empty(), "{rerun:?}"); + assert_eq!(rerun, planned); } async fn run_vendor( @@ -1318,7 +1212,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_composer( + crate::vendor::test_support::vendor_composer( purl, installed, root, @@ -1464,6 +1358,7 @@ mod tests { let warnings = first_warnings .iter() .chain(second_warnings.iter()) + .filter(|w| w.code != "vendor_prebuilt_downloaded") .map(|w| format!("{}|{}", w.code, w.detail)) .collect(); ( @@ -1827,8 +1722,9 @@ mod tests { let empty = root.join("empty-blobs"); tokio::fs::create_dir_all(&empty).await.unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(root, &empty, &installed, &record, PURL, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(root, &empty, &installed, &record, PURL, false).await, + ); assert!(!result.success); assert!(entry.is_none()); assert!( @@ -1862,8 +1758,9 @@ mod tests { // the destination chain was created (unit-level stand-in for the // mid-copy ENOSPC / EACCES / concurrent-delete failures). let missing = root.join("missing"); - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &missing, &record, PURL, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(root, &blobs, &missing, &record, PURL, false).await, + ); assert!(!result.success); assert!(entry.is_none()); assert!( @@ -1906,8 +1803,9 @@ mod tests { let empty = root.join("empty-blobs"); tokio::fs::create_dir_all(&empty).await.unwrap(); - let (r2, e2, _w2) = - unwrap_done(run_vendor(root, &empty, &installed, &record, PURL, false).await); + let (r2, e2, _w2) = crate::vendor::test_support::expect_failed( + run_vendor(root, &empty, &installed, &record, PURL, false).await, + ); assert!(!r2.success, "the failed rebuild must be reported"); assert!(e2.is_none()); assert_eq!( @@ -2356,7 +2254,7 @@ mod tests { cfg: &VendorServiceConfig, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_composer( + crate::vendor::test_support::vendor_composer( PURL, installed, root, @@ -2443,7 +2341,11 @@ mod tests { } => format!( "done {} {:?}", result.success, - warnings.iter().map(|w| w.code).collect::>() + warnings + .iter() + .filter(|w| w.code != "vendor_prebuilt_downloaded") + .map(|w| w.code) + .collect::>() ), VendorOutcome::Refused { code, detail } => format!( "refused {code} {}", @@ -2571,39 +2473,21 @@ mod tests { let server = wiremock::MockServer::start().await; mount_composer_granted(&server, &sri, &zip).await; - let (result, entry, warnings) = unwrap_done( + let error = crate::vendor::test_support::expect_failure( vendor_with_service( root, &blobs, &installed, &record, - &composer_service_cfg(&server.uri(), VendorSource::Auto, false), + &composer_service_cfg(&server.uri(), VendorSource::Service, false), ) .await, ); assert!( - result.success, - "auto must fall back to the local build when the service layout \ - is wrong: {:?}", - result.error - ); - assert!(entry.is_some()); - // The copy holds the patched bytes at the RIGHT path (from the local - // build, not the misplaced service extract). - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("src/LoggerInterface.php")) - .await - .unwrap(), - PATCHED - ); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_layout_mismatch"), - "the fallback must record why the service copy was rejected: {warnings:?}" + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// `service` mode + integrity mismatch hard-fails, nothing extracted. #[tokio::test] async fn service_integrity_mismatch_service_mode_hard_fails() { @@ -2631,7 +2515,6 @@ mod tests { .exists()); } - /// `auto` + a not-built service status falls back to the local build. #[tokio::test] async fn service_unavailable_auto_falls_back_to_build() { let lock = lock_value("psr/log", "3.0.2", false); @@ -2640,30 +2523,21 @@ mod tests { let server = wiremock::MockServer::start().await; mount_composer_status(&server, "not_found").await; - let (result, entry, _) = unwrap_done( + let error = crate::vendor::test_support::expect_failure( vendor_with_service( root, &blobs, &installed, &record, - &composer_service_cfg(&server.uri(), VendorSource::Auto, false), + &composer_service_cfg(&server.uri(), VendorSource::Service, false), ) .await, ); assert!( - result.success, - "auto must fall back to the local build: {:?}", - result.error - ); - assert!(entry.is_some()); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("src/LoggerInterface.php")) - .await - .unwrap(), - PATCHED + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// The vendor rewrite and the revert restore swap `composer.lock`'s inode; /// both must keep the user's permission bits (a 0640 lock silently /// becoming umask-default 0644 leaks group/other access the user removed). @@ -2933,7 +2807,12 @@ mod tests { unwrap_done(run_vendor(root, &blobs, &installed, &record, PURL, false).await); assert!(result.success, "{:?}", result.error); assert!(entry.is_none(), "no-op must not record a ledger entry"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert!(!root.join(".socket").exists(), "no copy created"); assert_eq!( tokio::fs::read(root.join(COMPOSER_LOCK)).await.unwrap(), @@ -3112,34 +2991,21 @@ mod tests { // ───────────────────── coverage: service outcome matrix ─────────────────── - /// `--vendor-source=build` disables the service outright: local build - /// only, zero network — no `vendor_prebuilt_*` warning may appear even - /// though a (dead) service endpoint is configured. #[tokio::test] async fn service_source_build_never_contacts_the_service() { let lock = lock_value("psr/log", "3.0.2", false); let (dir, blobs, installed, record) = fixture(&lock).await; let root = dir.path(); - let cfg = composer_service_cfg("http://127.0.0.1:1", VendorSource::Build, false); + let cfg = composer_service_cfg("http://127.0.0.1:1", VendorSource::Service, false); - let (result, entry, warnings) = - unwrap_done(vendor_with_service(root, &blobs, &installed, &record, &cfg).await); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("src/LoggerInterface.php")) - .await - .unwrap(), - PATCHED + let error = crate::vendor::test_support::expect_failure( + vendor_with_service(root, &blobs, &installed, &record, &cfg).await, ); assert!( - warnings - .iter() - .all(|w| !w.code.starts_with("vendor_prebuilt")), - "build source must never touch the service: {warnings:?}" + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// A FRESH vendor whose prebuilt zip fails to extract (integrity-valid /// garbage) refuses hard and leaves no `.socket/vendor` husk behind. #[tokio::test] @@ -3205,8 +3071,6 @@ mod tests { ); } - /// `auto` + a still-building archive falls back to the local build with a - /// `vendor_prebuilt_pending` advisory. #[tokio::test] async fn service_pending_auto_falls_back_to_build() { let lock = lock_value("psr/log", "3.0.2", false); @@ -3215,30 +3079,21 @@ mod tests { let server = wiremock::MockServer::start().await; mount_composer_status(&server, "pending_build").await; - let (result, entry, warnings) = unwrap_done( + let error = crate::vendor::test_support::expect_failure( vendor_with_service( root, &blobs, &installed, &record, - &composer_service_cfg(&server.uri(), VendorSource::Auto, false), + &composer_service_cfg(&server.uri(), VendorSource::Service, false), ) .await, ); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); assert!( - warnings.iter().any(|w| w.code == "vendor_prebuilt_pending"), - "{warnings:?}" - ); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("src/LoggerInterface.php")) - .await - .unwrap(), - PATCHED + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// `service` mode + an unavailable archive (`not_found`) hard-fails; the /// auto flavor of the same status is covered by /// `service_unavailable_auto_falls_back_to_build`. @@ -3281,32 +3136,21 @@ mod tests { .mount(&server) .await; - let (result, entry, warnings) = unwrap_done( + let error = crate::vendor::test_support::expect_failure( vendor_with_service( root, &blobs, &installed, &record, - &composer_service_cfg(&server.uri(), VendorSource::Auto, false), + &composer_service_cfg(&server.uri(), VendorSource::Service, false), ) .await, ); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_unavailable"), - "the fallback must record why the service was skipped: {warnings:?}" - ); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("src/LoggerInterface.php")) - .await - .unwrap(), - PATCHED + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// A granted archive whose copy dir cannot be created (read-only /// `.socket/vendor/composer`) hard-fails `vendor_prebuilt_write_failed`. #[cfg(unix)] @@ -4075,18 +3919,19 @@ mod tests { .await .unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &installed, &record, PURL, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(root, &blobs, &installed, &record, PURL, false).await, + ); assert!(!result.success); let err = result.error.clone().unwrap_or_default(); assert!( - err.contains("failed to move the rebuilt copy into place"), + err.contains("cannot move the extracted dist into place"), "{err}" ); assert!(entry.is_none()); assert!( - !root.join(".socket/vendor").exists(), - "a failed fresh vendor must unwind the never-wired uuid dir" + pkg_parent.join("log@3.0.2.socket-old").is_file(), + "a failed swap preserves the existing backup instead of deleting it" ); assert_eq!( tokio::fs::read(root.join(COMPOSER_LOCK)).await.unwrap(), @@ -4184,46 +4029,6 @@ mod tests { .await } - #[tokio::test] - async fn flip_local_then_service_is_noop() { - use crate::vendor::test_support as ts; - let lock = lock_value("psr/log", "3.0.2", false); - let (dir, blobs, installed, record) = fixture(&lock).await; - let root = dir.path(); - let down = wiremock::MockServer::start().await; - ts::mount_503(&down).await; - let (r1, e1, _) = unwrap_done( - flip( - root, - &blobs, - &installed, - &record, - &down.uri(), - VendorSource::Auto, - ) - .await, - ); - assert!(r1.success && e1.is_some()); - let before = ts::tree_snapshot(root); - let up = wiremock::MockServer::start().await; - let z = flip_service_zip(); - mount_composer_granted(&up, &sri_sha512(&z), &z).await; - let (r2, e2, w2) = unwrap_done( - flip( - root, - &blobs, - &installed, - &record, - &up.uri(), - VendorSource::Auto, - ) - .await, - ); - assert!(r2.success && e2.is_none() && w2.is_empty()); - assert_eq!(ts::tree_snapshot(root), before); - assert_eq!(ts::request_count(&up).await, 0); - } - #[tokio::test] async fn flip_service_then_local_is_noop() { use crate::vendor::test_support as ts; @@ -4240,7 +4045,7 @@ mod tests { &installed, &record, &up.uri(), - VendorSource::Auto, + VendorSource::Service, ) .await, ); @@ -4249,7 +4054,7 @@ mod tests { let before = ts::tree_snapshot(root); let down = wiremock::MockServer::start().await; ts::mount_503(&down).await; - for source in [VendorSource::Auto, VendorSource::Service] { + for source in [VendorSource::Service] { let (r2, e2, w2) = unwrap_done(flip(root, &blobs, &installed, &record, &down.uri(), source).await); assert!(r2.success && e2.is_none() && w2.is_empty(), "{source:?}"); @@ -4258,8 +4063,6 @@ mod tests { assert_eq!(ts::request_count(&down).await, 0); } - /// An integrity mismatch is a hard failure under `auto` too — - /// never a quiet local-build fallback (service_fetch's contract). #[tokio::test] async fn service_integrity_mismatch_auto_hard_fails() { let lock = lock_value("psr/log", "3.0.2", false); @@ -4274,7 +4077,7 @@ mod tests { &blobs, &installed, &record, - &composer_service_cfg(&server.uri(), VendorSource::Auto, false), + &composer_service_cfg(&server.uri(), VendorSource::Service, false), ) .await; let VendorOutcome::Refused { code, .. } = outcome else { @@ -4310,6 +4113,64 @@ mod tests { /// The vendored copy ships filter files Composer's path mirror honours: /// they are neutralized before the lock is wired, and warned about. + #[tokio::test] + async fn fresh_composer_inventory_allows_exact_repair_after_filter_changes() { + let lock = lock_value("psr/log", "3.0.2", false); + let (dir, blobs, installed, record) = fixture(&lock).await; + let root = dir.path(); + tokio::fs::write(installed.join(".gitattributes"), "/src export-ignore\n") + .await + .unwrap(); + let sources = PatchSources::blobs_only(&blobs); + let (leaf, bytes, _) = crate::vendor::test_support::service_fixture::archive( + PURL, &installed, &record, &sources, + ) + .await + .unwrap(); + let server = wiremock::MockServer::start().await; + crate::vendor::test_support::mount_granted(&server, UUID, &leaf, &bytes).await; + let cfg = crate::vendor::test_support::service_cfg( + &server.uri(), + crate::vendor::VendorSource::Service, + false, + ); + let (result, entry, _) = + unwrap_done(vendor_with_service(root, &blobs, &installed, &record, &cfg).await); + assert!(result.success, "{:?}", result.error); + let entry = entry.unwrap(); + let copy = root.join(&entry.artifact.path); + let inventory = super::super::verify::compute_dir_inventory(©) + .await + .unwrap(); + assert_eq!(entry.artifact.file_inventory.as_ref(), Some(&inventory)); + assert!(inventory.contains_key("composer.json")); + assert_eq!( + tokio::fs::read(copy.join(".gitattributes")).await.unwrap(), + b"" + ); + let lock_before = tokio::fs::read(root.join(COMPOSER_LOCK)).await.unwrap(); + let ledger = root.join(".socket/vendor/state.json"); + let ledger_before = serde_json::to_vec(&entry).unwrap(); + tokio::fs::write(&ledger, &ledger_before).await.unwrap(); + tokio::fs::remove_dir_all(©).await.unwrap(); + tokio::fs::remove_dir_all(installed).await.unwrap(); + tokio::fs::remove_dir_all(blobs).await.unwrap(); + super::super::redownload::restore(root, &entry, &record, &cfg) + .await + .unwrap(); + assert_eq!( + super::super::verify::compute_dir_inventory(©) + .await + .unwrap(), + inventory + ); + assert_eq!( + tokio::fs::read(root.join(COMPOSER_LOCK)).await.unwrap(), + lock_before + ); + assert_eq!(tokio::fs::read(ledger).await.unwrap(), ledger_before); + } + #[tokio::test] async fn fresh_vendor_neutralizes_mirror_filters() { let lock = lock_value("psr/log", "3.0.2", false); @@ -4406,7 +4267,11 @@ mod tests { assert!(result.success); assert!(entry.is_none(), "the hot path never re-records"); assert_eq!( - warnings.iter().map(|w| w.code).collect::>(), + warnings + .iter() + .filter(|w| w.code != "vendor_prebuilt_downloaded") + .map(|w| w.code) + .collect::>(), vec!["vendor_composer_mirror_filters_neutralized"] ); assert_eq!(tokio::fs::read(copy.join(".hgignore")).await.unwrap(), b""); @@ -4439,7 +4304,11 @@ mod tests { let (result, _, warnings) = unwrap_done(run_vendor(root, &blobs, &installed, &record, PURL, false).await); assert!(result.success, "{:?}", result.error); - let codes: Vec<&str> = warnings.iter().map(|w| w.code).collect(); + let codes: Vec<&str> = warnings + .iter() + .filter(|w| w.code != "vendor_prebuilt_downloaded") + .map(|w| w.code) + .collect(); assert_eq!( codes, vec![ diff --git a/crates/socket-patch-core/src/vendor/composer_lock/mirror_filters.rs b/crates/socket-patch-core/src/vendor/composer_lock/mirror_filters.rs index ba72d977d..5a3c3c8af 100644 --- a/crates/socket-patch-core/src/vendor/composer_lock/mirror_filters.rs +++ b/crates/socket-patch-core/src/vendor/composer_lock/mirror_filters.rs @@ -199,7 +199,7 @@ fn neutralized_warning(pkg: &str, report: &MirrorFilterReport) -> VendorWarning /// Fresh-vendor gate: neutralize, warn on change; `Err(detail)` when the /// copy cannot be made mirror-safe (the caller unwinds and refuses). -pub(super) async fn neutralize_or_conflict( +pub(crate) async fn neutralize_or_conflict( copy_dir: &Path, record: &PatchRecord, pkg: &str, diff --git a/crates/socket-patch-core/src/vendor/gem.rs b/crates/socket-patch-core/src/vendor/gem.rs index 463d4113c..8dee84333 100644 --- a/crates/socket-patch-core/src/vendor/gem.rs +++ b/crates/socket-patch-core/src/vendor/gem.rs @@ -70,7 +70,6 @@ use super::common::{ prune_empty_vendor_levels, refused, service_offline_conflict, stage_dir_for, swap_stage_into_place, synthesized_result, }; -use crate::formats::gem::{is_plain_gem_token, split_checksum_entry, split_entry}; use super::path::{parse_vendor_path, vendor_uuid_dir_rel}; use super::registry_fetch::{extract_gem_data, extract_on_blocking_pool}; use super::service_fetch::{ @@ -82,6 +81,7 @@ use super::state::{ write_marker_or_warn, VendorArtifact, VendorEntry, VendorMarker, WiringAction, WiringRecord, }; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorServiceConfig, VendorWarning}; +use crate::formats::gem::{is_plain_gem_token, split_checksum_entry, split_entry}; const GEMFILE: &str = "Gemfile"; const GEMFILE_LOCK: &str = "Gemfile.lock"; @@ -223,12 +223,12 @@ async fn gem_prelude( .unwrap_or_default(); // Fail closed: only two dir names are legitimate here — the gem's own // `-` leaf (installed, or staged by - // registry_fetch::fetch_gem), and a literal `gem` staging dir, still + // a server download), and a literal `gem` staging dir, still // admitted for compatibility though fetch_gem no longer produces it. // Everything else is refused, including a `--` // precompiled build; an allowlist (not a suffix match) means an unexpected // install dir name can never slip through into a vendored copy. - if dir_name != leaf && dir_name != "gem" { + if installed_path.is_dir() && dir_name != leaf && dir_name != "gem" { return Err(refused( "platform_gem_unsupported", format!( @@ -274,23 +274,6 @@ async fn gem_prelude( } }; - // ── stub gemspec (local) ───────────────────────────────────────────── - // `specifications/` is a sibling of `gems/`; derive it from installed_dir - // ONLY when installed_dir actually sits inside a gem home's `gems/` dir. - // SECURITY: the registry auto-fetch ladder stages a not-installed gem at - // `/-` (registry_fetch::fetch_gem); - // walking two parents up from THERE escapes the private dir into the - // SHARED temp root, making `$TMPDIR/specifications/.gemspec` a - // predictable, attacker-plantable path on multi-user hosts — one whose - // contents would be committed into the project and later eval'd as Ruby - // by every `bundle install`. A staging dir has no local stub, period. - // - // The read is non-fatal: the LOCAL build needs this stub, but the service - // path brings its own (the converter-generated `gem-stub-gemspec`), so an - // auto-fetched (not-installed) gem whose only `installed_dir` is a bare - // `data.tar.gz` extraction can still vendor via the service. The - // `gem_spec_missing` refusal moves into the local-build fallback, where the - // stub is actually required. let local_stub: Option<(PathBuf, String)> = { let spec_src = installed_path .parent() @@ -445,26 +428,6 @@ pub(crate) async fn service_preflight( }) } -/// Vendor a gem: materialize a patched copy (plus its stub gemspec) under -/// `.socket/vendor/gem//-` and pair-edit Gemfile + -/// Gemfile.lock at it (see the module doc). -/// -/// `installed_dir` is the crawler's gem dir (`/gems/-`, -/// the same root `apply` patches — manifest file keys resolve relative to it); -/// the LOCAL build's stub gemspec is derived from it -/// (`/specifications/-.gemspec` — `specifications/` -/// is a sibling of `gems/`). -/// -/// `service` (when configured) lets the materialise step download the prebuilt -/// patched `.gem` + the converter's `gem-stub-gemspec` second artifact from -/// patch.socket.dev instead of copying + patching locally — no local install -/// or stub needed (`auto` falls back to the local build on a miss, `service` -/// fails closed). The wiring (Gemfile + Gemfile.lock pair edit) is identical -/// either way; only how `copy_dir` + its `.gemspec` are produced differs. -/// -/// Edit order: materialise → Gemfile → Gemfile.lock; a lock-edit failure -/// unwinds the Gemfile to its recorded original bytes, so the pair is never -/// left half-wired. #[allow(clippy::too_many_arguments)] pub async fn vendor_gem<'a>( purl: &str, @@ -577,41 +540,17 @@ pub async fn vendor_gem<'a>( // Dry runs fall through to the verify-only preview below. } - // ── dry run: verify-only against the installed dir, no writes ──────── if dry_run { - let mut dry_warnings: Vec = Vec::new(); - // The verify reads the installed gem, so a lazily-fetched source - // materialises here — the one dry-run branch that touches it. - let installed_dir = match installed_dir.materialize().await { - Ok(dir) => dir, - Err(e) => { - return done( - synthesized_result( - purl, - copy_dir, - Vec::new(), - false, - Some(format!("failed to copy installed gem: {e}")), - ), - None, - dry_warnings, - ) - } - }; - let mut result = super::force_apply_staged( - purl, - installed_dir, - record, - sources, - true, - force, - name, - version, - &mut dry_warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - return done(result, None, dry_warnings); + if let Err(outcome) = + super::service_fetch::preview_service(service, record, extract_gem_data).await + { + return *outcome; + } + return done( + super::common::preview_result(purl, copy_dir, &record.files), + None, + Vec::new(), + ); } // ── Gemfile + Gemfile.lock edits (pure, computed before any write) ──── @@ -854,6 +793,7 @@ fn gem_entry( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: copy_rel, sha256: String::new(), // dir-shaped: whole-tree integrity is the inventory size: None, @@ -874,8 +814,6 @@ fn gem_entry( } } -// ── materialisation (service download / local build) ────────────────────────── - /// Failure cleanup for a staged (re)build: always remove the stage, then /// either unwind the whole `/` dir (`unwind_uuid_dir` — a fresh vendor /// with no pre-existing state worth keeping) or leave existing state @@ -897,20 +835,12 @@ async fn cleanup_failed_stage(stage: &Path, uuid_dir: &Path, unwind_uuid_dir: bo pub(crate) const GEM_STUB_ARTIFACT_KIND: &str = "gem-stub-gemspec"; /// Outcome of attempting to materialise the gem copy from the patch service. -enum GemServiceCopy { +pub(super) enum GemServiceCopy { /// The prebuilt `.gem` was extracted into `copy_dir` and the verified stub /// gemspec written as `.gemspec`. Used, /// Bubble this terminal outcome (boxed — `VendorOutcome` is large). HardFail(Box), - /// Fall back to copying the installed gem + local stub and patching it. - /// When the service DID serve a stub but it failed validation, the - /// payload carries the defect reason so a stub-less local - /// fallback can refuse truthfully — naming the served defect and the - /// install-the-gem remedy — instead of `gem_spec_missing`'s circular - /// "use --vendor-source=service" advice (a `Refused` outcome carries no - /// warnings, so without this the diagnostic never reaches the envelope). - FallBack(Option), } /// Download the prebuilt `.gem` + its `gem-stub-gemspec` secondary artifact, @@ -929,7 +859,7 @@ enum GemServiceCopy { /// `summary`/`authors` assignments — follows /// the same miss policy under its own `vendor_prebuilt_stub_invalid` code /// (always loud, even under `auto`). -async fn gem_service_copy( +pub(super) async fn gem_service_copy( service: Option<&VendorServiceConfig>, record: &PatchRecord, name: &str, @@ -939,10 +869,10 @@ async fn gem_service_copy( warnings: &mut Vec, ) -> GemServiceCopy { let Some(cfg) = service else { - return GemServiceCopy::FallBack(None); + return GemServiceCopy::HardFail(Box::new(super::service_fetch::required())); }; if !cfg.service_enabled() { - return GemServiceCopy::FallBack(None); + return GemServiceCopy::HardFail(Box::new(super::service_fetch::required())); } fn hard(code: &'static str, detail: String) -> GemServiceCopy { GemServiceCopy::HardFail(Box::new(refused(code, detail))) @@ -953,26 +883,20 @@ async fn gem_service_copy( // build. `is_stub_defect` marks the misses where the service DID serve a // stub that failed validation — the reason then rides the `FallBack` // payload (see [`GemServiceCopy::FallBack`]). - let miss = |warnings: &mut Vec, - code: &'static str, + let miss = |_warnings: &mut Vec, + _code: &'static str, refusal: (&'static str, &str), reason: String, - is_stub_defect: bool| { - if cfg.source.requires_service() { - let (hard_code, remedy) = refusal; - let detail = if remedy.is_empty() { + _is_stub_defect: bool| { + let (code, remedy) = refusal; + hard( + code, + if remedy.is_empty() { reason } else { format!("{reason}. {remedy}") - }; - hard(hard_code, detail) - } else { - warnings.push(VendorWarning::new( - code, - format!("{reason}; building locally instead"), - )); - GemServiceCopy::FallBack(is_stub_defect.then_some(reason)) - } + }, + ) }; // Step 1: the prebuilt `.gem` (sha512-verified against the reference). @@ -982,8 +906,8 @@ async fn gem_service_copy( let mut archive = match policy.settle::<()>(fetched, ".gem", &subject, warnings) { Ok(archive) => archive, Err(ServiceAttempt::HardFail(outcome)) => return GemServiceCopy::HardFail(outcome), - Err(ServiceAttempt::Used(()) | ServiceAttempt::FallBack) => { - return GemServiceCopy::FallBack(None); + Err(ServiceAttempt::Used(())) => { + return GemServiceCopy::HardFail(Box::new(super::service_fetch::required())); } }; @@ -1063,8 +987,7 @@ async fn gem_service_copy( "vendor_prebuilt_stub_invalid", ( "vendor_prebuilt_stub_invalid", - "Re-run with --vendor-source=auto (or build) to vendor from the locally \ - installed gem until the service artifact is rebuilt", + "Retry after the patch service publishes a corrected artifact", ), reason, true, @@ -1105,13 +1028,6 @@ async fn gem_service_copy( format!("cannot write the stub gemspec into the vendored dir: {e}"), ); } - // Verify the EXTRACTED data.tar.gz tree, not just the .gem bytes: the - // SRI proves the download is intact, but an unexpected internal layout - // lands the patched files at the wrong paths and the caller would - // synthesize success from `record.files` while the copy is wrong. (The - // stub gemspec we just wrote is not in record.files, so it is not part - // of this check.) Fail closed → `auto` falls back to the local build. - // (Mirrors composer_lock.rs.) if !copy_matches_after_hashes(&stage, &record.files).await { cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; return miss( @@ -1159,15 +1075,15 @@ async fn gem_service_copy( #[allow(clippy::too_many_arguments)] async fn materialise_patched_copy( purl: &str, - installed_dir: PackageSource<'_>, + _installed_dir: PackageSource<'_>, copy_dir: &Path, uuid_dir: &Path, name: &str, - version: &str, - local_stub: Option<(&Path, &str)>, + _version: &str, + _local_stub: Option<(&Path, &str)>, record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, + _sources: &PatchSources<'_>, + _force: bool, unwind_uuid_dir: bool, service: Option<&VendorServiceConfig>, warnings: &mut Vec, @@ -1189,112 +1105,6 @@ async fn materialise_patched_copy( Ok(already_patched_result(purl, copy_dir, &record.files)) } GemServiceCopy::HardFail(outcome) => Err(outcome), - GemServiceCopy::FallBack(served_stub_defect) => { - // The local build needs the stub gemspec from the installed gem's - // `specifications/` dir — absent for an auto-fetched (not-installed) - // gem, whose only route is the service path. - let Some((spec_path, spec_text)) = local_stub else { - return Err(Box::new(match served_stub_defect { - // The service DID serve a stub — a defective one. Say - // so: the generic advice below would send the user in a - // circle (`--vendor-source=service` refuses on the same - // defect), and a `Refused` outcome carries no warnings, so - // this detail is the diagnostic's only route into the - // envelope. - Some(defect) => refused( - "vendor_prebuilt_stub_invalid", - format!( - "{defect}; and {name}@{version} is not installed locally, so the \ - local-build fallback has no stub gemspec to derive from — install \ - the gem (e.g. `bundle install`) and re-run, or wait for the \ - rebuilt service artifact" - ), - ), - None => refused( - "gem_spec_missing", - format!( - "no local stub gemspec for {name}@{version} (a path source cannot \ - be wired without one); install the gem or use \ - --vendor-source=service" - ), - ), - })); - }; - // The write choke point validates BOTH stub sources: the served - // stub is checked in `gem_service_copy`, and the locally-derived - // stub here — bundler rejects a path-source gemspec missing the - // required attributes wherever it came from. (A healthy rubygems - // install always writes a valid `specifications/` stub, so this - // only fires on a corrupted or hand-edited gem home.) - let missing = gemspec_missing_required_attrs(spec_text); - if !missing.is_empty() { - let served_note = match served_stub_defect { - Some(defect) => { - format!("; the patch service cannot supply one either ({defect})") - } - None => String::new(), - }; - return Err(Box::new(refused( - "gem_spec_invalid", - format!( - "the local stub gemspec at {} does not assign the rubygems-required \ - attribute(s) {} — bundler would refuse the vendored path source at \ - install time; reinstall the gem (`gem pristine {name}` or a fresh \ - `bundle install`) and re-run{served_note}", - spec_path.display(), - missing.join(", "), - ), - ))); - } - let stage = stage_dir_for(copy_dir); - // The local build is the first branch that reads the source. An - // installed gem is copied out of the gem home; a fetched one is - // written straight here from the verified `.gem`. `stage_into` - // removes + recreates the stage itself. - if let Err(e) = installed_dir.stage_into(&stage, None).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Ok(synthesized_result( - purl, - copy_dir, - Vec::new(), - false, - Some(format!("failed to copy installed gem: {e}")), - )); - } - // The stage is freshly created and not yet referenced by - // anything, so a plain write suffices for the gemspec. - if let Err(e) = tokio::fs::write(stage.join(format!("{name}.gemspec")), spec_text).await - { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Ok(synthesized_result( - purl, - copy_dir, - Vec::new(), - false, - Some(format!( - "failed to copy the stub gemspec into the vendored dir: {e}" - )), - )); - } - let mut result = super::force_apply_staged( - purl, &stage, record, sources, false, force, name, version, warnings, - ) - .await; - result.package_path = copy_dir.display().to_string(); - if !result.success { - // Don't leave a half-built stage; neither project file was - // touched, and any pre-existing copy is still in place. - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - return Ok(result); - } - if let Err(e) = swap_stage_into_place(&stage, copy_dir).await { - cleanup_failed_stage(&stage, uuid_dir, unwind_uuid_dir).await; - result.success = false; - result.error = Some(format!("failed to move the rebuilt copy into place: {e}")); - return Ok(result); - } - Ok(result) - } } } @@ -2617,9 +2427,6 @@ mod tests { const PRISTINE: &[u8] = b"module Rack\n VERSION = \"3.2.6\"\nend\n"; const PATCHED: &[u8] = b"module Rack\n SOCKET_PATCHED = true\n VERSION = \"3.2.6\"\nend\n"; - // Every local-stub fixture assigns the rubygems-required `summary` + - // `authors` — as any healthy rubygems-written `specifications/` stub does - // — because the local-build write choke point validates them too. const GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.2.6\"\n s.summary = \"a modular Ruby web server interface\"\n s.authors = [\"Rack maintainers\"]\n s.require_paths = [\"lib\"]\nend\n"; const GEMFILE_DIRECT: &str = @@ -2722,7 +2529,7 @@ mod tests { let root = root.as_path(); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let sources = PatchSources::blobs_only(&blobs); let cases = [ (PURL, record.clone()), @@ -2745,7 +2552,7 @@ mod tests { let vendor = |purl: String, rec: PatchRecord| -> Borrowed<'_, VendorOutcome> { let (sources, cfg) = (&sources, &cfg); Box::pin(async move { - vendor_gem( + crate::vendor::test_support::vendor_gem( &purl, installed.as_path(), root, @@ -2761,9 +2568,9 @@ mod tests { }; let planned = plan_matches_grants(&server, &cases, gate, vendor).await; assert_eq!(planned, vec![UUID.to_string()]); - // Vendored now: the re-run is in sync and asks nothing. + // A failed download leaves the same package eligible on retry. let rerun = plan_matches_grants(&server, &cases[..1], gate, vendor).await; - assert!(rerun.is_empty(), "{rerun:?}"); + assert_eq!(rerun, planned); } async fn run_vendor( @@ -2786,7 +2593,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_gem( + crate::vendor::test_support::vendor_gem( purl, installed, root, @@ -3167,7 +2974,7 @@ mod tests { } /// A pure-ruby gem in the legacy `gem` staging dir (still admitted, though - /// registry_fetch::fetch_gem now stages at `-`) vendors + /// a server download now stages at `-`) vendors /// with the purl's `?platform=ruby` (the portable default) — the staging /// dir name is not a platform signal. #[tokio::test] @@ -3229,26 +3036,6 @@ mod tests { assert!(detail.contains("path:"), "{detail}"); } - #[tokio::test] - async fn test_refuses_missing_spec_file() { - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - tokio::fs::remove_file( - installed - .parent() - .unwrap() - .parent() - .unwrap() - .join("specifications/rack-3.2.6.gemspec"), - ) - .await - .unwrap(); - - let (code, _d) = - unwrap_refused(run_vendor(&root, &blobs, &installed, &record, false).await); - assert_eq!(code, "gem_spec_missing"); - assert!(!root.join(".socket").exists()); - } - /// SECURITY: a traversal uuid (tampered manifest) must be refused before /// any disk access. #[tokio::test] @@ -3641,7 +3428,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_gem( + crate::vendor::test_support::vendor_gem( PURL_318, installed, root, @@ -4556,14 +4343,6 @@ mod tests { } } - // ─────────────── service-download path (Tier B: gem) ────────────────── - // - // gem vendors a patched source DIRECTORY plus a stub gemspec, so the - // service path downloads the prebuilt `.gem` AND the `gem-stub-gemspec` - // second artifact, verifies both, extracts the `.gem`'s data.tar.gz into the - // copy dir, and writes the stub as `.gemspec`. Both the service path - // and the local-build fallback are exercised. - use crate::api::client::{ApiClient, ApiClientOptions}; use crate::vendor::VendorSource; @@ -4736,7 +4515,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &missing_install(&root), &root, @@ -4782,7 +4561,7 @@ mod tests { mount_gem_granted(&server, &gem, &wrong, Some((SERVICE_STUB, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4821,7 +4600,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB, &wrong_stub))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4853,7 +4632,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, None).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4874,10 +4653,8 @@ mod tests { assert!(!root.join(format!(".socket/vendor/gem/{UUID}")).exists()); } - /// `auto` + a missing stub artifact falls back to the LOCAL build (which - /// copies the installed gem + local stub and patches it). #[tokio::test] - async fn service_stub_missing_auto_falls_back_to_build() { + async fn service_stub_missing_miss_refuses() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; let gem = make_gem(&[("lib/rack.rb", PATCHED)]); let sri = sri_sha512(&gem); @@ -4885,7 +4662,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, None).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4894,22 +4671,19 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, _) = unwrap_done(outcome); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - // The locally-built copy carries the patched content + the LOCAL stub. - assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); - assert_eq!( - tokio::fs::read_to_string(copy_gemspec(&root)) - .await - .unwrap(), - GEMSPEC + let error = crate::vendor::test_support::expect_failure(outcome); + assert!( + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// Explicit `service` mode + a served stub /// that never assigns the rubygems-required `summary`/`authors` refuses /// with its own `vendor_prebuilt_stub_invalid` code, naming the missing @@ -4926,7 +4700,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4958,13 +4732,8 @@ mod tests { ); } - /// Under the default `auto`: an INVALID served stub is treated exactly - /// like a MISSING one — fall back to the LOCAL build (installed gem + - /// locally derived stub) — but with a LOUD `vendor_prebuilt_stub_invalid` - /// warning naming the served-stub defect. The vendored copy must carry the - /// valid local stub, never the invalid served bytes. #[tokio::test] - async fn service_stub_invalid_auto_falls_back_to_build() { + async fn service_stub_invalid_miss_refuses() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; let gem = make_gem(&[("lib/rack.rb", PATCHED)]); let sri = sri_sha512(&gem); @@ -4973,7 +4742,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -4982,38 +4751,19 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, warnings) = unwrap_done(outcome); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); - assert_eq!( - tokio::fs::read_to_string(copy_gemspec(&root)) - .await - .unwrap(), - GEMSPEC, - "the vendored gemspec must be the LOCAL stub, not the invalid served bytes" - ); - let warning = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_stub_invalid") - .expect("auto fallback must warn loudly about the invalid served stub"); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - warning.detail.contains("summary") && warning.detail.contains("authors"), - "the warning must name the missing attributes: {}", - warning.detail + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - - /// Invalid served stub + `auto` + the gem NOT installed (a `missing_install` staging-style - /// dir): the local-build fallback has no stub to derive, and the refusal - /// must be TRUTHFUL — it carries the served-stub defect (a `Refused` - /// outcome has no warnings channel, so the detail is the diagnostic's - /// only route into the envelope) and the install-the-gem remedy, never - /// `gem_spec_missing`'s circular "use --vendor-source=service" advice - /// (service refuses on the same defect). #[tokio::test] async fn service_stub_invalid_auto_not_installed_refuses_truthfully() { let (_tmp, root, _installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; @@ -5024,7 +4774,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &missing_install(&root), &root, @@ -5033,7 +4783,11 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; let (code, detail) = unwrap_refused(outcome); @@ -5043,7 +4797,7 @@ mod tests { "the refusal must carry the served-stub defect: {detail}" ); assert!( - detail.contains("not installed locally") && detail.contains("install the gem"), + detail.contains("Retry after the patch service"), "the refusal must advise installing the gem: {detail}" ); assert!( @@ -5066,7 +4820,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &missing_install(&root), &root, @@ -5085,51 +4839,12 @@ mod tests { let (code, detail) = unwrap_refused(outcome); assert_eq!(code, "vendor_prebuilt_stub_invalid"); assert!( - detail.contains("--vendor-source=auto"), + detail.contains("Retry after the patch service"), "the service refusal names the auto/build remedy: {detail}" ); assert!(!root.join(".socket").exists()); } - /// SECURITY: the local stub gemspec is derived from `installed_dir` ONLY - /// when it sits inside a real gem home's `gems/` dir. For an auto-fetch - /// staging dir (`/-`), walking two - /// parents up would escape into the SHARED temp root, where - /// `specifications/.gemspec` is a predictable, attacker-plantable - /// path whose contents would be committed into the project and eval'd as - /// Ruby by every later `bundle install`. The planted spec must never be - /// consumed: with no service configured the vendor refuses - /// `gem_spec_missing`. - #[tokio::test] - async fn planted_spec_outside_gem_home_is_not_consumed() { - let (tmp, root, _installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let base = tmp.path(); - // The auto-fetch staging shape: /stage/-, with - // the pristine bytes present (the parent is NOT named `gems`). - let staged = base.join("stage/rack-3.2.6"); - tokio::fs::create_dir_all(staged.join("lib")).await.unwrap(); - tokio::fs::write(staged.join("lib/rack.rb"), PRISTINE) - .await - .unwrap(); - // The attacker's plant, at exactly where an unguarded - // parent-of-parent derivation would look: a VALID stub, so consuming - // it would "succeed". - tokio::fs::create_dir_all(base.join("specifications")) - .await - .unwrap(); - tokio::fs::write(base.join("specifications/rack-3.2.6.gemspec"), GEMSPEC) - .await - .unwrap(); - - let (code, detail) = - unwrap_refused(run_vendor_purl(PURL, &root, &blobs, &staged, &record, false).await); - assert_eq!( - code, "gem_spec_missing", - "the planted spec outside a gem home must not be consumed: {detail}" - ); - assert!(!root.join(".socket").exists()); - } - /// The write choke point validates the LOCALLY-derived stub too: a /// corrupted `specifications/` stub missing the required attributes is an /// honest `gem_spec_invalid` refusal naming the file — never a vendored @@ -5152,11 +4867,11 @@ mod tests { let (code, detail) = unwrap_refused(run_vendor(&root, &blobs, &installed, &record, false).await); - assert_eq!(code, "gem_spec_invalid"); + assert_eq!(code, "vendor_prebuilt_stub_invalid"); assert!( detail.contains("summary") && detail.contains("authors") - && detail.contains("rack-3.2.6.gemspec"), + && detail.contains("served stub gemspec for rack"), "the refusal names the file and the missing attributes: {detail}" ); assert!(!root.join(".socket").exists()); @@ -5250,7 +4965,9 @@ mod tests { let empty = root.join(".socket/empty-blobs"); tokio::fs::create_dir_all(&empty).await.unwrap(); - let (r2, e2, _) = unwrap_done(run_vendor(&root, &empty, &installed, &record, false).await); + let (r2, e2, _) = crate::vendor::test_support::expect_failed( + run_vendor(&root, &empty, &installed, &record, false).await, + ); assert!(!r2.success, "rebuild must fail without patch content"); assert!(e2.is_none()); @@ -5335,7 +5052,7 @@ mod tests { mount_gem_granted(&server, &garbage, &sri, Some((SERVICE_STUB, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -5369,15 +5086,14 @@ mod tests { ); } - /// `auto` + a not-built service status falls back to the local build. #[tokio::test] - async fn service_unavailable_auto_falls_back_to_build() { + async fn service_unavailable_miss_refuses() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; let server = wiremock::MockServer::start().await; mount_gem_status(&server, "not_found").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -5386,15 +5102,19 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let (result, entry, _) = unwrap_done(outcome); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); + let error = crate::vendor::test_support::expect_failure(outcome); + assert!( + error.contains("prebuilt") || error.contains("patch service"), + "{error}" + ); } - /// A served stub that declares native extensions is refused (defense in /// depth — the converter should never emit one). #[tokio::test] @@ -5407,7 +5127,7 @@ mod tests { mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_NATIVE, &stub_sri))).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &missing_install(&root), &root, @@ -5432,7 +5152,7 @@ mod tests { async fn offline_service_mode_refuses() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &installed, &root, @@ -5741,7 +5461,7 @@ mod tests { cfg: &VendorServiceConfig, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_gem( + crate::vendor::test_support::vendor_gem( PURL, installed, root, @@ -5820,7 +5540,12 @@ mod tests { unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); assert!(result.success, "{:?}", result.error); assert!(entry.is_none(), "a no-op records no ledger entry"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert!(!root.join(".socket").exists(), "no writes at all"); assert_eq!( tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), @@ -5921,58 +5646,21 @@ mod tests { ); } - /// Wired pair + stale copy + no service and no local stub gemspec: the - /// artifact rebuild hard-fails `gem_spec_missing`, and the live pair - /// edit is left exactly as it was. - #[tokio::test] - async fn wired_missing_copy_rebuild_without_stub_refuses() { - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let (r1, _, _) = unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); - assert!(r1.success, "{:?}", r1.error); - let gemfile1 = tokio::fs::read(root.join(GEMFILE)).await.unwrap(); - let lock1 = tokio::fs::read(root.join(GEMFILE_LOCK)).await.unwrap(); - crate::patch::copy_tree::remove_tree(&root.join(copy_rel())) - .await - .unwrap(); - tokio::fs::remove_file( - installed - .parent() - .unwrap() - .parent() - .unwrap() - .join("specifications/rack-3.2.6.gemspec"), - ) - .await - .unwrap(); - - let (code, _d) = - unwrap_refused(run_vendor(&root, &blobs, &installed, &record, false).await); - assert_eq!(code, "gem_spec_missing"); - assert_eq!(tokio::fs::read(root.join(GEMFILE)).await.unwrap(), gemfile1); - assert_eq!( - tokio::fs::read(root.join(GEMFILE_LOCK)).await.unwrap(), - lock1 - ); - } - - /// Local-build failure: the installed gem dir is gone (spec stub still - /// present). The result is an un-successful Done with no ledger entry, - /// the uuid dir (and the empty `.socket/vendor` levels this run created) - /// removed — no committable husk — and neither project file touched. #[tokio::test] async fn fresh_copy_failure_cleans_up_and_touches_nothing() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; tokio::fs::remove_dir_all(&installed).await.unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(&root, &blobs, &installed, &record, false).await, + ); assert!(!result.success); assert!( result .error .as_deref() .unwrap_or("") - .contains("failed to copy installed gem"), + .contains("patch service request failed"), "{:?}", result.error ); @@ -5997,75 +5685,6 @@ mod tests { ); } - /// Local-build failure: the after-hash blob is missing, so the staged - /// apply fails. Same contract: un-successful Done, no entry, no husk, - /// pair untouched. - #[tokio::test] - async fn missing_blob_apply_failure_cleans_up_and_touches_nothing() { - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let after = compute_git_sha256_from_bytes(PATCHED); - tokio::fs::remove_file(blobs.join(&after)).await.unwrap(); - - let (result, entry, _w) = - unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); - assert!(!result.success, "apply must fail without the blob"); - assert!(result.error.is_some()); - assert!(entry.is_none()); - assert!( - !root.join(format!(".socket/vendor/gem/{UUID}")).exists(), - "no uuid-dir husk" - ); - assert_eq!( - tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), - GEMFILE_DIRECT - ); - assert_eq!( - tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) - .await - .unwrap(), - LOCK_DIRECT - ); - } - - /// Local-build failure: a DIRECTORY named `rack.gemspec` inside the - /// installed gem rides fresh_copy into the stage, so the stub-gemspec - /// write fails (EISDIR). Same cleanup contract. - #[tokio::test] - async fn stub_write_failure_cleans_up_and_touches_nothing() { - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - tokio::fs::create_dir_all(installed.join("rack.gemspec")) - .await - .unwrap(); - - let (result, entry, _w) = - unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); - assert!(!result.success); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .contains("failed to copy the stub gemspec"), - "{:?}", - result.error - ); - assert!(entry.is_none()); - assert!( - !root.join(format!(".socket/vendor/gem/{UUID}")).exists(), - "no uuid-dir husk" - ); - assert_eq!( - tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), - GEMFILE_DIRECT - ); - assert_eq!( - tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) - .await - .unwrap(), - LOCK_DIRECT - ); - } - /// A marker-write failure is informational only (state.json is the /// ledger of record): a DIRECTORY squatting the marker path survives /// materialise (which only rebuilds the copy dir) and makes the atomic @@ -6101,45 +5720,6 @@ mod tests { ); } - /// An uninventoriable copy vendors like a pre-inventory entry (fail-soft - /// contract): a file over the inventory hash cap (a sparse `set_len` - /// file — no real disk use) makes `compute_dir_inventory` refuse, so the - /// entry records `file_inventory: None` plus the - /// `vendor_inventory_unrecorded` warning, while the vendor itself - /// succeeds. - #[tokio::test] - async fn uninventoriable_copy_degrades_to_warning() { - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let big = std::fs::File::create(installed.join("lib/huge.bin")).unwrap(); - big.set_len(512 * 1024 * 1024 + 1).unwrap(); - drop(big); - - let (result, entry, warnings) = - unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); - assert!(result.success, "{:?}", result.error); - let entry = entry.expect("vendor still records the entry"); - assert!( - entry.artifact.file_inventory.is_none(), - "inventory must be absent, not partial" - ); - let warning = warnings - .iter() - .find(|w| w.code == "vendor_inventory_unrecorded") - .expect("the gap is surfaced"); - assert!( - warning.detail.contains("drift in its unpatched files"), - "{}", - warning.detail - ); - // The pair edit itself is unaffected. - assert_eq!( - tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) - .await - .unwrap(), - expected_lock_direct() - ); - } - /// A Gemfile atomic-write failure (read-only project root: the stage /// file cannot be created) unwinds the freshly-built uuid dir and /// reports an un-successful Done with both project files byte-untouched. @@ -6235,37 +5815,22 @@ mod tests { .await; } - /// A configured-but-disabled service (`--vendor-source=build`, or `auto` - /// while offline) silently uses the local build: no request is made (the - /// URI is a dead port) and no `vendor_prebuilt_*` advisory fires. #[tokio::test] async fn disabled_service_config_silently_builds_locally() { for cfg in [ - gem_service_cfg("http://127.0.0.1:1", VendorSource::Build, false), - gem_service_cfg("http://127.0.0.1:1", VendorSource::Auto, true), + gem_service_cfg("http://127.0.0.1:1", VendorSource::Service, false), + gem_service_cfg("http://127.0.0.1:1", VendorSource::Service, true), ] { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let (result, entry, warnings) = - unwrap_done(run_vendor_service(&root, &blobs, &installed, &record, &cfg).await); - assert!(result.success, "{:?}: {:?}", cfg.source, result.error); - assert!(entry.is_some()); - assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); - assert_eq!( - tokio::fs::read_to_string(copy_gemspec(&root)) - .await - .unwrap(), - GEMSPEC, - "the LOCAL stub is used" + let error = crate::vendor::test_support::expect_failure( + run_vendor_service(&root, &blobs, &installed, &record, &cfg).await, ); assert!( - !warnings - .iter() - .any(|w| w.code.starts_with("vendor_prebuilt")), - "silent local fallback, no service advisories: {warnings:?}" + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } } - /// `service` mode + a still-building archive (`pending_build`) refuses /// with the "still building" detail; nothing is written. #[tokio::test] @@ -6288,31 +5853,21 @@ mod tests { ); } - /// `auto` + `pending_build` warns under `vendor_prebuilt_pending` and - /// falls back to the local build. #[tokio::test] async fn service_pending_auto_warns_and_builds_locally() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; let server = wiremock::MockServer::start().await; mount_gem_status(&server, "pending_build").await; - let cfg = gem_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = gem_service_cfg(&server.uri(), VendorSource::Service, false); - let (result, entry, warnings) = - unwrap_done(run_vendor_service(&root, &blobs, &installed, &record, &cfg).await); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); - let warning = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_pending") - .expect("the pending miss is surfaced"); + let error = crate::vendor::test_support::expect_failure( + run_vendor_service(&root, &blobs, &installed, &record, &cfg).await, + ); assert!( - warning.detail.contains("building locally instead"), - "{}", - warning.detail + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// `service` mode + a terminal miss (`not_found`) hard-fails naming the /// unavailability (the `auto` fallback leg is covered above). #[tokio::test] @@ -6361,30 +5916,16 @@ mod tests { let stub_sri = sri_sha512(SERVICE_STUB); let server = wiremock::MockServer::start().await; mount_gem_granted_stub_get_fails(&server, &gem, &sri, &stub_sri).await; - let cfg = gem_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = gem_service_cfg(&server.uri(), VendorSource::Service, false); - let (result, entry, warnings) = - unwrap_done(run_vendor_service(&root, &blobs, &installed, &record, &cfg).await); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - assert_eq!( - tokio::fs::read_to_string(copy_gemspec(&root)) - .await - .unwrap(), - GEMSPEC, - "the LOCAL stub is used" + let error = crate::vendor::test_support::expect_failure( + run_vendor_service(&root, &blobs, &installed, &record, &cfg).await, ); - let warning = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_unavailable") - .expect("the fetch failure is surfaced"); assert!( - warning.detail.contains("could not fetch the stub gemspec"), - "{}", - warning.detail + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// `service` mode + a served `.gem` whose extracted layout misses the /// recorded file paths fails closed (`vendor_prebuilt_layout_mismatch` /// miss → `vendor_prebuilt_required` refusal); no husk is left. @@ -6421,29 +5962,16 @@ mod tests { let stub_sri = sri_sha512(SERVICE_STUB); let server = wiremock::MockServer::start().await; mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB, &stub_sri))).await; - let cfg = gem_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = gem_service_cfg(&server.uri(), VendorSource::Service, false); - let (result, entry, warnings) = - unwrap_done(run_vendor_service(&root, &blobs, &installed, &record, &cfg).await); - assert!(result.success, "auto must fall back: {:?}", result.error); - assert!(entry.is_some()); - assert_eq!( - tokio::fs::read(copy_lib(&root)).await.unwrap(), - PATCHED, - "the LOCAL build's patched file, at the recorded path" + let error = crate::vendor::test_support::expect_failure( + run_vendor_service(&root, &blobs, &installed, &record, &cfg).await, ); assert!( - !root.join(copy_rel()).join("wrong/rack.rb").exists(), - "the mismatched service layout never lands" - ); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_layout_mismatch"), - "{warnings:?}" + error.contains("prebuilt") || error.contains("service"), + "{error}" ); } - /// A served `.gem` whose data.tar.gz carries a DIRECTORY at the stub /// path (`rack.gemspec/…`) makes the stub write fail — a hard /// `vendor_prebuilt_write_failed`, no husk. @@ -6536,10 +6064,6 @@ mod tests { assert!(!root.join(".socket").exists()); } - /// Invalid served stub + `auto` + a CORRUPTED local stub: the local-build write choke - /// point refuses `gem_spec_invalid`, and the detail honestly notes the - /// service cannot supply a stub either (its served stub is defective) — - /// never circular "use --vendor-source=service" advice. #[tokio::test] async fn invalid_served_and_local_stub_refuses_with_honest_note() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; @@ -6559,13 +6083,13 @@ mod tests { let stub_sri = sri_sha512(SERVICE_STUB_INVALID); let server = wiremock::MockServer::start().await; mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; - let cfg = gem_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = gem_service_cfg(&server.uri(), VendorSource::Service, false); let (code, detail) = unwrap_refused(run_vendor_service(&root, &blobs, &installed, &record, &cfg).await); - assert_eq!(code, "gem_spec_invalid"); + assert_eq!(code, "vendor_prebuilt_stub_invalid"); assert!( - detail.contains("cannot supply one either"), + detail.contains("Retry after the patch service publishes a corrected artifact"), "the served-stub defect must ride the local refusal: {detail}" ); assert!(!root.join(".socket").exists()); @@ -7535,10 +7059,6 @@ mod tests { ); } - /// The LOCAL build's final swap failing (a stale regular FILE squatting - /// the backup sibling — a dir-tree remover cannot clear it): the run - /// reports the move failure, wires neither project file, and unwinds the - /// fresh uuid dir. #[tokio::test] async fn local_swap_failure_reports_move_error_and_touches_nothing() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; @@ -7548,13 +7068,14 @@ mod tests { .await .unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(&root, &blobs, &installed, &record, false).await, + ); assert!(!result.success, "the swap failure must fail the vendor"); assert!(entry.is_none(), "no ledger entry for a failed vendor"); let err = result.error.as_deref().unwrap_or(""); assert!( - err.contains("failed to move the rebuilt copy into place"), + err.contains("cannot move the extracted .gem into place"), "{err}" ); assert_eq!( @@ -7592,7 +7113,7 @@ mod tests { .await .unwrap(); - let outcome = vendor_gem( + let outcome = crate::vendor::test_support::vendor_gem( PURL, &missing_install(&root), &root, @@ -7698,7 +7219,7 @@ mod tests { ) -> (ApplyResult, Option, Vec) { let sources = PatchSources::blobs_only(blobs); unwrap_done( - vendor_gem( + crate::vendor::test_support::vendor_gem( PURL, installed, root, @@ -7707,7 +7228,11 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await, ) @@ -7722,31 +7247,6 @@ mod tests { server } - #[tokio::test] - async fn flip_local_then_service_is_noop() { - use crate::vendor::test_support as ts; - let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; - let down = wiremock::MockServer::start().await; - ts::mount_503(&down).await; - let (r1, e1, w1) = flip_run(&root, &installed, &blobs, &record, &down).await; - assert!(r1.success && e1.is_some()); - assert!(ts::has_warning(&w1, "vendor_prebuilt_unavailable")); - let before = ts::tree_snapshot(&root); - let up = flip_granted().await; - let (r2, e2, _) = flip_run(&root, &installed, &blobs, &record, &up).await; - assert!(r2.success, "{:?}", r2.error); - assert!( - e2.is_none(), - "re-run must be the in-sync no-op (entry None)" - ); - assert!(r2 - .files_verified - .iter() - .all(|f| f.status == VerifyStatus::AlreadyPatched)); - assert_eq!(before, ts::tree_snapshot(&root), "tree byte-identical"); - assert_eq!(ts::request_count(&up).await, 0); - } - #[tokio::test] async fn flip_service_then_local_is_noop() { use crate::vendor::test_support as ts; @@ -7765,7 +7265,10 @@ mod tests { let (r2, e2, w2) = flip_run(&root, &installed, &blobs, &record, &down).await; assert!(r2.success); assert!(e2.is_none()); - assert!(w2.is_empty(), "{w2:?}"); + assert!( + w2.iter().all(|w| w.code == "vendor_prebuilt_downloaded"), + "{w2:?}" + ); assert_eq!(before, ts::tree_snapshot(&root)); assert_eq!(ts::request_count(&down).await, 0); } @@ -7841,8 +7344,6 @@ mod tests { ); } - /// A `.gem` or stub that fails integrity verification is a hard - /// failure under `auto` too — never a quiet local-build fallback. #[tokio::test] async fn integrity_mismatch_hard_fails_under_auto() { let gem = make_gem(&[("lib/rack.rb", PATCHED)]); @@ -7855,7 +7356,7 @@ mod tests { (sri_sha512(b"different bytes"), sri_sha512(SERVICE_STUB)) }; mount_gem_granted(&server, &gem, &gem_sri, Some((SERVICE_STUB, &stub_sri))).await; - let cfg = gem_service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = gem_service_cfg(&server.uri(), VendorSource::Service, false); let outcome = run_vendor_service(&root, &blobs, &installed, &record, &cfg).await; let VendorOutcome::Refused { code, .. } = outcome else { panic!("bad_stub={bad_stub}: tampered bytes fell back: {outcome:?}"); diff --git a/crates/socket-patch-core/src/vendor/golang.rs b/crates/socket-patch-core/src/vendor/golang.rs index 1b14ff72b..6a17908e0 100644 --- a/crates/socket-patch-core/src/vendor/golang.rs +++ b/crates/socket-patch-core/src/vendor/golang.rs @@ -19,10 +19,10 @@ use std::path::{Path, PathBuf}; use crate::manifest::schema::PatchRecord; -use crate::patch::apply::{MismatchPolicy, PatchSources}; +use crate::patch::apply::PatchSources; use crate::patch::copy_tree::remove_tree; use crate::patch::redirect::golang_local::{ - apply_go_redirect, are_safe_redirect_coords, copy_dir_for, ensure_module_go_mod, + are_safe_redirect_coords, copy_dir_for, ensure_module_go_mod, }; use crate::utils::purl::{parse_golang_purl, strip_purl_qualifiers}; use crate::vendor::go_mod_edit::{ @@ -30,9 +30,8 @@ use crate::vendor::go_mod_edit::{ }; use super::common::{ - already_patched_result, copy_matches_after_hashes, done, failed_result, - prune_empty_vendor_levels, refused, service_offline_conflict, stage_dir_for, - swap_stage_into_place, + already_patched_result, copy_matches_after_hashes, done, prune_empty_vendor_levels, refused, + service_offline_conflict, stage_dir_for, swap_stage_into_place, }; use super::path::vendor_uuid_dir_rel; use super::registry_fetch::{extract_on_blocking_pool, extract_zip_with_prefix}; @@ -199,11 +198,10 @@ pub(crate) async fn service_preflight( /// Vendor one Go module: patched copy in the uuid dir + a vendor-owned /// `replace` directive + marker, returning the ledger entry to persist. /// -/// * `pristine_src` — the crawler's module-cache dir (case-encoded on disk). -/// It is copied, never mutated. +/// * `pristine_src` — retained for caller compatibility; acquisition uses the service. /// * `vendored_at` — caller-formatted RFC3339 timestamp for the marker. /// -/// `dry_run` writes nothing (read-only verify against `pristine_src`); +/// `dry_run` verifies the server artifact without writes; /// `entry` is then `None`. A user-authored `replace` for the same /// module+version surfaces as a failed result (the engine's `go.mod` editor /// refuses it), not a refusal — the verify report is still useful. @@ -213,13 +211,13 @@ pub async fn vendor_go_module<'a>( pristine_src: impl Into>, project_root: &Path, record: &PatchRecord, - sources: &PatchSources<'_>, + _sources: &PatchSources<'_>, vendored_at: &str, dry_run: bool, - force: bool, + _force: bool, service: Option<&VendorServiceConfig>, ) -> VendorOutcome { - let pristine_src = pristine_src.into(); + let _pristine_src = pristine_src.into(); let GoPrelude { module, version, @@ -239,15 +237,8 @@ pub async fn vendor_go_module<'a>( let mut warnings: Vec = Vec::new(); - // Hot path (mirrors cargo.rs / composer_lock.rs): already wired to this - // uuid with the committed copy intact → touch nothing and never consult - // `pristine_src` — a pruned/partial module-cache copy must not fail a - // healthy re-run. The engine's `redirect_in_sync` would answer the same, - // but only after the `!force` missing-target pre-check below consulted - // the pristine source; returning here keeps that pre-check scoped to - // runs that actually rebuild from it. Dry runs keep the engine's - // read-only verify as their preview. - if copy_was_ok && !dry_run { + // A healthy committed copy requires neither a local module cache nor the service. + if copy_was_ok || record.files.is_empty() { return done( already_patched_result(purl, ©_dir, &record.files), None, @@ -264,11 +255,8 @@ pub async fn vendor_go_module<'a>( } } - // Acquire the patched module: prefer the prebuilt module zip from the patch - // service (download → verify → extract → wire the `replace`, no pristine - // source needed); else let the engine copy the pristine source, patch it, - // and wire the `replace`. - let result = match go_service_redirect( + // Download, verify and extract the server module before wiring its `replace`. + let (result, file_inventory) = match go_service_redirect( service, record, module, @@ -283,81 +271,19 @@ pub async fn vendor_go_module<'a>( ) .await { - GoServiceRedirect::Used(()) => { - // No local apply to verify (the downloaded zip IS the patched - // module), so every patched file reads as `AlreadyPatched` — trust - // is the verified service integrity (sha512 + the `h1:` dirhash). - already_patched_result(purl, ©_dir, &record.files) - } + GoServiceRedirect::Used(inventory) => ( + already_patched_result(purl, ©_dir, &record.files), + inventory, + ), GoServiceRedirect::HardFail(outcome) => return *outcome, - GoServiceRedirect::FallBack => { - // Vendor auto-force policy (the engine's copy is staged from the - // pristine source, never the user's tree — see `force_apply_staged`): - // missing patch targets still fail closed unless the caller's own - // `--force` asked for the skip tolerance, then the engine apply runs - // forced so a beforeHash mismatch (already-applied module, or a - // patch built against different bytes) overwrites with the verified - // patched content. The engine is shared with the in-place `apply` - // redirect path, whose strict semantics stay unchanged. - if !force { - // The pre-check reads the pristine tree, so a lazily-fetched - // source materialises here; the engine's own copy below then - // comes from that tree rather than a second inflate. - let probe = match pristine_src.materialize().await { - Ok(dir) => dir, - Err(e) => { - return done( - failed_result( - purl, - Path::new(""), - format!("failed to copy pristine source: {e}"), - ), - None, - warnings, - ) - } - }; - let missing = super::missing_existing_patch_files(probe, &record.files).await; - if let Some(first) = missing.first() { - return done( - failed_result( - purl, - Path::new(""), - format!("Cannot apply patch: {first} - File not found"), - ), - None, - warnings, - ); - } - } - // The engine does the heavy lifting: fresh copy → hardened apply - // pipeline → `replace` upsert (refuses a user-authored same-version - // pin). The copy is a content-verified artifact, so its patched - // files are written without an fsync; the `go.mod` edit stays a - // durable commit point (see `crate::utils::durability`). - let result = crate::utils::durability::artifact_writes(apply_go_redirect( - purl, - module, - version, - pristine_src, - project_root, - &base_rel, - &record.files, - sources, - Some(&record.uuid), - dry_run, - MismatchPolicy::Force, - )) - .await; - if result.success { - warnings.extend(super::mismatch_overwrite_warnings(&result, module, version)); - } - result - } }; if dry_run { - return done(result, None, warnings); + return done( + super::common::preview_result(purl, ©_dir, &record.files), + None, + warnings, + ); } if !result.success { // The engine already rolled back a half-built copy, but its rollback @@ -460,11 +386,12 @@ pub async fn vendor_go_module<'a>( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!("{base_rel}/{module}@{version}"), sha256: String::new(), // dir-shaped: integrity is per-file afterHashes size: None, platform_locked: None, - file_inventory: None, + file_inventory, }, wiring: vec![WiringRecord { file: "go.mod".to_string(), @@ -500,7 +427,7 @@ pub async fn vendor_go_module<'a>( /// Outcome of attempting to materialise the go copy from the patch service /// (`Used`: the prebuilt module zip was extracted and the `replace` wired). -type GoServiceRedirect = ServiceAttempt<()>; +type GoServiceRedirect = ServiceAttempt>>; /// Download the prebuilt module zip, verify it (sha512 + the `h1:` dirhash, /// done by `fetch_verified_archive`), extract it into `copy_dir` (stripping its @@ -526,23 +453,32 @@ async fn go_service_redirect( wired: bool, warnings: &mut Vec, ) -> GoServiceRedirect { - let Some(cfg) = service else { - return GoServiceRedirect::FallBack; - }; - // Dry runs never reach the service: every leg below writes for real - // (copy-dir replace, go.mod upsert) — the engine's read-only verify is - // the preview (the same gate the npm/pypi backends apply). And an intact - // wired copy is already byte-identical to the verified service end state: - // never tear it down for a re-download whose failure would strand go.mod - // pointing at a deleted dir. - if dry_run || copy_was_ok { - return GoServiceRedirect::FallBack; - } - // An empty-files patch is a degenerate no-op; let the engine's empty - // handling deal with it rather than downloading anything. - if !cfg.service_enabled() || record.files.is_empty() { - return GoServiceRedirect::FallBack; + if record.files.is_empty() || copy_was_ok { + if let Err(e) = + go_mod_edit::ensure_replace_entry(project_root, module, version, base_rel, dry_run) + .await + { + return GoServiceRedirect::HardFail(Box::new(refused( + "vendor_prebuilt_wire_failed", + e.to_string(), + ))); + } + return GoServiceRedirect::Used(None); + } + if dry_run { + let prefix = format!("{module}@{version}/"); + return match super::service_fetch::preview_service(service, record, move |bytes, dest| { + extract_zip_with_prefix(bytes, dest, &prefix) + }) + .await + { + Ok(()) => GoServiceRedirect::Used(None), + Err(outcome) => GoServiceRedirect::HardFail(outcome), + }; } + let Some(cfg) = service.filter(|cfg| cfg.service_enabled()) else { + return GoServiceRedirect::HardFail(Box::new(super::service_fetch::required())); + }; let policy = ServicePolicy::new(cfg, ServiceTerminal::Refused); let fetched = fetch_verified_archive(cfg, &record.uuid).await; let subject = format!("module zip for {module}"); @@ -594,14 +530,6 @@ async fn go_service_redirect( format!("cannot synthesize go.mod for the copy: {e}"), ); } - // Verify the EXTRACTED TREE before it replaces the copy or the - // consumer's go.mod is wired: the SRI proves the zip bytes are - // intact, but an unexpected internal layout (the - // `{module}@{version}/` prefix strip mismatching) lands the - // patched files at the wrong paths, and the caller would - // synthesize success from `record.files` while the copy is - // wrong. Fail closed → `auto` falls back to the local build; - // nothing points at the bad stage. (Mirrors composer_lock.rs.) if !copy_matches_after_hashes(&stage, &record.files).await { cleanup_failed_service_stage(&stage, project_root, base_rel, copy_dir, module, wired).await; return policy.miss( @@ -614,6 +542,14 @@ async fn go_service_redirect( ), ); } + let file_inventory = match super::verify::compute_dir_inventory(&stage).await { + Ok(inventory) => inventory, + Err(error) => { + cleanup_failed_service_stage(&stage, project_root, base_rel, copy_dir, module, wired) + .await; + return policy.hard("vendor_inventory_unavailable", error); + } + }; if let Err(e) = swap_stage_into_place(&stage, copy_dir).await { cleanup_failed_service_stage(&stage, project_root, base_rel, copy_dir, module, wired).await; return policy.hard( @@ -644,7 +580,7 @@ async fn go_service_redirect( archive.source_url ), )); - GoServiceRedirect::Used(()) + GoServiceRedirect::Used(Some(file_inventory)) } /// Failure cleanup for the service legs (the vendor-side sibling of the @@ -796,6 +732,8 @@ mod tests { use crate::hash::git_sha256::compute_git_sha256_from_bytes; use crate::manifest::schema::{PatchFileInfo, VulnerabilityInfo}; use crate::patch::apply::ApplyResult; + use crate::patch::apply::MismatchPolicy; + use crate::patch::redirect::golang_local::apply_go_redirect; use crate::vendor::state::VENDOR_MARKER_FILE; use std::collections::HashMap; use std::path::PathBuf; @@ -894,7 +832,7 @@ mod tests { let root = dir.path(); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let sources = PatchSources::blobs_only(&blobs); let cases = [ (PURL, record.clone()), @@ -915,7 +853,7 @@ mod tests { let vendor = |purl: String, rec: PatchRecord| -> Borrowed<'_, VendorOutcome> { let (pristine, sources, cfg) = (&pristine, &sources, &cfg); Box::pin(async move { - vendor_go_module( + crate::vendor::test_support::vendor_go_module( &purl, pristine.as_path(), root, @@ -942,7 +880,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_go_module( + crate::vendor::test_support::vendor_go_module( purl, pristine, root, @@ -995,7 +933,12 @@ mod tests { let (result, entry, warnings) = expect_done(run_vendor(&qualified, root, &blobs, &pristine, &record, false).await); assert!(result.success, "vendor failed: {:?}", result.error); - assert!(warnings.is_empty(), "unexpected warnings: {warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "unexpected warnings: {warnings:?}" + ); // Copy holds the patched bytes inside the uuid dir. let copy = root.join(copy_rel()); @@ -1216,7 +1159,12 @@ mod tests { "an in-sync re-run records no entry — the first run's ledger \ entry holds the only pre-vendor original" ); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert_eq!( tokio::fs::read(©).await.unwrap(), copy1, @@ -1258,7 +1206,12 @@ mod tests { result.error ); assert!(entry.is_none(), "no re-recorded entry"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert_eq!( tokio::fs::read(©).await.unwrap(), copy1, @@ -1374,8 +1327,9 @@ mod tests { .await .unwrap(); - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); + let (result, entry, _warnings) = crate::vendor::test_support::expect_failed( + run_vendor(PURL, root, &blobs, &pristine, &record, false).await, + ); assert!(!result.success); assert!(entry.is_none()); // go.mod untouched and the failed copy fully unwound (no uuid husks). @@ -1562,13 +1516,6 @@ mod tests { .any(|e| e.module == MODULE && e.owner == Some(ReplaceOwner::Vendor))); } - /// Cross-mode policy regression (docs/ecosystems.md#go-directory-replaces-and-gosum): vendor - /// takes over a hosted-mode replace through the LOCAL build leg too — only - /// local *apply* refuses a Hosted-owned directive (its go-patches copy is - /// uncommitted, so the takeover would break other machines). The hosted - /// directive is rewritten in place to the vendor path, the takeover is - /// surfaced, and the wiring `original` records the hosted module-target - /// text so `--revert` can name the go.sum recovery. #[tokio::test] async fn test_local_vendor_takes_over_hosted_replace() { let (dir, blobs, pristine, record) = fixture().await; @@ -1649,7 +1596,9 @@ mod tests { expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); assert!(result.success); assert!(entry.is_none(), "nothing vendored, nothing recorded"); - assert!(warnings.is_empty()); + assert!(warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded")); assert!( read_replace_entries(root).await.is_empty(), "no replace written" @@ -1775,7 +1724,7 @@ mod tests { let sources = PatchSources::blobs_only(&blobs); let bogus_pristine = root.join("no-such-cache"); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &bogus_pristine, root, @@ -1828,7 +1777,7 @@ mod tests { .await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -1844,16 +1793,15 @@ mod tests { assert!(!root.join(format!(".socket/vendor/golang/{UUID}")).exists()); } - /// `auto` + a not-built service status falls back to the local build. #[tokio::test] - async fn service_unavailable_auto_falls_back_to_build() { + async fn service_unavailable_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let server = wiremock::MockServer::start().await; mount_go_status(&server, "not_found").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -1862,29 +1810,20 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; - let (result, entry, _) = expect_done(outcome); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - result.success, - "auto must fall back to the local build: {:?}", - result.error - ); - assert!(entry.is_some()); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("bar.go")) - .await - .unwrap(), - PATCHED + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// Dry-run must write nothing and stay off the network even when the /// service path is enabled (auto/service + client): the prebuilt download /// would delete/recreate the copy dir and rewrite go.mod for real. #[tokio::test] - async fn dry_run_with_service_enabled_writes_nothing_and_stays_offline() { + async fn dry_run_verifies_service_artifact_without_project_writes() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let zip = make_module_zip(&[ @@ -1899,7 +1838,7 @@ mod tests { .unwrap(); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -1908,7 +1847,7 @@ mod tests { "2026-06-09T00:00:00Z", /*dry_run=*/ true, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; let (result, entry, _warnings) = expect_done(outcome); @@ -1926,8 +1865,13 @@ mod tests { "no copy dir created" ); assert!( - server.received_requests().await.unwrap().is_empty(), - "dry-run must not contact the vendor service" + server + .received_requests() + .await + .unwrap() + .iter() + .any(|r| r.method == wiremock::http::Method::GET), + "dry-run verifies the server artifact without writing into the project" ); } @@ -1940,7 +1884,6 @@ mod tests { async fn service_rerun_with_intact_copy_never_degrades_wired_state() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); - // First run: local build wires copy + replace (no service). expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); let copy = root.join(copy_rel()).join("bar.go"); let gomod = root.join("go.mod"); @@ -1952,7 +1895,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_go_granted(&server, &sri_sha512(junk), None, junk).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -1961,7 +1904,7 @@ mod tests { "2026-06-10T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; @@ -1984,42 +1927,6 @@ mod tests { ); } - /// A failed rebuild of a wired-but-stale copy must not leave the vendor - /// `replace` directive pointing at the removed uuid dir (go: "replacement - /// directory does not exist" — build bricked). The failure must fall back - /// to the unpatched-module end state: dir gone AND directive gone. - #[tokio::test] - async fn failed_stale_copy_rebuild_drops_dangling_directive() { - let (dir, blobs, pristine, record) = fixture().await; - let root = dir.path(); - expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); - - // The committed copy drifts AND the patched blob is gone: the - // artifact rebuild has no source for afterHash content and fails. - tokio::fs::write(root.join(copy_rel()).join("bar.go"), b"drifted\n") - .await - .unwrap(); - tokio::fs::remove_file(blobs.join(git_sha(PATCHED))) - .await - .unwrap(); - - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); - assert!(!result.success, "rebuild without blobs must fail"); - assert!(entry.is_none()); - assert!( - !root.join(format!(".socket/vendor/golang/{UUID}")).exists(), - "uuid dir cleared" - ); - assert!( - read_replace_entries(root) - .await - .iter() - .all(|e| e.module != MODULE), - "no dangling replace directive at the deleted copy" - ); - } - /// The service legs stage the download and swap it in only once verified /// (the cargo / composer / gem shape): when the service rebuild of a /// wired-but-STALE copy fails mid-materialisation (corrupt zip), the copy @@ -2042,7 +1949,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_go_granted(&server, &sri_sha512(junk), None, junk).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2051,7 +1958,7 @@ mod tests { "2026-06-10T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; expect_refused(outcome, "vendor_prebuilt_extract_failed"); @@ -2084,7 +1991,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_go_granted(&server, &sri_sha512(junk), None, junk).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2093,7 +2000,7 @@ mod tests { "2026-06-10T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; expect_refused(outcome, "vendor_prebuilt_extract_failed"); @@ -2116,7 +2023,7 @@ mod tests { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2148,7 +2055,7 @@ mod tests { assert!(entry.is_some()); let gomod = tokio::fs::read(root.join("go.mod")).await.unwrap(); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2167,7 +2074,12 @@ mod tests { let (result, entry, warnings) = expect_done(outcome); assert!(result.success, "{:?}", result.error); assert!(entry.is_none(), "in sync: nothing recorded"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert_eq!(tokio::fs::read(root.join("go.mod")).await.unwrap(), gomod); } @@ -2183,7 +2095,7 @@ mod tests { assert!(result.success, "{:?}", result.error); let gomod = tokio::fs::read(root.join("go.mod")).await.unwrap(); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2205,92 +2117,6 @@ mod tests { assert_eq!(tokio::fs::read(root.join("go.mod")).await.unwrap(), gomod); } - // ── missing-patch-target pre-check (fail-closed vs `--force`) ───────── - - /// A patch-target file absent from the pristine module cache fails closed - /// on a first (non-`--force`) run: the vendor pre-check reports the - /// missing file BEFORE the engine's force-apply could silently skip it, - /// and nothing is written. - #[tokio::test] - async fn test_missing_patch_target_fails_closed_without_force() { - let (dir, blobs, pristine, record) = fixture().await; - let root = dir.path(); - let gomod_before = tokio::fs::read_to_string(root.join("go.mod")) - .await - .unwrap(); - // The module cache lost the beforeHash target. - tokio::fs::remove_file(pristine.join("bar.go")) - .await - .unwrap(); - - let (result, entry, _warnings) = - expect_done(run_vendor(PURL, root, &blobs, &pristine, &record, false).await); - assert!(!result.success, "missing target must fail closed"); - assert_eq!( - result.error.as_deref(), - Some("Cannot apply patch: package/bar.go - File not found") - ); - assert!(entry.is_none()); - // Nothing was written: no uuid dir husk, no replace, go.mod untouched. - assert!(!root.join(format!(".socket/vendor/golang/{UUID}")).exists()); - assert!(read_replace_entries(root).await.is_empty()); - assert_eq!( - tokio::fs::read_to_string(root.join("go.mod")) - .await - .unwrap(), - gomod_before - ); - } - - /// `--force` bypasses the fail-closed pre-check: the engine owns the - /// outcome, and its force policy SKIPS the missing file (its own skip - /// message, not the pre-check's "File not found") while still wiring the - /// vendor `replace` and recording the ledger entry. - #[tokio::test] - async fn test_force_bypasses_missing_target_precheck() { - let (dir, blobs, pristine, record) = fixture().await; - let root = dir.path(); - tokio::fs::remove_file(pristine.join("bar.go")) - .await - .unwrap(); - - let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( - PURL, - &pristine, - root, - &record, - &sources, - "2026-06-09T00:00:00Z", - false, - /*force=*/ true, - None, - ) - .await; - let (result, entry, _warnings) = expect_done(outcome); - assert!( - result.success, - "force skips the missing target: {:?}", - result.error - ); - let err = result.error.expect("force skip is surfaced in the result"); - assert!( - err.contains("not found on disk (--force)"), - "the engine's skip message, not the pre-check text: {err}" - ); - assert!(entry.is_some(), "a forced vendor still records the entry"); - let entries = read_replace_entries(root).await; - let e = entries - .iter() - .find(|e| e.module == MODULE) - .expect("replace wired despite the skip"); - assert_eq!(e.owner, Some(ReplaceOwner::Vendor)); - assert_eq!( - e.path.as_deref(), - Some(format!("./{}", copy_rel()).as_str()) - ); - } - // ── takeover husk-prune: multi-module go-patches layouts ────────────── /// The takeover prune walks empty parent husks upward but must stop at @@ -2353,17 +2179,15 @@ mod tests { // ── service status legs: pending / unavailable / request-failed ─────── - /// `auto` + a still-building prebuilt zip falls back to the local build - /// with a `vendor_prebuilt_pending` advisory naming the degradation. #[tokio::test] - async fn service_pending_auto_falls_back_to_build() { + async fn service_pending_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let server = wiremock::MockServer::start().await; mount_go_status(&server, "pending_build").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2372,31 +2196,15 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; - let (result, entry, warnings) = expect_done(outcome); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_pending") - .unwrap_or_else(|| panic!("pending advisory: {warnings:?}")); - assert!(w.detail.contains("still building"), "{}", w.detail); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - w.detail.contains("building locally instead"), - "{}", - w.detail - ); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("bar.go")) - .await - .unwrap(), - PATCHED, - "the local build produced the patched copy" + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// `service` mode + a still-building prebuilt zip hard-fails (no /// fallback), writing nothing. #[tokio::test] @@ -2408,7 +2216,7 @@ mod tests { mount_go_status(&server, "pending_build").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2442,7 +2250,7 @@ mod tests { mount_go_status(&server, "not_found").await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2470,7 +2278,7 @@ mod tests { /// A failed service REQUEST (HTTP 500 on the grant endpoint) under `auto` /// warns `vendor_prebuilt_unavailable` and builds locally. #[tokio::test] - async fn service_request_failure_auto_warns_and_builds_locally() { + async fn service_request_failure_miss_refuses() { use wiremock::matchers::{method, path}; use wiremock::{Mock, ResponseTemplate}; let (dir, blobs, pristine, record) = fixture().await; @@ -2483,7 +2291,7 @@ mod tests { .await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2492,29 +2300,15 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; - let (result, entry, warnings) = expect_done(outcome); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_unavailable") - .unwrap_or_else(|| panic!("fallback reason recorded: {warnings:?}")); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - w.detail.contains("patch service request failed"), - "{}", - w.detail - ); - assert_eq!( - tokio::fs::read(root.join(copy_rel()).join("bar.go")) - .await - .unwrap(), - PATCHED + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// The same failed request under `service` mode hard-fails, writing /// nothing. #[tokio::test] @@ -2532,7 +2326,7 @@ mod tests { .await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2561,7 +2355,7 @@ mod tests { /// the extracted-tree verify fails closed and `auto` rebuilds locally — /// the bad extract is torn down, never left behind. #[tokio::test] - async fn service_layout_mismatch_auto_falls_back_to_build() { + async fn service_layout_mismatch_miss_refuses() { let (dir, blobs, pristine, record) = fixture().await; let root = dir.path(); let zip = make_module_zip(&[ @@ -2573,7 +2367,7 @@ mod tests { mount_go_granted(&server, &sri, None, &zip).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2582,33 +2376,15 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; - let (result, entry, warnings) = expect_done(outcome); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_layout_mismatch"), - "{warnings:?}" - ); - let copy = root.join(copy_rel()); - assert_eq!( - tokio::fs::read(copy.join("bar.go")).await.unwrap(), - PATCHED, - "the local rebuild produced the patched copy" - ); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - !copy.join("wrong.go").exists(), - "the bad extract was torn down before the local rebuild" + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); - let entries = read_replace_entries(root).await; - let e = entries.iter().find(|e| e.module == MODULE).unwrap(); - assert_eq!(e.owner, Some(ReplaceOwner::Vendor)); } - /// The same layout mismatch under `service` mode refuses (no local /// fallback allowed): the extracted uuid dir is torn down and go.mod was /// never edited (the verify runs BEFORE the wire). @@ -2626,7 +2402,7 @@ mod tests { mount_go_granted(&server, &sri, None, &zip).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2678,7 +2454,7 @@ mod tests { mount_go_granted(&server, &sri, None, &zip).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2718,7 +2494,7 @@ mod tests { mount_go_granted(&server, &sri_sha512(junk), None, junk).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2817,7 +2593,7 @@ mod tests { ) -> (ApplyResult, Option, Vec) { let sources = PatchSources::blobs_only(blobs); expect_done( - vendor_go_module( + crate::vendor::test_support::vendor_go_module( PURL, pristine, root, @@ -2826,7 +2602,7 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(uri, VendorSource::Auto, false)), + Some(&go_service_cfg(uri, VendorSource::Service, false)), ) .await, ) @@ -2852,26 +2628,6 @@ mod tests { .unwrap(); } - #[tokio::test] - async fn flip_local_then_service_is_noop() { - use crate::vendor::test_support as ts; - let (dir, blobs, pristine, record) = fixture().await; - let root = dir.path(); - flip_go_sum(root).await; - let down = wiremock::MockServer::start().await; - ts::mount_503(&down).await; - let (r1, e1, _) = flip_run(root, &blobs, &pristine, &record, &down.uri()).await; - assert!(r1.success && e1.is_some()); - let before = ts::tree_snapshot(root); - let up = wiremock::MockServer::start().await; - let z = flip_service_zip(); - mount_go_granted(&up, &sri_sha512(&z), None, &z).await; - let (r2, e2, w2) = flip_run(root, &blobs, &pristine, &record, &up.uri()).await; - assert!(r2.success && e2.is_none() && w2.is_empty()); - assert_eq!(ts::tree_snapshot(root), before); - assert_eq!(ts::request_count(&up).await, 0); - } - #[tokio::test] async fn flip_service_then_local_is_noop() { use crate::vendor::test_support as ts; @@ -2893,8 +2649,6 @@ mod tests { assert_eq!(ts::request_count(&down).await, 0); } - /// An integrity mismatch is a hard failure under `auto` too — - /// never a quiet local-build fallback (service_fetch's contract). #[tokio::test] async fn service_integrity_mismatch_auto_hard_fails() { let (dir, blobs, pristine, record) = fixture().await; @@ -2908,7 +2662,7 @@ mod tests { let server = wiremock::MockServer::start().await; mount_go_granted(&server, &wrong, None, &zip).await; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, @@ -2917,7 +2671,7 @@ mod tests { "2026-06-09T00:00:00Z", false, false, - Some(&go_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&go_service_cfg(&server.uri(), VendorSource::Service, false)), ) .await; expect_refused(outcome, "vendor_prebuilt_integrity_mismatch"); @@ -2938,7 +2692,7 @@ mod tests { let mut cfg = go_service_cfg("http://127.0.0.1:1", VendorSource::Service, false); cfg.client = None; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_go_module( + let outcome = crate::vendor::test_support::vendor_go_module( PURL, &pristine, root, diff --git a/crates/socket-patch-core/src/vendor/jvm/apply.rs b/crates/socket-patch-core/src/vendor/jvm/apply.rs index 9ad6fc780..9788b62fe 100644 --- a/crates/socket-patch-core/src/vendor/jvm/apply.rs +++ b/crates/socket-patch-core/src/vendor/jvm/apply.rs @@ -809,7 +809,11 @@ pub fn check_entry( /// The vendored jar of the JVM `entry` for `uuid`, project-relative: the /// artifact path must be the entry's own tree jar (the Maven directory /// carries the uuid; the Gradle one's marker must name it). -pub fn checked_tree_jar(root: &Path, entry: &VendorEntry, uuid: &str) -> Result { +pub(crate) fn checked_tree_jar_path( + root: &Path, + entry: &VendorEntry, + uuid: &str, +) -> Result { let unsafe_path = || "vendor_path_unsafe".to_string(); if !is_jvm_entry(entry) { return Err(unsafe_path()); @@ -840,16 +844,29 @@ pub fn checked_tree_jar(root: &Path, entry: &VendorEntry, uuid: &str) -> Result< if rel != gradle_jar { return Err(unsafe_path()); } - let marker = ProjectReader::new(root) - .read(&format!("{}/{}", gradle::tree_dir(&c), gradle::MARKER_NAME)) - .and_then(|m| serde_json::from_slice::(&m).ok()); - match marker - .as_ref() - .and_then(|m| m.get("uuid")) - .and_then(Value::as_str) - { - Some(u) if u == uuid => Ok(gradle_jar), - _ => Err("vendor_uuid_mismatch".to_string()), + Ok(gradle_jar) +} + +pub fn checked_tree_jar(root: &Path, entry: &VendorEntry, uuid: &str) -> Result { + let rel = checked_tree_jar_path(root, entry, uuid)?; + if rel.starts_with(".socket/vendor/maven2/") { + return Ok(rel); + } + let marker_path = format!( + "{}/{}", + rel.rsplit_once('/').ok_or("vendor_path_unsafe")?.0, + gradle::MARKER_NAME + ); + let reader = ProjectReader::new(root); + let bytes = reader.read(&marker_path).ok_or("vendor_artifact_missing")?; + if reader.escaped().is_some() { + return Err("vendor_path_unsafe".into()); + } + let marker: Value = serde_json::from_slice(&bytes).map_err(|_| "vendor_artifact_unreadable")?; + match marker.get("uuid").and_then(Value::as_str) { + Some(u) if u == uuid => Ok(rel), + Some(_) => Err("vendor_uuid_mismatch".into()), + None => Err("vendor_artifact_unreadable".into()), } } diff --git a/crates/socket-patch-core/src/vendor/jvm/archive.rs b/crates/socket-patch-core/src/vendor/jvm/archive.rs deleted file mode 100644 index c8df0f6af..000000000 --- a/crates/socket-patch-core/src/vendor/jvm/archive.rs +++ /dev/null @@ -1,140 +0,0 @@ -//! Server-compatible JAR encoding (archiver 7's stored ZIP format). -//! Keep upstream entry order and executable bits; append new entries in name -//! order, drop signature metadata, and write fixed timestamps without extras. - -use std::collections::{BTreeMap, BTreeSet}; -use std::io::Cursor; - -pub(crate) fn is_signature(name: &str) -> bool { - let upper = name.to_ascii_uppercase(); - upper == ".SIGNATURE.P7S" - || upper.strip_prefix("META-INF/").is_some_and(|n| { - !n.contains('/') - && [".SF", ".RSA", ".DSA", ".EC"] - .iter() - .any(|ext| n.ends_with(ext)) - }) -} - -/// Canonicalize a locally patched JAR using the upstream archive's ordering. -/// Both archives are decoded with the vendor verifier's size/entry limits. -pub(crate) fn canonical_jar(upstream: &[u8], patched: &[u8]) -> Result, String> { - super::super::verify::read_zip_bytes_to_map(upstream)?; - let mut bodies = super::super::verify::read_zip_bytes_to_map(patched)?; - let mut original = zip::ZipArchive::new(Cursor::new(upstream)).map_err(|e| e.to_string())?; - let mut order = Vec::new(); - let mut seen = BTreeSet::new(); - for i in 0..original.len() { - let file = original.by_index(i).map_err(|e| e.to_string())?; - if file.is_dir() { - continue; - } - let name = file.name().to_string(); - if !seen.insert(name.clone()) { - return Err(format!("duplicate upstream JAR entry: {name}")); - } - let mode = if file.unix_mode().is_some_and(|m| m & 0o111 != 0) { - 0o100755 - } else { - 0o100644 - }; - order.push((name, mode)); - } - let mut additions: Vec<_> = bodies - .keys() - .filter(|name| !seen.contains(*name)) - .cloned() - .collect(); - additions.sort(); - order.extend(additions.into_iter().map(|name| (name, 0o100644))); - let mut patched_zip = zip::ZipArchive::new(Cursor::new(patched)).map_err(|e| e.to_string())?; - let mut crcs = BTreeMap::new(); - for i in 0..patched_zip.len() { - let file = patched_zip.by_index(i).map_err(|e| e.to_string())?; - crcs.insert(file.name().to_string(), file.crc32()); - } - let mut out = Vec::new(); - let mut directory = Vec::new(); - let mut count = 0u16; - for (name, mode) in order { - let Some(bytes) = bodies.remove(&name) else { - continue; - }; - if is_signature(&name) { - continue; - } - let offset = u32::try_from(out.len()).map_err(|_| "JAR exceeds ZIP32 limits")?; - let size = u32::try_from(bytes.len()).map_err(|_| "JAR member exceeds ZIP32 limits")?; - let len = u16::try_from(name.len()).map_err(|_| "JAR member name too long")?; - let flags = if name.is_ascii() { 0 } else { 0x800 }; - let crc = crcs[&name]; - u32s(&mut out, &[0x04034b50]); - u16s(&mut out, &[10, flags, 0, 0, 33]); - u32s(&mut out, &[crc, size, size]); - u16s(&mut out, &[len, 0]); - out.extend_from_slice(name.as_bytes()); - out.extend_from_slice(&bytes); - u32s(&mut directory, &[0x02014b50]); - u16s(&mut directory, &[0x032d, 10, flags, 0, 0, 33]); - u32s(&mut directory, &[crc, size, size]); - u16s(&mut directory, &[len, 0, 0, 0, 0]); - u32s(&mut directory, &[(mode << 16) | 0x20, offset]); - directory.extend_from_slice(name.as_bytes()); - count = count.checked_add(1).ok_or("too many JAR members")?; - } - let offset = u32::try_from(out.len()).map_err(|_| "JAR exceeds ZIP32 limits")?; - let size = u32::try_from(directory.len()).map_err(|_| "JAR directory exceeds ZIP32 limits")?; - out.extend_from_slice(&directory); - u32s(&mut out, &[0x06054b50]); - u16s(&mut out, &[0, 0, count, count]); - u32s(&mut out, &[size, offset]); - u16s(&mut out, &[0]); - Ok(out) -} - -fn u16s(out: &mut Vec, values: &[u16]) { - for v in values { - out.extend_from_slice(&v.to_le_bytes()); - } -} -fn u32s(out: &mut Vec, values: &[u32]) { - for v in values { - out.extend_from_slice(&v.to_le_bytes()); - } -} - -#[cfg(test)] -mod tests { - use super::*; - use std::io::Write; - - #[test] - fn matches_archiver_7_byte_for_byte() { - // Fixture generated by fixtures/repack/generate.cjs. - let expected = include_bytes!("fixtures/repack/archiver-7.0.1.jar"); - let mut upstream = zip::ZipWriter::new(Cursor::new(Vec::new())); - for (name, bytes, mode) in [ - ("z.txt", "first\n", 0o644), - ("META-INF/NOTICE.txt", "original\n", 0o644), - ("bin/run", "exec\n", 0o755), - ("café.txt", "unicode\n", 0o644), - ("META-INF/SIGN.RSA", "signature", 0o644), - ] { - upstream - .start_file( - name, - zip::write::SimpleFileOptions::default().unix_permissions(mode), - ) - .unwrap(); - upstream.write_all(bytes.as_bytes()).unwrap(); - } - let upstream = upstream.finish().unwrap().into_inner(); - let mut bodies = super::super::super::verify::read_zip_bytes_to_map(&upstream).unwrap(); - bodies.insert("META-INF/NOTICE.txt".into(), b"patched\n".to_vec()); - let mut entries: Vec<_> = bodies.into_iter().map(|(n, b)| (n, b, 0o644)).collect(); - entries.sort_by(|a, b| a.0.cmp(&b.0)); - let rebuilt = super::super::super::common::write_zip_entries(&entries).unwrap(); - assert_eq!(canonical_jar(&upstream, &rebuilt).unwrap(), expected); - assert_eq!(canonical_jar(expected, expected).unwrap(), expected); - } -} diff --git a/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/archiver-7.0.1.jar b/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/archiver-7.0.1.jar deleted file mode 100644 index 4fabed191..000000000 Binary files a/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/archiver-7.0.1.jar and /dev/null differ diff --git a/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/generate.cjs b/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/generate.cjs deleted file mode 100644 index ddc9a0553..000000000 --- a/crates/socket-patch-core/src/vendor/jvm/fixtures/repack/generate.cjs +++ /dev/null @@ -1,17 +0,0 @@ -// Regenerate with archiver@7.0.1, the patch server's repackDirToZip encoder. -// npm install --prefix /tmp/jvm-repack --ignore-scripts archiver@7.0.1 -// NODE_PATH=/tmp/jvm-repack/node_modules node generate.cjs -const fs = require('node:fs'); -const path = require('node:path'); -const archiver = require('archiver'); -(async () => { - const zip = archiver('zip', { zlib: { level: 0 }, store: true }); - const chunks = []; - zip.on('data', chunk => chunks.push(chunk)); - zip.on('error', error => { throw error; }); - zip.on('end', () => fs.writeFileSync(path.join(__dirname, 'archiver-7.0.1.jar'), Buffer.concat(chunks))); - for (const [name, body, mode] of [['z.txt', 'first\n', 0o644], ['META-INF/NOTICE.txt', 'patched\n', 0o644], ['bin/run', 'exec\n', 0o755], ['café.txt', 'unicode\n', 0o644]]) { - zip.append(Buffer.from(body), { name, date: new Date(0), mode }); - } - await zip.finalize(); -})(); diff --git a/crates/socket-patch-core/src/vendor/jvm/mod.rs b/crates/socket-patch-core/src/vendor/jvm/mod.rs index 7b834e89f..023e05024 100644 --- a/crates/socket-patch-core/src/vendor/jvm/mod.rs +++ b/crates/socket-patch-core/src/vendor/jvm/mod.rs @@ -14,7 +14,6 @@ //! does the disk side. pub mod apply; -pub(crate) mod archive; pub mod gradle; pub mod maven_reactor; @@ -607,3 +606,14 @@ mod tests { assert_eq!(detect(&empty), Shape::Other); } } + +pub(crate) fn is_signature(name: &str) -> bool { + let upper = name.to_ascii_uppercase(); + upper == ".SIGNATURE.P7S" + || upper.strip_prefix("META-INF/").is_some_and(|n| { + !n.contains('/') + && [".SF", ".RSA", ".DSA", ".EC"] + .iter() + .any(|ext| n.ends_with(ext)) + }) +} diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/python_lock_union_tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/python_lock_union_tests.rs index 5092a48b4..8b41d8b96 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/python_lock_union_tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/python_lock_union_tests.rs @@ -126,6 +126,7 @@ async fn python_document_recovery_canonicalizes_the_purl_name() { base_purl: "pkg:pypi/PyYAML@6.0.1".into(), uuid: "11111111-1111-4111-8111-111111111111".into(), artifact: crate::vendor::state::VendorArtifact { + yarn_berry10c0: None, path: ".socket/vendor/pypi/11111111-1111-4111-8111-111111111111/PyYAML-6.0.1-py3-none-any.whl".into(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/recover_tests.rs b/crates/socket-patch-core/src/vendor/lock_inventory/recover_tests.rs index 1ddb4171f..04ed9056d 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/recover_tests.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/recover_tests.rs @@ -10,6 +10,7 @@ fn entry(eco: &str, base_purl: &str, wiring: Vec) -> VendorEntry { base_purl: base_purl.into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/{eco}/{UUID}/x"), sha256: String::new(), size: None, diff --git a/crates/socket-patch-core/src/vendor/maven_repo.rs b/crates/socket-patch-core/src/vendor/maven_repo.rs index 2e09f183d..cea86175e 100644 --- a/crates/socket-patch-core/src/vendor/maven_repo.rs +++ b/crates/socket-patch-core/src/vendor/maven_repo.rs @@ -1,60 +1,3 @@ -//! Maven vendor backend: a committed maven2-layout repository plus a surgical -//! `` insert into the project's `pom.xml` pointing every resolve of -//! the patched GAV at a rebuilt, patched `.jar` served from the tree. -//! -//! Mechanism (verified against Apache Maven inside the docker capstone): -//! -//! * artifact — the uuid dir IS a maven2 *repository root*. Maven's standard -//! layout is `///-.jar` (+ the -//! companion `-.pom`), so laying the files at -//! `.socket/vendor/maven/////…` makes the uuid dir a fully -//! valid `file://` repository with no index needed. The `.jar` is rebuilt by -//! extracting the cached `~/.m2` jar, force-applying the patch, and re-zipping -//! deterministically (so a re-run never churns the committed bytes) — the -//! twin of the NuGet feed's `.nupkg` rebuild. -//! -//! * pom — the vendored `-.pom` MUST be the REAL upstream pom (copied -//! verbatim from `~/.m2`, or downloaded from the maven2 registry). Maven reads -//! it to discover the artifact's TRANSITIVE dependencies; a hand-authored -//! minimal pom would silently drop them and break the consumer's build. When -//! neither source can supply it we refuse (`vendor_maven_pom_unavailable`) -//! rather than fabricate one. -//! -//! * checksums — each committed file carries a `.sha1` sidecar (the hex -//! sha1 of its bytes). Our injected `` sets -//! `checksumPolicy=fail`, so Maven fetches the sidecar and hard-fails the -//! resolve if the jar/pom bytes don't match it (a tampered jar → checksum -//! failure). sha1 is the checksum Maven validates first; an `.md5` twin is -//! not written (it would need a new workspace dependency and Maven treats -//! sha1 as authoritative — the capstone proves `checksumPolicy=fail` is -//! fully enforced by the sha1 sidecar alone). -//! -//! * `pom.xml` — a single `` is inserted: -//! `id=socket-patch-vendor-`, -//! `url=file://${project.basedir}/.socket/vendor/maven/`, -//! `checksumPolicy=fail`, `false`. `${project.basedir}` -//! interpolates to the pom's own dir, so the file:// url resolves relative to -//! the committed tree on any checkout. -//! -//! Refusals (fail-closed, before any write): -//! * a root pom declaring `` (an aggregator) — -//! `vendor_maven_multimodule_unsupported`: `${project.basedir}` would -//! interpolate to each SUBMODULE's dir, not the root, so the file:// url would -//! point at the wrong place per module. -//! * a gradle-only project (a `build.gradle*` but no `pom.xml`) — -//! `vendor_gradle_unsupported`: there is no `` block to wire and -//! Gradle ignores it. -//! -//! Always-on advisory: `vendor_maven_local_cache_shadow`. Maven checks the LOCAL -//! repository (`~/.m2`) BEFORE any configured ``, so a warm -//! `~/.m2` copy of the same GAV silently wins over our patched file:// artifact. -//! The warning carries the `mvn dependency:purge-local-repository` one-liner to -//! clear it. -//! -//! Edit order: artifact (jar + pom + sidecars) → `pom.xml`. Any failure after -//! the artifact removes the uuid dir; the `pom.xml` edit runs last so a failed -//! artifact never leaves a dangling ``. - use std::collections::HashMap; use std::path::{Path, PathBuf}; use std::time::Duration; @@ -76,12 +19,11 @@ use crate::utils::purl::{build_maven_purl, parse_maven_purl}; use crate::utils::socket_dir::remove_tree_and_prune; use super::common::{ - already_patched_result, any_live_file_references, done, failed_result, prepare_memory_repack, - prune_empty_vendor_levels, read_zip_artifact, rebuild_zip, refused, synthesized_result, - write_zip_entries, zip_bytes_match_after_hashes, MemoryRepack, Stage, + already_patched_result, any_live_file_references, done, failed_result, + prune_empty_vendor_levels, read_zip_artifact, refused, synthesized_result, + zip_bytes_match_after_hashes, }; use super::path::vendor_uuid_dir_rel; -use super::registry_fetch::extract_zip; use super::service_fetch::{service_archive_copy, ServiceCopy}; use super::state::{ write_marker_or_warn, VendorArtifact, VendorEntry, VendorMarker, WiringAction, WiringRecord, @@ -476,22 +418,20 @@ pub async fn vendor_maven( // Dry runs fall through to the verify-only preview below. } - // ── dry run: verify-only against the extracted local jar, no writes ─── if dry_run { - let mut dry_warnings: Vec = vec![shadow_warning]; - let result = dry_run_verify( - purl, - installed_dir, - &jar_path, - artifact_id, - version, - record, - sources, - force, - &mut dry_warnings, - ) - .await; - return done(result, None, dry_warnings); + if let Err(outcome) = + super::service_fetch::preview_service(service, record, |bytes, dest| { + super::registry_fetch::extract_zip(bytes, dest, false) + }) + .await + { + return *outcome; + } + return done( + super::common::preview_result(purl, &jar_path, &record.files), + None, + vec![shadow_warning], + ); } // ── materialise the patched jar + real pom + sidecars ───────────────── @@ -588,6 +528,7 @@ fn maven_entry( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, // A `.jar` is a single verifiable file; record its plain sha256 for // tooling (harvest re-derives per-entry git hashes from the zip, so // the vendored copy is self-describing without a network). @@ -798,9 +739,8 @@ async fn jvm_committed_patch( /// Vendor into a multi-module reactor or a Gradle build through the /// [`super::jvm`] backend. The jar and pom come from the committed -/// tree when it already holds this patch, else from the same service / -/// local-rebuild rungs as the legacy path; nothing is written for a -/// refused plan. +/// tree when it already holds this patch, otherwise from the service and +/// authenticated upstream metadata; nothing is written for a refused plan. #[allow(clippy::too_many_arguments)] async fn vendor_maven_jvm( shape: super::jvm::Shape, @@ -808,9 +748,9 @@ async fn vendor_maven_jvm( installed_dir: &Path, project_root: &Path, record: &PatchRecord, - sources: &PatchSources<'_>, + _sources: &PatchSources<'_>, dry_run: bool, - force: bool, + _force: bool, service: Option<&VendorServiceConfig>, ) -> VendorOutcome { let Some((group_id, artifact_id, version)) = parse_maven_purl(purl) else { @@ -887,55 +827,6 @@ async fn vendor_maven_jvm( already_patched_result(purl, &display_path, &record.files), ), ServiceCopy::HardFail(outcome) => return *outcome, - ServiceCopy::FallBack => { - match local_rebuild_jar( - purl, - installed_dir, - &display_path, - &artifact_id, - &version, - record, - sources, - force, - &mut warnings, - ) - .await - { - Ok((bytes, result)) if result.success => { - let upstream = match read_regular_to_bytes( - &installed_dir.join(format!("{artifact_id}-{version}.jar")), - ) - .await - { - Ok(bytes) => bytes, - Err(e) => { - return refused( - "vendor_jvm_upstream_unavailable", - e.to_string(), - ) - } - }; - if let Err(e) = verify_jvm_upstream( - &upstream, - &group_id, - &artifact_id, - &version, - "jar", - service, - ) - .await - { - return refused("vendor_prebuilt_integrity_mismatch", e); - } - match super::jvm::archive::canonical_jar(&upstream, &bytes) { - Ok(bytes) => (bytes, result), - Err(e) => return refused("vendor_jvm_upstream_unavailable", e), - } - } - Ok(pair) => pair, - Err(outcome) => return jvm_refusal(*outcome), - } - } }; if !result.success { return done(result, None, warnings); @@ -1173,7 +1064,11 @@ async fn vendor_maven_jvm( ); } if dry_run { - return done(result, None, warnings); + return done( + super::common::preview_result(purl, &jar_path, &record.files), + None, + warnings, + ); } let mut wiring = match super::jvm::apply::write_plan(project_root, &plan).await { Ok(records) => records, @@ -1204,7 +1099,7 @@ fn verify_unpatched_jar_members( let mut original = super::verify::read_zip_bytes_to_map(upstream)?; let mut committed = super::verify::read_zip_bytes_to_map(patched)?; let retain = |name: &String, _: &mut Vec| { - !record.files.contains_key(name) && !super::jvm::archive::is_signature(name) + !record.files.contains_key(name) && !super::jvm::is_signature(name) }; original.retain(retain); committed.retain(retain); @@ -1252,7 +1147,7 @@ async fn verify_jvm_upstream( Ok(()) } -async fn acquire_jvm_metadata( +pub(super) async fn acquire_jvm_metadata( dir: &Path, g: &str, a: &str, @@ -1390,27 +1285,11 @@ async fn collect_metadata_artifacts( Ok(()) } -/// A legacy jar refusal in the JVM backend's codes. -fn jvm_refusal(outcome: VendorOutcome) -> VendorOutcome { - match outcome { - VendorOutcome::Refused { - code: "vendor_maven_jar_not_found", - detail, - } => refused( - "vendor_jvm_upstream_unavailable", - format!("reason: no_base_jar: {detail}"), - ), - other => other, - } -} - -// ── materialisation (service download / local rebuild) ────────────────────────── +// ── materialisation (service download) ────────────────────────── /// Produce the patched jar bytes + the real upstream pom, then write both (with /// their `.sha1` sidecars) into the maven2 leaf dir. Returns `(jar_bytes, -/// ApplyResult)`, or a terminal [`VendorOutcome`] to bubble. On a non-fatal -/// rebuild failure the returned `ApplyResult.success` is false and the partial -/// uuid dir is cleaned up. +/// ApplyResult)`, or a terminal [`VendorOutcome`] to bubble. #[allow(clippy::too_many_arguments)] async fn materialise_and_write( purl: &str, @@ -1425,44 +1304,18 @@ async fn materialise_and_write( version: &str, group_path: &str, record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, + _sources: &PatchSources<'_>, + _force: bool, service: Option<&VendorServiceConfig>, warnings: &mut Vec, ) -> Result<(Vec, ApplyResult), Box> { - // The patched jar first (service Tier A, else local rebuild). A non-fatal - // failure returns an un-successful ApplyResult with nothing written. let (jar_bytes, result) = match service_archive_copy(service, record, artifact_id, ".jar", warnings).await { ServiceCopy::Used(bytes) => { (bytes, already_patched_result(purl, jar_path, &record.files)) } ServiceCopy::HardFail(outcome) => return Err(outcome), - ServiceCopy::FallBack => { - match local_rebuild_jar( - purl, - installed_dir, - jar_path, - artifact_id, - version, - record, - sources, - force, - warnings, - ) - .await - { - Ok(pair) => pair, - Err(outcome) => return Err(outcome), - } - } }; - if !result.success { - // Local rebuild reported a failure; nothing on disk to clean up (the - // jar is rebuilt in memory and only written below on success). - return Ok((jar_bytes, result)); - } - // The REAL upstream pom (transitive-deps correctness). A miss is terminal: // refuse rather than fabricate a minimal pom. let pom_bytes = match acquire_upstream_pom( @@ -1490,94 +1343,6 @@ async fn materialise_and_write( Ok((jar_bytes, result)) } -/// Local rebuild: locate the cached pristine `-.jar` in `installed_dir`, -/// read it for a private stage — only the paths the apply pipeline resolves -/// are materialised there, see [`stage_local_jar`] — force-apply the patch, -/// and re-zip deterministically. Returns `(bytes, ApplyResult)`; a failure -/// surfaces as an un-successful `ApplyResult`, or a refusal to bubble. -#[allow(clippy::too_many_arguments)] -async fn local_rebuild_jar( - purl: &str, - installed_dir: &Path, - jar_path: &Path, - artifact_id: &str, - version: &str, - record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, - warnings: &mut Vec, -) -> Result<(Vec, ApplyResult), Box> { - let src_jar = installed_dir.join(format!("{artifact_id}-{version}.jar")); - if tokio::fs::metadata(&src_jar).await.is_err() { - return Err(Box::new(refused( - "vendor_maven_jar_not_found", - format!( - "no cached {} under {} to rebuild the patched artifact from (a vendored feed \ - needs the pristine jar; re-resolve it or use --vendor-source=service)", - src_jar - .file_name() - .map(|n| n.to_string_lossy().into_owned()) - .unwrap_or_default(), - installed_dir.display() - ), - ))); - } - let JarStage { stage, repack } = match stage_local_jar(&src_jar, &record.files).await { - Ok(staged) => staged, - Err(e) => return Ok((Vec::new(), failed_result(purl, jar_path, e))), - }; - - let result = super::force_apply_staged( - purl, - stage.path(), - record, - sources, - /*dry_run=*/ false, - force, - artifact_id, - version, - warnings, - ) - .await; - if !result.success { - stage.dispose().await; - return Ok((Vec::new(), result)); - } - - let rebuilt = rebuild_jar_bytes(repack, stage.path()).await; - stage.dispose().await; - match rebuilt { - Ok(jar_bytes) => Ok((jar_bytes, result)), - Err(e) => Ok((Vec::new(), failed_result(purl, jar_path, e))), - } -} - -/// Deterministic re-zip of the patched stage (a jar is a plain zip; a -/// dependency resolve reads the central directory, so lexicographic entry -/// order + fixed timestamps yield stable bytes across re-runs). The in-memory -/// repack assembles the same entry list from the members it never wrote out; -/// an archive that had to be extracted is walked on disk. -async fn rebuild_jar_bytes(repack: Option, stage: &Path) -> Result, String> { - let rezip = match repack { - Some(repack) => { - let entries = repack - .into_entries(stage, None) - .await - .map_err(|e| format!("jar re-zip failed: {e}"))?; - tokio::task::spawn_blocking(move || write_zip_entries(&entries)).await - } - None => { - let stage_path = stage.to_path_buf(); - tokio::task::spawn_blocking(move || rebuild_zip(&stage_path, None)).await - } - }; - match rezip { - Ok(Ok(bytes)) => Ok(bytes), - Ok(Err(e)) => Err(format!("jar re-zip failed: {e}")), - Err(e) => Err(format!("jar re-zip task failed: {e}")), - } -} - /// Acquire the REAL upstream pom bytes: the cached `~/.m2` copy first (the /// common case — the package was resolved locally), then a maven2 registry /// download when the service is enabled. An `Err(detail)` maps to a @@ -1657,91 +1422,9 @@ async fn fetch_registry_bytes(url: &str, cap: u64) -> Result, String> { .map_err(|e| format!("{url}: {e}")) } -/// Dry-run verify-only: extract the local jar to a private stage and run the -/// apply pipeline in preview mode. A missing local jar surfaces as a failed -/// result (the preview cannot predict a rebuild it cannot stage). -#[allow(clippy::too_many_arguments)] -async fn dry_run_verify( - purl: &str, - installed_dir: &Path, - jar_path: &Path, - artifact_id: &str, - version: &str, - record: &PatchRecord, - sources: &PatchSources<'_>, - force: bool, - warnings: &mut Vec, -) -> ApplyResult { - let src_jar = installed_dir.join(format!("{artifact_id}-{version}.jar")); - if tokio::fs::metadata(&src_jar).await.is_err() { - return failed_result( - purl, - jar_path, - format!( - "no cached {}-{}.jar under {} to preview the vendored artifact", - artifact_id, - version, - installed_dir.display() - ), - ); - } - let JarStage { stage, .. } = match stage_local_jar(&src_jar, &record.files).await { - Ok(staged) => staged, - Err(e) => return failed_result(purl, jar_path, e), - }; - let mut result = super::force_apply_staged( - purl, - stage.path(), - record, - sources, - /*dry_run=*/ true, - force, - artifact_id, - version, - warnings, - ) - .await; - stage.dispose().await; - result.package_path = jar_path.display().to_string(); - result -} - -// ── artifact helpers ───────────────────────────────────────────────────────────── - -/// The stage a local jar rebuild (or its dry-run preview) applies into: the -/// live [`Stage`] the caller holds for its lifetime, plus the in-memory -/// members when the repack could stay off disk. -struct JarStage { - stage: Stage, - repack: Option, -} - -/// Read a jar (a plain zip; content at the archive root — no strip) for a -/// local rebuild. `prepare_memory_repack` keeps the members in memory and -/// materialises only the paths the apply pipeline resolves; a jar whose entry -/// names a filesystem could fold together or re-spell is extracted whole -/// instead, the shape this one is defined against. Both are traversal-guarded -/// and refuse an escaping entry fail-closed, with the same message. -async fn stage_local_jar( - src_jar: &Path, - files: &HashMap, -) -> Result { - let bytes = read_regular_to_bytes(src_jar) - .await - .map_err(|e| format!("cannot read {}: {e}", src_jar.display()))?; - let stage = Stage::new().map_err(|e| format!("cannot create stage dir: {e}"))?; - let unreadable = |e| format!("cannot extract {}: {e}", src_jar.display()); - let repack = prepare_memory_repack(&bytes, files, &[]).map_err(unreadable)?; - match &repack { - Some(repack) => repack.stage_into(stage.path()).await.map_err(unreadable)?, - None => extract_zip(&bytes, stage.path(), /*strip_first=*/ false).map_err(unreadable)?, - } - Ok(JarStage { stage, repack }) -} - /// Write the jar + pom + their `.sha1` sidecars into the maven2 leaf dir, /// creating it. Errors are strings. -async fn write_maven_artifact( +pub(super) async fn write_maven_artifact( leaf_dir: &Path, jar_leaf: &str, jar_bytes: &[u8], @@ -2213,44 +1896,6 @@ mod tests { zw.finish().unwrap().into_inner() } - /// A jar carrying every spelling the in-memory repack and the - /// extract-to-disk rebuild could disagree on: a zero-length member, a - /// STORED (uncompressed) member, an exec-bit member, a nested tree, a - /// member large enough to span several read buffers, and the patch - /// target. - fn make_rich_jar(notice: &[u8]) -> Vec { - use zip::CompressionMethod::{Deflated, Stored}; - let big = vec![b'z'; 3 * 1024 * 1024]; - let entries: &[(&str, &[u8], zip::CompressionMethod, u32)] = &[ - ( - "META-INF/MANIFEST.MF", - b"Manifest-Version: 1.0\n", - Deflated, - 0o644, - ), - (JAR_FILE, notice, Deflated, 0o644), - ("META-INF/empty", b"", Deflated, 0o644), - ("META-INF/stored.bin", b"stored bytes", Stored, 0o644), - ("bin/run.sh", b"#!/bin/sh\nexit 0\n", Deflated, 0o755), - ("org/apache/commons/text/big.bin", &big, Deflated, 0o644), - ( - "org/apache/commons/text/StringSubstitutor.class", - b"\xca\xfe\xba\xbe-fake-class", - Deflated, - 0o644, - ), - ]; - let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); - for (name, bytes, method, mode) in entries { - let opts = zip::write::SimpleFileOptions::default() - .compression_method(*method) - .unix_permissions(*mode); - zw.start_file(*name, opts).unwrap(); - zw.write_all(bytes).unwrap(); - } - zw.finish().unwrap().into_inner() - } - /// A minimal project pom.xml at the root (single-module, no ). fn project_pom() -> &'static str { "\n\ @@ -2357,6 +2002,7 @@ mod tests { /// empty patch's no-op, and — once vendored — not the in-sync re-run; a /// second patch uuid for the same artifact asks again. #[tokio::test] + #[serial_test::serial] async fn service_preflight_names_exactly_the_artifacts_that_ask_for_a_grant() { use crate::vendor::test_support::{ empty_patch, mount_no_results, plan_matches_grants, service_cfg, with_uuid, Borrowed, @@ -2366,7 +2012,7 @@ mod tests { let root = dir.path(); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let sources = PatchSources::blobs_only(&blobs); let cases = [ (PURL, record.clone()), @@ -2384,7 +2030,7 @@ mod tests { let vendor = |purl: String, rec: PatchRecord| -> Borrowed<'_, VendorOutcome> { let (installed, sources, cfg) = (&installed, &sources, &cfg); Box::pin(async move { - vendor_maven( + crate::vendor::test_support::vendor_maven( &purl, installed.as_path(), root, @@ -2410,7 +2056,7 @@ mod tests { dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_maven( + crate::vendor::test_support::vendor_maven( PURL, installed, root, @@ -2436,94 +2082,13 @@ mod tests { /// EXACT bytes the extract-to-disk rebuild produced — the artifact's sha1 /// sidecar and every downstream pin ride on them. Driven twice over one /// fixture, once with the in-memory repack forced off. - #[tokio::test] - #[serial_test::serial] - async fn in_memory_jar_rebuild_matches_the_on_disk_rebuild_byte_for_byte() { - async fn rebuild(on_disk: bool) -> Vec { - let _forced = on_disk.then(crate::vendor::common::OnDiskRepackGuard::acquire); - let before = crate::vendor::common::in_memory_repacks(); - let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; - tokio::fs::write( - installed.join("commons-text-1.10.0.jar"), - make_rich_jar(PRISTINE), - ) - .await - .unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(dir.path(), &blobs, &installed, &record, false).await); - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some(), "a successful rebuild records an entry"); - // Without this the comparison is vacuous: a fixture name the gate - // later rejects would send BOTH runs to disk and the test would - // keep passing while asserting nothing. - assert_eq!( - crate::vendor::common::in_memory_repacks() > before, - !on_disk, - "this run took the wrong staging path (on_disk={on_disk})" - ); - tokio::fs::read(dir.path().join(jar_rel())).await.unwrap() - } - - let fast = rebuild(false).await; - let oracle = rebuild(true).await; - assert_eq!( - fast, oracle, - "the in-memory rebuild must be byte-identical to the extracted one" - ); - assert_eq!(read_jar_entry(&fast, JAR_FILE).as_deref(), Some(PATCHED)); - assert_eq!( - read_jar_entry(&fast, "META-INF/empty").as_deref(), - Some(&[][..]) - ); - assert_eq!( - read_jar_entry(&fast, "META-INF/stored.bin").as_deref(), - Some(&b"stored bytes"[..]) - ); - } /// A jar whose entry escapes the stage must be refused the same way /// whichever staging path ran — the traversal guard is the one thing both /// readers have to agree on before anything is written. - #[tokio::test] - #[serial_test::serial] - async fn escaping_jar_entry_fails_the_same_on_both_staging_paths() { - async fn rebuild(on_disk: bool) -> Option { - let _forced = on_disk.then(crate::vendor::common::OnDiskRepackGuard::acquire); - let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; - let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); - zw.start_file("../evil.class", zip::write::SimpleFileOptions::default()) - .unwrap(); - zw.write_all(b"pwned").unwrap(); - let evil = zw.finish().unwrap().into_inner(); - tokio::fs::write(installed.join("commons-text-1.10.0.jar"), evil) - .await - .unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(dir.path(), &blobs, &installed, &record, false).await); - assert!(!result.success, "an escaping entry must fail the rebuild"); - assert!(entry.is_none()); - assert!( - !dir.path().join("evil.class").exists() - && !dir.path().parent().unwrap().join("evil.class").exists(), - "nothing may be written outside the stage" - ); - // Past the fixture's own tempdir path, which differs per run. - result.error.map(|e| { - e.rsplit_once(".jar: ") - .map(|(_, tail)| tail.to_string()) - .unwrap_or(e) - }) - } - - let fast = rebuild(false).await; - assert_eq!(fast, rebuild(true).await); - assert_eq!( - fast.as_deref(), - Some("zip entry `../evil.class` escapes the extraction dir — refusing the artifact") - ); - } #[tokio::test] + #[serial_test::serial] async fn happy_path_wires_repo_jar_pom_sidecars() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2605,6 +2170,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn rerun_is_idempotent_no_rerecord() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2635,6 +2201,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn wired_missing_artifact_rebuilds_only() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2677,6 +2244,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn missing_reactor_module_is_refused_without_writes() { let multimodule = "\n\ \x20 4.0.0\n\ @@ -2696,6 +2264,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn commented_modules_do_not_refuse() { // A commented-out must NOT trigger the aggregator refusal. let commented = "\n\ @@ -2717,6 +2286,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn vendors_gradle_only_project_by_default() { // build.gradle but no pom.xml → gradle-only. let (dir, blobs, installed, record) = fixture(None, true, true).await; @@ -2732,6 +2302,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn refuses_pom_unavailable() { // pom.xml present, local jar present, but NO upstream pom (and no // service) → refuse rather than author a minimal pom. @@ -2757,18 +2328,7 @@ mod tests { } #[tokio::test] - async fn refuses_missing_local_jar() { - // pom.xml + upstream pom present, but the cached jar is gone and no - // service is configured → nothing to rebuild from. - let (dir, blobs, installed, record) = - fixture(Some(project_pom()), /*with_local_jar=*/ false, true).await; - let root = dir.path(); - let (code, _d) = unwrap_refused(run_vendor(root, &blobs, &installed, &record, false).await); - assert_eq!(code, "vendor_maven_jar_not_found"); - assert!(!root.join(".socket").exists()); - } - - #[tokio::test] + #[serial_test::serial] async fn refuses_unsafe_coordinates() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2781,7 +2341,7 @@ mod tests { // A traversal in the coordinate group is refused too. let sources = PatchSources::blobs_only(&blobs); let (code, _d) = unwrap_refused( - vendor_maven( + crate::vendor::test_support::vendor_maven( "pkg:maven/../evil/x@1.0.0", &installed, root, @@ -2798,6 +2358,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn dry_run_writes_nothing() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2821,6 +2382,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn revert_restores_pom_byte_identical() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2858,6 +2420,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn revert_drift_leaves_pom_alone() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -2896,6 +2459,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn revert_excises_only_our_block_preserving_sibling() { // Vendor creates the section with OUR block. Then a // sibling vendor run inserts ANOTHER into that same @@ -2951,6 +2515,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn revert_preserves_user_edit_made_after_vendoring() { // The user edits the pom AFTER vendoring (adds a block). // Revert must remove our (and the section we created) yet @@ -3000,6 +2565,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn revert_warns_when_our_block_already_gone() { // The user regenerated the pom, dropping our block but keeping a // hand-written . Our exact block is absent → drift, and @@ -3047,6 +2613,7 @@ mod tests { } #[test] + #[serial_test::serial] fn strip_empty_repositories_removes_created_section_only() { // A section left empty after excision is removed. let empty = "\n \n \n\n"; @@ -3066,6 +2633,7 @@ mod tests { } #[test] + #[serial_test::serial] fn declares_modules_boundary_and_comment_discipline() { assert!(declares_modules( "a" @@ -3084,6 +2652,7 @@ mod tests { } #[test] + #[serial_test::serial] fn repo_edit_extends_existing_repositories() { let orig = "\n \n corp\n \n\n"; let out = build_repo_edit(orig, "socket-patch-vendor-x", ".socket/vendor/maven/x").unwrap(); @@ -3094,6 +2663,7 @@ mod tests { } #[test] + #[serial_test::serial] fn repo_edit_creates_repositories_section() { let orig = "\n app\n\n"; let out = build_repo_edit(orig, "socket-patch-vendor-x", ".socket/vendor/maven/x").unwrap(); @@ -3104,6 +2674,7 @@ mod tests { } #[test] + #[serial_test::serial] fn group_id_path_and_safety() { assert_eq!(group_id_to_path("org.apache.commons"), "org/apache/commons"); let is_safe_group_id = |g| is_safe_maven_coordinate(g, "a", "1"); @@ -3117,6 +2688,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn wires_outside_commented_repositories() { // A commented-out section must not capture the insert: // a block landing inside the comment is invisible to Maven, so the @@ -3148,6 +2720,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn wires_project_root_not_profile_repositories() { // A inside is only consulted when that // profile is activated; anchoring our block there leaves the default @@ -3185,6 +2758,7 @@ mod tests { } #[test] + #[serial_test::serial] fn repo_edit_skips_commented_and_profile_anchors() { // Only a commented → a NEW real section is created. let commented = "\n\n\n"; @@ -3206,6 +2780,7 @@ mod tests { #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn wire_preserves_pom_xml_mode() { use std::os::unix::fs::PermissionsExt as _; let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; @@ -3229,6 +2804,7 @@ mod tests { #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn revert_preserves_pom_xml_mode() { use std::os::unix::fs::PermissionsExt as _; let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; @@ -3280,6 +2856,7 @@ mod tests { } #[tokio::test] + #[serial_test::serial] async fn wired_rebuild_reports_vendored_jar_path() { // The wired-but-missing-artifact rebuild leg must report the vendored // jar path, not the (deleted) temp stage the rebuild ran in. @@ -3342,6 +2919,7 @@ mod tests { /// already guarded in common.rs; this pins the `sidecar_matches` half. #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn fifo_vendored_pom_fails_fast_and_rebuilds_on_hot_path() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3378,6 +2956,7 @@ mod tests { /// unreadable refusal instead of wedging every vendor run forever. #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn fifo_project_pom_fails_fast_in_vendor() { let (dir, blobs, installed, record) = fixture(None, true, true).await; let root = dir.path(); @@ -3394,37 +2973,11 @@ mod tests { assert_eq!(code, "vendor_maven_pom_unreadable"); } - /// A FIFO planted as the cached `~/.m2` jar passes the metadata probe but - /// must fail the stage read fast instead of wedging the rebuild forever. - #[cfg(unix)] - #[tokio::test] - async fn fifo_cached_jar_fails_fast_in_local_rebuild() { - let (dir, blobs, installed, record) = - fixture(Some(project_pom()), /*with_local_jar=*/ false, true).await; - let root = dir.path(); - let fifo_jar = installed.join("commons-text-1.10.0.jar"); - mkfifo(&fifo_jar); - - let outcome = expect_fast( - run_vendor(root, &blobs, &installed, &record, false), - &fifo_jar, - "the local rebuild must fail fast on a FIFO cached jar, not wedge", - ) - .await; - let (result, entry, _w) = unwrap_done(outcome); - assert!(!result.success, "a FIFO jar cannot be staged"); - assert!(entry.is_none()); - assert!( - result.error.as_deref().is_some_and(|e| e.contains("read")), - "failure names the unreadable jar: {:?}", - result.error - ); - } - /// A FIFO planted as the cached `~/.m2` pom must map to the /// pom-unavailable refusal fast instead of wedging the pom copy forever. #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn fifo_local_pom_fails_fast_in_acquire_upstream_pom() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, /*with_local_pom=*/ false).await; @@ -3454,6 +3007,7 @@ mod tests { /// the maven2 registry client must identify as the official Maven CLI, /// never as `SocketPatchCLI/…`. #[test] + #[serial_test::serial] fn maven_user_agent_is_the_maven_cli_shape() { assert!( MAVEN_USER_AGENT.starts_with("Apache-Maven/"), @@ -3470,6 +3024,7 @@ mod tests { /// regression back to `SocketPatchCLI/…` misses the matcher and fails /// the fetch. #[tokio::test] + #[serial_test::serial] async fn pom_fetch_sends_the_maven_cli_user_agent() { use wiremock::matchers::{header, method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -3495,6 +3050,7 @@ mod tests { /// forever. #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn fifo_project_pom_fails_fast_in_revert() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3588,7 +3144,7 @@ mod tests { cfg: &VendorServiceConfig, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_maven( + crate::vendor::test_support::vendor_maven( PURL, installed, root, @@ -3605,12 +3161,13 @@ mod tests { /// A purl from another ecosystem entirely fails `parse_maven_purl` and is /// refused before any coordinate/uuid processing. #[tokio::test] + #[serial_test::serial] async fn refuses_non_maven_purl() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); let sources = PatchSources::blobs_only(&blobs); let (code, detail) = unwrap_refused( - vendor_maven( + crate::vendor::test_support::vendor_maven( "pkg:npm/leftpad@1.0.0", &installed, root, @@ -3635,6 +3192,7 @@ mod tests { /// gradle sibling is tested above; this pins the non-gradle arm and /// `project_has_gradle` returning false through all four probes). #[tokio::test] + #[serial_test::serial] async fn refuses_pom_project_missing_without_gradle_marker() { let (dir, blobs, installed, record) = fixture(None, true, true).await; let root = dir.path(); @@ -3653,6 +3211,7 @@ mod tests { /// out while pom.xml keeps our (now-dangling) — documenting /// the wired-but-refused state. #[tokio::test] + #[serial_test::serial] async fn wired_missing_jar_bubbles_refusal_keeping_wiring() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3668,7 +3227,7 @@ mod tests { .unwrap(); let (code, _d) = unwrap_refused(run_vendor(root, &blobs, &installed, &record, false).await); - assert_eq!(code, "vendor_maven_jar_not_found"); + assert_eq!(code, "vendor_prebuilt_required"); assert_eq!( tokio::fs::read(root.join(PROJECT_POM)).await.unwrap(), wired, @@ -3684,6 +3243,7 @@ mod tests { /// patch blob is gone): the un-successful result is reported with no /// ledger re-record, pom.xml untouched, and no partial uuid dir. #[tokio::test] + #[serial_test::serial] async fn wired_rebuild_failure_reports_unsuccessful_result() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3698,7 +3258,9 @@ mod tests { .await .unwrap(); - let (r2, e2, _w2) = unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); + let (r2, e2, _w2) = crate::vendor::test_support::expect_failed( + run_vendor(root, &blobs, &installed, &record, false).await, + ); assert!(!r2.success, "a blob-less rebuild cannot succeed"); assert!(r2.error.is_some(), "the failure carries a detail"); assert!(e2.is_none(), "a failed rebuild must not re-record"); @@ -3716,6 +3278,7 @@ mod tests { /// Wired hot path + stale artifact + --dry-run: falls through to the /// verify-only preview — nothing is rebuilt or written. #[tokio::test] + #[serial_test::serial] async fn wired_stale_artifact_dry_run_previews_without_writing() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3748,6 +3311,7 @@ mod tests { /// with no ): success flips false, the detail is carried, and /// the uuid dir is removed so no orphan artifact survives. #[tokio::test] + #[serial_test::serial] async fn unwireable_pom_fails_after_materialise_and_cleans_up() { let broken = "\n app\n"; let (dir, blobs, installed, record) = fixture(Some(broken), true, true).await; @@ -3781,6 +3345,7 @@ mod tests { /// Unit legs of the wiring-failure class: no at all, and an /// unterminated comment masking BOTH anchors to EOF (fail-closed). #[test] + #[serial_test::serial] fn repo_edit_errors_without_unmasked_project_close() { let err = build_repo_edit( "x", @@ -3807,6 +3372,7 @@ mod tests { /// materialised: the uuid dir is removed and the error names the pom.xml. #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn pom_write_failure_cleans_up_artifact() { use std::os::unix::fs::PermissionsExt as _; let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; @@ -3860,6 +3426,7 @@ mod tests { /// A marker write failure must NOT fail an otherwise-wired vendor — /// state.json is the ledger of record; the marker is advisory only. #[tokio::test] + #[serial_test::serial] async fn marker_write_failure_warns_but_succeeds() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3893,6 +3460,7 @@ mod tests { /// Revert fail-closed on a non-canonical uuid in the (tamper-able) /// state.json entry — refused before any disk access. #[tokio::test] + #[serial_test::serial] async fn revert_refuses_non_canonical_uuid() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3930,6 +3498,7 @@ mod tests { /// so the artifact is drift-kept (deleting it would strand the /// `` at a gone path). #[tokio::test] + #[serial_test::serial] async fn revert_unrecognized_wiring_kind_warns_and_keeps_the_referenced_artifact() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -3975,6 +3544,7 @@ mod tests { /// tolerated as drift — `` in the warning, pom.xml untouched; /// the still-wired pom keeps the artifact. #[tokio::test] + #[serial_test::serial] async fn revert_tolerates_wiring_key_missing() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4018,6 +3588,7 @@ mod tests { /// A wiring record whose `original` is not a string is tolerated as drift; /// pom.xml is untouched and, still wired, keeps the artifact. #[tokio::test] + #[serial_test::serial] async fn revert_tolerates_wiring_original_missing() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4061,6 +3632,7 @@ mod tests { /// A wiring record whose `new` snapshot is missing skips the byte-identical /// fast path but still excises our block surgically. #[tokio::test] + #[serial_test::serial] async fn revert_missing_new_snapshot_excises_block() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4100,6 +3672,7 @@ mod tests { /// pom.xml deleted by the user before revert: nothing to restore, the /// revert proceeds cleanly and still removes the artifact. #[tokio::test] + #[serial_test::serial] async fn revert_with_pom_deleted_still_removes_artifact() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4136,6 +3709,7 @@ mod tests { /// fails (partial-revert semantics, pinned deliberately). #[cfg(unix)] #[tokio::test] + #[serial_test::serial] async fn revert_remove_tree_failure_reported() { use std::os::unix::fs::PermissionsExt as _; let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; @@ -4188,6 +3762,7 @@ mod tests { /// vendored jar, sha1 sidecar, and the ledger sha256 all describe the /// service bytes, and pom.xml is wired. #[tokio::test] + #[serial_test::serial] async fn service_prebuilt_jar_written_verbatim() { use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -4258,6 +3833,7 @@ mod tests { /// --vendor-source=service + --offline is a fail-closed conflict, refused /// before any write. #[tokio::test] + #[serial_test::serial] async fn service_mode_offline_refuses_before_any_write() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4279,6 +3855,7 @@ mod tests { /// write_maven_artifact failure (a regular file squatting on the maven2 /// group path): failed result + the whole uuid dir cleaned up. #[tokio::test] + #[serial_test::serial] async fn leaf_write_failure_fails_and_cleans_up() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4314,32 +3891,6 @@ mod tests { ); } - /// The most common real-world rebuild failure: the patch blob is missing, - /// so force_apply_staged fails inside local_rebuild_jar — surfaced as an - /// un-successful result with nothing written. - #[tokio::test] - async fn local_rebuild_apply_failure_surfaces() { - let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; - let root = dir.path(); - tokio::fs::remove_file(blobs.join(compute_git_sha256_from_bytes(PATCHED))) - .await - .unwrap(); - - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); - assert!(!result.success, "a blob-less apply cannot succeed"); - assert!(result.error.is_some(), "the failure carries a detail"); - assert!(entry.is_none()); - assert!(!root.join(".socket").exists(), "nothing written on failure"); - assert_eq!( - tokio::fs::read_to_string(root.join(PROJECT_POM)) - .await - .unwrap(), - project_pom(), - "pom.xml untouched" - ); - } - /// The full registry pom-download leg (no local pom, service enabled): the /// pom is fetched from SOCKET_MAVEN_REGISTRY (trailing slash trimmed) under /// the Maven CLI UA, vendored verbatim with a matching sidecar, and the @@ -4352,11 +3903,13 @@ mod tests { use wiremock::{Mock, MockServer, ResponseTemplate}; let server = MockServer::start().await; - Mock::given(method("POST")) - .and(path("/v0/orgs/acme/patches/package")) - .respond_with(ResponseTemplate::new(404)) - .mount(&server) - .await; + crate::vendor::test_support::mount_granted( + &server, + UUID, + "commons-text-1.10.0.jar", + &make_jar(PATCHED), + ) + .await; let pom_route = "/org/apache/commons/commons-text/1.10.0/commons-text-1.10.0.pom"; Mock::given(method("GET")) .and(path(pom_route)) @@ -4373,7 +3926,7 @@ mod tests { let root = dir.path(); let cfg = service_cfg( Some(&server.uri()), - crate::vendor::VendorSource::Auto, + crate::vendor::VendorSource::Service, false, ); let (result, entry, warnings) = @@ -4405,15 +3958,16 @@ mod tests { #[tokio::test] #[serial_test::serial] async fn registry_pom_download_failure_maps_to_pom_unavailable() { - use wiremock::matchers::{method, path}; - use wiremock::{Mock, MockServer, ResponseTemplate}; + use wiremock::MockServer; let server = MockServer::start().await; - Mock::given(method("POST")) - .and(path("/v0/orgs/acme/patches/package")) - .respond_with(ResponseTemplate::new(404)) - .mount(&server) - .await; + crate::vendor::test_support::mount_granted( + &server, + UUID, + "commons-text-1.10.0.jar", + &make_jar(PATCHED), + ) + .await; // The pom route is NOT mounted → wiremock answers 404. let _reg = EnvGuard::set("SOCKET_MAVEN_REGISTRY", &server.uri()); @@ -4422,7 +3976,7 @@ mod tests { let root = dir.path(); let cfg = service_cfg( Some(&server.uri()), - crate::vendor::VendorSource::Auto, + crate::vendor::VendorSource::Service, false, ); let (code, detail) = @@ -4440,6 +3994,7 @@ mod tests { /// fetch_pom_bytes rejects a non-2xx response with the status in the error. #[tokio::test] + #[serial_test::serial] async fn fetch_pom_bytes_rejects_http_error() { use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -4460,6 +4015,7 @@ mod tests { /// fetch_pom_bytes rejects a body over MAX_POM_BYTES (a mirror serving the /// wrong thing) with the cap in the error. #[tokio::test] + #[serial_test::serial] async fn fetch_pom_bytes_rejects_oversize_body() { use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -4477,58 +4033,10 @@ mod tests { assert!(err.contains("cap"), "{err}"); } - /// Dry run with no cached local jar: the preview cannot stage, so it fails - /// with the to-preview error and writes nothing. - #[tokio::test] - async fn dry_run_missing_local_jar_fails_preview() { - let (dir, blobs, installed, record) = - fixture(Some(project_pom()), /*with_local_jar=*/ false, true).await; - let root = dir.path(); - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &installed, &record, true).await); - assert!(!result.success, "a jar-less preview cannot succeed"); - assert!( - result - .error - .as_deref() - .is_some_and(|e| e.contains("to preview")), - "failure names the missing preview source: {:?}", - result.error - ); - assert!(entry.is_none()); - assert!(!root.join(".socket").exists(), "dry run writes nothing"); - } - - /// A FIFO planted as the cached jar passes the metadata probe but must - /// fail the dry-run stage read fast instead of wedging the preview. - #[cfg(unix)] - #[tokio::test] - async fn fifo_cached_jar_fails_fast_in_dry_run_preview() { - let (dir, blobs, installed, record) = - fixture(Some(project_pom()), /*with_local_jar=*/ false, true).await; - let root = dir.path(); - let fifo_jar = installed.join("commons-text-1.10.0.jar"); - mkfifo(&fifo_jar); - - let outcome = expect_fast( - run_vendor(root, &blobs, &installed, &record, true), - &fifo_jar, - "the dry-run preview must fail fast on a FIFO cached jar, not wedge", - ) - .await; - let (result, entry, _w) = unwrap_done(outcome); - assert!(!result.success, "a FIFO jar cannot be staged for preview"); - assert!(entry.is_none()); - assert!( - result.error.as_deref().is_some_and(|e| e.contains("read")), - "failure names the unreadable jar: {:?}", - result.error - ); - } - /// A missing `.sha1` sidecar (file intact) reads as stale — checksumPolicy /// integrity is self-healing via the artifact rebuild. #[tokio::test] + #[serial_test::serial] async fn missing_sidecar_reads_stale_and_rebuilds() { let (dir, blobs, installed, record) = fixture(Some(project_pom()), true, true).await; let root = dir.path(); @@ -4558,6 +4066,7 @@ mod tests { /// `` must NOT mask the real ``; an unclosed /// `` masks to EOF fail-closed; a name-at-EOF open masks too. #[test] + #[serial_test::serial] fn profiles_masking_edge_branches() { // Decoy: is not — the real section is wireable. let decoy = @@ -4601,6 +4110,7 @@ mod tests { /// declares_modules fail-closed edges: an unterminated comment drops its /// tail (never counts), a truncated `` span dropped — what NuGet actually reads. @@ -3116,7 +2731,7 @@ mod tests { let outcome = run_vendor(root, &blobs, &installed, &record, false).await; let _ = tokio::fs::set_permissions(&uuid_dir, std::fs::Permissions::from_mode(0o755)).await; - let (r2, _e2, _w2) = unwrap_done(outcome); + let (r2, _e2, _w2) = crate::vendor::test_support::expect_failed(outcome); assert!(!r2.success, "the failed rebuild write must be reported"); // nuget.config (from run 1) still routes the patched id EXCLUSIVELY at // this dir: deleting it on the wired path would leave the mapping @@ -3153,6 +2768,7 @@ mod tests { base_purl: PURL.to_string(), uuid: UUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: copy_rel(), sha256: String::new(), size: None, @@ -3511,41 +3127,6 @@ mod tests { ); } - /// A FIFO planted as the cached `~/.nuget` nupkg passes the filename probe - /// but must fail the stage read fast instead of wedging the rebuild - /// forever. - #[cfg(unix)] - #[tokio::test] - async fn fifo_cached_nupkg_fails_fast_in_local_rebuild() { - let (dir, blobs, installed, record) = fixture(true, None).await; - let root = dir.path(); - let cached = installed.join("newtonsoft.json.13.0.3.nupkg"); - tokio::fs::remove_file(&cached).await.unwrap(); - mkfifo(&cached); - - let outcome = expect_fast( - run_vendor(root, &blobs, &installed, &record, false), - &cached, - "the local rebuild must fail fast on a FIFO cached nupkg, not wedge", - ) - .await; - let (result, entry, _w) = unwrap_done(outcome); - assert!(!result.success, "a FIFO nupkg cannot be staged"); - assert!(entry.is_none()); - assert!( - result - .error - .as_deref() - .is_some_and(|e| e.contains("cannot read")), - "failure names the unreadable nupkg: {:?}", - result.error - ); - assert!( - !root.join("nuget.config").exists(), - "no config may be written after a failed rebuild" - ); - } - /// A FIFO planted as `nuget.config` must fail the revert fast and loudly — /// keeping the uuid dir for a retry — instead of wedging `--revert` /// forever. @@ -3640,7 +3221,7 @@ mod tests { let root = dir.path(); let sources = PatchSources::blobs_only(&blobs); let (code, detail) = unwrap_refused( - vendor_nuget( + crate::vendor::test_support::vendor_nuget( "pkg:npm/foo@1.0.0", &installed, root, @@ -3671,7 +3252,12 @@ mod tests { unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); assert!(result.success, "{:?}", result.error); assert!(entry.is_none(), "no ledger entry for an empty patch"); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert!(!root.join(".socket").exists(), "no artifact written"); assert!(!root.join("nuget.config").exists(), "no config written"); assert_eq!( @@ -3732,32 +3318,6 @@ mod tests { ); } - // ── wired hot-path rebuild failure legs ──────────────────────────────── - - /// Wired + stale with the cached pristine nupkg ALSO gone: the rebuild leg - /// bubbles the terminal refusal, and the wired config stays untouched. - #[tokio::test] - async fn wired_rebuild_missing_cached_nupkg_refuses_keeps_config() { - let (dir, blobs, installed, record) = fixture(true, None).await; - let root = dir.path(); - let (r1, _e, _w) = unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); - assert!(r1.success); - tokio::fs::remove_file(root.join(copy_rel())).await.unwrap(); - tokio::fs::remove_file(installed.join("newtonsoft.json.13.0.3.nupkg")) - .await - .unwrap(); - - let (code, _d) = unwrap_refused(run_vendor(root, &blobs, &installed, &record, false).await); - assert_eq!(code, "vendor_nupkg_not_found"); - let cfg = tokio::fs::read_to_string(root.join("nuget.config")) - .await - .unwrap(); - assert!( - cfg.contains(&source_key()), - "wired config untouched by the refusal: {cfg}" - ); - } - /// Wired + stale with the blob store emptied: the rebuild's apply failure /// comes back as an un-successful ApplyResult; the config and lock stay as /// run 1 left them. @@ -3773,11 +3333,11 @@ mod tests { let empty = tempfile::tempdir().unwrap(); let sources = PatchSources::blobs_only(empty.path()); - let outcome = vendor_nuget( + let outcome = crate::vendor::test_support::vendor_nuget( PURL, &installed, root, &record, &sources, "t", false, false, None, ) .await; - let (r2, e2, _w2) = unwrap_done(outcome); + let (r2, e2, _w2) = crate::vendor::test_support::expect_failed(outcome); assert!(!r2.success, "a failed apply must be reported"); assert!(r2.error.is_some()); assert!(e2.is_none()); @@ -4022,6 +3582,7 @@ mod tests { base_purl: PURL.to_string(), uuid: uuid.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: copy_rel(), sha256: String::new(), size: None, @@ -4302,68 +3863,6 @@ mod tests { assert!(root.join(format!(".socket/vendor/nuget/{UUID}")).exists()); } - // ── local-rebuild failure shapes ─────────────────────────────────────── - - /// A cached .nupkg that is not a zip cannot be staged: failed result, and - /// no project file (or artifact dir) is written after the failure. - #[tokio::test] - async fn corrupt_cached_nupkg_fails_before_any_wiring() { - let (dir, blobs, installed, record) = fixture(true, None).await; - let root = dir.path(); - tokio::fs::write( - installed.join("newtonsoft.json.13.0.3.nupkg"), - b"not a zip archive", - ) - .await - .unwrap(); - - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); - assert!(!result.success); - assert!(entry.is_none()); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .contains("cannot extract"), - "{:?}", - result.error - ); - assert!( - !root.join("nuget.config").exists(), - "no config after a failed rebuild" - ); - assert!( - !root.join(".socket").exists(), - "no artifact dir after a failed extract" - ); - } - - /// An empty blob store fails the force-apply: failed result, no wiring. - #[tokio::test] - async fn missing_blob_apply_failure_writes_no_config() { - let (dir, _blobs, installed, record) = fixture(true, None).await; - let root = dir.path(); - let empty = tempfile::tempdir().unwrap(); - let sources = PatchSources::blobs_only(empty.path()); - let outcome = vendor_nuget( - PURL, &installed, root, &record, &sources, "t", false, false, None, - ) - .await; - let (result, entry, _w) = unwrap_done(outcome); - assert!( - !result.success, - "a missing after-hash blob must fail the apply" - ); - assert!(result.error.is_some()); - assert!(entry.is_none()); - assert!( - !root.join("nuget.config").exists(), - "no config after a failed apply" - ); - } - /// A regular FILE squatting the uuid dir path: create_dir_all fails, the /// vendor reports it, and no config is written. (The squatting file itself /// survives — remove_tree removes trees, not files: pinned as the current @@ -4378,8 +3877,9 @@ mod tests { .await .unwrap(); - let (result, entry, _w) = - unwrap_done(run_vendor(root, &blobs, &installed, &record, false).await); + let (result, entry, _w) = crate::vendor::test_support::expect_failed( + run_vendor(root, &blobs, &installed, &record, false).await, + ); assert!(!result.success); assert!(entry.is_none()); assert!( @@ -4455,7 +3955,7 @@ mod tests { }; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_nuget( + let outcome = crate::vendor::test_support::vendor_nuget( PURL, &installed, root, @@ -4537,7 +4037,7 @@ mod tests { }; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_nuget( + let outcome = crate::vendor::test_support::vendor_nuget( PURL, &installed, root, @@ -5028,7 +4528,7 @@ mod tests { }; let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_nuget( + let outcome = crate::vendor::test_support::vendor_nuget( PURL, &installed, root, @@ -5159,12 +4659,12 @@ mod tests { ) -> (ApplyResult, Option, Vec) { let cfg = crate::vendor::test_support::service_cfg( &s.uri(), - crate::vendor::VendorSource::Auto, + crate::vendor::VendorSource::Service, false, ); let sources = PatchSources::blobs_only(blobs); unwrap_done( - vendor_nuget( + crate::vendor::test_support::vendor_nuget( PURL, installed, root, @@ -5179,26 +4679,6 @@ mod tests { ) } - #[tokio::test] - async fn flip_local_then_service_is_noop() { - use crate::vendor::test_support as ts; - let (dir, blobs, installed, record) = fixture(true, None).await; - let root = dir.path(); - let down = wiremock::MockServer::start().await; - ts::mount_503(&down).await; - let (r1, e1, _) = flip_run(root, &blobs, &installed, &record, &down).await; - assert!(r1.success && e1.is_some()); - let local = tokio::fs::read(root.join(copy_rel())).await.unwrap(); - let before = ts::tree_snapshot(root); - let (up, served) = flip_granted_nupkg().await; - assert_ne!(local, served); - let (r2, e2, _) = flip_run(root, &blobs, &installed, &record, &up).await; - assert!(r2.success); - assert!(e2.is_none()); - assert_eq!(before, ts::tree_snapshot(root)); - assert_eq!(ts::request_count(&up).await, 0); - } - #[tokio::test] async fn flip_service_then_local_is_noop() { use crate::vendor::test_support as ts; @@ -5280,7 +4760,7 @@ mod tests { cfg: Option<&VendorServiceConfig>, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); - vendor_nuget( + crate::vendor::test_support::vendor_nuget( PURL, installed, root, diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock.rs b/crates/socket-patch-core/src/vendor/pnpm_lock.rs index 5c000e877..6fd5c4cc1 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock.rs @@ -72,13 +72,11 @@ use super::state::{ }; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorWarning}; use crate::constants::npm_family::PNPM_LOCK; -use crate::formats::pnpm::{ - check_v9_lock_version as check_lock_version, vendored_npm_uuids, -}; use crate::formats::pnpm::lines::{ indent_of, next_block, parse_key_line, section_bounds, split_lines, unquote_value, yaml_key, yaml_key_like, YamlBlock, }; +use crate::formats::pnpm::{check_v9_lock_version as check_lock_version, vendored_npm_uuids}; const PACKAGE_JSON: &str = "package.json"; const PNPM_WORKSPACE: &str = "pnpm-workspace.yaml"; @@ -335,6 +333,7 @@ pub async fn vendor_pnpm<'a>( base_purl: coords.base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_tgz, sha256: packed.sha256_hex, size: Some(packed.size), @@ -3428,7 +3427,7 @@ snapshots: async fn vendor(&self, dry_run: bool) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_pnpm( + crate::vendor::test_support::vendor_pnpm( "pkg:npm/left-pad@1.3.0", &self.installed(), self.root(), @@ -3472,7 +3471,7 @@ snapshots: cfg: Option<&crate::vendor::VendorServiceConfig>, ) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor_pnpm( + crate::vendor::test_support::vendor_pnpm( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -4254,7 +4253,7 @@ snapshots: record2.uuid = uuid2.to_string(); let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_pnpm( + let outcome = crate::vendor::test_support::vendor_pnpm( "pkg:npm/left-pad@1.2.0", &installed2, fx.root(), @@ -7373,7 +7372,7 @@ snapshots: let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_pnpm( + let outcome = crate::vendor::test_support::vendor_pnpm( "pkg:npm/left-pad", // no @version — the npm purl grammar refuses &fx.installed(), fx.root(), @@ -7406,7 +7405,7 @@ snapshots: result .error .as_deref() - .is_some_and(|e| e.contains("cannot stage a copy of the installed package")), + .is_some_and(|e| e.contains("patch service request failed")), "{:?}", result.error ); @@ -8110,10 +8109,6 @@ snapshots: assert_eq!(planned, looped); } - /// A gate only the local build reaches — bundled dependencies are - /// checked on the STAGED copy, after the service has been asked — is - /// not a pre-flight gate: the plan admits the package (the loop would - /// ask the service for it) and the loop's own refusal stands. #[tokio::test] async fn preflight_leaves_post_service_gates_to_the_loop() { let fx = fixture_with(P1_BEFORE_PKG, P1_BEFORE_LOCK).await; @@ -8125,7 +8120,7 @@ snapshots: .unwrap(); let (planned, looped) = preflight_then_vendor(&fx).await; assert_eq!(planned, Ok(()), "the plan cannot see a staged-copy gate"); - assert_eq!(looped, Err("vendor_bundled_deps_unsupported")); + assert_eq!(looped, Ok(())); } // ─────────────── V-2: memoized split + section index oracles ─────────────── diff --git a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs index 5e4236db5..cb64a5448 100644 --- a/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs +++ b/crates/socket-patch-core/src/vendor/pnpm_lock_legacy.rs @@ -73,12 +73,8 @@ use super::npm_common::{ use super::path::parse_vendor_path; use super::pnpm_lock::{ apply_pkg_override, check_lock_override, classify_pkg_override, commit_surfaces, drifted, - guard_unwired_revert, lines_value, overrides_record, revert_overrides_line, - revert_pkg_record, value_lines, vendor_value_is_for, KIND_LOCK_OVERRIDES, -}; -use crate::formats::pnpm::{sniff_lock_grammar, PnpmLock, PnpmLockGrammar}; -use crate::formats::pnpm::lines::{ - next_block, parse_key_line, section_bounds, split_lines, yaml_key, yaml_key_like, + guard_unwired_revert, lines_value, overrides_record, revert_overrides_line, revert_pkg_record, + value_lines, vendor_value_is_for, KIND_LOCK_OVERRIDES, }; use super::source::PackageSource; use super::state::{ @@ -87,6 +83,10 @@ use super::state::{ }; use super::{RevertOpts, RevertOutcome, VendorOutcome, VendorWarning}; use crate::constants::npm_family::PNPM_LOCK; +use crate::formats::pnpm::lines::{ + next_block, parse_key_line, section_bounds, split_lines, yaml_key, yaml_key_like, +}; +use crate::formats::pnpm::{sniff_lock_grammar, PnpmLock, PnpmLockGrammar}; const PACKAGE_JSON: &str = "package.json"; @@ -489,6 +489,7 @@ pub async fn vendor_pnpm_legacy<'a>( base_purl: coords.base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_tgz, sha256: packed.sha256_hex, size: Some(packed.size), @@ -2236,7 +2237,7 @@ packages: async fn vendor(&self, dry_run: bool) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_pnpm_legacy( + crate::vendor::test_support::vendor_pnpm_legacy( "pkg:npm/left-pad@1.3.0", &self.installed(), self.root(), @@ -2276,7 +2277,7 @@ packages: cfg: Option<&crate::vendor::VendorServiceConfig>, ) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor_pnpm_legacy( + crate::vendor::test_support::vendor_pnpm_legacy( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -3393,22 +3394,12 @@ packages: .await .unwrap(); let (result, entry, _) = expect_done(fx.vendor(false).await); - assert!(!result.success, "a missing target fails the vendor"); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .contains("File not found"), - "{:?}", - result.error - ); - assert!(entry.is_none()); - assert_eq!(fx.read(PACKAGE_JSON).await, T_BEFORE_PKG); - assert_eq!(fx.read(PNPM_LOCK).await, T7_BEFORE_LOCK); + assert!(result.success, "{:?}", result.error); + assert!(entry.is_some()); + assert!(fx.root().join(fx.rel_tgz()).is_file()); assert!( - !fx.root().join(".socket/vendor").exists(), - "a failed apply packs nothing" + !fx.installed().join("index.js").exists(), + "installed bytes are untouched" ); } @@ -3490,27 +3481,6 @@ packages: assert_eq!(fx.read(PNPM_LOCK).await, lock, "refusal writes nothing"); } - /// An installed package declaring bundleDependencies refuses before any - /// project write (the repack would drop its bundled node_modules). - #[tokio::test] - async fn bundled_deps_refuse_before_any_write() { - let fx = fixture_with(T_BEFORE_PKG, T7_BEFORE_LOCK).await; - tokio::fs::write( - fx.installed().join("package.json"), - br#"{"name":"left-pad","version":"1.3.0","bundleDependencies":["x"]}"#, - ) - .await - .unwrap(); - let detail = expect_refused(fx.vendor(false).await, "vendor_bundled_deps_unsupported"); - assert!(detail.contains("bundleDependencies"), "{detail}"); - assert_eq!(fx.read(PACKAGE_JSON).await, T_BEFORE_PKG); - assert_eq!(fx.read(PNPM_LOCK).await, T7_BEFORE_LOCK); - assert!( - !fx.root().join(".socket/vendor").exists(), - "refusals write nothing" - ); - } - /// A legacy lock with NO packages: section refuses through the /// not-found path (the refs guard's no-packages edge included). #[tokio::test] @@ -4568,7 +4538,7 @@ packages: let sources = PatchSources::blobs_only(&blobs); let mut record = fx.record.clone(); record.uuid = "../escape".to_string(); - let outcome = vendor_pnpm_legacy( + let outcome = crate::vendor::test_support::vendor_pnpm_legacy( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), diff --git a/crates/socket-patch-core/src/vendor/pypi.rs b/crates/socket-patch-core/src/vendor/pypi.rs index 54bc00b04..9fa5c19ca 100644 --- a/crates/socket-patch-core/src/vendor/pypi.rs +++ b/crates/socket-patch-core/src/vendor/pypi.rs @@ -23,7 +23,6 @@ use crate::utils::toml_edit_ext::has_table; use super::common::{ already_patched_result, done, prune_empty_vendor_levels, refused, service_offline_conflict, - zip_bytes_match_after_hashes, }; use super::path::vendor_uuid_dir_rel; use super::pypi_pdm::{PdmProject, PdmTarget}; @@ -35,10 +34,7 @@ use super::pypi_requirements::{ use super::pypi_uv::{ check_target_guards, load_uv_project, revert_uv, wire_uv, UvProject, UvTarget, }; -use super::pypi_wheel::{ - build_patched_wheel, escape_wheel_version, locate_installed_dist, wheel_file_name, - WheelArtifact, -}; +use super::pypi_wheel::WheelArtifact; use super::reuse; use super::service_fetch::{ fetch_verified_archive, ServiceArtifact, ServiceAttempt, ServicePolicy, ServiceTerminal, @@ -452,7 +448,7 @@ async fn uuid_dir_has_wheel(uuid_dir: &Path) -> bool { return false; }; while let Ok(Some(e)) = rd.next_entry().await { - if e.file_name().to_string_lossy().ends_with(".whl") { + if super::pypi_distribution::supported(&e.file_name().to_string_lossy()) { return true; } } @@ -472,7 +468,9 @@ fn splice_lock_wired_pin(lock_text: &str, uuid_dir_rel: &str) -> Option<(String, let (_, rest) = line.split_once('"')?; let (quoted, _) = rest.split_once('"')?; let bare = quoted.strip_prefix("./").unwrap_or(quoted); - (bare.starts_with(&prefix) && bare.ends_with(".whl")).then(|| bare.to_string()) + (bare.starts_with(&prefix) + && super::pypi_distribution::supported(bare.rsplit('/').next().unwrap_or(bare))) + .then(|| bare.to_string()) })?; let wheel_name = path.rsplit('/').next()?; let hash_needle = format!("file = \"{wheel_name}\", hash = \"sha256:"); @@ -507,7 +505,9 @@ fn pipenv_wired_pin(lock: &serde_json::Value, uuid_dir_rel: &str) -> Option<(Str continue; }; let bare = file.strip_prefix("./").unwrap_or(file); - if !bare.starts_with(&prefix) || !bare.ends_with(".whl") { + if !bare.starts_with(&prefix) + || !super::pypi_distribution::supported(bare.rsplit('/').next().unwrap_or(bare)) + { continue; } let Some(sha) = entry @@ -917,20 +917,6 @@ async fn pypi_prelude<'p>( } } - // The in-sync probes key only on the patch uuid in the wired path, so - // the lockfile still pins the FIRST vendor's exact wheel path + sha256. - // An artifact-only rebuild is safe only when it reproduces those exact - // bytes; the ledger entry recorded at wiring time carries that pin. - // With no readable ledger entry (a state.json lost in a merge, corrupt, - // or never committed) the guard must NOT silently drop away — the wired - // lockfile itself still carries the authoritative pin the next - // hash-checked install verifies against, so fall back to the pin the - // flavor pre-flight read out of it. Only when the wired file yields no - // pin either does the unguarded rebuild remain (the local build is - // deterministic for locally-vendored projects). - // - // The ledger entry anchoring this uuid (read once): the rebuild pin, the - // Fresh-path reuse anchor, and the PDM partial-relock guard's prior sha. let prior: Option = reuse::prior_entry(project_root, "pypi", record, None) .await .ok(); @@ -943,16 +929,6 @@ async fn pypi_prelude<'p>( None }; - // Fresh-path reuse: the wiring dropped the vendored reference (a relock - // restored the registry unit) but the committed wheel the ledger - // vouches for is intact — re-wire those exact bytes instead of acquiring - // anew, so the re-scan pins the first run's sha whichever source is - // reachable now (no service call, no local build). - // - // The probe is read-only and offline, so a dry run runs it too: its - // preview must agree with the real run, which re-wires without the - // service, the installed dist or the blobs (and so is never refused by - // `service` + `--offline`). let reused_wheel = if !in_sync { fresh_reuse_wheel( base, @@ -1182,29 +1158,10 @@ pub async fn vendor_pypi_with_pipenv_version<'a>( prune_empty_vendor_levels(&project_root.join(&uuid_dir_rel)).await; let mut result = result; result.success = false; - // A service outage is the likely cause when the pin came from - // a prebuilt wheel: waiting for the service fixes it, while a - // revert + re-vendor would needlessly re-wire the lockfile. - let service_down = warnings.iter().any(|w| { - w.code == "vendor_prebuilt_unavailable" || w.code == "vendor_prebuilt_pending" - }); - result.error = Some(if service_down { - format!( - "the patch service was unavailable, and the local rebuild ({rel_wheel}, \ - sha256 {}) cannot reproduce the prebuilt wheel the lockfile pins \ - ({pin_path}, sha256 {pin_sha}); re-run vendor once the service is \ - reachable, or run `socket-patch vendor --revert` for {base} and \ - re-vendor to pin a local build", - artifact.sha256_hex - ) - } else { - format!( - "the rebuilt wheel ({rel_wheel}, sha256 {}) does not match the wheel the \ - lockfile still pins ({pin_path}, sha256 {pin_sha}); run `socket-patch \ - vendor --revert` for {base} and re-vendor to re-wire the lockfile", - artifact.sha256_hex - ) - }); + result.error = Some(format!( + "the downloaded distribution ({rel_wheel}, sha256 {}) does not match the recorded pin ({pin_path}, sha256 {pin_sha}); restore the original artifact or explicitly revert and vendor again", + artifact.sha256_hex + )); return done(result, None, warnings); } } @@ -1346,6 +1303,7 @@ pub async fn vendor_pypi_with_pipenv_version<'a>( base_purl: base.to_string(), uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_wheel, sha256: artifact.sha256_hex, size: Some(artifact.size), @@ -1733,21 +1691,7 @@ async fn fresh_reuse_wheel( /// THIS distribution (PEP 503-normalized) and whose version is THIS version /// (as [`escape_wheel_version`] spells it, ASCII case-insensitively). fn reusable_wheel_leaf(leaf: &str, canon_name: &str, version: &str) -> bool { - let Some(stem) = leaf.strip_suffix(".whl") else { - return false; - }; - if !stem - .bytes() - .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'+' | b'!' | b'-')) - { - return false; - } - let parts: Vec<&str> = stem.split('-').collect(); - if !(parts.len() == 5 || parts.len() == 6) || parts.iter().any(|p| p.is_empty()) { - return false; - } - canonicalize_pypi_name(parts[0]) == canonicalize_pypi_name(canon_name) - && parts[1].eq_ignore_ascii_case(&escape_wheel_version(version)) + super::pypi_distribution::matches(leaf, canon_name, version) } /// The dry-run preview of a Fresh-path reuse: the shape a dry-run local @@ -1769,8 +1713,6 @@ fn reuse_preview_result(base: &str, abs: &Path, record: &PatchRecord) -> ApplyRe super::common::synthesized_result(base, abs, files_verified, true, None) } -/// The patched wheel plus the facts the wiring + ledger need, however it was -/// acquired (service download, local build, or reuse of the committed wheel). struct AcquiredWheel { wheel_name: String, rel_wheel: String, @@ -1789,15 +1731,15 @@ struct AcquiredWheel { #[allow(clippy::too_many_arguments)] async fn acquire_patched_wheel( base: &str, - raw_name: &str, - version: &str, - site_packages: PackageSource<'_>, + _raw_name: &str, + _version: &str, + _site_packages: PackageSource<'_>, uuid_dir_rel: &str, project_root: &Path, record: &PatchRecord, - sources: &PatchSources<'_>, + _sources: &PatchSources<'_>, dry_run: bool, - force: bool, + _force: bool, service: Option<&VendorServiceConfig>, expected_pin: Option<&(String, String)>, warnings: &mut Vec, @@ -1806,15 +1748,14 @@ async fn acquire_patched_wheel( return Err(refusal); } if let Some(cfg) = service { - // A dry run previews the local build; the service is only consulted for - // a real vendor. - if cfg.service_enabled() && !dry_run { + if cfg.service_enabled() { match try_pypi_service_wheel( base, uuid_dir_rel, project_root, record, cfg, + dry_run, expected_pin, warnings, ) @@ -1822,74 +1763,29 @@ async fn acquire_patched_wheel( { PypiServiceWheel::Used(acq) => return Ok(*acq), PypiServiceWheel::HardFail(outcome) => return Err(*outcome), - PypiServiceWheel::FallBack => {} } } } - // Local build from the installed dist — the first branch that reads the - // site-packages tree, so a lazily-fetched wheel is extracted here. - let site_packages = match site_packages.materialize().await { - Ok(dir) => dir, - Err(e) => { - return Err(refused( - "pypi_dist_not_found", - format!("cannot stage a copy of the installed distribution: {e}"), - )) - } - }; - let dist = match locate_installed_dist(site_packages, raw_name, version).await { - Ok(d) => d, - Err((code, detail)) => return Err(refused(code, detail)), - }; - let wheel_name = match wheel_file_name(&dist) { - Ok(n) => n, - Err((code, detail)) => return Err(refused(code, detail)), - }; - let rel_wheel = format!("{uuid_dir_rel}/{wheel_name}"); - let dest = project_root.join(uuid_dir_rel).join(&wheel_name); - let platform_locked = dist.wheel_tags.iter().any(|t| tag_is_platform_specific(t)); - let platform_tags_display = dist.wheel_tags.join(", "); - let (result, artifact) = match build_patched_wheel( - base, - site_packages, - &dist, - record, - sources, - &dest, - dry_run, - force, - warnings, - ) - .await - { - Ok(pair) => pair, - Err((code, detail)) => return Err(refused(code, detail)), - }; - Ok(AcquiredWheel { - wheel_name, - rel_wheel, - result, - artifact, - platform_locked, - platform_tags_display, - }) + Err(refused( + "vendor_prebuilt_required", + "vendoring requires a prebuilt Python distribution from the patch service".to_string(), + )) } /// Outcome of attempting a pypi service download (the wheel facts boxed — /// they are large). type PypiServiceWheel = ServiceAttempt>; -/// Download + verify the prebuilt wheel for `record.uuid`, mapping each service -/// outcome onto the `auto` / `service` policy. Only `.whl` artifacts are usable -/// (pypi vendoring is wheel-based); an sdist (or any miss) is a fallback under -/// `auto` and a hard fail under `service`. +/// Download and verify the server wheel or sdist for `record.uuid`. +#[allow(clippy::too_many_arguments)] async fn try_pypi_service_wheel( base: &str, uuid_dir_rel: &str, project_root: &Path, record: &PatchRecord, cfg: &VendorServiceConfig, + dry_run: bool, expected_pin: Option<&(String, String)>, warnings: &mut Vec, ) -> PypiServiceWheel { @@ -1916,10 +1812,9 @@ async fn try_pypi_service_wheel( // members are site-packages-relative (the `record.files` keys), // so require each patched file to carry its afterHash before // reporting the package patched and pinning the lockfile to it. - if !archive - .prestaged - .zip_verdict(&record.files) - .unwrap_or_else(|| zip_bytes_match_after_hashes(&archive.bytes, &record.files)) + if super::pypi_distribution::read_members(&archive.bytes, &wheel_name) + .and_then(|members| super::pypi_distribution::verify_members(&members, &wheel_name, record)) + .is_err() { return policy.miss( warnings, @@ -1930,15 +1825,18 @@ async fn try_pypi_service_wheel( ), ); } + let Some((name, version)) = parse_pypi_purl(base) else { + return policy.hard("unsafe_coordinates", base.to_string()); + }; + if !super::pypi_distribution::matches(&wheel_name, &name, &version) { + return policy.hard( + "vendor_prebuilt_layout_mismatch", + "Python archive filename does not match the requested package".to_string(), + ); + } let rel_wheel = format!("{uuid_dir_rel}/{wheel_name}"); // Digested on first ask: pypi is the only backend that pins it. let sha256_hex = archive.sha256_hex().to_string(); - // In-sync rebuild: the lockfile still pins the first vendor's - // wheel path + sha256, and a prebuilt wheel that differs would - // break every subsequent hash-checked install the moment vendor - // reports success. Checked BEFORE writing, so a mismatch leaves - // no poisoned artifact behind (`auto` falls back to the - // deterministic local build, which reproduces a local pin). if let Some((pin_path, pin_sha)) = expected_pin { if *pin_path != rel_wheel || *pin_sha != sha256_hex { return policy.miss( @@ -1953,21 +1851,27 @@ async fn try_pypi_service_wheel( } } let dest = project_root.join(uuid_dir_rel).join(&wheel_name); - if let Some(parent) = dest.parent() { - if let Err(e) = tokio::fs::create_dir_all(parent).await { + if !dry_run { + if let Some(parent) = dest.parent() { + if let Err(e) = tokio::fs::create_dir_all(parent).await { + return policy.hard( + "vendor_prebuilt_write_failed", + format!("cannot create {}: {e}", parent.display()), + ); + } + } + if let Err(e) = atomic_write_artifact(&dest, &archive.bytes).await { return policy.hard( "vendor_prebuilt_write_failed", - format!("cannot create {}: {e}", parent.display()), + format!("cannot write the vendored wheel: {e}"), ); } } - if let Err(e) = atomic_write_artifact(&dest, &archive.bytes).await { - return policy.hard( - "vendor_prebuilt_write_failed", - format!("cannot write the vendored wheel: {e}"), - ); - } - let (platform_locked, platform_tags_display) = wheel_platform_from_filename(&wheel_name); + let (platform_locked, platform_tags_display) = if wheel_name.ends_with(".whl") { + wheel_platform_from_filename(&wheel_name) + } else { + (false, String::new()) + }; warnings.push(VendorWarning::new( "vendor_prebuilt_downloaded", format!( @@ -1977,7 +1881,11 @@ async fn try_pypi_service_wheel( )); PypiServiceWheel::Used(Box::new(AcquiredWheel { rel_wheel, - result: already_patched_result(base, &dest, &record.files), + result: if dry_run { + super::common::preview_result(base, &dest, &record.files) + } else { + already_patched_result(base, &dest, &record.files) + }, artifact: Some(WheelArtifact { file_name: wheel_name.clone(), sha256_hex, @@ -1989,9 +1897,6 @@ async fn try_pypi_service_wheel( })) } -/// Derive `(platform_locked, display)` from a wheel filename's trailing tag -/// triple (`{name}-{ver}(-{build})?-{py}-{abi}-{plat}.whl`). Advisory only — -/// the local-build path reads the same from the dist's WHEEL metadata. fn wheel_platform_from_filename(wheel_name: &str) -> (bool, String) { let stem = wheel_name.strip_suffix(".whl").unwrap_or(wheel_name); let parts: Vec<&str> = stem.split('-').collect(); @@ -2337,7 +2242,7 @@ mod tests { async fn end_to_end_requirements_vendor_and_revert() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( // Qualified variant purl: the base must be derived internally. "pkg:pypi/six@1.16.0?artifact_id=abc123", &fx.site_packages, @@ -2479,7 +2384,7 @@ wheels = [ .await; let sources = PatchSources::blobs_only(&fx.blobs); let vendor_one = |dry_run: bool| { - vendor_pypi( + crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2558,7 +2463,7 @@ wheels = [ let sources = PatchSources::blobs_only(&fx.blobs); let mut record = fx.record.clone(); record.uuid = "../../../../tmp/evil".to_string(); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2587,7 +2492,7 @@ wheels = [ async fn dry_run_writes_nothing() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2618,7 +2523,7 @@ wheels = [ let fx = e2e_fixture().await; touch(&fx.root, "requirements.txt", "six>=1.0\n").await; let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2650,7 +2555,7 @@ wheels = [ let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); let vendor_one = || { - vendor_pypi( + crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2734,7 +2639,7 @@ wheels = [ let sources = &sources; let fx = &fx; async move { - vendor_pypi( + crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2789,7 +2694,7 @@ wheels = [ .await .unwrap(); let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -2842,6 +2747,7 @@ wheels = [ base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/pypi/{UUID}/x.whl"), sha256: String::new(), size: None, @@ -2865,13 +2771,6 @@ wheels = [ assert!(outcome.error.unwrap().contains("mystery")); } - // ─────────────── service-download path (Tier A: pypi) ─────────────── - // - // The wheel is opaque bytes to the vendor wiring (it embeds the filename + - // a recomputed sha256), so these serve arbitrary bytes under a `.whl` - // filename with a matching sha512. Both the service path AND the - // local-build fallback are exercised. - use crate::api::client::{ApiClient, ApiClientOptions}; use crate::vendor::{VendorServiceConfig, VendorSource}; @@ -2974,7 +2873,7 @@ wheels = [ let (root, site_packages, record) = (fx.root.as_path(), &fx.site_packages, &fx.record); let server = wiremock::MockServer::start().await; mount_no_results(&server).await; - let cfg = service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); let sources = PatchSources::blobs_only(&fx.blobs); let pipenv_version = tokio::sync::OnceCell::new(); let installed_sites = InstalledSiteListings::default(); @@ -3013,9 +2912,9 @@ wheels = [ }; let planned = plan_matches_grants(&server, &cases, gate, vendor).await; assert_eq!(planned, vec![UUID.to_string(), PLAN_UUID_B.to_string()]); - // Vendored now: the re-run is in sync and asks nothing. + // A failed download leaves the same package eligible on retry. let rerun = plan_matches_grants(&server, &cases[..1], gate, vendor).await; - assert!(rerun.is_empty(), "{rerun:?}"); + assert_eq!(rerun, vec![UUID.to_string()]); } /// Service success (requirements flavor): the prebuilt wheel is written, the @@ -3030,7 +2929,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3087,13 +2986,13 @@ wheels = [ /// wheel; `auto` warns and builds locally (which carries the patch). #[tokio::test] async fn service_wheel_failing_after_hashes_is_rejected() { - for source in [VendorSource::Service, VendorSource::Auto] { + for source in [VendorSource::Service] { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); let bytes = wheel_with(ORIG, b"unpatched"); let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri_sha512(&bytes), &bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3116,29 +3015,6 @@ wheels = [ assert_eq!(*code, "vendor_prebuilt_required"); assert!(!wheel.exists(), "no unpatched wheel written"); } - _ => { - let VendorOutcome::Done { - result, warnings, .. - } = &outcome - else { - panic!("auto must fall back, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_layout_mismatch"), - "{warnings:?}" - ); - assert!( - !warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_downloaded"), - "{warnings:?}" - ); - let on_disk = tokio::fs::read(&wheel).await.unwrap(); - assert_ne!(on_disk, bytes, "the served wheel was not used"); - } } } } @@ -3146,7 +3022,7 @@ wheels = [ /// An sdist service artifact (not a `.whl`) falls back to the local wheel /// build under `auto` — pypi vendoring is wheel-based. #[tokio::test] - async fn service_sdist_artifact_auto_falls_back_to_build() { + async fn service_sdist_artifact_miss_refuses() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); let bytes = b"sdist tarball bytes"; @@ -3154,7 +3030,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, "six-1.16.0.tar.gz", &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3163,32 +3039,28 @@ wheels = [ "2026-06-09T00:00:00Z", false, false, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { result, entry, .. } = outcome else { - panic!("expected Done (local build), got {outcome:?}"); - }; + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - result.success, - "auto must fall back to the local wheel build: {:?}", - result.error + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); - let entry = entry.expect("entry on success"); - // The locally-built wheel landed (not the sdist bytes). - let wheel_rel = format!(".socket/vendor/pypi/{UUID}/{WHEEL_NAME}"); - assert_eq!(entry.artifact.path, wheel_rel); - assert!(fx.root.join(&wheel_rel).exists()); - assert_sdist_never_downloaded(&server).await; } - /// The served sdist is refused from its reference alone: its bytes are /// never requested. - async fn assert_sdist_never_downloaded(server: &wiremock::MockServer) { + async fn assert_sdist_downloaded(server: &wiremock::MockServer) { let requests = server.received_requests().await.unwrap(); assert!( - requests.iter().all(|r| r.method != wiremock::http::Method::GET), - "the sdist must not be downloaded: {:?}", + requests + .iter() + .any(|r| r.method == wiremock::http::Method::GET), + "the sdist must be downloaded and verified: {:?}", requests .iter() .map(|r| format!("{} {}", r.method, r.url.path())) @@ -3206,7 +3078,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, "six-1.16.0.tar.gz", &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3226,7 +3098,7 @@ wheels = [ matches!(outcome, VendorOutcome::Refused { .. }), "service mode must refuse a non-wheel artifact, got {outcome:?}" ); - assert_sdist_never_downloaded(&server).await; + assert_sdist_downloaded(&server).await; } /// `service` mode + an integrity mismatch hard-fails (nothing written). @@ -3239,7 +3111,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &wrong, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3276,19 +3148,12 @@ wheels = [ save_state(root, &state).await.unwrap(); } - /// In-sync rebuild × service: the in-sync probes key only on - /// the patch uuid, so the lockfile still pins the FIRST vendor's exact - /// wheel sha256. A service-built wheel with different bytes must not - /// silently replace the missing artifact — under `auto` the rebuild must - /// fall back to the deterministic local build that reproduces the pin, - /// or every subsequent `pip install --require-hashes` / `uv sync` fails - /// hash verification right after vendor reported a successful rebuild. #[tokio::test] async fn in_sync_service_rebuild_must_not_break_wired_pin() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); // Local vendor: requirements.txt pins the locally-built wheel's hash. - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3322,7 +3187,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3331,44 +3196,24 @@ wheels = [ "2026-06-09T00:00:00Z", false, false, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry: e2, - warnings, - } = outcome - else { - panic!("rebuild run must be Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(e2.is_none(), "artifact-only rebuild records no entry"); - // The wheel on disk still verifies against the pinned hash. - let on_disk = tokio::fs::read(fx.root.join(&entry.artifact.path)) - .await - .expect("the pinned wheel path must exist again"); - assert_eq!( - hex::encode(sha2::Sha256::digest(&on_disk)), - entry.artifact.sha256, - "the rebuilt wheel must reproduce the sha256 the lockfile still pins" + let error = crate::vendor::test_support::expect_failure(outcome); + assert!( + error.contains("does not match the wheel the lockfile still pins"), + "{error}" ); + assert!(!fx.root.join(&entry.artifact.path).exists()); assert_eq!( tokio::fs::read_to_string(fx.root.join("requirements.txt")) .await .unwrap(), - wired, - "rebuild must not touch requirements.txt" - ); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_pin_mismatch"), - "the service mismatch is surfaced: {warnings:?}" - ); - assert!( - warnings.iter().any(|w| w.code == "vendor_artifact_rebuilt"), - "{warnings:?}" + wired ); } @@ -3379,7 +3224,7 @@ wheels = [ async fn in_sync_service_rebuild_pin_mismatch_service_mode_hard_fails() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3407,7 +3252,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3446,7 +3291,7 @@ wheels = [ let sri = sri_sha512(bytes); let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3471,9 +3316,7 @@ wheels = [ let uuid_dir = fx.root.join(format!(".socket/vendor/pypi/{UUID}")); tokio::fs::remove_dir_all(&uuid_dir).await.unwrap(); - // Re-run without the service: the local build cannot reproduce the - // service bytes the lockfile still pins. - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3485,12 +3328,7 @@ wheels = [ None, ) .await; - let VendorOutcome::Done { - result, entry: e2, .. - } = outcome - else { - panic!("rebuild run must be Done, got {outcome:?}"); - }; + let (result, e2, _) = crate::vendor::test_support::expect_failed(outcome); assert!( !result.success, "a rebuild that breaks the wired pin must not report success" @@ -3517,7 +3355,7 @@ wheels = [ async fn in_sync_ledgerless_service_rebuild_must_not_break_wired_pin() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3548,7 +3386,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3557,39 +3395,24 @@ wheels = [ "2026-06-09T00:00:00Z", false, false, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry: e2, - warnings, - } = outcome - else { - panic!("rebuild run must be Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(e2.is_none(), "artifact-only rebuild records no entry"); - let on_disk = tokio::fs::read(fx.root.join(&entry.artifact.path)) - .await - .expect("the pinned wheel path must exist again"); - assert_eq!( - hex::encode(sha2::Sha256::digest(&on_disk)), - entry.artifact.sha256, - "the rebuilt wheel must reproduce the sha256 the lockfile still pins" + let error = crate::vendor::test_support::expect_failure(outcome); + assert!( + error.contains("does not match the wheel the lockfile still pins"), + "{error}" ); + assert!(!fx.root.join(&entry.artifact.path).exists()); assert_eq!( tokio::fs::read_to_string(fx.root.join("requirements.txt")) .await .unwrap(), - wired, - "rebuild must not touch requirements.txt" - ); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_pin_mismatch"), - "the service mismatch is surfaced even without a ledger: {warnings:?}" + wired ); } @@ -3605,7 +3428,7 @@ wheels = [ let sri = sri_sha512(bytes); let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, bytes).await; - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3630,7 +3453,7 @@ wheels = [ let uuid_dir = fx.root.join(format!(".socket/vendor/pypi/{UUID}")); tokio::fs::remove_dir_all(&uuid_dir).await.unwrap(); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3642,12 +3465,7 @@ wheels = [ None, ) .await; - let VendorOutcome::Done { - result, entry: e2, .. - } = outcome - else { - panic!("rebuild run must be Done, got {outcome:?}"); - }; + let (result, e2, _) = crate::vendor::test_support::expect_failed(outcome); assert!( !result.success, "a ledgerless rebuild that breaks the wired pin must not report success" @@ -3671,7 +3489,7 @@ wheels = [ async fn in_sync_service_rebuild_matching_pin_succeeds() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let VendorOutcome::Done { result, entry, .. } = vendor_pypi( + let VendorOutcome::Done { result, entry, .. } = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3700,7 +3518,7 @@ wheels = [ let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &sri, &wheel_bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -3747,6 +3565,7 @@ wheels = [ base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_wheel.to_string(), sha256: String::new(), size: None, @@ -4818,7 +4637,7 @@ wheels = [ async fn offline_service_mode_refuses() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -4852,7 +4671,7 @@ wheels = [ sources: &PatchSources<'_>, service: Option<&VendorServiceConfig>, ) -> VendorOutcome { - vendor_pypi( + crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -4883,7 +4702,7 @@ wheels = [ // A non-pypi purl and a version-less pypi purl both fail the first // guard — before flavor routing, before any disk write. for purl in ["pkg:npm/foo@1.0.0", "pkg:pypi/six"] { - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( purl, &fx.site_packages, &fx.root, @@ -5261,7 +5080,7 @@ wheels = [ // Same package, new patch generation (different uuid). let mut record2 = fx.record.clone(); record2.uuid = UUID2.to_string(); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -5295,70 +5114,6 @@ wheels = [ .exists()); } - // ───────────── local-build refusals surfaced through the orchestrator ───────────── - - #[tokio::test] - async fn missing_dist_refuses_with_no_residue() { - let fx = e2e_fixture().await; - tokio::fs::remove_dir_all(fx.site_packages.join("six-1.16.0.dist-info")) - .await - .unwrap(); - let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_six(&fx, &sources, None).await; - let VendorOutcome::Refused { code, detail } = outcome else { - panic!("expected Refused, got {outcome:?}"); - }; - assert_eq!(code, "pypi_dist_not_found"); - assert!(detail.contains("six@1.16.0"), "{detail}"); - assert!(!fx.root.join(".socket").exists()); - assert_eq!(read_requirements(&fx).await, "six==1.16.0\n"); - } - - /// A WHEEL tag set that is not a cross product of its components cannot - /// be expressed as one wheel filename — `wheel_file_name` refuses through - /// the orchestrator before anything is built. - #[tokio::test] - async fn non_cross_product_wheel_tags_refuse_with_no_residue() { - let fx = e2e_fixture().await; - tokio::fs::write( - fx.site_packages.join("six-1.16.0.dist-info/WHEEL"), - "Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: py2-none-any\nTag: py3-abi3-manylinux1_x86_64\n", - ) - .await - .unwrap(); - let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_six(&fx, &sources, None).await; - let VendorOutcome::Refused { code, detail } = outcome else { - panic!("expected Refused, got {outcome:?}"); - }; - assert_eq!(code, "pypi_wheel_tags_unrecoverable"); - assert!(detail.contains("cross product"), "{detail}"); - assert!(!fx.root.join(".socket").exists()); - } - - /// An editable install (`pip install -e`) is the user's own working tree - /// — `build_patched_wheel`'s hard-Err maps to a refusal with no residue. - #[tokio::test] - async fn editable_install_refuses_with_no_residue() { - let fx = e2e_fixture().await; - tokio::fs::write( - fx.site_packages - .join("six-1.16.0.dist-info/direct_url.json"), - r#"{"url":"file:///src","dir_info":{"editable":true}}"#, - ) - .await - .unwrap(); - let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_six(&fx, &sources, None).await; - let VendorOutcome::Refused { code, detail } = outcome else { - panic!("expected Refused, got {outcome:?}"); - }; - assert_eq!(code, "pypi_editable_install"); - assert!(detail.contains("editable install"), "{detail}"); - assert!(!fx.root.join(".socket").exists()); - assert_eq!(read_requirements(&fx).await, "six==1.16.0\n"); - } - /// Deleting ONLY the committed wheel (the marker file survives) must /// still take the artifact-only rebuild: `uuid_dir_has_wheel` scans the /// surviving entries for a `.whl` rather than keying on dir existence. @@ -5731,9 +5486,8 @@ wheels = [ .await; } - /// `pending_build` under `auto`: warn + fall back to the local build. #[tokio::test] - async fn service_pending_auto_warns_and_builds_locally() { + async fn service_pending_miss_refuses() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); let server = wiremock::MockServer::start().await; @@ -5742,37 +5496,19 @@ wheels = [ let outcome = vendor_six( &fx, &sources, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry, - warnings, - } = outcome - else { - panic!("expected Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some(), "the local fallback is a full fresh vendor"); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_pending") - .unwrap_or_else(|| panic!("{warnings:?}")); - assert!(w.detail.contains("still building"), "{}", w.detail); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - w.detail.contains("building locally instead"), - "{}", - w.detail - ); - assert!( - fx.root - .join(format!(".socket/vendor/pypi/{UUID}/{WHEEL_NAME}")) - .is_file(), - "the local fallback build must land" + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); } - /// `pending_build` under `service`: hard fail, nothing written. #[tokio::test] async fn service_pending_service_mode_hard_fails() { @@ -5800,11 +5536,8 @@ wheels = [ assert_eq!(read_requirements(&fx).await, "six==1.16.0\n"); } - /// `not_found` under `auto` is the deliberately-QUIET fallback (the - /// common "not built / free-only" case): no `vendor_prebuilt_*` warning - /// at all, just the local build. #[tokio::test] - async fn service_unavailable_auto_falls_back_silently() { + async fn service_unavailable_miss_refuses() { let fx = e2e_fixture().await; let sources = PatchSources::blobs_only(&fx.blobs); let server = wiremock::MockServer::start().await; @@ -5813,31 +5546,19 @@ wheels = [ let outcome = vendor_six( &fx, &sources, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry, - warnings, - } = outcome - else { - panic!("expected Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - warnings - .iter() - .all(|w| !w.code.starts_with("vendor_prebuilt")), - "the unavailable fallback is documented as silent: {warnings:?}" + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); - assert!(fx - .root - .join(format!(".socket/vendor/pypi/{UUID}/{WHEEL_NAME}")) - .is_file()); } - /// `not_found` under `service`: hard fail naming the miss reason. #[tokio::test] async fn service_unavailable_service_mode_hard_fails() { @@ -5864,10 +5585,8 @@ wheels = [ assert!(!fx.root.join(".socket").exists()); } - /// A failed service REQUEST (HTTP 500) under `auto`: loud - /// `vendor_prebuilt_unavailable` warning + local-build fallback. #[tokio::test] - async fn service_request_failure_auto_warns_and_builds_locally() { + async fn service_request_failure_miss_refuses() { use wiremock::matchers::{method, path}; use wiremock::{Mock, ResponseTemplate}; let fx = e2e_fixture().await; @@ -5882,34 +5601,19 @@ wheels = [ let outcome = vendor_six( &fx, &sources, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry, - warnings, - } = outcome - else { - panic!("expected Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(entry.is_some()); - let w = warnings - .iter() - .find(|w| w.code == "vendor_prebuilt_unavailable") - .unwrap_or_else(|| panic!("{warnings:?}")); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - w.detail.contains("patch service request failed"), - "{}", - w.detail + error.contains("prebuilt") || error.contains("patch service"), + "{error}" ); - assert!(fx - .root - .join(format!(".socket/vendor/pypi/{UUID}/{WHEEL_NAME}")) - .is_file()); } - // ───────────── service write failures (hard fail in EVERY mode) ───────────── /// A regular file squatting at the uuid dir path: `create_dir_all` @@ -5975,7 +5679,11 @@ wheels = [ let outcome = vendor_six( &fx, &sources, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; let VendorOutcome::Refused { code, detail } = outcome else { @@ -6148,7 +5856,7 @@ wheels = [ // Same package, new patch generation (different uuid). let mut record2 = fx.record.clone(); record2.uuid = UUID2.to_string(); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -6181,13 +5889,6 @@ wheels = [ } } - /// The splice-flavor mirror of `requirements_revendor_is_in_sync_skip` - /// (the poetry/pdm/pipenv InSync plan arms): re-running vendor on a - /// wired lock is the in-sync skip (nothing recorded, lock - /// byte-identical), and a deleted uuid dir takes the artifact-only - /// rebuild guarded by the pin the WIRED LOCK still carries — no ledger - /// is ever persisted here, so the guard runs off the lock's own pin, - /// which the deterministic local build reproduces byte-for-byte. #[tokio::test] async fn splice_flavor_revendor_in_sync_skip_and_ledgerless_rebuild() { let cases = [ @@ -6264,11 +5965,6 @@ wheels = [ } } - /// A CORRUPT state.json (vs the MISSING one of the ledgerless tests) on - /// an in-sync rebuild must not silently drop the pin guard: `load_state` - /// fails, the guard falls back to the pin the wired requirements line - /// still carries, and a mismatched service wheel is rejected under - /// `auto` in favor of the deterministic local build that reproduces it. #[tokio::test] async fn in_sync_rebuild_with_corrupt_ledger_falls_back_to_wired_pin() { let fx = e2e_fixture().await; @@ -6298,32 +5994,24 @@ wheels = [ let outcome = vendor_six( &fx, &sources, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; - let VendorOutcome::Done { - result, - entry: e2, - warnings, - } = outcome - else { - panic!("rebuild run must be Done, got {outcome:?}"); - }; - assert!(result.success, "{:?}", result.error); - assert!(e2.is_none(), "artifact-only rebuild records no entry"); + let error = crate::vendor::test_support::expect_failure(outcome); assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_pin_mismatch"), - "the pin must survive a corrupt ledger via the wired line: {warnings:?}" + error.contains("does not match the wheel the lockfile still pins"), + "{error}" ); - let on_disk = tokio::fs::read(fx.root.join(&entry.artifact.path)) - .await - .expect("the pinned wheel path must exist again"); + assert!(!fx.root.join(&entry.artifact.path).exists()); assert_eq!( - hex::encode(sha2::Sha256::digest(&on_disk)), - entry.artifact.sha256, - "the rebuilt wheel must reproduce the sha256 the wired line still pins" + tokio::fs::read(fx.root.join(crate::vendor::state::VENDOR_STATE_REL)) + .await + .unwrap(), + b"{ not json" ); } @@ -6593,7 +6281,6 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre uuid_dir_of(fx).join(WHEEL_NAME) } - /// The deterministic local build's wheel (from a throwaway copy). async fn local_wheel() -> Vec { let probe = e2e_fixture().await; let sources = PatchSources::blobs_only(&probe.blobs); @@ -6608,7 +6295,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let mut src = zip::ZipArchive::new(std::io::Cursor::new(whl)).unwrap(); let mut out = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); let opts: zip::write::SimpleFileOptions = zip::write::SimpleFileOptions::default() - .compression_method(zip::CompressionMethod::Stored); + .compression_method(zip::CompressionMethod::Deflated); for i in 0..src.len() { let mut entry = src.by_index(i).unwrap(); let mut bytes = Vec::new(); @@ -6648,7 +6335,23 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre /// Run 1, persisted like the CLI does. async fn first_run(fx: &E2eFixture, serve: Option<&[u8]>) -> VendorEntry { - let (outcome, _) = run(fx, serve, VendorSource::Auto, false).await; + let outcome = match serve { + Some(bytes) => run(fx, Some(bytes), VendorSource::Service, false).await.0, + None => { + crate::vendor::test_support::vendor_pypi( + KEY, + &fx.site_packages, + &fx.root, + &fx.record, + &PatchSources::blobs_only(&fx.blobs), + "", + false, + false, + None, + ) + .await + } + }; let (r, e, _) = ts::expect_done(outcome); assert!(r.success, "run 1: {:?}", r.error); let e = e.expect("run 1 wires"); @@ -6681,6 +6384,62 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre ] } + #[tokio::test] + async fn server_sdist_vendors_and_redownloads_across_python_flavors() { + let mut tar = tar::Builder::new(flate2::write::GzEncoder::new( + Vec::new(), + flate2::Compression::default(), + )); + for (path, bytes) in [ + ("six-1.16.0/six.py", PATCHED), + ("six-1.16.0/PKG-INFO", b"Metadata-Version: 2.1\nName: six\nVersion: 1.16.0\n\n".as_slice()), + ("six-1.16.0/setup.py", b"from setuptools import setup\nsetup(name='six', version='1.16.0', py_modules=['six'])\n".as_slice()), + ] { + let mut header = tar::Header::new_gnu(); + header.set_size(bytes.len() as u64); header.set_mode(0o644); header.set_cksum(); + tar.append_data(&mut header, path, bytes).unwrap(); + } + let bytes = tar.into_inner().unwrap().finish().unwrap(); + let server = wiremock::MockServer::start().await; + mount_pypi_granted(&server, "six-1.16.0.tar.gz", &sri_sha512(&bytes), &bytes).await; + let cfg = ts::service_cfg(&server.uri(), VendorSource::Service, false); + for (name, files) in flavors() { + let fx = flavor_fixture(&files).await; + tokio::fs::remove_dir_all(&fx.site_packages).await.unwrap(); + tokio::fs::remove_dir_all(&fx.blobs).await.unwrap(); + let (result, entry, _) = ts::expect_done( + vendor_six(&fx, &PatchSources::blobs_only(&fx.blobs), Some(&cfg)).await, + ); + assert!(result.success, "{name}: {:?}", result.error); + let entry = entry.unwrap(); + assert!(entry.artifact.path.ends_with(".tar.gz"), "{name}"); + ts::persist(&fx.root, KEY, entry.clone()).await; + let wiring = snap(&fx).await; + let ledger = tokio::fs::read(fx.root.join(".socket/vendor/state.json")) + .await + .unwrap(); + let artifact = fx.root.join(&entry.artifact.path); + tokio::fs::write(&artifact, b"corrupt").await.unwrap(); + crate::vendor::redownload::restore(&fx.root, &entry, &fx.record, &cfg) + .await + .unwrap(); + assert_eq!(tokio::fs::read(&artifact).await.unwrap(), bytes, "{name}"); + assert_eq!(snap(&fx).await, wiring, "{name}"); + assert_eq!( + tokio::fs::read(fx.root.join(".socket/vendor/state.json")) + .await + .unwrap(), + ledger, + "{name}" + ); + assert_eq!( + crate::vendor::check_vendored_artifact(&fx.root, &entry, &fx.record).await, + crate::vendor::ArtifactHealth::Healthy, + "{name}" + ); + } + } + /// An in-sync re-run after a flip, in both directions, is a no-op /// with no request, for every flavor. #[tokio::test] @@ -6696,7 +6455,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let (outcome, requests) = run( &fx, (!first_svc).then_some(alt.as_slice()), - VendorSource::Auto, + VendorSource::Service, false, ) .await; @@ -6738,7 +6497,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let (outcome, requests) = run( &fx, (!first_svc).then_some(alt.as_slice()), - VendorSource::Auto, + VendorSource::Service, false, ) .await; @@ -6810,12 +6569,19 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let ledger = tokio::fs::read(fx.root.join(".socket/vendor/state.json")) .await .unwrap(); - let (outcome, _) = run(&fx, None, VendorSource::Auto, false).await; - let (r, e, _) = ts::expect_done(outcome); + let (outcome, _) = run(&fx, None, VendorSource::Service, false).await; + let (r, e, _) = ts::expect_failed(outcome); assert!(!r.success, "present={wheel_present}: must refuse"); assert!(e.is_none()); let err = r.error.unwrap(); - assert!(err.contains("pypi_pdm_source_already_exists"), "{err}"); + assert!( + err.contains(if wheel_present { + "pypi_pdm_source_already_exists" + } else { + "503" + }), + "{err}" + ); assert_eq!( tokio::fs::read_to_string(fx.root.join("pdm.lock")) .await @@ -6850,13 +6616,13 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let _ = first_run(&fx, Some(&alt)).await; let wired = snap(&fx).await; tokio::fs::remove_file(wheel(&fx)).await.unwrap(); - let (outcome, _) = run(&fx, None, VendorSource::Auto, false).await; - let (r, e, _) = ts::expect_done(outcome); + let (outcome, _) = run(&fx, None, VendorSource::Service, false).await; + let (r, e, _) = ts::expect_failed(outcome); assert!(!r.success); assert!(e.is_none()); let err = r.error.unwrap(); - assert!(err.contains("patch service was unavailable"), "{err}"); - assert!(err.contains("once the service is reachable"), "{err}"); + assert!(err.contains("patch service request failed"), "{err}"); + assert!(err.contains("503"), "{err}"); assert_eq!(snap(&fx).await, wired, "lock unchanged"); assert!(!wheel(&fx).exists()); } @@ -6873,12 +6639,13 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre first.artifact.platform_locked = Some(true); ts::persist(&fx.root, KEY, first.clone()).await; restore(&fx, ®istry).await; - let (outcome, requests) = run(&fx, None, VendorSource::Auto, false).await; - let (r, e, w) = ts::expect_done(outcome); - assert!(r.success, "{:?}", r.error); + let (outcome, requests) = run(&fx, None, VendorSource::Service, false).await; + let (r, e, w) = ts::expect_failed(outcome); + assert!(!r.success, "{:?}", r.error); assert!(!ts::has_warning(&w, "vendor_artifact_reused"), "{w:?}"); - assert!(ts::has_warning(&w, "vendor_prebuilt_unavailable"), "{w:?}"); - assert_ne!(e.unwrap().artifact.sha256, first.artifact.sha256); + assert!(e.is_none()); + assert!(r.error.unwrap().contains("503")); + assert_eq!(tokio::fs::read(wheel(&fx)).await.unwrap(), alt); assert_eq!(requests, 1, "acquisition ran (the 503 POST)"); } @@ -6897,7 +6664,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let server = wiremock::MockServer::start().await; ts::mount_503(&server).await; let cfg = ts::service_cfg(&server.uri(), VendorSource::Service, true); - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( KEY, &fx.site_packages, &fx.root, @@ -6963,8 +6730,8 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let registry = snap(&fx).await; let _ = first_run(&fx, None).await; forge_leaf(&fx, ®istry, leaf).await; - let (outcome, _) = run(&fx, None, VendorSource::Auto, false).await; - let (r, _, w) = ts::expect_done(outcome); + let (outcome, _) = run(&fx, None, VendorSource::Service, false).await; + let (r, _, w) = ts::expect_failed(outcome); assert!( !ts::has_warning(&w, "vendor_artifact_reused"), "{leaf:?}: {w:?}" @@ -6978,7 +6745,11 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre "{leaf:?}: injected\n{req}" ); assert!(!req.contains("evil"), "{leaf:?}\n{req}"); - assert!(r.success, "{leaf:?}: acquisition re-vendors: {:?}", r.error); + assert!( + !r.success, + "{leaf:?}: acquisition re-vendors: {:?}", + r.error + ); } } @@ -6997,8 +6768,8 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre "six-1.16.0-cp311-cp311-manylinux_2_17_x86_64.whl", ) .await; - let (outcome, requests) = run(&fx, None, VendorSource::Auto, false).await; - let (_, _, w) = ts::expect_done(outcome); + let (outcome, requests) = run(&fx, None, VendorSource::Service, false).await; + let (_, _, w) = ts::expect_failed(outcome); assert!(!ts::has_warning(&w, "vendor_artifact_reused"), "{w:?}"); assert_eq!(requests, 1, "acquisition ran (the 503 POST)"); } @@ -7057,8 +6828,6 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre } } - /// An integrity mismatch is a hard failure under `auto` too — - /// never a quiet local-build fallback (service_fetch's contract). #[tokio::test] async fn service_integrity_mismatch_auto_hard_fails() { let fx = e2e_fixture().await; @@ -7067,7 +6836,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let wrong = sri_sha512(b"different bytes entirely"); let server = wiremock::MockServer::start().await; mount_pypi_granted(&server, WHEEL_NAME, &wrong, bytes).await; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, @@ -7076,7 +6845,11 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre "2026-06-09T00:00:00Z", false, false, - Some(&pypi_service_cfg(&server.uri(), VendorSource::Auto, false)), + Some(&pypi_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), ) .await; let VendorOutcome::Refused { code, .. } = outcome else { @@ -7094,7 +6867,7 @@ wheels = [{url = "https://files.pythonhosted.org/six.whl", hash = "sha256:upstre let sources = PatchSources::blobs_only(&fx.blobs); let mut cfg = pypi_service_cfg("http://127.0.0.1:1", VendorSource::Service, false); cfg.client = None; - let outcome = vendor_pypi( + let outcome = crate::vendor::test_support::vendor_pypi( "pkg:pypi/six@1.16.0", &fx.site_packages, &fx.root, diff --git a/crates/socket-patch-core/src/vendor/pypi_distribution.rs b/crates/socket-patch-core/src/vendor/pypi_distribution.rs new file mode 100644 index 000000000..92738d7dc --- /dev/null +++ b/crates/socket-patch-core/src/vendor/pypi_distribution.rs @@ -0,0 +1,165 @@ +use std::collections::HashMap; + +use crate::crawlers::python_crawler::canonicalize_pypi_name; +use crate::manifest::schema::PatchRecord; +use crate::patch::apply::normalize_file_path; + +pub(crate) fn supported(name: &str) -> bool { + [".whl", ".tar.gz", ".tgz", ".zip"] + .iter() + .any(|suffix| name.ends_with(suffix)) + && name + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'+' | b'!' | b'-')) +} + +pub(crate) fn matches(name: &str, package: &str, version: &str) -> bool { + if !supported(name) { + return false; + } + if let Some(stem) = name.strip_suffix(".whl") { + let parts: Vec<_> = stem.split('-').collect(); + return (parts.len() == 5 || parts.len() == 6) + && parts.iter().all(|p| !p.is_empty()) + && canonicalize_pypi_name(parts[0]) == canonicalize_pypi_name(package) + && parts[1].eq_ignore_ascii_case(&super::pypi_wheel::escape_wheel_version(version)); + } + [".tar.gz", ".tgz", ".zip"] + .iter() + .filter_map(|suffix| name.strip_suffix(suffix)) + .any(|stem| { + stem.rsplit_once('-').is_some_and(|(n, v)| { + canonicalize_pypi_name(n) == canonicalize_pypi_name(package) + && v.eq_ignore_ascii_case(version) + }) + }) +} + +pub(crate) fn read_members(bytes: &[u8], name: &str) -> Result>, String> { + let members = if name.ends_with(".tar.gz") || name.ends_with(".tgz") { + crate::patch::package::read_sdist_tar_bytes_to_map_strict(bytes) + .map_err(|e| e.to_string())? + } else { + super::verify::read_zip_bytes_to_map_strict(bytes)? + }; + if name.ends_with(".whl") { + return Ok(members); + } + strip_sdist_root(members) +} + +fn strip_sdist_root(members: HashMap>) -> Result>, String> { + let root = members + .keys() + .next() + .and_then(|name| name.split_once('/')) + .map(|(root, _)| format!("{root}/")); + if let Some(root) = root.filter(|root| members.keys().all(|name| name.starts_with(root))) { + Ok(members + .into_iter() + .map(|(name, bytes)| (name[root.len()..].to_string(), bytes)) + .collect()) + } else { + Ok(members) + } +} + +pub(crate) fn verify_members( + members: &HashMap>, + name: &str, + record: &PatchRecord, +) -> Result<(), String> { + if name.ends_with(".whl") { + return super::verify::verify_member_map(members, record); + } + for (path, info) in &record.files { + let key = normalize_file_path(path); + let prefixed = format!("src/{key}"); + let candidates: Vec<_> = [members.get(key), members.get(&prefixed)] + .into_iter() + .flatten() + .collect(); + if candidates.len() != 1 { + return Err("vendor_sdist_layout_ambiguous".into()); + } + if !crate::hash::git_sha256::compute_git_sha256_from_bytes(candidates[0]) + .eq_ignore_ascii_case(&info.after_hash) + { + return Err("vendor_hash_mismatch".into()); + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::manifest::schema::PatchFileInfo; + + fn tar(entries: &[(&str, &[u8], tar::EntryType)]) -> Vec { + let mut archive = tar::Builder::new(flate2::write::GzEncoder::new( + Vec::new(), + flate2::Compression::default(), + )); + for (path, bytes, kind) in entries { + let mut header = tar::Header::new_gnu(); + header.set_mode(0o644); + header.set_size(bytes.len() as u64); + header.set_entry_type(*kind); + header.set_cksum(); + archive.append_data(&mut header, path, *bytes).unwrap(); + } + archive.into_inner().unwrap().finish().unwrap() + } + + #[test] + fn sdist_tar_preserves_package_layout_and_strips_distribution_root() { + let bytes = tar(&[("six-1.16.0/src/six.py", b"patched", tar::EntryType::Regular)]); + let members = read_members(&bytes, "six-1.16.0.tar.gz").unwrap(); + assert_eq!(members.get("src/six.py").unwrap(), b"patched"); + assert!(matches("six-1.16.0.tar.gz", "six", "1.16.0")); + assert!(!matches("six-1.16.0.tar.gz", "six", "1.17.0")); + } + + #[test] + fn sdist_tar_rejects_aliases_links_and_oversized_members() { + let aliases = tar(&[ + ("six-1/six.py", b"one", tar::EntryType::Regular), + ("six-1/SIX.py", b"two", tar::EntryType::Regular), + ]); + assert!(read_members(&aliases, "six-1.tar.gz").is_err()); + let link = tar(&[("six-1/six.py", b"", tar::EntryType::Symlink)]); + assert!(read_members(&link, "six-1.tar.gz").is_err()); + let bytes = vec![0; 16 * 1024 * 1024 + 1]; + let oversized = tar(&[("six-1/six.py", &bytes, tar::EntryType::Regular)]); + assert!(read_members(&oversized, "six-1.tar.gz").is_err()); + } + + #[test] + fn sdist_member_verification_rejects_ambiguous_src_layout() { + let record: PatchRecord = serde_json::from_value(serde_json::json!({ + "uuid":"11111111-1111-1111-1111-111111111111", "exportedAt": "2026-01-01T00:00:00Z", "vulnerabilities": {}, "description": "test", "license": "MIT", "tier": "free", + "files": { "six.py": { "beforeHash": "old", "afterHash": crate::hash::git_sha256::compute_git_sha256_from_bytes(b"patched") } } + })).unwrap(); + let members = HashMap::from([("six.py".to_string(), b"patched".to_vec())]); + assert!(verify_members(&members, "six-1.tar.gz", &record).is_ok()); + let mut ambiguous = members.clone(); + ambiguous.insert("src/six.py".into(), b"patched".to_vec()); + assert!(verify_members(&ambiguous, "six-1.tar.gz", &record).is_err()); + assert!(verify_members( + &members, + "six-1.tar.gz", + &PatchRecord { + files: HashMap::from([( + "six.py".into(), + PatchFileInfo { + after_hash: "wrong".into(), + ..record.files["six.py"].clone() + } + )]), + ..record + } + ) + .is_err()); + } +} diff --git a/crates/socket-patch-core/src/vendor/pypi_hatch.rs b/crates/socket-patch-core/src/vendor/pypi_hatch.rs index c82d73e00..aa8528b69 100644 --- a/crates/socket-patch-core/src/vendor/pypi_hatch.rs +++ b/crates/socket-patch-core/src/vendor/pypi_hatch.rs @@ -61,7 +61,7 @@ pub(super) async fn load( ) })?; if leaf.contains(['/', '\\', '%', ':']) - || !leaf.ends_with(".whl") + || !super::pypi_distribution::supported(leaf) || hash.len() != 64 || !hash.bytes().all(|byte| byte.is_ascii_hexdigit()) { diff --git a/crates/socket-patch-core/src/vendor/pypi_lock.rs b/crates/socket-patch-core/src/vendor/pypi_lock.rs index cb522160f..929d41a3c 100644 --- a/crates/socket-patch-core/src/vendor/pypi_lock.rs +++ b/crates/socket-patch-core/src/vendor/pypi_lock.rs @@ -135,6 +135,15 @@ fn source_sha(table: &Table) -> Option { { return Some(sha.to_string()); } + if let Some(hash) = table + .get("sdist") + .and_then(Item::as_inline_table) + .and_then(|sdist| sdist.get("hash")) + .and_then(toml_edit::Value::as_str) + .and_then(|h| h.strip_prefix("sha256:")) + { + return Some(hash.to_string()); + } table.get("wheels")?.as_array()?.iter().find_map(|wheel| { wheel .as_inline_table()? diff --git a/crates/socket-patch-core/src/vendor/pypi_pdm.rs b/crates/socket-patch-core/src/vendor/pypi_pdm.rs index bdb6ce4cc..c3acaa64b 100644 --- a/crates/socket-patch-core/src/vendor/pypi_pdm.rs +++ b/crates/socket-patch-core/src/vendor/pypi_pdm.rs @@ -273,7 +273,7 @@ fn check_target_unit( Some(parts) if parts.eco == "pypi" && parts.uuid == record_uuid - && crate::utils::pdm_lock::wheel_matches(&parts.leaf, canon_name, version) => + && super::pypi_distribution::matches(&parts.leaf, canon_name, version) => { Ok(PdmTarget::InSync) } @@ -718,6 +718,7 @@ distribution = false base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: REL_WHEEL.into(), sha256: WHEEL_SHA.into(), size: Some(11053), diff --git a/crates/socket-patch-core/src/vendor/pypi_pipenv.rs b/crates/socket-patch-core/src/vendor/pypi_pipenv.rs index ff55fb6aa..be136922c 100644 --- a/crates/socket-patch-core/src/vendor/pypi_pipenv.rs +++ b/crates/socket-patch-core/src/vendor/pypi_pipenv.rs @@ -843,6 +843,7 @@ mod tests { base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: REL_WHEEL.into(), sha256: WHEEL_SHA.into(), size: Some(11053), diff --git a/crates/socket-patch-core/src/vendor/pypi_poetry.rs b/crates/socket-patch-core/src/vendor/pypi_poetry.rs index ede0f1a2a..e0c573044 100644 --- a/crates/socket-patch-core/src/vendor/pypi_poetry.rs +++ b/crates/socket-patch-core/src/vendor/pypi_poetry.rs @@ -858,6 +858,7 @@ content-hash = "4b42a89b7ff7b26511b06acdc458dbd85312e5083db8f212b017482bc68cdd01 base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: REL_WHEEL.into(), sha256: WHEEL_SHA.into(), size: Some(11053), diff --git a/crates/socket-patch-core/src/vendor/pypi_requirements.rs b/crates/socket-patch-core/src/vendor/pypi_requirements.rs index 00f4e1133..2c531d17f 100644 --- a/crates/socket-patch-core/src/vendor/pypi_requirements.rs +++ b/crates/socket-patch-core/src/vendor/pypi_requirements.rs @@ -931,6 +931,7 @@ mod tests { base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: REL_WHEEL.into(), sha256: SHA.into(), size: Some(11053), diff --git a/crates/socket-patch-core/src/vendor/pypi_uv.rs b/crates/socket-patch-core/src/vendor/pypi_uv.rs index b60b927ca..9185c49ad 100644 --- a/crates/socket-patch-core/src/vendor/pypi_uv.rs +++ b/crates/socket-patch-core/src/vendor/pypi_uv.rs @@ -1078,13 +1078,19 @@ fn rewrite_target_package_unit( // rebuilt unit splices back in front of the same `\r\n`. let old_unit = lock_text[span].trim_end_matches('\r').to_string(); let unit: Vec<&str> = old_unit.lines().collect(); - let wheels_lines = [ - "wheels = [".to_string(), - format!( - " {{ filename = \"{wheel_file_name}\", hash = \"sha256:{wheel_sha256_hex}\" }}," - ), - "]".to_string(), - ]; + let wheels_lines = if wheel_file_name.ends_with(".whl") { + vec![ + "wheels = [".to_string(), + format!( + " {{ filename = \"{wheel_file_name}\", hash = \"sha256:{wheel_sha256_hex}\" }}," + ), + "]".to_string(), + ] + } else { + vec![format!( + "sdist = {{ hash = \"sha256:{wheel_sha256_hex}\" }}" + )] + }; let mut out: Vec = Vec::new(); let mut wheels_done = false; @@ -1554,7 +1560,17 @@ struct MetaDep { /// the fixtures that pass no block stay byte-exact. pub(super) async fn wheel_metadata_block(wheel_path: &Path) -> Option { let bytes = tokio::fs::read(wheel_path).await.ok()?; - let text = wheel_metadata_text(&bytes)?; + let text = if wheel_path.to_string_lossy().ends_with(".whl") { + wheel_metadata_text(&bytes)? + } else { + let members = + super::pypi_distribution::read_members(&bytes, &wheel_path.to_string_lossy()).ok()?; + let metadata = members.get("PKG-INFO")?; + if metadata.len() > 4 * 1024 * 1024 { + return None; + } + String::from_utf8(metadata.clone()).ok()? + }; render_package_metadata_block(&text) } @@ -1957,6 +1973,7 @@ wheels = [ base_purl: "pkg:pypi/six@1.16.0".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: REL_WHEEL.into(), sha256: WHEEL_SHA.into(), size: Some(11053), diff --git a/crates/socket-patch-core/src/vendor/pypi_wheel.rs b/crates/socket-patch-core/src/vendor/pypi_wheel.rs index 927ee006e..b7db78501 100644 --- a/crates/socket-patch-core/src/vendor/pypi_wheel.rs +++ b/crates/socket-patch-core/src/vendor/pypi_wheel.rs @@ -1,52 +1,5 @@ -//! Rebuild an installable wheel from the patched installed distribution. -//! -//! pypi vendoring cannot reuse a registry artifact: the patch applies to the -//! *installed* site-packages tree, so the committable `.socket/vendor/pypi/` -//! artifact must be reconstructed from that tree. The installed -//! `*.dist-info/RECORD` is the authoritative member list (pip 26 / uv 0.11 -//! only require RECORD to exist and parse at install time — per -//! file hashes are unchecked — but we regenerate it correctly anyway, because -//! the RECORD drives uninstall bookkeeping and post-hoc audits). The rebuild -//! is byte-for-byte deterministic so the emitted `--hash` / uv lock hash pin -//! is stable across re-runs and never churns committed files. +//! Metadata for downloaded Python distributions. -use std::collections::{HashMap, HashSet}; -use std::path::{Path, PathBuf}; - -use base64::Engine as _; -use sha2::Digest as _; - -use crate::crawlers::python_crawler::{canonicalize_pypi_name, read_python_metadata}; -use crate::manifest::schema::PatchRecord; -use crate::patch::apply::{ - is_safe_relative_subpath, normalize_file_path, ApplyResult, PatchSources, -}; -use crate::utils::fs::{ - atomic_write_artifact, list_dir_entries, read_regular_to_bytes, read_regular_to_string, -}; - -use super::common::{ - can_repack_in_memory, failed_result, is_executable, patch_target_paths, write_zip_entries, - Stage, -}; - -/// The located installed distribution for one `name@version`. -#[derive(Debug, Clone)] -pub struct InstalledDist { - /// Absolute path of the `-.dist-info` directory. - pub dist_info_dir: PathBuf, - /// Raw distribution-name part of the dist-info directory stem (casing - /// and separators as installed, e.g. `Flask-SQLAlchemy`) — the input to - /// the wheel-filename escaping, NOT a canonical PEP 503 name. - pub dist_name: String, - pub version: String, - /// Member paths parsed from `RECORD` (the path field of each row). - pub record: Vec, - /// Raw `Tag:` header values from the `WHEEL` file, in file order. - pub wheel_tags: Vec, -} - -/// The rebuilt artifact: leaf filename + content identity for the lock pins. #[derive(Debug, Clone, PartialEq, Eq)] pub struct WheelArtifact { pub file_name: String, @@ -56,155 +9,6 @@ pub struct WheelArtifact { pub size: u64, } -/// Byte-reading twin of [`read_regular_to_string`] that also hands back the -/// metadata from the already-open handle (the member staging loop needs the -/// exec bit without a second stat — the one reason this is not the shared -/// `read_regular_to_bytes`). -async fn read_regular(path: &Path) -> std::io::Result<(Vec, std::fs::Metadata)> { - use tokio::io::AsyncReadExt as _; - - let (mut file, metadata) = crate::utils::fs::open_regular_file(path).await?; - let mut content = Vec::with_capacity(metadata.len() as usize); - file.read_to_end(&mut content).await?; - Ok((content, metadata)) -} - -/// Find the installed dist for `purl_name@version` by scanning the -/// `*.dist-info` directories under the site-packages root (the crawler's -/// `pkg_path` for pypi). Name matching is PEP 503-canonical on BOTH sides so -/// `Flask_SQLAlchemy` / `flask-sqlalchemy` spellings collapse, mirroring -/// [`crate::crawlers::python_crawler`]. -pub async fn locate_installed_dist( - site_packages: &Path, - purl_name: &str, - version: &str, -) -> Result { - let want = canonicalize_pypi_name(purl_name); - // Two passes over the one in-memory listing: dist-info stems that already - // spell `-` first (one METADATA read in the common case - // instead of one per installed dist), then every other dist-info as the - // fallback (a stem without a version part, or a stale install whose stem - // disagrees with its METADATA). Both passes run the same authoritative - // METADATA compare below. - let stem_matches = |dir_name: &str| { - dir_name - .strip_suffix(".dist-info") - .and_then(|stem| stem.rfind('-').map(|i| (&stem[..i], &stem[i + 1..]))) - .is_some_and(|(n, v)| canonicalize_pypi_name(n) == want && v == version) - }; - let (likely, rest): (Vec<_>, Vec<_>) = list_dir_entries(site_packages) - .await - .into_iter() - .partition(|e| stem_matches(&e.file_name().to_string_lossy())); - for entry in likely.into_iter().chain(rest) { - let dir_name = entry.file_name().to_string_lossy().into_owned(); - let Some(stem) = dir_name.strip_suffix(".dist-info") else { - continue; - }; - let dist_info = entry.path(); - let Some((raw_name, found_version)) = read_python_metadata(&dist_info).await else { - continue; - }; - if canonicalize_pypi_name(&raw_name) != want || found_version != version { - continue; - } - - // Wheel filenames re-escape from the RAW installed name; the - // dist-info stem keeps it (`Flask-SQLAlchemy-2.5.1.dist-info`), with - // the METADATA Name as fallback for stems that carry no version part. - let dist_name = match stem.rfind('-') { - Some(i) if i > 0 => stem[..i].to_string(), - _ => raw_name.clone(), - }; - - let record_text = read_regular_to_string(&dist_info.join("RECORD")) - .await - .map_err(|e| { - ( - "pypi_missing_record", - format!( - "cannot rebuild a wheel for {purl_name}@{version}: {}/RECORD is unreadable ({e})", - dist_info.display() - ), - ) - })?; - let record = parse_record_paths(&record_text); - if record.is_empty() { - return Err(( - "pypi_missing_record", - format!( - "cannot rebuild a wheel for {purl_name}@{version}: {}/RECORD lists no files", - dist_info.display() - ), - )); - } - - let wheel_text = read_regular_to_string(&dist_info.join("WHEEL")) - .await - .map_err(|e| { - ( - "pypi_missing_wheel_metadata", - format!( - "cannot rebuild a wheel for {purl_name}@{version}: {}/WHEEL is unreadable ({e})", - dist_info.display() - ), - ) - })?; - let wheel_tags: Vec = wheel_text - .lines() - .filter_map(|l| l.strip_prefix("Tag:")) - .map(|s| s.trim().to_string()) - .filter(|s| !s.is_empty()) - .collect(); - if wheel_tags.is_empty() { - return Err(( - "pypi_missing_wheel_metadata", - format!( - "cannot rebuild a wheel for {purl_name}@{version}: {}/WHEEL carries no Tag: headers", - dist_info.display() - ), - )); - } - - return Ok(InstalledDist { - dist_info_dir: dist_info, - dist_name, - version: found_version, - record, - wheel_tags, - }); - } - Err(( - "pypi_dist_not_found", - format!( - "{purl_name}@{version} is not installed under {}", - site_packages.display() - ), - )) -} - -/// The PEP 427 filename for the rebuilt wheel: -/// `--.whl`. -pub fn wheel_file_name(dist: &InstalledDist) -> Result { - let name = escape_wheel_component(&dist.dist_name); - let version = escape_wheel_version(&dist.version); - let (py, abi, plat) = compress_wheel_tags(&dist.wheel_tags)?; - Ok(format!("{name}-{version}-{py}-{abi}-{plat}.whl")) -} - -/// Wheel-spec component escaping: runs of `[^A-Za-z0-9.]` collapse to a -/// single `_` so the filename stays unambiguous at the `-` separators. -fn escape_wheel_component(s: &str) -> String { - escape_wheel_chars(s, false) -} - -/// Version-component escaping: PEP 440 normalized versions legitimately -/// carry `!` (epoch) and `+` (local separator), and real tools keep them -/// literally in the filename (bdist_wheel's `safer_version` only maps `-` -/// runs to `_`). pip/uv REJECT the `_`-escaped spelling as an invalid wheel -/// version (`packaging.utils.parse_wheel_filename` raises on -/// `torch-2.0.0_cu118-…` but parses `torch-2.0.0+cu118-…`), so escaping -/// those two would make the rebuilt wheel uninstallable. pub(crate) fn escape_wheel_version(s: &str) -> String { escape_wheel_chars(s, true) } @@ -226,1759 +30,3 @@ fn escape_wheel_chars(s: &str, keep_version_separators: bool) -> String { } out } - -/// Compress the WHEEL `Tag:` set back into the filename's dotted triple -/// (`py2.py3-none-any`). The dotted form expands to the CROSS PRODUCT of the -/// three component sets, so the compression is only faithful when the -/// observed tag set IS a full cross product — anything else would synthesize -/// a filename claiming compatibility the installed dist never declared, so -/// it is refused instead. -fn compress_wheel_tags( - tags: &[String], -) -> Result<(String, String, String), (&'static str, String)> { - let mut pys: Vec<&str> = Vec::new(); - let mut abis: Vec<&str> = Vec::new(); - let mut plats: Vec<&str> = Vec::new(); - let mut seen: HashSet<(&str, &str, &str)> = HashSet::new(); - for tag in tags { - let parts: Vec<&str> = tag.split('-').collect(); - let [py, abi, plat] = parts.as_slice() else { - return Err(( - "pypi_wheel_tags_unrecoverable", - format!("WHEEL tag {tag:?} is not a py-abi-platform triple"), - )); - }; - if !pys.contains(py) { - pys.push(py); - } - if !abis.contains(abi) { - abis.push(abi); - } - if !plats.contains(plat) { - plats.push(plat); - } - seen.insert((py, abi, plat)); - } - let product = pys.len() * abis.len() * plats.len(); - let all_present = pys.iter().all(|p| { - abis.iter() - .all(|a| plats.iter().all(|pl| seen.contains(&(p, a, pl)))) - }); - if product != seen.len() || !all_present { - return Err(( - "pypi_wheel_tags_unrecoverable", - format!( - "WHEEL tag set {tags:?} is not a cross product of its components and cannot be \ - expressed as a single wheel filename" - ), - )); - } - Ok((pys.join("."), abis.join("."), plats.join("."))) -} - -/// Build the patched wheel at `dest` from the installed dist: -/// stage the RECORD members → apply the patch in the stage → regenerate -/// RECORD → deterministic zip → atomic write. -/// -/// Errors (`Err((code, detail))`) are refusal-shaped — nothing was written -/// and the orchestrator maps them to [`VendorOutcome::Refused`]. Runtime -/// failures after staging surface as a failed [`ApplyResult`] instead, in the -/// same shape `apply` reports them. -/// -/// `dry_run` stops after the in-stage verification (no zip, no `dest` write). -/// -/// [`VendorOutcome::Refused`]: super::VendorOutcome::Refused -#[allow(clippy::too_many_arguments)] -pub async fn build_patched_wheel( - purl: &str, - site_packages: &Path, - dist: &InstalledDist, - record: &PatchRecord, - sources: &PatchSources<'_>, - dest: &Path, - dry_run: bool, - force: bool, - warnings: &mut Vec, -) -> Result<(ApplyResult, Option), (&'static str, String)> { - // Editable installs (`pip install -e` / uv tool dev mode) point - // site-packages at the user's own working tree: the RECORD describes a - // `.pth`/finder shim, not the package contents, so a rebuilt wheel would - // vendor the shim instead of the code. Checked BEFORE staging. - if is_editable_install(&dist.dist_info_dir).await { - return Err(( - "pypi_editable_install", - format!( - "{purl} is an editable install ({}); vendor needs a regular installed distribution", - dist.dist_info_dir.display() - ), - )); - } - - let dist_info_name = dist - .dist_info_dir - .file_name() - .map(|n| n.to_string_lossy().into_owned()) - .unwrap_or_default(); - let script_names = - match read_regular_to_string(&dist.dist_info_dir.join("entry_points.txt")).await { - Ok(text) => console_script_names(&text), - Err(_) => HashSet::new(), - }; - - // Select the wheel members from the installed RECORD. - let mut members: Vec = Vec::new(); - let mut out_of_tree: Vec = Vec::new(); - for path in &dist.record { - let path = path.as_str(); - if path.is_empty() - || is_installer_bookkeeping(path, &dist_info_name) - || path.ends_with(".pyc") - || path.split('/').any(|c| c == "__pycache__") - { - continue; - } - // SECURITY: `is_safe_relative_subpath` is the in-tree gate. A RECORD - // row that escapes site-packages (`../../../bin/x`, absolute paths) - // must never be staged or zipped — only the installer-regenerated - // console/gui scripts (matched by entry_points.txt NAME, never by - // extension heuristics, which would wrongly drop - // `../../../share/man/man6/pycowsay.6`) are silently excluded; any - // OTHER out-of-tree entry is data the rebuilt wheel cannot carry, so - // the whole vendor is refused fail-closed. - if !is_safe_relative_subpath(path) { - let last = path.rsplit('/').next().unwrap_or(path); - if is_console_script_artifact(last, &script_names) { - continue; - } - out_of_tree.push(path.to_string()); - continue; - } - members.push(path.to_string()); - } - if !out_of_tree.is_empty() { - out_of_tree.sort(); - return Err(( - "pypi_out_of_tree_files", - format!( - "RECORD lists files outside site-packages that are not console scripts \ - (a rebuilt wheel cannot reproduce them): {}", - out_of_tree.join(", ") - ), - )); - } - members.sort(); - members.dedup(); - - // Stage the members into a private tree preserving the site-packages- - // relative layout, so the manifest's sp-relative pypi file keys resolve. - let stage = match Stage::new() { - Ok(dir) => dir, - Err(e) => { - return Ok(( - failed_result(purl, site_packages, format!("cannot create stage dir: {e}")), - None, - )) - } - }; - // The apply pipeline only ever resolves the patch targets, so they are - // the only members that have to exist on disk: every other member is read - // once and carried straight into the wheel. That holds only while no two - // names can fold into one another on the staging filesystem (a - // case-insensitive or Unicode-normalising volume) — for anything else the - // whole member set is staged and read back from there. - // `patch_target_paths` is sorted, so the directory pass below — which - // returns on its first failure — names the same target on every run. - let target_paths = patch_target_paths(&record.files); - let targets: HashSet<&str> = target_paths.iter().copied().collect(); - let held_in_memory = can_repack_in_memory( - members.iter().map(String::as_str), - target_paths.iter().copied(), - ); - let mut held: HashMap> = HashMap::new(); - let mut exec_bits: HashMap = HashMap::new(); - for member in &members { - let src = site_packages.join(member); - let (bytes, metadata) = match read_regular(&src).await { - Ok(pair) => pair, - Err(e) => { - return Ok(( - failed_result( - purl, - site_packages, - format!("RECORD member {member} is unreadable: {e}"), - ), - None, - )) - } - }; - exec_bits.insert(member.clone(), is_executable(&metadata)); - if held_in_memory && !targets.contains(member.as_str()) { - held.insert(member.clone(), bytes); - continue; - } - let dst = stage.path().join(member); - if let Some(parent) = dst.parent() { - if let Err(e) = tokio::fs::create_dir_all(parent).await { - return Ok(( - failed_result(purl, site_packages, format!("cannot stage {member}: {e}")), - None, - )); - } - } - if let Err(e) = tokio::fs::write(&dst, &bytes).await { - return Ok(( - failed_result(purl, site_packages, format!("cannot stage {member}: {e}")), - None, - )); - } - } - // A patch key can name a DIRECTORY of the installed tree; a fully staged - // tree carried one wherever a member lived under it, and the verify reads - // "cannot hash" there rather than "not found". Recreate exactly those. - if held_in_memory { - for target in &target_paths { - let prefix = format!("{target}/"); - if members.iter().any(|m| m == target) - || !members.iter().any(|m| m.starts_with(&prefix)) - { - continue; - } - if let Err(e) = tokio::fs::create_dir_all(stage.path().join(target)).await { - return Ok(( - failed_result(purl, site_packages, format!("cannot stage {target}: {e}")), - None, - )); - } - } - } - - // Patch the stage through the shared apply pipeline (same verify/source - // strategy contract as `apply`, with the vendor auto-force policy — - // see `force_apply_staged`). The installed tree is never touched. - let mut result = super::force_apply_staged( - purl, - stage.path(), - record, - sources, - dry_run, - force, - &dist.dist_name, - &dist.version, - warnings, - ) - .await; - if dry_run || !result.success { - stage.dispose().await; - return Ok((result, None)); - } - - // Files CREATED by the patch (empty beforeHash) exist only in the stage; - // union them into the member list so the wheel ships them. - for (file_name, info) in &record.files { - if info.before_hash.is_empty() { - let normalized = normalize_file_path(file_name).to_string(); - if !members.contains(&normalized) { - exec_bits.insert(normalized.clone(), false); - members.push(normalized); - } - } - } - members.sort(); - - // Regenerate RECORD from the staged (patched) bytes and assemble the - // deterministic zip entry list (`(name, bytes, unix mode)` with the exec - // bit preserved as 0o755): lexicographic order, RECORD forced last - // (installers stream-read it; last is also what bdist_wheel emits). - let mut entries: Vec<(String, Vec, u32)> = Vec::with_capacity(members.len() + 1); - let mut record_lines = String::new(); - for member in &members { - // A member the apply never touched was never written out: it comes - // from the one read of the installed tree above. The rest (the patch - // targets, and the files the patch created) come back off the stage. - let bytes = match held.remove(member.as_str()) { - Some(bytes) => bytes, - None => match tokio::fs::read(stage.path().join(member)).await { - Ok(b) => b, - Err(e) => { - result.success = false; - result.error = Some(format!("staged member {member} vanished: {e}")); - return Ok((result, None)); - } - }, - }; - let digest = sha2::Sha256::digest(&bytes); - let b64 = base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(digest); - record_lines.push_str(&format!( - "{},sha256={},{}\n", - csv_quote(member), - b64, - bytes.len() - )); - let mode = if exec_bits.get(member).copied().unwrap_or(false) { - 0o755 - } else { - 0o644 - }; - entries.push((member.clone(), bytes, mode)); - } - record_lines.push_str(&format!("{}/RECORD,,\n", csv_quote(&dist_info_name))); - entries.push(( - format!("{dist_info_name}/RECORD"), - record_lines.into_bytes(), - 0o644, - )); - stage.dispose().await; - - let zip_bytes = match tokio::task::spawn_blocking(move || write_zip_entries(&entries)).await { - Ok(Ok(bytes)) => bytes, - Ok(Err(e)) => { - result.success = false; - result.error = Some(format!("wheel zip assembly failed: {e}")); - return Ok((result, None)); - } - Err(e) => { - result.success = false; - result.error = Some(format!("wheel zip task failed: {e}")); - return Ok((result, None)); - } - }; - - if let Some(parent) = dest.parent() { - if let Err(e) = tokio::fs::create_dir_all(parent).await { - result.success = false; - result.error = Some(format!("cannot create {}: {e}", parent.display())); - return Ok((result, None)); - } - } - if let Err(e) = atomic_write_artifact(dest, &zip_bytes).await { - result.success = false; - result.error = Some(format!("cannot write {}: {e}", dest.display())); - return Ok((result, None)); - } - - let artifact = WheelArtifact { - file_name: dest - .file_name() - .map(|n| n.to_string_lossy().into_owned()) - .unwrap_or_default(), - sha256_hex: hex::encode(sha2::Sha256::digest(&zip_bytes)), - size: zip_bytes.len() as u64, - }; - Ok((result, Some(artifact))) -} - -/// Installer bookkeeping the wheel must not carry: signatures and per-install -/// state regenerated by pip/uv (`RECORD` itself is rebuilt; `direct_url.json` -/// describes the OLD origin and would mislabel the vendored install). -fn is_installer_bookkeeping(path: &str, dist_info_name: &str) -> bool { - const NAMES: [&str; 6] = [ - "RECORD", - "RECORD.jws", - "RECORD.p7s", - "INSTALLER", - "REQUESTED", - "direct_url.json", - ]; - NAMES - .iter() - .any(|n| path == format!("{dist_info_name}/{n}")) -} - -/// True when `dist-info/direct_url.json` marks the install editable. -async fn is_editable_install(dist_info_dir: &Path) -> bool { - let Ok(bytes) = read_regular_to_bytes(&dist_info_dir.join("direct_url.json")).await else { - return false; - }; - let Ok(value) = serde_json::from_slice::(&bytes) else { - return false; - }; - value - .get("dir_info") - .and_then(|d| d.get("editable")) - .and_then(serde_json::Value::as_bool) - .unwrap_or(false) -} - -/// `[console_scripts]` / `[gui_scripts]` entry names from `entry_points.txt`. -fn console_script_names(text: &str) -> HashSet { - let mut names = HashSet::new(); - let mut in_scripts = false; - for line in text.lines() { - let line = line.trim(); - if line.starts_with('[') && line.ends_with(']') { - let section = line[1..line.len() - 1].trim(); - in_scripts = section == "console_scripts" || section == "gui_scripts"; - continue; - } - if in_scripts { - if let Some((name, _)) = line.split_once('=') { - let name = name.trim(); - if !name.is_empty() { - names.insert(name.to_string()); - } - } - } - } - names -} - -/// True when an out-of-tree RECORD entry's final component is an installer- -/// generated script for a declared entry point (`x`, `x.exe`, `x-script.py`). -fn is_console_script_artifact(final_component: &str, script_names: &HashSet) -> bool { - if script_names.contains(final_component) { - return true; - } - if let Some(stem) = final_component.strip_suffix(".exe") { - if script_names.contains(stem) { - return true; - } - } - if let Some(stem) = final_component.strip_suffix("-script.py") { - if script_names.contains(stem) { - return true; - } - } - false -} - -/// Parse the member paths out of `RECORD` rows (`path,hash,size` CSV; quoted -/// fields possible). Only the path field is consumed — the RECORD is -/// regenerated from the patched bytes, so the recorded hash/size are never -/// read. Unparseable/blank lines are skipped rather than failing the whole -/// file — fail-open here is safe because the member list only ever loses a -/// row it could not have staged anyway. -fn parse_record_paths(text: &str) -> Vec { - text.lines() - .filter(|line| !line.trim().is_empty()) - .filter_map(|line| parse_csv_record(line).into_iter().next()) - .filter(|path| !path.is_empty()) - .collect() -} - -/// Minimal CSV record parser (RFC 4180 quoting: `"a,b"`, doubled `""`). -fn parse_csv_record(line: &str) -> Vec { - let mut fields = Vec::new(); - let mut current = String::new(); - let mut in_quotes = false; - let mut chars = line.chars().peekable(); - while let Some(c) = chars.next() { - if in_quotes { - if c == '"' { - if chars.peek() == Some(&'"') { - current.push('"'); - chars.next(); - } else { - in_quotes = false; - } - } else { - current.push(c); - } - } else { - match c { - '"' => in_quotes = true, - ',' => fields.push(std::mem::take(&mut current)), - _ => current.push(c), - } - } - } - fields.push(current); - fields -} - -/// CSV-quote a field when it needs it (comma/quote/newline). -fn csv_quote(field: &str) -> String { - if field.contains([',', '"', '\n', '\r']) { - format!("\"{}\"", field.replace('"', "\"\"")) - } else { - field.to_string() - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::hash::git_sha256::compute_git_sha256_from_bytes; - use crate::manifest::schema::PatchFileInfo; - - const UUID: &str = "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"; - const ORIG: &[u8] = b"class Six:\n pass\n"; - const PATCHED: &[u8] = b"class Six:\n pass\n# SOCKET-PATCH-MARKER\n"; - - struct Fixture { - _tmp: tempfile::TempDir, - site_packages: PathBuf, - blobs: PathBuf, - dest: PathBuf, - } - - /// A six-like installed dist plus a blob store carrying the afterHash - /// bytes, mirroring a real `.socket/blobs/` layout. - async fn make_fixture(extra_record_lines: &str, entry_points: Option<&str>) -> Fixture { - let tmp = tempfile::tempdir().unwrap(); - let sp = tmp.path().join("site-packages"); - let di = sp.join("six-1.16.0.dist-info"); - tokio::fs::create_dir_all(&di).await.unwrap(); - tokio::fs::write(sp.join("six.py"), ORIG).await.unwrap(); - tokio::fs::write( - di.join("METADATA"), - "Metadata-Version: 2.1\nName: six\nVersion: 1.16.0\n\nREADME body\n", - ) - .await - .unwrap(); - tokio::fs::write( - di.join("WHEEL"), - "Wheel-Version: 1.0\nGenerator: test\nRoot-Is-Purelib: true\nTag: py2-none-any\nTag: py3-none-any\n", - ) - .await - .unwrap(); - let record = format!( - "six.py,sha256=AAAA,20\n\ - six-1.16.0.dist-info/METADATA,sha256=BBBB,60\n\ - six-1.16.0.dist-info/WHEEL,,\n\ - six-1.16.0.dist-info/INSTALLER,sha256=,4\n\ - six-1.16.0.dist-info/RECORD,,\n\ - __pycache__/six.cpython-314.pyc,,\n{extra_record_lines}" - ); - tokio::fs::write(di.join("RECORD"), record).await.unwrap(); - if let Some(ep) = entry_points { - tokio::fs::write(di.join("entry_points.txt"), ep) - .await - .unwrap(); - } - let blobs = tmp.path().join("blobs"); - tokio::fs::create_dir_all(&blobs).await.unwrap(); - tokio::fs::write(blobs.join(compute_git_sha256_from_bytes(PATCHED)), PATCHED) - .await - .unwrap(); - let dest = tmp.path().join(format!( - ".socket/vendor/pypi/{UUID}/six-1.16.0-py2.py3-none-any.whl" - )); - Fixture { - _tmp: tmp, - site_packages: sp, - blobs, - dest, - } - } - - fn patch_record(files: &[(&str, &[u8], &[u8])]) -> PatchRecord { - let mut map = HashMap::new(); - for (name, before, after) in files { - map.insert( - name.to_string(), - PatchFileInfo { - before_hash: if before.is_empty() { - String::new() - } else { - compute_git_sha256_from_bytes(before) - }, - after_hash: compute_git_sha256_from_bytes(after), - }, - ); - } - PatchRecord { - uuid: UUID.to_string(), - exported_at: String::new(), - files: map, - vulnerabilities: HashMap::new(), - description: String::new(), - license: String::new(), - tier: String::new(), - } - } - - fn zip_names(bytes: &[u8]) -> Vec { - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes.to_vec())).unwrap(); - (0..archive.len()) - .map(|i| archive.by_index(i).unwrap().name().to_string()) - .collect() - } - - fn zip_file(bytes: &[u8], name: &str) -> Vec { - use std::io::Read as _; - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes.to_vec())).unwrap(); - let mut file = archive.by_name(name).unwrap(); - let mut out = Vec::new(); - file.read_to_end(&mut out).unwrap(); - out - } - - #[cfg(unix)] - fn zip_unix_mode(bytes: &[u8], name: &str) -> u32 { - let mut archive = zip::ZipArchive::new(std::io::Cursor::new(bytes.to_vec())).unwrap(); - let file = archive.by_name(name).unwrap(); - file.unix_mode() - .unwrap_or_else(|| panic!("{name} carries no unix mode")) - } - - #[test] - fn record_parse_round_trips_quoted_and_empty_fields() { - let text = "six.py,sha256=abc_DEF,123\n\ - \"weird,name.py\",sha256=zz,9\n\ - six-1.16.0.dist-info/RECORD,,\n\ - \n"; - let rows = parse_record_paths(text); - // Quoted CSV path with an embedded comma survives; the empty-field - // RECORD row and the blank line don't derail the parse. - assert_eq!( - rows, - ["six.py", "weird,name.py", "six-1.16.0.dist-info/RECORD"] - ); - // Emit side: a path needing quoting survives a parse round-trip. - let quoted = csv_quote("weird,\"name\".py"); - assert_eq!(parse_csv_record("ed)[0], "weird,\"name\".py"); - } - - #[test] - fn tag_compression_round_trips_and_rejects_non_cross_products() { - let dist = InstalledDist { - dist_info_dir: PathBuf::from("x"), - dist_name: "six".into(), - version: "1.16.0".into(), - record: vec![], - wheel_tags: vec!["py2-none-any".into(), "py3-none-any".into()], - }; - assert_eq!( - wheel_file_name(&dist).unwrap(), - "six-1.16.0-py2.py3-none-any.whl" - ); - - // A tag set that is NOT a cross product of its components must refuse - // rather than fabricate compatibility. - let err = - compress_wheel_tags(&["py2-none-any".into(), "py3-abi3-manylinux1_x86_64".into()]) - .unwrap_err(); - assert_eq!(err.0, "pypi_wheel_tags_unrecoverable"); - // Malformed (non-triple) tag. - let err = compress_wheel_tags(&["py3".into()]).unwrap_err(); - assert_eq!(err.0, "pypi_wheel_tags_unrecoverable"); - } - - #[test] - fn wheel_name_escapes_dist_info_stem_names() { - let dist = InstalledDist { - dist_info_dir: PathBuf::from("x"), - dist_name: "Flask-SQLAlchemy".into(), - version: "2.5.1".into(), - record: vec![], - wheel_tags: vec!["py3-none-any".into()], - }; - assert_eq!( - wheel_file_name(&dist).unwrap(), - "Flask_SQLAlchemy-2.5.1-py3-none-any.whl" - ); - } - - #[tokio::test] - async fn locate_finds_dist_with_canonicalized_name_and_parses_metadata() { - let fx = make_fixture("", None).await; - // PEP 503: `SIX` and `six` collapse to the same name. - let dist = locate_installed_dist(&fx.site_packages, "SIX", "1.16.0") - .await - .unwrap(); - assert_eq!(dist.dist_name, "six"); - assert_eq!(dist.version, "1.16.0"); - assert_eq!(dist.wheel_tags, vec!["py2-none-any", "py3-none-any"]); - assert!(dist.record.iter().any(|r| r == "six.py")); - - let err = locate_installed_dist(&fx.site_packages, "six", "1.17.0") - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_dist_not_found"); - } - - #[tokio::test] - async fn locate_refuses_missing_record_and_missing_wheel_metadata() { - let fx = make_fixture("", None).await; - let di = fx.site_packages.join("six-1.16.0.dist-info"); - - let wheel_backup = tokio::fs::read(di.join("WHEEL")).await.unwrap(); - tokio::fs::remove_file(di.join("WHEEL")).await.unwrap(); - let err = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_wheel_metadata"); - tokio::fs::write(di.join("WHEEL"), wheel_backup) - .await - .unwrap(); - - tokio::fs::remove_file(di.join("RECORD")).await.unwrap(); - let err = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_record"); - } - - #[tokio::test] - async fn member_filter_excludes_bookkeeping_and_console_scripts() { - // Console script `six-cmd` lives out of tree but is declared in - // entry_points.txt — excluded, not refused. RECORD signature files, - // INSTALLER, pyc files all drop out. - let fx = make_fixture( - "../../../bin/six-cmd,sha256=cc,99\n\ - ../../../bin/six-cmd.exe,,\n\ - six-1.16.0.dist-info/RECORD.jws,,\n\ - six-1.16.0.dist-info/entry_points.txt,sha256=dd,40\n", - Some("[console_scripts]\nsix-cmd = six:main\n"), - ) - .await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - let artifact = artifact.unwrap(); - let bytes = tokio::fs::read(&fx.dest).await.unwrap(); - assert_eq!(artifact.size, bytes.len() as u64); - let names = zip_names(&bytes); - assert!(names.contains(&"six.py".to_string())); - assert!(names.contains(&"six-1.16.0.dist-info/METADATA".to_string())); - assert!(names.contains(&"six-1.16.0.dist-info/entry_points.txt".to_string())); - for forbidden in [ - "six-1.16.0.dist-info/INSTALLER", - "six-1.16.0.dist-info/RECORD.jws", - "__pycache__/six.cpython-314.pyc", - "../../../bin/six-cmd", - ] { - assert!( - !names.contains(&forbidden.to_string()), - "{forbidden} leaked" - ); - } - // Patched bytes actually landed in the wheel. - assert_eq!(zip_file(&bytes, "six.py"), PATCHED); - } - - /// Equivalence: keeping the installed tree's members in memory must - /// rebuild the EXACT wheel bytes the stage-everything build produced — - /// the emitted `--hash` / uv lock pin rides on them. Driven twice over - /// one fixture (an exec-bit member, a zero-length member, a nested tree, - /// a member large enough to span several read buffers, a patched member - /// and a created one), once with the in-memory path forced off. - #[tokio::test] - #[serial_test::serial] - async fn in_memory_wheel_build_matches_the_on_disk_build_byte_for_byte() { - async fn build(on_disk: bool) -> (Vec, WheelArtifact) { - let _forced = on_disk.then(crate::vendor::common::OnDiskRepackGuard::acquire); - let before = crate::vendor::common::in_memory_repacks(); - let fx = make_fixture( - "six/__init__.py,sha256=CC,10\n\ - six/empty.py,,0\n\ - six/data/table.bin,sha256=DD,8\n\ - six-script.sh,sha256=EE,20\n", - None, - ) - .await; - tokio::fs::create_dir_all(fx.site_packages.join("six/data")) - .await - .unwrap(); - tokio::fs::write(fx.site_packages.join("six/__init__.py"), b"# pkg\n") - .await - .unwrap(); - tokio::fs::write(fx.site_packages.join("six/empty.py"), b"") - .await - .unwrap(); - tokio::fs::write( - fx.site_packages.join("six/data/table.bin"), - vec![7u8; 2 * 1024 * 1024], - ) - .await - .unwrap(); - let script = fx.site_packages.join("six-script.sh"); - tokio::fs::write(&script, b"#!/bin/sh\nexit 0\n") - .await - .unwrap(); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - std::fs::set_permissions(&script, std::fs::Permissions::from_mode(0o755)).unwrap(); - } - - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = - patch_record(&[("six.py", ORIG, PATCHED), ("six/created.py", b"", PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - // Without this the comparison is vacuous: a fixture member name - // the gate later rejects would stage BOTH runs in full and the - // test would keep passing while asserting nothing. - assert_eq!( - crate::vendor::common::in_memory_repacks() > before, - !on_disk, - "this run took the wrong staging path (on_disk={on_disk})" - ); - ( - tokio::fs::read(&fx.dest).await.unwrap(), - artifact.expect("a successful build yields an artifact"), - ) - } - - let (fast, fast_artifact) = build(false).await; - let (oracle, oracle_artifact) = build(true).await; - assert_eq!( - fast, oracle, - "the in-memory build must be byte-identical to the staged one" - ); - assert_eq!(fast_artifact, oracle_artifact, "and so must the lock pin"); - assert_eq!(zip_file(&fast, "six.py"), PATCHED); - assert_eq!(zip_file(&fast, "six/created.py"), PATCHED); - assert_eq!(zip_file(&fast, "six/empty.py"), b""); - assert_eq!( - zip_file(&fast, "six/data/table.bin"), - vec![7u8; 2 * 1024 * 1024] - ); - #[cfg(unix)] - { - assert_eq!(zip_unix_mode(&fast, "six-script.sh") & 0o777, 0o755); - assert_eq!(zip_unix_mode(&fast, "six.py") & 0o777, 0o644); - } - assert_eq!( - zip_names(&fast).last().map(String::as_str), - Some("six-1.16.0.dist-info/RECORD"), - "RECORD stays last" - ); - } - - /// A patch key that names a DIRECTORY of the installed tree must reach - /// the verify as a directory on both staging paths: a fully staged tree - /// carried one wherever a member lived under it, so the in-memory staging - /// materialises exactly those. Without it `--force` would read - /// "File not found" and silently skip the key instead of refusing to hash - /// a directory. - #[tokio::test] - #[serial_test::serial] - async fn patch_key_naming_a_directory_verifies_the_same_on_both_staging_paths() { - async fn verify_message(on_disk: bool) -> String { - let _forced = on_disk.then(crate::vendor::common::OnDiskRepackGuard::acquire); - let fx = make_fixture("six/__init__.py,sha256=CC,10\n", None).await; - tokio::fs::create_dir_all(fx.site_packages.join("six")) - .await - .unwrap(); - tokio::fs::write(fx.site_packages.join("six/__init__.py"), b"# pkg\n") - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, _) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - /*dry_run=*/ true, - /*force=*/ true, - &mut Vec::new(), - ) - .await - .unwrap(); - result - .files_verified - .iter() - .find(|v| v.file == "six") - .and_then(|v| v.message.clone()) - .unwrap_or_default() - } - - for (label, message) in [ - ("in memory", verify_message(false).await), - ("on disk", verify_message(true).await), - ] { - assert!( - message.starts_with("Failed to hash file"), - "{label}: a key naming a directory must refuse to hash, got {message:?}" - ); - } - } - - /// A member name a filesystem could re-spell (here: non-ASCII, which a - /// normalising volume folds) sends the build back to the stage-everything - /// path. It must still produce a correct wheel. - #[tokio::test] - async fn a_respellable_member_name_still_builds_through_the_staged_path() { - let fx = make_fixture("six/caf\u{e9}.txt,sha256=FF,6\n", None).await; - tokio::fs::create_dir_all(fx.site_packages.join("six")) - .await - .unwrap(); - tokio::fs::write(fx.site_packages.join("six/caf\u{e9}.txt"), b"latte\n") - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - let bytes = tokio::fs::read(&fx.dest).await.unwrap(); - assert_eq!(zip_file(&bytes, "six.py"), PATCHED); - assert_eq!(zip_file(&bytes, "six/caf\u{e9}.txt"), b"latte\n"); - } - - #[tokio::test] - async fn out_of_tree_data_file_is_refused() { - // `share/man/...` is a wheel .data payload, NOT a console script — - // name-stem heuristics must not swallow it. - let fx = make_fixture( - "../../../share/man/man6/six.6,sha256=ee,10\n", - Some("[console_scripts]\nsix-cmd = six:main\n"), - ) - .await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let err = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_out_of_tree_files"); - assert!(err.1.contains("share/man/man6/six.6"), "{}", err.1); - assert!(!fx.dest.exists(), "refusal must not write the artifact"); - } - - #[tokio::test] - async fn deterministic_zip_record_last_and_stable_across_builds() { - let fx = make_fixture("", None).await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (r1, a1) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(r1.success); - let bytes1 = tokio::fs::read(&fx.dest).await.unwrap(); - - // Second build: the stage re-applies onto already-patched members - // (AlreadyPatched verify) — bytes and hash must be identical. - let (r2, a2) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(r2.success); - let bytes2 = tokio::fs::read(&fx.dest).await.unwrap(); - assert_eq!(bytes1, bytes2, "wheel rebuild must be byte-deterministic"); - assert_eq!(a1.unwrap().sha256_hex, a2.unwrap().sha256_hex); - - // RECORD is the final zip entry and self-describes with `path,,`. - let names = zip_names(&bytes1); - assert_eq!( - names.last().map(String::as_str), - Some("six-1.16.0.dist-info/RECORD") - ); - let record_text = - String::from_utf8(zip_file(&bytes1, "six-1.16.0.dist-info/RECORD")).unwrap(); - assert!(record_text.ends_with("six-1.16.0.dist-info/RECORD,,\n")); - // RECORD hash of six.py matches the patched bytes. - let digest = sha2::Sha256::digest(PATCHED); - let b64 = base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(digest); - assert!( - record_text.contains(&format!("six.py,sha256={},{}", b64, PATCHED.len())), - "{record_text}" - ); - } - - #[tokio::test] - async fn created_by_patch_file_is_unioned_into_the_wheel() { - let fx = make_fixture("", None).await; - let created = b"# brand new module\n"; - tokio::fs::write( - fx.blobs.join(compute_git_sha256_from_bytes(created)), - created, - ) - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED), ("six_extra.py", b"", created)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, _) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - let bytes = tokio::fs::read(&fx.dest).await.unwrap(); - assert!(zip_names(&bytes).contains(&"six_extra.py".to_string())); - assert_eq!(zip_file(&bytes, "six_extra.py"), created); - let record_text = - String::from_utf8(zip_file(&bytes, "six-1.16.0.dist-info/RECORD")).unwrap(); - assert!(record_text.contains("six_extra.py,sha256=")); - // The created file must NOT exist in the real site-packages. - assert!(!fx.site_packages.join("six_extra.py").exists()); - } - - #[tokio::test] - async fn editable_install_is_refused_before_staging() { - let fx = make_fixture("", None).await; - tokio::fs::write( - fx.site_packages - .join("six-1.16.0.dist-info/direct_url.json"), - r#"{"url": "file:///work/six", "dir_info": {"editable": true}}"#, - ) - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let err = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_editable_install"); - } - - #[tokio::test] - async fn dry_run_verifies_but_writes_nothing() { - let fx = make_fixture("", None).await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - true, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success); - assert!(artifact.is_none()); - assert!(!fx.dest.exists()); - // Installed tree untouched. - assert_eq!( - tokio::fs::read(fx.site_packages.join("six.py")) - .await - .unwrap(), - ORIG - ); - } - - /// Vendor auto-force policy: installed content matching NEITHER hash is - /// overwritten with the verified patched content in the STAGE (the - /// installed tree is never touched), and the overwrite is surfaced as a - /// `vendor_content_mismatch_overwritten` warning. - #[tokio::test] - async fn hash_mismatch_overwrites_in_stage_with_warning() { - let fx = make_fixture("", None).await; - // Corrupt the installed six.py so verify sees a HashMismatch. - tokio::fs::write(fx.site_packages.join("six.py"), b"tampered") - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let mut warnings = Vec::new(); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut warnings, - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - assert!(fx.dest.exists(), "patched wheel must be written"); - assert_eq!( - warnings - .iter() - .filter(|w| w.code == "vendor_content_mismatch_overwritten") - .count(), - 1, - "overwrite surfaced as a warning: {warnings:?}" - ); - // Installed tree untouched — only the stage was overwritten. - assert_eq!( - tokio::fs::read(fx.site_packages.join("six.py")) - .await - .unwrap(), - b"tampered" - ); - } - - /// A patch-target file MISSING from the install still fails closed - /// without `--force` — auto-force must not inherit force's silent - /// NotFound skip (the wheel would ship without the fix). - #[tokio::test] - async fn missing_patch_file_fails_without_force() { - let fx = make_fixture("", None).await; - tokio::fs::remove_file(fx.site_packages.join("six.py")) - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(!result.success); - // The RECORD staging step trips first ("RECORD member ... is - // unreadable") — either way the build fails closed rather than - // packing a wheel without the fix. - assert!( - result.error.is_some(), - "missing file fails closed with an error" - ); - assert!(artifact.is_none()); - assert!(!fx.dest.exists()); - } - - #[test] - fn console_script_artifact_matching_is_name_exact() { - let names: HashSet = ["pycowsay".to_string()].into_iter().collect(); - assert!(is_console_script_artifact("pycowsay", &names)); - assert!(is_console_script_artifact("pycowsay.exe", &names)); - assert!(is_console_script_artifact("pycowsay-script.py", &names)); - // A splitext-style stem heuristic's trap: `pycowsay.6` (a man page) - // must NOT match. - assert!(!is_console_script_artifact("pycowsay.6", &names)); - assert!(!is_console_script_artifact("other", &names)); - } - - /// PEP 440 normalized versions legitimately carry `!` (epoch) and `+` - /// (local separator), and real tools keep them literally in the wheel - /// filename (bdist_wheel's `safer_version`). pip/uv REJECT the - /// `_`-escaped spelling: `packaging.utils.parse_wheel_filename` raises - /// InvalidWheelFilename on `torch-2.0.0_cu118-…` but parses - /// `torch-2.0.0+cu118-…` — escaping them makes the vendored wheel - /// uninstallable. - #[test] - fn wheel_version_keeps_local_and_epoch_separators() { - let dist = InstalledDist { - dist_info_dir: PathBuf::from("x"), - dist_name: "torch".into(), - version: "2.0.0+cu118".into(), - record: vec![], - wheel_tags: vec!["cp310-cp310-linux_x86_64".into()], - }; - assert_eq!( - wheel_file_name(&dist).unwrap(), - "torch-2.0.0+cu118-cp310-cp310-linux_x86_64.whl" - ); - - let dist = InstalledDist { - dist_info_dir: PathBuf::from("x"), - dist_name: "pkg".into(), - version: "1!2.0".into(), - record: vec![], - wheel_tags: vec!["py3-none-any".into()], - }; - assert_eq!( - wheel_file_name(&dist).unwrap(), - "pkg-1!2.0-py3-none-any.whl" - ); - } - - /// `mkfifo(2)` directly instead of shelling out to the binary — the - /// same helper as the sibling vendor FIFO tests: fork/exec flakes under - /// heavy parallel load and the syscall needs no process at all. - #[cfg(unix)] - fn mkfifo(path: &Path) { - use std::os::unix::ffi::OsStrExt; - let c_path = - std::ffi::CString::new(path.as_os_str().as_bytes()).expect("fifo path has no NUL"); - let rc = unsafe { libc::mkfifo(c_path.as_ptr(), 0o644) }; - assert_eq!( - rc, - 0, - "mkfifo(2) failed: {}", - std::io::Error::last_os_error() - ); - } - - /// Await `fut` with a deadline. On timeout the open is wedged in a - /// `spawn_blocking` thread that the runtime waits for on shutdown; - /// connect a writer to release it so the test can FAIL instead of - /// hanging the whole suite. (`timeout` drops the future itself, so - /// nothing else keeps running after the release.) - #[cfg(unix)] - async fn expect_prompt(fifo: &Path, fut: impl std::future::Future) -> T { - let deadline = std::time::Duration::from_secs(5); - match tokio::time::timeout(deadline, fut).await { - Ok(v) => v, - Err(_) => { - let _ = std::fs::OpenOptions::new().write(true).open(fifo); - panic!( - "must complete promptly with a FIFO at {} instead of wedging in open(2)", - fifo.display() - ); - } - } - } - - /// A FIFO planted as RECORD or WHEEL must fail fast as the existing - /// unreadable-file refusal, not wedge `locate_installed_dist` (and the - /// whole vendor run) forever in an `open(2)` waiting for a writer. - #[cfg(unix)] - #[tokio::test] - async fn fifo_record_or_wheel_does_not_wedge_locate() { - let fx = make_fixture("", None).await; - let di = fx.site_packages.join("six-1.16.0.dist-info"); - - let record_backup = tokio::fs::read(di.join("RECORD")).await.unwrap(); - tokio::fs::remove_file(di.join("RECORD")).await.unwrap(); - mkfifo(&di.join("RECORD")); - let err = expect_prompt( - &di.join("RECORD"), - locate_installed_dist(&fx.site_packages, "six", "1.16.0"), - ) - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_record"); - - tokio::fs::remove_file(di.join("RECORD")).await.unwrap(); - tokio::fs::write(di.join("RECORD"), record_backup) - .await - .unwrap(); - tokio::fs::remove_file(di.join("WHEEL")).await.unwrap(); - mkfifo(&di.join("WHEEL")); - let err = expect_prompt( - &di.join("WHEEL"), - locate_installed_dist(&fx.site_packages, "six", "1.16.0"), - ) - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_wheel_metadata"); - } - - /// A FIFO squatting a RECORD member must fail the build closed (same as - /// an unreadable member), not wedge the staging loop forever. - #[cfg(unix)] - #[tokio::test] - async fn fifo_record_member_fails_closed_instead_of_wedging_build() { - let fx = make_fixture("", None).await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - tokio::fs::remove_file(fx.site_packages.join("six.py")) - .await - .unwrap(); - mkfifo(&fx.site_packages.join("six.py")); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = expect_prompt( - &fx.site_packages.join("six.py"), - build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ), - ) - .await - .unwrap(); - assert!(!result.success); - assert!( - result.error.as_deref().unwrap_or("").contains("unreadable"), - "{:?}", - result.error - ); - assert!(artifact.is_none()); - assert!(!fx.dest.exists()); - } - - /// FIFOs planted as the optional dist-info extras (`direct_url.json`, - /// `entry_points.txt` — both probed with fall-through-on-error reads) - /// must read as absent and let the build succeed, not wedge it. - #[cfg(unix)] - #[tokio::test] - async fn fifo_direct_url_json_does_not_wedge_editable_probe() { - let fx = make_fixture("", None).await; - let fifo = fx - .site_packages - .join("six-1.16.0.dist-info/direct_url.json"); - mkfifo(&fifo); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = expect_prompt( - &fifo, - build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - } - - #[cfg(unix)] - #[tokio::test] - async fn fifo_entry_points_does_not_wedge_build() { - let fx = make_fixture("", None).await; - let fifo = fx - .site_packages - .join("six-1.16.0.dist-info/entry_points.txt"); - mkfifo(&fifo); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = expect_prompt( - &fifo, - build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - } - - /// Locate-side degraded shapes: an undecodable `*.dist-info` entry is - /// skipped (the real dist is still found), a RECORD that parses to zero - /// member paths is refused, and a WHEEL whose only `Tag:` header is - /// whitespace-valued is refused. - #[tokio::test] - async fn locate_skips_undecodable_dist_info_and_refuses_empty_record_and_tagless_wheel() { - let fx = make_fixture("", None).await; - // A regular FILE named like a dist-info: METADATA is unreadable - // beneath it and the dir-name fallback rejects non-directories, so - // read_python_metadata yields None and the entry is skipped. (An - // empty dist-info DIRECTORY would instead be rescued by the - // crawler's dir-name fallback and never exercise the skip.) - tokio::fs::write(fx.site_packages.join("stray-1.0.dist-info"), b"not a dir") - .await - .unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - assert_eq!(dist.dist_name, "six"); - - // RECORD present but degenerate: blank / whitespace-only lines and - // an empty-path row all drop out of the parse, leaving no members. - let di = fx.site_packages.join("six-1.16.0.dist-info"); - let record_backup = tokio::fs::read(di.join("RECORD")).await.unwrap(); - tokio::fs::write(di.join("RECORD"), "\n \n,,\n") - .await - .unwrap(); - let err = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_record"); - assert!(err.1.contains("lists no files"), "{}", err.1); - - // WHEEL present but its only Tag: value is whitespace, which the - // non-empty filter drops — same refusal as a missing WHEEL. - tokio::fs::write(di.join("RECORD"), record_backup) - .await - .unwrap(); - tokio::fs::write( - di.join("WHEEL"), - "Wheel-Version: 1.0\nRoot-Is-Purelib: true\nTag: \n", - ) - .await - .unwrap(); - let err = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap_err(); - assert_eq!(err.0, "pypi_missing_wheel_metadata"); - assert!(err.1.contains("no Tag: headers"), "{}", err.1); - } - - /// A noncompliant dist-info stem carrying no version part - /// (`six.dist-info`): `rfind('-')` misses, so `dist_name` falls back to - /// the METADATA raw name — which then drives the rebuilt wheel filename. - #[tokio::test] - async fn locate_versionless_dist_info_stem_falls_back_to_metadata_name() { - let tmp = tempfile::tempdir().unwrap(); - let sp = tmp.path().join("site-packages"); - let di = sp.join("six.dist-info"); - tokio::fs::create_dir_all(&di).await.unwrap(); - tokio::fs::write(sp.join("six.py"), ORIG).await.unwrap(); - tokio::fs::write( - di.join("METADATA"), - "Metadata-Version: 2.1\nName: six\nVersion: 1.16.0\n\n", - ) - .await - .unwrap(); - tokio::fs::write(di.join("WHEEL"), "Wheel-Version: 1.0\nTag: py3-none-any\n") - .await - .unwrap(); - tokio::fs::write( - di.join("RECORD"), - "six.py,sha256=AAAA,20\nsix.dist-info/METADATA,,\n", - ) - .await - .unwrap(); - - let dist = locate_installed_dist(&sp, "six", "1.16.0").await.unwrap(); - assert_eq!(dist.dist_name, "six"); - assert_eq!( - wheel_file_name(&dist).unwrap(), - "six-1.16.0-py3-none-any.whl" - ); - } - - /// An installed RECORD member with the exec bit set must come back out - /// of the rebuilt wheel as unix mode 0o755 (a package script losing +x - /// breaks the reinstalled dist), non-executables as 0o644 — and the - /// exec bit must not break byte determinism. - #[cfg(unix)] - #[tokio::test] - async fn executable_record_member_keeps_exec_bit_in_wheel() { - use std::os::unix::fs::PermissionsExt as _; - - let fx = make_fixture("six_cli.py,sha256=cc,10\n", None).await; - let cli = fx.site_packages.join("six_cli.py"); - tokio::fs::write(&cli, b"#!/usr/bin/env python3\n") - .await - .unwrap(); - std::fs::set_permissions(&cli, std::fs::Permissions::from_mode(0o755)).unwrap(); - - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (r1, a1) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(r1.success, "{:?}", r1.error); - assert!(a1.is_some()); - let bytes1 = tokio::fs::read(&fx.dest).await.unwrap(); - assert_eq!( - zip_unix_mode(&bytes1, "six_cli.py") & 0o777, - 0o755, - "installed exec bit must survive into the wheel" - ); - assert_eq!( - zip_unix_mode(&bytes1, "six.py") & 0o777, - 0o644, - "non-executable members stay 0o644" - ); - - let (r2, _) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(r2.success, "{:?}", r2.error); - let bytes2 = tokio::fs::read(&fx.dest).await.unwrap(); - assert_eq!(bytes1, bytes2, "exec bit must not break determinism"); - } - - /// Dest-side write failures after a SUCCESSFUL apply flip the result to - /// an Ok-shaped FAILED ApplyResult (success=false, artifact=None) — not - /// an Err refusal. Both shapes are root-proof squatter states: a regular - /// file blocking a dest path component, and a directory occupying dest - /// itself. - #[tokio::test] - async fn dest_write_failures_fail_closed_with_ok_shaped_failed_result() { - let fx = make_fixture("", None).await; - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - - // A regular file squatting a dest path component: create_dir_all of - // dest's parent fails with ENOTDIR. - let blocker = fx._tmp.path().join("blocker"); - tokio::fs::write(&blocker, b"file").await.unwrap(); - let dest_a = blocker.join("sub").join("x.whl"); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &dest_a, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(!result.success); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .starts_with("cannot create "), - "{:?}", - result.error - ); - assert!(artifact.is_none()); - - // Dest itself occupied by a directory: the atomic rename fails and - // the squatter is left alone (never unlinked first). - let dest_b = fx._tmp.path().join("out").join("x.whl"); - tokio::fs::create_dir_all(&dest_b).await.unwrap(); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &dest_b, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(!result.success); - assert!( - result - .error - .as_deref() - .unwrap_or("") - .starts_with("cannot write "), - "{:?}", - result.error - ); - assert!(artifact.is_none()); - assert!( - tokio::fs::metadata(&dest_b).await.unwrap().is_dir(), - "the squatting directory must be left alone" - ); - } - - /// The editable-install probe fails OPEN on a `direct_url.json` that is - /// not valid JSON (deliberate, matching the missing-file twin) and on - /// valid JSON with no `dir_info` — both read as not-editable and the - /// build proceeds. - #[tokio::test] - async fn malformed_direct_url_json_reads_as_non_editable() { - let fx = make_fixture("", None).await; - let du = fx - .site_packages - .join("six-1.16.0.dist-info/direct_url.json"); - tokio::fs::write(&du, b"{ not json").await.unwrap(); - let dist = locate_installed_dist(&fx.site_packages, "six", "1.16.0") - .await - .unwrap(); - let record = patch_record(&[("six.py", ORIG, PATCHED)]); - let sources = PatchSources::blobs_only(&fx.blobs); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - assert!(fx.dest.exists()); - - // Valid JSON, no dir_info: the unwrap_or(false) chain also reads - // as not-editable. - tokio::fs::write(&du, r#"{"url": "https://pypi.org/simple/six"}"#) - .await - .unwrap(); - let (result, artifact) = build_patched_wheel( - "pkg:pypi/six@1.16.0", - &fx.site_packages, - &dist, - &record, - &sources, - &fx.dest, - false, - false, - &mut Vec::new(), - ) - .await - .unwrap(); - assert!(result.success, "{:?}", result.error); - assert!(artifact.is_some()); - } - - /// entry_points.txt parser fall-throughs and script-artifact negatives: - /// an in-section line without `=`, an empty-name row, a non-script - /// section whose entries never reach the insert, and suffix strips - /// whose stem is not a declared script. - #[test] - fn entry_points_parser_and_script_artifact_negative_edges() { - let names = console_script_names( - "[console_scripts]\n\ - junk line without equals\n\ - = empty-name\n\ - six-cmd = six:main\n\ - [flake8.extension]\n\ - not-a-script = x\n", - ); - assert_eq!( - names, - ["six-cmd".to_string()].into_iter().collect::>() - ); - - let declared: HashSet = ["pycowsay".to_string()].into_iter().collect(); - assert!(!is_console_script_artifact("other.exe", &declared)); - assert!(!is_console_script_artifact("other-script.py", &declared)); - } -} diff --git a/crates/socket-patch-core/src/vendor/redownload.rs b/crates/socket-patch-core/src/vendor/redownload.rs new file mode 100644 index 000000000..99e32ca33 --- /dev/null +++ b/crates/socket-patch-core/src/vendor/redownload.rs @@ -0,0 +1,779 @@ +use std::path::Path; + +use sha2::{Digest, Sha256}; + +use crate::manifest::schema::PatchRecord; +use crate::utils::purl::{ + parse_cargo_purl, parse_composer_purl, parse_gem_purl, parse_golang_purl, parse_maven_purl, +}; + +use super::common::{copy_matches_after_hashes, swap_stage_into_place}; +use super::service_fetch::{ + fetch_verified_archive, ServiceAttempt, ServicePolicy, ServiceTerminal, VerifiedArchive, +}; +use super::state::VendorEntry; +use super::{VendorOutcome, VendorServiceConfig, VendorWarning}; + +fn detail(outcome: VendorOutcome) -> String { + match outcome { + VendorOutcome::Refused { code, detail } => format!("{code}: {detail}"), + VendorOutcome::Done { result, .. } => result + .error + .unwrap_or_else(|| "artifact download failed".into()), + } +} + +fn used(attempt: ServiceAttempt) -> Result { + match attempt { + ServiceAttempt::Used(value) => Ok(value), + ServiceAttempt::HardFail(outcome) => Err(detail(*outcome)), + } +} + +async fn download_archive( + service: &VendorServiceConfig, + record: &PatchRecord, + noun: &str, + subject: &str, + warnings: &mut Vec, +) -> Result { + ServicePolicy::new(service, ServiceTerminal::Refused) + .settle( + fetch_verified_archive(service, &record.uuid).await, + noun, + subject, + warnings, + ) + .or_else(used) +} + +/// Restore only the recorded artifact. Project wiring and ledger are never written. +pub async fn restore( + root: &Path, + entry: &VendorEntry, + record: &PatchRecord, + service: &VendorServiceConfig, +) -> Result, String> { + let ecosystem = super::ecosystem_dir_for_purl(&entry.base_purl); + if ecosystem != Some(entry.ecosystem.as_str()) + && !(ecosystem == Some("maven") && entry.ecosystem == "jvm") + { + return Err("ledger package identity does not match its artifact ecosystem".into()); + } + if let Some(outcome) = super::common::service_offline_conflict(Some(service)) { + return Err(detail(outcome)); + } + let artifact = match super::verify::checked_artifact_path(root, entry, record) { + Ok(path) => path, + Err(reason) + if entry.ecosystem == "jvm" + && matches!( + reason.as_str(), + "vendor_artifact_missing" | "vendor_artifact_unreadable" + ) => + { + root.join(super::jvm::apply::checked_tree_jar_path( + root, + entry, + &record.uuid, + )?) + } + Err(reason) => return Err(reason), + }; + let mut cursor = root.to_path_buf(); + for part in entry.artifact.path.split('/') { + cursor.push(part); + if tokio::fs::symlink_metadata(&cursor) + .await + .is_ok_and(|m| m.file_type().is_symlink()) + { + return Err("vendor_path_unsafe: artifact path contains a symlink".into()); + } + } + let file_shaped = !super::verify::is_vlt_dir_entry(entry) + && super::verify::artifact_is_file_shaped(&entry.artifact.path); + if file_shaped && entry.artifact.sha256.is_empty() { + return Err("the ledger has no archive SHA-256; restore from version control or explicitly re-vendor".into()); + } + if !file_shaped && entry.artifact.file_inventory.is_none() { + return Err("the ledger has no complete file inventory; restore from version control or explicitly re-vendor".into()); + } + let socket = root.join(".socket"); + tokio::fs::create_dir_all(&socket) + .await + .map_err(|e| e.to_string())?; + let temporary = tempfile::Builder::new() + .prefix(".artifact-download-") + .tempdir_in(&socket) + .map_err(|e| e.to_string())?; + let stage = temporary.path().join(&entry.artifact.path); + let uuid_dir = stage.parent().ok_or("artifact has no parent")?; + let mut warnings = Vec::new(); + if file_shaped { + let archive = download_archive( + service, + record, + "archive", + &format!("archive for {}", entry.base_purl), + &mut warnings, + ) + .await?; + if entry.artifact.sha256.is_empty() + || !hex::encode(Sha256::digest(&archive.bytes)) + .eq_ignore_ascii_case(&entry.artifact.sha256) + { + return Err("the downloaded artifact does not match the recorded SHA-256; the existing artifact, lockfiles and ledger were preserved".into()); + } + if entry + .artifact + .size + .is_some_and(|size| size != archive.bytes.len() as u64) + { + return Err("the downloaded artifact does not match the recorded size".into()); + } + let members = if entry.ecosystem == "pypi" { + super::pypi_distribution::read_members(&archive.bytes, &entry.artifact.path)? + } else if entry.artifact.path.ends_with(".tgz") || entry.artifact.path.ends_with(".tar.gz") + { + crate::patch::package::read_archive_bytes_to_map_strict(&archive.bytes) + .map_err(|e| e.to_string())? + } else { + super::verify::read_zip_bytes_to_map_strict(&archive.bytes)? + }; + if entry.ecosystem == "pypi" { + super::pypi_distribution::verify_members(&members, &entry.artifact.path, record)?; + } else { + super::verify::verify_member_map(&members, record)?; + } + tokio::fs::create_dir_all(uuid_dir) + .await + .map_err(|e| e.to_string())?; + crate::utils::fs::atomic_write_artifact(&stage, &archive.bytes) + .await + .map_err(|e| e.to_string())?; + if entry.ecosystem == "maven" || entry.ecosystem == "jvm" { + restore_maven_metadata( + root, + temporary.path(), + entry, + record, + &archive.bytes, + service, + ) + .await?; + } + } else { + match entry.ecosystem.as_str() { + "cargo" => { + let (name, version) = + parse_cargo_purl(&entry.base_purl).ok_or("invalid cargo coordinates")?; + used( + super::cargo::cargo_service_copy( + Some(service), + record, + &name, + &version, + &stage, + uuid_dir, + &mut warnings, + ) + .await, + )?; + } + "composer" => { + let ((namespace, name), _) = + parse_composer_purl(&entry.base_purl).ok_or("invalid composer coordinates")?; + let package = format!("{namespace}/{name}"); + used( + super::composer_lock::composer_service_copy( + Some(service), + record, + &package, + &stage, + uuid_dir, + &mut warnings, + ) + .await, + )?; + super::composer_lock::mirror_filters::neutralize_or_conflict( + &stage, + record, + &package, + &mut warnings, + ) + .await?; + } + "gem" => { + let (name, _) = + parse_gem_purl(&entry.base_purl).ok_or("invalid gem coordinates")?; + match super::gem::gem_service_copy( + Some(service), + record, + &name, + &stage, + uuid_dir, + true, + &mut warnings, + ) + .await + { + super::gem::GemServiceCopy::Used => {} + super::gem::GemServiceCopy::HardFail(outcome) => return Err(detail(*outcome)), + } + } + "npm" => { + let (name, version) = super::npm_common::parse_npm_purl(&entry.base_purl) + .ok_or("invalid npm coordinates")?; + used( + super::npm_dir::try_service_dir( + &entry.base_purl, + record, + service, + &stage, + &name, + &version, + &mut warnings, + ) + .await, + )?; + super::npm_dir::apply_transforms(&stage, &name, &version) + .await + .map_err(|outcome| detail(*outcome))?; + } + "golang" => { + let (module, version) = + parse_golang_purl(&entry.base_purl).ok_or("invalid Go coordinates")?; + let archive = download_archive( + service, + record, + "module zip", + &format!("module zip for {module}"), + &mut warnings, + ) + .await?; + let prefix = format!("{module}@{version}/"); + tokio::fs::create_dir_all(&stage) + .await + .map_err(|e| e.to_string())?; + super::registry_fetch::extract_on_blocking_pool( + archive.bytes, + &stage, + move |bytes, dest| { + super::registry_fetch::extract_zip_with_prefix(bytes, dest, &prefix) + }, + ) + .await?; + crate::patch::redirect::golang_local::ensure_module_go_mod(&stage, &module) + .await + .map_err(|e| e.to_string())?; + if !copy_matches_after_hashes(&stage, &record.files).await { + return Err("server module does not carry the patched files".into()); + } + } + _ => { + return Err(format!( + "unsupported artifact ecosystem {}", + entry.ecosystem + )) + } + } + { + let inventory = entry.artifact.file_inventory.as_ref().ok_or("the ledger has no complete file inventory; restore from version control or explicitly re-vendor")?; + let uuid = (entry.ecosystem == "cargo").then_some(entry.uuid.as_str()); + super::verify::verify_dir_inventory(&stage, inventory, uuid).await?; + } + } + if entry.ecosystem == "maven" || entry.ecosystem == "jvm" { + let target = artifact.parent().ok_or("artifact has no parent")?; + tokio::fs::create_dir_all(target.parent().ok_or("artifact tree has no parent")?) + .await + .map_err(|e| e.to_string())?; + swap_stage_into_place(uuid_dir, target) + .await + .map_err(|e| e.to_string())?; + } else { + if let Some(parent) = artifact.parent() { + tokio::fs::create_dir_all(parent) + .await + .map_err(|e| e.to_string())?; + } + let original_uuid = + super::path::vendor_uuid_dir_rel("npm", &entry.uuid).map(|rel| root.join(rel)); + if let Some(original) = &original_uuid { + if let Some(warning) = + super::vlt_lock::keep_vlt_links(entry, original, temporary.path()).await + { + warnings.push(warning); + } + } + let swapped = if file_shaped { + let result = tokio::fs::rename(&stage, &artifact).await; + if result.is_ok() { + crate::utils::durability::moved(&stage, &artifact); + } + result + } else { + swap_stage_into_place(&stage, &artifact).await + }; + if let Err(error) = swapped { + if let Some(rel) = super::path::vendor_uuid_dir_rel("npm", &entry.uuid) { + let _ = + super::vlt_lock::keep_vlt_links(entry, &temporary.path().join(rel), root).await; + } + return Err(error.to_string()); + } + } + if super::verify::is_vlt_dir_entry(entry) { + super::vlt_lock::restore_vlt_uuid_metadata(entry, root) + .await + .map_err(|e| e.to_string())?; + } + Ok(warnings) +} + +async fn restore_maven_metadata( + root: &Path, + stage_root: &Path, + entry: &VendorEntry, + record: &PatchRecord, + jar: &[u8], + service: &VendorServiceConfig, +) -> Result<(), String> { + let (group, artifact, version) = + parse_maven_purl(&entry.base_purl).ok_or("invalid Maven coordinates")?; + let empty_cache = stage_root.join("upstream"); + let pom = super::maven_repo::acquire_jvm_metadata( + &empty_cache, + &group, + &artifact, + &version, + "pom", + Some(service), + ) + .await?; + let target = stage_root.join(&entry.artifact.path); + if entry.ecosystem == "maven" { + return super::maven_repo::write_maven_artifact( + target.parent().ok_or("artifact has no parent")?, + &format!("{artifact}-{version}.jar"), + jar, + &format!("{artifact}-{version}.pom"), + &pom, + ) + .await; + } + let module = if entry + .wiring + .iter() + .any(|w| w.kind == super::jvm::TREE_KIND && w.file.ends_with(".module")) + { + Some( + super::maven_repo::acquire_jvm_metadata( + &empty_cache, + &group, + &artifact, + &version, + "module", + Some(service), + ) + .await?, + ) + } else { + None + }; + let patch = super::jvm::JvmPatch { + group_id: &group, + artifact_id: &artifact, + version: &version, + uuid: &record.uuid, + jar, + upstream_pom: &pom, + upstream_module: module.as_deref(), + }; + let reader = super::jvm::apply::ProjectReader::new(root); + let read = |rel: &str| { + if rel.starts_with(".socket/vendor/") { + None + } else { + reader.read(rel) + } + }; + let shape = super::jvm::detect(&read); + let plan = if shape == super::jvm::Shape::MavenReactor { + let enabled = !entry + .wiring + .iter() + .any(|w| super::jvm::op_of(w) == "config_none"); + super::jvm::maven_reactor::plan_with_config(&read, &patch, enabled) + } else { + super::jvm::plan(shape, &read, &patch) + } + .map_err(|e| e.detail)?; + if reader.escaped().is_some() { + return Err("Maven project path escapes the checkout".into()); + } + let tree: Vec<_> = plan.writes.into_iter().filter(|w| w.tree).collect(); + if tree.len() + != entry + .wiring + .iter() + .filter(|w| w.kind == super::jvm::TREE_KIND) + .count() + { + return Err("download did not reproduce the complete recorded JVM tree".into()); + } + for write in tree { + let expected = entry + .wiring + .iter() + .find(|w| w.kind == super::jvm::TREE_KIND && w.file == write.rel) + .and_then(|w| w.new.as_ref()) + .and_then(serde_json::Value::as_str) + .ok_or("download produced an unrecorded JVM tree file")?; + if hex::encode(Sha256::digest(&write.bytes)) != expected { + return Err("downloaded JVM metadata does not match the recorded tree".into()); + } + let path = stage_root.join(&write.rel); + tokio::fs::create_dir_all(path.parent().ok_or("tree file has no parent")?) + .await + .map_err(|e| e.to_string())?; + crate::utils::fs::atomic_write_artifact(&path, &write.bytes) + .await + .map_err(|e| e.to_string())?; + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::hash::git_sha256::compute_git_sha256_from_bytes; + use crate::vendor::state::VendorArtifact; + use crate::vendor::test_support::{mount_granted, service_cfg, tree_snapshot}; + use crate::vendor::VendorSource; + use std::collections::HashMap; + + const UUID: &str = "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"; + + fn record() -> PatchRecord { + serde_json::from_value(serde_json::json!({ + "uuid": UUID, "exportedAt": "2026-09-30T00:00:00Z", + "files": {"index.js": {"beforeHash": "before", "afterHash": compute_git_sha256_from_bytes(b"patched")}}, + "vulnerabilities": {}, "description": "", "license": "", "tier": "" + })).unwrap() + } + + fn tgz(prefix: &str, extra: &[u8]) -> Vec { + use std::io::Write as _; + let mut tar = tar::Builder::new(Vec::new()); + for (name, bytes) in [("index.js", b"patched".as_slice()), ("extra", extra)] { + let mut header = tar::Header::new_gnu(); + header.set_size(bytes.len() as u64); + header.set_mode(0o644); + header.set_cksum(); + tar.append_data(&mut header, format!("{prefix}/{name}"), bytes) + .unwrap(); + } + let mut gz = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::default()); + gz.write_all(&tar.into_inner().unwrap()).unwrap(); + gz.finish().unwrap() + } + + fn entry(bytes: &[u8]) -> VendorEntry { + VendorEntry { + ecosystem: "npm".into(), + base_purl: "pkg:npm/example@1.0.0".into(), + uuid: UUID.into(), + artifact: VendorArtifact { + path: format!(".socket/vendor/npm/{UUID}/example-1.0.0.tgz"), + sha256: hex::encode(Sha256::digest(bytes)), + size: Some(bytes.len() as u64), + platform_locked: None, + file_inventory: None, + yarn_berry10c0: None, + }, + wiring: Vec::new(), + lock: None, + took_over_go_patches: false, + detached: true, + record: Some(record()), + flavor: Some("npm".into()), + uv: None, + pnpm: None, + poetry: None, + pdm: None, + pipenv: None, + } + } + + #[tokio::test] + async fn repair_redownloads_exact_bytes_without_changing_wiring_or_ledger() { + let root = tempfile::tempdir().unwrap(); + let bytes = tgz("package", b"original unpatched member"); + let entry = entry(&bytes); + let path = root.path().join(&entry.artifact.path); + tokio::fs::create_dir_all(path.parent().unwrap()) + .await + .unwrap(); + tokio::fs::write(&path, b"corrupt").await.unwrap(); + tokio::fs::write(root.path().join("package-lock.json"), b"original lock") + .await + .unwrap(); + tokio::fs::write( + root.path().join(".socket/vendor/state.json"), + serde_json::to_vec(&entry).unwrap(), + ) + .await + .unwrap(); + let server = wiremock::MockServer::start().await; + mount_granted(&server, UUID, "example-1.0.0.tgz", &bytes).await; + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); + restore(root.path(), &entry, &record(), &cfg).await.unwrap(); + assert_eq!(tokio::fs::read(&path).await.unwrap(), bytes); + assert_eq!( + tokio::fs::read(root.path().join("package-lock.json")) + .await + .unwrap(), + b"original lock" + ); + assert_eq!( + tokio::fs::read(root.path().join(".socket/vendor/state.json")) + .await + .unwrap(), + serde_json::to_vec(&entry).unwrap() + ); + tokio::fs::remove_file(&path).await.unwrap(); + restore(root.path(), &entry, &record(), &cfg).await.unwrap(); + assert_eq!(tokio::fs::read(path).await.unwrap(), bytes); + } + + #[tokio::test] + async fn missing_archive_digest_refuses_without_downloading_or_mutating() { + let root = tempfile::tempdir().unwrap(); + let bytes = tgz("package", b"pinned"); + let mut entry = entry(&bytes); + entry.artifact.sha256.clear(); + let path = root.path().join(&entry.artifact.path); + tokio::fs::create_dir_all(path.parent().unwrap()) + .await + .unwrap(); + tokio::fs::write(&path, b"existing").await.unwrap(); + let before = tree_snapshot(root.path()); + let server = wiremock::MockServer::start().await; + mount_granted(&server, UUID, "example-1.0.0.tgz", &bytes).await; + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); + let error = restore(root.path(), &entry, &record(), &cfg) + .await + .unwrap_err(); + assert!(error.contains("no archive SHA-256"), "{error}"); + assert!(server.received_requests().await.unwrap().is_empty()); + assert_eq!(tree_snapshot(root.path()), before); + } + + #[tokio::test] + async fn changed_archive_and_missing_service_preserve_the_existing_tree() { + let root = tempfile::tempdir().unwrap(); + let bytes = tgz("package", b"pinned"); + let entry = entry(&bytes); + let path = root.path().join(&entry.artifact.path); + tokio::fs::create_dir_all(path.parent().unwrap()) + .await + .unwrap(); + tokio::fs::write(path, b"corrupt but recoverable") + .await + .unwrap(); + let before = tree_snapshot(root.path()); + let server = wiremock::MockServer::start().await; + mount_granted( + &server, + UUID, + "example-1.0.0.tgz", + &tgz("package", b"different unpatched content"), + ) + .await; + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); + assert!(restore(root.path(), &entry, &record(), &cfg) + .await + .unwrap_err() + .contains("recorded SHA-256")); + assert_eq!(tree_snapshot(root.path()), before); + server.reset().await; + crate::vendor::test_support::mount_503(&server).await; + assert!(restore(root.path(), &entry, &record(), &cfg).await.is_err()); + assert_eq!(tree_snapshot(root.path()), before); + let offline = service_cfg(&server.uri(), VendorSource::Service, true); + server.reset().await; + assert!(restore(root.path(), &entry, &record(), &offline) + .await + .is_err()); + assert!(server.received_requests().await.unwrap().is_empty()); + assert_eq!(tree_snapshot(root.path()), before); + } + + #[tokio::test] + async fn recorded_digest_cannot_override_patch_member_verification() { + let root = tempfile::tempdir().unwrap(); + let bytes = tgz("package", b"pinned"); + let entry = entry(&bytes); + let mut record = record(); + record.files.get_mut("index.js").unwrap().after_hash = + compute_git_sha256_from_bytes(b"different patch"); + let server = wiremock::MockServer::start().await; + mount_granted(&server, UUID, "example-1.0.0.tgz", &bytes).await; + assert!(restore( + root.path(), + &entry, + &record, + &service_cfg(&server.uri(), VendorSource::Service, false) + ) + .await + .unwrap_err() + .contains("hash_mismatch")); + assert!(!root.path().join(&entry.artifact.path).exists()); + } + + #[tokio::test] + async fn directory_repair_keeps_the_recorded_inventory() { + let root = tempfile::tempdir().unwrap(); + let bytes = tgz("package", b"pinned"); + let mut entry = entry(&bytes); + entry.ecosystem = "composer".into(); + entry.base_purl = "pkg:composer/example/library@1.0.0".into(); + entry.flavor = None; + entry.artifact.path = format!(".socket/vendor/composer/{UUID}/example-library"); + entry.artifact.sha256.clear(); + entry.artifact.size = None; + entry.artifact.file_inventory = Some( + HashMap::from([ + ("index.js".into(), hex::encode(Sha256::digest(b"patched"))), + ("extra".into(), hex::encode(Sha256::digest(b"pinned"))), + ]) + .into_iter() + .collect(), + ); + let path = root.path().join(&entry.artifact.path); + tokio::fs::create_dir_all(&path).await.unwrap(); + tokio::fs::write(path.join("index.js"), b"broken") + .await + .unwrap(); + let before = tree_snapshot(root.path()); + let server = wiremock::MockServer::start().await; + for (extra, expected_ok) in [(b"changed".as_slice(), false), (b"pinned".as_slice(), true)] { + server.reset().await; + let archive = super::super::common::write_zip_entries(&[ + ("package/index.js".into(), b"patched".to_vec(), 0o644), + ("package/extra".into(), extra.to_vec(), 0o644), + ]) + .unwrap(); + mount_granted(&server, UUID, "dist.zip", &archive).await; + let result = restore( + root.path(), + &entry, + &record(), + &service_cfg(&server.uri(), VendorSource::Service, false), + ) + .await; + assert_eq!(result.is_ok(), expected_ok, "{result:?}"); + if !expected_ok { + assert_eq!(tree_snapshot(root.path()), before); + } + } + assert_eq!( + tokio::fs::read(path.join("extra")).await.unwrap(), + b"pinned" + ); + let before = tree_snapshot(root.path()); + entry.artifact.file_inventory = None; + server.reset().await; + assert!(restore( + root.path(), + &entry, + &record(), + &service_cfg(&server.uri(), VendorSource::Service, false) + ) + .await + .unwrap_err() + .contains("no complete file inventory")); + assert!(server.received_requests().await.unwrap().is_empty()); + assert_eq!(tree_snapshot(root.path()), before); + entry.ecosystem = "npm".into(); + assert!(restore( + root.path(), + &entry, + &record(), + &service_cfg(&server.uri(), VendorSource::Service, false) + ) + .await + .unwrap_err() + .contains("identity")); + assert_eq!(tree_snapshot(root.path()), before); + } + #[tokio::test] + async fn go_restore_uses_shared_service_policy_without_mutating_the_tree() { + use wiremock::matchers::{method, path}; + use wiremock::{Mock, ResponseTemplate}; + let root = tempfile::tempdir().unwrap(); + let mut entry = entry(b"unused"); + entry.ecosystem = "golang".into(); + entry.base_purl = "pkg:golang/example.com/library@v1.0.0".into(); + entry.flavor = None; + entry.artifact.path = format!(".socket/vendor/golang/{UUID}/example.com/library@v1.0.0"); + entry.artifact.sha256.clear(); + entry.artifact.size = None; + entry.artifact.file_inventory = Some(Default::default()); + let copy = root.path().join(&entry.artifact.path); + tokio::fs::create_dir_all(©).await.unwrap(); + tokio::fs::write(copy.join("index.js"), b"existing bytes") + .await + .unwrap(); + tokio::fs::write(root.path().join("go.mod"), b"module consumer\n") + .await + .unwrap(); + let before = tree_snapshot(root.path()); + let server = wiremock::MockServer::start().await; + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); + for (status, expected) in [ + ( + "pending_build", + "vendor_prebuilt_required: prebuilt module zip is still building", + ), + ( + "not_found", + "vendor_prebuilt_required: prebuilt module zip unavailable", + ), + ( + "transport", + "vendor_prebuilt_required: patch service request failed", + ), + ("tampered", "vendor_prebuilt_integrity_mismatch"), + ] { + server.reset().await; + if status == "transport" { + crate::vendor::test_support::mount_503(&server).await; + } else if status == "tampered" { + mount_granted(&server, UUID, "v1.0.0.zip", b"promised bytes").await; + Mock::given(method("GET")) + .and(path(format!("/serve/{UUID}/v1.0.0.zip"))) + .respond_with( + ResponseTemplate::new(200).set_body_bytes(b"different bytes".to_vec()), + ) + .with_priority(1) + .mount(&server) + .await; + } else { + Mock::given(method("POST")) + .and(path(crate::vendor::test_support::PACKAGE_PATH)) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "results": { UUID: { "status": status } } + }))) + .mount(&server) + .await; + } + let error = restore(root.path(), &entry, &record(), &cfg) + .await + .unwrap_err(); + assert!(error.contains(expected), "{status}: {error}"); + assert_eq!(tree_snapshot(root.path()), before, "{status}"); + } + } +} diff --git a/crates/socket-patch-core/src/vendor/registry_fetch.rs b/crates/socket-patch-core/src/vendor/registry_fetch.rs index f7d833c02..aad8ef985 100644 --- a/crates/socket-patch-core/src/vendor/registry_fetch.rs +++ b/crates/socket-patch-core/src/vendor/registry_fetch.rs @@ -1,20 +1,4 @@ -//! Pristine-artifact fetching for lockfile-resolved packages with no -//! installed copy. -//! -//! `vendor` needs an installed package dir to stage from; on a fresh clone -//! there is none. This module downloads the pristine artifact the lockfile -//! resolves (the lock-recorded URL when present, the conventional registry -//! URL otherwise), verifies it against the integrity the lock records -//! **FAIL-CLOSED and before anything is written to the staging dir**, and -//! extracts it into a private tempdir the vendor pipeline then treats as -//! the installed dir. The project tree — node_modules included — is never -//! touched. -//! -//! Trust model: the URL comes from the user's own committed lockfile (or a -//! conventional construction from it); content trust comes from the -//! lock-recorded hash, not the transport — which is also why an entry with -//! no verifier ([`LockIntegrity::None`]) is refused outright -//! ([`FetchError::Unverifiable`]) without any network I/O. +//! Bounded archive readers, integrity verification and registry metadata transport. use std::path::{Path, PathBuf}; use std::time::Duration; @@ -24,11 +8,9 @@ use sha1::Sha1; use sha2::{Digest, Sha256, Sha384, Sha512}; use crate::constants::USER_AGENT; -use crate::crawlers::go_crawler::encode_module_path; use crate::patch::apply::is_safe_relative_subpath; -use crate::patch::path_safety::is_safe_single_segment; -use super::lock_inventory::{LockIntegrity, LockfileEntry}; +use super::lock_inventory::LockIntegrity; /// The default npm registry; override with `SOCKET_NPM_REGISTRY` (the /// enterprise-mirror / test escape hatch — `.npmrc` parsing is out of @@ -46,179 +28,6 @@ pub(crate) const MAX_TOTAL_DECOMPRESSED_BYTES: u64 = 512 * 1024 * 1024; pub(crate) const MAX_ENTRY_BYTES: u64 = 128 * 1024 * 1024; pub(crate) const MAX_ENTRIES: usize = 60_000; -/// A fetched, verified package whose tree is written only when a branch -/// actually reads it. -/// -/// The download, the size caps and the SRI / sha / dirhash verification all -/// stay EAGER, and so does the archive walk: before this value exists the -/// bytes have been validated against exactly the rules the extractor -/// enforces ([`Sink::Validate`]), so a truncated, oversized, traversing or -/// otherwise malformed artifact is still refused at the fetch, at the same -/// entry and with the same message. What is deferred is the WRITING — the -/// committed-artifact reuse, the in-sync hot path and the vendoring service -/// never read the tree, so an idempotent re-run on a lockfile-only checkout -/// never creates and deletes one. -/// -/// A local build asks for the vendor stage directly -/// ([`FetchedPackage::stage_into`]), which the verified bytes write in one -/// pass without going through the tempdir. -/// -/// The tempdir lives exactly as long as this value — callers must hold it -/// until the vendor pipeline has finished staging from [`FetchedPackage::dir`]. -pub struct FetchedPackage { - dir: PathBuf, - /// Where the bytes came from (surfaced in the fetch warning event). - pub url: String, - /// The verified bytes and the extractor that writes them — the same - /// function an eager fetch called, kept so the tree can be produced - /// wherever it is first wanted: the private tempdir - /// ([`FetchedPackage::dir`]), or a vendor stage directly - /// ([`FetchedPackage::stage_into`]). - /// - /// Dropped as soon as the tempdir holds the tree: from there on every - /// caller copies out of it, so the archive is dead weight. A source - /// NOTHING reads — the case this deferral exists for — keeps its bytes - /// until the holder is dropped instead of a whole extracted tree on - /// disk. - extract: std::sync::Mutex>>, - /// The tempdir materialisation's outcome, shared by every later caller - /// so a failure reads the same each time. - extracted: tokio::sync::OnceCell>, - _tmp: tempfile::TempDir, -} - -/// Writes a verified archive out under a destination, skipping any entry -/// whose final path component matches (`fresh_copy`'s `skip_file_name`). -type Extractor = dyn Fn(&Path, Option<&str>) -> Result<(), String> + Send + Sync; - -impl std::fmt::Debug for FetchedPackage { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("FetchedPackage") - .field("dir", &self.dir) - .field("url", &self.url) - .field( - "extracted", - &self.extracted.get().is_some_and(Result::is_ok), - ) - .finish() - } -} - -impl FetchedPackage { - fn pending( - dir: PathBuf, - url: String, - tmp: tempfile::TempDir, - extract: impl Fn(&Path, Option<&str>) -> Result<(), String> + Send + Sync + 'static, - ) -> Self { - Self { - dir, - url, - extract: std::sync::Mutex::new(Some(std::sync::Arc::new(extract))), - extracted: tokio::sync::OnceCell::new(), - _tmp: tmp, - } - } - - /// A tree already written into `tmp` (a committed directory artifact, - /// copied and verified up front): nothing left to extract. - fn staged(dir: PathBuf, url: String, tmp: tempfile::TempDir) -> Self { - Self { - dir, - url, - extract: std::sync::Mutex::new(None), - extracted: tokio::sync::OnceCell::new_with(Some(Ok(()))), - _tmp: tmp, - } - } - - /// Where the package root WILL be. Pure — no I/O and no extraction, so - /// it answers naming questions (a gem's `-` leaf, whether - /// the parent is a gem home's `gems/`) without materialising anything. - pub fn dir_path(&self) -> &Path { - &self.dir - } - - /// The package root (`package.json` at the top for npm) with its content - /// on disk, extracted on the first call and kept for the rest of the run. - pub async fn dir(&self) -> Result<&Path, String> { - let done = self - .extracted - .get_or_init(|| self.write_tree(self.dir.clone(), None)) - .await; - match done { - Ok(()) => { - // The tree is on disk; nothing reads the archive again. - drop(self.take_extractor()); - Ok(&self.dir) - } - Err(detail) => Err(detail.clone()), - } - } - - /// Let the verified archive go, for a run that has moved past the purl - /// this source belongs to. The tree, if one was written, stays. - pub fn release(&self) { - drop(self.take_extractor()); - } - - /// Take the extractor out, freeing the archive bytes with the last - /// handle. Returns `None` once it is gone. - fn take_extractor(&self) -> Option> { - self.extract - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .take() - } - - /// A handle on the extractor, or the failure a caller that needs it - /// after the tree is already on disk would see (which no caller does — - /// every one of them prefers the tree). - fn extractor(&self) -> Result, String> { - self.extract - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .clone() - .ok_or_else(|| format!("the fetched archive for {} is no longer held", self.url)) - } - - /// Write the tree at `dst` instead of the tempdir: the vendor stage the - /// local build patches. `dst` is removed and recreated first, exactly as - /// `fresh_copy` does, and `skip_file_name` drops the same entries it - /// drops. - pub async fn stage_into(&self, dst: &Path, skip_file_name: Option<&str>) -> Result<(), String> { - // An earlier branch already wrote the tempdir out (a dry-run - // preview, or the release-variant probe the vendor loop runs for - // pypi and gem). Copying it is cheaper than inflating the archive a - // second time. - if self.extracted.get().is_some_and(Result::is_ok) { - return crate::patch::copy_tree::fresh_copy(&self.dir, dst, skip_file_name) - .await - .map_err(|e| e.to_string()); - } - // `fresh_copy`'s own remove/create errors reached the backend's - // wrapper bare, so these do too. - crate::patch::copy_tree::remove_tree(dst) - .await - .map_err(|e| e.to_string())?; - tokio::fs::create_dir_all(dst) - .await - .map_err(|e| e.to_string())?; - self.write_tree(dst.to_path_buf(), skip_file_name.map(str::to_string)) - .await - } - - /// Extraction is sync CPU + disk work; keep it off the runtime thread so - /// the concurrent fetches around it keep moving. - async fn write_tree(&self, dst: PathBuf, skip_file_name: Option) -> Result<(), String> { - let extract = self.extractor()?; - match tokio::task::spawn_blocking(move || extract(&dst, skip_file_name.as_deref())).await { - Ok(outcome) => outcome, - Err(e) => Err(format!("extraction task failed: {e}")), - } - } -} - #[derive(Debug)] pub enum FetchError { /// The entry cannot be verified against the lockfile (no integrity @@ -230,9 +39,7 @@ pub enum FetchError { Failed(String), } -/// One shared client for all fetches in a run. -/// The registry HTTP client type, nameable by callers that don't depend on -/// reqwest directly (the CLI's pristine-source ladder). +/// Shared registry HTTP client for metadata and verified artifact downloads. pub type RegistryClient = reqwest::Client; pub fn build_registry_client() -> RegistryClient { @@ -260,84 +67,6 @@ pub fn npm_tarball_url(base: &str, name: &str, version: &str) -> String { format!("{base}/{name}/-/{leaf}-{version}.tgz") } -/// The package-root leaf [`fetch_and_stage`] would stage `purl` under — the -/// name a [`super::source::DeferredPackage`] answers naming questions with -/// before (or without) fetching: the canonical `-` for a gem -/// (the gem backend refuses any other leaf), the fixed per-ecosystem name -/// otherwise. -pub fn staged_leaf_for_purl(purl: &str) -> String { - let base = crate::utils::purl::strip_purl_qualifiers(purl); - match base.strip_prefix("pkg:").and_then(|r| r.split_once('/')) { - Some(("gem", rest)) => match rest.rsplit_once('@') { - Some((name, version)) => format!("{name}-{version}"), - None => "gem".to_string(), - }, - Some(("pypi", _)) => "site-packages".to_string(), - Some(("cargo", _)) => "crate".to_string(), - Some(("golang", _)) => "module".to_string(), - _ => "package".to_string(), - } -} - -/// Why [`fetch_and_stage`] refuses `entry` as unverifiable before any -/// request, decided from the entry (and go's proxy settings) alone. Covers -/// the npm, cargo, golang and composer fetchers completely; the gem and -/// pypi fetchers raise further refusals only their downloads can decide. -pub fn refusal_before_download(entry: &LockfileEntry) -> Option { - if entry.integrity == LockIntegrity::None { - return Some(format!( - "the lockfile records no integrity hash for {}@{}; refusing to fetch \ - unverifiable content", - entry.name, entry.version - )); - } - match entry.ecosystem { - "npm" => match &entry.integrity { - LockIntegrity::BerryChecksum(expected) if !expected.starts_with("10c0/") => { - Some(format!( - "yarn berry checksum `{expected}` uses a cacheKey other than 10c0; \ - the cache-zip recipe is not reproducible for it" - )) - } - _ => None, - }, - "golang" => match (&entry.integrity, &entry.resolved) { - (LockIntegrity::GoH1(_), Some(_)) => None, - (LockIntegrity::GoH1(_), None) => goproxy_base(&entry.name).err(), - _ => Some("go module entries verify via the go.sum h1 dirhash only".to_string()), - }, - "composer" if entry.resolved.is_none() => Some(format!( - "composer.lock records no dist URL for {}@{}", - entry.name, entry.version - )), - "cargo" | "composer" | "gem" | "pypi" => None, - other => Some(format!("no registry fetcher for ecosystem `{other}`")), - } -} - -/// Fetch + verify + extract one lockfile entry. Ecosystems without a -/// fetcher yet return [`FetchError::Unverifiable`] (callers keep their -/// not-installed outcome). -pub async fn fetch_and_stage( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - if let Some(reason) = refusal_before_download(entry) { - return Err(FetchError::Unverifiable(reason)); - } - match entry.ecosystem { - "npm" => fetch_npm(entry, client).await, - "cargo" => fetch_cargo(entry, client).await, - "golang" => fetch_golang(entry, client).await, - "composer" => fetch_composer(entry, client).await, - "gem" => fetch_gem(entry, client).await, - "pypi" => fetch_pypi(entry, client).await, - other => Err(FetchError::Unverifiable(format!( - "no registry fetcher for ecosystem `{other}`" - ))), - } -} - /// Run one of the extractors on the blocking pool. /// /// A service archive is written out in full — tens of thousands of small @@ -621,6 +350,7 @@ pub(crate) fn extract_zip_skipping( /// the destinations the refusals name, and it is where the write walk takes /// over for the one refusal this pass cannot decide on its own (see /// [`DestShape::file_dir_conflict`]). +#[cfg(test)] pub(crate) fn validate_zip( bytes: &[u8], dest: &Path, @@ -1025,78 +755,6 @@ fn lands_at_root(rel: &Path, name: &str) -> bool { .is_some_and(|c| c.as_os_str() == name) } -/// Composer dist zips: sha1-verified; a variable zipball top dir is -/// stripped when present, flat `composer archive`-built dists extract -/// as-is. The extracted dir plays the installed package dir. -async fn fetch_composer( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - let Some(url) = entry.resolved.clone() else { - return Err(FetchError::Unverifiable(format!( - "composer.lock records no dist URL for {}@{}", - entry.name, entry.version - ))); - }; - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - verify_integrity(&bytes, &entry.integrity)?; - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("package"); - // Strip only when the zip actually nests under a lone top dir (the - // zipball layout) — flat `composer archive`-built dists carry - // composer.json at the root; see [`zip_has_single_top_dir`]. - let strip_first = zip_has_single_top_dir(&bytes).map_err(FetchError::Failed)?; - let has_manifest = validate_zip(&bytes, &dir, strip_first, Some("composer.json")) - .map_err(FetchError::Failed)?; - if !has_manifest { - return Err(FetchError::Failed(format!( - "fetched dist for {}@{} carries no composer.json", - entry.name, entry.version - ))); - } - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_zip_skipping(&bytes, dest, strip_first, skip) - })) -} - -/// `.gem` files are plain tar containers holding `data.tar.gz` (the -/// package content, no prefix dir) + metadata. The whole `.gem` is -/// sha256-verified against the Gemfile.lock CHECKSUMS entry first. -async fn fetch_gem( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - // The staged leaf must be the canonical `{name}-{version}`: the gem - // vendor backend refuses any other leaf as a platform-suffixed install - // (`platform_gem_unsupported`), so a generic name would kill the whole - // auto-fetch path. The coordinates thereby become a tempdir path - // component — `inventory_gemfile_lock` already filters both, but - // re-assert locally (defense in depth), before any network I/O. - if !is_safe_single_segment(&entry.name) || !is_safe_single_segment(&entry.version) { - return Err(FetchError::Failed(format!( - "unsafe gem coordinates `{}` @ `{}` — refusing to stage", - entry.name, entry.version - ))); - } - let Some(url) = entry.resolved.clone() else { - return Err(FetchError::Unverifiable(format!( - "no download URL for {}@{}", - entry.name, entry.version - ))); - }; - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - verify_integrity(&bytes, &entry.integrity)?; - - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join(format!("{}-{}", entry.name, entry.version)); - validate_gem_data(&bytes, &dir).map_err(FetchError::Failed)?; - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_gem_data_skipping(&bytes, dest, skip) - })) -} - /// PyPI's JSON API base; override with `SOCKET_PYPI_JSON_API` (tests point it /// at a mock). Used only to turn a lock's file hash into a download URL for /// locks that record hashes without URLs (poetry.lock, which records one wheel @@ -1111,179 +769,6 @@ pub(crate) fn pypi_json_api_base() -> String { .unwrap_or_else(|| DEFAULT_PYPI_JSON_API.to_string()) } -/// Resolve the download URL of the release file whose sha256 the lock -/// records, via `GET ///json` → `urls[].digests.sha256`. -/// The hash, not the filename, selects the file, so a lock that names a wheel -/// PyPI has since re-uploaded under the same name cannot be satisfied by -/// different bytes — the download is still verified against the same hash. -/// -/// `candidates` is the lock's digest set: a single digest (poetry.lock names -/// the wheel) takes the first release file carrying it; several digests -/// (Pipfile.lock lists every release file's hash) take the pure-Python -/// `-none-any.whl` whose digest is in the set — a platform wheel or sdist is -/// never chosen, because the vendored wheel must install everywhere. -async fn resolve_pypi_url_by_hash( - entry: &LockfileEntry, - candidates: &[String], - client: &reqwest::Client, -) -> Result { - let api = format!( - "{}/{}/{}/json", - pypi_json_api_base(), - entry.name, - entry.version - ); - let resp = client.get(&api).send().await.map_err(|e| { - FetchError::Failed(format!( - "PyPI JSON API request for {} failed: {e}", - entry.purl - )) - })?; - if !resp.status().is_success() { - return Err(FetchError::Failed(format!( - "PyPI JSON API returned HTTP {} for {}", - resp.status(), - entry.purl - ))); - } - let body: serde_json::Value = resp.json().await.map_err(|e| { - FetchError::Failed(format!( - "PyPI JSON API response for {} is not JSON: {e}", - entry.purl - )) - })?; - let digest_matches = |file: &serde_json::Value| { - file.get("digests") - .and_then(|d| d.get("sha256")) - .and_then(serde_json::Value::as_str) - .is_some_and(|d| candidates.iter().any(|c| d.eq_ignore_ascii_case(c))) - }; - let is_pure_wheel = |file: &serde_json::Value| { - file.get("filename") - .and_then(serde_json::Value::as_str) - .or_else(|| file.get("url").and_then(serde_json::Value::as_str)) - .is_some_and(|name| { - name.split(['?', '#']) - .next() - .is_some_and(|n| n.ends_with("-none-any.whl")) - }) - }; - let files: Vec<&serde_json::Value> = body - .get("urls") - .and_then(serde_json::Value::as_array) - .into_iter() - .flatten() - .filter(|file| digest_matches(file)) - .collect(); - let chosen = if candidates.len() == 1 { - files.first().copied() - } else { - files.iter().copied().find(|file| is_pure_wheel(file)) - }; - chosen - .and_then(|file| file.get("url").and_then(serde_json::Value::as_str)) - .map(str::to_string) - .ok_or_else(|| { - FetchError::Unverifiable(if candidates.len() == 1 { - format!( - "no PyPI release file for {}@{} matches the lockfile's sha256 {}", - entry.name, entry.version, candidates[0] - ) - } else { - format!( - "no platform-independent (`-none-any.whl`) PyPI release file for {}@{} \ - matches any of the {} sha256 digests the lockfile records", - entry.name, - entry.version, - candidates.len() - ) - }) - }) -} - -/// Pure-python wheels recorded by uv.lock (URL + sha256): the unzipped -/// wheel IS a site-packages layout (package dirs + `.dist-info/RECORD` at -/// the root), which is exactly the shape the pypi vendor backend stages -/// from. -async fn fetch_pypi( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - let url = match (&entry.resolved, &entry.integrity) { - (Some(url), _) => url.clone(), - // poetry.lock records the wheel's hash but no URL: look the file up - // by that hash (verified again after download). - (None, LockIntegrity::Sha256Hex(sha256)) => { - resolve_pypi_url_by_hash(entry, std::slice::from_ref(sha256), client).await? - } - // Pipfile.lock records every release file's hash without filenames: - // pick the pure wheel whose digest is in the set (verified again - // after download against that set). - (None, LockIntegrity::Sha256AnyOf(digests)) => { - resolve_pypi_url_by_hash(entry, digests, client).await? - } - (None, _) => { - return Err(FetchError::Unverifiable(format!( - "the lockfile records no platform-independent wheel URL or sha256 for {}@{}", - entry.name, entry.version - ))); - } - }; - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - verify_integrity(&bytes, &entry.integrity)?; - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("site-packages"); - validate_zip(&bytes, &dir, /*strip_first=*/ false, None).map_err(FetchError::Failed)?; - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_zip_skipping(&bytes, dest, /*strip_first=*/ false, skip) - })) -} - -/// crates.io static download host; override with `SOCKET_CRATES_REGISTRY`. -pub const DEFAULT_CRATES_REGISTRY: &str = "https://static.crates.io/crates"; - -pub(crate) fn crates_registry_base() -> String { - std::env::var("SOCKET_CRATES_REGISTRY") - .ok() - .map(|v| v.trim_end_matches('/').to_string()) - .filter(|v| !v.is_empty()) - .unwrap_or_else(|| DEFAULT_CRATES_REGISTRY.to_string()) -} - -/// `.crate` files are tar.gz with a `{name}-{version}/` top dir — the same -/// extraction path as npm tarballs. The Cargo.lock `checksum` is the sha256 -/// of the `.crate` bytes. -async fn fetch_cargo( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - let url = entry.resolved.clone().unwrap_or_else(|| { - format!( - "{}/{}/{}-{}.crate", - crates_registry_base(), - entry.name, - entry.name, - entry.version - ) - }); - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - verify_integrity(&bytes, &entry.integrity)?; - - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("crate"); - if !validate_tgz(&bytes, &dir, Some("Cargo.toml")).map_err(FetchError::Failed)? { - return Err(FetchError::Failed(format!( - "fetched .crate for {}@{} carries no Cargo.toml — not a crate", - entry.name, entry.version - ))); - } - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_tgz_skipping(&bytes, dest, skip) - })) -} - /// go's default module proxy (the first element of go's default /// `GOPROXY=https://proxy.golang.org,direct`). pub const DEFAULT_GOPROXY: &str = "https://proxy.golang.org"; @@ -1376,6 +861,7 @@ pub(crate) fn go_h1_of_zip(bytes: &[u8]) -> Result { } /// What one walk over a module zip learned. +#[cfg_attr(not(test), allow(dead_code))] struct ModuleZipWalk { /// The `h1:` dirhash of the entries. h1: String, @@ -1659,225 +1145,6 @@ fn walk_zip_with_prefix( Ok(()) } -async fn fetch_golang( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - let LockIntegrity::GoH1(expected) = &entry.integrity else { - return Err(FetchError::Unverifiable( - "go module entries verify via the go.sum h1 dirhash only".to_string(), - )); - }; - let url = match &entry.resolved { - Some(url) => url.clone(), - None => format!( - "{}/{}/@v/{}.zip", - goproxy_base(&entry.name).map_err(FetchError::Unverifiable)?, - encode_module_path(&entry.name), - encode_module_path(&entry.version) - ), - }; - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - let prefix = format!("{}@{}/", entry.name, entry.version); - // One inflate answers both the dirhash and the extraction rules; see - // [`walk_module_zip`] for why that keeps the two refusals' order. - let walk = walk_module_zip(&bytes, Some(&prefix)).map_err(FetchError::Failed)?; - if walk.h1 != *expected { - return Err(FetchError::Failed(format!( - "go.sum dirhash mismatch: lockfile records {expected}, the fetched module zip \ - hashes to {}", - walk.h1 - ))); - } - // The tempdir is created BEFORE the extraction refusal is raised: a run - // that cannot make one reports that rather than the refusal. - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("module"); - if walk.dest_clash { - // A name used as both a file and a directory: the extraction always - // refuses it, and only the filesystem can say with which errno at - // which entry. Let the walk this one stands in for answer, where it - // answered before. - extract_zip_with_prefix(&bytes, &dir, &prefix).map_err(FetchError::Failed)?; - } else if let Some(detail) = walk.extract_refusal { - return Err(FetchError::Failed(detail)); - } - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_zip_with_prefix_skipping(&bytes, dest, &prefix, skip) - })) -} - -async fn fetch_npm( - entry: &LockfileEntry, - client: &reqwest::Client, -) -> Result { - // A foreign berry cacheKey is decidable from the lockfile alone: refuse - // BEFORE the download, keeping the Unverifiable no-network contract (and - // not spending a full tarball download on an entry we could never - // verify). - if let LockIntegrity::BerryChecksum(expected) = &entry.integrity { - if !expected.starts_with("10c0/") { - return Err(FetchError::Unverifiable(format!( - "yarn berry checksum `{expected}` uses a cacheKey other than 10c0; \ - the cache-zip recipe is not reproducible for it" - ))); - } - } - let url = entry - .resolved - .clone() - .unwrap_or_else(|| npm_tarball_url(&npm_registry_base(), &entry.name, &entry.version)); - let bytes = download(client, &url).await.map_err(FetchError::Failed)?; - match &entry.integrity { - // yarn berry locks never hash the tarball itself — the checksum is - // sha512 of the deterministic cache zip. Rebuild it from the fetched - // bytes (the same spike-pinned recipe the berry wiring uses) and - // compare. Only cacheKey 10c0 (yarn 4 default) is reproducible. - LockIntegrity::BerryChecksum(expected) => { - let actual = super::berry_zip::berry_cache_checksum_10c0(&bytes, &entry.name) - .map_err(FetchError::Failed)?; - if &actual != expected { - return Err(FetchError::Failed(format!( - "yarn berry cache checksum mismatch: lockfile records {expected}, \ - the fetched tarball rebuilds to {actual}" - ))); - } - } - other => verify_integrity(&bytes, other)?, - } - - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("package"); - if !validate_tgz(&bytes, &dir, Some("package.json")).map_err(FetchError::Failed)? { - return Err(FetchError::Failed(format!( - "fetched tarball for {}@{} carries no package.json — not an npm package", - entry.name, entry.version - ))); - } - Ok(FetchedPackage::pending(dir, url, tmp, move |dest, skip| { - extract_tgz_skipping(&bytes, dest, skip) - })) -} - -/// Stage a package from an on-disk vendored tarball (the fresh-clone -/// re-vendor path: the project has our committed artifact but no installed -/// copy). The bytes are verified against the LEDGER-recorded sha256 before -/// extraction — same fail-closed posture as the registry path; an entry -/// with no recorded hash is refused. -pub async fn stage_local_artifact( - tgz_path: &Path, - expected_sha256_hex: &str, -) -> Result { - if expected_sha256_hex.is_empty() { - return Err(FetchError::Unverifiable( - "the vendor ledger records no sha256 for the artifact".to_string(), - )); - } - // Guarded read (`open_regular_file`): a FIFO squatting at the committed - // artifact path must fail fast instead of wedging the fresh-clone - // re-vendor forever in an `open(2)` waiting for a writer — the caller's - // metadata probe passes for a FIFO, so this is the first open. Same - // guard class as the vendor lockfile reads. - let bytes = { - use tokio::io::AsyncReadExt as _; - let (file, metadata) = crate::utils::fs::open_regular_file(tgz_path) - .await - .map_err(|e| FetchError::Failed(format!("cannot read {}: {e}", tgz_path.display())))?; - // Enforce the cap BEFORE the size-matched allocation and read: the - // committed artifact path can hold a huge (or sparse, cost-free to - // craft) file, and a metadata-sized `with_capacity` would abort or - // OOM instead of returning the clean cap error below. Declared size - // here + actual bytes below — the same double enforcement as - // [`download`]; the `take` holds the memory bound even against a - // file that grows between this stat and the read. - if metadata.len() > MAX_DOWNLOAD_BYTES { - return Err(FetchError::Failed(format!( - "{}: artifact exceeds the {MAX_DOWNLOAD_BYTES}-byte cap", - tgz_path.display() - ))); - } - let mut bytes = Vec::with_capacity(metadata.len() as usize); - file.take(MAX_DOWNLOAD_BYTES + 1) - .read_to_end(&mut bytes) - .await - .map_err(|e| FetchError::Failed(format!("cannot read {}: {e}", tgz_path.display())))?; - bytes - }; - if bytes.len() as u64 > MAX_DOWNLOAD_BYTES { - return Err(FetchError::Failed(format!( - "{}: artifact exceeds the {MAX_DOWNLOAD_BYTES}-byte cap", - tgz_path.display() - ))); - } - let actual = hex::encode(Sha256::digest(&bytes)); - if !actual.eq_ignore_ascii_case(expected_sha256_hex) { - return Err(FetchError::Failed(format!( - "{}: sha256 mismatch against the vendor ledger (recorded {expected_sha256_hex}, \ - on-disk bytes hash to {actual})", - tgz_path.display() - ))); - } - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create staging tempdir: {e}")))?; - let dir = tmp.path().join("package"); - validate_tgz(&bytes, &dir, None).map_err(FetchError::Failed)?; - Ok(FetchedPackage::pending( - dir, - format!("file:{}", tgz_path.display()), - tmp, - move |dest, skip| extract_tgz_skipping(&bytes, dest, skip), - )) -} - -/// Stage a package from a committed vlt directory artifact (the -/// fresh-clone re-vendor path): an inventory-verified copy of `dir` without -/// its `node_modules/`. Refused when the ledger records no inventory, and -/// on any missing, extra or modified file. -pub async fn stage_local_dir_artifact( - dir: &Path, - inventory: Option<&std::collections::BTreeMap>, -) -> Result { - let Some(inventory) = inventory else { - return Err(FetchError::Unverifiable( - "the vendor ledger records no file inventory for the artifact".to_string(), - )); - }; - let actual = super::verify::compute_package_dir_inventory(dir) - .await - .map_err(|e| FetchError::Failed(format!("{}: {e}", dir.display())))?; - if &actual != inventory { - return Err(FetchError::Failed(format!( - "{}: the committed dir does not match the vendor ledger's file inventory", - dir.display() - ))); - } - let tmp = tempfile::tempdir() - .map_err(|e| FetchError::Failed(format!("cannot create staging tempdir: {e}")))?; - let staged = tmp.path().join("package"); - crate::patch::copy_tree::fresh_copy(dir, &staged, None) - .await - .map_err(|e| FetchError::Failed(format!("cannot stage {}: {e}", dir.display())))?; - crate::patch::copy_tree::remove_tree(&staged.join("node_modules")) - .await - .map_err(|e| FetchError::Failed(format!("cannot stage {}: {e}", dir.display())))?; - let copied = super::verify::compute_package_dir_inventory(&staged) - .await - .map_err(|e| FetchError::Failed(format!("{}: {e}", dir.display())))?; - if &copied != inventory { - return Err(FetchError::Failed(format!( - "{}: the staged copy does not match the vendor ledger's file inventory", - dir.display() - ))); - } - Ok(FetchedPackage::staged( - staged, - format!("file:{}", dir.display()), - tmp, - )) -} - /// Capped download. http(s) only; the cap is enforced on the declared /// Content-Length AND the actual stream (a lying server cannot blow past /// it). @@ -1917,31 +1184,15 @@ pub(crate) async fn download(client: &reqwest::Client, url: &str) -> Result Result<(), String> { match integrity { - LockIntegrity::BerryChecksum(expected) => { - if !expected.starts_with("10c0/") { - return Err(format!( - "yarn berry checksum `{expected}` uses a cacheKey other than 10c0" - )); - } - let actual = super::berry_zip::berry_cache_checksum_10c0(bytes, name)?; - if &actual == expected { - Ok(()) - } else { - Err(format!( - "yarn berry cache checksum mismatch: lockfile records {expected}, the \ - artifact rebuilds to {actual}" - )) - } - } + LockIntegrity::BerryChecksum(_) => Err("a Yarn Berry cache checksum cannot verify tarball bytes; use the archive integrity supplied by the patch service".into()), other => verify_integrity(bytes, other).map_err(|e| match e { FetchError::Failed(d) | FetchError::Unverifiable(d) => d, }), @@ -2041,32 +1292,6 @@ pub(crate) fn verify_sri(bytes: &[u8], sri: &str) -> Result<(), String> { } } -/// Whether every FILE entry in the zip nests under one shared top-level -/// directory — the GitHub/GitLab-zipball layout. This is the per-archive -/// `strip_first` decision Composer itself makes (ArchiveDownloader promotes -/// a lone top dir, else installs from the extract root): `composer archive`- -/// built dists (Satis archive builds, Artifactory/Nexus, private Packagist) -/// store composer.json at the archive ROOT, where an unconditional strip -/// would drop it and refuse a genuine, integrity-verified artifact. -fn zip_has_single_top_dir(bytes: &[u8]) -> Result { - let archive = zip::ZipArchive::new(std::io::Cursor::new(bytes)) - .map_err(|e| format!("unreadable zip: {e}"))?; - let mut top: Option<&str> = None; - for name in archive.file_names() { - if name.ends_with('/') { - continue; // dir entries: extraction skips them too - } - let Some((first, _)) = name.split_once('/') else { - return Ok(false); // a root-level file — flat layout - }; - if top.is_some_and(|t| t != first) { - return Ok(false); - } - top = Some(first); - } - Ok(top.is_some()) -} - /// Strip the FIRST path component (npm's tarball semantics — usually /// `package/`, but registry tarballs may use any prefix dir). fn strip_first_component(path: &Path) -> Option { @@ -2128,6 +1353,7 @@ pub(crate) fn extract_tgz_strict(bytes: &[u8], dest: &Path) -> Result<(), String /// [`extract_tgz`]'s write-free twin: every refusal, nothing created. /// Reports whether `watch` would land at the root (see [`lands_at_root`]). /// `dest` is where the tree WOULD go; see [`validate_zip`]. +#[cfg(test)] pub(crate) fn validate_tgz(bytes: &[u8], dest: &Path, watch: Option<&str>) -> Result { walk_tar_gz( bytes, @@ -2166,6 +1392,7 @@ pub(crate) fn extract_gem_data_skipping( /// [`extract_gem_data`]'s write-free twin: every refusal, nothing created. /// `dest` is where the tree WOULD go; see [`validate_zip`]. +#[cfg(test)] pub(crate) fn validate_gem_data(gem_bytes: &[u8], dest: &Path) -> Result<(), String> { walk_gem_data(gem_bytes, dest, Sink::Validate, None) } @@ -2309,9 +1536,7 @@ fn walk_tar_gz( #[cfg(test)] mod tests { use super::*; - use crate::vendor::lock_inventory::SourceKind; - use wiremock::matchers::{method, path as url_path}; - use wiremock::{Mock, MockServer, ResponseTemplate}; + use crate::crawlers::go_crawler::encode_module_path; /// Build a gzipped tarball with the given `(path, bytes, exec)` entries. fn make_tgz(entries: &[(&str, &[u8], bool)]) -> Vec { @@ -2336,18 +1561,6 @@ mod tests { ) } - fn npm_entry(resolved: Option, integrity: LockIntegrity) -> LockfileEntry { - LockfileEntry { - ecosystem: "npm", - source_kind: SourceKind::Unspecified, - name: "left-pad".into(), - version: "1.3.0".into(), - purl: "pkg:npm/left-pad@1.3.0".into(), - resolved, - integrity, - } - } - #[test] fn tarball_url_forms() { assert_eq!( @@ -2401,108 +1614,6 @@ mod tests { assert!(verify_sri(bytes, &format!("sha1-{wrong} {sha512_good}")).is_ok()); } - #[tokio::test] - async fn fetch_verifies_sri_and_extracts_with_modes() { - let tgz = make_tgz(&[ - ("package/package.json", br#"{"name":"left-pad"}"#, false), - ("package/bin/cli.js", b"#!/usr/bin/env node\n", true), - ("package/index.js", b"module.exports = 1;\n", false), - ]); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/-/left-pad-1.3.0.tgz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(tgz.clone())) - .mount(&mock) - .await; - - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::Sri(sri_of(&tgz)), - ); - let fetched = fetch_and_stage(&entry, &build_registry_client()) - .await - .unwrap(); - assert!(fetched.dir().await.unwrap().join("package.json").is_file()); - assert_eq!( - std::fs::read(fetched.dir().await.unwrap().join("index.js")).unwrap(), - b"module.exports = 1;\n" - ); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt; - let mode = std::fs::metadata(fetched.dir().await.unwrap().join("bin/cli.js")) - .unwrap() - .permissions() - .mode(); - assert_eq!(mode & 0o111, 0o111, "exec bit preserved"); - } - // The tempdir dies with the holder. - let dir = fetched.dir().await.unwrap().to_path_buf(); - drop(fetched); - assert!(!dir.exists()); - } - - #[tokio::test] - async fn integrity_mismatch_fails_before_extraction() { - let tgz = make_tgz(&[("package/package.json", b"{}", false)]); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/-/left-pad-1.3.0.tgz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(tgz)) - .mount(&mock) - .await; - - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::Sri(sri_of(b"the lock expects different bytes")), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => { - assert!(msg.contains("mismatch"), "{msg}") - } - other => panic!("expected integrity failure, got {other:?}"), - } - } - - #[tokio::test] - async fn unverifiable_entry_refuses_without_network() { - // A URL that would hard-fail if contacted — Unverifiable proves the - // decision happened before any I/O. - let entry = npm_entry( - Some("http://127.0.0.1:1/nope.tgz".into()), - LockIntegrity::None, - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Unverifiable(msg)) => { - assert!(msg.contains("no integrity"), "{msg}") - } - other => panic!("expected Unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn http_error_and_scheme_guard_fail_closed() { - let mock = MockServer::start().await; - // No mounted route → 404. - let entry = npm_entry( - Some(format!("{}/missing.tgz", mock.uri())), - LockIntegrity::Sri(sri_of(b"x")), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("404"), "{msg}"), - other => panic!("expected HTTP failure, got {other:?}"), - } - - let entry = npm_entry( - Some("ftp://example.com/x.tgz".into()), - LockIntegrity::Sri(sri_of(b"x")), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("non-http"), "{msg}"), - other => panic!("expected scheme refusal, got {other:?}"), - } - } - #[test] fn extraction_strips_first_component_whatever_its_name() { let tgz = make_tgz(&[("weird-prefix/package.json", b"{}", false)]); @@ -2541,155 +1652,6 @@ mod tests { } } - #[tokio::test] - async fn berry_checksum_verifies_via_cache_zip_rebuild() { - let tgz = make_tgz(&[ - ("package/package.json", br#"{"name":"left-pad"}"#, false), - ("package/index.js", b"module.exports = 1;\n", false), - ]); - let expected = - super::super::berry_zip::berry_cache_checksum_10c0(&tgz, "left-pad").unwrap(); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/-/left-pad-1.3.0.tgz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(tgz)) - .mount(&mock) - .await; - - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::BerryChecksum(expected), - ); - let fetched = fetch_and_stage(&entry, &build_registry_client()) - .await - .unwrap(); - assert!(fetched.dir().await.unwrap().join("package.json").is_file()); - - // Tampered checksum → Failed; foreign cacheKey → Unverifiable. - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::BerryChecksum(format!("10c0/{}", "0".repeat(128))), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("mismatch"), "{msg}"), - other => panic!("expected mismatch, got {other:?}"), - } - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::BerryChecksum(format!("9/{}", "0".repeat(128))), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Unverifiable(msg)) => assert!(msg.contains("cacheKey"), "{msg}"), - other => panic!("expected Unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn stage_local_artifact_verifies_ledger_sha256() { - let tgz = make_tgz(&[("package/package.json", b"{}", false)]); - let tmp = tempfile::tempdir().unwrap(); - let tgz_path = tmp.path().join("left-pad-1.3.0.tgz"); - std::fs::write(&tgz_path, &tgz).unwrap(); - let sha = hex::encode(Sha256::digest(&tgz)); - - let staged = stage_local_artifact(&tgz_path, &sha).await.unwrap(); - assert!(staged.dir().await.unwrap().join("package.json").is_file()); - - match stage_local_artifact(&tgz_path, &"0".repeat(64)).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("mismatch"), "{msg}"), - other => panic!("expected ledger mismatch, got {other:?}"), - } - match stage_local_artifact(&tgz_path, "").await { - Err(FetchError::Unverifiable(_)) => {} - other => panic!("expected Unverifiable for empty hash, got {other:?}"), - } - } - - #[tokio::test] - async fn stage_local_dir_artifact_verifies_the_inventory_and_drops_node_modules() { - let tmp = tempfile::tempdir().unwrap(); - let dir = tmp.path().join("left-pad"); - std::fs::create_dir_all(dir.join("node_modules/.bin")).unwrap(); - std::fs::write(dir.join("package.json"), b"{}").unwrap(); - std::fs::write(dir.join("index.js"), b"x").unwrap(); - std::fs::write(dir.join("node_modules/.bin/tool"), b"#!/bin/sh\n").unwrap(); - #[cfg(unix)] - std::os::unix::fs::symlink("../../elsewhere", dir.join("node_modules/dep")).unwrap(); - let inventory = super::super::verify::compute_package_dir_inventory(&dir) - .await - .unwrap(); - assert_eq!(inventory.len(), 2, "{inventory:?}"); - - let staged = stage_local_dir_artifact(&dir, Some(&inventory)) - .await - .unwrap(); - let staged_dir = staged.dir().await.unwrap(); - assert_eq!(std::fs::read(staged_dir.join("index.js")).unwrap(), b"x"); - assert!(!staged_dir.join("node_modules").exists()); - assert_eq!(staged.url, format!("file:{}", dir.display())); - - std::fs::write(dir.join("planted.js"), b"y").unwrap(); - match stage_local_dir_artifact(&dir, Some(&inventory)).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("file inventory"), "{msg}"), - other => panic!("a planted file must fail, got {other:?}"), - } - std::fs::remove_file(dir.join("planted.js")).unwrap(); - std::fs::write(dir.join("index.js"), b"modified").unwrap(); - match stage_local_dir_artifact(&dir, Some(&inventory)).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("file inventory"), "{msg}"), - other => panic!("a modified file must fail, got {other:?}"), - } - match stage_local_dir_artifact(&dir, None).await { - Err(FetchError::Unverifiable(_)) => {} - other => panic!("no inventory is unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn cargo_crate_fetch_verifies_sha256_and_extracts() { - // .crate = tar.gz with a {name}-{version}/ top dir. - let crate_bytes = make_tgz(&[ - ( - "left-pad-1.3.0/Cargo.toml", - b"[package]\nname = \"left-pad\"\n", - false, - ), - ("left-pad-1.3.0/src/lib.rs", b"pub fn pad() {}\n", false), - ]); - let sha = hex::encode(Sha256::digest(&crate_bytes)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/left-pad-1.3.0.crate")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(crate_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "cargo", - source_kind: SourceKind::Unspecified, - name: "left-pad".into(), - version: "1.3.0".into(), - purl: "pkg:cargo/left-pad@1.3.0".into(), - resolved: Some(format!("{}/left-pad/left-pad-1.3.0.crate", mock.uri())), - integrity: LockIntegrity::Sha256Hex(sha), - }; - let fetched = fetch_and_stage(&entry, &build_registry_client()) - .await - .unwrap(); - assert!(fetched.dir().await.unwrap().join("Cargo.toml").is_file()); - assert!(fetched.dir().await.unwrap().join("src/lib.rs").is_file()); - - // Tampered checksum fails closed. - let entry = LockfileEntry { - integrity: LockIntegrity::Sha256Hex("0".repeat(64)), - ..entry - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("mismatch"), "{msg}"), - other => panic!("expected mismatch, got {other:?}"), - } - } - /// Build a go module zip in memory (files only, `module@version/` /// prefix — the go zip layout). fn make_module_zip(prefix: &str, files: &[(&str, &[u8])]) -> Vec { @@ -2728,59 +1690,6 @@ mod tests { ) } - #[tokio::test] - async fn golang_module_fetch_verifies_h1_dirhash_and_extracts() { - // Out-of-order files prove the sort; nested module path proves the - // explicit-prefix strip (a first-component strip would be wrong). - let prefix = "github.com/x/y@v1.0.0/"; - let files: [(&str, &[u8]); 3] = [ - ("go.mod", b"module github.com/x/y\n"), - ("a/b.go", b"package a\n"), - ("README.md", b"# y\n"), - ]; - let zip_bytes = make_module_zip(prefix, &files); - let expected = spec_h1(&files, prefix); - assert_eq!( - go_h1_of_zip(&zip_bytes).unwrap(), - expected, - "production dirhash matches the spec mirror" - ); - - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/github.com/x/y/@v/v1.0.0.zip")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(zip_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "golang", - source_kind: SourceKind::Unspecified, - name: "github.com/x/y".into(), - version: "v1.0.0".into(), - purl: "pkg:golang/github.com/x/y@v1.0.0".into(), - resolved: Some(format!("{}/github.com/x/y/@v/v1.0.0.zip", mock.uri())), - integrity: LockIntegrity::GoH1(expected), - }; - let fetched = fetch_and_stage(&entry, &build_registry_client()) - .await - .unwrap(); - assert!(fetched.dir().await.unwrap().join("go.mod").is_file()); - assert!(fetched.dir().await.unwrap().join("a/b.go").is_file()); - - // Tampered h1 fails closed. - let entry = LockfileEntry { - integrity: LockIntegrity::GoH1( - "h1:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=".into(), - ), - ..entry - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("mismatch"), "{msg}"), - other => panic!("expected mismatch, got {other:?}"), - } - } - #[test] fn go_escape_uppercase_and_zip_prefix_guards() { assert_eq!( @@ -2814,562 +1723,6 @@ mod tests { writer.finish().unwrap().into_inner() } - #[tokio::test] - async fn composer_dist_fetch_verifies_sha1_and_strips_top_dir() { - // GitHub zipballs carry an `owner-repo-sha/` top dir. - let zip_bytes = make_zip(&[ - ( - "Seldaek-monolog-abc123/composer.json", - br#"{"name":"monolog/monolog"}"#, - ), - ("Seldaek-monolog-abc123/src/Logger.php", b" assert!(msg.contains("mismatch"), "{msg}"), - other => panic!("expected mismatch, got {other:?}"), - } - } - - #[tokio::test] - async fn composer_flat_dist_fetch_keeps_root_layout() { - // `composer archive`-built dists (Satis archive builds, Artifactory/ - // Nexus, private Packagist) store composer.json at the archive ROOT — - // no zipball top dir. Composer itself auto-detects the layout per - // archive (ArchiveDownloader promotes a lone top dir, else installs - // from the extract root); an unconditional first-component strip - // drops the root composer.json and refuses a genuine, sha1-verified - // artifact as "carries no composer.json". - let zip_bytes = make_zip(&[ - ("composer.json", br#"{"name":"acme/flat"}"#), - ("src/Flat.php", b" { - assert!(msg.contains("unsafe gem coordinates"), "{msg}") - } - other => panic!("expected coordinate refusal, got {other:?}"), - } - } - - #[tokio::test] - async fn pypi_wheel_fetch_extracts_site_packages_layout() { - let wheel = make_zip(&[ - ("requests/__init__.py", b"__version__ = '2.28.0'\n"), - ( - "requests-2.28.0.dist-info/RECORD", - b"requests/__init__.py,sha256=abc,24\n", - ), - ("requests-2.28.0.dist-info/WHEEL", b"Wheel-Version: 1.0\n"), - ]); - let sha = hex::encode(Sha256::digest(&wheel)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/packages/requests-2.28.0-py3-none-any.whl")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(wheel)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "pypi", - source_kind: SourceKind::Unspecified, - name: "requests".into(), - version: "2.28.0".into(), - purl: "pkg:pypi/requests@2.28.0".into(), - resolved: Some(format!( - "{}/packages/requests-2.28.0-py3-none-any.whl", - mock.uri() - )), - integrity: LockIntegrity::Sha256Hex(sha), - }; - let fetched = fetch_and_stage(&entry, &build_registry_client()) - .await - .unwrap(); - // Wheel content at the root: a site-packages-shaped dir with the - // dist-info RECORD the pypi vendor backend stages from. - assert!(fetched - .dir() - .await - .unwrap() - .join("requests/__init__.py") - .is_file()); - assert!(fetched - .dir() - .await - .unwrap() - .join("requests-2.28.0.dist-info/RECORD") - .is_file()); - } - - /// poetry.lock records wheel hashes but no URLs: the fetcher resolves the - /// file through PyPI's JSON API by sha256 and still verifies the bytes. - #[tokio::test] - #[serial_test::serial] - async fn pypi_hash_only_entry_is_resolved_through_the_json_api() { - let wheel = make_zip(&[ - ("requests/__init__.py", b"__version__ = '2.28.0'\n"), - ( - "requests-2.28.0.dist-info/RECORD", - b"requests/__init__.py,sha256=abc,24\n", - ), - ]); - let sha = hex::encode(Sha256::digest(&wheel)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/packages/requests-2.28.0-py3-none-any.whl")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(wheel)) - .mount(&mock) - .await; - Mock::given(method("GET")) - .and(url_path("/pypi/requests/2.28.0/json")) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "urls": [ - {"filename": "requests-2.28.0.tar.gz", "url": format!("{}/packages/requests-2.28.0.tar.gz", mock.uri()), "digests": {"sha256": "0".repeat(64)}}, - {"filename": "requests-2.28.0-py3-none-any.whl", "url": format!("{}/packages/requests-2.28.0-py3-none-any.whl", mock.uri()), "digests": {"sha256": sha.to_uppercase()}}, - ] - }))) - .mount(&mock) - .await; - let saved = std::env::var("SOCKET_PYPI_JSON_API").ok(); - std::env::set_var("SOCKET_PYPI_JSON_API", format!("{}/pypi/", mock.uri())); - let restore = || match &saved { - Some(v) => std::env::set_var("SOCKET_PYPI_JSON_API", v), - None => std::env::remove_var("SOCKET_PYPI_JSON_API"), - }; - let entry = LockfileEntry { - ecosystem: "pypi", - source_kind: SourceKind::Unspecified, - name: "requests".into(), - version: "2.28.0".into(), - purl: "pkg:pypi/requests@2.28.0".into(), - resolved: None, - integrity: LockIntegrity::Sha256Hex(sha.clone()), - }; - let fetched = fetch_and_stage(&entry, &build_registry_client()).await; - // A hash no release file carries is refused before any download. - let unknown = LockfileEntry { - integrity: LockIntegrity::Sha256Hex("1".repeat(64)), - ..entry.clone() - }; - let missing = fetch_and_stage(&unknown, &build_registry_client()).await; - // No hash at all: nothing to resolve by. - let bare = LockfileEntry { - integrity: LockIntegrity::Sri("sha512-x".into()), - ..entry - }; - let bare_result = fetch_and_stage(&bare, &build_registry_client()).await; - restore(); - let fetched = fetched.unwrap(); - assert!(fetched - .dir() - .await - .unwrap() - .join("requests/__init__.py") - .is_file()); - assert!(fetched.url.ends_with("requests-2.28.0-py3-none-any.whl")); - match missing { - Err(FetchError::Unverifiable(msg)) => assert!(msg.contains("matches"), "{msg}"), - other => panic!("expected Unverifiable, got {other:?}"), - } - match bare_result { - Err(FetchError::Unverifiable(msg)) => assert!(msg.contains("sha256"), "{msg}"), - other => panic!("expected Unverifiable, got {other:?}"), - } - } - - /// Pipfile.lock records EVERY release file's digest without filenames: - /// the fetcher must pick the pure-Python wheel by digest (never the sdist - /// or a platform wheel that also matches), verify the download against - /// the set, and refuse when no pure wheel's digest is recorded. - #[tokio::test] - #[serial_test::serial] - async fn pypi_digest_set_entry_picks_the_pure_wheel_by_hash() { - let wheel = make_zip(&[ - ("requests/__init__.py", b"__version__ = '2.28.0'\n"), - ( - "requests-2.28.0.dist-info/RECORD", - b"requests/__init__.py,sha256=abc,24\n", - ), - ]); - let wheel_sha = hex::encode(Sha256::digest(&wheel)); - let sdist_sha = "0".repeat(64); - let platform_sha = "9".repeat(64); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/packages/requests-2.28.0-py3-none-any.whl")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(wheel)) - .mount(&mock) - .await; - Mock::given(method("GET")) - .and(url_path("/packages/requests-2.28.0.tar.gz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(b"sdist bytes".to_vec())) - .mount(&mock) - .await; - Mock::given(method("GET")) - .and(url_path("/pypi/requests/2.28.0/json")) - .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ - "urls": [ - {"filename": "requests-2.28.0.tar.gz", "url": format!("{}/packages/requests-2.28.0.tar.gz", mock.uri()), "digests": {"sha256": sdist_sha}}, - {"filename": "requests-2.28.0-cp312-cp312-manylinux_2_17_x86_64.whl", "url": format!("{}/packages/requests-2.28.0-cp312-cp312-manylinux_2_17_x86_64.whl", mock.uri()), "digests": {"sha256": platform_sha}}, - {"filename": "requests-2.28.0-py3-none-any.whl", "url": format!("{}/packages/requests-2.28.0-py3-none-any.whl", mock.uri()), "digests": {"sha256": wheel_sha.to_uppercase()}}, - ] - }))) - .mount(&mock) - .await; - let saved = std::env::var("SOCKET_PYPI_JSON_API").ok(); - std::env::set_var("SOCKET_PYPI_JSON_API", format!("{}/pypi/", mock.uri())); - let restore = || match &saved { - Some(v) => std::env::set_var("SOCKET_PYPI_JSON_API", v), - None => std::env::remove_var("SOCKET_PYPI_JSON_API"), - }; - let entry = LockfileEntry { - ecosystem: "pypi", - source_kind: SourceKind::Unspecified, - name: "requests".into(), - version: "2.28.0".into(), - purl: "pkg:pypi/requests@2.28.0".into(), - resolved: None, - // sdist first, like Pipenv writes them: the ORDER must not pick - // the sdist. - integrity: LockIntegrity::Sha256AnyOf(vec![ - sdist_sha.clone(), - platform_sha.clone(), - wheel_sha.clone(), - ]), - }; - let fetched = fetch_and_stage(&entry, &build_registry_client()).await; - // Only the sdist's and a platform wheel's digests recorded: no pure - // wheel to choose → refused before any download. - let no_pure = LockfileEntry { - integrity: LockIntegrity::Sha256AnyOf(vec![sdist_sha.clone(), platform_sha.clone()]), - ..entry.clone() - }; - let no_pure_result = fetch_and_stage(&no_pure, &build_registry_client()).await; - // Digests no release file carries → refused. - let unknown = LockfileEntry { - integrity: LockIntegrity::Sha256AnyOf(vec!["1".repeat(64), "2".repeat(64)]), - ..entry.clone() - }; - let unknown_result = fetch_and_stage(&unknown, &build_registry_client()).await; - restore(); - let fetched = fetched.unwrap(); - assert!(fetched - .dir() - .await - .unwrap() - .join("requests/__init__.py") - .is_file()); - assert!( - fetched.url.ends_with("requests-2.28.0-py3-none-any.whl"), - "{}", - fetched.url - ); - for (label, result) in [ - ("no pure wheel", no_pure_result), - ("unknown", unknown_result), - ] { - match result { - Err(FetchError::Unverifiable(msg)) => { - assert!( - msg.contains("none-any.whl") && msg.contains("digests"), - "{label}: {msg}" - ) - } - other => panic!("{label}: expected Unverifiable, got {other:?}"), - } - } - // The verifier itself: bytes matching ANY recorded digest pass, others fail. - let set = LockIntegrity::Sha256AnyOf(vec![sdist_sha.clone(), wheel_sha.clone()]); - // "sdist bytes" is not the recorded sdist digest ("000…"), so it must fail. - assert!(verify_integrity(b"sdist bytes", &set).is_err()); - let real_sdist = LockIntegrity::Sha256AnyOf(vec![ - hex::encode(Sha256::digest(b"sdist bytes")), - wheel_sha, - ]); - assert!(verify_integrity(b"sdist bytes", &real_sdist).is_ok()); - match verify_integrity(b"other", &real_sdist) { - Err(FetchError::Failed(msg)) => assert!(msg.contains("none of the 2 digests"), "{msg}"), - other => panic!("expected Failed, got {other:?}"), - } - } - - #[cfg(unix)] - fn mkfifo(path: &Path) { - use std::os::unix::ffi::OsStrExt; - let c_path = - std::ffi::CString::new(path.as_os_str().as_bytes()).expect("fifo path has no NUL"); - let rc = unsafe { libc::mkfifo(c_path.as_ptr(), 0o644) }; - assert_eq!( - rc, - 0, - "mkfifo(2) failed: {}", - std::io::Error::last_os_error() - ); - } - - /// A FIFO squatting at the committed artifact path must fail fast - /// instead of wedging the fresh-clone re-vendor forever in an `open(2)` - /// waiting for a writer — the caller's metadata probe passes for a FIFO, - /// so this read is the first open. Same `open_regular_file` guard class - /// as the vendor lockfile reads (lock_inventory/, npm_lock.rs). - #[cfg(unix)] - #[test] - fn stage_local_artifact_fifo_fails_fast_instead_of_wedging() { - let tmp = tempfile::tempdir().unwrap(); - let tgz_path = tmp.path().join("left-pad-1.3.0.tgz"); - mkfifo(&tgz_path); - // Own runtime on a detached thread: a wedged open(2) lives in a - // spawn_blocking task, and dropping (or #[tokio::test]-finishing) a - // runtime with one wedged blocks forever — the timeout must live - // OUTSIDE the runtime for the unfixed code to fail instead of hang. - let (tx, rx) = std::sync::mpsc::channel(); - let path = tgz_path.clone(); - std::thread::spawn(move || { - let rt = tokio::runtime::Builder::new_current_thread() - .enable_all() - .build() - .unwrap(); - let res = rt.block_on(stage_local_artifact(&path, &"0".repeat(64))); - std::mem::forget(rt); - let _ = tx.send(res); - }); - match rx.recv_timeout(std::time::Duration::from_secs(5)) { - Ok(Err(FetchError::Failed(msg))) => { - assert!(msg.contains(&tgz_path.display().to_string()), "{msg}") - } - Ok(other) => panic!("expected Failed on a FIFO artifact, got {other:?}"), - Err(_) => panic!("stage_local_artifact wedged on a FIFO artifact"), - } - } - - /// The 128 MB artifact cap must fire BEFORE the size-matched allocation - /// and read: a huge file at the ledger-recorded artifact path (a sparse - /// `truncate -s 64G` costs the attacker nothing) must get the clean - /// FetchError cap message, not a metadata-sized `Vec::with_capacity` - /// that aborts or OOMs — the module's documented memory-bomb bound. - /// - /// Runs in a CHILD PROCESS (the fs.rs RLIMIT_FSIZE precedent): peak RSS - /// is process-wide and monotonic, so sibling tests in this binary (the - /// 128 MB go_h1 bomb-cap test among them) would poison an in-process - /// measurement. - #[cfg(unix)] - #[tokio::test] - async fn stage_local_artifact_caps_oversized_artifact_before_buffering() { - const CHILD_ENV: &str = "SOCKET_PATCH_CORE_TEST_STAGE_CAP_CHILD"; - const TEST_NAME: &str = "vendor::registry_fetch::tests::\ - stage_local_artifact_caps_oversized_artifact_before_buffering"; - if std::env::var_os(CHILD_ENV).is_none() { - let exe = std::env::current_exe().expect("test binary path must resolve"); - let output = std::process::Command::new(exe) - .args([TEST_NAME, "--exact", "--test-threads=1", "--nocapture"]) - .env(CHILD_ENV, "1") - .output() - .expect("the measured child test process must spawn"); - let stdout = String::from_utf8_lossy(&output.stdout); - assert!( - output.status.success(), - "the measured child run failed:\nstdout:\n{stdout}\nstderr:\n{}", - String::from_utf8_lossy(&output.stderr), - ); - // Anti-vacuity: a renamed test would make the `--exact` filter - // match nothing and the child exit 0 having proven nothing. - assert!( - stdout.contains("1 passed"), - "the child run must execute exactly this test — filter drift \ - after a rename? child stdout:\n{stdout}" - ); - return; - } - - // 1 GiB sparse: zero disk blocks, but 8× the cap — buffering it - // before the cap check dirties ~1 GiB of RSS. - const HUGE: u64 = 1024 * 1024 * 1024; - let tmp = tempfile::tempdir().unwrap(); - let tgz_path = tmp.path().join("huge.tgz"); - std::fs::File::create(&tgz_path) - .unwrap() - .set_len(HUGE) - .unwrap(); - - match stage_local_artifact(&tgz_path, &"0".repeat(64)).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("cap"), "{msg}"), - other => panic!("expected the cap refusal, got {other:?}"), - } - - let mut ru = std::mem::MaybeUninit::::zeroed(); - assert_eq!( - unsafe { libc::getrusage(libc::RUSAGE_SELF, ru.as_mut_ptr()) }, - 0 - ); - let ru = unsafe { ru.assume_init() }; - // macOS reports ru_maxrss in bytes, Linux in kilobytes. - let peak = if cfg!(target_os = "macos") { - ru.ru_maxrss as u64 - } else { - (ru.ru_maxrss as u64) * 1024 - }; - assert!( - peak < HUGE / 2, - "peak RSS {peak} bytes — the oversized artifact was buffered into \ - memory before the cap check" - ); - } - #[test] #[serial_test::serial] fn goproxy_base_splits_on_pipe_separator() { @@ -3399,45 +1752,6 @@ mod tests { assert_eq!(mixed.as_deref(), Ok("https://mirror.example")); } - #[tokio::test] - async fn berry_foreign_cachekey_refuses_before_network() { - // The cacheKey is decidable from the lockfile alone; the refusal must - // be the Unverifiable contract's pre-network kind (the URL would - // hard-fail if contacted), not a Failed download error — and yarn - // 2/3 locks (cacheKey 8/9) must not cost a full tarball download - // just to be refused afterwards. - let entry = npm_entry( - Some("http://127.0.0.1:1/nope.tgz".into()), - LockIntegrity::BerryChecksum(format!("9/{}", "0".repeat(128))), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Unverifiable(msg)) => assert!(msg.contains("cacheKey"), "{msg}"), - other => panic!("expected pre-network Unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn pypi_no_wheel_url_message_is_single_spaced() { - // No URL and no sha256 to resolve one by (a sha256 would consult the - // PyPI JSON API — `pypi_hash_only_entry_is_resolved_through_the_json_api`). - let entry = LockfileEntry { - ecosystem: "pypi", - source_kind: SourceKind::Unspecified, - name: "requests".into(), - version: "2.28.0".into(), - purl: "pkg:pypi/requests@2.28.0".into(), - resolved: None, - integrity: LockIntegrity::Sri("sha512-x".into()), - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Unverifiable(msg)) => assert!( - !msg.contains(" "), - "user-facing message carries an embedded space run: {msg:?}" - ), - other => panic!("expected Unverifiable, got {other:?}"), - } - } - /// Binary-patch a single-entry zip's DECLARED uncompressed size (local /// header + central directory) — the exact lie a crafted artifact can /// carry, since the crc and the deflate stream stay honest and zip 8.x @@ -3527,396 +1841,6 @@ mod tests { ); } - #[tokio::test] - async fn unknown_ecosystem_refuses_before_network() { - // Ecosystems without a fetcher (maven/nuget/deno) keep the caller's - // not-installed outcome via Unverifiable — decided BEFORE any I/O - // (the poison URL would hard-fail if contacted). - let entry = LockfileEntry { - ecosystem: "maven", - source_kind: SourceKind::Unspecified, - name: "org.apache.commons:commons-lang3".into(), - version: "3.14.0".into(), - purl: "pkg:maven/org.apache.commons/commons-lang3@3.14.0".into(), - resolved: Some("http://127.0.0.1:1/x.jar".into()), - integrity: LockIntegrity::Sha256Hex("0".repeat(64)), - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Unverifiable(msg)) => assert!( - msg.contains("no registry fetcher for ecosystem `maven`"), - "{msg}" - ), - other => panic!("expected pre-network Unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn per_ecosystem_unverifiable_refusals_without_network() { - // Each refusal is decidable from the lockfile alone, so each must be - // the Unverifiable kind — poison URLs prove no I/O happened. - let client = build_registry_client(); - - // composer.lock entry with no dist URL. - let entry = LockfileEntry { - ecosystem: "composer", - source_kind: SourceKind::Unspecified, - name: "monolog/monolog".into(), - version: "3.5.0".into(), - purl: "pkg:composer/monolog/monolog@3.5.0".into(), - resolved: None, - integrity: LockIntegrity::Sha1Hex("0".repeat(40)), - }; - match fetch_and_stage(&entry, &client).await { - Err(FetchError::Unverifiable(msg)) => { - assert!(msg.contains("no dist URL"), "{msg}") - } - other => panic!("expected composer Unverifiable, got {other:?}"), - } - - // Gem entry (safe coordinates) with no download URL. - let entry = LockfileEntry { - ecosystem: "gem", - source_kind: SourceKind::Unspecified, - name: "rails".into(), - version: "7.1.0".into(), - purl: "pkg:gem/rails@7.1.0".into(), - resolved: None, - integrity: LockIntegrity::Sha256Hex("0".repeat(64)), - }; - match fetch_and_stage(&entry, &client).await { - Err(FetchError::Unverifiable(msg)) => { - assert!(msg.contains("no download URL"), "{msg}") - } - other => panic!("expected gem Unverifiable, got {other:?}"), - } - - // Go modules verify via the go.sum h1 dirhash ONLY: any other - // integrity kind refuses before the URL is even built. - let entry = LockfileEntry { - ecosystem: "golang", - source_kind: SourceKind::Unspecified, - name: "github.com/x/y".into(), - version: "v1.0.0".into(), - purl: "pkg:golang/github.com/x/y@v1.0.0".into(), - resolved: Some("http://127.0.0.1:1/m.zip".into()), - integrity: LockIntegrity::Sha256Hex("0".repeat(64)), - }; - match fetch_and_stage(&entry, &client).await { - Err(FetchError::Unverifiable(msg)) => { - assert!(msg.contains("h1 dirhash"), "{msg}") - } - other => panic!("expected golang Unverifiable, got {other:?}"), - } - } - - #[tokio::test] - async fn cargo_crate_without_cargo_toml_refuses() { - // A sha256-VERIFIED .crate that extracts without a Cargo.toml is not - // a crate — the post-extraction shape check must fail the fetch. - let crate_bytes = make_tgz(&[("left-pad-1.3.0/src/lib.rs", b"pub fn pad() {}\n", false)]); - let sha = hex::encode(Sha256::digest(&crate_bytes)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/left-pad-1.3.0.crate")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(crate_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "cargo", - source_kind: SourceKind::Unspecified, - name: "left-pad".into(), - version: "1.3.0".into(), - purl: "pkg:cargo/left-pad@1.3.0".into(), - resolved: Some(format!("{}/left-pad/left-pad-1.3.0.crate", mock.uri())), - integrity: LockIntegrity::Sha256Hex(sha), - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("no Cargo.toml"), "{msg}"), - other => panic!("expected shape refusal, got {other:?}"), - } - } - - #[tokio::test] - async fn composer_dist_without_composer_json_refuses() { - // sha1-verified zipball whose lone top dir carries no composer.json: - // the layout detection strips the top dir, finds nothing, refuses. - let zip_bytes = make_zip(&[("pkg-1.0/README.md", b"# not a composer package\n")]); - let sha1 = hex::encode(Sha1::digest(&zip_bytes)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/dists/pkg-1.0.zip")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(zip_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "composer", - source_kind: SourceKind::Unspecified, - name: "acme/pkg".into(), - version: "1.0.0".into(), - purl: "pkg:composer/acme/pkg@1.0.0".into(), - resolved: Some(format!("{}/dists/pkg-1.0.zip", mock.uri())), - integrity: LockIntegrity::Sha1Hex(sha1), - }; - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("no composer.json"), "{msg}"), - other => panic!("expected shape refusal, got {other:?}"), - } - } - - #[tokio::test] - async fn npm_tarball_without_package_json_refuses() { - // SRI-verified tarball with no package.json — not an npm package. - let tgz = make_tgz(&[("package/index.js", b"module.exports = 1;\n", false)]); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/-/left-pad-1.3.0.tgz")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(tgz.clone())) - .mount(&mock) - .await; - - let entry = npm_entry( - Some(format!("{}/left-pad/-/left-pad-1.3.0.tgz", mock.uri())), - LockIntegrity::Sri(sri_of(&tgz)), - ); - match fetch_and_stage(&entry, &build_registry_client()).await { - Err(FetchError::Failed(msg)) => assert!(msg.contains("no package.json"), "{msg}"), - other => panic!("expected shape refusal, got {other:?}"), - } - } - - #[tokio::test] - #[serial_test::serial] - async fn cargo_conventional_url_honors_registry_override() { - // Cargo.lock records no `resolved` URL for registry crates — the - // conventional `{base}/{name}/{name}-{version}.crate` construction - // (and the SOCKET_CRATES_REGISTRY override feeding it) must run. - let crate_bytes = make_tgz(&[( - "left-pad-1.3.0/Cargo.toml", - b"[package]\nname = \"left-pad\"\n", - false, - )]); - let sha = hex::encode(Sha256::digest(&crate_bytes)); - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/left-pad/left-pad-1.3.0.crate")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(crate_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "cargo", - source_kind: SourceKind::Unspecified, - name: "left-pad".into(), - version: "1.3.0".into(), - purl: "pkg:cargo/left-pad@1.3.0".into(), - resolved: None, - integrity: LockIntegrity::Sha256Hex(sha), - }; - let saved = std::env::var("SOCKET_CRATES_REGISTRY").ok(); - // Trailing slash on purpose: the base must be trimmed before use. - std::env::set_var("SOCKET_CRATES_REGISTRY", format!("{}/", mock.uri())); - let result = fetch_and_stage(&entry, &build_registry_client()).await; - match saved { - Some(v) => std::env::set_var("SOCKET_CRATES_REGISTRY", v), - None => std::env::remove_var("SOCKET_CRATES_REGISTRY"), - } - let fetched = result.expect("the conventional crate URL must fetch"); - assert_eq!( - fetched.url, - format!("{}/left-pad/left-pad-1.3.0.crate", mock.uri()), - "conventional URL: {{base}}/{{name}}/{{name}}-{{version}}.crate" - ); - assert!(fetched.dir().await.unwrap().join("Cargo.toml").is_file()); - } - - #[tokio::test] - #[serial_test::serial] - async fn golang_conventional_url_escapes_name_and_version() { - // No resolved URL → the conventional GOPROXY zip URL, with the - // module-path CASE ESCAPING applied to BOTH the name and the version - // (an uppercase letter becomes `!lowercase` in the URL, while the - // zip's interior prefix keeps the unescaped coordinates). - let prefix = "github.com/Azure/y@v1.0.0-RC1/"; - let files: [(&str, &[u8]); 1] = [("go.mod", b"module github.com/Azure/y\n")]; - let zip_bytes = make_module_zip(prefix, &files); - let expected_h1 = spec_h1(&files, prefix); - - let mock = MockServer::start().await; - Mock::given(method("GET")) - .and(url_path("/github.com/!azure/y/@v/v1.0.0-!r!c1.zip")) - .respond_with(ResponseTemplate::new(200).set_body_bytes(zip_bytes)) - .mount(&mock) - .await; - - let entry = LockfileEntry { - ecosystem: "golang", - source_kind: SourceKind::Unspecified, - name: "github.com/Azure/y".into(), - version: "v1.0.0-RC1".into(), - purl: "pkg:golang/github.com/Azure/y@v1.0.0-RC1".into(), - resolved: None, - integrity: LockIntegrity::GoH1(expected_h1), - }; - let saved_socket = std::env::var("SOCKET_GOPROXY").ok(); - let saved = std::env::var("GOPROXY").ok(); - std::env::set_var("SOCKET_GOPROXY", mock.uri()); - std::env::remove_var("GOPROXY"); - let result = fetch_and_stage(&entry, &build_registry_client()).await; - match saved_socket { - Some(v) => std::env::set_var("SOCKET_GOPROXY", v), - None => std::env::remove_var("SOCKET_GOPROXY"), - } - match saved { - Some(v) => std::env::set_var("GOPROXY", v), - None => std::env::remove_var("GOPROXY"), - } - let fetched = result.expect("the conventional module zip URL must fetch"); - assert_eq!( - fetched.url, - format!("{}/github.com/!azure/y/@v/v1.0.0-!r!c1.zip", mock.uri()), - "case escaping must apply to the name AND the version" - ); - assert!(fetched.dir().await.unwrap().join("go.mod").is_file()); - } - - /// The pre-download refusals the vendor loop's service deferral mirrors - /// are exactly the ones `fetch_and_stage` raises before any request. - #[tokio::test] - #[serial_test::serial] - async fn refusal_before_download_matches_the_fetchers_first_refusals() { - let entry = |ecosystem: &'static str, resolved: Option<&str>, integrity: LockIntegrity| { - LockfileEntry { - ecosystem, - name: "example.com/mod".into(), - version: "1.0.0".into(), - purl: format!("pkg:{ecosystem}/example.com/mod@1.0.0"), - resolved: resolved.map(str::to_string), - integrity, - source_kind: SourceKind::Unspecified, - } - }; - let saved = std::env::var("GOPROXY").ok(); - std::env::set_var("GOPROXY", "off"); - let refused = [ - entry("npm", None, LockIntegrity::None), - entry( - "npm", - Some("http://127.0.0.1:1/x.tgz"), - LockIntegrity::BerryChecksum("8/abc".into()), - ), - entry("golang", None, LockIntegrity::GoH1("h1:AAAA".into())), - entry("golang", None, LockIntegrity::Sri("sha512-AAAA".into())), - entry("composer", None, LockIntegrity::Sha1Hex("aa".into())), - entry("nuget", Some("http://127.0.0.1:1/x"), LockIntegrity::Sri("x".into())), - ]; - for e in &refused { - let reason = refusal_before_download(e).expect("refused"); - match fetch_and_stage(e, &build_registry_client()).await { - Err(FetchError::Unverifiable(d)) => assert_eq!(d, reason), - other => panic!("{e:?}: {:?}", other.err()), - } - } - match saved { - Some(v) => std::env::set_var("GOPROXY", v), - None => std::env::remove_var("GOPROXY"), - } - let fetchable = [ - entry("npm", Some("http://127.0.0.1:1/x.tgz"), LockIntegrity::Sri("x".into())), - entry( - "npm", - Some("http://127.0.0.1:1/x.tgz"), - LockIntegrity::BerryChecksum("10c0/abc".into()), - ), - entry("cargo", None, LockIntegrity::Sha256Hex("aa".into())), - entry( - "golang", - Some("http://127.0.0.1:1/x.zip"), - LockIntegrity::GoH1("h1:AAAA".into()), - ), - entry( - "composer", - Some("http://127.0.0.1:1/x.zip"), - LockIntegrity::Sha1Hex("aa".into()), - ), - ]; - for e in &fetchable { - assert_eq!(refusal_before_download(e), None, "{e:?}"); - } - } - - /// go never sends a module path to a proxy when GOPROXY starts with - /// `off` / `direct`, or when the module matches GONOPROXY (defaulting to - /// GOPRIVATE). The pristine fetch must not either: it refuses before any - /// network I/O instead of falling back to proxy.golang.org. - #[tokio::test] - #[serial_test::serial] - async fn golang_fetch_never_uses_a_proxy_go_would_not() { - let mock = MockServer::start().await; - let entry = LockfileEntry { - ecosystem: "golang", - name: "example.com/private/mod".into(), - version: "v1.0.0".into(), - purl: "pkg:golang/example.com/private/mod@v1.0.0".into(), - resolved: None, - integrity: LockIntegrity::GoH1("h1:AAAA".into()), - source_kind: SourceKind::Unspecified, - }; - let keys = ["SOCKET_GOPROXY", "GOPROXY", "GOPRIVATE", "GONOPROXY"]; - let saved: Vec> = keys.iter().map(|k| std::env::var(k).ok()).collect(); - for k in keys { - std::env::remove_var(k); - } - let proxy = mock.uri(); - let cases: Vec<(String, &str, &str, bool)> = vec![ - ("off".into(), "", "", false), - ("direct".into(), "", "", false), - (format!("off,{proxy}"), "", "", false), - (format!("direct|{proxy}"), "", "", false), - (proxy.clone(), "example.com/private", "", false), - (proxy.clone(), "example.com/*", "", false), - (proxy.clone(), "*.example", "", true), - (proxy.clone(), "example.com/private", "other.example", true), - ]; - let mut outcomes = Vec::new(); - for (goproxy, goprivate, gonoproxy, uses_proxy) in &cases { - std::env::set_var("GOPROXY", goproxy); - std::env::set_var("GOPRIVATE", goprivate); - std::env::set_var("GONOPROXY", gonoproxy); - let result = fetch_and_stage(&entry, &build_registry_client()).await; - outcomes.push(( - goproxy.clone(), - *goprivate, - *gonoproxy, - *uses_proxy, - result.err(), - )); - } - for (k, v) in keys.iter().zip(saved) { - match v { - Some(v) => std::env::set_var(k, v), - None => std::env::remove_var(k), - } - } - for (goproxy, goprivate, gonoproxy, uses_proxy, err) in &outcomes { - let case = format!("GOPROXY={goproxy} GOPRIVATE={goprivate} GONOPROXY={gonoproxy}"); - if *uses_proxy { - assert!( - matches!(err, Some(FetchError::Failed(_))), - "{case}: {err:?}" - ); - } else { - assert!( - matches!(err, Some(FetchError::Unverifiable(d)) if d.contains("GO")), - "{case}: {err:?}" - ); - } - } - let hits = mock.received_requests().await.unwrap_or_default().len(); - assert_eq!(hits, 2, "only the two proxy-eligible cases reach the proxy"); - } - #[test] #[serial_test::serial] fn goproxy_base_env_precedence() { @@ -4525,7 +2449,6 @@ mod tests { // did not. A flat `composer archive`-built dist is the shape that // reaches this (the zipball layout is stripped first). let flat = make_zip(&[("root.txt", b"x"), ("./composer.json", b"{}")]); - assert!(!zip_has_single_top_dir(&flat).unwrap()); let extracted = tempfile::tempdir().unwrap(); extract_zip(&flat, extracted.path(), /*strip_first=*/ false).unwrap(); assert!( @@ -4544,60 +2467,6 @@ mod tests { assert!(validate_tgz(&dotted, &nowhere, Some("Cargo.toml")).unwrap()); } - /// A deferred source extracts to exactly what the eager fetch wrote — - /// same tree, same bytes, same modes — and says so only once. - #[tokio::test] - async fn deferred_extraction_materializes_the_eager_tree() { - let tgz = make_tgz(&[ - ("package/package.json", br#"{"name":"left-pad"}"#, false), - ("package/index.js", b"module.exports=1\n", false), - ("package/bin/cli.js", b"#!/usr/bin/env node\n", true), - ]); - let eager = tempfile::tempdir().unwrap(); - extract_tgz(&tgz, eager.path()).unwrap(); - - let tmp = tempfile::tempdir().unwrap(); - let dir = tmp.path().join("package"); - let bytes = tgz.clone(); - let fetched = FetchedPackage::pending( - dir.clone(), - "https://example.invalid/left-pad.tgz".to_string(), - tmp, - move |dest, skip| extract_tgz_skipping(&bytes, dest, skip), - ); - // The path is known before anything is written, and nothing is. - assert_eq!(fetched.dir_path(), dir); - assert!(!dir.exists(), "a pending source writes nothing until read"); - - assert_eq!(fetched.dir().await.unwrap(), dir); - for rel in ["package.json", "index.js", "bin/cli.js"] { - assert_eq!( - std::fs::read(dir.join(rel)).unwrap(), - std::fs::read(eager.path().join(rel)).unwrap(), - "{rel}" - ); - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt as _; - assert_eq!( - std::fs::metadata(dir.join(rel)) - .unwrap() - .permissions() - .mode() - & 0o777, - std::fs::metadata(eager.path().join(rel)) - .unwrap() - .permissions() - .mode() - & 0o777, - "{rel} mode" - ); - } - } - // A second read is the same answer, not a second extraction. - assert_eq!(fetched.dir().await.unwrap(), dir); - } - /// A zip with a unix mode per entry, so the parallel walk's `fchmod` /// can be checked against what the archive declares. fn make_zip_with_modes(files: &[(&str, &[u8], u32)]) -> Vec { @@ -4841,197 +2710,6 @@ mod tests { ); } - /// Staging a pending source straight into the vendor stage must leave - /// exactly what extracting it and copying the tree out left: same - /// files, same bytes, same modes, same skip. - #[tokio::test] - async fn staging_a_pending_source_equals_extract_then_copy() { - let tgz = make_tgz(&[ - ("crate/Cargo.toml", b"[package]\nname=\"x\"\n", false), - ("crate/src/lib.rs", b"pub fn x() {}\n", false), - ("crate/build.sh", b"#!/bin/sh\n", true), - ("crate/.cargo-checksum.json", b"{}", false), - ("crate/vendor/.cargo-checksum.json", b"{}", false), - ]); - for skip in [None, Some(".cargo-checksum.json")] { - // The oracle: what the eager fetch + `fresh_copy` produced. - let tmp = tempfile::tempdir().unwrap(); - let extracted = tmp.path().join("crate"); - extract_tgz(&tgz, &extracted).unwrap(); - let oracle = tempfile::tempdir().unwrap(); - let oracle_stage = oracle.path().join("stage"); - crate::patch::copy_tree::fresh_copy(&extracted, &oracle_stage, skip) - .await - .unwrap(); - - let holder = tempfile::tempdir().unwrap(); - let bytes = tgz.clone(); - let fetched = FetchedPackage::pending( - holder.path().join("crate"), - "https://example.invalid/x.crate".to_string(), - holder, - move |dest, skip| extract_tgz_skipping(&bytes, dest, skip), - ); - let staged_root = tempfile::tempdir().unwrap(); - let staged = staged_root.path().join("stage"); - fetched.stage_into(&staged, skip).await.unwrap(); - assert!( - !fetched.dir_path().exists(), - "a direct stage writes no tempdir tree" - ); - assert_eq!(tree_of(&staged), tree_of(&oracle_stage), "skip: {skip:?}"); - assert_eq!(dirs_of(&staged), dirs_of(&oracle_stage), "skip: {skip:?}"); - } - } - - /// And when something read the tree first, the stage is still the same - /// — it just comes off that tree instead of a second inflate. - #[tokio::test] - async fn staging_after_materializing_still_matches() { - let tgz = make_tgz(&[ - ("pkg/a.rb", b"A\n", false), - ("pkg/bin/run", b"#!/bin/sh\n", true), - ]); - let holder = tempfile::tempdir().unwrap(); - let bytes = tgz.clone(); - let fetched = FetchedPackage::pending( - holder.path().join("pkg"), - "https://example.invalid/x.gem".to_string(), - holder, - move |dest, skip| extract_tgz_skipping(&bytes, dest, skip), - ); - let materialized = fetched.dir().await.unwrap().to_path_buf(); - let staged_root = tempfile::tempdir().unwrap(); - let staged = staged_root.path().join("stage"); - fetched.stage_into(&staged, None).await.unwrap(); - assert_eq!(tree_of(&staged), tree_of(&materialized)); - assert_eq!(dirs_of(&staged), dirs_of(&materialized)); - } - - /// Once the tree is on disk the archive is dead weight: a run that - /// materialises its sources must not carry every one of them to the end - /// of the vendor loop, which is more than the eager fetch ever held. - #[tokio::test] - async fn materializing_frees_the_archive_bytes() { - struct Tattle { - bytes: Vec, - freed: std::sync::Arc, - } - impl Drop for Tattle { - fn drop(&mut self) { - self.freed.store(true, std::sync::atomic::Ordering::SeqCst); - } - } - let freed = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); - let tattle = Tattle { - bytes: make_tgz(&[("pkg/a.txt", b"hello", false)]), - freed: std::sync::Arc::clone(&freed), - }; - let holder = tempfile::tempdir().unwrap(); - let fetched = FetchedPackage::pending( - holder.path().join("pkg"), - "https://example.invalid/x.tgz".to_string(), - holder, - move |dest, skip| extract_tgz_skipping(&tattle.bytes, dest, skip), - ); - assert!(!freed.load(std::sync::atomic::Ordering::SeqCst)); - fetched.dir().await.unwrap(); - assert!( - freed.load(std::sync::atomic::Ordering::SeqCst), - "the archive is still held after its tree reached the tempdir" - ); - // And the tree is still the one thing every later caller reads. - let stage_root = tempfile::tempdir().unwrap(); - let stage = stage_root.path().join("stage"); - fetched.stage_into(&stage, None).await.unwrap(); - assert_eq!(tree_of(&stage), tree_of(fetched.dir().await.unwrap())); - } - - /// And the source nothing ever reads — the case the deferral exists - /// for — is let go when the loop moves past its purl, so a run holds - /// one archive rather than every one it fetched. - #[test] - fn releasing_an_unread_source_frees_the_archive_bytes() { - struct Tattle { - bytes: Vec, - freed: std::sync::Arc, - } - impl Drop for Tattle { - fn drop(&mut self) { - self.freed.store(true, std::sync::atomic::Ordering::SeqCst); - } - } - let freed = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); - let tattle = Tattle { - bytes: make_tgz(&[("pkg/a.txt", b"hello", false)]), - freed: std::sync::Arc::clone(&freed), - }; - let holder = tempfile::tempdir().unwrap(); - let fetched = FetchedPackage::pending( - holder.path().join("pkg"), - "https://example.invalid/x.tgz".to_string(), - holder, - move |dest, skip| extract_tgz_skipping(&tattle.bytes, dest, skip), - ); - crate::vendor::source::PackageSource::Pending(&fetched).release(); - assert!( - freed.load(std::sync::atomic::Ordering::SeqCst), - "a released source is still holding its archive" - ); - // Releasing twice is the same nothing. - fetched.release(); - } - - /// A stage that cannot be cleared or created reports what the copy out - /// of the tempdir reported — the backends wrap it in their own wording. - #[tokio::test] - async fn a_stage_that_cannot_be_made_reads_as_the_copy_read() { - let tgz = make_tgz(&[("pkg/a.txt", b"hello", false)]); - let bytes = tgz.clone(); - let holder = tempfile::tempdir().unwrap(); - let fetched = FetchedPackage::pending( - holder.path().join("pkg"), - "https://example.invalid/x.tgz".to_string(), - holder, - move |dest, skip| extract_tgz_skipping(&bytes, dest, skip), - ); - // A stage whose parent is a FILE: `create_dir_all` fails the same - // way for `fresh_copy` and for a direct stage. - let root = tempfile::tempdir().unwrap(); - let blocker = root.path().join("blocked"); - std::fs::write(&blocker, b"not a dir").unwrap(); - let stage = blocker.join("stage"); - - let installed = tempfile::tempdir().unwrap(); - let oracle = crate::patch::copy_tree::fresh_copy(installed.path(), &stage, None) - .await - .unwrap_err() - .to_string(); - let direct = fetched.stage_into(&stage, None).await.unwrap_err(); - assert_eq!(direct, oracle); - } - - /// Every DIRECTORY under `root`, relative. `tree_of` lists files, so it - /// cannot see a stage that dropped a directory whose only member was - /// skipped — which is exactly what the fixtures below are built to - /// catch. - fn dirs_of(root: &Path) -> Vec { - let mut out: Vec = walkdir::WalkDir::new(root) - .into_iter() - .flatten() - .filter(|e| e.file_type().is_dir() && e.path() != root) - .map(|e| { - e.path() - .strip_prefix(root) - .unwrap() - .to_string_lossy() - .into_owned() - }) - .collect(); - out.sort(); - out - } - /// Every file under `root`, relative, with its bytes and unix mode. fn tree_of(root: &Path) -> Vec<(String, Vec, u32)> { let mut out: Vec<(String, Vec, u32)> = walkdir::WalkDir::new(root) @@ -5060,29 +2738,6 @@ mod tests { out } - /// An extraction that cannot be written reports the same failure to - /// every later caller, and never half-answers. - #[tokio::test] - async fn deferred_extraction_failure_is_sticky() { - let tmp = tempfile::tempdir().unwrap(); - let dir = tmp.path().join("package"); - let fetched = FetchedPackage::pending( - dir.clone(), - "https://example.invalid/x.tgz".to_string(), - tmp, - |_, _| Err("cannot create /nope: nope".to_string()), - ); - assert_eq!( - fetched.dir().await.unwrap_err(), - "cannot create /nope: nope" - ); - assert_eq!( - fetched.dir().await.unwrap_err(), - "cannot create /nope: nope", - "the outcome is decided once and shared" - ); - } - #[test] fn verify_go_h1_accepts_matching_dirhash() { // The SUCCESS path is the golang service-download content verifier — @@ -5109,21 +2764,18 @@ mod tests { &LockIntegrity::BerryChecksum(format!("8/{}", "0".repeat(128))), ) .unwrap_err(); - assert!(err.contains("cacheKey other than 10c0"), "{err}"); + assert!(err.contains("cannot verify tarball bytes"), "{err}"); // 10c0: the cache-zip rebuild round-trips, and a tampered checksum // names the mismatch. let tgz = make_tgz(&[("package/package.json", br#"{"name":"left-pad"}"#, false)]); - let good = super::super::berry_zip::berry_cache_checksum_10c0(&tgz, "left-pad").unwrap(); - artifact_matches_integrity(&tgz, "left-pad", &LockIntegrity::BerryChecksum(good)) - .expect("the rebuilt cache checksum must match"); let err = artifact_matches_integrity( &tgz, "left-pad", &LockIntegrity::BerryChecksum(format!("10c0/{}", "0".repeat(128))), ) .unwrap_err(); - assert!(err.contains("mismatch"), "{err}"); + assert!(err.contains("cannot verify tarball bytes"), "{err}"); // GoH1 has a dedicated fetch-path verifier; None is reachable from a // repair against an npm-era lock recording no integrity. Both refuse. diff --git a/crates/socket-patch-core/src/vendor/reuse.rs b/crates/socket-patch-core/src/vendor/reuse.rs index 539d0bc4c..9fa5bc1a0 100644 --- a/crates/socket-patch-core/src/vendor/reuse.rs +++ b/crates/socket-patch-core/src/vendor/reuse.rs @@ -1,50 +1,3 @@ -//! Reuse of an already-committed, file-shaped vendored artifact (npm -//! tarball, pypi wheel) instead of acquiring a new one. -//! -//! The directory-shaped backends (cargo, golang, composer, gem, maven, -//! nuget) decide "in sync" from the COMMITTED artifact before they ever -//! consult the patch service. The archive-shaped backends acquire (service -//! download, else a local deterministic pack) and compare the lock's digests -//! with those NEW bytes, so without this a prebuilt ↔ local source flip -//! between two runs (a service outage, or its recovery) would rewrite the -//! lock and the tarball even though nothing needed vendoring. This module -//! gives them the same rule: when the ledger vouches for the committed -//! artifact and the bytes verify, reuse them. -//! -//! Anchor: the vendor ledger entry (`.socket/vendor/state.json`) recorded -//! the artifact's path + sha256 when it was wired. Reuse requires, fail -//! closed at every step (any miss falls through to the caller's normal -//! acquisition): -//! -//! 1. a non-empty patch record (nothing to verify ⇒ never reused); -//! 2. a canonical, uuid-bound artifact path (`checked_artifact_path`) — an -//! artifact under another uuid's directory is never reused; -//! 3. no symlink anywhere on the path below the project root; -//! 4. a regular file (FIFO-safe open), at most `MAX_HEALTH_HASH_BYTES`, read -//! ONCE into memory — every later check runs on that one buffer; -//! 5. `sha256(bytes)` == the ledger sha256 (and the ledger size, when -//! recorded) — the tamper anchor for unpatched members and re-encodings; -//! 6. the archive is CANONICAL (strict decode: every tarball entry under -//! `package/`, only regular/directory entries, no exact or case-folded -//! duplicate names; the same name rules for wheels) — so the decoded -//! members are exactly what an installer extracts, and the afterHash -//! check below cannot be satisfied by one entry while a sibling the -//! installer prefers (another top-level dir, a type-`7` twin, a -//! case-variant name) carries different bytes; -//! 7. every `record.files` afterHash verifies inside the decoded members. -//! -//! The lockfile is deliberately NOT an input: the flavor's own in-sync code -//! runs afterwards against the reused bytes' facts, so a lock that already -//! pins them is a true no-op and a lock that drifted is re-pinned to the -//! verified committed bytes. -//! -//! Residual trust (the same level `repair` and `vex`'s -//! `check_vendored_artifact` already grant the ledger): an attacker who edits -//! an unpatched member AND rewrites the ledger sha256 keeps the edit, because -//! the afterHash check only covers patched members. -//! -//! Read-only: nothing here writes or touches the network. - use std::collections::HashMap; use std::path::Path; @@ -282,12 +235,21 @@ pub(crate) async fn verify_committed_artifact( // Members from the SAME buffer. let is_tarball = rel_path.ends_with(".tgz") || rel_path.ends_with(".tar.gz"); - let is_wheel = rel_path.ends_with(".whl"); + let is_wheel = rel_path.ends_with(".whl") || rel_path.ends_with(".zip"); if !is_tarball && !is_wheel { return Err(ReuseMiss::Unreadable); } + let python_name = (entry.ecosystem == "pypi").then(|| rel_path.clone()); let (bytes, members) = tokio::task::spawn_blocking(move || { - let members = if is_tarball { + let members = if let Some(name) = &python_name { + super::pypi_distribution::read_members(&bytes, name).map_err(|error| { + if error == "vendor_artifact_non_canonical" { + ReuseMiss::NonCanonical + } else { + ReuseMiss::Unreadable + } + }) + } else if is_tarball { crate::patch::package::read_archive_bytes_to_map_strict(&bytes).map_err(|e| match e { crate::patch::package::ArchiveError::NonCanonical(_) => ReuseMiss::NonCanonical, _ => ReuseMiss::Unreadable, @@ -306,7 +268,12 @@ pub(crate) async fn verify_committed_artifact( .await .map_err(|_| ReuseMiss::Unreadable)?; let members = members?; - verify_member_map(&members, record).map_err(ReuseMiss::MemberMismatch)?; + if entry.ecosystem == "pypi" { + super::pypi_distribution::verify_members(&members, &rel_path, record) + } else { + verify_member_map(&members, record) + } + .map_err(ReuseMiss::MemberMismatch)?; Ok(CommittedArtifact { rel_path, @@ -444,6 +411,7 @@ mod tests { base_purl: "pkg:npm/left-pad@1.3.0".into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel.into(), sha256: hex::encode(Sha256::digest(bytes)), size: Some(bytes.len() as u64), diff --git a/crates/socket-patch-core/src/vendor/service_fetch.rs b/crates/socket-patch-core/src/vendor/service_fetch.rs index 3975db269..7a3aeb3d9 100644 --- a/crates/socket-patch-core/src/vendor/service_fetch.rs +++ b/crates/socket-patch-core/src/vendor/service_fetch.rs @@ -20,15 +20,55 @@ use crate::vendor::{ VendorOutcome, VendorWarning, }; -/// A service archive whose bytes have passed integrity verification. -/// -/// Deliberately minimal: every consumer recomputes the hashes it needs from -/// `bytes` (so a service-downloaded artifact describes itself byte-identically -/// to a local build), so the service-reported sha1/md5/size are not re-carried. -/// The one exception is [`Self::sha256_hex`], which is OUR digest of the same -/// bytes, taken where they are already being walked. +pub(crate) fn required() -> VendorOutcome { + refused( + "vendor_prebuilt_required", + "vendoring requires a verified artifact from the patch service".to_string(), + ) +} + +pub(crate) async fn preview_service( + service: Option<&VendorServiceConfig>, + record: &PatchRecord, + extract: impl FnOnce(&[u8], &std::path::Path) -> Result<(), String> + Send + 'static, +) -> Result<(), Box> { + if let Some(outcome) = service_offline_conflict(service) { + return Err(Box::new(outcome)); + } + let cfg = service + .filter(|cfg| cfg.service_enabled()) + .ok_or_else(|| Box::new(required()))?; + let policy = ServicePolicy::new(cfg, ServiceTerminal::Refused); + let archive = match policy.settle::<()>( + fetch_verified_archive(cfg, &record.uuid).await, + "archive", + &record.uuid, + &mut Vec::new(), + ) { + Ok(archive) => archive, + Err(ServiceAttempt::HardFail(outcome)) => return Err(outcome), + _ => return Err(Box::new(required())), + }; + let stage = tempfile::tempdir() + .map_err(|e| Box::new(refused("vendor_prebuilt_extract_failed", e.to_string())))?; + super::registry_fetch::extract_on_blocking_pool(archive.bytes, stage.path(), extract) + .await + .map_err(|e| Box::new(refused("vendor_prebuilt_extract_failed", e)))?; + if !super::common::copy_matches_after_hashes(stage.path(), &record.files).await { + return Err(Box::new(refused( + "vendor_prebuilt_layout_mismatch", + format!( + "prebuilt archive for {} does not carry its patched files", + record.uuid + ), + ))); + } + Ok(()) +} + #[derive(Debug)] pub(crate) struct VerifiedArchive { + pub yarn_berry10c0: Option, /// The verified archive bytes (npm `.tgz`, pypi `.whl`/sdist, cargo /// `.crate`, golang/composer `.zip`, gem `.gem`, …). pub bytes: Vec, @@ -61,12 +101,6 @@ impl VerifiedArchive { } } -/// Result of attempting a service download for one patch UUID. -/// -/// The backends map this onto the `auto` / `service` policy: `Ready` → use it; -/// `Pending` / `Unavailable` / `Failed` → fall back to a local build under -/// `auto` (or hard-fail under `service`); `IntegrityMismatch` → ALWAYS a hard -/// error regardless of mode. #[derive(Debug)] pub(crate) enum ServiceArtifact { Ready(VerifiedArchive), @@ -128,6 +162,7 @@ pub(crate) async fn fetch_verified_archive( } ServiceArtifact::Ready(VerifiedArchive { + yarn_berry10c0: pkg.yarn_berry10c0, bytes: pkg.tarball, integrity_sri: pkg.integrity_sri, sha256_hex: std::sync::OnceLock::new(), @@ -152,16 +187,13 @@ pub(crate) async fn claim_prestaged( } } -/// Outcome of a backend's service fast path, mapped onto the `auto` / -/// `service` fallback policy by [`ServicePolicy`]. +/// Outcome of a backend's verified service download. pub(crate) enum ServiceAttempt { /// The verified service artifact was used; `T` is what the backend made /// of it. Used(T), /// Bubble this terminal outcome (boxed — `VendorOutcome` is large). HardFail(Box), - /// Fall back to the local rebuild. - FallBack, } /// The single-file outcome for the Tier-A backends (maven `.jar`, nuget @@ -177,21 +209,14 @@ pub(crate) enum ServiceTerminal<'a> { Failure(&'a str), } -/// The `auto` / `service` fallback policy every service-backed backend -/// shares: `service` refuses every miss, `auto` warns (or, for a plain -/// `Unavailable`, stays quiet) and builds locally. Tampered bytes are always -/// terminal. +/// Every service miss is terminal; backends never construct an archive locally. pub(crate) struct ServicePolicy<'a> { - requires_service: bool, terminal: ServiceTerminal<'a>, } impl<'a> ServicePolicy<'a> { - pub(crate) fn new(cfg: &VendorServiceConfig, terminal: ServiceTerminal<'a>) -> Self { - Self { - requires_service: cfg.source.requires_service(), - terminal, - } + pub(crate) fn new(_cfg: &VendorServiceConfig, terminal: ServiceTerminal<'a>) -> Self { + Self { terminal } } pub(crate) fn hard(&self, code: &'static str, detail: String) -> ServiceAttempt { @@ -201,23 +226,15 @@ impl<'a> ServicePolicy<'a> { })) } - /// `service` refuses with `reason`; `auto` warns under `code` and falls - /// back. + /// Refuse an unavailable server artifact. pub(crate) fn miss( &self, warnings: &mut Vec, code: &'static str, reason: String, ) -> ServiceAttempt { - if self.requires_service { - self.hard("vendor_prebuilt_required", reason) - } else { - warnings.push(VendorWarning::new( - code, - format!("{reason}; building locally instead"), - )); - ServiceAttempt::FallBack - } + let _ = (warnings, code); + self.hard("vendor_prebuilt_required", reason) } /// The verified archive of a `Ready` outcome, or every other outcome @@ -232,10 +249,6 @@ impl<'a> ServicePolicy<'a> { ) -> Result> { match artifact { ServiceArtifact::Ready(archive) => Ok(archive), - // Bytes that fail integrity verification are an active tamper - // signal: ALWAYS a hard error, in `auto` exactly as in `service` - // — never a quiet local-build fallback ([`ServiceArtifact`]'s - // documented contract). ServiceArtifact::IntegrityMismatch(reason) => Err(self.hard( "vendor_prebuilt_integrity_mismatch", format!( @@ -248,12 +261,11 @@ impl<'a> ServicePolicy<'a> { "vendor_prebuilt_pending", format!("prebuilt {noun} is still building"), )), - // The common, quiet miss: not built / free-only / not found. - ServiceArtifact::Unavailable(reason) if self.requires_service => Err(self.hard( + // No artifact is available for these coordinates or entitlements. + ServiceArtifact::Unavailable(reason) => Err(self.hard( "vendor_prebuilt_required", format!("prebuilt {noun} unavailable: {reason}"), )), - ServiceArtifact::Unavailable(_) => Err(ServiceAttempt::FallBack), ServiceArtifact::Failed(reason) => Err(self.miss( warnings, "vendor_prebuilt_unavailable", @@ -264,8 +276,7 @@ impl<'a> ServicePolicy<'a> { } /// Download + integrity-verify the prebuilt patched archive for the Tier-A -/// backends, mapping each service outcome onto the `auto` / `service` fallback -/// policy. `noun` is the artifact kind used in messages (".jar" / ".nupkg"). +/// backends. `noun` is the artifact kind used in messages (".jar" / ".nupkg"). pub(crate) async fn service_archive_copy( service: Option<&VendorServiceConfig>, record: &PatchRecord, @@ -280,10 +291,10 @@ pub(crate) async fn service_archive_copy( return ServiceCopy::HardFail(Box::new(refusal)); } let Some(cfg) = service else { - return ServiceCopy::FallBack; + return ServiceCopy::HardFail(Box::new(super::service_fetch::required())); }; if !cfg.service_enabled() { - return ServiceCopy::FallBack; + return ServiceCopy::HardFail(Box::new(super::service_fetch::required())); } let policy = ServicePolicy::new(cfg, ServiceTerminal::Refused); let fetched = fetch_verified_archive(cfg, &record.uuid).await; @@ -294,8 +305,7 @@ pub(crate) async fn service_archive_copy( // The SRI proves the download is intact, not that it carries the // patch: the bytes are written verbatim and reported AlreadyPatched, // so every patched member must hash to its afterHash first (the - // Tier-B backends' extracted-tree check). Fail closed → `auto` - // falls back to the local rebuild. + // extracted-tree check). A mismatching artifact always fails closed. if !archive .prestaged .zip_verdict(&record.files) @@ -519,7 +529,7 @@ mod tests { let wrong = PackedTarball::from_bytes(b"completely different bytes").integrity; mount_granted(&server, &wrong, body).await; let mut cfg = cfg_for(&server); - cfg.source = VendorSource::Auto; + cfg.source = VendorSource::Service; let mut warnings = Vec::new(); match service_archive_copy(Some(&cfg), &record(), "x", ".jar", &mut warnings).await { ServiceCopy::HardFail(outcome) => match *outcome { @@ -529,16 +539,9 @@ mod tests { other => panic!("expected Refused, got {other:?}"), }, ServiceCopy::Used(_) => panic!("tampered bytes must never be used"), - ServiceCopy::FallBack => { - panic!("auto fell back to a local build on tampered bytes") - } } } - /// `--vendor-source=service --offline` is a fail-closed refusal (the same - /// `vendor_service_offline_conflict` the other backends give via - /// `service_offline_conflict`), never a silent local-build fallback — - /// maven/nuget funnel through here and have no earlier guard. #[tokio::test] async fn service_copy_offline_conflict_hard_fails() { let server = MockServer::start().await; @@ -553,25 +556,21 @@ mod tests { other => panic!("expected Refused, got {other:?}"), }, ServiceCopy::Used(_) => panic!("offline run must not download"), - ServiceCopy::FallBack => { - panic!("--vendor-source=service --offline fell back to a local build") - } } } - /// Under `auto`, offline stays a quiet fallback to the local build. #[tokio::test] - async fn service_copy_offline_auto_falls_back() { + async fn service_copy_offline_refuses_without_advisory() { let server = MockServer::start().await; let mut cfg = cfg_for(&server); - cfg.source = VendorSource::Auto; + cfg.source = VendorSource::Service; cfg.offline = true; let mut warnings = Vec::new(); assert!(matches!( service_archive_copy(Some(&cfg), &record(), "x", ".jar", &mut warnings).await, - ServiceCopy::FallBack + ServiceCopy::HardFail(_) )); - assert!(warnings.is_empty(), "quiet fallback, no warning"); + assert!(warnings.is_empty(), "the refusal needs no advisory"); } /// A config without a client is a quiet Unavailable, not a panic. @@ -579,7 +578,7 @@ mod tests { async fn unavailable_when_client_absent() { let cfg = VendorServiceConfig { maven_config: None, - source: VendorSource::Auto, + source: VendorSource::Service, client: None, use_public_proxy: false, vendor_url: None, @@ -649,7 +648,6 @@ mod tests { { ServiceCopy::Used(bytes) => assert_eq!(bytes, body), ServiceCopy::HardFail(outcome) => panic!("expected Used, got HardFail({outcome:?})"), - ServiceCopy::FallBack => panic!("expected Used, got FallBack"), } assert_eq!(warnings.len(), 1, "exactly one downloaded advisory"); assert_eq!(warnings[0].code, "vendor_prebuilt_downloaded"); @@ -667,29 +665,23 @@ mod tests { ); } - /// Pending under `auto`: warn (`vendor_prebuilt_pending`, "still - /// building; building locally instead") and fall back to the local build. #[tokio::test] - async fn service_copy_pending_auto_warns_and_falls_back() { + async fn service_copy_pending_refuses_without_fallback() { let server = MockServer::start().await; mount_status(&server, "pending_build").await; let mut cfg = cfg_for(&server); - cfg.source = VendorSource::Auto; + cfg.source = VendorSource::Service; let mut warnings = Vec::new(); assert!(matches!( service_archive_copy(Some(&cfg), &record(), "x", ".jar", &mut warnings).await, - ServiceCopy::FallBack + ServiceCopy::HardFail(_) )); - assert_eq!(warnings.len(), 1); - assert_eq!(warnings[0].code, "vendor_prebuilt_pending"); - assert_eq!( - warnings[0].detail, - "prebuilt .jar is still building; building locally instead" + assert!( + warnings.is_empty(), + "a hard failure is not a fallback advisory" ); } - /// Pending under `--vendor-source=service`: a hard `vendor_prebuilt_required` - /// refusal (never a local-build fallback), and no warning alongside it. #[tokio::test] async fn service_copy_pending_service_hard_fails() { let server = MockServer::start().await; @@ -712,9 +704,6 @@ mod tests { other => panic!("expected Refused, got {other:?}"), }, ServiceCopy::Used(_) => panic!("pending build must not yield bytes"), - ServiceCopy::FallBack => { - panic!("--vendor-source=service fell back on a pending build") - } } assert!(warnings.is_empty(), "the hard-fail path must not warn"); } @@ -743,26 +732,21 @@ mod tests { other => panic!("expected Refused, got {other:?}"), }, ServiceCopy::Used(_) => panic!("unavailable archive must not yield bytes"), - ServiceCopy::FallBack => { - panic!("--vendor-source=service fell back on an unavailable archive") - } } assert!(warnings.is_empty(), "the hard-fail path must not warn"); } - /// Unavailable under `auto` is a QUIET fallback — no warning. This is the - /// deliberate asymmetry with Pending/Failed (mirrors the golang backend's - /// mapping): a terminal miss is routine, not noteworthy. + /// An unavailable artifact is a refusal, with no fallback advisory. #[tokio::test] - async fn service_copy_unavailable_auto_is_quiet_fallback() { + async fn service_copy_unavailable_refuses_without_fallback() { let server = MockServer::start().await; mount_status(&server, "not_found").await; let mut cfg = cfg_for(&server); - cfg.source = VendorSource::Auto; + cfg.source = VendorSource::Service; let mut warnings = Vec::new(); assert!(matches!( service_archive_copy(Some(&cfg), &record(), "x", ".jar", &mut warnings).await, - ServiceCopy::FallBack + ServiceCopy::HardFail(_) )); assert!( warnings.is_empty(), @@ -774,7 +758,7 @@ mod tests { /// deliberately reuses the `vendor_prebuilt_unavailable` warning code /// (matching the golang mapping) rather than a dedicated one. #[tokio::test] - async fn service_copy_failed_auto_warns_and_falls_back() { + async fn service_copy_failed_refuses_without_fallback() { let server = MockServer::start().await; Mock::given(method("POST")) .and(path("/v0/orgs/acme/patches/package")) @@ -782,30 +766,18 @@ mod tests { .mount(&server) .await; let mut cfg = cfg_for(&server); - cfg.source = VendorSource::Auto; + cfg.source = VendorSource::Service; let mut warnings = Vec::new(); assert!(matches!( service_archive_copy(Some(&cfg), &record(), "x", ".jar", &mut warnings).await, - ServiceCopy::FallBack + ServiceCopy::HardFail(_) )); - assert_eq!(warnings.len(), 1); - assert_eq!(warnings[0].code, "vendor_prebuilt_unavailable"); - assert!( - warnings[0] - .detail - .starts_with("patch service request failed ("), - "{}", - warnings[0].detail - ); assert!( - warnings[0].detail.ends_with("; building locally instead"), - "{}", - warnings[0].detail + warnings.is_empty(), + "a hard failure is not a fallback advisory" ); } - /// Transport failure under `--vendor-source=service`: hard refusal - /// (`vendor_prebuilt_required`), never a local-build fallback. #[tokio::test] async fn service_copy_failed_service_hard_fails() { let server = MockServer::start().await; @@ -835,9 +807,6 @@ mod tests { other => panic!("expected Refused, got {other:?}"), }, ServiceCopy::Used(_) => panic!("a failed request must not yield bytes"), - ServiceCopy::FallBack => { - panic!("--vendor-source=service fell back on a transport failure") - } } assert!(warnings.is_empty(), "the hard-fail path must not warn"); } @@ -853,6 +822,7 @@ mod tests { .mount(&server) .await; let archive = VerifiedArchive { + yarn_berry10c0: None, bytes: Vec::new(), integrity_sri: String::new(), sha256_hex: std::sync::OnceLock::new(), @@ -879,10 +849,7 @@ mod tests { } } - /// A served archive that passes its SRI but does not carry the - /// record's patched bytes is never `Used`: `service` refuses, `auto` - /// falls back loudly — and neither pushes the `vendor_prebuilt_downloaded` - /// advisory for bytes it rejected. + /// Valid transfer integrity cannot substitute for the record's patched bytes. #[tokio::test] async fn service_copy_ready_failing_after_hashes_is_rejected() { use crate::hash::git_sha256::compute_git_sha256_from_bytes; @@ -903,7 +870,7 @@ mod tests { after_hash: compute_git_sha256_from_bytes(b"patched"), }, ); - for source in [VendorSource::Service, VendorSource::Auto] { + for source in [VendorSource::Service] { let server = MockServer::start().await; let sri = PackedTarball::from_bytes(&body).integrity; mount_granted(&server, &sri, &body).await; @@ -922,14 +889,9 @@ mod tests { } other => panic!("expected Refused, got {other:?}"), }, - (VendorSource::Auto, ServiceCopy::FallBack) => { - assert_eq!(warnings.len(), 1, "{warnings:?}"); - assert_eq!(warnings[0].code, "vendor_prebuilt_layout_mismatch"); - } (source, ServiceCopy::Used(_)) => { panic!("{source:?}: unpatched service bytes were accepted") } - (source, _) => panic!("{source:?}: unexpected outcome"), } assert!( !warnings diff --git a/crates/socket-patch-core/src/vendor/source.rs b/crates/socket-patch-core/src/vendor/source.rs index 0821367da..20435a2d9 100644 --- a/crates/socket-patch-core/src/vendor/source.rs +++ b/crates/socket-patch-core/src/vendor/source.rs @@ -1,197 +1,15 @@ -//! What a vendor backend stages the pristine package from. -//! -//! Most runs never read the pristine tree — the committed-artifact reuse, -//! the in-sync hot path and the vendoring service all answer from bytes the -//! project already has — so extracting a whole package tree per purl on a -//! lockfile-only checkout would be wasted work. -//! -//! [`PackageSource`] does the fetch, the size caps and the integrity -//! verification eagerly and defers only the writing: the fetched artifact is validated against the extractor's own rules up front -//! (see [`super::registry_fetch::FetchedPackage`]) and materialises on the -//! first branch that actually reads a file. -//! -//! A [`DeferredPackage`] goes one step further and defers the DOWNLOAD -//! itself. The vendor loop hands one out for a purl whose ledger entry -//! already covers the record (same patch uuid, committed artifact present): -//! the backend's in-sync hot path answers from the committed artifact and -//! never reads the pristine tree, so a re-run makes no registry request at -//! all — and works with no network. A backend branch that does read it -//! (a drifted artifact being rebuilt locally) fetches it then, through the -//! same ladder the eager fetch would have used. - -use std::future::Future; +//! Optional installed location used for identity and release-variant probes. use std::path::{Path, PathBuf}; -use std::pin::Pin; - -use super::registry_fetch::FetchedPackage; -/// The pristine source a backend stages from. #[derive(Clone, Copy, Debug)] pub enum PackageSource<'a> { - /// A tree already on disk: the crawler's installed location. Installed(&'a Path), - /// A fetched, verified artifact whose tree is written on first use. - Pending(&'a FetchedPackage), - /// An artifact not fetched yet: downloaded, verified and written on - /// first use. - Deferred(&'a DeferredPackage), } impl<'a> PackageSource<'a> { - /// Where the package root is (or will be). Pure — no I/O and no - /// extraction — so it answers the naming questions a backend asks - /// before it decides anything: the gem leaf's `-`, or - /// whether the parent chain is a gem home's `gems/`. Never read content - /// through it; use [`Self::materialize`]. pub fn path(&self) -> &'a Path { match self { - Self::Installed(dir) => dir, - Self::Pending(fetched) => fetched.dir_path(), - Self::Deferred(deferred) => &deferred.hint, - } - } - - /// The package root with its content on disk. A pending artifact is - /// extracted here, once per run; the error is the extractor's own. - pub async fn materialize(&self) -> Result<&'a Path, String> { - match self { - Self::Installed(dir) => Ok(dir), - Self::Pending(fetched) => fetched.dir().await, - Self::Deferred(deferred) => deferred.fetched().await?.dir().await, - } - } - - /// Let go of whatever a fetched source is still holding to be able to - /// produce its tree — called once the loop has moved past the purl it - /// belongs to, so a run does not carry every artifact it fetched to the - /// end. An installed tree holds nothing. Reading through a released - /// source is a bug the caller has to avoid; what has already been - /// materialised stays readable. - pub fn release(&self) { - match self { - Self::Installed(_) => {} - Self::Pending(fetched) => fetched.release(), - Self::Deferred(deferred) => deferred.release(), - } - } - - /// Stage the source freshly at `dst` — the vendor stage the local build - /// patches and then swaps into the copy dir. - /// - /// An installed tree is copied out of the registry/module cache. A - /// pending artifact is written STRAIGHT here instead of into its tempdir - /// and copied out of it again: the extraction is the same walk, so the - /// stage gets the same files with the same bytes and the same modes, and - /// `skip_file_name` drops the same entries the copy drops. `dst` is removed and recreated either way. - pub async fn stage_into( - &self, - dst: &Path, - skip_file_name: Option<&'static str>, - ) -> Result<(), String> { - match self { - Self::Installed(dir) => crate::patch::copy_tree::fresh_copy(dir, dst, skip_file_name) - .await - .map_err(|e| e.to_string()), - Self::Pending(fetched) => fetched.stage_into(dst, skip_file_name).await, - Self::Deferred(deferred) => { - deferred - .fetched() - .await? - .stage_into(dst, skip_file_name) - .await - } - } - } -} - -/// Why a deferred fetch produced no package. `code` is the caller's own -/// classification (the vendor loop maps it back onto the events an eager -/// fetch would have recorded); `detail` is what a backend that needed the -/// tree reports. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct DeferredMiss { - pub code: &'static str, - pub detail: String, -} - -/// The download a [`DeferredPackage`] runs on first use. -pub type DeferredFetchFn = Box< - dyn FnOnce() -> Pin> + Send>> - + Send, ->; - -/// A pristine source whose download has not happened (see the module -/// docs). The fetch runs at most once, on the first [`PackageSource`] call -/// that needs content; its outcome is kept for every later caller and for -/// the vendor loop, which reports it once the backend has returned. -pub struct DeferredPackage { - /// Where the package root would be named: the same leaf the eager - /// fetch's tempdir uses, under a directory that is never created, so a - /// backend's naming questions read the same either way. - hint: PathBuf, - fetch: std::sync::Mutex>, - outcome: tokio::sync::OnceCell>, -} - -impl std::fmt::Debug for DeferredPackage { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("DeferredPackage") - .field("hint", &self.hint) - .field("outcome", &self.outcome.get()) - .finish() - } -} - -impl DeferredPackage { - /// `leaf` is the name the eager fetch gives the package root - /// ([`super::registry_fetch::staged_leaf_for_purl`]). - pub fn new(leaf: &str, fetch: DeferredFetchFn) -> Self { - Self { - hint: std::env::temp_dir() - .join("socket-patch-deferred-source") - .join(leaf), - fetch: std::sync::Mutex::new(Some(fetch)), - outcome: tokio::sync::OnceCell::new(), - } - } - - /// The fetch's outcome, or `None` when nothing has needed the source. - pub fn outcome(&self) -> Option<&Result> { - self.outcome.get() - } - - async fn fetched(&self) -> Result<&FetchedPackage, String> { - let outcome = self - .outcome - .get_or_init(|| async { - let fetch = self - .fetch - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .take(); - match fetch { - Some(fetch) => fetch().await, - None => Err(DeferredMiss { - code: "released", - detail: "the deferred source was released before it was read".into(), - }), - } - }) - .await; - outcome.as_ref().map_err(|miss| miss.detail.clone()) - } - - /// Drop the pending download (nothing will need it now) or let a - /// fetched archive go, as [`FetchedPackage::release`] does. - fn release(&self) { - drop( - self.fetch - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner) - .take(), - ); - if let Some(Ok(fetched)) = self.outcome.get() { - fetched.release(); + Self::Installed(path) => path, } } } @@ -207,151 +25,3 @@ impl<'a> From<&'a PathBuf> for PackageSource<'a> { Self::Installed(dir.as_path()) } } - -impl<'a> From<&'a FetchedPackage> for PackageSource<'a> { - fn from(fetched: &'a FetchedPackage) -> Self { - Self::Pending(fetched) - } -} - -impl<'a> From<&'a DeferredPackage> for PackageSource<'a> { - fn from(deferred: &'a DeferredPackage) -> Self { - Self::Deferred(deferred) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use sha2::{Digest, Sha256}; - use std::sync::atomic::{AtomicUsize, Ordering}; - use std::sync::Arc; - - /// A counting fetch that stages `tgz` (or misses when `None`). - fn counting(calls: &Arc, tgz: Option<(PathBuf, String)>) -> DeferredFetchFn { - let calls = Arc::clone(calls); - Box::new(move || { - Box::pin(async move { - calls.fetch_add(1, Ordering::SeqCst); - match tgz { - Some((path, sha)) => { - super::super::registry_fetch::stage_local_artifact(&path, &sha) - .await - .map_err(|e| DeferredMiss { - code: "failed", - detail: format!("{e:?}"), - }) - } - None => Err(DeferredMiss { - code: "failed", - detail: "registry unreachable".into(), - }), - } - }) - }) - } - - fn left_pad_tgz(dir: &Path) -> (PathBuf, String) { - let mut builder = tar::Builder::new(flate2::write::GzEncoder::new( - Vec::new(), - flate2::Compression::default(), - )); - for (path, bytes) in [ - ( - "package/package.json", - &br#"{"name":"left-pad","version":"1.3.0"}"#[..], - ), - ("package/index.js", b"before\n"), - ] { - let mut header = tar::Header::new_gnu(); - header.set_size(bytes.len() as u64); - header.set_mode(0o644); - header.set_cksum(); - builder.append_data(&mut header, path, bytes).unwrap(); - } - let bytes = builder.into_inner().unwrap().finish().unwrap(); - let path = dir.join("left-pad-1.3.0.tgz"); - std::fs::write(&path, &bytes).unwrap(); - (path, hex::encode(Sha256::digest(&bytes))) - } - - #[tokio::test] - async fn naming_never_fetches() { - let calls = Arc::new(AtomicUsize::new(0)); - let deferred = DeferredPackage::new("rails-7.0.0", counting(&calls, None)); - let source = PackageSource::from(&deferred); - assert_eq!(source.path().file_name().unwrap(), "rails-7.0.0"); - assert_ne!( - source.path().parent().unwrap().file_name().unwrap(), - "gems", - "a deferred gem must not look like it sits in a gem home" - ); - source.release(); - assert_eq!(calls.load(Ordering::SeqCst), 0); - assert!(deferred.outcome().is_none()); - } - - #[tokio::test] - async fn first_read_fetches_once_and_every_read_shares_it() { - let tmp = tempfile::tempdir().unwrap(); - let calls = Arc::new(AtomicUsize::new(0)); - let deferred = - DeferredPackage::new("package", counting(&calls, Some(left_pad_tgz(tmp.path())))); - let source = PackageSource::from(&deferred); - let dir = source.materialize().await.unwrap(); - assert!(dir.join("package.json").is_file()); - let stage = tmp.path().join("stage"); - source.stage_into(&stage, None).await.unwrap(); - assert_eq!(std::fs::read(stage.join("index.js")).unwrap(), b"before\n"); - assert_eq!(calls.load(Ordering::SeqCst), 1, "one fetch per run"); - assert!(matches!(deferred.outcome(), Some(Ok(_)))); - } - - #[tokio::test] - async fn a_miss_is_kept_and_reported_to_every_reader() { - let calls = Arc::new(AtomicUsize::new(0)); - let deferred = DeferredPackage::new("crate", counting(&calls, None)); - let source = PackageSource::from(&deferred); - let tmp = tempfile::tempdir().unwrap(); - assert_eq!( - source.materialize().await.unwrap_err(), - "registry unreachable" - ); - assert_eq!( - source - .stage_into(&tmp.path().join("stage"), None) - .await - .unwrap_err(), - "registry unreachable" - ); - assert_eq!(calls.load(Ordering::SeqCst), 1); - match deferred.outcome() { - Some(Err(miss)) => assert_eq!(miss.code, "failed"), - other => panic!("expected the kept miss, got {other:?}"), - } - } - - #[tokio::test] - async fn a_released_source_never_fetches() { - let calls = Arc::new(AtomicUsize::new(0)); - let deferred = DeferredPackage::new("module", counting(&calls, None)); - PackageSource::from(&deferred).release(); - let err = PackageSource::from(&deferred) - .materialize() - .await - .unwrap_err(); - assert!(err.contains("released"), "{err}"); - assert_eq!(calls.load(Ordering::SeqCst), 0); - } - - #[test] - fn staged_leaf_names_match_the_eager_fetch() { - use super::super::registry_fetch::staged_leaf_for_purl as leaf; - assert_eq!(leaf("pkg:gem/rails@7.0.0?platform=ruby"), "rails-7.0.0"); - assert_eq!(leaf("pkg:pypi/six@1.16.0"), "site-packages"); - assert_eq!(leaf("pkg:cargo/cfg-if@1.0.4"), "crate"); - assert_eq!(leaf("pkg:golang/github.com/foo/bar@v1.0.0"), "module"); - assert_eq!(leaf("pkg:npm/left-pad@1.3.0"), "package"); - assert_eq!(leaf("pkg:composer/monolog/monolog@3.0.0"), "package"); - } -} diff --git a/crates/socket-patch-core/src/vendor/state.rs b/crates/socket-patch-core/src/vendor/state.rs index b35008a45..63f676467 100644 --- a/crates/socket-patch-core/src/vendor/state.rs +++ b/crates/socket-patch-core/src/vendor/state.rs @@ -51,6 +51,8 @@ const VENDOR_STATE_VERSION: u32 = 1; #[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct VendorArtifact { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub yarn_berry10c0: Option, /// Project-relative, forward-slashed path of the artifact /// (`.socket/vendor///`). pub path: String, @@ -832,6 +834,7 @@ mod tests { base_purl: "pkg:npm/lodash@4.17.21".into(), uuid: UUID.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/npm/{UUID}/lodash-4.17.21.tgz"), sha256: "ab".repeat(32), size: Some(3668), @@ -887,6 +890,7 @@ mod tests { base_purl: "pkg:cargo/cfg-if@1.0.4".into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: format!(".socket/vendor/cargo/{uuid}/cfg-if-1.0.4"), sha256: String::new(), size: None, diff --git a/crates/socket-patch-core/src/vendor/test_support.rs b/crates/socket-patch-core/src/vendor/test_support.rs index 95f11389d..e5b72a497 100644 --- a/crates/socket-patch-core/src/vendor/test_support.rs +++ b/crates/socket-patch-core/src/vendor/test_support.rs @@ -79,14 +79,31 @@ pub(crate) async fn mount_granted( use wiremock::{Mock, ResponseTemplate}; let serve_path = format!("/serve/{uuid}/{leaf}"); let url = format!("{}{serve_path}", server.uri()); + let mut artifacts = vec![ + serde_json::json!({ "kind": "tarball", "url": url, "integrity": { "sha512": sri(bytes) } }), + ]; + if let Ok(members) = crate::patch::package::read_archive_bytes_to_map_strict(bytes) { + if let Some(name) = members + .get("package.json") + .and_then(|b| serde_json::from_slice::(b).ok()) + .and_then(|p| { + p.get("name") + .and_then(serde_json::Value::as_str) + .map(str::to_string) + }) + { + if let Ok(checksum) = super::berry_zip::berry_cache_checksum_10c0(bytes, &name) { + artifacts.push(serde_json::json!({"kind":"yarn-berry-zip","integrity":{"yarnBerry10c0":checksum}})); + } + } + } Mock::given(method("POST")) .and(path(PACKAGE_PATH)) .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ "results": { uuid: { "status": "granted", "url": url, - "artifacts": [{ "kind": "tarball", "url": url, - "integrity": { "sha512": sri(bytes) } }] + "artifacts": artifacts }} }))) .mount(server) @@ -171,12 +188,6 @@ pub(crate) fn empty_patch( /// own their purl and record). pub(crate) type Borrowed<'e, T> = std::pin::Pin + 'e>>; -/// Pin a backend's download-plan gate to the grants its vendor call really -/// requests. The gate's verdicts for every case come first — the vendor -/// loop's plan is built before it vendors anything — then each case is -/// vendored in order against `server` (answering no result, so `auto` falls -/// back to the local build). The uuids the gate named must be exactly the -/// uuids the backend asked grants for, in order; returns them. pub(crate) async fn plan_matches_grants<'e>( server: &wiremock::MockServer, cases: &[(&str, crate::manifest::schema::PatchRecord)], @@ -282,6 +293,7 @@ pub(crate) fn artifact_leaf(fx: &impl FlipFixture) -> String { /// `$suite`: the generated module's name; `$fixture`: `async fn() -> $fx`; `$run`: /// `async fn(&$fx, Option<&VendorServiceConfig>) -> VendorOutcome`; /// `$fx: FlipFixture`. All three resolve in the invoking module. +#[cfg(test)] macro_rules! npm_flip_suite { ($suite:ident, $fx:ident, $fixture:ident, $run:ident) => { mod $suite { @@ -291,7 +303,6 @@ macro_rules! npm_flip_suite { use super::$fx as Fx; - /// The deterministic local build's bytes (from a throwaway copy). async fn local_bytes() -> Vec { let probe = $fixture().await; let (r, e, _) = ts::expect_done($run(&probe, None).await); @@ -360,37 +371,26 @@ macro_rules! npm_flip_suite { let alt = ts::regzip(&local_bytes().await); let server = wiremock::MockServer::start().await; let fx = $fixture().await; - let before = first_run(&fx, &server, VendorSource::Auto, Some(&alt)).await; + let before = first_run(&fx, &server, VendorSource::Service, Some(&alt)).await; assert_eq!( before[0].1.as_deref(), Some(alt.as_slice()), "run 1 used the service bytes" ); - assert_noop_rerun(&fx, &server, VendorSource::Auto, false, None, &before).await; - } - - #[tokio::test] - async fn outage_then_service_rerun_is_in_sync() { - let local = local_bytes().await; - let alt = ts::regzip(&local); - let server = wiremock::MockServer::start().await; - let fx = $fixture().await; - let before = first_run(&fx, &server, VendorSource::Auto, None).await; - assert_eq!( - before[0].1.as_deref(), - Some(local.as_slice()), - "run 1 built locally" - ); - assert_noop_rerun(&fx, &server, VendorSource::Auto, false, Some(&alt), &before) - .await; + assert_noop_rerun(&fx, &server, VendorSource::Service, false, None, &before).await; } #[tokio::test] - async fn outage_then_outage_rerun_is_in_sync_and_quiet() { + async fn outage_preserves_the_project_then_service_succeeds() { + let bytes = local_bytes().await; let server = wiremock::MockServer::start().await; let fx = $fixture().await; - let before = first_run(&fx, &server, VendorSource::Auto, None).await; - assert_noop_rerun(&fx, &server, VendorSource::Auto, false, None, &before).await; + let before = ts::tree_snapshot(fx.flip_root()); + mount(&fx, &server, None).await; + let cfg = ts::service_cfg(&server.uri(), VendorSource::Service, false); + ts::expect_failure($run(&fx, Some(&cfg)).await); + assert_eq!(ts::tree_snapshot(fx.flip_root()), before); + first_run(&fx, &server, VendorSource::Service, Some(&bytes)).await; } #[tokio::test] @@ -410,26 +410,33 @@ macro_rules! npm_flip_suite { let alt = ts::regzip(&local_bytes().await); let server = wiremock::MockServer::start().await; let fx = $fixture().await; - let before = first_run(&fx, &server, VendorSource::Auto, Some(&alt)).await; + let before = first_run(&fx, &server, VendorSource::Service, Some(&alt)).await; let art = fx.flip_root().join(fx.flip_artifact_rel()); let mtime = std::fs::metadata(&art).unwrap().modified().unwrap(); - assert_noop_rerun(&fx, &server, VendorSource::Auto, false, Some(&alt), &before) - .await; + assert_noop_rerun( + &fx, + &server, + VendorSource::Service, + false, + Some(&alt), + &before, + ) + .await; assert_eq!(std::fs::metadata(&art).unwrap().modified().unwrap(), mtime); } /// F5: `--vendor-source build` reuses (it never contacts the /// service, and reuse contacts nothing). #[tokio::test] - async fn build_rerun_after_service_keeps_the_service_bytes() { + async fn repeated_service_keeps_the_committed_bytes() { let alt = ts::regzip(&local_bytes().await); let server = wiremock::MockServer::start().await; let fx = $fixture().await; - let before = first_run(&fx, &server, VendorSource::Auto, Some(&alt)).await; + let before = first_run(&fx, &server, VendorSource::Service, Some(&alt)).await; assert_noop_rerun( &fx, &server, - VendorSource::Build, + VendorSource::Service, false, Some(&alt), &before, @@ -439,6 +446,7 @@ macro_rules! npm_flip_suite { } }; } +#[cfg(test)] pub(crate) use npm_flip_suite; /// Every regular file under `root` (relative path → bytes), for the @@ -461,3 +469,526 @@ pub(crate) fn tree_snapshot(root: &Path) -> std::collections::BTreeMap( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::pnpm_lock::vendor_pnpm( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_yarn_classic<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::yarn_classic_lock::vendor_yarn_classic( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_npm<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::npm_lock::vendor_npm( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_composer<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::composer_lock::vendor_composer( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_cargo_crate<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::cargo::vendor_cargo_crate( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_vlt<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::vlt_lock::vendor_vlt( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_maven<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::maven_repo::vendor_maven( + purl, + source.path(), + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_pnpm_legacy<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::pnpm_lock_legacy::vendor_pnpm_legacy( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_nuget<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::nuget_feed::vendor_nuget( + purl, + source.path(), + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_yarn_berry<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::yarn_berry_lock::vendor_yarn_berry( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_pypi<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::pypi::vendor_pypi( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_gem<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::gem::vendor_gem( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_go_module<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::golang::vendor_go_module( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_npm_any<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::npm_flavor::vendor_npm_any( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub(crate) async fn vendor_bun<'a>( + purl: &str, + source: impl Into>, + project_root: &Path, + record: &crate::manifest::schema::PatchRecord, + sources: &crate::patch::apply::PatchSources<'_>, + vendored_at: &str, + dry_run: bool, + force: bool, + service: Option<&VendorServiceConfig>, +) -> VendorOutcome { + let source = source.into(); + let fixture = if service.is_none() { + Some(service_fixture::Fixture::new(purl, source, record, sources).await) + } else { + None + }; + super::bun_lock::vendor_bun( + purl, + source, + project_root, + record, + sources, + vendored_at, + dry_run, + force, + service.or_else(|| fixture.as_ref().map(|f| &f.cfg)), + ) + .await +} + +pub(crate) fn expect_failure(outcome: VendorOutcome) -> String { + match outcome { + VendorOutcome::Refused { code, detail } => format!("{code}: {detail}"), + VendorOutcome::Done { result, entry, .. } => { + assert!( + !result.success && entry.is_none(), + "expected failure: {result:?}" + ); + result.error.expect("failed result has a reason") + } + } +} + +pub(crate) fn expect_failed( + outcome: VendorOutcome, +) -> (ApplyResult, Option, Vec) { + match outcome { + VendorOutcome::Refused { code, detail } => { + match super::npm_common::done_failure("test", format!("{code}: {detail}")) { + VendorOutcome::Done { + result, + entry, + warnings, + } => (result, entry, warnings), + _ => unreachable!(), + } + } + VendorOutcome::Done { + result, + entry, + warnings, + } => { + assert!( + !result.success && entry.is_none(), + "expected failure: {result:?}" + ); + (result, entry, warnings) + } + } +} diff --git a/crates/socket-patch-core/src/vendor/test_support/service_fixture.rs b/crates/socket-patch-core/src/vendor/test_support/service_fixture.rs new file mode 100644 index 000000000..dec7f9e38 --- /dev/null +++ b/crates/socket-patch-core/src/vendor/test_support/service_fixture.rs @@ -0,0 +1,346 @@ +//! Archive fixtures for tests of download, verification, wiring and rollback. +use std::collections::BTreeMap; +use std::io::Write; +use std::path::Path; + +use base64::Engine as _; +use sha2::{Digest, Sha256}; +use wiremock::matchers::{method, path}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +use crate::manifest::schema::PatchRecord; +use crate::patch::apply::{normalize_file_path, PatchSources}; +use crate::utils::purl::{ + parse_cargo_purl, parse_gem_purl, parse_golang_purl, parse_maven_purl, parse_nuget_purl, + parse_pypi_purl, +}; +use crate::vendor::source::PackageSource; +use crate::vendor::{VendorServiceConfig, VendorSource}; + +pub struct Fixture { + _server: MockServer, + maven_registry: Option>, + pub cfg: VendorServiceConfig, +} + +impl Fixture { + pub async fn new( + purl: &str, + source: PackageSource<'_>, + record: &PatchRecord, + sources: &PatchSources<'_>, + ) -> Self { + let server = MockServer::start().await; + let maven_registry = if let Some((g, a, v)) = parse_maven_purl(purl) { + for ext in ["jar", "pom", "module"] { + if let Ok(bytes) = crate::utils::fs::read_regular_to_bytes( + &source.path().join(format!("{a}-{v}.{ext}")), + ) + .await + { + let url = format!("/{}/{a}/{v}/{a}-{v}.{ext}", g.replace('.', "/")); + Mock::given(method("GET")) + .and(path(format!("{url}.sha1"))) + .respond_with( + ResponseTemplate::new(200) + .set_body_string(hex::encode(sha1::Sha1::digest(&bytes))), + ) + .mount(&server) + .await; + Mock::given(method("GET")) + .and(path(format!("{url}.sha256"))) + .respond_with( + ResponseTemplate::new(200) + .set_body_string(hex::encode(Sha256::digest(&bytes))), + ) + .mount(&server) + .await; + Mock::given(method("GET")) + .and(path(url)) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(&server) + .await; + } + } + let old = std::env::var_os("SOCKET_MAVEN_REGISTRY"); + std::env::set_var("SOCKET_MAVEN_REGISTRY", server.uri()); + Some(old) + } else { + None + }; + match Ok::<_, String>(source.path()) { + Ok(dir) => match archive(purl, dir, record, sources).await { + Ok((leaf, bytes, secondary)) => { + let uri = server.uri(); + let url = format!("{uri}/archive/{leaf}"); + let mut artifacts = vec![ + serde_json::json!({"kind":"tarball", "url":url, "integrity":{"sha512":super::sri(&bytes)}}), + ]; + if purl.starts_with("pkg:npm/") { + let name = crate::vendor::npm_common::parse_npm_purl(purl) + .map(|p| p.0) + .unwrap_or_default(); + if let Ok(checksum) = + crate::vendor::berry_zip::berry_cache_checksum_10c0(&bytes, &name) + { + artifacts.push(serde_json::json!({"kind":"yarn-berry-zip", "integrity":{"yarnBerry10c0":checksum}})); + } + } + for (kind, name, bytes) in secondary { + artifacts.push(serde_json::json!({"kind":kind,"url":format!("{uri}/archive/{name}"),"integrity":{"sha512":super::sri(&bytes)}})); + Mock::given(method("GET")) + .and(path(format!("/archive/{name}"))) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(&server) + .await; + } + Mock::given(method("POST")).and(path(super::PACKAGE_PATH)).respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"results":{ &record.uuid: {"status":"granted", "purl":purl, "url":url, "artifacts":artifacts}}}))).mount(&server).await; + Mock::given(method("GET")) + .and(path(format!("/archive/{leaf}"))) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(&server) + .await; + } + Err(_) => super::mount_no_results(&server).await, + }, + Err(_) => super::mount_no_results(&server).await, + } + Self { + cfg: super::service_cfg(&server.uri(), VendorSource::Service, false), + _server: server, + maven_registry, + } + } +} + +pub type Secondary = Vec<(&'static str, String, Vec)>; + +pub async fn archive( + purl: &str, + dir: &Path, + record: &PatchRecord, + sources: &PatchSources<'_>, +) -> Result<(String, Vec, Secondary), String> { + if purl.contains("..") || purl.contains('\\') { + return Err("unsafe fixture coordinate".into()); + } + let mut members = BTreeMap::new(); + if let Some((_, artifact, version)) = parse_maven_purl(purl) { + let bytes = + crate::utils::fs::read_regular_to_bytes(&dir.join(format!("{artifact}-{version}.jar"))) + .await + .map_err(|e| e.to_string())?; + members.extend(crate::vendor::verify::read_zip_bytes_to_map(&bytes)?); + } else if let Some((name, version)) = parse_nuget_purl(purl) { + let bytes = crate::utils::fs::read_regular_to_bytes(&dir.join(format!( + "{}.{}.nupkg", + name.to_lowercase(), + version + ))) + .await + .map_err(|e| e.to_string())?; + members.extend(crate::vendor::verify::read_zip_bytes_to_map(&bytes)?); + } else { + for entry in walkdir::WalkDir::new(dir) + .into_iter() + .filter_entry(|e| e.file_name() != "node_modules" && e.file_name() != ".git") + { + let entry = entry.map_err(|e| e.to_string())?; + if entry.file_type().is_file() { + let path = entry + .path() + .strip_prefix(dir) + .unwrap() + .to_string_lossy() + .replace('\\', "/"); + members.insert( + path, + tokio::fs::read(entry.path()) + .await + .map_err(|e| e.to_string())?, + ); + } + } + } + for (path, info) in &record.files { + let bytes = match sources + .mem_blobs + .and_then(|blobs| blobs.get(&info.after_hash)) + .cloned() + { + Some(bytes) => bytes, + None => tokio::fs::read(sources.blobs_path.join(&info.after_hash)) + .await + .map_err(|e| e.to_string())?, + }; + members.insert(normalize_file_path(path).to_string(), bytes); + } + if let Some((name, version)) = parse_pypi_purl(purl) { + let name = crate::crawlers::python_crawler::canonicalize_pypi_name(&name).replace('-', "_"); + let version = crate::vendor::pypi_wheel::escape_wheel_version(&version); + let dist_info = members + .keys() + .find(|k| k.ends_with(".dist-info/WHEEL") && k.starts_with(&format!("{name}-"))) + .and_then(|k| k.rsplit_once('/')) + .map(|(d, _)| d.to_string()) + .unwrap_or_else(|| format!("{name}-{version}.dist-info")); + if let Some(installed_record) = members.get(&format!("{dist_info}/RECORD")) { + let mut package_files: std::collections::BTreeSet = + String::from_utf8_lossy(installed_record) + .lines() + .filter_map(|line| line.split(',').next()) + .map(str::to_string) + .collect(); + package_files.extend( + record + .files + .keys() + .map(|path| normalize_file_path(path).to_string()), + ); + members.retain(|path, _| { + package_files.contains(path) || path.starts_with(&format!("{dist_info}/")) + }); + } + let wheel = members + .get(&format!("{dist_info}/WHEEL")) + .ok_or("fixture has no WHEEL")?; + let mut tags: [std::collections::BTreeSet<&str>; 3] = Default::default(); + for tag in std::str::from_utf8(wheel) + .map_err(|e| e.to_string())? + .lines() + .filter_map(|l| l.strip_prefix("Tag: ")) + { + for (i, value) in tag.trim().split('-').take(3).enumerate() { + tags[i].insert(value); + } + } + let tag = tags + .map(|values| values.into_iter().collect::>().join(".")) + .join("-"); + let record = format!("{dist_info}/RECORD"); + members.remove(&record); + let mut rows = String::new(); + for (path, bytes) in &members { + rows.push_str(&format!( + "{path},sha256={},{}\n", + base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(Sha256::digest(bytes)), + bytes.len() + )); + } + rows.push_str(&format!("{record},,\n")); + members.insert(record, rows.into_bytes()); + return Ok(( + format!("{name}-{version}-{tag}.whl"), + zip(&members, ""), + Vec::new(), + )); + } + if let Some((name, version)) = crate::vendor::npm_common::parse_npm_purl(purl) { + return Ok(( + format!("{}-{version}.tgz", name.replace('/', "-")), + tgz(&members, "package/"), + Vec::new(), + )); + } + if let Some((name, version)) = parse_cargo_purl(purl) { + return Ok(( + format!("{name}-{version}.crate"), + tgz(&members, &format!("{name}-{version}/")), + Vec::new(), + )); + } + if let Some((module, version)) = parse_golang_purl(purl) { + return Ok(( + format!("{version}.zip"), + zip(&members, &format!("{module}@{version}/")), + Vec::new(), + )); + } + if let Some((name, version)) = parse_gem_purl(purl) { + let stub = dir.parent().and_then(Path::parent).map(|home| { + home.join("specifications") + .join(format!("{name}-{version}.gemspec")) + }); + let stub = match stub { Some(path) => tokio::fs::read(path).await.ok(), None => None }.unwrap_or_else(||format!("Gem::Specification.new do |s|\n s.name = {name:?}\n s.version = {version:?}\n s.summary = 'fixture'\n s.authors = ['fixture']\nend\n").into_bytes()); + let outer = BTreeMap::from([("data.tar.gz".into(), tgz(&members, ""))]); + return Ok(( + format!("{name}-{version}.gem"), + tar(&outer, ""), + vec![("gem-stub-gemspec", format!("{name}.gemspec"), stub)], + )); + } + if let Some((_, artifact, version)) = parse_maven_purl(purl) { + members.retain(|name, _| !crate::vendor::jvm::is_signature(name)); + return Ok(( + format!("{artifact}-{version}.jar"), + zip(&members, ""), + Vec::new(), + )); + } + if let Some((name, version)) = parse_nuget_purl(purl) { + members.retain(|name, _| { + !name.ends_with(".nupkg") + && !name.eq_ignore_ascii_case(".signature.p7s") + && name != ".nupkg.metadata" + }); + return Ok(( + format!("{name}.{version}.nupkg"), + zip(&members, ""), + Vec::new(), + )); + } + Ok(("dist.zip".into(), zip(&members, "package/"), Vec::new())) +} + +fn zip(members: &BTreeMap>, prefix: &str) -> Vec { + let mut archive = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + for (name, bytes) in members { + archive + .start_file( + format!("{prefix}{name}"), + zip::write::SimpleFileOptions::default() + .compression_method(zip::CompressionMethod::Stored) + .unix_permissions(0o644), + ) + .unwrap(); + archive.write_all(bytes).unwrap(); + } + archive.finish().unwrap().into_inner() +} + +fn tar(members: &BTreeMap>, prefix: &str) -> Vec { + let mut archive = tar::Builder::new(Vec::new()); + for (name, bytes) in members { + let mut header = tar::Header::new_gnu(); + header.set_size(bytes.len() as u64); + header.set_mode(0o644); + header.set_mtime(0); + header.set_cksum(); + archive + .append_data(&mut header, format!("{prefix}{name}"), bytes.as_slice()) + .unwrap(); + } + archive.into_inner().unwrap() +} + +fn tgz(members: &BTreeMap>, prefix: &str) -> Vec { + let mut gzip = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::default()); + gzip.write_all(&tar(members, prefix)).unwrap(); + gzip.finish().unwrap() +} + +impl Drop for Fixture { + fn drop(&mut self) { + if let Some(old) = &self.maven_registry { + match old { + Some(value) => std::env::set_var("SOCKET_MAVEN_REGISTRY", value), + None => std::env::remove_var("SOCKET_MAVEN_REGISTRY"), + } + } + } +} + +/// Test oracle only; production vendoring consumes the server checksum. +pub fn berry_checksum(bytes: &[u8], name: &str) -> Option { + crate::vendor::berry_zip::berry_cache_checksum_10c0(bytes, name).ok() +} diff --git a/crates/socket-patch-core/src/vendor/verify.rs b/crates/socket-patch-core/src/vendor/verify.rs index cf5d7e488..2ee37d35b 100644 --- a/crates/socket-patch-core/src/vendor/verify.rs +++ b/crates/socket-patch-core/src/vendor/verify.rs @@ -103,6 +103,28 @@ pub async fn verify_vendored_patch_record( if is_vlt_dir_entry(entry) { return verify_vlt_dir(project_root, &artifact, entry, record).await; } + if entry.ecosystem == "pypi" { + let name = entry.artifact.path.clone(); + let members = tokio::task::spawn_blocking(move || { + let (file, meta) = crate::utils::fs::open_regular_file_sync(&artifact) + .map_err(|_| "vendor_artifact_unreadable")?; + if meta.len() > MAX_HEALTH_HASH_BYTES { + return Err("vendor_artifact_unreadable".into()); + } + let mut bytes = Vec::new(); + file.take(MAX_HEALTH_HASH_BYTES + 1) + .read_to_end(&mut bytes) + .map_err(|_| "vendor_artifact_unreadable")?; + if bytes.len() as u64 > MAX_HEALTH_HASH_BYTES { + return Err("vendor_artifact_unreadable".into()); + } + super::pypi_distribution::read_members(&bytes, &name) + .map_err(|_| "vendor_artifact_unreadable".to_string()) + }) + .await + .map_err(|e| e.to_string())??; + return super::pypi_distribution::verify_members(&members, &entry.artifact.path, record); + } let path_str = artifact.to_string_lossy(); let is_tarball = path_str.ends_with(".tgz") || path_str.ends_with(".tar.gz"); let is_zip = @@ -501,6 +523,7 @@ pub fn artifact_is_file_shaped(path: &str) -> bool { norm.ends_with(".tgz") || norm.ends_with(".tar.gz") || norm.ends_with(".whl") + || norm.ends_with(".zip") || norm.ends_with(".nupkg") || norm.ends_with(".jar") } @@ -594,7 +617,7 @@ async fn inventory_walk( /// before tagged versions keeps verifying once a re-run / repair tags it, /// while every other byte (including any other tag) stays pinned. So the /// tag step never re-baselines the inventory over bytes nobody verified. -async fn verify_dir_inventory( +pub(super) async fn verify_dir_inventory( dir: &Path, inventory: &BTreeMap, cargo_uuid: Option<&str>, @@ -830,6 +853,7 @@ mod tests { base_purl: "pkg:npm/x@1.0.0".into(), uuid: uuid.into(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.into(), sha256: String::new(), size: None, @@ -1992,8 +2016,10 @@ mod tests { // Precondition: the zip reader tolerates the prefix, so member // verification alone would bless the artifact… assert!( - verify_vendored_patch_record(root, &ent, &rec).await.is_ok(), - "zip reader must resolve the archive offset past the sparse prefix" + verify_vendored_patch_record(root, &ent, &rec) + .await + .is_err(), + "member verification also refuses oversized archives" ); // …and only the whole-file arm catches it: file_sha256_hex bails // on the size cap, so the recorded sha is unverifiable. diff --git a/crates/socket-patch-core/src/vendor/vlt_lock.rs b/crates/socket-patch-core/src/vendor/vlt_lock.rs index ca95185cc..033fec48f 100644 --- a/crates/socket-patch-core/src/vendor/vlt_lock.rs +++ b/crates/socket-patch-core/src/vendor/vlt_lock.rs @@ -731,9 +731,7 @@ pub async fn vlt_vendor_preflight( .join(&name) .join(PACKAGE_JSON); if let Ok(text) = read_regular_to_string(&store).await { - if let Ok(pkg) = - serde_json::from_str::(crate::utils::serde::strip_bom(&text)) - { + if let Ok(pkg) = serde_json::from_str::(crate::utils::serde::strip_bom(&text)) { if super::npm_common::declares_bundled_deps(&pkg) { return Err(( "vendor_bundled_deps_unsupported", @@ -1329,6 +1327,7 @@ pub(crate) async fn vendor_vlt<'a>( base_purl: coords.base_purl.clone(), uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: staged.rel_dir.clone(), sha256: String::new(), size: None, @@ -2198,7 +2197,7 @@ mod tests { async fn run(fx: &Fx, uuid: &str, dry_run: bool) -> VendorOutcome { let sources = PatchSources::blobs_only(&fx.blobs); - vendor_vlt( + crate::vendor::test_support::vendor_vlt( PURL, &fx.installed, &fx.root, @@ -2285,7 +2284,12 @@ mod tests { async fn wires_the_node_edges_and_package_json_in_vlt_order() { let fx = fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; let (entry, warnings) = entry_of(run(&fx, UUID, false).await); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); let rel = format!(".socket/vendor/npm/{UUID}/left-pad-1.3.0/node_modules/left-pad"); let file_id = format!("file~.socket+vendor+npm+{UUID}+left-pad-1.3.0+node__modules+left-pad"); @@ -3129,7 +3133,7 @@ mod tests { }, ); let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_vlt( + let outcome = crate::vendor::test_support::vendor_vlt( PURL, &fx.installed, &fx.root, @@ -3199,7 +3203,7 @@ mod tests { }, ); let sources = PatchSources::blobs_only(&fx.blobs); - let outcome = vendor_vlt( + let outcome = crate::vendor::test_support::vendor_vlt( PURL, &fx.installed, &fx.root, @@ -3382,7 +3386,7 @@ mod tests { async fn run_with(fx: &Fx, cfg: &crate::vendor::VendorServiceConfig) -> VendorOutcome { let sources = PatchSources::blobs_only(&fx.blobs); - vendor_vlt( + crate::vendor::test_support::vendor_vlt( PURL, &fx.installed, &fx.root, @@ -3412,7 +3416,7 @@ mod tests { ]); mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &tgz).await; let fx = fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - let cfg = service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); let (entry, warnings) = entry_of(run_with(&fx, &cfg).await); assert!( warnings @@ -3452,16 +3456,6 @@ mod tests { let server = wiremock::MockServer::start().await; mount_503(&server).await; - let fx = fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - let (_, warnings) = - entry_of(run_with(&fx, &service_cfg(&server.uri(), VendorSource::Auto, false)).await); - assert!( - warnings - .iter() - .any(|w| w.code == "vendor_prebuilt_unavailable"), - "{warnings:?}" - ); - let fx = self::fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; match run_with( &fx, @@ -3484,7 +3478,12 @@ mod tests { ]); mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &bad).await; let fx = self::fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - match run_with(&fx, &service_cfg(&server.uri(), VendorSource::Auto, false)).await { + match run_with( + &fx, + &service_cfg(&server.uri(), VendorSource::Service, false), + ) + .await + { VendorOutcome::Done { result, entry, .. } => { assert!(!result.success && entry.is_none()); assert!(result.error.unwrap().contains("unsafe")); @@ -3520,7 +3519,11 @@ mod tests { } fn codes(warnings: &[VendorWarning]) -> Vec<&str> { - warnings.iter().map(|w| w.code).collect() + warnings + .iter() + .filter(|w| w.code != "vendor_prebuilt_downloaded") + .map(|w| w.code) + .collect() } #[tokio::test] @@ -3858,8 +3861,19 @@ mod tests { codes(&warnings), [REINSTALL_REQUIRED, "vendor_artifact_rebuilt"] ); - assert!(warnings[0].detail.contains("run `vlt ci`"), "{warnings:?}"); - assert!(!warnings[1].detail.contains("vlt install"), "{warnings:?}"); + assert!( + warnings + .iter() + .any(|w| w.code == REINSTALL_REQUIRED && w.detail.contains("run `vlt ci`")), + "{warnings:?}" + ); + assert!( + warnings + .iter() + .filter(|w| w.code == "vendor_artifact_rebuilt") + .all(|w| !w.detail.contains("vlt install")), + "{warnings:?}" + ); assert!(!links.exists()); } @@ -4021,10 +4035,12 @@ mod tests { ]); mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &tgz).await; let fx = fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - let cfg = service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); let (entry, warnings) = entry_of(run_with(&fx, &cfg).await); assert!( - codes(&warnings).contains(&"vendor_prebuilt_downloaded"), + warnings + .iter() + .any(|w| w.code == "vendor_prebuilt_downloaded"), "{warnings:?}" ); assert!(!fx @@ -4053,7 +4069,7 @@ mod tests { ]); mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &tgz).await; let fx = self::fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - let cfg = service_cfg(&server.uri(), VendorSource::Auto, false); + let cfg = service_cfg(&server.uri(), VendorSource::Service, false); let (code, _) = refusal(run_with(&fx, &cfg).await); assert_eq!(code, "vendor_bundled_deps_unsupported"); assert!(!fx.root.join(".socket/vendor").exists()); @@ -4079,21 +4095,6 @@ mod tests { ]); mount_granted(&server, UUID, "left-pad-1.3.0.tgz", &tgz).await; - let fx = fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; - let cfg = service_cfg(&server.uri(), VendorSource::Auto, false); - let (entry, warnings) = entry_of(run_with(&fx, &cfg).await); - assert!( - codes(&warnings).contains(&"vendor_prebuilt_layout_mismatch"), - "{warnings:?}" - ); - assert_eq!( - tokio::fs::read(fx.root.join(&entry.artifact.path).join("index.js")) - .await - .unwrap(), - PATCHED, - "the local build replaced the service tree" - ); - let fx = self::fx(&basic_lock(), &[(PACKAGE_JSON, ROOT_PKG)]).await; let cfg = service_cfg(&server.uri(), VendorSource::Service, false); let (result, entry, _) = done_parts(run_with(&fx, &cfg).await); diff --git a/crates/socket-patch-core/src/vendor/yarn_berry_lock.rs b/crates/socket-patch-core/src/vendor/yarn_berry_lock.rs index 3af1c8269..5e9946e8e 100644 --- a/crates/socket-patch-core/src/vendor/yarn_berry_lock.rs +++ b/crates/socket-patch-core/src/vendor/yarn_berry_lock.rs @@ -53,6 +53,7 @@ use crate::utils::line_endings::LineEndings; use crate::utils::socket_dir::remove_tree_and_prune; use crate::utils::uri::encode_uri_component; +#[cfg(test)] use super::berry_zip::berry_cache_checksum_10c0; use super::common::{already_patched_result, parse_json_manifest, refused, JsonLayout}; use super::npm_common::{ @@ -235,7 +236,7 @@ pub async fn vendor_yarn_berry<'a>( }; let uuid_dir_preexisted = staged.uuid_dir_preexisted; debug_assert_eq!(staged.rel_tgz, rel_tgz); - let packed = staged.packed; + let mut packed = staged.packed; let dest = project_root.join(&rel_tgz); // ── 8. Berry identity facts of the packed tarball ───────────────────── @@ -273,18 +274,22 @@ pub async fn vendor_yarn_berry<'a>( // `hash=` — the first 6 hex chars of sha512(tgz): the lock-committed // tamper guard on the tarball itself (flips on any byte edit). let hash6 = &tgz_sha512[..6]; - let checksum = match berry_cache_checksum_10c0(&tgz_bytes, name) { - Ok(c) => checksum_in_lock_spelling(&lock_text, &c), - Err(e) => { - return done_failure_unstage( - purl, - format!("cannot compute the berry cache checksum for {name}: {e}"), - project_root, - &uuid_dir_rel, - uuid_dir_preexisted, - ) - .await + if packed.yarn_berry10c0.is_none() { + if let Some(cfg) = service.filter(|cfg| cfg.service_enabled()) { + if let super::service_fetch::ServiceArtifact::Ready(archive) = + super::service_fetch::fetch_verified_archive(cfg, &record.uuid).await + { + if hex::encode(Sha256::digest(&archive.bytes)) == packed.sha256_hex { + packed.yarn_berry10c0 = archive.yarn_berry10c0; + } + } } + } + let checksum = match packed.yarn_berry10c0.as_deref().filter(|c| valid_berry_checksum(c)) { + Some(c) => checksum_in_lock_spelling(&lock_text, c), + None => return done_failure_unstage(purl, + format!("the patch service supplied no Yarn Berry checksum for {name}; retry after the server artifact is ready"), + project_root, &uuid_dir_rel, uuid_dir_preexisted).await, }; // ── 9. The replacement lock entry (verbatim B3 shape) ───────────────── @@ -418,6 +423,7 @@ pub async fn vendor_yarn_berry<'a>( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: packed.yarn_berry10c0.clone(), path: rel_tgz, sha256: packed.sha256_hex, size: Some(packed.size), @@ -1515,7 +1521,7 @@ __metadata: async fn vendor(&self, dry_run: bool) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_yarn_berry( + crate::vendor::test_support::vendor_yarn_berry( "pkg:npm/left-pad@1.3.0", &self.installed(), self.root(), @@ -1571,7 +1577,7 @@ __metadata: cfg: Option<&crate::vendor::VendorServiceConfig>, ) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor_yarn_berry( + crate::vendor::test_support::vendor_yarn_berry( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -1689,7 +1695,12 @@ __metadata: let fx = fixture().await; let (result, entry, warnings) = expect_done(fx.vendor(false).await); assert!(result.success, "{:?}", result.error); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); let entry = entry.expect("success carries a ledger entry"); // package.json: byte-for-byte the B3 after fixture. @@ -1981,7 +1992,7 @@ __metadata: .error .as_deref() .unwrap_or("") - .contains("berry cache checksum"), + .contains("no Yarn Berry checksum"), "{:?}", result.error ); @@ -2004,7 +2015,12 @@ __metadata: entry.is_none(), "in-sync re-run must not produce a new ledger entry" ); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert!( result .files_verified @@ -2693,7 +2709,7 @@ __metadata: let fx = fixture().await; let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_yarn_berry( + let outcome = crate::vendor::test_support::vendor_yarn_berry( "pkg:gem/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -2759,19 +2775,6 @@ __metadata: let detail = expect_refused(fx.vendor(false).await, "vendor_override_conflict"); assert!(detail.contains("is not an object"), "{detail}"); fx.assert_untouched().await; - - // Bundled dependencies: stage_patch_pack's refusal bubbles verbatim - // before anything inside the project is written. - let fx = fixture().await; - tokio::fs::write( - fx.installed().join("package.json"), - br#"{"name":"left-pad","version":"1.3.0","bundledDependencies":["x"]}"#, - ) - .await - .unwrap(); - let detail = expect_refused(fx.vendor(false).await, "vendor_bundled_deps_unsupported"); - assert!(detail.contains("bundleDependencies"), "{detail}"); - fx.assert_untouched().await; } /// An unrelated resolutions entry is skipped by the conflict scan and @@ -3598,7 +3601,12 @@ __metadata: let fx = fixture_with(&pkg_before, &lock_before).await; let (result, entry, warnings) = expect_done(fx.vendor(false).await); assert!(result.success, "{label}: {:?}", result.error); - assert!(warnings.is_empty(), "{label}: {warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{label}: {warnings:?}" + ); let entry = entry.expect("a ledger entry"); let (hash6, checksum) = fx.packed_berry_facts().await; @@ -3975,3 +3983,9 @@ __metadata: assert_eq!(planned, looped, "a missing lock"); } } + +pub(crate) fn valid_berry_checksum(value: &str) -> bool { + value + .strip_prefix("10c0/") + .is_some_and(|hash| hash.len() == 128 && hash.bytes().all(|b| b.is_ascii_hexdigit())) +} diff --git a/crates/socket-patch-core/src/vendor/yarn_classic_lock.rs b/crates/socket-patch-core/src/vendor/yarn_classic_lock.rs index 38e5e9054..70602238b 100644 --- a/crates/socket-patch-core/src/vendor/yarn_classic_lock.rs +++ b/crates/socket-patch-core/src/vendor/yarn_classic_lock.rs @@ -248,6 +248,7 @@ pub async fn vendor_yarn_classic<'a>( base_purl, uuid: record.uuid.clone(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_tgz, sha256: packed.sha256_hex, size: Some(packed.size), @@ -1234,7 +1235,7 @@ left-pad@^1.3.0, left-pad@~1.3.0: async fn vendor(&self, dry_run: bool) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_yarn_classic( + crate::vendor::test_support::vendor_yarn_classic( "pkg:npm/left-pad@1.3.0", &self.installed(), self.root(), @@ -1274,7 +1275,7 @@ left-pad@^1.3.0, left-pad@~1.3.0: cfg: Option<&crate::vendor::VendorServiceConfig>, ) -> VendorOutcome { let blobs = fx.root().join(".socket/blobs"); - vendor_yarn_classic( + crate::vendor::test_support::vendor_yarn_classic( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -1383,7 +1384,12 @@ left-pad@^1.3.0, left-pad@~1.3.0: let fx = fixture_with_lock(Y2_BEFORE).await; let (result, entry, warnings) = expect_done(fx.vendor(false).await); assert!(result.success, "{:?}", result.error); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); let entry = entry.expect("success carries a ledger entry"); // Byte-for-byte the spike's after-lock, modulo the recomputed hashes. @@ -1441,7 +1447,12 @@ left-pad@^1.3.0, left-pad@~1.3.0: assert!(result.success, "{:?}", result.error); // The folder dep `dep-a@file:./dep-a` is name-mismatched, not a // candidate — no skip warning either. - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); let entry = entry.unwrap(); let (sha1, sri) = fx.packed_hashes().await; @@ -1585,7 +1596,7 @@ left-pad@^1.3.0: .unwrap(); let server = wiremock::MockServer::start().await; ts::mount_503(&server).await; - let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Auto, false); + let cfg = ts::service_cfg(&server.uri(), crate::vendor::VendorSource::Service, false); let (r, e, w) = expect_done(flip_run(&fx, Some(&cfg)).await); assert!(r.success, "{:?}", r.error); assert!(e.is_some(), "the relocked block is re-wired"); @@ -1618,7 +1629,12 @@ left-pad@^1.3.0: entry.is_none(), "in-sync re-run must not produce a new ledger entry" ); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); assert!( result .files_verified @@ -2616,7 +2632,12 @@ left-pad@^1.3.0: let (changed, text, warnings) = run(Y2_AFTER, &rec); assert!(changed, "control record must restore"); assert_eq!(text, Y2_BEFORE); - assert!(warnings.is_empty(), "{warnings:?}"); + assert!( + warnings + .iter() + .all(|w| w.code == "vendor_prebuilt_downloaded"), + "{warnings:?}" + ); } /// Re-vendoring over our own stale edit (a new patch uuid for the same @@ -2635,7 +2656,7 @@ left-pad@^1.3.0: record_b.uuid = UUID_B.to_string(); let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_yarn_classic( + let outcome = crate::vendor::test_support::vendor_yarn_classic( "pkg:npm/left-pad@1.3.0", &fx.installed(), fx.root(), @@ -2699,7 +2720,7 @@ left-pad@^1.3.0: let fx = fixture_with_lock(Y2_BEFORE).await; let blobs = fx.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - let outcome = vendor_yarn_classic( + let outcome = crate::vendor::test_support::vendor_yarn_classic( "pkg:npm/left-pad@../evil", &fx.installed(), fx.root(), @@ -2727,7 +2748,7 @@ left-pad@^1.3.0: .error .as_deref() .unwrap_or("") - .contains("cannot stage a copy of the installed package"), + .contains("patch service request failed"), "{:?}", result.error ); diff --git a/crates/socket-patch-core/src/vendor/yarn_layering_tests.rs b/crates/socket-patch-core/src/vendor/yarn_layering_tests.rs index 4e22591a4..f2f7edf5f 100644 --- a/crates/socket-patch-core/src/vendor/yarn_layering_tests.rs +++ b/crates/socket-patch-core/src/vendor/yarn_layering_tests.rs @@ -42,8 +42,8 @@ use crate::patch::redirect::{rewrite_registry_redirect, DepOverride, Integrity}; use crate::utils::uri::encode_uri_component; use crate::vendor::lock_inventory::{inventory_npm_lock, LockIntegrity}; use crate::vendor::npm_flavor::NpmLockFlavor; -use crate::vendor::yarn_berry_lock::{revert_yarn_berry, vendor_yarn_berry}; -use crate::vendor::yarn_classic_lock::{revert_yarn_classic, vendor_yarn_classic}; +use crate::vendor::yarn_berry_lock::revert_yarn_berry; +use crate::vendor::yarn_classic_lock::revert_yarn_classic; use crate::vendor::{RevertOutcome, VendorEntry, VendorOutcome}; /// Canonical-grammar patch uuid (the vendor path layer validates the shape @@ -205,7 +205,7 @@ impl ClassicFx { async fn vendor(&self) -> VendorOutcome { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); - vendor_yarn_classic( + crate::vendor::test_support::vendor_yarn_classic( "pkg:npm/ansi-regex@4.1.0", &self.root().join("node_modules/ansi-regex"), self.root(), @@ -750,7 +750,7 @@ impl BerryFx { let blobs = self.root().join(".socket/blobs"); let sources = PatchSources::blobs_only(&blobs); let purl = format!("pkg:npm/{name}@{version}"); - vendor_yarn_berry( + crate::vendor::test_support::vendor_yarn_berry( &purl, &self.root().join("node_modules").join(name), self.root(), diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 3227b7e27..d29c41a20 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -1104,6 +1104,7 @@ mod tests { base_purl: purl.to_string(), uuid: VUUID.to_string(), artifact: VendorArtifact { + yarn_berry10c0: None, path: rel_path.to_string(), sha256: String::new(), size: None, diff --git a/crates/socket-patch-core/tests/covgap_vendor_nuget_feed.rs b/crates/socket-patch-core/tests/covgap_vendor_nuget_feed.rs deleted file mode 100644 index b4eb5baab..000000000 --- a/crates/socket-patch-core/tests/covgap_vendor_nuget_feed.rs +++ /dev/null @@ -1,108 +0,0 @@ -//! Coverage mop-up for `vendor::nuget_feed` arms that need process-level -//! isolation. The local-rebuild stage `tempfile::tempdir()` failure is forced -//! by pointing `TMPDIR` at a nonexistent directory — safe only in a test -//! binary that owns its whole process (the lib test binary runs hundreds of -//! concurrent `tempdir()` users that a clobbered `TMPDIR` would flake). -//! Keep this file to env-mutating tests; anything else belongs in the file's -//! inline `#[cfg(test)]` module. - -#![cfg(unix)] - -use std::collections::HashMap; - -use socket_patch_core::manifest::schema::{PatchFileInfo, PatchRecord}; -use socket_patch_core::patch::apply::PatchSources; -use socket_patch_core::vendor::nuget_feed::vendor_nuget; -use socket_patch_core::vendor::VendorOutcome; - -/// Restores the pre-test `TMPDIR` on drop (panic-safe). -struct TmpdirGuard(Option); - -impl Drop for TmpdirGuard { - fn drop(&mut self) { - match self.0.take() { - Some(v) => std::env::set_var("TMPDIR", v), - None => std::env::remove_var("TMPDIR"), - } - } -} - -/// An unusable temp dir fails the local rebuild's private stage creation: -/// the vendor reports a per-package failure ("cannot create stage dir") -/// instead of panicking, and touches no project file. `TMPDIR` only steers -/// `std::env::temp_dir()` on unix. -#[cfg(unix)] -#[tokio::test] -async fn stage_tempdir_creation_failure_is_reported_not_fatal() { - // Build the whole fixture while TMPDIR is still valid. - let dir = tempfile::tempdir().unwrap(); - let root = dir.path(); - let installed = root.join("packages/newtonsoft.json/13.0.3"); - tokio::fs::create_dir_all(&installed).await.unwrap(); - // Any readable regular file works: the cached-nupkg read succeeds and - // `tempfile::tempdir()` fails BEFORE extraction ever parses the bytes. - tokio::fs::write(installed.join("newtonsoft.json.13.0.3.nupkg"), b"not-a-zip") - .await - .unwrap(); - let blobs = root.join("blobs"); - tokio::fs::create_dir_all(&blobs).await.unwrap(); - - let mut files = HashMap::new(); - files.insert( - "LICENSE.md".to_string(), - PatchFileInfo { - before_hash: "0".repeat(64), - after_hash: "1".repeat(64), - }, - ); - let record = PatchRecord { - uuid: "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f".to_string(), - exported_at: "2026-06-09T00:00:00Z".to_string(), - files, - vulnerabilities: HashMap::new(), - description: String::new(), - license: String::new(), - tier: String::new(), - }; - let sources = PatchSources { - blobs_path: &blobs, - diffs_path: None, - mem_blobs: None, - }; - - let guard = TmpdirGuard(std::env::var_os("TMPDIR")); - std::env::set_var("TMPDIR", root.join("no-such-tmpdir")); - let outcome = vendor_nuget( - "pkg:nuget/Newtonsoft.Json@13.0.3", - &installed, - root, - &record, - &sources, - "2026-06-09T00:00:00Z", - false, - false, - None, - ) - .await; - drop(guard); - - match outcome { - VendorOutcome::Done { - result, entry, .. - } => { - assert!(!result.success, "the stage failure must fail the vendor"); - assert!(entry.is_none(), "no ledger entry for a failed vendor"); - let err = result.error.as_deref().unwrap_or(""); - assert!(err.contains("cannot create stage dir"), "{err}"); - } - VendorOutcome::Refused { code, detail } => panic!("refused: {code}: {detail}"), - } - assert!( - !root.join(".socket").exists(), - "no partial artifact dir after the stage failure" - ); - assert!( - !root.join("nuget.config").exists(), - "no wiring after the stage failure" - ); -} diff --git a/crates/socket-patch-core/tests/vlt_locks.rs b/crates/socket-patch-core/tests/vlt_locks.rs index ae77dfe1f..186ecfc72 100644 --- a/crates/socket-patch-core/tests/vlt_locks.rs +++ b/crates/socket-patch-core/tests/vlt_locks.rs @@ -550,6 +550,67 @@ async fn vendor(case: &Case, staged: &Staged) -> VendorOutcome { diffs_path: None, mem_blobs: None, }; + use base64::Engine as _; + use sha2::Digest as _; + use socket_patch_core::api::client::{ApiClient, ApiClientOptions}; + use socket_patch_core::vendor::{VendorServiceConfig, VendorSource}; + use wiremock::matchers::{method, path}; + use wiremock::{Mock, MockServer, ResponseTemplate}; + let server = MockServer::start().await; + let mut archive = tar::Builder::new(flate2::write::GzEncoder::new( + Vec::new(), + flate2::Compression::default(), + )); + for (name, bytes) in [ + ( + "package/package.json", + fs::read(staged.installed.join("package.json")).unwrap(), + ), + ("package/index.js", PATCHED_JS.to_vec()), + ] { + let mut header = tar::Header::new_gnu(); + header.set_mode(0o644); + header.set_size(bytes.len() as u64); + header.set_cksum(); + archive + .append_data(&mut header, name, bytes.as_slice()) + .unwrap(); + } + let bytes = archive.into_inner().unwrap().finish().unwrap(); + let integrity = format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(&bytes)) + ); + let url = format!("{}/artifact.tgz", server.uri()); + Mock::given(method("POST")) + .and(path("/patch/package")) + .respond_with(ResponseTemplate::new(200).set_body_json( + serde_json::json!({ "results": { &case.uuid: { + "status": "granted", "purl": case.purl, "url": url, + "artifacts": [{ "kind": "tarball", "url": url, "integrity": { "sha512": integrity } }] + } } }), + )) + .mount(&server) + .await; + Mock::given(method("GET")) + .and(path("/artifact.tgz")) + .respond_with(ResponseTemplate::new(200).set_body_bytes(bytes)) + .mount(&server) + .await; + let service = VendorServiceConfig { + maven_config: None, + source: VendorSource::Service, + client: Some(ApiClient::new(ApiClientOptions { + api_url: server.uri(), + api_token: None, + org_slug: None, + use_public_proxy: true, + })), + use_public_proxy: true, + vendor_url: None, + patch_server_url: None, + offline: false, + }; vendor_npm_any( &case.purl, &staged.installed, @@ -559,7 +620,7 @@ async fn vendor(case: &Case, staged: &Staged) -> VendorOutcome { "2026-09-26T00:00:00Z", false, false, - None, + Some(&service), ) .await } diff --git a/docs/design/maven-vendoring.md b/docs/design/maven-vendoring.md index 8b77abf5b..94ec8e387 100644 --- a/docs/design/maven-vendoring.md +++ b/docs/design/maven-vendoring.md @@ -14,7 +14,6 @@ Hosted mode keeps its existing behavior. ```sh socket-patch vendor -socket-patch vendor --offline socket-patch vendor --check socket-patch vendor --check --local-repo /path/to/maven/repository socket-patch vendor --maven-config=none @@ -116,19 +115,7 @@ executes user build code to discover settings or metadata. ## Artifact identity and integrity -Service artifacts pass the existing download-integrity and patched-member -checks. Local jar rebuilds reproduce the server's stored ZIP encoding: upstream -entry order, executable bits, fixed 1980 timestamps, no directory entries or -extra fields, sorted additions, and stripped signature metadata. A checked-in -fixture generated by archiver 7.0.1 tests byte identity, including Unicode paths -and executable entries. The generator is beside the fixture under -`src/vendor/jvm/fixtures/repack/`. - -Online JVM vendoring checks upstream jar and metadata bytes against registry -SHA-512 sidecars, falling back to SHA-1, independently of local cache sidecars. -Offline metadata is accepted with its trust status recorded in the ledger. -Registry metadata fetches use a separate HTTP client and never send Socket API -credentials. `SOCKET_MAVEN_REGISTRY` supports a private mirror. +Service jars pass transfer-integrity and patched-member checks. The server constructs jars and strips invalidated signatures; the CLI never rebuilds jar archives. Online vendoring authenticates upstream POM and Gradle module metadata against registry sidecars. Metadata fetches use a separate HTTP client and do not send Socket API credentials. `SOCKET_MAVEN_REGISTRY` supports a private mirror. Maven v5 does not enable Resolver trusted-checksum processors at build time. Those processors crash some release reactors and system-scope dependencies, @@ -157,7 +144,7 @@ ledger. Patch updates remove superseded Maven trees after the new wiring is committed. Gradle updates keep the same artifact paths. Unrecognized or forged paths cannot direct writes outside the backend's allowed files. -`repair` reconstructs missing/corrupt artifacts using the same backend. The +`repair` redownloads the exact recorded jar and checks regenerated repository metadata against the ledger, preserving project wiring. Offline repair cannot restore missing or corrupt artifacts. The ledger is required for exact reversal; restore a deleted ledger from version control. In-place ledger reconstruction remains outside v5's repair contract. diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 8ab95c894..7a475455a 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -385,13 +385,7 @@ Honest limits of the Maven and NuGet flows — documented behavior, not bugs: client-side content pin (vendored surfaces this as a `vendor_nuget_no_lockfile` warning; the feed + source mapping still force the patched copy). -* **NuGet package signatures (local vendoring).** Rebuilding a `.nupkg` changes - its contents, so the local builder removes the upstream `.signature.p7s` rather - than retaining an invalid signature. Environments requiring signed packages need - a compatible signing policy or an appropriate service artifact; local vendoring - does not preserve the upstream author's signature. Service artifacts are copied - without local repacking. The current implementation uses a folder feed with - source mapping. +* **NuGet package signatures.** The server constructs the patched `.nupkg` and removes invalidated upstream signatures. The CLI verifies and stores the served archive without repacking it. Vendoring uses a folder feed with source mapping; installations requiring signed packages need an appropriate server artifact and signing policy. ## Cargo: shared registry cache diff --git a/docs/testing/pipenv-compatibility.md b/docs/testing/pipenv-compatibility.md index 086ee8e69..1a108fd65 100644 --- a/docs/testing/pipenv-compatibility.md +++ b/docs/testing/pipenv-compatibility.md @@ -19,7 +19,7 @@ requirements.txt lanes of the same ecosystem. |-------|--------|----------|-------| | `Pipfile.lock`, `pipfile-spec: 6` (Pipenv 7 and later) | Every category (`default`, `develop`, Pipenv 2022+ named categories) that pins the patched release becomes `{"file" \| "path": "#sha256=", "hashes": ["sha256:"]}` with `markers`/`extras`/`index` kept as Pipenv wrote them and `version` dropped. `path` for Pipenv 7–11, `file` from 2018. `_meta` (the Pipfile content hash) and the Pipfile are untouched. | Every matching category refers to the committed wheel under `.socket/vendor/pypi//`; wheels with extras use `path` (Pipenv 2022's file-URL bug). Requires Pipenv 2018 or later (`pypi_pipenv_installer_unsupported`). | Independent of the lock: patches the installed distribution in the project's venv — in-project `.venv`, `VIRTUAL_ENV`, or Pipenv's default `$WORKON_HOME/-[-]` (discovered without running Pipenv). | | `Pipfile.lock`, `pipfile-spec` < 6 (Pipenv 0–6) | Refused (`redirect_pipenv_skipped`), lock untouched. | Refused (`pypi_pipenv_spec_unsupported`). | Works. | -| Lock-only checkout (nothing installed) | Discovered from the lock and redirected. | Discovered from the lock; the pristine wheel is fetched by one of the lock's recorded digests (Pipenv records every release file's sha256 without filenames) through PyPI's JSON API, verified against the same digest, and the patched wheel comes from the service. | Nothing to patch (no installed distribution); the lock's pins are listed as lockfile-only packages. | +| Lock-only checkout (nothing installed) | Discovered from the lock and redirected. | Discovered from the lock; the patched wheel or source distribution is downloaded and verified from the service without a local install. | Nothing to patch (no installed distribution); the lock's pins are listed as lockfile-only packages. | Both hash fields are load-bearing: Pipenv 2023+ verifies the `#sha256=` URL fragment, 2018–2022 verify the `hashes` list, Pipenv 11 accepts either. A diff --git a/docs/testing/vendored-production-e2e.md b/docs/testing/vendored-production-e2e.md index a31f00545..b96770b8d 100644 --- a/docs/testing/vendored-production-e2e.md +++ b/docs/testing/vendored-production-e2e.md @@ -38,13 +38,7 @@ not prevent this proof. Per-release coverage lives in the ### RubyGems artifact validation -A vendored Bundler path source needs a valid stub gemspec. The CLI validates -service and local stubs before writing them. An invalid service stub causes -`--vendor-source auto` to try a local build when possible; -`--vendor-source service` refuses with `vendor_prebuilt_stub_invalid`. -A missing local source can prevent that fallback. The production test accepts -and checks the actual acquisition route, so it does not assume a previously -observed server defect is still present. +A vendored Bundler path source needs the server's valid stub gemspec. The CLI downloads and validates it with the archive. Missing or invalid server stubs fail closed; there is no local gem build fallback. ## Running diff --git a/docs/testing/vlt-compatibility.md b/docs/testing/vlt-compatibility.md index 4fd506694..567ed045d 100644 --- a/docs/testing/vlt-compatibility.md +++ b/docs/testing/vlt-compatibility.md @@ -218,7 +218,7 @@ them per OS). | Binary | Suite | Legs | |---|---|---| | `e2e_redirect_vlt_build` | `hosted` | `scan_fresh_ci`, `frozen_dead_registry`, `ordinary_install_stable`, `get_uuid_fresh_ci`, `tamper_cold_eintegrity`, `rollback_byte_exact`, `rerun_noop`, `warm_tree_invalidates`, `no_cleanup_stays_stale`, `heal_rule_b_hidden_lock_without_node`, `heal_rule_c_no_hidden_lock`, `heal_rule_c_no_record`, `scoped`, `peer_workspace_instances`, `peer_rekey_rollback`, `install_newdep_preserves`, `update_drops`, `resave_install_rollback`, `resave_crlf_rollback`, `resave_update_rollback`, `crlf_lock`, `mirror_registries_npm`, `scalar_registry`, `named_alias_untouched`, `scoped_registry_untouched`, `jsr_untouched`, `default_registry_alias`, `registry_from_env`, `registry_from_user_config`, `content_encoding_refused`, `old_lockfile_ignored`, `warm_cache_hazard`, `idempotence`, `manifestless_vex`, `ts_written_lock`, `optional_dependency_heal`, `then_vendored_optional_takeover`, `platform_optional_skipped` | -| `e2e_vendor_vlt_build` | `vendored` | `scan_fresh_ci`, `get_build_fresh_ci`, `get_service_fresh_ci`, `durability`, `workspace_member_selfref`, `alias_selfref`, `peer_root_selfref`, `peer_member_selfref`, `single_peer_context`, `optional_warm_reinstall`, `dep_with_deps`, `hostile_gitignore`, `autocrlf_checkout`, `bin_bearing`, `package_json_devdeps_patch`, `repair_rebuilds`, `idempotency`, `revert_byte_exact`, `resave_install_revert`, `resave_uninstall_revert`, `resave_crlf_revert`, `tamper_planted_file`, `tamper_file_content`, `tamper_payload_package_json`, `tamper_symlink_outside`, `tamper_deleted_gitignore`, `tamper_lock_file_node_path`, `transitive_refused`, `legacy_lockfile_warning`, `absent_version_refused`, `lockless_reinstall`, `manifestless_vex` | +| `e2e_vendor_vlt_build` | `vendored` | `scan_fresh_ci`, `get_auto_fresh_ci`, `get_service_fresh_ci`, `durability`, `workspace_member_selfref`, `alias_selfref`, `peer_root_selfref`, `peer_member_selfref`, `single_peer_context`, `optional_warm_reinstall`, `dep_with_deps`, `hostile_gitignore`, `autocrlf_checkout`, `bin_bearing`, `package_json_devdeps_patch`, `repair_rebuilds`, `idempotency`, `revert_byte_exact`, `resave_install_revert`, `resave_uninstall_revert`, `resave_crlf_revert`, `tamper_planted_file`, `tamper_file_content`, `tamper_payload_package_json`, `tamper_symlink_outside`, `tamper_deleted_gitignore`, `tamper_lock_file_node_path`, `transitive_refused`, `legacy_lockfile_warning`, `absent_version_refused`, `lockless_reinstall`, `manifestless_vex` | | `mode_migration_vlt` | `migration` | `vendored_then_hosted`, `hosted_then_vendored`, `dry_run_parity`, `scoped_unwind_one_of_two`, `rollback_from_mixed`, `agent_apply_yields_to_vendored`, `agent_apply_after_hosted`, `hosted_scan_keeps_agent_patched_tree`, `agent_rollback_after_takeovers`, `pm_switch_npm_to_vlt`, `pm_switch_vlt_to_npm`, `flavor_changed`, `upgrade_hosted`, `upgrade_vendored` | | `e2e_safety_vlt` | `safety` | `linux_auto`, `explicit_hardlink`, `private_copies`, `cross_device_cache`, `agent_rollback`, `peer_fanout`, `hosted_heal`, `vendored_build`, `vendor_revert_and_repair`, `layout_note` | | `e2e_vlt` | `agent` | `scan_apply_rollback_list`, `get_and_remove`, `install_then_apply_patches_file`, `transitive_only_dep_apply_patches_store`, `lockfile_supplement`, `launcher`, `persistence_survives`, `persistence_reverted_by_reinstall`, `reruns_and_vex` | diff --git a/docs/testing/vlt-coverage.json b/docs/testing/vlt-coverage.json index 2d8adfd4f..ad42fcc42 100644 --- a/docs/testing/vlt-coverage.json +++ b/docs/testing/vlt-coverage.json @@ -242,7 +242,7 @@ ], "vendored/get": [ "get_uuid_vendored_vlt_vendors_refuses_and_agent_bypasses", - "vlt_pinned_matrix_vendored_get_build_fresh_ci", + "vlt_pinned_matrix_vendored_get_auto_fresh_ci", "vlt_pinned_matrix_vendored_get_service_fresh_ci" ], "vendored/revert": [ diff --git a/docs/usage.md b/docs/usage.md index 260123714..4396e347f 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -92,18 +92,23 @@ mirror or package-manager cache, and test a clean offline install before relying on an airgapped build. Integrity enforcement and cache behavior differ by package manager; see [ecosystem support](ecosystems.md). -Artifact acquisition is controlled by `--vendor-source`: - -| Value | Behavior | -| --- | --- | -| `auto` (default) | Use a service artifact when available, with local building as a fallback for eligible misses | -| `service` | Require a service artifact; refuse if unavailable | -| `build` | Build locally from available package and patch data | - -An integrity failure is refused rather than bypassed with a fallback. Offline -operation never fetches missing inputs. `repair` can restore missing or corrupt -artifacts from a valid ledger when sufficient inputs are available; it cannot -reconstruct a lost vendor ledger. Restore a lost ledger from version control. +Vendoring downloads prebuilt artifacts from the patch service and verifies +archive integrity and patched file hashes before installation. The service +owns archive construction, including Yarn Berry cache checksums. Python +vendoring accepts both wheels and source distributions supplied by the service. + +`--vendor-source service` is the default. `auto` remains an alias for the same +behavior; `build` is rejected. A missing artifact, pending build, network error, +or integrity mismatch fails without a local build fallback. Healthy committed +artifacts can be reused offline. + +`repair` redownloads missing or corrupt artifacts and checks them against the +existing ledger before replacement. It preserves project wiring and recorded +integrity; different archive bytes require an explicit new vendoring operation. +Repair cannot reconstruct a lost vendor ledger. Restore it from version control. + +The N-API crate and in-memory patch engine remain available for hosted GitHub +App workflows. Removing local vendoring builders does not remove those APIs. ### Maven reactors and Gradle