Fix pnpm modulesDir store being skipped (#661, #696)

CHANGELOG.md

@@ -206,6 +206,13 @@ limits, and required install commands.
   `virtualStoreDir`, instead of reporting them `package_not_installed` (#359,
   #362). A store outside the project, such as pnpm's global virtual store, is
   shared with other projects and is still not patched in place.
+- Agent mode and `vex` find packages installed under pnpm's `modulesDir`
+  (`modulesDir:` in `pnpm-workspace.yaml`, or `modules-dir` in `.npmrc`).
+  From pnpm 10.12 the virtual store moves there (`<modulesDir>/.pnpm`), so
+  `apply` exited 0 with the package unpatched as "not installed", and
+  hosted `vex` attested `not_affected` over the unpatched install. Hosted
+  `vex` also no longer attests a pinned npm package the crawler cannot see
+  because pnpm keeps the installed store outside the project (#661, #696).
 - npm locks keep their own layout when edited. `scan --mode hosted`,
   `scan --mode vendored`, `rollback` and `vendor --revert`
   re-serialized `package-lock.json` / `npm-shrinkwrap.json` with LF line

crates/socket-patch-cli/CLI_CONTRACT.md

@@ -386,7 +386,7 @@ Recognition rules that hold for every ecosystem:
 | Wiring | Evidence (verify mode) | Marker |
 |---|---|---|
 | Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync`. | `(vendored)` |
-| Hosted: a discovered patch-host reference (or a live pre-v5 redirect-ledger record) | The installed copies the build **consumes** through the hosted wiring are hash-verified when any exist: the Go replacement module, never the pristine `M@v` in the module cache; the Socket-registry cargo source dir; maven's suffixed version. Installed evidence wins: `hash_mismatch` / `not_applied` are omitted. With **nothing installed**, a discovered reference whose lock pins the artifact (or whose format's rewriter never writes a pin) attests from that pin, which is the same evidence as in-run `scan --mode hosted --vex`. A pre-v5 ledger-only record, or a reference whose required pin is missing, stays `package_not_found`. So do purls that `--ecosystems` kept out of the crawl, because "not installed" has to mean the crawler looked. | `(redirected)` |
+| Hosted: a discovered patch-host reference (or a live pre-v5 redirect-ledger record) | The installed copies the build **consumes** through the hosted wiring are hash-verified when any exist: the Go replacement module, never the pristine `M@v` in the module cache; the Socket-registry cargo source dir; maven's suffixed version. Installed evidence wins: `hash_mismatch` / `not_applied` are omitted. With **nothing installed**, a discovered reference whose lock pins the artifact (or whose format's rewriter never writes a pin) attests from that pin, which is the same evidence as in-run `scan --mode hosted --vex`. A pre-v5 ledger-only record, or a reference whose required pin is missing, stays `package_not_found`. So do purls that `--ecosystems` kept out of the crawl, because "not installed" has to mean the crawler looked. The same goes for npm purls when an installed pnpm tree records its virtual store outside the project (`enableGlobalVirtualStore`, or a `virtualStoreDir` that climbs out): transitive deps there are invisible to the crawler. A pnpm `modulesDir` inside the project is crawled. | `(redirected)` |
 | Agent: a manifest record with no live hosted/vendored wiring | The installed tree, unchanged. **Every** installed copy the crawler finds for the purl (npm nests duplicates of one `name@version`) must hash to the patched bytes, as `apply` patches every copy. One unpatched copy omits the purl with that copy's tag (`not_applied` / `hash_mismatch`). | none |
 
 **Liveness gates.** These gates run before hashing, and `--no-verify` / `--vex-no-verify` skips only the hashing, never the gates:

crates/socket-patch-cli/src/commands/vex.rs

@@ -598,11 +598,21 @@ async fn generate_vex(
             )
             .is_empty()
         };
+        //
+        // Nor did it look when pnpm keeps the installed virtual store
+        // outside the project (its global virtual store, or a
+        // `virtualStoreDir` that climbs out): only direct deps are linked
+        // into the project, so an npm purl not found may be an installed,
+        // unpatched transitive dep (#696).
+        let npm_store_hidden =
+            socket_patch_core::crawlers::npm_crawler::pnpm_store_outside_project(&common.cwd);
+        let hidden = |purl: &str| npm_store_hidden && purl.starts_with("pkg:npm/");
         let mut lockfile_attested = Vec::new();
         outcome.failed.retain(|f| {
             let excused = f.reason == "package_not_found"
                 && plan.lockfile_basis.contains(&f.purl)
-                && crawled(&f.purl);
+                && crawled(&f.purl)
+                && !hidden(&f.purl);
             if excused {
                 lockfile_attested.push(f.purl.clone());
             }

crates/socket-patch-cli/tests/e2e_vex_redirect.rs

@@ -2294,6 +2294,144 @@ fn yarn_modules_folder_install_is_hash_verified_not_lockfile_attested() {
     );
 }
 
+/// A pnpm v9 lock whose only package, `left-pad@1.3.0`, is pinned to its
+/// hosted patch artifact.
+fn pnpm_lock_pinning_left_pad() -> String {
+    format!(
+        "lockfileVersion: '9.0'\n\nimporters:\n  .:\n    dependencies:\n      left-pad:\n        \
+         specifier: 1.3.0\n        version: 1.3.0\n\npackages:\n  left-pad@1.3.0:\n    \
+         resolution: {{integrity: {SRI}, tarball: {}}}\n\nsnapshots:\n  left-pad@1.3.0: {{}}\n",
+        hosted_npm_url("left-pad", "1.3.0", UUID)
+    )
+}
+
+/// REGRESSION (#696): pnpm 10.12+ with `modulesDir: deps` installs into
+/// `deps/.pnpm` and there is no `node_modules`. The crawler never looked
+/// there, so the unpatched installed copy read as "nothing installed" and
+/// the pinned hosted lock attested `not_affected`. Installed evidence
+/// wins: the copy is hash-checked and omitted. With nothing installed the
+/// lock basis still attests.
+#[test]
+fn pnpm_modules_dir_install_is_hash_verified_not_lockfile_attested() {
+    for (file, text) in [
+        ("pnpm-workspace.yaml", "modulesDir: deps\n"),
+        (".npmrc", "modules-dir=deps\n"),
+    ] {
+        let tmp = tempfile::tempdir().unwrap();
+        let cwd = tmp.path();
+        let purl = "pkg:npm/left-pad@1.3.0";
+        std::fs::write(
+            cwd.join("package.json"),
+            r#"{ "name": "app", "version": "1.0.0", "dependencies": { "left-pad": "1.3.0" } }"#,
+        )
+        .unwrap();
+        std::fs::write(cwd.join(file), text).unwrap();
+        std::fs::write(cwd.join("pnpm-lock.yaml"), pnpm_lock_pinning_left_pad()).unwrap();
+        let pkg = cwd.join("deps/.pnpm/left-pad@1.3.0/node_modules/left-pad");
+        std::fs::create_dir_all(&pkg).unwrap();
+        std::fs::write(
+            pkg.join("package.json"),
+            r#"{ "name": "left-pad", "version": "1.3.0" }"#,
+        )
+        .unwrap();
+        std::fs::write(pkg.join("index.js"), b"unpatched upstream bytes\n").unwrap();
+        let patched = b"hosted patched index\n";
+        let (_rt, server) = serve_patch_views(vec![(
+            UUID.to_string(),
+            left_pad_view(&compute_git_sha256_from_bytes(patched)),
+        )]);
+
+        let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+        assert_eq!(
+            code,
+            Some(1),
+            "{file}: the unpatched deps/.pnpm copy must not attest: {env}"
+        );
+        assert_eq!(skipped_reason(&env, purl), "hash_mismatch", "{file}: {env}");
+
+        std::fs::write(pkg.join("index.js"), patched).unwrap();
+        let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+        assert_eq!(code, Some(0), "{file}: a patched store copy attests: {env}");
+
+        std::fs::remove_dir_all(cwd.join("deps")).unwrap();
+        let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+        assert_eq!(
+            code,
+            Some(0),
+            "{file}: nothing installed: the lock basis attests: {env}"
+        );
+    }
+}
+
+/// REGRESSION (#696, follow-up variants): with pnpm's global virtual
+/// store, or a `virtualStoreDir` outside the project, a TRANSITIVE dep is
+/// installed only in that outside store, which the crawler does not walk.
+/// "Not found" then is not "not installed": the pinned lock must not
+/// attest over an install the crawler could not inspect. Without an
+/// install the lock basis still attests.
+#[test]
+fn pnpm_store_outside_project_is_not_lockfile_attested() {
+    let outside = tempfile::tempdir().unwrap();
+    let store = outside.path().join("v11/links");
+    let copy = store.join("@/left-pad/1.3.0/abc/node_modules/left-pad");
+    std::fs::create_dir_all(&copy).unwrap();
+    std::fs::write(
+        copy.join("package.json"),
+        r#"{ "name": "left-pad", "version": "1.3.0" }"#,
+    )
+    .unwrap();
+    std::fs::write(copy.join("index.js"), b"unpatched upstream bytes\n").unwrap();
+    let tmp = tempfile::tempdir().unwrap();
+    let cwd = tmp.path();
+    let purl = "pkg:npm/left-pad@1.3.0";
+    std::fs::write(
+        cwd.join("package.json"),
+        r#"{ "name": "app", "version": "1.0.0" }"#,
+    )
+    .unwrap();
+    std::fs::write(cwd.join("pnpm-lock.yaml"), pnpm_lock_pinning_left_pad()).unwrap();
+    let nm = cwd.join("node_modules");
+    std::fs::create_dir_all(&nm).unwrap();
+    let recorded = format!("{}", store.display()).replace('\\', "\\\\");
+    std::fs::write(
+        nm.join(".modules.yaml"),
+        format!("{{\"layoutVersion\": 5, \"virtualStoreDir\": \"{recorded}\"}}"),
+    )
+    .unwrap();
+    let patched = b"hosted patched index\n";
+    let (_rt, server) = serve_patch_views(vec![(
+        UUID.to_string(),
+        left_pad_view(&compute_git_sha256_from_bytes(patched)),
+    )]);
+
+    let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+    assert_eq!(
+        code,
+        Some(1),
+        "an install the crawler cannot see must not attest: {env}"
+    );
+    assert_eq!(skipped_reason(&env, purl), "package_not_found", "{env}");
+
+    // A store inside the project is walked, so its absence is real.
+    let inside = cwd.join(".vstore");
+    std::fs::create_dir_all(&inside).unwrap();
+    std::fs::write(
+        nm.join(".modules.yaml"),
+        "{\"layoutVersion\": 5, \"virtualStoreDir\": \"../.vstore\"}",
+    )
+    .unwrap();
+    let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+    assert_eq!(code, Some(0), "an in-project store hides nothing: {env}");
+
+    std::fs::remove_dir_all(&nm).unwrap();
+    let (code, env) = vex_json(cwd, &["--proxy-url", &server.uri()]);
+    assert_eq!(
+        code,
+        Some(0),
+        "nothing installed: the lock basis attests: {env}"
+    );
+}
+
 /// REGRESSION (#518): Rush installs every package into
 /// `common/temp/node_modules/.pnpm` and the projects' `node_modules` only
 /// link their DIRECT deps. The crawler pruned `temp`, so an unpatched

crates/socket-patch-cli/tests/in_process_alternate_installers.rs

@@ -1342,6 +1342,125 @@ async fn rush_transitive_dep_in_common_temp_store_is_patched() {
     );
 }
 
+/// REGRESSION (#661), real pnpm: with `modulesDir: deps` (pnpm 10+
+/// reads `pnpm-workspace.yaml`, older pnpm `.npmrc` `modules-dir`), pnpm
+/// 10.12+ puts the virtual store in `deps/.pnpm` and no package under
+/// `node_modules`. Agent-mode apply must patch the store copy. Older pnpm
+/// keeps the store in `node_modules/.pnpm`, which is not this layout, so
+/// the leg skips there.
+#[tokio::test]
+#[serial]
+async fn pnpm_modules_dir_install_then_apply_patches_file() {
+    if !has("pnpm") {
+        println!("SKIP: pnpm not on PATH");
+        return;
+    }
+
+    let tmp = tempfile::tempdir().unwrap();
+    std::fs::write(
+        tmp.path().join("package.json"),
+        r#"{ "name": "pnpm-md-test", "version": "0.0.0", "dependencies": { "ms": "2.1.3" } }"#,
+    )
+    .unwrap();
+    std::fs::write(tmp.path().join("pnpm-workspace.yaml"), "modulesDir: deps\n").unwrap();
+    std::fs::write(tmp.path().join(".npmrc"), "modules-dir=deps\n").unwrap();
+
+    let status = pm_command("pnpm", &["npm_config_"])
+        .args(["install", "--silent", "--no-frozen-lockfile"])
+        .current_dir(tmp.path())
+        .stdout(std::process::Stdio::piped())
+        .stderr(std::process::Stdio::piped())
+        .output()
+        .expect("pnpm install");
+    if !status.status.success() {
+        println!(
+            "SKIP: pnpm install failed: {}",
+            String::from_utf8_lossy(&status.stderr)
+        );
+        return;
+    }
+    let ms_index = tmp
+        .path()
+        .join("deps/.pnpm/ms@2.1.3/node_modules/ms/index.js");
+    if !ms_index.exists() {
+        println!("SKIP: this pnpm keeps its virtual store outside modulesDir (< 10.12)");
+        return;
+    }
+    assert!(!tmp.path().join("node_modules/.pnpm").exists());
+
+    let original = std::fs::read(&ms_index).expect("read ms/index.js");
+    let before_hash = git_sha256(&original);
+    let mut patched = original.clone();
+    patched.extend_from_slice(b"\n// SOCKET-PATCH-PNPM-MODULES-DIR-REAL-MARKER\n");
+    let after_hash = git_sha256(&patched);
+
+    let socket = tmp.path().join(".socket");
+    write_manifest(&socket, "pkg:npm/ms@2.1.3", &before_hash, &after_hash);
+    let blobs = socket.join("blobs");
+    std::fs::create_dir_all(&blobs).unwrap();
+    std::fs::write(blobs.join(&after_hash), &patched).unwrap();
+
+    let code = apply_run(default_apply(tmp.path())).await;
+    assert_eq!(code, 0, "apply must patch the pnpm modulesDir install");
+    assert_patched(&ms_index, &patched, &before_hash, &after_hash);
+}
+
+/// REGRESSION (#661): with pnpm 10.12+ `modulesDir: deps`, pnpm installs
+/// into `deps/.pnpm` and there is no `node_modules`. Agent-mode apply
+/// reported the package not installed ("resolved by the project
+/// lockfile") and exited 0 with it unpatched; the store copy is now
+/// patched.
+#[tokio::test]
+#[serial]
+async fn pnpm_modules_dir_install_is_patched() {
+    for (file, text) in [
+        ("pnpm-workspace.yaml", "modulesDir: deps\n"),
+        (".npmrc", "modules-dir=deps\n"),
+    ] {
+        let tmp = tempfile::tempdir().unwrap();
+        let root = tmp.path();
+        std::fs::write(
+            root.join("package.json"),
+            r#"{ "name": "app", "version": "1.0.0", "dependencies": { "left-pad": "1.3.0" } }"#,
+        )
+        .unwrap();
+        std::fs::write(root.join(file), text).unwrap();
+        std::fs::write(
+            root.join("pnpm-lock.yaml"),
+            "lockfileVersion: '9.0'\n\nimporters:\n  .:\n    dependencies:\n      left-pad:\n        \
+             specifier: 1.3.0\n        version: 1.3.0\n\npackages:\n  left-pad@1.3.0:\n    \
+             resolution: {integrity: sha512-upstream==}\n\nsnapshots:\n  left-pad@1.3.0: {}\n",
+        )
+        .unwrap();
+        let pkg = root.join("deps/.pnpm/left-pad@1.3.0/node_modules/left-pad");
+        std::fs::create_dir_all(&pkg).unwrap();
+        std::fs::write(
+            pkg.join("package.json"),
+            r#"{ "name": "left-pad", "version": "1.3.0" }"#,
+        )
+        .unwrap();
+        let original = b"module.exports = leftPad;\n".to_vec();
+        std::fs::write(pkg.join("index.js"), &original).unwrap();
+        let before_hash = git_sha256(&original);
+        let mut patched = original.clone();
+        patched.extend_from_slice(b"\n// SOCKET-PATCH-PNPM-MODULES-DIR-MARKER\n");
+        let after_hash = git_sha256(&patched);
+        let socket = root.join(".socket");
+        write_manifest(&socket, "pkg:npm/left-pad@1.3.0", &before_hash, &after_hash);
+        let blobs = socket.join("blobs");
+        std::fs::create_dir_all(&blobs).unwrap();
+        std::fs::write(blobs.join(&after_hash), &patched).unwrap();
+
+        let code = apply_run(default_apply(root)).await;
+        assert_eq!(
+            code, 0,
+            "{file}: apply must find the pnpm modulesDir store copy"
+        );
+        assert_patched(&pkg.join("index.js"), &patched, &before_hash, &after_hash);
+        assert!(!root.join("node_modules").exists());
+    }
+}
+
 /// REGRESSION (#493): with `.yarnrc` `--modules-folder deps`, yarn classic
 /// installs into `deps/` and there is no `node_modules`. Agent-mode apply
 /// reported the package `package_not_installed` (exit 0 from `scan`,