Skip to content

fix: improve MCP dependency pinning audit 🤖🤖🤖 - #3852

Open
tomelias10 wants to merge 2 commits into
github:mainfrom
tomelias10:fix/mcp-security-audit-dependency-pinning
Open

tomelias10 wants to merge 2 commits into
github:mainfrom
tomelias10:fix/mcp-security-audit-dependency-pinning

Conversation

@tomelias10

Copy link
Copy Markdown

Summary

Improves the existing mcp-security-audit skill's dependency-pinning check so it catches mutable package references beyond only @latest.

The current example logic misses bare npx package references and treats npx without -y as a security finding. It also demonstrates remediation by substituting an arbitrary @1.2.3, which can imply a version was reviewed when that evidence is unknown.

This change:

  • detects bare package names and @latest as mutable references
  • detects non-exact selectors/ranges separately
  • handles scoped packages correctly
  • recognizes common JS package runners (npx, npm exec/npm x, bunx/bun x, pnpm dlx, yarn dlx)
  • treats dependency mutability as a reproducibility/review-boundary signal, not proof of compromise
  • recommends pinning the exact version the team actually reviewed rather than inventing today's/current version
  • treats -y / --yes as CI ergonomics/context rather than a vulnerability by itself
  • updates the generated skills README description so the expanded coverage is discoverable

Validation

  • npm run skill:validate — all 423 skills valid
  • npm run build — completed successfully
  • git diff --check — clean
  • behavior matrix: 9/9 cases passed covering bare, @latest, scoped ranges, scoped exact versions, npm exec, pnpm dlx, bunx, Windows npx.cmd, and unsupported commands

The generated docs/README.agents.md changed locally only because the build fetched newer live MCP Registry metadata; that unrelated registry drift was intentionally excluded from this PR.

Scope

This does not execute MCP servers or resolve/download referenced packages. It only improves the static review guidance already present in the skill.

@github-actions github-actions Bot added the skills PR touches skills label Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 1 changed file(s).

Severity Count
🔴 High 0
🟠 Medium 7
ℹ️ Info 0
Severity Rule File Line Match
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 8 - Detecting mutable npm/npx, bunx, pnpm dlx, yarn dlx, and npm exec references in MCP configurations
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 177 if command in {"npx", "npx.cmd", "bunx"}:
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 220 { "command": "npx", "args": ["-y", "my-mcp-server@​​2.1.0"] }
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 225 { "command": "npx", "args": ["-y", "my-mcp-server@​​latest"] }
🟠 unpinned-version-indicator skills/mcp-security-audit/SKILL.md 225 { "command": "npx", "args": ["-y", "my-mcp-server@​​latest"] }
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 226 { "command": "npx", "args": ["-y", "my-mcp-server"] }
🟠 package-exec-command skills/mcp-security-audit/SKILL.md 227 { "command": "npx", "args": ["-y", "@​​scope/server@​​^2.1.0"] }

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Contributor Reputation Check: HIGH risk

Check Risk
Profile HIGH
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the needs-review:HIGH Contributor reputation check flagged HIGH risk label Sep 25, 2026
@tomelias10
tomelias10 force-pushed the fix/mcp-security-audit-dependency-pinning branch from 2811332 to e090bb5 Compare September 25, 2026 10:23
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Vally Lint Results

✅ All checks passed

Scope Checked
Skills 1
Agents 0
Total 1
Severity Count
❌ Errors 0
⚠️ Warnings 0
ℹ️ Advisories 0

Summary

Level Finding
ℹ️ ✅ mcp-security-audit (2/2 checks passed)
ℹ️ ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
ℹ️ ✓ spec-compliance: All spec checks passed.
ℹ️ ✓ [valid-refs] All file references across 1 skill(s) are valid.
ℹ️ ✓ valid-refs: All file references resolve to existing files within the skill directory.
ℹ️ 1 skill(s) linted, 1 passed
Full linter output
### Linting skills/mcp-security-audit
✅ mcp-security-audit (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

@tomelias10

Copy link
Copy Markdown
Author

The PR Risk Scan's 7 MEDIUM hits are expected review targets for this change: the skill is specifically teaching Copilot to identify package-runner commands (npx, npm exec, etc.) and mutable selectors such as @latest.

They appear only in static-analysis logic and inert configuration examples. This change does not execute, install, resolve, or download any referenced MCP package. The updated guidance also explicitly says that a mutable reference is a reproducibility/review-boundary signal, not proof of compromise, and that -y/--yes is not a vulnerability by itself.

I left the examples unobfuscated so the security scanner and human reviewers can see the exact patterns being audited.

@github-actions github-actions Bot added merge-risk:high requires-submitter-fixes Submission has quality-gate findings that submitter must fix before maintainer review labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

🚦 Submission status: 🛠️ Requires submitter fixes

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 2 approvals from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier
  • Label needs-review:HIGH flags a high contributor-risk signal

Automated checks

Check Status Details
Line endings ✅ Passed Passed · logs
Spelling ✅ Passed Passed · logs
Generated README consistency ❌ Contribution failure Failed step: Fail workflow if files need updating · logs
Skill validation ⏳ Pending Waiting for the check to start
Skill lint (vally) ✅ Passed Passed · logs
Risk scan ✅ Passed Passed · logs
Contributor reputation ✅ Passed Passed · logs
Duplicate resource scan ✅ Passed Passed · logs
PR quality signal ⏭️ Skipped Skipped by its workflow · logs
Contributor risk signal 🔧 Infrastructure failure The contributor check succeeded but its result artifact was missing, unreadable, or for another commit · logs

Action needed

  • Generated README consistency failed. Run npm start locally and commit the regenerated files.
  • 🔧 Contributor risk signal hit an automation problem that is not caused by your contribution. Comment /rerun-checks to retry; maintainers are notified if it keeps failing.

Review

  • Approvals: 0/2
  • Assigned reviewer: aaronpowell
  • Review target date: not set
  • Still needed: 2 more approval(s); an approval from a maintainer with admin or maintain permission
  • The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command Who What it does
/rerun-checks PR author, maintainers Re-runs failed or incomplete checks and re-evaluates this gate
/request-review PR author, maintainers Asks the review rotation to assign a reviewer (adds needs-reviewer)

Updated for e090bb5 · This comment is maintained automatically — see submission gate docs.

@tomelias10
tomelias10 requested review from a team as code owners October 1, 2026 03:13

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk:high needs-review:HIGH Contributor reputation check flagged HIGH risk requires-submitter-fixes Submission has quality-gate findings that submitter must fix before maintainer review skills PR touches skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant