Skip to content

Add shepherd-task campaign automation plugin 🤖🤖🤖 - #3721

Draft
edburns wants to merge 191 commits into
github:mainfrom
edburns:edburns/dd-3068479-shepherd-task
Draft

edburns wants to merge 191 commits into
github:mainfrom
edburns:edburns/dd-3068479-shepherd-task

Conversation

@edburns

@edburns edburns commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory.
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot.
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the main branch for this pull request.

Files in this PR outside the shepherd-task system

The following files are outside skills/shepherd-task* and
plugins/shepherd-task:

  • .github/plugin/marketplace.json — generated marketplace metadata registering
    the plugin.
  • AGENTS.md — documents the repository-wide pluginFiles composition field
    and its validation requirements.
  • CONTRIBUTING.md — documents how plugin authors declare plugin-owned support
    files and directories.
  • docs/README.plugins.md — generated plugin index containing the
    shepherd-task plugin.
  • docs/README.skills.md — generated skill index containing the
    shepherd-task-* skills.
  • eng/lib/plugin-files.mjs — validates repository-wide pluginFiles
    declarations, including nested symlink containment.
  • eng/materialize-plugins.mjs — handles pluginFiles while materializing
    plugin manifests.
  • eng/materialize-plugins.test.mjs — provides regression coverage for
    pluginFiles materialization.
  • eng/validate-plugins.mjs — integrates pluginFiles validation into the
    repository-wide plugin validator.
  • eng/validate-plugins.test.mjs — provides regression coverage for valid,
    malformed, escaping, and dangling pluginFiles references.

shepherd-task is not composed solely of reusable skills. Its installed plugin also requires plugin-owned runtime content that already lives beneath plugins/shepherd-task, including the Bash and PowerShell orchestration scripts, version metadata, installation utilities, and contract fixtures used to verify an installation. The repository’s existing composition fields could materialize shared agents, hooks, skills, and extensions, but there was no declarative way for a plugin to identify these plugin-local support files as part of its complete distributable estate.

The new repository-only pluginFiles field fills that gap. plugins/shepherd-task/plugin.json uses it to declare the local files and directories that must remain in the packaged plugin. The materializer validates those declarations and removes the repository-specific composition metadata from the served Agent Plugins manifest. Validation requires references to exist, remain within the plugin root, use normalized paths, and avoid escaping or dangling symbolic links. This prevents a malformed declaration from accidentally including content outside the plugin.

These shared eng/ changes are therefore necessary to package and validate shepherd-task correctly; they are not unrelated changes to the repository’s plugin infrastructure. The accompanying tests ensure that the new composition behavior remains deterministic and that the root-containment guarantees apply to both directly declared paths and content nested within declared directories.

Website integration

The repository’s Deploy preview website to GitHub Pages workflow passed at PR head cf2993f. This workflow runs the complete production website build, including regeneration of website data from repository plugins and skills and the Astro static-site build.

shepherd-task requires no bespoke website components. Its plugin and six skills are consumed through the website’s existing generated plugin and skill catalogs and rendered through the same generic detail-page routes used by all other repository plugins and skills. The generated plugin index, skill index, and marketplace metadata are included in this PR and pass repository validation.

Because this is a fork-based PR, the workflow intentionally skips publishing a live GitHub Pages preview, but the complete production build succeeds.

See this comment for proof that the local website generation is successful.

Description

This PR adds the shepherd-task plugin, an end-to-end system for running an ordered engineering campaign through GitHub issues and pull requests. It coordinates GitHub Copilot Coding Agent, Copilot code review, local Copilot CLI sessions, GitHub Actions, and gh while keeping GitHub state authoritative.

The campaign lifecycle is divided into explicit stages:

  • 00: initialize durable campaign metadata and lesson state;
  • 10: create an ignorance-reduction plan when implementation issues do not already exist;
  • 15: derive and validate the inputs for issue creation;
  • 20: create and order implementation issues from the resolved plan;
  • 25: dispatch the selected issues serially;
  • 30: assign an issue to Copilot Coding Agent and shepherd its draft PR through CI;
  • 40: request review, resolve findings, publish lessons, and merge to the campaign base branch;
  • 50: create an evidence-based campaign post-mortem.

The plugin includes:

  • six reusable shepherd-task-* skills;
  • Bash 3.2+ and PowerShell 7 orchestration scripts;
  • installers, uninstallers, version-management commands, and a versioned artifact contract;
  • campaign identity, resume, retry, and failure-recovery behavior;
  • optional campaign lesson propagation, defaulting to off;
  • redacted JSONL session artifacts and post-mortem inputs;
  • Linux, macOS, Git Bash, and PowerShell contract fixtures;
  • detailed operating documentation and sequence diagrams.

Task PRs target one non-main campaign base branch and run serially, so each merged task becomes the starting point for the next. The final campaign-base-to-main PR remains a separate human-controlled step.

This PR also adds the repository-side pluginFiles composition field. It allows a plugin to declare plugin-owned runtime files and directories that already live beneath the plugin root, such as scripts, tests, and version tooling. Validation rejects missing, unsorted, malformed, duplicate, or root-escaping references, and materialization strips this repository-only metadata from the served Agent Plugins manifest.

This supersedes the abandoned initial submission in #2329. The implementation has since been rebased onto current main, expanded into a complete staged campaign lifecycle, hardened through treatment/control campaigns, and validated on both Linux and Windows.

Usage notes

The system intentionally invokes copilot --yolo for bounded lifecycle stages. Users must explicitly accept that behavior and provide:

  • authenticated gh and Copilot CLI installations;
  • Copilot Coding Agent and Copilot code review enabled in the target repository;
  • GitHub Actions and Issues;
  • git, jq, uuidgen, and PowerShell 7 for PowerShell entrypoints;
  • a local environment capable of running the repository's real gating tests;
  • a checked-out non-main campaign base branch.

The full setup, installation, lifecycle, recovery, and uninstall procedures are documented in plugins/shepherd-task/README.md.

Validation

Repository validation:

  • npm run plugin:validate
  • npm run skill:validate
  • node --test eng/materialize-plugins.test.mjs eng/validate-plugins.test.mjs
  • npm start
  • bash eng/fix-line-endings.sh
  • git diff --check upstream/main

Cross-platform contract validation:

  • all 23 Bash contract tests passed on Linux after the Windows compatibility fixes;
  • all 32 PowerShell contract tests passed on Windows, including the .cmd mock-based contracts unavailable on Linux;
  • all shepherd-task Bash and PowerShell files passed syntax parsing;
  • shepherd-task JSON contracts parsed successfully;
  • shell executable modes and generated plugin/skill/marketplace outputs were verified.

The existing warnings for unrelated external marketplace entries with mutable source locators or non-SPDX license strings are pre-existing catalog warnings.


Type of Contribution

  • New instruction file.
  • New prompt file.
  • New agent file.
  • New plugin.
  • New skill file.
  • New agentic workflow.
  • New canvas extension.
  • Update to existing instruction, prompt, agent, plugin, skill, workflow, or canvas extension.
  • Other (please specify): repository build and validation support for plugin-owned pluginFiles.

Additional Notes

The contribution is intentionally technical and neutral. It documents Copilot and GitHub prerequisites because they are necessary to operate the workflow; it does not promote an unrelated paid product or direct users to marketing or signup pages.

The large test estate is included because the orchestration spans GitHub state transitions, CI approval, review convergence, cross-platform shell behavior, durable campaign artifacts, versioned installation, retries, and recovery. The fixtures provide regression coverage for those contracts without requiring reviewers to run a live multi-day campaign.

The generated plugin and skill indexes and .github/plugin/marketplace.json have been updated. plugins/external.json is unchanged.

See the full documentation at README.


By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

Copilot AI balanced review requested due to automatic review settings September 23, 2026 22:53
@github-actions github-actions Bot added new-submission PR adds at least one new contribution plugin PR touches plugins skills PR touches skills labels Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 192 changed file(s).

Severity Count
🔴 High 0
🟠 Medium 0
ℹ️ Info 0

✅ No matching risk patterns were detected in changed files.

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Contributor Reputation Check: HIGH risk

Check Risk
Profile HIGH
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor reputation check flagged MEDIUM risk label Sep 23, 2026
@edburns
edburns marked this pull request as draft September 23, 2026 22:54
@github-actions github-actions Bot added the skill-check-error Skill validator reported errors label Sep 23, 2026
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🔍 Vally Lint Results

⛔ Findings need attention

Scope Checked
Skills 6
Agents 0
Total 6
Severity Count
❌ Errors 1
⚠️ Warnings 0
ℹ️ Advisories 0

Summary

Level Finding
❌ shepherd-task-30-from-assignment-to-ready (0/2 checks passed, 2 failed)
Full linter output
### Linting skills/shepherd-task-10-create-ignorance-reduction-plan
✅ shepherd-task-10-create-ignorance-reduction-plan (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

### Linting skills/shepherd-task-20-create-issues-from-plan
✅ shepherd-task-20-create-issues-from-plan (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

### Linting skills/shepherd-task-30-from-assignment-to-ready
❌ shepherd-task-30-from-assignment-to-ready (0/2 checks passed, 2 failed)
    ✗ [spec-compliance] 1 of 1 skill(s) have spec violations.
        ✗ spec-compliance: Spec checks failed.
            ✗ File length (834 lines) exceeds limit (500).
    ✗ [valid-refs] 1 of 1 skill(s) have invalid file references.
        ✗ valid-refs: Found 1 invalid file reference(s): ($changedFilesOutput (missing).

1 skill(s) linted, 1 failed

### Linting skills/shepherd-task-40-from-ready-to-merged-to-base
✅ shepherd-task-40-from-ready-to-merged-to-base (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

### Linting skills/shepherd-task-50-create-post-mortem
✅ shepherd-task-50-create-post-mortem (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

### Linting skills/shepherd-task-approve-workflows-and-wait-for-completion
✅ shepherd-task-approve-workflows-and-wait-for-completion (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

Note: Vally lint returned a non-zero exit code. Please review the findings above before merge.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate issues affect orchestration safety, completion gates, cross-platform behavior, and plugin-file integrity.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 9 High severity · 5 Medium severity · 5 Low severity

Open (19)
What changed in this PR

Adds the shepherd-task plugin for staged GitHub issue and pull-request campaign automation across Bash and PowerShell.

Changes:

  • Adds campaign lifecycle skills, orchestration scripts, recovery tooling, and documentation.
  • Adds cross-platform contract fixtures.
  • Adds validation and materialization support for plugin-owned pluginFiles.
File Description
skills/​shepherd-task-approve-workflows-and-wait-for-completion/​SKILL.md Defines workflow approval and check waiting.
skills/​shepherd-task-20-create-issues-from-plan/​examples/​04-28-python-agent-demo.md Adds a Python issue-generation example.
skills/​shepherd-task-20-create-issues-from-plan/​examples/​03-dd-3017826-java-real-estate-demo.md Adds a Java real-estate example.
skills/​shepherd-task-20-create-issues-from-plan/​examples/​02-1810-java-tool-as-lambda.md Adds a Java Lambda example.
skills/​shepherd-task-20-create-issues-from-plan/​examples/​01-1682-java-tool-ergonomics.md Adds a Java tool ergonomics example.
plugins/​shepherd-task/​test/​simple-math/​README.md Documents the simple-math fixture.
plugins/​shepherd-task/​test/​simple-math/​get-copilot-skill-list.sh Adds a Bash skill-list helper.
plugins/​shepherd-task/​test/​simple-math/​get-copilot-skill-list.ps1 Adds a PowerShell skill-list helper.
plugins/​shepherd-task/​test/​simple-math/​11-stage15-installed-path-contract.sh Tests installed Bash paths.
plugins/​shepherd-task/​test/​simple-math/​11-stage15-installed-path-contract.ps1 Tests installed PowerShell paths.
plugins/​shepherd-task/​test/​simple-math/​10-simple-math-fixture-contract.sh Validates the simple-math fixture.
plugins/​shepherd-task/​test/​simple-math/​09-skill-powershell-contract.sh Checks embedded PowerShell guidance.
plugins/​shepherd-task/​test/​simple-math/​08-psncpps-contract.sh Tests Bash failure propagation.
plugins/​shepherd-task/​test/​simple-math/​08-psncpps-contract.ps1 Tests PowerShell native commands.
plugins/​shepherd-task/​test/​simple-math/​06-stage40-review-contract.sh Tests Bash review behavior.
plugins/​shepherd-task/​test/​simple-math/​06-stage40-review-contract.ps1 Tests PowerShell review behavior.
plugins/​shepherd-task/​test/​simple-math/​03-resolve-repository-remote.sh Tests Bash remote resolution.
plugins/​shepherd-task/​test/​simple-math/​03-resolve-repository-remote.ps1 Tests PowerShell remote resolution.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​get-copilot-skill-list.ps1 Adds a treatment skill-list helper.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​08-psncpps-contract.ps1 Tests treatment native commands.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​07-driver-encoding-contract.ps1 Tests UTF-8 output handling.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​06-stage40-review-contract.sh Tests treatment review behavior.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​06-stage40-review-contract.ps1 Tests treatment review behavior.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​03-resolve-repository-remote.sh Tests treatment Bash remotes.
plugins/​shepherd-task/​test/​simple-math-treatment-control/​03-resolve-repository-remote.ps1 Tests treatment PowerShell remotes.
plugins/​shepherd-task/​test/​macos-bash-compatibility-contract.sh Enforces macOS Bash compatibility.
plugins/​shepherd-task/​test/​lesson-propagation-default-contract.sh Tests lesson-propagation defaults.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​get-copilot-skill-list.sh Adds a Cargo Tracker skill helper.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​get-copilot-skill-list.ps1 Adds a PowerShell skill helper.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​12-session-outcome-contract.sh Tests Bash session outcomes.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​12-session-outcome-contract.ps1 Tests PowerShell session outcomes.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​11-stage15-plan-discovery-contract.sh Tests plan discovery.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​08-psncpps-contract.sh Tests Bash pipeline handling.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​08-psncpps-contract.ps1 Tests PowerShell pipeline handling.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​06-stage40-review-contract.ps1 Tests Stage 40 behavior.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​03-resolve-repository-remote.sh Tests Bash remote resolution.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature/​03-resolve-repository-remote.ps1 Tests PowerShell remote resolution.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​get-copilot-skill-list.ps1 Adds a treatment skill helper.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​13-resume-driver-contract.ps1 Tests preserved-run recovery.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​12-session-outcome-contract.sh Tests treatment session outcomes.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​11-stage15-plan-discovery-contract.sh Tests treatment plan discovery.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​11-stage15-plan-discovery-contract.ps1 Tests PowerShell plan discovery.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​08-psncpps-contract.ps1 Tests native-command handling.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​07-driver-encoding-contract.ps1 Tests driver encoding.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​06-stage40-review-contract.ps1 Tests review behavior.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​03-resolve-repository-remote.sh Tests treatment Bash remotes.
plugins/​shepherd-task/​test/​cargotracker-add-change-arrival-deadline-feature-treatment-control/​03-resolve-repository-remote.ps1 Tests treatment PowerShell remotes.
plugins/​shepherd-task/​shepherd-task-version-contract.json Defines protocol and artifact versions.
plugins/​shepherd-task/​scripts/​validate-stage20-drafts.sh Validates Bash issue drafts.
plugins/​shepherd-task/​scripts/​validate-stage20-drafts.ps1 Validates PowerShell issue drafts.
plugins/​shepherd-task/​scripts/​uninstall-task-shepherd.sh Adds the Bash uninstaller.
plugins/​shepherd-task/​scripts/​uninstall-task-shepherd.ps1 Adds the PowerShell uninstaller.
plugins/​shepherd-task/​scripts/​shepherd-task-inspect-otel-token-summary.sh Summarizes Bash telemetry.
plugins/​shepherd-task/​scripts/​shepherd-task-inspect-otel-token-summary.ps1 Summarizes PowerShell telemetry.
plugins/​shepherd-task/​scripts/​shepherd-task-inspect-json.sh Inspects Bash JSONL logs.
plugins/​shepherd-task/​scripts/​shepherd-task-inspect-json.ps1 Inspects PowerShell JSONL logs.
plugins/​shepherd-task/​scripts/​resolve-repository-remote.sh Resolves Bash Git remotes.
plugins/​shepherd-task/​scripts/​resolve-repository-remote.ps1 Resolves PowerShell Git remotes.
plugins/​shepherd-task/​scripts/​redact-secrets.sh Redacts sensitive JSON fields.
plugins/​shepherd-task/​scripts/​read-shepherd-task-version.sh Reads Bash version contracts.
plugins/​shepherd-task/​scripts/​read-shepherd-task-version.ps1 Reads PowerShell version contracts.
plugins/​shepherd-task/​scripts/​assert-stage20-result.sh Validates Bash Stage 20 results.
plugins/​shepherd-task/​scripts/​assert-stage20-result.ps1 Validates PowerShell Stage 20 results.
plugins/​shepherd-task/​scripts/​assert-shepherd-session-outcome.sh Validates Bash session outcomes.
plugins/​shepherd-task/​scripts/​assert-shepherd-session-outcome.ps1 Validates PowerShell session outcomes.
plugins/​shepherd-task/​plugin.json Defines plugin metadata and runtime files.
plugins/​shepherd-task/​figure-05-post-mortem.md Documents Stage 50.
plugins/​shepherd-task/​figure-04-from-ready-to-merged.md Documents Stage 40.
plugins/​shepherd-task/​figure-03-from-assigned-to-ready.md Documents Stage 30.
plugins/​shepherd-task/​figure-02-shepherd-task.md Documents per-issue orchestration.
plugins/​shepherd-task/​figure-01-shepherd-task-25-given-list.md Documents batch orchestration.
eng/​validate-plugins.test.mjs Tests pluginFiles validation.
eng/​validate-plugins.mjs Integrates pluginFiles validation.
eng/​materialize-plugins.test.mjs Tests plugin materialization.
eng/​materialize-plugins.mjs Supports plugin-local files during materialization.
eng/​lib/​plugin-files.mjs Implements pluginFiles inspection.
docs/​README.skills.md Registers the new skills.
docs/​README.plugins.md Registers the new plugin.
CONTRIBUTING.md Documents pluginFiles.
AGENTS.md Updates plugin contribution guidance.
.github/​plugin/​marketplace.json Adds the marketplace entry.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eng/lib/plugin-files.mjs
Comment thread plugins/shepherd-task/scripts/shepherd-task-25-given-list.ps1
Comment thread plugins/shepherd-task/scripts/shepherd-task-inspect-otel-token-summary.sh Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1 Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1 Outdated
Comment thread plugins/shepherd-task/figure-01-shepherd-task-25-given-list.md Outdated
Comment thread plugins/shepherd-task/figure-02-shepherd-task.md Outdated
Comment thread skills/shepherd-task-10-create-ignorance-reduction-plan/SKILL.md Outdated
Comment thread skills/shepherd-task-30-from-assignment-to-ready/SKILL.md Outdated
Comment thread skills/shepherd-task-40-from-ready-to-merged-to-base/SKILL.md Outdated
@github-actions github-actions Bot removed the skill-check-error Skill validator reported errors label Sep 23, 2026
@edburns
edburns requested a balanced review from Copilot September 24, 2026 02:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate issues affect security, portability, validation, monitoring, and merge safety.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 8 High severity · 6 Medium severity · 2 Low severity

Open (16)
Resolved since last review (19)

Comment thread plugins/shepherd-task/scripts/shepherd-task-15-prepare-create-issues.sh Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task-25-given-list.ps1
Comment thread plugins/shepherd-task/scripts/shepherd-task-monitor.sh Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1
Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1 Outdated
Comment thread skills/shepherd-task-30-from-assignment-to-ready/SKILL.md Outdated
Comment thread skills/shepherd-task-30-from-assignment-to-ready/SKILL.md Outdated
Comment thread plugins/shepherd-task/README.md Outdated
Comment thread plugins/shepherd-task/figure-05-post-mortem.md Outdated
@edburns

edburns commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author
Image Image

@edburns
edburns marked this pull request as ready for review September 24, 2026 16:16
Copilot AI review requested due to automatic review settings September 24, 2026 16:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread plugins/shepherd-task/scripts/assert-stage20-result.ps1 Outdated
Comment thread plugins/shepherd-task/scripts/read-shepherd-task-version.ps1 Outdated
Comment thread plugins/shepherd-task/figure-01-shepherd-task-25-given-list.md Outdated
Copilot AI review requested due to automatic review settings September 24, 2026 17:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Campaign initialization, stage-40 resumption, and Bash issue-body comparison contain correctness defects that can break supported workflows.

Review effort: Balanced
Findings: None

Resolved since last review (6)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Reject campaign branch when it matches the repository default

plugins/​shepherd-task/​scripts/​shepherd-task-00-init-campaign.ps1:67

This validates only the literal branch name main, although the plugin contract requires the campaign branch to differ from the repository's default branch. In a repository whose default is master (or any other name), initialization accepts that branch and the campaign can merge task PRs directly into it, bypassing the intended final human-controlled PR. Query defaultBranchRef for Repo and reject equality before creating campaign state; the Bash entrypoint needs the same correction.

Medium severity Reject campaign branch when it matches the repository default

plugins/​shepherd-task/​scripts/​shepherd-task-00-init-campaign.sh:45

This validates only the literal branch name main, although the plugin contract requires the campaign branch to differ from the repository's default branch. In a repository whose default is master (or any other name), initialization accepts that branch and the campaign can merge task PRs directly into it, bypassing the intended final human-controlled PR. Query defaultBranchRef for REPO and reject equality before creating campaign state; the PowerShell entrypoint needs the same correction.

Medium severity Emit raw jq output for correct file equivalence comparisons

plugins/​shepherd-task/​scripts/​verify-github-issue-body.sh:40

jq is currently emitting a JSON string (for example, "body\\n") rather than normalized raw bytes. Consequently, equivalent_files appends a newline after the closing quote, so its documented one-trailing-newline tolerance can never match a newline difference inside the body. Emit raw output so the subsequent byte comparison and newline adjustment operate on the body itself.

Copilot AI review requested due to automatic review settings September 24, 2026 17:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Failed sessions can expose unredacted telemetry to post-mortem processing, and Stage 30 retries can incorrectly reassign tasks with existing pull requests.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity

Open (2)

Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1 Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.sh Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ee3b3d65-a828-4ded-a757-e5f978303649
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ee3b3d65-a828-4ded-a757-e5f978303649
Move final child count, identity uniqueness, and plan-order checks into a tested platform-specific helper backed by one jq contract. Wire the helper through Stage 15 and require Stage 20 to use it instead of generating ad hoc verification logic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 0776a0ce-2054-4396-b8d0-b768262ebf03
Require generated PowerShell runners to consume the body verifier's object directly with try/catch instead of applying native-command or JSON text semantics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 478065b5-e440-4b9d-b267-a96eb3fb2513
Copilot AI review requested due to automatic review settings September 25, 2026 18:43
@edburns
edburns force-pushed the edburns/dd-3068479-shepherd-task branch from a7192da to 26faee4 Compare September 25, 2026 18:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Resume can repeat Copilot assignment, and unpaginated timeline lookups can miss linked PRs and break idempotent recovery.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 5 Medium severity

Open (5)
Resolved since last review (2)

Comment thread plugins/shepherd-task/scripts/shepherd-task-monitor.ps1 Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task-monitor.sh Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.ps1 Outdated
Comment thread plugins/shepherd-task/scripts/shepherd-task.sh Outdated
Comment thread skills/shepherd-task-30-from-assignment-to-ready/SKILL.md Outdated
@edburns
edburns marked this pull request as draft September 25, 2026 18:51
@edburns

edburns commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

@codemillmatt wrote:

Hey Ed - I've been dealing w/ okta access for the last 2 weeks so I'm just going off what copilot is telling me as I can't see the actual PR. I did run an automated review of it and here are some findings. I think the biggest one is the first one. But I'm kinda flying in the dark here w/o seeing the PR itself. So anyway - does the below seem valid to you?

Finding Impact
The published plugin’s startup breaks. Its version reader requires metadata that the publishing process deliberately removes. The source-checkout installation works differently from the marketplace package; the published version fails before campaign initialization.
Recovery gets stuck after a PR becomes ready for review. Theorchestrator restarts stage 30, which requires a draft PR. An interrupted stage-40 run cannot resume through the documented runner without hitting the draft-only gate.
The independent CI gate accepts pending checks. The checker only looks for failures. The agent instructions say to wait, but the script does not independently enforce the completion guarantee it claims.
The new shared packaging validator mishandles symlinked paths. Itcompares a canonical root against an uncanonicalized target. Valid plugin files are rejected through symlinked ancestors, including macOS temporary-directory paths. This affects shared repository tooling, not just Shepherd.

Add a durable runtime estate manifest for published plugins, migrate Bash and PowerShell consumers, and validate served-layout installation at version 1.0.5.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
@edburns

edburns commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author

@codemillmatt wrote:

The published plugin’s startup breaks. Its version reader requires metadata that the publishing process deliberately removes.

Fixed in

commit 5f726c5 (HEAD -> edburns/dd-3068479-shepherd-task, origin/edburns/dd-3068479-shepherd-task)
Author: Ed Burns
Date: Fri Sep 25 16:38:57 2026 -0400

Fix shepherd-task published estate metadata

Add a durable runtime estate manifest for published plugins, migrate Bash and PowerShell consumers, and validate served-layout installation at version 1.0.5.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf

Route draft pull requests back through stage 30, but resume stage 40 for ready pull requests only when the run directory contains successful stage-30 evidence for the same task and PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Treat only passing, skipped, and the named remove-before-merge failure as acceptable in the independent post-stage gate. Pending, cancelled, failed, and unknown buckets now fail closed in both Bash and PowerShell.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
@edburns

edburns commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

@codemillmatt wrote:

| The independent CI gate accepts pending checks. The checker only looks for failures. | The agent instructions say to wait, but the script does not independently enforce the completion guarantee it claims. |

Fixed in

commit 9448564 (HEAD -> edburns/dd-3068479-shepherd-task, origin/edburns/dd-3068479-shepherd-task)

Reject incomplete shepherd CI checks

Treat only passing, skipped, and the named remove-before-merge failure as acceptable in the independent post-stage gate. Pending, cancelled, failed, and unknown buckets now fail closed in both Bash and PowerShell.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Compare lexical plugin paths against a lexical root before comparing resolved targets against the canonical root. Cover valid file and directory references through a linked ancestor while preserving top-level and nested symlink escape rejection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
@edburns

edburns commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

@codemillmatt wrote:

| The new shared packaging validator mishandles symlinked paths. Itcompares a canonical root against an uncanonicalized target. | Valid plugin files are rejected through symlinked ancestors, including macOS temporary-directory paths. This affects shared repository tooling, not just Shepherd. |

Fixed in

commit ff5ac2e (HEAD -> edburns/dd-3068479-shepherd-task, origin/edburns/dd-3068479-shepherd-task)
Date: Sat Sep 26 18:58:14 2026 -0400

Handle symlinked plugin roots

Compare lexical plugin paths against a lexical root before comparing resolved targets against the canonical root. Cover valid file and directory references through a linked ancestor while preserving top-level and nested symlink escape rejection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf

edburns and others added 4 commits September 26, 2026 19:50
Request the documented maximum timeline page size and follow pagination links when discovering linked pull requests. Require both behaviors in the Bash and PowerShell monitor contracts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Request the maximum issue-timeline page size and follow pagination links for linked-PR discovery. Add Bash and PowerShell contracts covering merged PRs found only through paginated timeline output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
@github-actions github-actions Bot added the skill-check-error Skill validator reported errors label Sep 27, 2026
edburns and others added 4 commits September 27, 2026 09:37
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
Require the exact Bash atomic-write helper and exercise ledger/result initialization before any GitHub mutation. Extend the Stage 20 contract to execute the documented helper under set -u.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 5e90e102-df88-4229-bd33-65808fbc6a9a
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 292755c3-37b5-4a61-8f01-6d2d399b05cf
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

🚦 Submission status: ⏳ Awaiting automation

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 2 approvals from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier
  • eng/lib/plugin-files.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • eng/materialize-plugins.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • eng/materialize-plugins.test.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • eng/validate-plugins.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • eng/validate-plugins.test.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/assert-shepherd-session-outcome.ps1 is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/assert-shepherd-session-outcome.sh is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/assert-stage20-result.ps1 is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/assert-stage20-result.sh is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/install-task-shepherd.ps1 is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/install-task-shepherd.sh is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/read-shepherd-task-version.ps1 is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/read-shepherd-task-version.sh is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/redact-secrets.ps1 is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • plugins/shepherd-task/scripts/redact-secrets.sh is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • …and 149 more

Automated checks

Check Status Details
Line endings ✅ Passed Passed · logs
Spelling ✅ Passed Passed · logs
Generated README consistency ✅ Passed Passed · logs
Plugin and extension validation ✅ Passed Passed · logs
Canvas extension validation ⏳ Pending Waiting for the check to start
Plugin structure ✅ Passed Passed · logs
Skill validation ⏳ Pending Waiting for the check to start
Skill lint (vally) ✅ Passed Passed · logs
Risk scan ✅ Passed Passed · logs
Contributor reputation ✅ Passed Passed · logs
Duplicate resource scan ✅ Passed Passed · logs
PR quality signal ⏭️ Skipped Skipped by its workflow · logs
Canvas/plugin smoke test ⏳ Pending Waiting for the check to start

Review

  • Approvals: 0/2
  • Assigned reviewer: aaronpowell
  • Review target date: not set
  • Still needed: 2 more approval(s); an approval from a maintainer with admin or maintain permission
  • The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command Who What it does
/rerun-checks PR author, maintainers Re-runs failed or incomplete checks and re-evaluates this gate
/request-review PR author, maintainers Asks the review rotation to assign a reviewer (adds needs-reviewer)

Updated for 13c8acb · This comment is maintained automatically — see submission gate docs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-automation merge-risk:high needs-review:HIGH Contributor reputation check flagged HIGH risk new-submission PR adds at least one new contribution plugin PR touches plugins skill-check-error Skill validator reported errors skills PR touches skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants