Skip to content

fix(search): refuse overlapping Search retirement runs with a session lock - #8524

Closed
waleedlatif1 wants to merge 1 commit into
stagingfrom
fix/search-retirement-run-lock
Closed

waleedlatif1 wants to merge 1 commit into
stagingfrom
fix/search-retirement-run-lock

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

This follows up on #8522 and addresses its Greptile finding, "Plain runs lack the documented lock".

The operator command's plain retirement run took no lock. The runbook says it holds a session advisory lock. Two overlapping runs could not corrupt progress, because every page locks the progress row, but they would double the load on the primary.

Retirement now takes a session try-lock (search-embedding-retirement), the same pattern maintenance already uses (0028). It refuses to start while another run holds the lock, and it releases the lock in finally.

Behaviour changes

  • A second concurrent retirement run fails at once with "Search retirement is already running" and touches nothing.
  • A single run is unchanged.

Test plan

  • New real-Postgres test: another session holds the lock, the run rejects and creates no cleanup tables, then a run without the competing lock completes. The test failed before the fix ("promise resolved instead of rejecting").
  • The lock-retry test now has the suite's 60 s timeout. Under the gentler default pacing, its lock wait (~1.5 s) is followed by a pause twice as long, which overran the default 5 s timeout when the whole suite ran in parallel. The full packages/db integration suite then passed 129/129 three times in a row.
  • Lint, the packages/db type-check and bun run check:audits pass.

… lock

A plain operator run took no lock, so two runs could double the load on the primary. Retirement
now takes a session try-lock, as maintenance already does, and refuses to start while another run
holds it.
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 1, 2026 5:38pm UTC

Request Review

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Adds session lock to prevent concurrent search retirement runs.

No new actionable issue prevents merging under this review.

Summary

The PR adds a session advisory lock to plain Search embedding retirement, tests rejection when another session holds it, extends one integration-test timeout, and updates the runbook.

Reviews (2) · Last reviewed commit: "fix(search): refuse overlapping Search r..."

Comment thread packages/db/script-migrations/0027_retire_search_embeddings.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread packages/db/script-migrations/0027_retire_search_embeddings.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

This branch was successfully deployed

1 active deployment
Preview — 401cdeec Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant