Skip to content

fix(audit): harden data drains and expand security event coverage - #8778

Merged
waleedlatif1 merged 3 commits into
stagingfrom
codex/audit-drain-reliability
Oct 8, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
codex/audit-drain-reliability

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Checkpoint acknowledged provider requests and drain chunks, fence overlapping workers, resume bounded backlogs, and execute the database queue fallback.
  • Bound source and archive reads, preserve tenant isolation, harden signed S3 transport, sanitize credential errors, and correct Snowflake and Datadog delivery.
  • Add authentication, session, SSO, table-row, and export audit events; preserve deployment audit retries and historical CSV exports.
  • Persist Vitest transforms across runs while retaining test isolation and time limits.

Type of Change

  • Bug fix

Testing

  • 74 PostgreSQL regressions with JSON reports in test-results/integration.json and test-results/data-drains-integration.json.
  • 738 focused unit tests, including the touched and sibling suites.
  • CI runs the complete script and workspace suites.
  • Lint, all-workspace type-checking, 58 repository audits, docs manifest, block registry, migration safety, and schema generation checks.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 8, 2026 3:25am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 55 files

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/data-drains/service.ts
Comment thread apps/sim/lib/data-drains/destinations/s3.ts Outdated
Comment thread apps/sim/lib/core/security/input-validation.server.ts Outdated
Comment thread apps/sim/lib/data-drains/sources/copilot-chats.ts
Comment thread apps/sim/lib/data-drains/destinations/datadog.ts
Comment thread apps/sim/lib/data-drains/enqueue.ts Outdated
Comment thread apps/sim/lib/auth/lifecycle-audit.ts
@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Refactors audit logging and data export infrastructure across auth and drain systems.

The PR appears safe to merge; no new actionable defect was established.

What we checked:

  • Stale workers cannot save progress: checkpoint locks both records and refuses the write when the run is no longer running or the saved cursor changed.
  • Historical exports keep organization scope: includeDeparted returns the organization predicate unchanged. It only removes the current-actor restriction.
  • Missing rows do not count: deleteRow throws when the database returns no deleted row. The use case never reaches its audit projection.

Summary

This PR adds acknowledged delivery checkpoints, bounded drain runs, queue continuations, and a working database fallback. It also strengthens S3 transport and expands audit coverage.

  • CSV exports share one formatter for escaping and byte limits.
  • Datadog saves each accepted request’s locator before sending another request.
  • Authentication, session, SSO, table-row, and export actions gain audit events.
  • Deployment audit writes are awaited and safe to retry.

Both earlier, unnumbered findings are fixed in the current code. Tests were inspected but not run during this review.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Scheduled or manual drain] --> B[Enqueue job]
  B --> C[Trigger.dev worker]
  B --> D[Database fallback]
  C --> E[Claim drain]
  D --> E
  E --> F[Read bounded page]
  F --> G[Deliver bounded chunk]
  G --> H[Save acknowledged cursor and locator]
  H --> F
  H --> I[Finish bounded run]
  I --> J{More rows remain?}
  J -->|Yes| B
  J -->|No| K[Done]
Loading

Reviews (3) · Last reviewed commit: "fix(audit): bound CSV allocation and dis..." · Reviewed by Greptile

Comment thread apps/sim/lib/audit-logs/application/export-audit-logs.ts Outdated
Comment thread apps/sim/lib/data-drains/destinations/datadog.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 61 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/audit-logs/application/export-audit-logs.ts Outdated
Comment thread apps/sim/lib/data-drains/destinations/datadog.ts Outdated
@waleedlatif1
waleedlatif1 force-pushed the codex/audit-drain-reliability branch from c8293ab to a6cc8a8 Compare October 8, 2026 03:19
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 61 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 59e749e into staging Oct 8, 2026
48 checks passed
@waleedlatif1
waleedlatif1 deleted the codex/audit-drain-reliability branch October 8, 2026 03:30

This branch was successfully deployed

1 active deployment
Preview — a6cc8a8b Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant