fix(db): keep residual clauses residual when combining WHERE clauses by Jaime02 · Pull Request #1977 · TanStack/db

Jaime02 · 2026-10-01T13:10:57Z

Changes

Outer joins could make the optimizer push the same filter into a source on each pass. The source gained repeated filters until the ten-pass limit. This PR keeps pushed predicates in their source once and retains their outer residual marker.

Jaime02 reported the repeated-filter case in #1977 after work on #1976. The original patch fixed the common case. A query with an existing residual clause still left three outer WHERE groups. This follow-up combines regular clauses and residual clauses separately. It also removes the second residual-reassembly step.

The loss audit found another error in the same WHERE grouping path. A filter that touched no source, such as where(() => new Value(false)), disappeared from a joined query. The live-query Collection then published rows even though the filter was false. The optimizer now keeps every clause without one eligible source in the outer WHERE.

Evidence and limits

The optimizer oracle checks predicate values, counts, residual markers, outer clause groups, and the extracted source filters. It checks the first optimizer result and a later pass. Its cases cover LEFT, RIGHT, INNER, and FULL joins, namespace predicates, source-free predicates, later RIGHT/FULL joins, and an ordered QueryRef that declines pushdown.

The source-free case also checks public rows. Before the fix, its first snapshot contained three rows instead of none. An empty-source-filter mutant and a predicate-value-blind recorder both fail at their intended oracle checks. The review record lists the checked boundaries and open paths.

The production diff has nine fewer lines than origin/main. The full packages/db/tests suite passed: 197 files and 7,675 tests. Package type checking, ESLint, Prettier, and the diff check passed. The suite used two Vitest threads and a 120-second test timeout.

Checklist

I tested this change locally with the full packages/db/tests suite.

Release impact

This change affects published code and includes a changeset.
This change affects only docs, CI, or development files.

Summary by CodeRabbit

Bug Fixes
- Fixed query optimization for filters used with outer joins. Filters now remain in the appropriate part of the query across repeated optimization passes, preserving correct results when conditions apply to both sides of a join.
- Improved predicate handling across LEFT, RIGHT, INNER, and FULL joins, including combined conditions and multi-join queries.
- Preserved source-free and malformed or unqualified predicates in the main query instead of dropping them. Single-source filters can still be pushed into source subqueries.

applyOptimizations combined the remaining WHERE clauses of an outer-join query into one plain AND, dropping the residual marker on predicates that were already pushed into a subquery. The next optimization pass treated them as new predicates and pushed them again, so optimizeQuery only stopped at its iteration limit and left duplicated predicates in the subquery. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-10-01T13:12:17Z

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 423e2108-3cd0-4a7e-a8f0-17e969826de8

📥 Commits

Reviewing files that changed from the base of the PR and between 0da9cb9 and 31502a2.

📒 Files selected for processing (6)

.changeset/optimizer-residual-convergence.md
docs/contributing/oracle-coverage.md
docs/contributing/oracle-reviews/2026-10-01-pr-1977-residual-convergence.md
packages/db/src/query/optimizer.ts
packages/db/tests/query/optimizer-semantics-oracle.test.ts
packages/db/tests/query/optimizer.test.ts

🚧 Files skipped from review as they are similar to previous changes (2)

.changeset/optimizer-residual-convergence.md
docs/contributing/oracle-coverage.md

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

Walkthrough

The optimizer now retains source-free predicates and separates regular WHERE clauses from residual clauses. Structural tests and live-query checks cover predicate placement across join types and repeated optimization.

Changes

Optimizer residual handling

Layer / File(s)	Summary
Retain and combine WHERE clauses `packages/db/src/query/optimizer.ts`, `packages/db/tests/query/optimizer.test.ts`, `.changeset/optimizer-residual-convergence.md`	The optimizer retains clauses that touch no sources and combines regular and residual clauses separately. Unit tests expect source-free, unqualified, and malformed predicates to remain in the main WHERE clause.
Verify predicate placement across optimization passes `packages/db/tests/query/optimizer-semantics-oracle.test.ts`	Structural tests check predicate placement and residual markers across join types and predicate forms. They also check repeated optimization and ordered `QueryRef` cases that decline pushdown.
Check public join results and record oracle scope `packages/db/tests/query/optimizer-semantics-oracle.test.ts`, `docs/contributing/oracle-coverage.md`, `docs/contributing/oracle-reviews/2026-10-01-pr-1977-residual-convergence.md`	Live-query tests compare LEFT, RIGHT, and chained-join snapshots with independently modeled rows. The coverage map and review record document test results and coverage limits.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested reviewers: kyleamathews

Merge Risk: ⚪ Minimal · up to 31502

The change fixes optimizer non-convergence for outer-join queries with multiple WHERE clauses and keeps source-free predicates that were previously dropped. No merge-blocking risk was found. The author reports that the one remaining suite failure also occurs on main.

Architecture Summary

Architecture risk: 🟡 Medium · up to 0da9c

The change affects 2 systems.

Changed systems: packages/db, docs

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

observed — packages/db (library) was modified; 3 changed files map to changed impact.
observed — docs (service) was modified; 2 changed files map to changed impact.

Before / after behavior

observed — Modified behavior in packages/db/tests/query/optimizer.test.ts: The test file now imports createResidualWhere from the query IR module.
observed — Modified behavior in packages/db/tests/query/optimizer.test.ts: Adds a test asserting that with a LEFT JOIN and both member and team filters, the team filter is pushed into the active-side source subquery, while the member filter and createResidualWhere(teamFilter) remain in the main query.
observed — Modified behavior in docs/contributing/oracle-reviews/2026-10-01-pr-1977-residual-convergence.md: Adds the review baseline and candidate identifiers and hashes, then states the pushdown and public-row contracts and the finite structural and array-model oracle coverage, including its checkpoints and assertions.
observed — Modified behavior in docs/contributing/oracle-reviews/2026-10-01-pr-1977-residual-convergence.md: Records the expanded oracle results for the original optimizer, supplied patch, candidate repair, and wrong-design controls; also distinguishes production-change line counts from test additions.

Reliability and maintainability

inferred — Risk-relevant change factors for packages/db: blast_radius_1; direct_dependents_1

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 3 files. (3 skipped: 3…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main optimizer fix: preserving residual clauses when combining WHERE clauses.
Description check	✅ Passed	The description explains the motivation, implementation, evidence, test results, checklist status, and release impact. It includes a changeset and local test validation. The heading uses lowercase in …

Full details: Docstring Coverage

Explanation

Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 3 files. (3 skipped: 3 unsupported.)

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

# Conflicts: # docs/contributing/oracle-coverage.md

pkg-pr-new · 2026-10-01T17:05:15Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1977

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1977

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1977

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1977

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1977

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1977

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1977

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1977

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1977

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1977

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1977

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1977

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1977

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1977

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1977

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1977

@tanstack/react-router-with-db

npm i https://pkg.pr.new/@tanstack/react-router-with-db@1977

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1977

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1977

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1977

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1977

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1977

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1977

commit: 31502a2

Jaime02 mentioned this pull request Oct 1, 2026

fix(db): preserve source identity in optimizer copies #1976

Closed

2 of 3 tasks

Isaac added 3 commits October 1, 2026 10:47

Merge origin/main into fix/optimizer-residual-convergence

1ca8284

fix(db): keep residual predicates stable across passes

4102398

Merge origin/main after source-identity fix

0da9cb9

# Conflicts: # docs/contributing/oracle-coverage.md

Isaac added 2 commits October 1, 2026 12:34

fix(db): preserve source-free join filters

92490bc

docs: record optimizer loss-audit evidence

31502a2

KyleAMathews merged commit 1dbf5f9 into TanStack:main Oct 1, 2026
12 checks passed

github-actions Bot mentioned this pull request Oct 1, 2026

ci: Version Packages #1971

Merged

Navigation Menu

fix(db): keep residual clauses residual when combining WHERE clauses - #1977

fix(db): keep residual clauses residual when combining WHERE clauses#1977
KyleAMathews merged 6 commits into
TanStack:mainfrom
Jaime02:fix/optimizer-residual-convergence

Jaime02 commented Oct 1, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 1, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

pkg-pr-new Bot commented Oct 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Jaime02 commented Oct 1, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Evidence and limits

Checklist

Release impact

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Architecture Summary

❌ Failed checks (1 warning)

Uh oh!

pkg-pr-new Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jaime02 commented Oct 1, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 1, 2026 •

edited

Loading

pkg-pr-new Bot commented Oct 1, 2026 •

edited

Loading