Skip to content

fix(sync): now gracefully skip oversized records instead of blocking - #4155

Open
Dilyxs wants to merge 1 commit into
atuinsh:mainfrom
Dilyxs:fix_command_very_large
Open

Dilyxs wants to merge 1 commit into
atuinsh:mainfrom
Dilyxs:fix_command_very_large

Conversation

@Dilyxs

@Dilyxs Dilyxs commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #2887
follow-up to #4051(failed PR)

What's currently going on

when a user has a record that is larger than the max record size, the user CANNOT sync they are blocked. Firstly, is it even desirable to have very large records?

Solution

As the issue pointed out, most users wouldn't want to save records that exceptionally large, thus we introduce a config that lets users choose a max size for records (for now set to 1MB), in this way, large records wouldn't even make their way to the db to sync.

What about large records that are already in?

One solution would be to either truncate or delete the large records, but what if the user switches server? Their data would have been deleted to a server which they don't even use anymore. Thus, for those records, the users sends an empty Record, the sync would work and when other clients try to sync, they simply skip over it(a warning log is emitted in both cases).

Tests

tests have been added to verify this sync behavior and to verify that the max_record_length works as expected.

Checks

  • I am happy for maintainers to push small adjustments to this PR, to speed up the review cycle
  • I have checked that there are no existing pull requests for the same thing

@Dilyxs
Dilyxs force-pushed the fix_command_very_large branch 4 times, most recently from 8f96ff1 to d2a0e76 Compare September 16, 2026 00:47
@Dilyxs
Dilyxs marked this pull request as ready for review September 16, 2026 01:04
@strix-security

strix-security Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 1 commit after the last Strix review (0de1615). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

1 open security finding on this PR:

Review summary

Reviewed PR #4155, which adds a client-side max_record_length save filter and a server-advertised max_record_size capability, and changes the sync upload path to replace oversized records with empty "tombstone" payloads. The one confirmed issue — "Sync client trusts server-controlled max_record_size to irreversibly replace user history with empty tombstones" — concerns the client making a destructive, irreversible data decision based on an unauthenticated server-supplied value, with the tombstone propagating silently to other devices. The remaining changes (the local save filter, the new capability type, server advertisement, tests, and documentation) were reviewed and raised no additional security concerns.

Fixed the findings? re-run the review, or tag @strix-security in a PR comment to run a fresh review.

Updated for 0de1615.


Reviewed by Strix
Re-run review · Configure security review settings

@greptile-apps

greptile-apps Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds configurable history-length filtering and server capability negotiation intended to prevent oversized records from blocking synchronization.

  • Filters newly captured commands using max_record_length.
  • Advertises the server’s encrypted-record size limit.
  • Replaces oversized upload payloads with encrypted empty payloads.

Confidence Score: 4/5

The PR is not safe to merge until upload decisions preserve the server’s actual size-limit semantics.

The new clamp treats an unlimited server as having a 1 MiB limit and ignores legitimate limits below 1 MiB, causing either unnecessary empty replacements or recurring whole-batch rejection. The previous empty-payload-overhead finding is otherwise addressed.

Files Needing Attention: crates/atuin-client/src/record/sync/mod.rs

Important Files Changed

Filename Overview
crates/atuin-client/src/record/sync/mod.rs Adds oversized-record replacement, but the capability clamp breaks zero and sub-1-MiB server limits.
crates/atuin-client/src/history.rs Applies the configured command-length limit through the shared history acceptance rule.
crates/atuin-server/src/router.rs Advertises the configured server record-size limit as a capability.
crates/atuin-server/tests/sync.rs Tests very large records but misses records between the configured server limit and the client floor, plus unlimited mode.

Reviews (2): Last reviewed commit: "Update crates/atuin-client/src/record/sy..." | Re-trigger Greptile

Comment thread crates/atuin-client/src/record/sync/mod.rs Outdated

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strix flagged a new security finding below. See the pinned summary comment for the full PR status.

Comment thread crates/atuin-client/src/record/sync/mod.rs Outdated
Comment thread crates/atuin-client/src/record/sync/mod.rs
Comment thread crates/atuin-client/src/record/sync/mod.rs
@Dilyxs
Dilyxs force-pushed the fix_command_very_large branch 2 times, most recently from 4da57c2 to 73b6586 Compare September 16, 2026 02:08
@Dilyxs Dilyxs changed the title fix: sync to now gracefully skip oversized records instead of blocking fix(sync): now gracefully skip oversized records instead of blocking Sep 16, 2026
Comment thread crates/atuin-client/src/record/sync/mod.rs Outdated
@Dilyxs
Dilyxs force-pushed the fix_command_very_large branch from 73b6586 to 0e1290e Compare September 16, 2026 03:09
@Dilyxs
Dilyxs force-pushed the fix_command_very_large branch from 50f3019 to 92ba6ac Compare September 16, 2026 13:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Sync: Content Too Large (HTTP 413)

1 participant