Repository navigation
Restore eth_getLogs range reduction after alloy migration - #6729
Draft
datanexus-vincent wants to merge 1 commit into
Draft
datanexus-vincent wants to merge 1 commit into
datanexus-vincent wants to merge 1 commit into
Conversation
log_stream shrinks the block range when a provider rejects eth_getLogs as too heavy. It recognised that by substring-matching the rendered error, including "ServerError(-32005)" and "ServerError(-32000)", which are rust-web3's Debug rendering. Since the alloy migration in v0.42.0 those strings never appear, so a provider enforcing a range or result cap got ten identical retries, "Unexpected RPC error", and a block stream restart loop instead of a smaller range. Classify the alloy error structurally instead: - A known cap message shrinks the range at once, without retrying. Caps are matched by message because most providers report them under a generic code such as -32602. - An ambiguous code (-32000, -32002, -32003, -32005) or a bare HTTP 503 is retried first and shrinks the range only if it keeps failing. These also cover transient failures such as a lagging node or a rate limit, which the old list shrank on immediately. - Anything else is retried and surfaced, as before. Cap messages are also read from HTTP 400/403/413/503 bodies and from 200 bodies alloy could not deserialize, since the transport hides the JSON-RPC error in both cases.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
log_streamshrinks theeth_getLogsblock range when a provider rejects a request as too heavy. It detected that by substring-matching the rendered error against a list that includedServerError(-32005)andServerError(-32000). Those are rust-web3'sDebugrendering of the error code.Since the alloy migration in v0.42.0 the error is an alloy
RpcError, which renders aserror code -32005: ...orErrorResp(ErrorPayload { code: -32005, .. }). The code fingerprints never match. A provider with a range or result cap belowGRAPH_ETHEREUM_MAX_BLOCK_RANGE_SIZEnow gets ten identical retries,Unexpected RPC error, and a block stream restart, and the subgraph never advances. Besu with--rpc-max-logs-rangeis the simplest reproduction.Change
The substring list is replaced by a structural classification of the alloy error with three outcomes:
Caps are matched by message, not code. geth and reth report a cap under -32602 and Alchemy under -32600, which they also use for genuinely malformed requests, so the code alone cannot be trusted.
Cap messages are also read from HTTP 400, 403, 413 and 503 bodies, and from 200 bodies alloy failed to deserialize. graph-node's HTTP transport turns any non-2xx response into an
HttpErrorcarrying the raw body, so a provider that pairs its cap with an error status was otherwise invisible. Ankr does this with a 413.Covered, each with its verbatim message in the tests: geth, reth, Besu, Alchemy, Infura, QuickNode, Ankr, zkSync Era, Monad.
Behaviour changes to be aware of
header not foundfrom a lagging node, Infura's rate limit, and a gateway with no backend. They are now retried first and shrunk only if the failure persists.The reduction policy itself, dividing the step by ten down to a single block, is untouched.
Testing
Unit tests cover every message fragment in isolation, the verbatim provider messages under their real codes, the ambiguous codes, unrelated errors, cap messages inside HTTP and undeserializable bodies, and statuses whose bodies must not be read.
cargo test -p graph-chain-ethereum,cargo clippy -p graph-chain-ethereum --all-targetsandcargo check -p graph-chain-ethereum --releasepass. The provider messages come from client sources and provider documentation, not from live requests.