[branch-55] Preserve field metadata when coercing INT96 timestamps (#24790)

[dwsmith1983]

Which issue does this PR close?

• Backport of Preserve field metadata when coercing INT96 timestamps #24790 to branch-55, for the 55.2.0 patch release (Release DataFusion 55.2.0 (patch) Release (Oct 2026) #25758).
• Closes Int96Coercer drops the metadata of struct, list and map fields. #24786 on branch-55.

Rationale for this change

With datafusion.execution.parquet.coerce_int96 set, a file with an INT96 column comes back with the metadata of every struct, list and map field emptied, while leaf fields keep theirs. Readers that match fields by Parquet field id lose the id of any container whose id sits only on the container. Apache DataFusion Comet always enables the coercion, so with Spark's field id reads on it null fills such a container where Spark reads it (apache/datafusion-comet#6131).

What changes are included in this PR?

A clean cherry-pick of #24790 (with -x): the three container constructors in schema_coercion.rs keep the original field's metadata, plus that PR's tests.

What is the testing strategy for this PR?

The tests from #24790 come with the cherry-pick. The datafusion-datasource-parquet crate's tests, cargo fmt and cargo clippy pass on this branch.

Are there any user-facing changes?

Struct, list and map fields read from files with INT96 columns keep their field metadata when coerce_int96 is set. No API changes.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.21%. Comparing base (c1514b5) to head (303d1a9).

Additional details and impacted files

@@             Coverage Diff             @@
##           branch-55   #25895    +/-   ##
===========================================
  Coverage      81.21%   81.21%            
===========================================
  Files           1110     1110            
  Lines         388506   388606   +100     
  Branches      388506   388606   +100     
===========================================
+ Hits          315527   315611    +84     
- Misses         54430    54448    +18     
+ Partials       18549    18547     -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[comphead]

Thanks @dwsmith1983 this backport makes sense to me. Would you mind adding more information how critical is not having this fix in the patchset? For backports we usually trying to merge the critical things.

[dwsmith1983]

Would you mind adding more information how critical is not having this fix in the patchset?

Without it the query returns wrong results and raises no error. With coerce_int96 on, struct, list and map fields read from a file that has an INT96 column lose their metadata. A reader that matches columns by Parquet field id then no longer finds those containers and fills them with nulls. Comet enables the coercion for every scan, so with Spark's field id reads on, where s.inner.x = 1 and where s is not null return no rows and count(s) is 0 where Spark reads the data (apache/datafusion-comet#6405). Until DataFusion carries this, Comet's stopgap (apache/datafusion-comet#6444) sends every scan that asks for an id on a nested field back to Spark.

The fix is three .with_metadata(...) calls in schema_coercion.rs. Files without an INT96 column are unaffected.

[dwsmith1983]

@alamb could this go into 55.2.0? It is a clean cherry-pick of #24790, which keeps field ids on struct, list and map fields through INT96 coercion. Comet needs it to match Parquet field ids on those fields, and is adding a fallback to Spark for those reads until it ships.

[alamb]

Tahnks @dwsmith1983 -- looks good to me