fix(table): align Data Evolution row-range pruning with Java - #628
fix(table): align Data Evolution row-range pruning with Java#628jerry-024 wants to merge 10 commits into
Conversation
…fest * upstream/main: fix(table): skip unrelated BLOB and vector files in row-range reads (apache#629) feat(datafusion): support database SQL statements (apache#627) fix(datafusion): apply dynamic options to vector search (apache#623) feat(io): let FileIO wrap a caller-provided filesystem operator (apache#618) feat(c): build a catalog-free Table from a resolved schema JSON (apache#595) feat: support batch incremental diff reads (apache#510) fix(spec): accept Java's long memory-size units and reject overflow (apache#621) fix(table): stop answering COUNT(*) from a placeholder row count (apache#624) feat(datafusion): support alter column (apache#625) fix(spec): validate bucket-key against table schema (apache#620) # Conflicts: # crates/paimon/src/table/data_evolution_reader.rs # crates/paimon/src/table/table_scan.rs
|
[Blocking] Null-fill partially uncovered ranges for nullable providers
For example, consider a normal anchor covering This makes the nullable missing-provider behavior inconsistent: adding a covered row turns a valid NULL-filled query into an error. Please make BLOB and vector sources emit NULL/placeholders for uncovered selected ranges, rather than NULL-filling only when the entire provider is absent. Please also add end-to-end tests covering mixed provider-covered and provider-gap ranges for both BLOB and vector columns. The existing |
Summary
Align Data Evolution row-range manifest pruning with Java two-stage direct intersection. This avoids retaining unrelated manifests and data files solely because their ranges belong to an overlapping transitive component, while preserving Java-compatible missing-provider semantics.
Changes
Testing
Notes
No planner memory option, temporary spill files, iterative candidate expansion, or additional scan trace fields are introduced.