René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"reader: ScanIterator.compareValues swallows ClassCastException, silently disabling zone-map pruning on type-mismatched filters","articleBody":"## Summary\n\n`ScanIterator.compareValues(Object a, Object b)` catches `ClassCastException` and returns `0`. Because the `canPruneChunk` arms treat `compareValues(...) \u003c 0` / `\u003e 0` as the prune condition, a `0` means **\"cannot prune\"** — so a filter whose value type does not match the stored zone-map stat type silently prunes **nothing** and the whole column is decoded. No error, no log: a correctness-safe but performance-silent failure.\n\n## Where\n\n`reader/src/main/java/io/github/dfa1/vortex/reader/ScanIterator.java`\n\n```java\nprivate static int compareValues(Object a, Object b) {\n    try {\n        return ((Comparable\u003cObject\u003e) a).compareTo(b);\n    } catch (ClassCastException _) {\n        return 0;   // \u003c-- mismatch =\u003e 0 =\u003e canPruneChunk yields false =\u003e no pruning\n    }\n}\n```\n\nInteger zone stats decode as `Long` (`ArrayStats.decodeScalar` returns `int64_value()`), and floats as `Double`. So a caller that builds a `RowFilter` with an `Integer` (or `Float`) value against an integer (or f32) column gets **zero pruning** even though the predicate is perfectly valid — the comparator just throws `ClassCastException` internally and is swallowed.\n\n## How it surfaced\n\nBuilding the Calcite adapter's filter push-down (`feat/vortex-calcite-demo`), a `WHERE date BETWEEN ...` on an `I32` column pruned **0 of 100** chunks until the literal was coerced from `Integer` to `Long`. The filter, expansion, and `canPruneChunk` path were all correct — only the boxed type differed, and the swallow hid it.\n\n## Impact\n\n- Silent performance cliff: a valid, selective filter degrades to a full scan with no signal.\n- Easy to hit from any caller that boxes filter values at the column's \"natural\" width (`Integer` for `I32`, `Float` for `F32`) rather than the stat's storage width (`Long` / `Double`).\n\n## Suggested fix (pick one)\n\n1. **Normalise numeric comparison** in `compareValues` — if both args are `Number` but different boxed types, compare via `Long`/`Double` (or `BigDecimal`) instead of returning `0`. Most robust; makes pruning width-agnostic.\n2. **Coerce on `RowFilter` construction** — normalise integer values to `Long` and float values to `Double` when a `RowFilter` is built, matching the stat storage types.\n3. At minimum, **do not return `0` on mismatch** — a genuine type mismatch (e.g. comparing a string filter to a numeric column) should be a clear error or a logged \"pruning skipped\", not a silent no-op.\n\nOption 1 is preferred: it fixes every caller, not just the careful ones.\n\n## Notes\n\nThe Calcite adapter worked around this by coercing integer literals to `Long` (commit on `feat/vortex-calcite-demo`), so it is unblocked — but the underlying reader trap should be fixed so the next caller does not rediscover it.","author":{"url":"https://github.com/dfa1","@type":"Person","name":"dfa1"},"datePublished":"2026-06-24T19:59:33.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/159/vortex-java/issues/159"}