QWP/UDP compares column names case-sensitively; the server and QWP/WS do not

[jerrinot]

Summary

QuestDB column names are case-insensitive. QWP/WebSocket honours that — it resolves columns through a lowercased key (lookup_column → column_lookup_key → lowercase_name_bytes). QWP/UDP compares column names byte-for-byte.

Consequently a QWP/UDP row can set the same server column twice — including with two different types — and the client returns Ok.

Found while reviewing #203. #203 is not the cause: the column → column route into this already exists on main. #203 removes a state-machine guard that had been blocking the symbol-after-column route by accident, so it adds one more way in.

Reproduction

Two case-variant names, one row:

buffer.table("trades")
      .symbol("Side", "buy")
      .symbol("side", "sell")   // same column per the server
      .at_now();

transport	result
QWP/UDP	Ok — emits two wire columns, Side=buy and side=sell
QWP/WS	Ok — one column, first value kept (Side=buy), "sell" dropped

Worse, with a type change:

buffer.table("trades")
      .column_i64("Qty", 1)
      .symbol("qty", "x")       // same column, different type
      .at_now();

transport	result
QWP/UDP	Ok — emits a row where one column is both a LONG and a SYMBOL
QWP/WS	Err: QWP/WebSocket column "qty" changes type within a batched table

Verified on main for the column→column spelling, so this predates #203:

column_i64("Qty",1) · column_str("qty","x")  ->  ACCEPTED   (already broken on main)
column_i64("Qty",1) · symbol("qty","x")      ->  REJECTED   (state machine blocked it; #203 lifts that)

Root cause

Two byte-exact comparisons in the QWP/UDP path (line numbers as of 84c79f4):

1. QwpBuffer::mark_pending_entry_name — questdb-rs/src/ingress/buffer/qwp.rs:1315, if entry_name == name_bytes — within-row duplicate detection.
2. RowGroupPlanner::find_column — questdb-rs/src/ingress/buffer/qwp.rs:7308, position(|c| &name_bytes[c.name.0.as_range()] == name) — cross-row column identity when planning the datagram.

Both need to fold case the way lowercase_name_bytes (qwp.rs:5232) does, so the two transports agree on when two names are one column.

Fixing (2) also repairs an existing guard: RowGroupPlanner::add_row already raises batched_type_change_error when a column's kind changes, but a byte-exact find_column hides the case-variant clash from it.

Impact

• QWP/UDP can emit a row containing the same server column twice, with conflicting values or conflicting types. Server-side outcome not verified here.
• The two transports silently disagree on the same input, so behaviour depends on which one is configured.
• Plausible without any obvious coding mistake: column names driven by JSON keys, CSV headers, or map iteration are not always consistently cased.

Note for whoever fixes it: there is a performance trap

I prototyped the fix and measured it. The naive version is a large regression on the UDP per-column path, because both call sites sit in per-row scans (find_column runs per entry per row — ~n² per row in the column count) and a case-folding compare is much more expensive than a SIMD memcmp.

Micro-benchmark, 200k rows × 13 columns, Buffer::new_qwp() row-build only, best of 7:

variant	13 same-length names	varied-length names
today (byte-exact)	25.9 Mcell/s	30.7 Mcell/s
naive case-insensitive both sites	13.0	—
is_ascii + eq_ignore_ascii_case	16.0 (−38%)	23.4 (−24%)

Two things I established that should save time:

• find_column is easy. Do an exact memcmp pass first and fall back to a folding pass only when it finds nothing. Rows spell a column the same way every time, so the exact pass answers for every row after the first. Measured 26.8 vs 25.9 Mcell/s — free.
• mark_pending_entry_name is the whole remaining cost. The two-pass trick does not apply: "no duplicate" is the common case, so both passes would always run. Per-comparison is_ascii() calls are what dominate — they exist only to decide whether Unicode folding is needed.

Suggested direction: track "all column names appended so far are ASCII" once on the buffer (set in append_name), and gate a pure-ASCII fast path on it, so the hot path never calls is_ascii() per comparison while non-ASCII names still fold correctly. A per-entry packed lowercase fingerprint (QWP/WS already has packed_lower_ascii_name, qwp.rs:5257) would also work and could beat today's numbers, at the cost of widening EntryMeta.

Open question

Table names look byte-exact on both transports (lookup_or_create_table uses the raw bytes; QWP/UDP groups segments on the table name). If table names are case-insensitive server-side too, that is a second, separate instance of this — but since both transports agree there, it is not a divergence and I have not investigated it.