Enterprise 24.0 files: page-type byte is lowercase, and the AP model's per-block `bv` / per-sector `step` do not hold

[pete-green]

I've been testing opensqlany against a QuickBooks Desktop Enterprise Solutions: Contractor Edition 24.0 company file (4096-byte pages, 7,634 pages, engine string 2182 SAP SE, Copyright (c)2015 17.0.4., CRC-32 valid). Everything below is from byte-level analysis of a file we lawfully own, with an independently written reader — no SAP or Intuit tooling, SDK, ODBC or exported data was used to produce or check any of it.

I can't attach the file: it contains real customer PII and payroll. Happy to produce synthetic reductions for anything you want to reproduce.

Three findings. The first is a one-line fix; the other two are corrections to the de-obfuscation model, with the evidence that contradicts it.

1. PageType::from_byte only accepts uppercase, and this file uses lowercase

page.rs classifies the trailer byte with

b'E' => PageType::Extent,
...
other => PageType::Other(other),

On this file the extent/data pages carry 'e' (0x65), not 'E' (0x45). Every one of them therefore lands in PageType::Other(0x65), and any consumer filtering on PageType::Extent — for example openqbw's systable.rs, which does if page.trailer().page_type() != PageType::Extent { continue } — skips the entire data population silently. No error, no warning; the file simply looks nearly empty.

I'd suggest normalising case at classification time rather than adding a second arm per type, since the same question will arise for the other letters:

pub fn from_byte(b: u8) -> PageType {
    match b.to_ascii_uppercase() {
        b'E' => PageType::Extent,
        // ... unchanged
        _ => PageType::Other(b),   // keep the ORIGINAL byte here
    }
}

Keeping the original byte in Other(_) matters: the case distinction may itself be meaningful, and discarding it would hide the variant from anyone looking later. If you'd rather treat lowercase as a distinct kind than fold it, that also works for us — the important part is that it stops being invisible.

2. bv is not constant per 16-page block on this file

ap.rs documents and implements

base(pn, si) = (bv(pn / 16) + pn + si - 4 * ((pn % 16) / 2)) mod 256

with "bv(bi) — a per-16-page-block calibration byte" and ApModel storing "at most one u8 per 16-page block".

That doesn't hold here. Across 2,920 pages that decode to >30% zeros, 394 of 439 blocks show 6–8 distinct per-page bv values. Block 4, for example, yields {61, 63, 65, 67, 77, 79, 81, 83}.

This is, I think, the direct cause of the low learn rate — with one slot per block, most blocks can't be satisfied by any single value. What works on this file is a per-page oracle: for each page independently, choose the bv that makes plain[0] == 0. That's what our reader uses throughout and it holds across the whole file.

I'd suggest keying learned bv by page rather than by block, or at minimum falling back to per-page recovery when a block's candidates disagree.

3. step looks like a deterministic global keystream, not a per-sector empirical value

ap.rs recovers step per sector by histogram peak ("the candidate step value that produces the highest histogram peak"). On this file step has visible global structure instead: values come in pairs, and the sequence shifts by one sector per page.

page 44 steps = (159,159,109,109, 67, 67,145,145)
page 45 steps = (233,159,159,109,109, 67, 67,145)   <- page44[:7], shifted right
page 46 steps = (109,109, 67, 67,145,145, 71, 71)

Measured over the file:

• the pairwise-repeat structure holds on 2,398 / 2,399 trusted pages;
• steps(pn+1)[1:] == steps(pn)[:7] on 723 / 1,443 adjacent trusted page pairs.

I want to be straight about the limits of this one, because it's a partial result. We could not close the index formula — the per-block sector-pair index appears permuted, and we didn't find the permutation. We also tried borrowing a step sequence from a trusted neighbouring page to decode dense pages, and it failed: 0 of 40. So this is evidence that the current model is the wrong shape, not a finished replacement for it, and it does not explain the slot-directory counts.

If the keystream is genuinely global, deriving it once would be both faster and more robust than per-sector histogram recovery — but someone would need to find the index permutation first.

Provenance

Clean-room: byte-level analysis only, single file, independent reader. Findings 2 and 3 are statistical over that one file, so I'd treat the exact counts as indicative and the structure as the claim. If it's useful I can re-run any of these measurements with different thresholds, or check a specific page range you care about.