Skip to content

Add reversible predict-only (1/3) wavelet via Part 2 ATK signaling - #344

Open
hmaarrfk wants to merge 3 commits into
aous72:masterfrom
hmaarrfk:rev13-wavelet
Open

Add reversible predict-only (1/3) wavelet via Part 2 ATK signaling#344
hmaarrfk wants to merge 3 commits into
aous72:masterfrom
hmaarrfk:rev13-wavelet

Conversation

@hmaarrfk

@hmaarrfk hmaarrfk commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Thank you for the discussion on the approach. This implements the 1/3 wavelet as a custom ATK, and reads the ATK to decode the image correctly.

Notably, we do NOT build a whole API to define the ATK, instead we define a OpenJPH specific wavelet that we can reference by name. Please let me know if you agree or disagree with this direction.

It seems to really help and your structure is really powerful that I think the changes are quite contained.

The comments are likely overly verbose, let me know if you want me to really trim them, but I think I mostly understand the changes.

Closes #261

Claude longer description

What this adds

A reversible predict-only ("1/3") wavelet: the 5/3 kernel with its update step nulled, signaled with a JPEG 2000 Part 2 ATK marker segment (index 2, Rsiz |= 0x8000 | 0x20). With this kernel, the LL band of every decomposition holds the even-indexed samples of the previous resolution untouched, so decoding with r skipped resolutions returns exactly image[::2^r, ::2^r] — no filtering, no values absent from the original — while full-resolution decoding remains lossless. The intended use is AI-generated label/mask images (microscopy), where interpolated "in-between" values are illegal.

  • param_atk::init_rev13() — the kernel. It keeps a null update step (A=0, B=0, E=0) rather than dropping it: with m_init = 0 the first synthesis step operates on the even-indexed subsequence, so a predict-only kernel needs an even number of steps (a literal 1-step kernel corrupts reconstruction).
  • param_atk::write() — ATK marker segment serialization (reversible, whole-sample symmetric, single-coefficient steps; byte layout mirrors read() and was cross-checked against a Kakadu-generated ATK stream).
  • Public param_cod::set_wavelet_kern() / get_wavelet_kern(); write_headers() emits the ATK segment for kernels ≥ 2.
  • param_qcd::make_quant_steps() and param_cap Ccap now route through is_reversible() so ATK-referenced kernels get correct QCD/CAP values (previously QCD was silently left uninitialized for wavelet_trans >= 2).
  • param_cod::is_predict_only() — detects exact-subsampling kernels by inspecting the ATK lifting steps, never the kernel index (which is file-local in Part 2; your test suite's simple_dec_irv53_bhvhb_low_latency.jph uses index 2 for an irreversible 5/3, and correctly reports false).

What did not change

The decode path. Your existing generic ATK machinery handles this kernel as-is — we verified that stock OpenJPH (≥ 0.30) decodes these codestreams bit-exactly at every resolution level, so only encoders need this patch.

Testing

New self-contained library-API tests in tests/test_rev13.cpp (in-memory encode/decode, no reference codestreams needed, modeled on test_mixed_coc.cpp):

  • full-res losslessness + image[::2^r, ::2^r] exactness at every level, for even/odd dimensions including single-row/column images;
  • ATK marker presence and step-based kernel detection (is_predict_only), with Part 1 kernels asserting the negative;
  • a handcrafted 8x8 image with a single impulse at odd coordinates (1, 1): the exact subsample is all zeros, and the test asserts the 5/3 update step leaks the impulse into resolution level 1 while the predict-only kernel returns exactly the all-zero subsample — both lossless at full resolution.

All 90 ctest tests pass (including the Kakadu ATK/DFS decode tests); shared+static builds.

  • On 4096×4096 masks this kernel also compresses ~27% better than rev53 (no update-step smearing), ~13–20 ms encode / ~11 ms decode per image.

Happy to split, rename (e.g. the rev13 name), add tests in your preferred style, or adjust the API surface (e.g. decoupling the public kernel enum from the ATK index) — whatever makes review easiest.

…rker

<details><summary>Claude's draft</summary>

Implements the "1/3 wavelet" discussed in issue aous72#261: the reversible 5/3
kernel with its update step nulled, so the low-pass subband of every
decomposition holds the even-indexed samples of the previous resolution
untouched. Decoding resolution level r of a losslessly-coded image yields
exactly image[::2^r, ::2^r] -- no filtering, no values absent from the
original -- while full-resolution decoding remains lossless. Intended for
label/mask images where intermediate values are illegal.

Encoder additions:
- param_atk::init_rev13(): 2-step kernel (null update A=0,B=0,E=0 +
  5/3 predict), ATK index 2. The null update is kept, rather than the
  step dropped, because with m_init = 0 the first synthesis step operates
  on the even-indexed subsequence, so a predict-only kernel needs an even
  number of steps.
- param_atk::write(): ATK marker segment serialization (reversible,
  whole-sample symmetric, single-coefficient kernels).
- public param_cod::set_wavelet_kern() / get_wavelet_kern(); write_headers
  emits the ATK segment and sets Rsiz (0x8000 | 0x20).
- param_qcd::make_quant_steps and param_cap Ccap now route through
  is_reversible() so ATK-referenced kernels get correct QCD/CAP values
  (previously QCD was left uninitialized for wavelet_trans >= 2).

Decode side is unchanged -- the existing generic ATK machinery already
handles this kernel, so stock OpenJPH >= 0.30 decodes these codestreams
bit-exactly (verified). New param_cod::is_predict_only() inspects the ATK
lifting steps (never the file-local kernel index) so readers can reliably
detect exact-subsampling codestreams from any encoder; verified against a
Kakadu stream that uses ATK index 2 for a different kernel.

All 86 ctest tests pass, including the Kakadu ATK/DFS decode tests.

Resume this Claude session:
```
cd /home/mark/git/jp2k
claude --resume 487c1e20-36f3-416f-8be1-437c25ed67fe
```
</details>
<details><summary>Claude's draft</summary>

Adds tests/test_rev13.cpp (self-contained gtest, in-memory encode/decode,
no reference codestreams needed), following the test_mixed_coc.cpp model:

- LevelsAreExactSubsampling: full-resolution decode is lossless and every
  skipped-resolution decode equals image[::2^r, ::2^r], across even/odd
  dimensions including single-row and single-column images.
- SignalingAndDetection: the main header carries an ATK marker segment,
  and a reader classifies the kernel from the ATK lifting steps
  (is_predict_only, is_reversible); Part 1 kernels classify as not
  predict-only and carry no ATK segment.
- ImpulseAtOddPositionRev53LeaksRev13DoesNot: a handcrafted 8x8 image,
  zero except one impulse at (1, 1). The exact subsample image[::2, ::2]
  is all zeros; the test asserts the 5/3 kernel's update step leaks the
  impulse into its first resolution level (not the exact subsample) while
  the predict-only kernel returns exactly the all-zero subsample -- with
  both kernels lossless at full resolution.
- Rev53RemainsLossless: guards the QCD/CAP code paths this feature
  reworked.

All 90 ctest tests pass.

Resume this Claude session:
```
cd /home/mark/git/jp2k
claude --resume 487c1e20-36f3-416f-8be1-437c25ed67fe
```
</details>
@hmaarrfk hmaarrfk changed the title Add reversible predict-only (1/3) wavelet via Part 2 ATK signaling (issue #261) Add reversible predict-only (1/3) wavelet via Part 2 ATK signaling Aug 8, 2026
@hmaarrfk
hmaarrfk marked this pull request as ready for review August 8, 2026 12:24
…ernel

<details><summary>Claude's draft</summary>

Three conformance fixes found while checking this work against the
freely-available 2002 edition of T.801:

- T.801 Table A.27 sizes the Batk parameter by the coefficient type
  Coeff_Typ, but param_atk::read() consumed a fixed 16-bit field for all
  coefficient types (and the new write() mirrored that). Both now size
  Batk by Coeff_Typ. This matters for reading conformant streams that
  use 8-bit reversible kernel parameters.
- T.801 Table A.2: Rsiz bit 5 (0x20) signals arbitrary decomposition
  styles and bit 7 (0x80) signals whole-sample symmetric kernels;
  param_siz::read() had the two assignments swapped, and the rev13
  encoder used 0x20 where 0x80 is correct. Verified against a
  Kakadu-produced stream (Rsiz = 0xC0A0 for DFS + WS-ATK usage).
- init_rev13 now uses 16-bit kernel coefficients (Coeff_Typ = 1,
  Satk = 0x5902, Latk = 17). With Coeff_Typ = 0 the previously written
  segment was 2 bytes longer than T.801 specifies (the Batk issue
  above); with Coeff_Typ = 1 the conformant layout is identical to what
  already-deployed OpenJPH releases parse, so the codestreams remain
  decodable by stock decoders (verified against the published ojph
  0.9.1 wheels, which bundle stock OpenJPH).

All 90 ctest tests pass.

Resume this Claude session:
```
cd /home/mark/git/jp2k
claude --resume 487c1e20-36f3-416f-8be1-437c25ed67fe
```
</details>
@hmaarrfk

hmaarrfk commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Pushed a conformance commit after checking the implementation against the freely-available 2002 edition of T.801:

  • Batk sizing (Table A.27): Batk is sized by Coeff_Typ, but param_atk::read() consumed a fixed 16-bit field for all coefficient types; read()/write() now size it correctly.
  • Rsiz bits (Table A.2): bit 5 (0x20) = arbitrary decomposition styles, bit 7 (0x80) = whole-sample symmetric kernels — param_siz::read() had the two assignments swapped, and this PR's encoder previously set 0x20 where 0x80 is correct (cross-checked against a Kakadu stream: Rsiz=0xC0A0 for DFS + WS-ATK usage).
  • rev13 kernel now uses 16-bit coefficients (Coeff_Typ=1, Satk=0x5902): with Coeff_Typ=0 the previously written segment was 2 bytes longer than T.801 specifies; with Coeff_Typ=1 the conformant layout is byte-identical to what already-released OpenJPH versions parse, so the streams remain decodable by stock decoders (re-verified against the published ojph 0.9.1 wheels bundling stock OpenJPH).

All 90 ctest tests pass.

@aous72

aous72 commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Hi Mark,

Thank you for PR and all the effort you put in.

This requires careful attention.
As you may have noticed, every change has consequences.
We also want a minimal set of changes.

I need to find the time to think about it.

More importantly, was this useful for your purpose? and how much is the improvement?

Kind regards,
Aous

@hmaarrfk

hmaarrfk commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

More importantly, was this useful for your purpose?

It was! I added a test to "demonstrates" it by reading the lower levels and ensuring they are "bit exact" at the subsampled locations.

and how much is the improvement?

I actually ended up taking it a step further for my own work and implemented (with the help of AI of course) the rev12 transform.
hmaarrfk#3

But the amount of changes there are insane, and I really don't to ask you to read through them.

┌──────────────────────┬──────────┬──────────┬─────────┬─────────┐
│        kernel        │ 6 levels │ 8 levels │ encode  │ decode  │
├──────────────────────┼──────────┼──────────┼─────────┼─────────┤
│ rev53 (standard 5/3) │ 300 kB   │ 278 kB   │ 14.0 ms │ 12.1 ms │
├──────────────────────┼──────────┼──────────┼─────────┼─────────┤
│ rev13 (1/3, PR #344) │ 218 kB   │ 194 kB   │ 14.1 ms │ 12.3 ms │
├──────────────────────┼──────────┼──────────┼─────────┼─────────┤
│ rev12 (1/2, new)     │ 151 kB   │ 126 kB   │ 13.1 ms │ 11.1 ms │
└──────────────────────┴──────────┴──────────┴─────────┴─────────┘

I need to find the time to think about it.

Take all the time you need.

I basically had to create my own ojph library since i knew you had told me that it would take you time. I did want to "give back"

PS. hmaarrfk#3 shows that you can get rid of much of your assembly written code for libhwy. Overall, my fork has about 2x the performance (barring me finding any bugs today) just because I sent AI loose on finding optimizations.

PPS. hmaarrfk#3 thinks that T.801 conformance fixes found against the spec (Batk sized by Coeff_Typ; Rsiz bits 0x20/0x40/0x80 corrected). I haven't had time to look into it. But I think it found it when i asked it to build out an 16 bit codepath in addition to the 32bit and 64bit codepaths you have.

Although I did use AI, i have read through this code and think that it is in the spirit of the rest of the codebase. It is "minimal" since the infrastructure to read through ATKs is already there, and mostly just adds the "write" path for this kind of tranform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request R1X1 "wavelet transform"

2 participants