Skip to content

perf(media): batch .zt JPEG frames into one ffmpeg MJPEG decode - #241

Open
gavindi wants to merge 1 commit into
Lexonight1:mainfrom
gavindi:zt-batch-mjpeg-decode
Open

gavindi wants to merge 1 commit into
Lexonight1:mainfrom
gavindi:zt-batch-mjpeg-decode

Conversation

@gavindi

@gavindi gavindi commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Split out of #229 per review — this half was called clean and self-contained, no dependency on that PR's PlayVideo threading split.

Concatenated JPEGs are a valid MJPEG byte stream, so ZtDecoder.decode() now feeds every frame's JPEG payload into one ffmpeg -f mjpeg -i pipe:0 -vf scale=W:H -f rawvideo -pix_fmt rgb24 pipe:1 call instead of spawning a subprocess per frame (_decode_jpeg_decode_jpegs). Falls back to blank frames (with a warning) if the batch decode fails or under-produces, matching the old per-frame failure behaviour.

  • Byte-parity: synthetic 5-frame .zt fixture decoded via the new batched path vs. the old per-frame logic (reimplemented standalone for comparison) — output bytes identical, frame-to-cursor mapping correct.
  • Speedup: 120-frame synthetic clip, 3.925s (old, per-frame) → 0.039s (new, batched) — 100.5x.

Test plan

  • ruff check src/trcc/services/media.py — clean
  • pyright src/trcc/services/media.py — 0 errors
  • pytest tests/ -k "video or zt or media" — 124 passed

Concatenated JPEGs are a valid MJPEG byte stream, so ZtDecoder.decode()
now feeds every frame's JPEG payload into one `ffmpeg -f mjpeg -i pipe:0
-vf scale=W:H -f rawvideo -pix_fmt rgb24 pipe:1` call instead of spawning
a subprocess per frame (_decode_jpeg -> _decode_jpegs). Falls back to
blank frames (with a warning) if the batch decode fails or under-produces,
matching the old per-frame failure behavior.

Verification:
- Byte-parity: synthetic 5-frame .zt fixture decoded via the new batched
  path vs. the old per-frame logic (reimplemented standalone for
  comparison) — output bytes identical, frame-to-cursor mapping correct.
- Speedup: 120-frame synthetic clip, 3.925s (old, per-frame) -> 0.039s
  (new, batched) — 100.5x.
- Full suite: 2927 tests — 2922 pass, 4 skip, 1 fail
  (test_nvml_init_state_logs_resolved_tuple — pre-existing, unrelated:
  NVML shared library absent in this sandbox).

Split out of Lexonight1#229 per review — self-contained, no dependency on that
PR's PlayVideo threading split.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Lexonight1

Copy link
Copy Markdown
Owner

@gavindi — first, an apology: you split this out of #229 exactly as the review asked, verified byte-parity, measured it, and then got seven weeks of silence. That's on me, not you.

Second, the harder part. Your diagnosis was right — spawning one ffmpeg per frame was the bottleneck, and batching into a single MJPEG stream is the correct fix. In the meantime ZtDecoder changed underneath you in the same direction, but further: it now keeps the JPEG payloads encoded and decodes one per tick through the renderer, so there is no ffmpeg call for .zt frames at all. _decode_payloads and _decode_jpeg are both gone; the only trace left is a comment explaining why.

So this PR is superseded — not because it was wrong, but because you pointed at the right bottleneck and the eventual fix went past it. Your 100x would have been a real improvement on the code as it stood.

I'm leaving it open rather than closing it myself; it's your work and your call. If you want to look at what landed, it's ZtDecoder.decode in services/media.py.

Genuinely sorry for the wait. #229 is still live and I'll come back to it properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants