feat(hang): define segment-addressed recordings - #2574
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ed2fe6e74
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
kixelated
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8dec4b58d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ddaf604060
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
moq-archive specified a generic, arrival-ordered chunk format at the moq-lite layer, with its own per-track index mapping group runs to byte ranges. It predated the timeline rework, which already publishes a per-segment index of group ranges per track. Addressing a recording by segment instead of by flush interval collapses the format: the segment number names the object, so the timeline is the only index and the separate index log, chunk numbering, byte offsets, partial spans, and overlap resolution all disappear. It also gives a reader one whole-object GET per segment per track, which byte-range splicing across chunks could not. That makes the format a hang concept rather than a moq-lite one, since segments are defined by the timeline and the timeline is hang. Specify it in the hang draft and delete the standalone draft, which was never submitted to the datatracker. Segment objects carry their own group boundaries so they parse without the timeline, and frames stay byte-identical to moq-lite FRAME so a recorded group is a FETCH response body unchanged. Known trade: a group arriving after its segment object is written is not recorded. Writers wait for completeness and bound that wait. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Cxx2ebpiGmWaD8AvJkRdi
The segmenter required every enrolled track to vote on boundaries and to report past a segment's end before its record could flush. That works for audio and video, whose groups arrive continuously, and stalls the timeline permanently for anything else: boundary() returns None the moment an enrolled track has no group past the threshold, and a catalog publishes a group only when the renditions change. So a broadcast could only segment tracks it could also pace, which left catalogs, metadata, and application tracks like a chat log with no way to be indexed at all. Add Producer::passive alongside Producer::track. A passive track's groups are recorded into whichever segment is open when they arrive, but it never votes on a boundary, never gates completeness, and never anchors the first segment (a catalog published while the encoder warms up would otherwise stretch segment 0 across the whole startup gap). It also does not create the timeline track, so segmentation stays opt-in by pacing track. Placement is by arrival rather than content time, since nothing waits for these groups: one that shows up after its segment flushed is recorded in the next. The frames still carry their own timestamps. A passive track whose group never closes is recorded once, in the segment its group opened in. Rolling the group at segment boundaries is the fix and is left to the publisher. Recording drops its special case for the catalog as a result: every track the timeline lists is stored the same way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Cxx2ebpiGmWaD8AvJkRdi
Persist immutable track metadata alongside per-track segment objects, and specify archive and duration-bounded DVR retention with the recording timeline as the authoritative durable index. Fix passive timeline tracks to drain by arrival and exclude their timestamps from final media duration. Add matching Rust and TypeScript regression coverage. Co-Authored-By: OpenAI Codex <noreply@openai.com>
Enroll catalog.json as a passive timeline track and report each published snapshot group so durable recordings include the catalog needed to interpret media segments. Specify uppercase percent escapes and require recording-owned timeline re-encoding so atomic gaps do not corrupt the shared DEFLATE dictionary. Co-Authored-By: OpenAI Codex <noreply@openai.com>
Make ordinary timeline track enrollment non-pacing so sparse catalog or metadata tracks cannot accidentally stall segment publication. Add explicit pacing_track and pacingTrack methods for continuous media that votes on boundaries and gates completeness. BREAKING CHANGE: Rust timeline Producer::track is now infallible and non-pacing, and TypeScript Producer.track is now non-pacing. Use pacing_track or pacingTrack for media tracks that drive segmentation. Co-Authored-By: OpenAI Codex <noreply@openai.com>
Specify an immutable .complete object written only after the final recording state is durable. Clarify that the application owns the prefix and distinguish archive caching from active DVR retention. Keep the catalog timeline's shared recorder lint-clean on wasm after rebasing onto the current producer state model. Co-Authored-By: GPT-5.6 Codex <noreply@openai.com>
ddaf604 to
5132f5c
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5132f5c226
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| self.hang.update(catalog)?; | ||
| self.catalog_timeline.lock().unwrap().record(&self.hang_track); |
There was a problem hiding this comment.
When the last reservation flush races with an update through another cloned Producer, the two Outputs::emit calls can overlap: one call can publish catalog group 0, the other can publish group 1, and both can then observe hang_track.latest() == 1. Group 0 is consequently never reported to the timeline, so a recording can omit the catalog that was in effect for an intervening media segment. Serialize the catalog update and timeline report across all Outputs clones rather than discovering the sequence afterward with latest(). (Written by GPT-5.6 Sol)
Useful? React with 👍 / 👎.
| ~~~ | ||
|
|
||
| The fields have the meanings and encodings defined by moq-lite `TRACK_INFO` {{moql}}. | ||
| The source broadcast's epoch and `Publisher Max Latency` are not stored because the recording is a new generation with its own epoch and retention policy. |
There was a problem hiding this comment.
The current moq-lite TRACK_INFO definition calls this property Publisher Max Age; Publisher Max Latency was renamed and no longer exists. Referring to the obsolete name here makes it unclear which property recording readers should reconstruct from their serving policy and can carry the old latency semantics into implementations. Rename this reference to Publisher Max Age. (Written by GPT-5.6 Sol)
Useful? React with 👍 / 👎.
2 participants
Summary
The root causes were four format and implementation gaps. Recorded frames lacked the immutable track properties needed to reconstruct TRACK_INFO. Sparse tracks were timestamp-drained as if they paced media. The production catalog bypassed timeline enrollment. A clean end was promised without a durable representation.
Public API changes
This is a semantic break to the existing low-level timeline API, so the dev target remains appropriate. High-level catalog and media producers select the correct mode internally.
Test plan
The Rust and TypeScript timeline implementations and regression tests remain in sync. No other cross-package sync row applies.
(written by GPT-5.6 Codex)