Ingest runs as a configurable sequence of processing steps, and each step currently writes its own preservation event, so a single SIP produces one event per step. There should be an option to consolidate this into a single preservation event per ingest, covering whichever steps actually ran, with enough detail to tell whether each one succeeded, failed, or was skipped, along with the agents involved. A SIP update goes through the same ingest process, so it should behave the same way.
This should be opt-in and off by default, must not affect ingests that already happened, and must not change how the same processing steps behave when run on their own outside of an ingest.
Acceptance Criteria
Notes
Every ingest step is implemented as its own plugin, and each one currently writes its own event independently — there's no single place today that already sees everything that happened across an entire ingest and could just be changed once. Delivering this means changing each step's own event-writing individually, not flipping one central switch.
The single event's structure needs to work within what a PREMIS event can actually represent and what the existing event-handling code can already provide, added to without breaking it — this is new, additive behavior, not a replacement of how events are built today for anything that isn't using this option.
Possible Approach (reasoned starting point, still needs validation on the specifics)
The mechanism every plugin already uses to write a preservation event (PluginHelper.createPluginEvent) is used far beyond ingest, so it shouldn't be changed directly — doing so would put every one of its callers at risk for the sake of one feature. Instead:
- A new, off-by-default option, set by the ingest workflow on its own parameters. It reaches every step automatically through the parameter-merging already done today when a plugin runs as part of an ingest — no new propagation code needed.
- A new method is added on
PluginHelper, alongside the existing one, reusing the same underlying logic (agent resolution, outcome messaging). When the option is off, it simply delegates to the existing method, unchanged. When the option is on, it captures a snapshot of what would have been written — into a small, purpose-built registry, grouped by job and by the specific object the snapshot belongs to — instead of writing it.
- Each step plugin is migrated to the new method individually, one at a time — a small, mechanical change per plugin, not a coordinated rewrite, since the existing method is never touched and keeps working for anything not yet migrated. During a partial rollout, a migrated step contributes to the one consolidated event, while a step not yet migrated keeps writing its own individual event alongside it — an expected interim state on the way to full coverage, not a bug.
- The decision to actually write the one consolidated event is made explicitly by the ingest workflow, at the exact point in its own existing loop where it already writes today's per-object "ingest end" event — never guessed at by the writing mechanism itself. At that point, for that one object, it reads back what was captured, builds one real event from it, writes it, and clears that object's entry immediately.
- When the option is off, or for a step plugin run on its own outside any ingest, nothing changes anywhere — the option is never set on that plugin's parameters in the first place.
What's still open, and should be validated against the code rather than assumed: the exact shape of the registry and its concurrency handling, and precisely how an abandoned entry gets cleaned up if an ingest fails partway through.
Ingest runs as a configurable sequence of processing steps, and each step currently writes its own preservation event, so a single SIP produces one event per step. There should be an option to consolidate this into a single preservation event per ingest, covering whichever steps actually ran, with enough detail to tell whether each one succeeded, failed, or was skipped, along with the agents involved. A SIP update goes through the same ingest process, so it should behave the same way.
This should be opt-in and off by default, must not affect ingests that already happened, and must not change how the same processing steps behave when run on their own outside of an ingest.
Acceptance Criteria
Notes
Every ingest step is implemented as its own plugin, and each one currently writes its own event independently — there's no single place today that already sees everything that happened across an entire ingest and could just be changed once. Delivering this means changing each step's own event-writing individually, not flipping one central switch.
The single event's structure needs to work within what a PREMIS event can actually represent and what the existing event-handling code can already provide, added to without breaking it — this is new, additive behavior, not a replacement of how events are built today for anything that isn't using this option.
Possible Approach (reasoned starting point, still needs validation on the specifics)
The mechanism every plugin already uses to write a preservation event (
PluginHelper.createPluginEvent) is used far beyond ingest, so it shouldn't be changed directly — doing so would put every one of its callers at risk for the sake of one feature. Instead:PluginHelper, alongside the existing one, reusing the same underlying logic (agent resolution, outcome messaging). When the option is off, it simply delegates to the existing method, unchanged. When the option is on, it captures a snapshot of what would have been written — into a small, purpose-built registry, grouped by job and by the specific object the snapshot belongs to — instead of writing it.What's still open, and should be validated against the code rather than assumed: the exact shape of the registry and its concurrency handling, and precisely how an abandoned entry gets cleaned up if an ingest fails partway through.