Skip to content

Simpler API for submitting SIPs for ingest, with configurable webhook in parameter, whitelisted #3751

Description

@006627

Submitting a SIP through the API currently takes two calls: upload the file, then start a job to ingest it. This proposal adds a single call that does both, plus a webhook parameter, building on the notification-by-email/file/HTTP capability the ingest plugin already has.

The webhook parameter only accepts a URL that's on an admin-configured list of allowed destinations and that doesn't resolve to an internal address. This whitelist requirement applies only to this new endpoint's webhook parameter; the existing way of setting a job completion callback keeps working exactly as it does today, unchanged.

Acceptance Criteria

  • A single API call accepts a SIP file and starts ingest, without a separate upload step first.
  • That call accepts a webhook URL as a normal, documented parameter.
  • RODA calls the given webhook URL when the ingest job finishes.
  • A webhook URL is only accepted if it's on an admin-configured list of allowed destinations, and only if it doesn't resolve to an internal/private address; a submission with a disallowed or unsafe URL is rejected immediately, not accepted and silently failed later.
  • The endpoint can be called by an external system using an API access token, without a browser session.

Notes

Authentication needs no new work: RODA's personal access tokens already authenticate any API call, including a new one, without a browser session.

Whether a submitted SIP still needs manual review or is accepted automatically depends on the parameters sent with the request, same as it does today for a job created the regular way.

Open Questions

  • Should the webhook call require an acknowledgment from the receiver to be considered delivered, or is a successful response to the call enough, as it is today?
  • Should large SIPs be uploadable in chunks through this new endpoint, or is a single upload (no size limit, no resume) good enough for now?

Out of Scope

  • Uploading large files in chunks (see open questions).
  • Any change to the existing upload/ingest screens in the web interface.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions