Skip to content

feat(rc-multipart): list and abort incomplete uploads#269

Closed
cxymds wants to merge 1 commit into
cxymds/issue-1362-output-v3from
cxymds/issue-1366-multipart
Closed

feat(rc-multipart): list and abort incomplete uploads#269
cxymds wants to merge 1 commit into
cxymds/issue-1362-output-v3from
cxymds/issue-1366-multipart

Conversation

@cxymds

@cxymds cxymds commented Jul 21, 2026

Copy link
Copy Markdown
Member

Related work

Background

RustFS 1.0.0-beta.10 exposes S3 multipart list and abort operations, but its ListMultipartUploads implementation interprets prefix as an exact object key. Bucket-wide, prefix, and recursive cleanup therefore cannot be implemented safely until rustfs/backlog#1384 is resolved.

This PR delivers the safe exact-key subset now and returns a typed unsupported-feature result before alias or network resolution for the affected modes. It intentionally does not close rustfs/backlog#1366 because recursive acceptance remains blocked by the server issue.

Solution

  • Add typed multipart upload records, paginated list options/results, identities, and abort requests to the rc-core object-store boundary.
  • Preserve source compatibility for external and mock ObjectStore implementations by providing default UnsupportedFeature bodies for the new trait methods.
  • Implement ListMultipartUploads and AbortMultipartUpload in rc-s3, including metadata mapping, retries, idempotent exact 404 NoSuchUpload handling, and defensive pagination validation.
  • Support rc ls alias/bucket/key --incomplete for one exact object key with deterministic human and output-v3 JSON results.
  • Support rc rm alias/bucket/key --incomplete for exact-key abort, dry-run planning, bounded concurrency, retry, deterministic ordering, and complete per-upload success/failure retention.
  • Reject bucket, prefix, and recursive incomplete-upload modes with a typed unsupported-feature result linked to rustfs/backlog#1384.
  • Emit output-v3 multipart listing, cleanup, typed error, and partial envelopes; partial cleanup retains both successful and failed upload records.
  • Inherit the explicit-stack async command launcher from feat(cli)!: add runtime discovery and replication workflows #272; this PR has no crates/cli/src/main.rs delta relative to its base.
  • Make the local S3 subprocess test server explicitly switch accepted sockets to blocking mode for cross-platform reliability.

RustFS beta.10 compatibility and safety

  • A truncated response is accepted only with a complete next-key/next-upload-id marker pair.
  • A fallback marker is synthesized only when both markers are absent and the final upload provides a safe advancing pair.
  • Partial marker pairs, empty unsafe fallbacks, cycles, and stalled pagination are rejected.
  • HTTP 401/403 is classified as authentication failure before any service-code handling.
  • Idempotence requires HTTP 404 plus the exact structured NoSuchUpload code; generic 404s and message substrings are not treated as success.
  • Exact-key CLI filtering prevents a nonconforming server response from widening cleanup to similarly prefixed objects.

Compatibility and migration

BREAKING contract process: this PR updates protected command-reference documentation and extends the output-v3 schema introduced by #266. It does not modify schemas/output_v1.json or schemas/output_v2.json, and it does not change their runtime contracts. Multipart commands added here explicitly emit v3 envelopes.

The dependency chain is main -> #272 -> #266 -> #269; the command-launcher fix remains isolated in #272 and the output-v3 foundation remains isolated in #266.

Validation

Static restack checks:

  • stable multipart patch-id preserved: 16de49fa3049b3cb05161b00c66232e06af1178f
  • range-diff shows only removal of the obsolete local main.rs launcher hunk
  • no crates/cli/src/main.rs delta relative to the feat(rc-output): introduce output schema v3 #266 base
  • no changes to output v1/v2 schemas
  • git diff --check passes

Final isolated gates in /tmp/rc-target-1366-final with CARGO_INCREMENTAL=0 and CARGO_BUILD_JOBS=1:

  • cargo fmt --all --check
  • cargo clippy --workspace -- -D warnings
  • cargo test --workspace
  • cargo test -p rustfs-cli --test multipart_incomplete (9 passed)
  • cargo test -p rustfs-cli --test output_schema_v3 (6 passed)
  • INSTA_UPDATE=no cargo test -p rustfs-cli --features golden --test golden (5 passed)
  • jq empty for all schemas and JSON fixtures
  • protected-files check with the required BREAKING marker

All local checks pass.

Final review resolution (2026-07-21)

  • Corrected service-error precedence so a structured NotImplemented response remains UnsupportedFeature even when RustFS returns HTTP 404.
  • Kept authentication first and exact 404 NoSuchUpload abort idempotence ahead of generic classification.
  • Added focused list/abort regression tests for the mixed HTTP-status/service-code responses.
  • Re-ran cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, multipart (9 passed), output-v3 (6 passed), and golden (5 passed) gates in /tmp/rc-target-1366-rereview; all pass.

@cxymds
cxymds force-pushed the cxymds/issue-1366-multipart branch from a29c77b to efec262 Compare July 21, 2026 05:57
@cxymds
cxymds changed the base branch from main to cxymds/issue-1362-output-v3 July 21, 2026 05:59
@cxymds
cxymds force-pushed the cxymds/issue-1366-multipart branch from efec262 to 9ec7923 Compare July 21, 2026 06:00
@cxymds
cxymds changed the base branch from cxymds/issue-1362-output-v3 to main July 21, 2026 06:01
@cxymds
cxymds force-pushed the cxymds/issue-1366-multipart branch from 9ec7923 to 8be3b56 Compare July 21, 2026 06:01
@cxymds
cxymds changed the base branch from main to cxymds/issue-1362-output-v3 July 21, 2026 06:01
@cxymds
cxymds force-pushed the cxymds/issue-1362-output-v3 branch from a50ab58 to 10d3f0b Compare July 21, 2026 06:51
@cxymds
cxymds changed the base branch from cxymds/issue-1362-output-v3 to main July 21, 2026 07:23
@cxymds
cxymds force-pushed the cxymds/issue-1366-multipart branch from 8be3b56 to 658f58d Compare July 21, 2026 07:23
@cxymds
cxymds changed the base branch from main to cxymds/issue-1362-output-v3 July 21, 2026 07:23
@cxymds
cxymds force-pushed the cxymds/issue-1366-multipart branch from 658f58d to 266541d Compare July 21, 2026 08:30
@cxymds
cxymds changed the base branch from cxymds/issue-1362-output-v3 to main July 21, 2026 08:31
@cxymds
cxymds changed the base branch from main to cxymds/issue-1362-output-v3 July 21, 2026 08:33
@cxymds
cxymds changed the base branch from cxymds/issue-1362-output-v3 to main July 21, 2026 08:33
@cxymds cxymds closed this Jul 21, 2026
@cxymds cxymds reopened this Jul 21, 2026
@cxymds
cxymds changed the base branch from main to cxymds/issue-1362-output-v3 July 21, 2026 08:33
@cxymds
cxymds deleted the branch cxymds/issue-1362-output-v3 July 22, 2026 13:02
@cxymds cxymds closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant