Skip to content

test: .zip data file uploads are stored as ZIP (DataSpaceBackend #206) - #136

Merged
saqibmanan merged 1 commit into
CIfrom
test-sync/DataSpaceBackend-pr206
Sep 25, 2026
Merged

saqibmanan merged 1 commit into
CIfrom
test-sync/DataSpaceBackend-pr206

Conversation

@saqibmanan

Copy link
Copy Markdown
Contributor

Source: CivicDataLab/DataSpaceBackend#206 (merge 92ed2b7e, merged to dev 2026-09-24). Base: CI.

The merge's own dev deploy failed provider-smoke and its rollback job also failed. The later dev deploy of 8850b1d6, which contains #206, succeeded, so dev is running this change.

What changed (from the diff)

createFileResources accepts a file only when its extension and its sniffed MIME type map to the same format (file_validation, api/utils/file_utils.py). The sniff used magic.from_buffer, and the server's libmagic (5.46) reports a zip buffer as application/octet-stream. That maps to no format, so every .zip upload failed with Unsupported file format. #206 sniffs the stored file with magic.from_file when it has a path, which reports application/zip → ZIP.

What this adds

tests/api/smoke/test_api_011_zip_upload.py (api + smoke):

  • test_zip_datafile_is_accepted_as_zip: create a draft dataset, upload a small real .zip through the same GraphQL multipart request the dataset editor sends, and assert no errors and fileDetails.format == "ZIP".
  • test_zip_bytes_named_csv_are_rejected: a control. The same zip bytes named .csv must still be refused as Unsupported file format. This shows the first test passes because zips are recognised, not because validation stopped running. It passed before #206 too.

Each test creates one draft dataset and deletes it in teardown. Not readonly: it writes, so the prod gate (suite: readonly) never selects it.

Why these categories: a backend upload-validation fix is API behaviour, so api. It's smoke so it actually runs: api-smoke selects -m "smoke", and a regression-only test would never run on any event. No UI test: the dataset editor just passes the backend's error through.

Proof (local, against dev)

Collected under CI's filter:

$ pytest tests/api/smoke -m smoke --collect-only -q
tests/api/smoke/test_api_011_zip_upload.py::test_zip_datafile_is_accepted_as_zip
tests/api/smoke/test_api_011_zip_upload.py::test_zip_bytes_named_csv_are_rejected
41/70 tests collected (29 deselected)

Green:

test_zip_datafile_is_accepted_as_zip PASSED
test_zip_bytes_named_csv_are_rejected PASSED
2 passed in 2.28s

Red (assertions flipped: expect CSV; expect the rejection to be absent):

E  AssertionError: Expected uploaded .zip to be stored with format 'ZIP', got 'ZIP'
E  assert 'ZIP' == 'CSV'
E  AssertionError: Expected zip bytes named .csv to be rejected ... got errors=[{'message': 'Unsupported file format.', ... 'path': ['createFileResources']}]
2 failed in 1.95s

Reverted → 2 passed in 1.92s.

Gaps

  • Not run against the pre-#206 backend. That needs a local backend with the server's libmagic 5.46; the misidentification is version-specific, so a local libmagic proves nothing.
  • The merge-SHA deploy's Rollback (smoke tests failed) job itself failed (run 36008274802). A later successful deploy covered it, but a failing rollback is worth a separate look.

@saqibmanan
saqibmanan marked this pull request as ready for review September 25, 2026 12:29
@saqibmanan
saqibmanan merged commit 6bd5edd into CI Sep 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant