Fix mws_l1b_nc reader not scaling and masking properly - #3449
Conversation
Also lowercases sensor name to 'mws'
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3449 +/- ##
==========================================
- Coverage 96.34% 96.34% -0.01%
==========================================
Files 465 465
Lines 59151 59177 +26
==========================================
+ Hits 56990 57015 +25
- Misses 2161 2162 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Manny7717
left a comment
There was a problem hiding this comment.
Verified locally (head cc06db5, base ce37614)
Regression-proven: the PR's rewritten test file (with PACKING_ATTRIBUTES import removed for base compatibility) yields 4 failures on base — test_sensor (MWS vs mws), test_get_dataset_get_channeldata_counts, test_get_dataset_get_channeldata_bts, test_get_global_attributes — and 24/24 pass on head. All 4 base failures trace to the intended behavior changes, not unrelated breakage.
Core bug confirmed in the old code:
_get_dataset_channel(old) applied only valid-range masking and NEVER appliedscale_factor/add_offset— brightness temperatures came back as raw packed counts. Newmask_and_scale()masks fill + valid range BEFORE scaling, casts to float32, applies scale/offset, and strips the packing attrs (they no longer describe the scaled array). Correct netCDF packing semantics._get_dataset_aux_data(old) only scaled when BOTHscale_factorANDadd_offsetexisted, and didvariable.attrs["missing_value"]— a hardKeyErrorwhen the attr is absent. New path is robust to missing attrs (falls back to netCDFdefault_fillvalsand ±inf range).- YAML:
mws_lonfile_key pointed atdata/navigation/mws_lat(copy-paste) — fixed tomws_lon. Real data bug. sensornow lowercases/attr/instrument, matching the reader configsensors: [mws]and satpy's case-sensitivesupports_sensorset intersection.
Edge-checked: counts path keeps integer dtype + _FillValue attr (dtype-typed fill); float path NaN-fills; fill-vs-range precedence (FillValue → _FillValue → missing_value) sensible; xr.set_options(keep_attrs=True) preserves attrs through where/astype.
No blocking issues. Clean, well-tested fix.
|
@Manny7717 Who are you? I'm sorry if I missed some conversation about this. Did someone ask you for these AI reviews? |
mraspaud
left a comment
There was a problem hiding this comment.
LGTM, thanks an lot for the simplifications!
Also lowercases sensor name to 'mws'.
Claude did almost all the coding on this one, but all code reviewed by me.
AUTHORS.mdif not there already