The FITS headers POCS writes are the input contract for
panoptes-pipeline, which reads
them to calibrate frames and produce the survey's photometry. That vocabulary
has drifted, in both directions, with nothing checking it.
What drifted
Checked against main at 7370af45. These keywords are read by
panoptes-pipeline's extract_metadata and are absent repo-wide from POCS:
| Keyword |
What it carried |
Consequence downstream |
WHTLVLN / WHTLVLS |
camera white level |
saturation masking falls back to a fleet-wide constant that is ~25% too high on both cameras measured |
CAMSN |
body serial number |
the only human-readable link from a frame to a physical body |
INTSN |
lens serial number |
lens provenance |
MEASRGGB |
measured RGGB levels |
Bayer phase check |
The pipeline reads them with header.get(...), so it degrades silently:
None for white level, and a "0 0 0 0" default for MEASRGGB. Nothing fails,
the numbers just get quietly worse.
Going the other way, POCS writes EGAIN and BITDEPTH, which the pipeline does
not currently read.
Frames already in the archive are unaffected — they carry these keywords. The
break is forward-looking, and it is only detectable today by diffing two
repositories by hand.
Why this is worth a contract rather than a bug fix
There is already a single authoritative fields map in
src/panoptes/pocs/camera/camera.py that defines the header vocabulary. That is
the right place; it just is not treated as an interface. A keyword removed from
it is an API change for every consumer of PANOPTES data, and today that change
is silent at both ends.
Proposed
Context
Reasoning is in
plans/data-contract.md,
sections 2.2 and 8, in the panoptes-pipeline repository. Related:
panoptes/panoptes-pipeline#166.
Filed by Claude Opus 5 · effort: high · 🤖 Claude Code
The FITS headers POCS writes are the input contract for
panoptes-pipeline, which reads
them to calibrate frames and produce the survey's photometry. That vocabulary
has drifted, in both directions, with nothing checking it.
What drifted
Checked against
mainat7370af45. These keywords are read bypanoptes-pipeline'sextract_metadataand are absent repo-wide from POCS:WHTLVLN/WHTLVLSCAMSNINTSNMEASRGGBThe pipeline reads them with
header.get(...), so it degrades silently:Nonefor white level, and a"0 0 0 0"default forMEASRGGB. Nothing fails,the numbers just get quietly worse.
Going the other way, POCS writes
EGAINandBITDEPTH, which the pipeline doesnot currently read.
Frames already in the archive are unaffected — they carry these keywords. The
break is forward-looking, and it is only detectable today by diffing two
repositories by hand.
Why this is worth a contract rather than a bug fix
There is already a single authoritative
fieldsmap insrc/panoptes/pocs/camera/camera.pythat defines the header vocabulary. That isthe right place; it just is not treated as an interface. A keyword removed from
it is an API change for every consumer of PANOPTES data, and today that change
is silent at both ends.
Proposed
fieldsmap as the declared vocabulary, and note in POCS thatremoving or renaming an entry is a breaking change for downstream consumers.
panoptes-pipelinedeclares which keywords it requires and fails loudlywhen one is missing, instead of defaulting. That half is
Declare which header keywords are required, and fail loudly when one is missing panoptes-pipeline#186.
tracked separately in Restore WHTLVLN/WHTLVLS (camera white level) to the FITS header #1480 (white level) and Make egain per-body rather than a hardcoded class constant #1481 (per-body gain).
Context
Reasoning is in
plans/data-contract.md,sections 2.2 and 8, in the
panoptes-pipelinerepository. Related:panoptes/panoptes-pipeline#166.
Filed by Claude Opus 5 · effort: high · 🤖 Claude Code