Skip to content

Feat/improved autofocuser - #129

Merged
ppp-one merged 16 commits into
ppp-one:mainfrom
dgegen:feat/improved-autofocuser
Aug 6, 2026
Merged

ppp-one merged 16 commits into
ppp-one:mainfrom
dgegen:feat/improved-autofocuser

Conversation

@dgegen

@dgegen dgegen commented Mar 19, 2026 •

Copy link
Copy Markdown
Collaborator

name: Pull request
about: Create a pull request


Description
Improves the Autofocuser class with a shared run_timestamp property so all output files from a single autofocus run share a consistent timestamp-based naming scheme. Also simplify the focus record handling to use only the in-memory record, and adds logging for the observatory subclass type at startup.

Changes Made

  • Log name of subclass during observatory startup
  • Add run_timestamp property to Autofocuser that lazily generates a single timestamp (format YYYYMMDD_HHMMSS) shared across all output files of one run
  • Add _output_path() helper that builds consistent output paths using run_timestamp
  • Refactor make_summary_plot() and save_result() to use _output_path() instead of ad-hoc timestamp/CSV-derived filenames
  • Remove fallback CSV-reading logic in make_summary_plot() — the in-memory focus_record is now the only source; failures emit warnings and return early
  • Also save the focus record DataFrame as CSV via make_summary_plot() using the shared timestamp
  • Log observatory subclass name at startup (observatory.py)
  • Fix self.observatory.error(...) → self.observatory.logger.error(...) call
  • Bump astrafocus dependency to 0.1.2. This fixes a few small bugs that propagated into Astra, especially when using secondary focus measure operators.

Checklist

  • Code follows project style guidelines
  • Tests added/updated
  • Documentation updated (if needed)
  • All checks pass

Copilot AI review requested due to automatic review settings March 19, 2026 18:23

This comment was marked as resolved.

@dgegen
dgegen force-pushed the feat/improved-autofocuser branch from b70600c to 8069d43 Compare March 24, 2026 10:21
Comment thread src/astra/autofocus.py Outdated
except Exception as e:
field_of_view = np.array([np.nan, np.nan])
self.observatory.error(
self.observatory.logger.error(

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning, instead?

@dgegen
dgegen force-pushed the feat/improved-autofocuser branch from 1633aa0 to 33ffe60 Compare July 21, 2026 09:09
dgegen added 8 commits July 21, 2026 11:09
Upgrades astrafocus from 0.1.2 to 0.1.3, which fixes autofocus runs
crashing with "Input must be a 2D array" when an exposure fails mid-sweep
(e.g. due to weather); the new version catches focus measure exceptions
and records NaN instead of aborting.

Also adds documentation (docstring and FIELD_DESCRIPTIONS) clarifying when
to prefer non-parametric operators (fft, normalized_variance) for coarse
searches vs. analytic operators (HFR) for fine tuning near the focus peak.
@dgegen
dgegen force-pushed the feat/improved-autofocuser branch from 33ffe60 to 9632c68 Compare July 21, 2026 09:26
@dgegen
dgegen force-pushed the feat/improved-autofocuser branch from 9632c68 to 6bee361 Compare July 21, 2026 09:28
dgegen added 7 commits July 21, 2026 15:25
Break the monolithic utils.py into astra/utils/{time,image,ephemeris,speculoos}.py
grouped by responsibility (JD/time conversions, image cleaning, sun/body
ephemeris, SPECULOOS error handling), and update all call sites and tests
to import from the new submodules.
Adds PairedDevices.calculate_fwhm(), estimating the DAOStarFinder FWHM
from the camera/telescope plate scale assuming ~2" seeing when
AutofocusConfig.fwhm is left unset. Also moves field-of-view
calculation from Autofocuser into PairedDevices.calculate_field_of_view()
for symmetry, reusing the shared math in astra.utils.optics.
…ogic

Break out of the IsMoving poll loop early if observatory conditions
fail during a focuser slew, rather than waiting for timeout. Also
extract the repeated save_path/last_image_path fallback logic in
Autofocuser into a shared _determine_save_dir() helper.
…he redundant prefix on check/ack function names

Reflects that the module handles AsTelOS telescope errors specifically, not SPECULOOS-wide utilities.
@dgegen

dgegen commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator Author

This led me to some cleanup related to, but also going beyond, the autofocuser improvements. With the acceptance of twirl/pull/44, the dependency tension between photutils>3.0.0 and twirl has been resolved. @ppp-one, I think this is ready to merge if you're happy with the changes.

@ppp-one

ppp-one commented Aug 6, 2026

Copy link
Copy Markdown
Owner

<3

@ppp-one
ppp-one merged commit 992b119 into ppp-one:main Aug 6, 2026
2 checks passed
ppp-one added a commit that referenced this pull request Sep 9, 2026
* feat: jpl horizons and tle support

* fix: Update TLE for non-sidereal tracking tests

* fix: addressed copilot comments

* fix: api: improve schedule function error handling and sanitize action values

* fix: enhance ObjectActionConfig visibility checker to handle non-sidereal target coordinates

* fix: adjust nonsidereal recenter interval handling and improve RA rate calculation for accuracy

* imports

* added docstrings

* fix: update nonsidereal tracking interval handling and improve RA rate assertion accuracy

* enhance: optimize non-sidereal rate application to reduce unnecessary updates

* fix: upgrade alpyca to PyPI release and fix subprocess device initialisation

Upgrade alpyca from a git pin to the stable PyPI release (>=3.1.2).

Move device instantiation from __init__ to run() so that the alpyca
device object (which opens network connections) is created inside the
subprocess rather than in the parent, avoiding pickling issues and
shared-socket hazards under multiprocessing.

Initialise subprocess-local state in run(): patch Device._put/_get to
default to a 60 s timeout and seed client IDs with a PID-derived random
value to prevent conflicts across processes.

Simplify callable dispatch in get__: replace the no_kwargs sentinel
with `if callable(data): data = data(**kwargs)`, which handles both
zero-arg methods (empty kwargs → data()) and methods with arguments,
and correctly skips the call for plain property reads.

* Update telescope.get calls in guiding.py

There are 4 occurrences of `self.telescope.get("PulseGuide")(...)`.
The new behaviour, get now auto-calls the callable in the subproces).
Consequently, this needed to be updated

* fix: preserve retry sentinel when callable dispatch raises in get__

If getattr() succeeded but the subsequent call raised, data was left
holding the bound method rather than the "not get" sentinel, causing
the retry loop and final attempt to be silently skipped, then failing
to pickle the method on send. Use a temporary variable so data is only
updated from the sentinel once the full operation completes.

* fix: validate device type in __init__ instead of run()

The previous late check in run() left the parent holding open pipes to a
dead subprocess, resulting in an EOFError on the next get() or set() call
rather than a meaningful error. Raising ValueError in __init__ fails fast
before start() is called and any pipes are created, giving callers an
immediate and unambiguous error message.

* fix: update PulseGuide call in calibrate_guiding to use get() kwargs

* fix: increase minimum guide interval to improve tracking stability

* fix: enhance boolean value handling in FITS header processing

* enhance: improve non-sidereal tracking and telescope slew handling

* Feat/improved autofocuser (#129) + refactor

* Log observatory subclass type during startup

* chore: update astrafocus dependency to version 0.1.2

* Add run_timestamp property and refactor output path handling in Autofocuser

* fix: ensure output directory exists and update log messages for clarity

* bump astrafocus to 0.1.3 and document focus measure operator guidance

Upgrades astrafocus from 0.1.2 to 0.1.3, which fixes autofocus runs
crashing with "Input must be a 2D array" when an exposure fails mid-sweep
(e.g. due to weather); the new version catches focus measure exceptions
and records NaN instead of aborting.

Also adds documentation (docstring and FIELD_DESCRIPTIONS) clarifying when
to prefer non-parametric operators (fft, normalized_variance) for coarse
searches vs. analytic operators (HFR) for fine tuning near the focus peak.

* fix: improve error handling in Autofocuser initialization and field of view determination

* Upgrade cabaret

* fix: update dependencies for astrafocus and cabaret to latest versions

* Update twirl and photutils dependencies

* Split astra.utils into a package with per-concern submodules

Break the monolithic utils.py into astra/utils/{time,image,ephemeris,speculoos}.py
grouped by responsibility (JD/time conversions, image cleaning, sun/body
ephemeris, SPECULOOS error handling), and update all call sites and tests
to import from the new submodules.

* Extract shared plate scale/FOV/FWHM math into astra.utils.optics

* Auto-derive default FWHM from plate scale instead of hardcoding 8px

Adds PairedDevices.calculate_fwhm(), estimating the DAOStarFinder FWHM
from the camera/telescope plate scale assuming ~2" seeing when
AutofocusConfig.fwhm is left unset. Also moves field-of-view
calculation from Autofocuser into PairedDevices.calculate_field_of_view()
for symmetry, reusing the shared math in astra.utils.optics.

* Abort focuser slew wait on failed conditions check; dedupe save_dir logic

Break out of the IsMoving poll loop early if observatory conditions
fail during a focuser slew, rather than waiting for timeout. Also
extract the repeated save_path/last_image_path fallback logic in
Autofocuser into a shared _determine_save_dir() helper.

* Rename astra.utils.speculoos to astra.utils.astelos_errors and drop the redundant prefix on check/ack function names

Reflects that the module handles AsTelOS telescope errors specifically, not SPECULOOS-wide utilities.

* Adapt photutils>=3.0.0 underscore syntax

* Bump astrafocus version

* feat: add requests dependency and improve slew handling with polling delays

* feat: enhance non-sidereal tracking support and validation checks

* Refactor non-sidereal tracking logic and improve rate update handling

- Adjust default lead time for non-sidereal tracking to 0 seconds, allowing immediate slewing to targets.
- Introduce a minimum rate update interval to prevent excessive rate commands to the mount.
- Implement drift calculations to determine if re-centering is necessary based on target movement.
- Enhance error handling for stale ephemeris data, ensuring tracking is only active for valid epochs.
- Update tests to reflect changes in default lead time and validate new rate update logic.
- Remove unnecessary parameters and streamline the ephemeris fetching process for non-sidereal targets.

* feat: add safety checks for non-sidereal re-centering and enhance tests

* Refactor non-sidereal tracking and improve handling of tracking rates

- Updated HeaderManager to allow None values in boolean conversion.
- Revised documentation in nonsidereal.py for clarity on tracking capabilities and integration notes.
- Enhanced NonSiderealManager to manage tracking rates more effectively, including improved recentering logic and state management.
- Implemented caching for telescope capabilities regarding differential tracking rates in Observatory.
- Added methods to clear stale tracking rates before slewing to prevent trailing on subsequent sequences.
- Improved tests for nonsidereal tracking, including checks for recentering behavior and rate application logic.
- Adjusted ephemeris utility functions to clarify rate calculations and ensure correct ASCOM units.
- Enhanced visibility checks for moving bodies to ensure proper configuration validation.

* feat: enhance equatorial system handling and coordinate conversions for mounts

* fix: add public fields utility and normalize action values in schedule

* Refactor ephemeris sampling and validation for moving targets

- Introduced automatic interval selection for ephemeris sampling based on target motion, improving accuracy for fast-moving bodies like satellites.
- Added new utility functions to calculate maximum sky rate and appropriate sampling intervals.
- Updated `precompute_ephemeris` to handle automatic interval selection and refined sampling for fast targets.
- Enhanced validation in `ObjectActionConfig` to reject fixed coordinates for moving targets, ensuring correct behavior during visibility checks.
- Adjusted tests to cover new functionality, including automatic interval refinement and validation of fixed coordinates alongside moving targets.
- Improved logging and documentation for clarity on ephemeris sampling behavior and constraints.

---------

Co-authored-by: Peter Pihlmann Pedersen <peter@ppp.one>
Co-authored-by: dgegen <ddegen@ethz.ch>
Co-authored-by: dgegen <66382194+dgegen@users.noreply.github.com>
ppp-one added a commit that referenced this pull request Sep 11, 2026
* feat: add non-sidereal tracking for solar system objects

Pre-computes a cubic-interpolated ephemeris at sequence start so that
ASCOM RightAscensionRate/DeclinationRate can be refreshed each exposure
without repeated get_body() calls. Periodic re-centering slews correct
long-term drift. Guiding is automatically disabled when non-sidereal
tracking is active.

- utils: add is_solar_system_body, precompute_ephemeris,
  compute_nonsidereal_rates_from_interp; cache _SOLAR_SYSTEM_BODIES
  and _SOLAR_TO_SIDEREAL at module level
- nonsidereal: new NonSiderealManager class encapsulating setup,
  rate application, re-centering, and cleanup
- observatory: integrate NonSiderealManager into the imaging sequence
  loop with guiding-incompatibility warning and guaranteed rate reset
  in a finally block
- action_configs: add nonsidereal_recenter_interval field and update
  guiding field description
- tests: fix monkeypatch targets for get_body; add full test suite for
  the three new utility functions

* Add example to documentation

- Future-proof logic selecting non-sidereal tracking
- Add updated uv.lock file

* fix: non-sidereal tracking and add integration test for Mars sequence

- Activate NonSiderealManager on lookup_name alone; recenter_interval=0
  now means "track without periodic re-slews" rather than "disabled"
- Use np.unwrap for continuous RA interpolation to avoid discontinuities
  at the 0°/360° boundary
- Re-apply tracking rates after a pointing correction in the exposure loop
- Add test_nonsidereal_object_action: verifies a Mars sequence completes,
  takes images, and resets RA/Dec rates on teardown
- Add set_location_for_body_visibility helper to position the simulator
  at the target's meridian so the visibility check passes at any time
- Fix action config: nonsidereal_object test case uses action_type=object
  with both object and lookup_name keys, allowing create_schedule_data
  to support an action_type override independent of the config key name

* refactor: pre-compute non-sidereal ephemeris at schedule load time

Move ephemeris computation from sequence start (NonSiderealManager._setup)
to schedule validation (ObjectActionConfig._resolve_lookup_name), so the
interpolators are built once when the schedule is loaded rather than
re-computed at runtime.

Key changes:
- Add NotMovingBodyError to distinguish solar system / minor bodies from
  fixed targets in precompute_ephemeris
- Extend precompute_ephemeris to query JPL Horizons via astroquery for
  comets and asteroids not in astropy's built-in ephemeris
- Store _nonsidereal flag and interpolators as private fields on
  ObjectActionConfig; NonSiderealManager reads them directly instead of
  repeating network calls
- Remove obs_location parameter from NonSiderealManager (no longer needed
  at runtime)
- Add astroquery>=0.4.11 dependency
- Add unit tests for NonSiderealManager and a network-marked integration
  test for minor-body ephemeris lookup

* fix: improve logging for non-sidereal tracking rate failures

* feat: Added JPL Horizons, and TLE Support (#136)

* feat: jpl horizons and tle support

* fix: Update TLE for non-sidereal tracking tests

* fix: addressed copilot comments

* fix: api: improve schedule function error handling and sanitize action values

* fix: enhance ObjectActionConfig visibility checker to handle non-sidereal target coordinates

* fix: adjust nonsidereal recenter interval handling and improve RA rate calculation for accuracy

* imports

* added docstrings

* fix: update nonsidereal tracking interval handling and improve RA rate assertion accuracy

* enhance: optimize non-sidereal rate application to reduce unnecessary updates

* fix: upgrade alpyca to PyPI release and fix subprocess device initialisation

Upgrade alpyca from a git pin to the stable PyPI release (>=3.1.2).

Move device instantiation from __init__ to run() so that the alpyca
device object (which opens network connections) is created inside the
subprocess rather than in the parent, avoiding pickling issues and
shared-socket hazards under multiprocessing.

Initialise subprocess-local state in run(): patch Device._put/_get to
default to a 60 s timeout and seed client IDs with a PID-derived random
value to prevent conflicts across processes.

Simplify callable dispatch in get__: replace the no_kwargs sentinel
with `if callable(data): data = data(**kwargs)`, which handles both
zero-arg methods (empty kwargs → data()) and methods with arguments,
and correctly skips the call for plain property reads.

* Update telescope.get calls in guiding.py

There are 4 occurrences of `self.telescope.get("PulseGuide")(...)`.
The new behaviour, get now auto-calls the callable in the subproces).
Consequently, this needed to be updated

* fix: preserve retry sentinel when callable dispatch raises in get__

If getattr() succeeded but the subsequent call raised, data was left
holding the bound method rather than the "not get" sentinel, causing
the retry loop and final attempt to be silently skipped, then failing
to pickle the method on send. Use a temporary variable so data is only
updated from the sentinel once the full operation completes.

* fix: validate device type in __init__ instead of run()

The previous late check in run() left the parent holding open pipes to a
dead subprocess, resulting in an EOFError on the next get() or set() call
rather than a meaningful error. Raising ValueError in __init__ fails fast
before start() is called and any pipes are created, giving callers an
immediate and unambiguous error message.

* fix: update PulseGuide call in calibrate_guiding to use get() kwargs

* fix: increase minimum guide interval to improve tracking stability

* fix: enhance boolean value handling in FITS header processing

* enhance: improve non-sidereal tracking and telescope slew handling

* Feat/improved autofocuser (#129) + refactor

* Log observatory subclass type during startup

* chore: update astrafocus dependency to version 0.1.2

* Add run_timestamp property and refactor output path handling in Autofocuser

* fix: ensure output directory exists and update log messages for clarity

* bump astrafocus to 0.1.3 and document focus measure operator guidance

Upgrades astrafocus from 0.1.2 to 0.1.3, which fixes autofocus runs
crashing with "Input must be a 2D array" when an exposure fails mid-sweep
(e.g. due to weather); the new version catches focus measure exceptions
and records NaN instead of aborting.

Also adds documentation (docstring and FIELD_DESCRIPTIONS) clarifying when
to prefer non-parametric operators (fft, normalized_variance) for coarse
searches vs. analytic operators (HFR) for fine tuning near the focus peak.

* fix: improve error handling in Autofocuser initialization and field of view determination

* Upgrade cabaret

* fix: update dependencies for astrafocus and cabaret to latest versions

* Update twirl and photutils dependencies

* Split astra.utils into a package with per-concern submodules

Break the monolithic utils.py into astra/utils/{time,image,ephemeris,speculoos}.py
grouped by responsibility (JD/time conversions, image cleaning, sun/body
ephemeris, SPECULOOS error handling), and update all call sites and tests
to import from the new submodules.

* Extract shared plate scale/FOV/FWHM math into astra.utils.optics

* Auto-derive default FWHM from plate scale instead of hardcoding 8px

Adds PairedDevices.calculate_fwhm(), estimating the DAOStarFinder FWHM
from the camera/telescope plate scale assuming ~2" seeing when
AutofocusConfig.fwhm is left unset. Also moves field-of-view
calculation from Autofocuser into PairedDevices.calculate_field_of_view()
for symmetry, reusing the shared math in astra.utils.optics.

* Abort focuser slew wait on failed conditions check; dedupe save_dir logic

Break out of the IsMoving poll loop early if observatory conditions
fail during a focuser slew, rather than waiting for timeout. Also
extract the repeated save_path/last_image_path fallback logic in
Autofocuser into a shared _determine_save_dir() helper.

* Rename astra.utils.speculoos to astra.utils.astelos_errors and drop the redundant prefix on check/ack function names

Reflects that the module handles AsTelOS telescope errors specifically, not SPECULOOS-wide utilities.

* Adapt photutils>=3.0.0 underscore syntax

* Bump astrafocus version

* feat: add requests dependency and improve slew handling with polling delays

* feat: enhance non-sidereal tracking support and validation checks

* Refactor non-sidereal tracking logic and improve rate update handling

- Adjust default lead time for non-sidereal tracking to 0 seconds, allowing immediate slewing to targets.
- Introduce a minimum rate update interval to prevent excessive rate commands to the mount.
- Implement drift calculations to determine if re-centering is necessary based on target movement.
- Enhance error handling for stale ephemeris data, ensuring tracking is only active for valid epochs.
- Update tests to reflect changes in default lead time and validate new rate update logic.
- Remove unnecessary parameters and streamline the ephemeris fetching process for non-sidereal targets.

* feat: add safety checks for non-sidereal re-centering and enhance tests

* Refactor non-sidereal tracking and improve handling of tracking rates

- Updated HeaderManager to allow None values in boolean conversion.
- Revised documentation in nonsidereal.py for clarity on tracking capabilities and integration notes.
- Enhanced NonSiderealManager to manage tracking rates more effectively, including improved recentering logic and state management.
- Implemented caching for telescope capabilities regarding differential tracking rates in Observatory.
- Added methods to clear stale tracking rates before slewing to prevent trailing on subsequent sequences.
- Improved tests for nonsidereal tracking, including checks for recentering behavior and rate application logic.
- Adjusted ephemeris utility functions to clarify rate calculations and ensure correct ASCOM units.
- Enhanced visibility checks for moving bodies to ensure proper configuration validation.

* feat: enhance equatorial system handling and coordinate conversions for mounts

* fix: add public fields utility and normalize action values in schedule

* Refactor ephemeris sampling and validation for moving targets

- Introduced automatic interval selection for ephemeris sampling based on target motion, improving accuracy for fast-moving bodies like satellites.
- Added new utility functions to calculate maximum sky rate and appropriate sampling intervals.
- Updated `precompute_ephemeris` to handle automatic interval selection and refined sampling for fast targets.
- Enhanced validation in `ObjectActionConfig` to reject fixed coordinates for moving targets, ensuring correct behavior during visibility checks.
- Adjusted tests to cover new functionality, including automatic interval refinement and validation of fixed coordinates alongside moving targets.
- Improved logging and documentation for clarity on ephemeris sampling behavior and constraints.

---------

Co-authored-by: Peter Pihlmann Pedersen <peter@ppp.one>
Co-authored-by: dgegen <ddegen@ethz.ch>
Co-authored-by: dgegen <66382194+dgegen@users.noreply.github.com>

---------

Co-authored-by: dgegen <ddegen@ethz.ch>
Co-authored-by: SaltyPotatoe <90574754+SaltyPotatoe@users.noreply.github.com>
Co-authored-by: dgegen <66382194+dgegen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants