Skip to content

Migrate the COMPAS build and dependency management to pixi - #1

Open
kcsapp wants to merge 8 commits into
devfrom
kcsapp/pixi
Open

kcsapp wants to merge 8 commits into
devfrom
kcsapp/pixi

Conversation

@kcsapp

@kcsapp kcsapp commented Aug 18, 2026

Copy link
Copy Markdown
Owner
  • Build updates: uv, aarch, lint & format
  • Convert build to pixi
  • merge features of setup.py into pyproject.toml
  • more refinement of the pixi setup and helper commands

pixi (https://pixi.prefix.dev/) is a package manager connected to the Anaconda packaging system, written in rust for speed and platform-agnostic features.

While working on setting up COMPAS locally, I decided to experiment with this to

  • isolate the packages and environment required to build COMPAS from the rest of my system;
  • lean on pre-built versions of packages that COMPAS depends on to avoid possible conflicts due to type mismatches, and for the speed advantages;
  • update the python build process and integrate it with the C++ build process; and
  • to be able to create simple and repeatable commands for building & installation (for both local installation and docker image building)

The file pyproject.toml now contains dependency specifications for both python-library dependencies and the core C++ COMPAS library's dependencies. The latter are retrieved from conda-forge by default, but any conda repo could theoretically be substituted; and they are managed locally by updating your shell environment, either in another shell (pixi shell) or scoped to a single execution of a command (pixi run) - the latter is typically how I use it. I also designed a few 'Tasks' (basically inline shell scripts) for common build & clean operations, that can be run as pixi run {task_name}:

  • version: prints the version, as retrieved from the changelog (via the python package)
  • build: run the C++ build locally, attempting to set some reasonable limits on parallelism (I was frequently running out of memory with the default nproc threads)
  • clean: runs make clean for the C++ build (nothing to do for python)
  • build-docker: build the docker image - this builds the COMPAS binary in the docker-build environment, and isolates the build cache for easier cleanup later
  • clean-docker: clean up resources produced while building the docker image - these artifacts can use up disk space quickly, this lets you clean up just those used for COMPAS (in case you want to preserve others)
    and of course
  • compas: runs the COMPAS binary with arguments! If the binary can be found on your local path, it uses that; otherwise, it runs using the docker image, whether built locally or pulled from dockerhub.

pixi installation instructions here

@kcsapp kcsapp self-assigned this Aug 18, 2026
Comment on lines -1 to -3
import matplotlib.pyplot as plt
from .conversions import chirp_mass_eta_to_m1_m2, m1_m2_to_eta_chirp_mass
from .gpu_utils import xp

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have the config for lint - format updates? Couldn't find which config generates these changes.

Comment thread Dockerfile Outdated
@@ -1,32 +1,24 @@
FROM ubuntu:22.04
COPY --from=ghcr.io/prefix-dev/pixi:latest /usr/local/bin/pixi /bin/pixi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend pinning the dependency here.

@ulgens

ulgens commented Sep 24, 2026

Copy link
Copy Markdown

We already discussed the overall purpose - approach with pixi, and I'm okay on that end but I believe this soon-to-be PR is doing a bit too much for a single PR. I created TeamCOMPAS#1490 to start modernizing the lint setup and TeamCOMPAS#1492 to start cleaning - updating CI pipelines; planning to move incrementally when they got merged. Overall, I was thinking of the following work items:

  • Base OS updates for CI and builds
  • CI dependency updates
  • Basic lint setup
  • Improved lint setup with ruff
  • Improved dependency and build management with uv
  • Then, pixi or some other generalized dependency manager.

@kcsapp

kcsapp commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

We already discussed the overall purpose - approach with pixi, and I'm okay on that end but I believe this soon-to-be PR is doing a bit too much for a single PR. I created TeamCOMPAS#1490 to start modernizing the lint setup and TeamCOMPAS#1492 to start cleaning - updating CI pipelines; planning to move incrementally when they got merged. Overall, I was thinking of the following work items:

  • Base OS updates for CI and builds
  • CI dependency updates
  • Basic lint setup
  • Improved lint setup with ruff
  • Improved dependency and build management with uv
  • Then, pixi or some other generalized dependency manager.

I think it's reasonable to break this PR up but I would like to avoid cycling through build tools. pixi is an extension of uv so I'd rather standardize on it if you want uv behavior, or choose something else independently (I don't think uv can't help with the C++ build).

Also before we introduce CI we will need to have a discussion with the repo authors - it imposes costs they may or may not be okay with.

I tried to avoid linting and formatting code here but I think my IDE got ahead of me - I will revert some of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants