Skip to content

Add benchmarking test framework and improve featurization performance - #48

Merged
d33bs merged 19 commits into
WayScience:mainfrom
d33bs:hortuscute
Aug 7, 2026
Merged

Add benchmarking test framework and improve featurization performance#48
d33bs merged 19 commits into
WayScience:mainfrom
d33bs:hortuscute

Conversation

@d33bs

@d33bs d33bs commented Aug 5, 2026

Copy link
Copy Markdown
Member

Description

This PR adds a benchmarking test-based framework and improves feature extraction performance generally. I focused on making relatively boring or safe changes which don't interfere with the static output we already have achieved in this project. Below is a table of what we're seeing.

Synthetic data:

┌───────────────────┬─────────┬─────────┬─────────┐
│      feature      │  main   │ branch  │ speedup │
├───────────────────┼─────────┼─────────┼─────────┤
│ neighbors         │ 0.162 s │ 0.009 s │ ~18×    │
├───────────────────┼─────────┼─────────┼─────────┤
│ intensity         │ 0.337 s │ 0.071 s │ ~4.7×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ volume_size_shape │ 0.082 s │ 0.043 s │ ~1.9×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ texture           │ 0.802 s │ 0.555 s │ ~1.4×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ colocalization    │ 0.458 s │ 0.383 s │ ~1.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ granularity       │ 0.113 s │ 0.104 s │ ~1.08×  │
└───────────────────┴─────────┴─────────┴─────────┘

Real-world (small CP tutorial) data:

┌───────────────────┬─────────┬─────────┬─────────┐
│      feature      │  main   │ branch  │ speedup │
├───────────────────┼─────────┼─────────┼─────────┤
│ intensity         │ 0.579 s │ 0.087 s │ ~6.7×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ neighbors         │ 0.286 s │ 0.035 s │ ~8.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ texture           │ 0.551 s │ 0.288 s │ ~1.9×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ volume_size_shape │ 0.186 s │ 0.087 s │ ~2.2×   │
├───────────────────┼─────────┼─────────┼─────────┤
│ colocalization    │ 0.457 s │ 0.315 s │ ~1.45×  │
├───────────────────┼─────────┼─────────┼─────────┤
│ granularity       │ 1.756 s │ ~1.77 s │ —       │
├───────────────────┼─────────┼─────────┼─────────┤
│ TOTAL             │ 3.82 s  │ 2.57 s  │ ~1.48×  │
└───────────────────┴─────────┴─────────┴─────────┘

What kind of change(s) are included?

  • Documentation (changes docs or other related content)
  • Bug fix (fixes an issue).
  • Enhancement (adds functionality).
  • Breaking change (these changes would cause existing functionality to not work as expected).

Checklist

Please ensure that all boxes are checked before indicating that this pull request is ready for review.

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have searched for existing content to ensure this is not a duplicate.
  • I have performed a self-review of these additions (including spelling, grammar, and related).
  • These changes pass all pre-commit checks.
  • I have added comments to my code to help provide understanding
  • I have added a test which covers the code changes found within this PR
  • I have deleted all non-relevant text in this pull request template.

📚 Documentation preview 📚: https://zedprofiler--48.org.readthedocs.build/en/48/

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@7d76edb). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #48   +/-   ##
=======================================
  Coverage        ?   93.38%           
=======================================
  Files           ?       15           
  Lines           ?     1376           
  Branches        ?        0           
=======================================
  Hits            ?     1285           
  Misses          ?       91           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@d33bs
d33bs marked this pull request as ready for review August 5, 2026 22:29
@d33bs
d33bs requested review from MikeLippincott and gwaybio August 5, 2026 22:29

@MikeLippincott MikeLippincott left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM I left some minor concerns

Comment thread docs/src/benchmarking.md Outdated
Comment thread src/zedprofiler/featurization/intensity.py
Comment thread src/zedprofiler/featurization/intensity.py
Comment thread src/zedprofiler/featurization/neighbors.py
Comment thread tests/featurization/test_neighbors.py
Comment thread tests/benchmarking.py Outdated
Comment thread tests/benchmarking.py Outdated
d33bs and others added 6 commits August 6, 2026 10:56
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
Co-Authored-By: Mike Lippincott <58147848+MikeLippincott@users.noreply.github.com>
@d33bs

d33bs commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

Thanks @MikeLippincott !

@d33bs
d33bs merged commit b44f5d3 into WayScience:main Aug 7, 2026
15 of 21 checks passed
@d33bs
d33bs deleted the hortuscute branch August 7, 2026 12:35
d33bs added a commit to d33bs/ZedProfiler that referenced this pull request Aug 7, 2026
Integrate the benchmarking framework and featurization performance
refactor from upstream (PR WayScience#48) with the per-shard CLI and
Metadata_Imaging_ImageID propagation work on this branch.

The two lines of work touched disjoint regions and merged cleanly, but
the upstream BenchmarkImageSet mock lacked the image_id attribute that
the cli featurization functions now read. Add image_id to
BenchmarkImageSet (mirroring ImageSetLoader's fallback to
image_set_name) and regenerate the six accuracy-lock signatures in
test_benchmark_contracts.py, which now include the
Metadata_Imaging_ImageID column.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants