Promote reach-end gridpoints to breakpoints, remove boundary - #694
Open
jpalm3r wants to merge 3 commits into
Open
Promote reach-end gridpoints to breakpoints, remove boundary#694jpalm3r wants to merge 3 commits into
jpalm3r wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request updates ModelSkill’s network loading and time-series handling to (1) make MIKE reach-end gridpoint data discoverable via standard breakpoint mechanisms (removing the unused NetworkNode.boundary concept), (2) add a quantities filter to Network.from_mike()/from_epanet() for more selective reads, and (3) standardize timezone-aware time series inputs by converting them to UTC timezone-naive datetime64[ns].
Changes:
- Promote MIKE reach start/end gridpoints to ordinary breakpoints and tag node↔breakpoint connections as zero-length
boundary=Trueedges. - Add
quantitiesfiltering to network constructors and propagate it through the Res1D adapter read path. - Normalize tz-aware time to UTC-naive and extend persistence support to include node-geometry comparers’ raw model data; update docs and tests accordingly.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_timeseries.py | Adds coverage for tz-aware inputs being converted to UTC-naive. |
| tests/test_network.py | Updates node-count expectations; adds tests for boundary edges, reach-end breakpoint behavior, quantity filtering, and filtered-load correctness. |
| tests/test_match.py | Adds matching tests for tz-aware vs tz-naive combinations and different timezones. |
| tests/test_comparercollection.py | Adds regression test for save/load round-trip of node-geometry comparer raw data. |
| tests/test_comparer.py | Adds regression test for tz-aware from_matched() normalization. |
| src/modelskill/timeseries/_timeseries.py | Replaces ns-only normalization with timezone + ns normalization and applies it during dataset validation. |
| src/modelskill/timeseries/_point.py | Switches to the new _normalize_time() helper. |
| src/modelskill/network.py | Removes NetworkNode.boundary, adds quantities filtering, shared chainage tolerance, and boundary-edge tagging/clamping in graph generation. |
| src/modelskill/model/adapters/_res1d.py | Implements quantities filtering and shared empty-frame usage for topology-only locations (with one inconsistency flagged in review). |
| src/modelskill/comparison/_comparison.py | Uses _normalize_time() and enables save/load of raw model data for node-geometry comparers. |
| docs/user-guide/network.qmd | Updates docs to remove boundary, explain boundary edges, and document quantities. |
| docs/user-guide/matching.qmd | Documents timezone handling and UTC-naive conversion behavior. |
Suppressed comments (1)
src/modelskill/model/adapters/_res1d.py:28
_simplify_colnames()returns a new emptyDataFrame()whennode.quantitiesis empty (e.g. MIKE 11 nodes). This defeats the new “share one empty frame” optimization and causes many unnecessary allocations; it also makes topology-only empties inconsistent with the_EMPTY_DATAsentinel used elsewhere.
# Some formats keep no timeseries at all on some locations - MIKE 11, for instance,
# stores everything on reach gridpoints, leaving the nodes empty. Asking mikeio1d
# for a dataframe there raises, so return an empty one instead.
if not node.quantities:
return pd.DataFrame()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…tadata A reach's first and last gridpoint sit at the same location as its start and end node, but were read separately and stashed as per-node boundary metadata. That made them a special case: a second loading rule, a dict on every node, and an abstract property every NetworkNode subclass had to implement even when it had nothing to put there. Make them ordinary breakpoints instead. Each is joined to its node by a zero-length edge tagged boundary=True, so the distinction survives in the graph while the loading rules collapse into one — notably, reaches=[] now skips these reads like any other breakpoint. Distances within a chainage tolerance of the reach ends are clamped to an exact 0.0. The two endpoints come from different upstream sources, so floating-point noise could otherwise leave a tiny nonzero weight where the true value is analytically zero. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The old PNG showed boundary nodes, which no longer exist. Redraw it as an SVG so it stays legible when scaled and can be edited as text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Document the zero-length boundary edges and the two remaining NetworkNode properties, and warn that length-weighted networkx algorithms treat a 0.0 edge as a free hop rather than erroring. Drop the boundary property from the subclassing examples and the notebook. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jpalm3r
force-pushed
the
expose-reach-quantities
branch
from
August 11, 2026 09:54
9872826 to
24d4600
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Stacked on #685 (
network-loading) — merge that first. This diff will include #685's changes until then; the description below covers only what this branch adds on top ofnetwork-loading.Summary
Reach-end gridpoint data (e.g. EPANET's
Flow/Velocity, or MIKE's reach-endWaterLevel, which is a distinct series from the node's own) was previously held in aboundarydict onNetworkNodethat nothing ever read. This PR removesboundaryand instead promotes a MIKE reach's first/last gridpoint to an ordinary breakpoint, so that data is reachable throughfind()/recall()/to_dataframe()like any other break point.Res1DReachnow includes a reach's own start/end gridpoint in its breakpoints (previously only the intermediate ones), gated bypopulate_gridpointslike every other breakpoint —reaches=[]now consistently means "no gridpoint data at all," including these._generate_graphconnects them with a zero-length edge, taggedboundary=Trueand clamped to exactly0.0(the raw value is a difference between two independently-sourced numbers, so floating-point noise could otherwise leave a tiny negative weight and break shortest-path algorithms).NetworkNode.boundary(andBasicNode/Res1DNode'sboundaryparam) are removed — a breaking change for any existingNetworkNodesubclass._build_dataframe(25 failed / 29 errors on this branch beforehand) — it turned out not to be needed, since breakpoints already flow into the dataframe through the same mechanism as any other graph node.boundaryare updated accordingly.Deliberately out of scope: EPANET. Its reaches have at most one synthetic gridpoint belonging to neither end, so the existing gate (
len(reach.gridpoints) > 2) already excludes them — EPANET keeps today's behavior (no breakpoints,length=None) unchanged.Closes #599.
Related to #680, which stays open: this only reaches MIKE's reach-end data. EPANET's single-gridpoint case needs its own placement rule and is left for a follow-up.
Test plan
uv run pytest tests/— 814 passed, 6 skippeduv run ruff check/uv run mypyon all touched filesnetwork.res1d/network_cali.res11fixtures