diff --git a/.github/workflows/full_test.yml b/.github/workflows/full_test.yml index bd62ad2b4..7881909ca 100644 --- a/.github/workflows/full_test.yml +++ b/.github/workflows/full_test.yml @@ -10,7 +10,7 @@ jobs: lint: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v4 - uses: astral-sh/ruff-action@v2 with: version: 0.6.2 @@ -24,7 +24,7 @@ jobs: pandas-version: ["pandas2", "pandas3"] # TODO: drop pandas2 once 3.x is well-established steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v4 - uses: extractions/setup-just@v3 @@ -58,7 +58,7 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v4 - uses: extractions/setup-just@v3 diff --git a/.gitignore b/.gitignore index b38653822..318f22d3b 100644 --- a/.gitignore +++ b/.gitignore @@ -153,6 +153,4 @@ docs/_site/ docs/_extensions/ docs/api/*.qmd -uv.lock - -tests/testdata/confidential/* \ No newline at end of file +uv.lock \ No newline at end of file diff --git a/adr/012-network-format-constructors.md b/adr/012-network-format-constructors.md deleted file mode 100644 index 7e1785027..000000000 --- a/adr/012-network-format-constructors.md +++ /dev/null @@ -1,77 +0,0 @@ -# ADR-012: One Network Constructor per Modelling Product - -**Status**: Draft - -**Date**: 2026-08 - -## Context - -`Network` is built from result files read through mikeio1d. Its single `Res1D` class opens nine extensions across five products — MIKE 1D (`.res1d`), MIKE 11 (`.res11`), MOUSE (`.prf`, `.crf`, `.xrf`), EPANET (`.res`), SWMM (`.out`), Water Hammer (`.whr`), and `.resx`, which is shared by the last three. There is no per-format reader and no per-format constructor argument, so from mikeio1d's side all nine look alike. - -modelskill's constructor was named `from_res1d`, and its extension guard was briefly widened to accept everything mikeio1d could read. That made the name misleading: it promised one format and read nine. - -Loading each of mikeio1d's own fixtures showed the nine are not interchangeable: - -- `.res1d` and `.res11` produce a full network with real reach lengths and gridpoints. `.res11` initially failed because MIKE 11 keeps its timeseries on reach gridpoints, leaving nodes with no quantities at all — a bug in modelskill's adapter, now fixed. -- `.res` (EPANET) loads, but as a link-node model it reports no reach length and one synthetic gridpoint per reach. mikeio1d signals the missing length by returning `0`, which is indistinguishable from a genuine zero. -- `.out` (SWMM) and `.resx` expose no reach start/end nodes. Looking closer, this is not an upstream API gap: the raw `StartNodeIndex` is `-1` on every reach, node coordinates are `nan`, and there are no chainages, so the connectivity is absent from the files themselves. It lives in a *companion* file — SWMM's `.inp` input file, and, for `.resx`, the sibling `.res` that defines the network the `.resx` adds results to. -- MOUSE and `.whr` have no test fixture anywhere, including in mikeio1d's testdata, so nothing about them can be verified. - -## Decision - -Name constructors after the product that writes the file, and only ship one where a committed fixture backs it: - -| Constructor | Extensions | -|---|---| -| `Network.from_mike` | `.res1d`, `.res11` | -| `Network.from_epanet` | `.res`, plus optional `.resx` and `.inp` companions | - -`from_res1d` is removed. It only ever shipped in the 1.4.0a3 alpha, and the network module is opt-in and absent from the API reference, so a deprecation shim would have added a second name for the tested path without protecting a real caller. - -Every extension mikeio1d can read is accounted for in one of three module-level tables in `network.py` — readable by `from_mike`, readable by `from_epanet`, or refused with a specific reason. A test asserts the three cover exactly `Res1D.get_supported_file_extensions()`, so a mikeio1d release that adds a tenth format fails CI and forces a decision rather than leaving the format silently unreachable. - -Four supporting rules: - -- **A constructor takes its product's companion files as arguments, not as separate constructors.** A product may write several files for one run, and the result file is not always the whole picture. `from_epanet(res, resx=..., inp=...)` reflects that: the `.resx` adds node quantities and the `.inp` adds the reach lengths that no result file carries. The alternative — a `from_resx()` — would have been wrong, since a companion file describes a network defined elsewhere and cannot stand on its own. A companion is validated against the main file (same time axis, no unknown IDs) rather than merged on trust, because two runs would line up silently and produce a network that nothing downstream would flag. -- **A constructor requires a fixture.** Naming a product in the API is a support claim; it should be backed by a test that builds a `Network` from a real file of that product. MOUSE and Water Hammer are refused today for exactly this reason, and each becomes a six-line addition once a redistributable fixture exists. -- **A refusal names the file that would lift it.** "mikeio1d does not expose reach start/end nodes" was accurate about our code path but wrong about the cause, and it left users with nothing to do. The `.out` refusal now names the companion `.inp` and the issue tracking it; the `.resx` refusal names `from_epanet(res, resx=...)`, which is a next action rather than a dead end. -- **Degenerate results are documented, not warned about.** EPANET's undefined reach lengths and absent breakpoints are stated in the `from_epanet` docstring and the user guide, and asserted in tests. A runtime warning would fire on correct usage and teach users to filter our warnings, and both consequences already raise where they bite. -- **An unreadable reach length is undefined, not zero.** `NetworkReach.length` is optional and defaults to `None`, and the adapter maps mikeio1d's `0` sentinel onto it. Reporting `0` would assert that an EPANET pipe has no extent, which is false — the length exists, mikeio1d just cannot read it — and it makes a length-weighted graph algorithm treat the reach as free to traverse. With `None`, `networkx` fails instead: shortest-path treats the edge as unreachable and weight-summing calls raise `TypeError`. Omitting the edge attribute altogether was rejected for the opposite reason, since `networkx` then defaults the weight to `1`. Nothing inside modelskill reads the length, so this only affects `Network.graph`; matching and extraction work from break point distances. - -## Alternatives Considered - -**One constructor per extension** — `from_res`, `from_out`, `from_whr` say nothing about the product they belong to, and MOUSE would need three identical methods. - -**A generic catch-all (`from_file` or `from_mikeio1d`)** — a second way to do the same thing. With every extension either read or explicitly refused, the catch-all's only remaining job is forward compatibility with formats mikeio1d adds later, which the drift test handles more usefully by demanding a decision. - -**Keep `from_res1d` permissive** — preserves the misleading name, and a constructor that accepts everything cannot tell an EPANET user which method to reach for instead. - -**Ship all five product constructors regardless of coverage** — MOUSE and Water Hammer would be unverifiable, so the method list would stop being a reliable statement of what works. SWMM is a different case: its `.inp` does carry the missing topology and a fixture for it exists upstream, so it is deferred rather than impossible ([#689](https://github.com/DHI/modelskill/issues/689)). - -**A `from_resx()` constructor, or a caller-supplied edge list** — both were considered while working out what `.resx` and `.out` needed. A `from_resx()` cannot work, because a companion file describes a network defined elsewhere. An edge-list argument for supplying topology by hand was redundant: `NetworkReach` is already an abstract base class, so that path exists without new API. - -## Consequences - -Positive: - -- The method list is the format list; `Network.from_` answers "which formats does this read". -- Refusals name the file that would lift them, so a user has somewhere to go. -- Passing a file the other constructor handles raises a `ValueError` naming that constructor. -- One private implementation (`Network._from_mikeio1d`) does the version guard, extension validation, `Res1D` construction, node/reach filtering and companion-file handling, so a new product constructor is a docstring and one call. -- EPANET networks get real edge lengths and two more node quantities, and the `.inp` reader added for it (`model/adapters/_inp.py`) is the same one SWMM support will need, since the two products share the `.inp` layout. - -Negative: - -- MIKE 11 is covered by a fixture but has no field-tested usage behind it yet. -- MOUSE and Water Hammer are refused even though mikeio1d may well read them correctly. This is deliberate: refusing with a reason is recoverable, while a method that silently produces a wrong graph is not. -- The `.inp` reader is ours to maintain. mikeio1d does not read `.inp` at all, and pulling in `wntr` or `swmmio` for two sections each would weigh more than the parser does (ADR-010). The cost is that an unusual `.inp` dialect is our bug to fix. -- `.resx` merges node quantities only. Its reach-level quantities need a data location on single-gridpoint reaches, which is #680. - -## Relationship to ADR-009 - -[ADR-009](009-factory-pattern.md) argues for auto-detecting entry points such as `model_result()` and `observation()` so users need not know the class hierarchy. That is not in tension with this decision. Auto-detection resolves *which class* to build from the shape of the data; these constructors resolve *which product wrote the file*, which is information the call site should state rather than have guessed — particularly when the answer decides whether reach-based matching will work at all. - -## See Also - -- [ADR-010](010-optional-domain-dependencies.md) — why mikeio1d is an optional dependency -- `tests/testdata/README.md` — provenance of the result fixtures diff --git a/adr/README.md b/adr/README.md index 59b6d3bf2..eeff80d44 100644 --- a/adr/README.md +++ b/adr/README.md @@ -30,7 +30,6 @@ Each ADR follows this structure: - [ADR-009](009-factory-pattern.md) - Factory pattern for type detection - [ADR-010](010-optional-domain-dependencies.md) - Optional dependencies for domain-specific model types (Draft) - [ADR-011](011-vertical-pre-extracted-columns.md) - VerticalModelResult ingests pre-extracted columns -- [ADR-012](012-network-format-constructors.md) - One Network constructor per modelling product (Draft) ## Contributing diff --git a/docs/user-guide/network.qmd b/docs/user-guide/network.qmd index 56df21b1b..dd8ed055d 100644 --- a/docs/user-guide/network.qmd +++ b/docs/user-guide/network.qmd @@ -132,33 +132,18 @@ Network → NetworkModelResult → match() → Comparer ## Building a Network -You can build a `Network` object by loading it from a supported network result file. Reading these files relies on [mikeio1d](https://github.com/DHI/mikeio1d), so install the `networks` dependency group first. +You can build a `Network` object by loading it from a supported network format. -There is one constructor per product that writes the file: +Currently, the only supported format is `mikeio.Res1D`. -| Constructor | Extensions | Product | -|---|---|---| -| `Network.from_mike` | `.res1d`, `.res11` | MIKE 1D, MIKE 11 | -| `Network.from_epanet` | `.res`, plus optional `.resx` and `.inp` | EPANET | - -The remaining formats mikeio1d can open cannot be turned into a `Network`, and say so when you try: - -| Extension | Why not | -|---|---| -| `.out` (SWMM) | The reach connectivity is not in the `.out` at all — it lives in the companion `.inp` input file, which modelskill does not read yet ([#689](https://github.com/DHI/modelskill/issues/689)). | -| `.resx` | Not a network on its own. It holds extra results for the network defined in the sibling `.res`, so pass it as `from_epanet(res, resx=...)` instead. | -| `.prf`, `.crf`, `.xrf` (MOUSE), `.whr` (Water Hammer) | No test fixture exists for these formats, so support cannot be verified. [Open an issue](https://github.com/DHI/modelskill/issues) if you need one. | +### Res1D file -### From a network result file - -The quickest way to get a `Network` is from the path to a result file: +The quickest way to get a `Network` is from the path to a MIKE 1D result file: ```{python} # | echo: false path_to_res1d = "../../tests/testdata/network.res1d" -path_to_res11 = "../../tests/testdata/network_cali.res11" -path_to_epanet = "../../tests/testdata/epanet.res" path_to_sensor_data_1 = "../../tests/testdata/network_sensor_1.csv" path_to_sensor_data_2 = "../../tests/testdata/network_sensor_2.csv" ``` @@ -166,7 +151,7 @@ path_to_sensor_data_2 = "../../tests/testdata/network_sensor_2.csv" ```{python} from modelskill.network import Network -network = Network.from_mike(path_to_res1d) +network = Network.from_res1d(path_to_res1d) network ``` @@ -176,81 +161,18 @@ or a `mikeio1d.Res1D` that has already been opened: from mikeio1d import Res1D res = Res1D(path_to_res1d) -network = Network.from_mike(res) -``` - -MIKE 11 files work the same way. Note that MIKE 11 keeps its timeseries on reach gridpoints rather than on nodes, so the nodes of such a network carry no data of their own: - -```{python} -Network.from_mike(path_to_res11) -``` - -EPANET results use `from_epanet`: - -```{python} -Network.from_epanet(path_to_epanet) -``` - -#### EPANET companion files - -An EPANET run writes more than one file, and the `.res` is not the whole picture: - -| File | What it adds | -|---|---| -| `.res` | The network and its main timeseries. Required. | -| `.resx` | Extra results — tank volume and pump energy. Merged onto matching nodes. | -| `.inp` | The model input. The only one of the three carrying reach lengths. | - -Pass the companions alongside the result file to get a fuller network: - -```{python} -# | echo: false -path_to_epanet_resx = "../../tests/testdata/epanet.resx" -path_to_epanet_inp = "../../tests/testdata/epanet.inp" -``` - -```{python} -network_epanet = Network.from_epanet( - path_to_epanet, - resx=path_to_epanet_resx, - inp=path_to_epanet_inp, -) -network_epanet -``` - -`Volume` and `Volume Percentage` come from the `.resx`, and the reach lengths from the `.inp`: - -```{python} -sorted( - d["length"] - for *_, d in network_epanet.graph.edges(data=True) - if d["length"] is not None -) +network = Network.from_res1d(res) ``` -::: {.callout-warning} -## EPANET reach geometry is limited - -EPANET is a link-node model, and mikeio1d reports no length and a single synthetic gridpoint for each reach. So for an EPANET network: - -* without `inp=`, every edge of `network.graph` has `length=None`. A length-weighted `networkx` call then fails rather than returning a meaningless number — shortest-path treats the edge as unreachable, and anything that sums the weights raises `TypeError`. With `inp=`, only pumps and valves stay `None`, since `[PIPES]` is the one section carrying lengths -* reaches have no breakpoints, so a `ReachObservation` cannot be matched — use `NodeObservation` instead -* `find(reach=..., distance=)` never resolves; only `distance="start"` and `distance="end"` work - -For the same reason, `resx=` merges node quantities only. Its reach-level quantities — pump energy, efficiency and costs — have no breakpoint to live on, which is tracked in [#680](https://github.com/DHI/modelskill/issues/680). - -Node timeseries, `to_dataframe()`, `to_dataset()`, `find(node=...)` and `recall()` are unaffected. -::: - -A MIKE 1D network contains multiple levels that are unified into a generic network structure as depicted in the image below. The image introduces concepts like _find_, _recall_ and _boundary_ which are explained in the following sections. +A `Res1D` network contains multiple levels that are unified into a generic network structure as depicted in the image below. The image introduces concepts like _find_, _recall_ and _boundary_ which are explained in the following sections. ![How a Res1D file maps to a Network object. Reaches and nodes are re-indexed as integers; boundary nodes expose `find()`/`recall()` round-trip lookups.](../images/res1d_network_mapping.png) #### Selective loading -Large result files can contain thousands of nodes and gridpoints. Loading all of that data into memory is slow and may cause memory issues — especially when you only need the timeseries at a handful of nodes where observations exist. +Large Res1D files can contain thousands of nodes and gridpoints. Loading all of that data into memory is slow and may cause memory issues — especially when you only need the timeseries at a handful of nodes where observations exist. -Both constructors accept the same two optional arguments to restrict what gets loaded: +`from_res1d` accepts two optional arguments to restrict what gets loaded: | Argument | Type | Effect | |---|---|---| @@ -264,7 +186,7 @@ Selective loading only controls **which timeseries are held in memory**. The ful The most memory-efficient setup — useful when you only care about specific junction nodes — is to pass the node IDs you need and skip all intermediate gridpoints with `reaches=[]`: ```{python} -network_subset = Network.from_mike( +network_subset = Network.from_res1d( path_to_res1d, nodes=["78", "46"], reaches=[], @@ -275,7 +197,7 @@ network_subset If you also need gridpoint data along a particular reach, pass its name (or a list of names): ```{python} -network_subset = Network.from_mike( +network_subset = Network.from_res1d( path_to_res1d, nodes=["78", "46"], reaches=["94l1"], @@ -463,9 +385,7 @@ Use `ReachObservation` when your measured quantity is representative of the whol In case you have your network data in a format that is not included in [Building a Network](#building-a-network), you can assemble a `Network` object by subclassing the abstract base classes `NetworkNode` and `NetworkReach`. `NetworkNode` requires three properties: `id`, `data`, and `boundary`. -`NetworkReach` requires four: `id`, `start`, `end`, and `breakpoints`. - -`NetworkReach.length` is optional and defaults to `None`. Reach length matters in some domains (rivers, sewer networks) and not in others (link-node water distribution models), so override it only where a length exists. Where it is left undefined, the reach contributes an edge with `length=None` to `network.graph`, which keeps length-weighted graph algorithms from quietly treating the reach as free. Nothing else in modelskill reads the length — matching and extraction work from break point distances alone. +`NetworkReach` requires five: `id`, `start`, `end`, `length`, and `breakpoints`. The following is a simple implementation example: @@ -532,7 +452,7 @@ class ExampleReach(NetworkReach): ``` ::: {.callout-tip} -The three abstract properties that **every** `NetworkNode` subclass must implement are `id`, `data` and `boundary`. If `boundary` is not relevant for your use case, define the property to return an empty dictionary, as in the example above. Similarly, a `NetworkReach` with no intermediate points can return an empty `breakpoints` list, and one with no meaningful length can leave the `length` property out altogether. +The three abstract properties that **every** `NetworkNode` subclass must implement are `id`, `data` and `boundary`. If `boundary` is not relevant for your use case, define the property to return an empty dictionary, as in the example above. Similarly, a `NetworkReach` with no intermediate points can return an empty `breakpoints` list. ::: diff --git a/notebooks/Collection_systems_network.ipynb b/notebooks/Collection_systems_network.ipynb index 89e9962e0..a79d769fe 100644 --- a/notebooks/Collection_systems_network.ipynb +++ b/notebooks/Collection_systems_network.ipynb @@ -37,7 +37,7 @@ "```python\n", "from modelskill.network import Network\n", "\n", - "network = Network.from_mike(\"path/to/results.res1d\")\n", + "network = Network.from_res1d(\"path/to/results.res1d\")\n", "``` \n", "\n", "### Custom network format\n", @@ -89,7 +89,7 @@ } ], "source": [ - "network = Network.from_mike(\"../tests/testdata/network.res1d\")\n", + "network = Network.from_res1d(\"../tests/testdata/network.res1d\")\n", "network" ] }, diff --git a/roadmap/features/network-models.md b/roadmap/features/network-models.md index fc7a7f96c..b4b586662 100644 --- a/roadmap/features/network-models.md +++ b/roadmap/features/network-models.md @@ -13,7 +13,7 @@ This reduces the effort required to produce quality-assured model deliverables a ## What This Enables -- Load MIKE 1D, MIKE 11 and EPANET simulation results as model results +- Load MIKE 1D simulation results (Res1D files) as model results - Match network model outputs against point observations at specific nodes, reaches, or catchments - Apply the full suite of ModelSkill metrics and visualisations to network model validation - Compare multiple network model scenarios side by side @@ -21,6 +21,4 @@ This reduces the effort required to produce quality-assured model deliverables a ## Current Status -In active development. MIKE 1D, MIKE 11 and EPANET result files can be read today. Integration with ModelSkill's validation workflow is underway. - -MOUSE and Water Hammer results are not read yet: no shareable result file exists for either format, so support cannot be verified. SWMM results cannot be supported until mikeio1d exposes reach connectivity for them. +In active development. Reading of MIKE 1D result files is already supported. Integration with ModelSkill's validation workflow is underway. diff --git a/src/modelskill/model/adapters/_inp.py b/src/modelskill/model/adapters/_inp.py deleted file mode 100644 index 329b2c92a..000000000 --- a/src/modelskill/model/adapters/_inp.py +++ /dev/null @@ -1,109 +0,0 @@ -"""Minimal reader for EPANET and SWMM ``.inp`` input files. - -mikeio1d reads only the binary result formats, so the ``.inp`` that accompanies a -result file has to be parsed here. Both products use the same layout: bracketed -section headers, ``;``-prefixed comments (including the ``;;Name Node1 ...`` -column headers the products write), whitespace-delimited data rows, and blank -lines to ignore. - -Only the sections modelskill needs are interpreted; everything else is kept as -raw fields for a caller to use, or ignored. -""" - -from __future__ import annotations - -from pathlib import Path - - -def read_sections(path: str | Path) -> dict[str, list[list[str]]]: - """Parse an ``.inp`` file into its sections. - - Parameters - ---------- - path : str or Path - Path to an EPANET or SWMM ``.inp`` file. - - Returns - ------- - dict[str, list[list[str]]] - Section name (upper case, without brackets) mapped to its data rows, - each row split into whitespace-delimited fields. Comment-only and blank - lines are dropped, as is any trailing comment on a data row. - - Examples - -------- - >>> sections = read_sections("model.inp") # doctest: +SKIP - >>> sections["PIPES"][0] # doctest: +SKIP - ['10', '10', '11', '3209.544', '304.8', '100', '0', 'Open'] - """ - sections: dict[str, list[list[str]]] = {} - current: list[list[str]] | None = None - - with open(path, "r", encoding="utf-8", errors="replace") as f: - for line in f: - # A comment can trail a data row, so strip it before anything else. - line = line.split(";", 1)[0].strip() - if not line: - continue - - if line.startswith("["): - name = line.strip("[]").strip().upper() - current = sections.setdefault(name, []) - continue - - if current is not None: - current.append(line.split()) - - return sections - - -def read_pipe_lengths(path: str | Path) -> dict[str, float]: - """Read reach lengths from the ``[PIPES]`` section of an EPANET ``.inp``. - - Parameters - ---------- - path : str or Path - Path to an EPANET ``.inp`` file. - - Returns - ------- - dict[str, float] - Pipe ID mapped to its length. Pumps and valves are links too, but carry - no length, so they are absent from the result rather than present with a - placeholder. - - Raises - ------ - ValueError - If the file has no ``[PIPES]`` section, or a row there has too few - fields to read a length from. - - Notes - ----- - ``[PIPES]`` rows are ``ID Node1 Node2 Length Diameter Roughness ...``, so the - length is the fourth field. The units are whatever the model declares in - ``[OPTIONS]``; no conversion is applied. - """ - sections = read_sections(path) - - try: - rows = sections["PIPES"] - except KeyError: - raise ValueError( - f"'{path}' has no [PIPES] section, so it does not look like an " - "EPANET input file. Available sections: " - f"{sorted(sections)}." - ) - - _ID, _LENGTH = 0, 3 - lengths: dict[str, float] = {} - for row in rows: - if len(row) <= _LENGTH: - raise ValueError( - f"Cannot read a pipe length from [PIPES] row {' '.join(row)!r} " - f"in '{path}': expected at least {_LENGTH + 1} fields " - f"(ID, Node1, Node2, Length), got {len(row)}." - ) - lengths[row[_ID]] = float(row[_LENGTH]) - - return lengths diff --git a/src/modelskill/model/adapters/_res1d.py b/src/modelskill/model/adapters/_res1d.py index 567f0d498..d6d391da0 100644 --- a/src/modelskill/model/adapters/_res1d.py +++ b/src/modelskill/model/adapters/_res1d.py @@ -13,12 +13,6 @@ def _simplify_colnames(node: ResultNode | ResultGridPoint) -> pd.DataFrame: # We remove suffixes and indexes so the columns contain only the quantity names - # Some formats keep no timeseries at all on some locations - MIKE 11, for instance, - # stores everything on reach gridpoints, leaving the nodes empty. Asking mikeio1d - # for a dataframe there raises, so return an empty one instead. - if not node.quantities: - return pd.DataFrame() - # The columns in a Res1D dataframe follow the convention "Quantity:Location:Sublocation" # where Location refers to the node id or the reach id followed by the chainage. RES1D_NAME_SEP = ":" @@ -39,44 +33,6 @@ def _simplify_colnames(node: ResultNode | ResultGridPoint) -> pd.DataFrame: return df.rename(columns=renamer_dict).copy() -def _merge_extra_quantities( - base: pd.DataFrame, extra: pd.DataFrame, *, node_id: str -) -> pd.DataFrame: - """Append a companion file's quantities to a node's frame as extra columns. - - Parameters - ---------- - base : pd.DataFrame - The node's frame from the main result file. - extra : pd.DataFrame - The same node's frame from the companion file, sharing its time index. - node_id : str - Node ID, used in error messages. - - Returns - ------- - pd.DataFrame - - Raises - ------ - ValueError - If a quantity appears in both frames. Concatenating would give the node - two columns of the same name, which is the state ``_simplify_colnames`` - already refuses. - """ - if extra.empty: - return base - - overlapping = base.columns.intersection(extra.columns) - if len(overlapping) > 0: - raise ValueError( - f"Node {node_id!r} already has {sorted(overlapping)} in the main " - "result file, so the companion file's copy cannot be merged in." - ) - - return pd.concat([base, extra], axis=1) - - class Res1DNode(NetworkNode): def __init__( self, @@ -128,18 +84,9 @@ def __init__( end_node: Res1DNode, *, populate_gridpoints: bool = True, - length: float | None = None, ): self._id = reach.name - # Must be checked separately: some formats (.resx) report None for both the - # reach and the node, which the identity checks below would let through. - if reach.start_node is None or reach.end_node is None: - raise ValueError( - f"mikeio1d reported no start/end node for reach {reach.name!r}; " - "this result format's topology cannot be represented as a Network." - ) - if start_node.id != reach.start_node: raise ValueError("Incorrect starting node.") if end_node.id != reach.end_node: @@ -151,14 +98,7 @@ def __init__( self._start = start_node self._end = end_node - - # A length read from a companion input file wins, since mikeio1d has none - # to offer for the formats that need one. Otherwise: mikeio1d returns 0 - # when it cannot read a reach length - link-node models such as EPANET - # report this for every reach. Report it as undefined rather than as a - # zero-length reach, which would make length-weighted graph algorithms - # treat the reach as free. The two cases cannot be told apart upstream. - self._length = length if length is not None else (reach.length or None) + self._length = reach.length self._breakpoints: list[ReachBreakPoint] = [ GridPoint( gridpoint.reach_name, @@ -181,7 +121,7 @@ def end(self) -> Res1DNode: return self._end @property - def length(self) -> float | None: + def length(self) -> float: return self._length @property diff --git a/src/modelskill/model/network.py b/src/modelskill/model/network.py index 328c1cdea..17c62035c 100644 --- a/src/modelskill/model/network.py +++ b/src/modelskill/model/network.py @@ -197,7 +197,7 @@ def _extract_node(self, observation: NodeObservation) -> NodeModelResult: raise ValueError( f"Node {node_id} exists in the network topology but its timeseries was not loaded. " f"Re-create the NetworkModelResult with the relevant nodes populated, " - f"e.g. Network.from_mike(path, nodes=[...])." + f"e.g. Network.from_res1d(nodes=[...])." ) return NodeModelResult( diff --git a/src/modelskill/network.py b/src/modelskill/network.py index 3aff11f48..071bc01f6 100644 --- a/src/modelskill/network.py +++ b/src/modelskill/network.py @@ -30,62 +30,6 @@ from .model.adapters._res1d import Res1DReach -_MIKE_EXTENSIONS = frozenset({".res1d", ".res11"}) -_EPANET_EXTENSIONS = frozenset({".res"}) - -_NO_FIXTURE = ( - "{product} results are not supported yet: modelskill has no test fixture for " - "this format, so support cannot be verified. Please open an issue if you need it." -) -# A result file that holds timeseries but no topology of its own. The connectivity -# is in a companion file we do not parse yet. -_TOPOLOGY_IN_COMPANION_FILE = ( - "SWMM '.out' files carry no reach connectivity of their own - it lives in the " - "companion '.inp' input file, which modelskill does not read yet. Tracked in " - "https://github.com/DHI/modelskill/issues/689." -) -# A companion result file: readable, but it describes a network defined elsewhere. -_COMPANION_RESULT_FILE = ( - "'.resx' holds extra EPANET results (tank volume, pump energy) for a network " - "defined in the sibling '.res' file, so it has no topology of its own. Read the " - "'.res' file and pass this one alongside it: " - "Network.from_epanet(res, resx=...)." -) - -# extension -> why modelskill will not read it, even though mikeio1d can -_UNSUPPORTED_EXTENSIONS: dict[str, str] = { - ".out": _TOPOLOGY_IN_COMPANION_FILE, - ".resx": _COMPANION_RESULT_FILE, - ".prf": _NO_FIXTURE.format(product="MOUSE"), - ".crf": _NO_FIXTURE.format(product="MOUSE"), - ".xrf": _NO_FIXTURE.format(product="MOUSE"), - ".whr": _NO_FIXTURE.format(product="Water Hammer"), -} - -# extension -> the constructor that reads it, for "use X instead" errors -_EXTENSION_CONSTRUCTORS: dict[str, str] = { - **{extension: "from_mike" for extension in _MIKE_EXTENSIONS}, - **{extension: "from_epanet" for extension in _EPANET_EXTENSIONS}, -} - - -def _check_file_path_is_str(res: Res1D) -> None: - """Reject a Res1D opened with a path object rather than a string. - - mikeio1d resolves reach topology with ``str.endswith`` on - ``Res1D.file_path``, which raises ``AttributeError`` from deep inside the - load when that attribute is a ``Path``. Fail here instead, where the cause - can be named. - """ - file_path = getattr(res, "file_path", None) - if file_path is not None and not isinstance(file_path, str): - raise TypeError( - f"This Res1D was opened with a {type(file_path).__name__} file_path, " - "which mikeio1d cannot resolve reach topology from. Re-open it as " - "Res1D(str(path)), or pass the path to the constructor directly." - ) - - class NetworkNode(ABC): """Abstract base class for a node in a network. @@ -185,7 +129,7 @@ def data(self) -> pd.DataFrame: @property def distance(self) -> float: - """Along-reach distance of this break point, measured from the start node.""" + """Along-reach distance of this break point (same units as :attr:`NetworkReach.length`).""" return self.id[1] @property @@ -202,31 +146,30 @@ class NetworkReach(ABC): a list of :class:`ReachBreakPoint` objects for intermediate chainage locations. - Subclass this to integrate your own network topology. Four properties + Subclass this to integrate your own network topology. Five properties must be implemented: * :attr:`id` - a unique string identifier for the reach. * :attr:`start` - the upstream/start :class:`NetworkNode`. * :attr:`end` - the downstream/end :class:`NetworkNode`. + * :attr:`length` - total reach length (in the units of your coordinate + system). * :attr:`breakpoints` - list of :class:`ReachBreakPoint` instances ordered by increasing distance from the start node (empty list if none). - :attr:`length` is optional and defaults to ``None``. Reach length matters - in some domains (rivers, sewer networks) and not in others (link-node water - distribution models), so override it only where a length exists. - The concrete helper :class:`BasicReach` is provided for the common case where all data is already available in memory. Examples -------- - Minimal subclass, without a length: + Minimal subclass: >>> class MyReach(NetworkReach): - ... def __init__(self, rid, start_node, end_node): + ... def __init__(self, rid, start_node, end_node, length): ... self._id = rid ... self._start = start_node ... self._end = end_node + ... self._length = length ... @property ... def id(self): return self._id ... @property @@ -234,16 +177,9 @@ class NetworkReach(ABC): ... @property ... def end(self): return self._end ... @property - ... def breakpoints(self): return [] - - Add a :attr:`length` property on top of that when the domain has one: - - >>> class MyMeasuredReach(MyReach): - ... def __init__(self, rid, start_node, end_node, length): - ... super().__init__(rid, start_node, end_node) - ... self._length = length - ... @property ... def length(self): return self._length + ... @property + ... def breakpoints(self): return [] See Also -------- @@ -272,9 +208,10 @@ def end(self) -> NetworkNode: pass @property - def length(self) -> float | None: - """Total length of this reach in network units, or ``None`` if undefined.""" - return None + @abstractmethod + def length(self) -> float: + """Total length of this reach in network units.""" + pass @property @abstractmethod @@ -341,18 +278,14 @@ class BasicReach(NetworkReach): Start node. end : NetworkNode End node. - length : float, optional - Reach length, by default None (undefined). + length : float + Reach length. breakpoints : list[ReachBreakPoint], optional Intermediate break points, by default empty. Examples -------- >>> reach = BasicReach("reach_1", node_a, node_b, length=250.0) - - Where the domain has no reach length, leave it out: - - >>> reach = BasicReach("pipe_1", node_a, node_b) """ def __init__( @@ -360,7 +293,7 @@ def __init__( id: str, start: NetworkNode, end: NetworkNode, - length: float | None = None, + length: float, breakpoints: list[ReachBreakPoint] | None = None, ) -> None: self._id = id @@ -382,7 +315,7 @@ def end(self) -> NetworkNode: return self._end @property - def length(self) -> float | None: + def length(self) -> float: return self._length @property @@ -416,20 +349,19 @@ def __repr__(self) -> str: return "\n".join(out) @classmethod - def from_mike( + def from_res1d( cls, res: str | Path | Res1D, *, nodes: str | list[str] | None = None, reaches: str | list[str] | None = None, ) -> Network: - """Create a Network from a MIKE 1D or MIKE 11 result file. + """Create a Network from a Res1D file or object. Parameters ---------- res : str, Path or Res1D - Path to a ``.res1d`` or ``.res11`` file, or an already-opened - :class:`mikeio1d.Res1D` object. + Path to a .res1d file, or an already-opened :class:`mikeio1d.Res1D` object. nodes : str, list of str, or None, optional Controls which nodes have their timeseries data loaded into memory. @@ -453,24 +385,17 @@ def from_mike( ------- Network - Raises - ------ - NotImplementedError - If the file extension is not one modelskill can read. - ValueError - If the extension belongs to another constructor, such as EPANET. - Examples -------- Load everything (default behaviour): >>> from modelskill.network import Network - >>> network = Network.from_mike("model.res1d") + >>> network = Network.from_res1d("model.res1d") Load data only for the two nodes where observations exist, and skip all intermediate gridpoint data to keep memory usage low: - >>> network = Network.from_mike( + >>> network = Network.from_res1d( ... "model.res1d", ... nodes=["node_a", "node_b"], ... reaches=[], @@ -478,155 +403,13 @@ def from_mike( Load data for selected nodes and gridpoints for one specific reach: - >>> network = Network.from_mike( + >>> network = Network.from_res1d( ... "model.res1d", ... nodes=["node_a", "node_b"], ... reaches=["reach_1"], ... ) - - Notes - ----- - MIKE 11 keeps its timeseries on reach gridpoints rather than on nodes, - so the nodes of a ``.res11`` network carry no data of their own. Pass - ``reaches`` rather than ``nodes`` to control what gets loaded. - - See Also - -------- - from_epanet : Read an EPANET result file. - """ - return cls._from_mikeio1d( - res, - nodes=nodes, - reaches=reaches, - allowed=_MIKE_EXTENSIONS, - caller="from_mike", - ) - - @classmethod - def from_epanet( - cls, - res: str | Path | Res1D, - *, - resx: str | Path | Res1D | None = None, - inp: str | Path | None = None, - nodes: str | list[str] | None = None, - reaches: str | list[str] | None = None, - ) -> Network: - """Create a Network from an EPANET result file and its companions. - - An EPANET run writes up to three files that modelskill can use. The - ``.res`` holds the network and its main timeseries; the optional - ``.resx`` holds extra results; and the optional ``.inp`` is the input - file, which is the only one of the three carrying reach lengths. - - Parameters - ---------- - res : str, Path or Res1D - Path to a ``.res`` file, or an already-opened - :class:`mikeio1d.Res1D` object. - resx : str, Path, Res1D or None, optional - Companion ``.resx`` file from the same run. Its extra node - quantities (tank ``Volume`` and ``Volume Percentage``) are merged - onto the matching nodes. By default None, and those quantities are - simply absent. - inp : str, Path or None, optional - EPANET ``.inp`` input file for the same model, read for its - ``[PIPES]`` lengths. By default None, and reach lengths are - undefined. - nodes : str, list of str, or None, optional - Which nodes get their timeseries loaded. See :meth:`from_mike`. - reaches : str, list of str, or None, optional - Which reaches get their gridpoint data loaded. See - :meth:`from_mike`. EPANET results have no intermediate gridpoints, - so this argument has no effect. - - Returns - ------- - Network - - Raises - ------ - NotImplementedError - If the file extension is not one modelskill can read. - ValueError - If the extension belongs to another constructor, such as MIKE, if a - companion file has the wrong extension, or if ``resx`` does not come - from the same run as ``res``. - - Examples - -------- - >>> from modelskill.network import Network - >>> network = Network.from_epanet("model.res") - - With both companions, for real edge lengths and the extra quantities: - - >>> network = Network.from_epanet( - ... "model.res", - ... resx="model.resx", - ... inp="model.inp", - ... ) - - Notes - ----- - EPANET is a link-node model, and mikeio1d reports no length and a - single synthetic gridpoint for each of its reaches. As a result: - - * without ``inp``, every edge of :attr:`graph` has ``length=None``, so a - length-weighted graph algorithm fails rather than returning a - meaningless number. Pumps and valves keep ``length=None`` even with - ``inp``, since ``[PIPES]`` is the only section carrying lengths - * reaches have no breakpoints, so - :class:`~modelskill.obs.ReachObservation` cannot be matched against - an EPANET network — use :class:`~modelskill.obs.NodeObservation` - * ``find(reach=..., distance=)`` never resolves; only - ``distance="start"`` and ``distance="end"`` work - - For the same reason, ``resx`` merges node quantities only. Its - reach-level quantities (pump energy, efficiency and costs) have no - breakpoint to live on, which is tracked in issue #680. - - Node timeseries, :meth:`to_dataframe`, :meth:`to_dataset`, - ``find(node=...)`` and :meth:`recall` are unaffected. - - See Also - -------- - from_mike : Read a MIKE 1D or MIKE 11 result file. """ - return cls._from_mikeio1d( - res, - nodes=nodes, - reaches=reaches, - allowed=_EPANET_EXTENSIONS, - caller="from_epanet", - resx=resx, - inp=inp, - ) - @classmethod - def _from_mikeio1d( - cls, - res: str | Path | Res1D, - *, - nodes: str | list[str] | None, - reaches: str | list[str] | None, - allowed: frozenset[str], - caller: str, - resx: str | Path | Res1D | None = None, - inp: str | Path | None = None, - ) -> Network: - """Shared implementation behind the public ``from_*`` constructors. - - Parameters - ---------- - allowed : frozenset of str - Extensions this constructor accepts. - caller : str - Name of the public method, used in error messages. - resx : str, Path, Res1D or None, optional - Companion result file whose node quantities are merged in. - inp : str, Path or None, optional - Companion input file read for reach lengths. - """ if sys.version_info >= (3, 14): raise NotImplementedError( f"Current version of 'mikeio1d' requires python < 3.14 and {sys.version} is being used." @@ -636,13 +419,12 @@ def _from_mikeio1d( if isinstance(res, (str, Path)): path = Path(res) - cls._validate_extension(path.suffix, allowed=allowed, caller=caller) + if path.suffix.lower() != ".res1d": + raise NotImplementedError( + f"Unsupported file extension '{path.suffix}'. Only .res1d files are supported." + ) res = _Res1D(str(path)) - elif isinstance(res, _Res1D): - _check_file_path_is_str(res) - suffix = Path(res.file_path).suffix - cls._validate_extension(suffix, allowed=allowed, caller=caller) - else: + elif not isinstance(res, _Res1D): raise TypeError( f"Expected a str, Path or Res1D object, got {type(res).__name__!r}" ) @@ -661,144 +443,22 @@ def _from_mikeio1d( else: reaches_list = list(reaches) - extra = None if resx is None else cls._open_companion_result(res, resx) - lengths = None if inp is None else cls._read_companion_lengths(inp) - - list_of_reaches = cls._load_res1d_network( - res, nodes_list, reaches_list, extra=extra, lengths=lengths - ) + list_of_reaches = cls._load_res1d_network(res, nodes_list, reaches_list) return cls(list_of_reaches) - - @staticmethod - def _read_companion_lengths(inp: str | Path) -> dict[str, float]: - """Read reach lengths from a companion ``.inp`` input file.""" - from modelskill.model.adapters._inp import read_pipe_lengths - - path = Path(inp) - if path.suffix.lower() != ".inp": - raise ValueError( - f"Expected an EPANET '.inp' input file, got '{path.suffix}'. " - "This argument reads reach lengths from the model input, not " - "from a result file." - ) - return read_pipe_lengths(path) - - @staticmethod - def _open_companion_result(res: Res1D, resx: str | Path | Res1D) -> Res1D: - """Open and validate a companion ``.resx`` result file. - - Raises - ------ - ValueError - If the extension is not ``.resx``, or if the file does not come from - the same run as ``res``. - """ - from mikeio1d import Res1D as _Res1D - - if isinstance(resx, (str, Path)): - path = Path(resx) - if path.suffix.lower() != ".resx": - raise ValueError( - f"Expected an EPANET '.resx' companion file, got '{path.suffix}'." - ) - extra = _Res1D(str(path)) - elif isinstance(resx, _Res1D): - _check_file_path_is_str(resx) - if Path(resx.file_path).suffix.lower() != ".resx": - raise ValueError( - "Expected an EPANET '.resx' companion file, got " - f"'{Path(resx.file_path).suffix}'." - ) - extra = resx - else: - raise TypeError( - f"Expected a str, Path or Res1D object, got {type(resx).__name__!r}" - ) - - # Merging two different runs would line up silently and produce a network - # that is wrong in a way no later error would reveal. - if not res.time_index.equals(extra.time_index): - raise ValueError( - "The '.resx' companion does not share a time axis with the " - "'.res' file, so the two are not from the same run. Got " - f"{len(extra.time_index)} steps ending {extra.end_time} against " - f"{len(res.time_index)} ending {res.end_time}." - ) - - unknown = set(extra.nodes) - set(res.nodes) - if unknown: - raise ValueError( - f"The '.resx' companion holds nodes {sorted(unknown)} that are " - "absent from the '.res' network, so the two files do not describe " - "the same model." - ) - - return extra - - @staticmethod - def _validate_extension( - suffix: str, *, allowed: frozenset[str], caller: str - ) -> None: - """Check a file extension against mikeio1d and against one constructor. - - Raises - ------ - NotImplementedError - If modelskill cannot read the extension, either because mikeio1d - does not support it or because modelskill does not. - ValueError - If another constructor is the one that reads this extension. - """ - from mikeio1d import Res1D as _Res1D - - extension = suffix.lower() - - # Checked before the supported set below, since these all *are* readable - # by mikeio1d - it is modelskill that cannot use the result. - reason = _UNSUPPORTED_EXTENSIONS.get(extension) - if reason is not None: - raise NotImplementedError(f"Cannot read '{suffix}' files. {reason}") - - supported = _Res1D.get_supported_file_extensions() - if extension not in supported: - readable = sorted(supported - set(_UNSUPPORTED_EXTENSIONS)) - raise NotImplementedError( - f"Unsupported file extension '{suffix}'. " - f"Supported extensions are {readable}." - ) - - if extension not in allowed: - constructor = _EXTENSION_CONSTRUCTORS.get(extension) - if constructor is None: - raise NotImplementedError( - f"File extension '{suffix}' is supported by mikeio1d but is not mapped " - "to a Network constructor in this version of modelskill. " - "Please upgrade modelskill or open an issue." - ) - raise ValueError( - f"Network.{caller}() reads {sorted(allowed)} files, got '{suffix}'. " - f"Use Network.{constructor}() instead." - ) - @staticmethod def _load_res1d_network( res: Res1D, nodes: list[str], reaches: list[str], - *, - extra: Res1D | None = None, - lengths: dict[str, float] | None = None, ) -> list[Res1DReach]: from modelskill.model.adapters._res1d import ( Res1DReach, Res1DNode, - _merge_extra_quantities, _simplify_colnames, ) nodes_set = set(nodes) reaches_set = set(reaches) - lengths = lengths or {} # In order to work with bigger files, we might want to select a subset of nodes and avoid # potential memory issues. For this reason, we create this intermediate step that populates @@ -810,12 +470,6 @@ def _init_node(reach: ResultReach, is_end: bool) -> Res1DNode: if id in nodes_set: node = res.nodes[id] df = _simplify_colnames(node) - # Merged here rather than up front so selective loading still - # decides what is held in memory. - if extra is not None and id in extra.nodes: - df = _merge_extra_quantities( - df, _simplify_colnames(extra.nodes[id]), node_id=id - ) overlapping_gridpoint = reach.gridpoints[gpt_idx] boundary = _simplify_colnames(overlapping_gridpoint) return Res1DNode(id, data=df, boundary={reach.name: boundary}) @@ -828,7 +482,6 @@ def _init_node(reach: ResultReach, is_end: bool) -> Res1DNode: _init_node(reach, False), _init_node(reach, True), populate_gridpoints=reach.name in reaches_set, - length=lengths.get(reach.name), ) for reach in res.reaches.values() ] @@ -838,9 +491,7 @@ def _generate_alias_map(g: nx.Graph) -> dict[str | tuple[str, float], int]: return {g.nodes[id]["alias"]: id for id in g.nodes()} @staticmethod - def _generate_reaches_dict( - reaches: Sequence[NetworkReach], - ) -> dict[str, NetworkReach]: + def _generate_reaches_dict(reaches: Sequence[NetworkReach]) -> dict[str, NetworkReach]: return {r.id: r for r in reaches} @staticmethod @@ -934,17 +585,11 @@ def _generate_graph(reaches: Sequence[NetworkReach]) -> nx.Graph: g0.add_node(bp_key, data=bp.data) g0.add_edge(start_key, bp_keys[0], length=reach.breakpoints[0].distance) - - # Only the final segment needs the total length. Break point - # distances are known even when the total is not, so a reach - # without a length still gets real lengths on every edge but - # this one. - tail_length = ( - None - if reach.length is None - else reach.length - reach.breakpoints[-1].distance + g0.add_edge( + bp_keys[-1], + end_key, + length=reach.length - reach.breakpoints[-1].distance, ) - g0.add_edge(bp_keys[-1], end_key, length=tail_length) # 3) Connect consecutive intermediate breakpoints for i in range(reach.n_breakpoints - 1): diff --git a/tests/notebooks/test_notebooks.py b/tests/notebooks/test_notebooks.py index 8b6fd62dc..e44be8975 100644 --- a/tests/notebooks/test_notebooks.py +++ b/tests/notebooks/test_notebooks.py @@ -8,7 +8,7 @@ _TEST_DIR = os.path.dirname(os.path.abspath(__file__)) PARENT_DIR = os.path.join(_TEST_DIR, "../..") SKIP_LIST = ["Download", "Metocean_track_comparison_global", "Metrics_widget", "Collection_systems_network"] -# We skip Collection_systems_network.ipynb since it uses Network.from_mike() which uses pythonnet and, currently, it does not support python 3.14 +# We skip Collection_systems_network.ipynb since it uses Network.from_res1d() which uses pythonnet and, currently, it does not support python 3.14 def _process_notebook(notebook_filename, notebook_path="notebooks"): diff --git a/tests/test_network.py b/tests/test_network.py index 150f043ba..931860102 100644 --- a/tests/test_network.py +++ b/tests/test_network.py @@ -2,7 +2,6 @@ # ruff: noqa: E402 import sys -from pathlib import Path import pytest pytest.importorskip("networkx") @@ -15,21 +14,10 @@ NetworkModelResult, NodeModelResult, ) -from modelskill.model.adapters._inp import read_pipe_lengths, read_sections -from modelskill.model.adapters._res1d import ( - Res1DNode, - Res1DReach, - _simplify_colnames, -) from modelskill.network import ( Network, BasicNode, BasicReach, - NetworkReach, - ReachBreakPoint, - _EPANET_EXTENSIONS, - _MIKE_EXTENSIONS, - _UNSUPPORTED_EXTENSIONS, ) from modelskill.obs import NodeObservation from modelskill.quantity import Quantity @@ -212,8 +200,7 @@ def test_extract_wrong_observation_type(self, sample_network): obs = ms.PointObservation(df, x=0.0, y=0.0) with pytest.raises( - TypeError, - match="NetworkModelResult supports NodeObservation and ReachObservation", + TypeError, match="NetworkModelResult supports NodeObservation and ReachObservation" ): nmr.extract(obs) @@ -455,7 +442,7 @@ def test_matching_workflow_multiple_nodes(self, sample_network, sample_node_data ) def test_open_res1d(): path_to_file = "./tests/testdata/network.res1d" - network = Network.from_mike(path_to_file) + network = Network.from_res1d(path_to_file) assert network.graph.number_of_nodes() == 259 @@ -464,7 +451,7 @@ def test_open_res1d(): ) def test_extract_reach_observation_happy_path(sample_node_data): path_to_file = "./tests/testdata/network.res1d" - network = Network.from_mike(path_to_file) + network = Network.from_res1d(path_to_file) nmr = NetworkModelResult(network, item="Discharge", name="network_model") obs_data = sample_node_data.rename(columns={"WaterLevel": "Discharge"}) obs = ms.ReachObservation(obs_data, reach="100l1", item="Discharge") @@ -481,12 +468,14 @@ def test_extract_reach_observation_happy_path(sample_node_data): ) def test_extract_reach_observation_non_equivalent_breakpoints_raises(sample_node_data): path_to_file = "./tests/testdata/network.res1d" - network = Network.from_mike(path_to_file) + network = Network.from_res1d(path_to_file) nmr = NetworkModelResult(network, item="Discharge") obs_data = sample_node_data.rename(columns={"WaterLevel": "Discharge"}) obs = ms.ReachObservation(obs_data, reach="113l1", item="Discharge") - with pytest.raises(ValueError, match="Not all data in breakpoints are equivalent"): + with pytest.raises( + ValueError, match="Not all data in breakpoints are equivalent" + ): nmr.extract(obs) @@ -497,7 +486,7 @@ def test_extract_reach_observation_with_reaches_not_populated_raises_valueerror( sample_node_data, ): path_to_file = "./tests/testdata/network.res1d" - network = Network.from_mike(path_to_file, reaches=[]) + network = Network.from_res1d(path_to_file, reaches=[]) nmr = NetworkModelResult(network, item="WaterLevel") obs = ms.ReachObservation(sample_node_data, reach="100l1", item="WaterLevel") @@ -512,7 +501,7 @@ def test_extract_reach_observation_breakpoint_node_missing_raises_valueerror( sample_node_data, ): path_to_file = "./tests/testdata/network.res1d" - network = Network.from_mike(path_to_file) + network = Network.from_res1d(path_to_file) nmr = NetworkModelResult(network, item="Discharge") obs_data = sample_node_data.rename(columns={"WaterLevel": "Discharge"}) baseline_obs = ms.ReachObservation(obs_data, reach="100l1", item="Discharge") @@ -533,13 +522,13 @@ def test_extract_reach_observation_breakpoint_node_missing_raises_valueerror( @pytest.mark.skipif( sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" ) -def test_from_mike_nodes_filter_creates_full_network(): +def test_from_res1d_nodes_filter_creates_full_network(): """When nodes is specified, the full network topology is created.""" path_to_file = "./tests/testdata/network.res1d" - full_network = Network.from_mike(path_to_file) + full_network = Network.from_res1d(path_to_file) selected_nodes = ["1", "108"] - partial_network = Network.from_mike(path_to_file, nodes=selected_nodes) + partial_network = Network.from_res1d(path_to_file, nodes=selected_nodes) # Full topology is preserved assert ( @@ -550,12 +539,12 @@ def test_from_mike_nodes_filter_creates_full_network(): @pytest.mark.skipif( sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" ) -def test_from_mike_nodes_filter_only_selected_have_data(): +def test_from_res1d_nodes_filter_only_selected_have_data(): """When nodes is specified, only selected nodes contain non-empty data.""" path_to_file = "./tests/testdata/network.res1d" selected_nodes = ["1", "108"] - network = Network.from_mike(path_to_file, nodes=selected_nodes, reaches=[]) + network = Network.from_res1d(path_to_file, nodes=selected_nodes, reaches=[]) g = network.graph.copy() n_nodes = network.graph.number_of_nodes() @@ -567,12 +556,12 @@ def test_from_mike_nodes_filter_only_selected_have_data(): @pytest.mark.skipif( sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" ) -def test_from_mike_nodes_single_string(): +def test_from_res1d_nodes_single_string(): """nodes argument accepts a single string (not just a list).""" path_to_file = "./tests/testdata/network.res1d" - full_network = Network.from_mike(path_to_file) + full_network = Network.from_res1d(path_to_file) - network = Network.from_mike(path_to_file, nodes="108", reaches=[]) + network = Network.from_res1d(path_to_file, nodes="108", reaches=[]) g = network.graph.copy() assert g.number_of_nodes() == full_network.graph.number_of_nodes() @@ -589,7 +578,7 @@ def test_dataframe_from_partial_network(): """nodes argument accepts a single string (not just a list).""" path_to_file = "./tests/testdata/network.res1d" selected_nodes = ["108", "101"] - network = Network.from_mike(path_to_file, nodes=selected_nodes, reaches=[]) + network = Network.from_res1d(path_to_file, nodes=selected_nodes, reaches=[]) nodes_in_df = network.to_dataframe().droplevel(axis=1, level=1).columns assert set(nodes_in_df) == set([network.find(n) for n in selected_nodes]) @@ -598,10 +587,10 @@ def test_dataframe_from_partial_network(): @pytest.mark.skipif( sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" ) -def test_from_mike_empty_nodes_and_reaches_keeps_topology_and_empty_outputs(): +def test_from_res1d_empty_nodes_and_reaches_keeps_topology_and_empty_outputs(): path_to_file = "./tests/testdata/network.res1d" - full_network = Network.from_mike(path_to_file) - network = Network.from_mike(path_to_file, nodes=[], reaches=[]) + full_network = Network.from_res1d(path_to_file) + network = Network.from_res1d(path_to_file, nodes=[], reaches=[]) assert network.graph.number_of_nodes() == full_network.graph.number_of_nodes() @@ -616,181 +605,6 @@ def test_from_mike_empty_nodes_and_reaches_keeps_topology_and_empty_outputs(): assert len(ds.data_vars) == 0 -# --------------------------------------------------------------------------- -# Optional reach length -# --------------------------------------------------------------------------- - - -class _StubBreakPoint(ReachBreakPoint): - """Minimal concrete ReachBreakPoint for building reaches by hand.""" - - def __init__(self, reach_id, distance, data=None): - self._id = (reach_id, distance) - self._data = pd.DataFrame() if data is None else data - - @property - def id(self): - return self._id - - @property - def data(self): - return self._data - - -def _two_node_pair(): - time = pd.date_range("2020", periods=3, freq="h") - df = pd.DataFrame({"WaterLevel": [1.0, 1.1, 1.2]}, index=time) - return BasicNode("a", df), BasicNode("b", df.copy()) - - -class TestOptionalReachLength: - """Reach length is undefined in some domains, so it must be omittable.""" - - def test_subclass_may_omit_length(self): - class LengthlessReach(NetworkReach): - def __init__(self, id, start, end): - self._id, self._start, self._end = id, start, end - - @property - def id(self): - return self._id - - @property - def start(self): - return self._start - - @property - def end(self): - return self._end - - @property - def breakpoints(self): - return [] - - a, b = _two_node_pair() - reach = LengthlessReach("r1", a, b) - - assert reach.length is None - assert Network([reach]).graph.number_of_nodes() == 2 - - def test_basic_reach_length_defaults_to_none(self): - a, b = _two_node_pair() - - assert BasicReach("r1", a, b).length is None - - def test_edge_length_is_none_when_undefined(self): - a, b = _two_node_pair() - - network = Network([BasicReach("r1", a, b)]) - - assert [d["length"] for *_, d in network.graph.edges(data=True)] == [None] - - def test_breakpoint_distances_survive_an_undefined_length(self): - """Only the final segment needs the total, so the rest keep real lengths.""" - a, b = _two_node_pair() - breakpoints = [_StubBreakPoint("r1", d) for d in (30.0, 70.0)] - - network = Network([BasicReach("r1", a, b, breakpoints=breakpoints)]) - - lengths = sorted( - (d["length"] for *_, d in network.graph.edges(data=True)), - key=lambda v: (v is None, v), - ) - assert lengths == [30.0, 40.0, None] - - def test_length_weighted_algorithms_fail_loudly(self): - """Storing None keeps networkx honest. - - Omitting the attribute instead would let networkx default the weight to - 1, so every call below would return a plausible but meaningless number. - With None, shortest-path treats the edge as hidden and the arithmetic - consumers raise. - """ - import networkx as nx - - a, b = _two_node_pair() - g = Network([BasicReach("r1", a, b)]).graph - - with pytest.raises(nx.NetworkXNoPath): - nx.shortest_path_length(g, 0, 1, weight="length") - - with pytest.raises(TypeError): - g.size(weight="length") - - def test_known_length_is_unchanged(self): - a, b = _two_node_pair() - breakpoints = [_StubBreakPoint("r1", 40.0)] - - network = Network([BasicReach("r1", a, b, 100.0, breakpoints)]) - - assert sorted(d["length"] for *_, d in network.graph.edges(data=True)) == [ - 40.0, - 60.0, - ] - - -# --------------------------------------------------------------------------- -# Which extensions each constructor accepts, and why the rest are refused -# --------------------------------------------------------------------------- - - -@pytest.mark.skipif( - sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" -) -class TestExtensionPolicy: - @pytest.mark.parametrize("suffix", [".res1d", ".res11", ".RES1D"]) - def test_from_mike_accepts_mike_extensions(self, tmp_path, suffix): - """The file does not exist, so mikeio1d - not the guard - is what complains.""" - with pytest.raises((FileExistsError, FileNotFoundError)): - Network.from_mike(tmp_path / f"network{suffix}") - - def test_error_lists_only_readable_extensions(self): - with pytest.raises(NotImplementedError) as excinfo: - Network.from_mike("network.nc") - - message = str(excinfo.value) - for extension in _MIKE_EXTENSIONS | _EPANET_EXTENSIONS: - assert extension in message - for extension in _UNSUPPORTED_EXTENSIONS: - assert extension not in message - - def test_swmm_refusal_names_the_companion_inp(self): - """A real file, so this fails the day SWMM support lands.""" - with pytest.raises(NotImplementedError, match=r"companion '\.inp'"): - Network.from_mike("./tests/testdata/swmm.out") - - def test_resx_refusal_points_at_the_resx_argument(self): - """'.resx' is a companion, so the message must name what to do instead.""" - with pytest.raises( - NotImplementedError, match=r"from_epanet\(res, resx=\.\.\.\)" - ): - Network.from_mike("./tests/testdata/epanet.resx") - - @pytest.mark.parametrize("suffix", [".prf", ".crf", ".xrf", ".whr"]) - def test_formats_without_a_fixture_are_refused(self, tmp_path, suffix): - with pytest.raises(NotImplementedError, match="no test fixture"): - Network.from_mike(tmp_path / f"network{suffix}") - - def test_every_mikeio1d_extension_is_accounted_for(self): - """A new mikeio1d format must be read or explicitly refused, never ignored.""" - from mikeio1d import Res1D - - accounted_for = ( - _MIKE_EXTENSIONS | _EPANET_EXTENSIONS | set(_UNSUPPORTED_EXTENSIONS) - ) - - assert accounted_for == Res1D.get_supported_file_extensions() - - def test_res1d_opened_with_a_path_is_refused(self): - """mikeio1d calls str.endswith on file_path, so a Path breaks it later on.""" - from mikeio1d import Res1D - - res = Res1D(Path("./tests/testdata/network.res1d")) - - with pytest.raises(TypeError, match="file_path"): - Network.from_mike(res) - - # --------------------------------------------------------------------------- # NodeObservation — alias / breakpoint node forms # --------------------------------------------------------------------------- @@ -971,413 +785,3 @@ def test_match_with_string_alias(self, sample_network, sample_node_data): comparer = ms.match(obs, nmr) assert comparer.n_points > 0 assert "Network_Model" in comparer.mod_names - - -# --------------------------------------------------------------------------- -# Res1D adapter — no mikeio1d required, the adapter is duck-typed -# --------------------------------------------------------------------------- - - -class _StubLocation: - """Stands in for a mikeio1d ResultNode / ResultGridPoint.""" - - def __init__(self, quantities, df=None): - self.quantities = quantities - self._df = df - - def to_dataframe(self): - if self._df is None: - raise AssertionError("to_dataframe() should not be called") - return self._df - - -class TestSimplifyColnames: - def test_location_without_quantities_gives_empty_frame(self): - """MIKE 11 keeps its data on gridpoints, leaving nodes with no quantities.""" - df = _simplify_colnames(_StubLocation(quantities=[])) - - assert df.empty - assert list(df.columns) == [] - - def test_quantity_columns_are_stripped_of_location_suffix(self): - time = pd.date_range("2020", periods=2, freq="h") - raw = pd.DataFrame({"WaterLevel:node_1": [1.0, 2.0]}, index=time) - - df = _simplify_colnames(_StubLocation(quantities=["WaterLevel"], df=raw)) - - assert list(df.columns) == ["WaterLevel"] - - -class _StubReach: - """Stands in for a mikeio1d ResultReach.""" - - def __init__(self, name="r1", start_node="a", end_node="b", length=100.0): - self.name = name - self.start_node = start_node - self.end_node = end_node - self.length = length - self.gridpoints = [] - - -class TestRes1DReachConnectivity: - """Formats that expose no reach connectivity must fail with a clear message.""" - - @pytest.mark.parametrize("missing", ["start_node", "end_node"]) - def test_missing_node_raises(self, missing): - reach = _StubReach(**{missing: None}) - - with pytest.raises(ValueError, match="no start/end node for reach 'r1'"): - Res1DReach(reach, Res1DNode("a"), Res1DNode("b")) - - def test_both_nodes_missing_raises(self): - """.resx reports None for both, which the identity checks alone would allow.""" - reach = _StubReach(start_node=None, end_node=None) - - with pytest.raises(ValueError, match="no start/end node"): - Res1DReach(reach, Res1DNode(None), Res1DNode(None)) # type: ignore[arg-type] - - def test_mismatched_start_node_still_raises(self): - with pytest.raises(ValueError, match="Incorrect starting node"): - Res1DReach(_StubReach(), Res1DNode("wrong"), Res1DNode("b")) - - -class TestRes1DReachLength: - """mikeio1d returns 0 when it cannot read a length; that is not a real zero.""" - - @pytest.mark.parametrize("reported", [0, 0.0]) - def test_zero_becomes_undefined(self, reported): - reach = Res1DReach(_StubReach(length=reported), Res1DNode("a"), Res1DNode("b")) - - assert reach.length is None - - def test_real_length_passes_through(self): - reach = Res1DReach(_StubReach(length=47.5), Res1DNode("a"), Res1DNode("b")) - - assert reach.length == 47.5 - - -# --------------------------------------------------------------------------- -# from_mike / from_epanet -# --------------------------------------------------------------------------- - -requires_mikeio1d = pytest.mark.skipif( - sys.version_info >= (3, 14), reason="mikeio1d requires Python < 3.14" -) - - -@requires_mikeio1d -class TestFromMike: - def test_res1d(self): - network = Network.from_mike("./tests/testdata/network.res1d") - - assert network.graph.number_of_nodes() == 259 - - def test_res11(self): - """MIKE 11 keeps its data on gridpoints, so its nodes are empty.""" - network = Network.from_mike("./tests/testdata/network_cali.res11") - - assert len(network._reaches) == 3 - assert network.graph.number_of_nodes() == 71 - assert set(network.quantities) == {"Discharge", "Water Level"} - assert [r.n_breakpoints for r in network._reaches.values()] == [23, 21, 23] - - def test_res11_reaches_have_real_lengths(self): - network = Network.from_mike("./tests/testdata/network_cali.res11") - - lengths = [d["length"] for *_, d in network.graph.edges(data=True)] - assert all(length > 0 for length in lengths) - - def test_open_res1d_object(self): - from mikeio1d import Res1D - - res = Res1D("./tests/testdata/network.res1d") - - network = Network.from_mike(res, nodes=[], reaches=[]) - - assert network.graph.number_of_nodes() == 259 - - def test_epanet_file_is_redirected(self): - with pytest.raises(ValueError, match=r"Use Network\.from_epanet\(\)"): - Network.from_mike("./tests/testdata/epanet.res") - - def test_unknown_extension(self): - with pytest.raises(NotImplementedError, match="Unsupported file extension"): - Network.from_mike("./tests/testdata/obs.dfs0") - - def test_unsupported_type(self): - with pytest.raises(TypeError, match="Expected a str, Path or Res1D object"): - Network.from_mike(42) # type: ignore[arg-type] - - -@requires_mikeio1d -class TestFromEpanet: - def test_epanet(self): - network = Network.from_epanet("./tests/testdata/epanet.res") - - assert network.graph.number_of_nodes() == 11 - assert len(network._reaches) == 13 - assert set(network.quantities) == { - "Demand", - "Head", - "Pressure", - "WaterQuality", - } - assert not network.to_dataframe().empty - - def test_link_node_reaches_have_no_length_or_breakpoints(self): - """Without inp=, mikeio1d reports neither - documented in the docstring.""" - network = Network.from_epanet("./tests/testdata/epanet.res") - - lengths = [d["length"] for *_, d in network.graph.edges(data=True)] - assert lengths and all(length is None for length in lengths) - assert all(r.n_breakpoints == 0 for r in network._reaches.values()) - - def test_reach_observation_cannot_be_matched(self, sample_node_data): - """Follows from having no breakpoints; also documented in the docstring.""" - network = Network.from_epanet("./tests/testdata/epanet.res") - nmr = NetworkModelResult(network, item="Pressure") - obs = ms.ReachObservation(sample_node_data, reach="10", item="WaterLevel") - - with pytest.raises(ValueError, match="breakpoints"): - nmr.extract(obs) - - def test_mike_file_is_redirected(self): - with pytest.raises(ValueError, match=r"Use Network\.from_mike\(\)"): - Network.from_epanet("./tests/testdata/network.res1d") - - def test_open_res1d_object_is_validated(self): - from mikeio1d import Res1D - - res = Res1D("./tests/testdata/network.res1d") - - with pytest.raises(ValueError, match=r"Use Network\.from_mike\(\)"): - Network.from_epanet(res) - - @pytest.mark.parametrize("suffix", [".res", ".RES"]) - def test_extension_is_case_insensitive(self, tmp_path, suffix): - with pytest.raises((FileExistsError, FileNotFoundError)): - Network.from_epanet(tmp_path / f"network{suffix}") - - -# --------------------------------------------------------------------------- -# EPANET companion files: .inp for reach lengths, .resx for extra quantities -# --------------------------------------------------------------------------- - -_EPANET_RES = "./tests/testdata/epanet.res" -_EPANET_RESX = "./tests/testdata/epanet.resx" -_EPANET_INP = "./tests/testdata/epanet.inp" - -# The 12 [PIPES] entries; reach "9" is the pump, which carries no length. -_PUMP_REACH = "9" - - -@requires_mikeio1d -class TestEpanetCompanionInp: - """`.inp` is the only one of the three files carrying reach lengths.""" - - def test_pipe_reaches_get_real_lengths(self): - network = Network.from_epanet(_EPANET_RES, inp=_EPANET_INP) - - lengths = {r.id: r.length for r in network._reaches.values()} - assert lengths["10"] == pytest.approx(3209.544) - assert lengths["110"] == pytest.approx(60.96) - - def test_pump_reach_stays_undefined(self): - """[PIPES] is the only section with lengths, so pumps keep None.""" - network = Network.from_epanet(_EPANET_RES, inp=_EPANET_INP) - - lengths = {r.id: r.length for r in network._reaches.values()} - assert lengths[_PUMP_REACH] is None - assert sum(v is None for v in lengths.values()) == 1 - - def test_graph_edges_carry_the_lengths(self): - network = Network.from_epanet(_EPANET_RES, inp=_EPANET_INP) - - lengths = [d["length"] for *_, d in network.graph.edges(data=True)] - assert sum(v is not None for v in lengths) == 12 - - def test_node_ids_overlapping_reach_ids_are_not_confused(self): - """Most IDs here name both a node and a reach, e.g. '9', '10', '21'.""" - network = Network.from_epanet(_EPANET_RES, inp=_EPANET_INP) - - assert set(network._reaches) & set(network._alias_map) # they do overlap - # Reach "10" is 3209.544 long; node "10" is untouched by the length map. - assert network._reaches["10"].length == pytest.approx(3209.544) - node_10 = network.find(node="10") - assert "Head" in network.to_dataframe()[node_10].columns - - def test_wrong_suffix_is_refused(self): - with pytest.raises(ValueError, match=r"Expected an EPANET '\.inp'"): - Network.from_epanet(_EPANET_RES, inp=_EPANET_RESX) - - def test_file_without_a_pipes_section_is_refused(self, tmp_path): - other = tmp_path / "not-epanet.inp" - other.write_text("[JUNCTIONS]\n;;Name\n9 1000\n") - - with pytest.raises(ValueError, match=r"no \[PIPES\] section"): - Network.from_epanet(_EPANET_RES, inp=other) - - -@requires_mikeio1d -class TestEpanetCompanionResx: - """`.resx` holds extra results for the network defined in the sibling `.res`.""" - - def test_extra_node_quantities_are_merged(self): - network = Network.from_epanet(_EPANET_RES, resx=_EPANET_RESX) - - assert set(network.quantities) == { - "Demand", - "Head", - "Pressure", - "WaterQuality", - "Volume", - "Volume Percentage", - } - - def test_only_the_nodes_present_in_the_resx_gain_them(self): - """The .resx covers the tank and the reservoir, not all eleven nodes.""" - network = Network.from_epanet(_EPANET_RES, resx=_EPANET_RESX) - df = network.to_dataframe() - - with_volume = { - node - for node in df.columns.get_level_values("node").unique() - if "Volume" in df[node].columns - } - # Node IDs are re-indexed to integers, so recall the original labels. - assert {network.recall(node)["node"] for node in with_volume} == {"2", "9"} - - def test_values_come_through(self): - network = Network.from_epanet(_EPANET_RES, resx=_EPANET_RESX) - - reservoir = network.find(node="9") - volume = network.to_dataframe()[(reservoir, "Volume Percentage")] - assert len(volume) == 25 - assert volume.notna().all() - - def test_selective_loading_still_governs_what_is_read(self): - network = Network.from_epanet(_EPANET_RES, resx=_EPANET_RESX, nodes=["2"]) - - df = network.to_dataframe() - tank = network.find(node="2") - assert set(df.columns.get_level_values("node").unique()) == {tank} - assert "Volume" in df[tank].columns - - def test_both_companions_together(self): - network = Network.from_epanet(_EPANET_RES, resx=_EPANET_RESX, inp=_EPANET_INP) - - assert "Volume" in network.quantities - assert network._reaches["10"].length == pytest.approx(3209.544) - - def test_an_open_res1d_object_is_accepted(self): - from mikeio1d import Res1D - - network = Network.from_epanet(_EPANET_RES, resx=Res1D(_EPANET_RESX)) - - assert "Volume" in network.quantities - - def test_wrong_suffix_is_refused(self): - with pytest.raises(ValueError, match=r"Expected an EPANET '\.resx'"): - Network.from_epanet(_EPANET_RES, resx=_EPANET_RES) - - def test_a_result_file_of_another_format_is_refused(self): - from mikeio1d import Res1D - - other = Res1D("./tests/testdata/network.res1d") - - with pytest.raises(ValueError, match=r"Expected an EPANET '\.resx'"): - Network.from_epanet(_EPANET_RES, resx=other) - - def test_a_companion_from_another_run_is_refused(self, monkeypatch): - """Merging two runs would line up silently and give a wrong network.""" - from mikeio1d import Res1D - - res = Res1D(_EPANET_RES) - resx = Res1D(_EPANET_RESX) - shifted = resx.time_index + pd.Timedelta("1D") - - # Both objects share the Res1D class, so shift only this one instance. - original = type(resx).time_index.fget - monkeypatch.setattr( - type(resx), - "time_index", - property(lambda self: shifted if self is resx else original(self)), - ) - - with pytest.raises(ValueError, match="does not share a time axis"): - Network.from_epanet(res, resx=resx) - - def test_a_companion_naming_an_unknown_node_is_refused(self, monkeypatch): - """A node the .res has never heard of means these are different models.""" - from mikeio1d import Res1D - - res = Res1D(_EPANET_RES) - resx = Res1D(_EPANET_RESX) - strangers = dict(resx.nodes) | {"not_in_the_res": None} - - original = type(resx).nodes.fget - monkeypatch.setattr( - type(resx), - "nodes", - property(lambda self: strangers if self is resx else original(self)), - ) - - with pytest.raises(ValueError, match="not_in_the_res"): - Network.from_epanet(res, resx=resx) - - def test_unsupported_type_is_refused(self): - with pytest.raises(TypeError, match="Expected a str, Path or Res1D object"): - Network.from_epanet(_EPANET_RES, resx=42) # type: ignore[arg-type] - - -class TestReadInp: - """Minimal .inp reader - see modelskill/model/adapters/_inp.py.""" - - def _write(self, tmp_path, text): - path = tmp_path / "model.inp" - path.write_text(text) - return path - - def test_sections_are_keyed_without_brackets_and_upper_cased(self, tmp_path): - path = self._write(tmp_path, "[Pipes]\n1 a b 10\n[TANKS]\n2 5\n") - - assert set(read_sections(path)) == {"PIPES", "TANKS"} - - def test_comment_and_blank_lines_are_dropped(self, tmp_path): - path = self._write( - tmp_path, - ";a leading banner\n\n[PIPES]\n" - ";;ID Node1 Node2 Length\n" - ";;-- ----- ----- ------\n" - "1 a b 10\n\n", - ) - - assert read_sections(path) == {"PIPES": [["1", "a", "b", "10"]]} - - def test_trailing_comment_is_stripped_from_a_data_row(self, tmp_path): - path = self._write(tmp_path, "[PIPES]\n1 a b 10 ; the short one\n") - - assert read_sections(path)["PIPES"] == [["1", "a", "b", "10"]] - - def test_rows_before_any_section_are_ignored(self, tmp_path): - path = self._write(tmp_path, "stray row\n[PIPES]\n1 a b 10\n") - - assert read_sections(path) == {"PIPES": [["1", "a", "b", "10"]]} - - def test_lengths_are_read_from_the_fourth_field(self, tmp_path): - path = self._write(tmp_path, "[PIPES]\n1 a b 10.5 300 100\n") - - assert read_pipe_lengths(path) == {"1": 10.5} - - def test_a_short_row_raises_rather_than_dropping_a_length(self, tmp_path): - path = self._write(tmp_path, "[PIPES]\n1 a b\n") - - with pytest.raises(ValueError, match="Cannot read a pipe length"): - read_pipe_lengths(path) - - def test_a_repeated_section_header_accumulates(self, tmp_path): - path = self._write( - tmp_path, "[PIPES]\n1 a b 10\n[TANKS]\n2 5\n[PIPES]\n3 c d 20\n" - ) - - assert read_pipe_lengths(path) == {"1": 10.0, "3": 20.0} diff --git a/tests/testdata/README.md b/tests/testdata/README.md deleted file mode 100644 index a6f4a71c6..000000000 --- a/tests/testdata/README.md +++ /dev/null @@ -1,25 +0,0 @@ -# Test data provenance - -Most files here were produced for modelskill. The exceptions are listed below. - -## From DHI/mikeio1d - -These network files come from -[DHI/mikeio1d](https://github.com/DHI/mikeio1d/tree/main/tests/testdata) -(commit `d937466`), copied unchanged. mikeio1d is MIT-licensed, as is modelskill. - -| File | Format | Used for | -|---|---|---| -| `network_cali.res11` | MIKE 11 | `Network.from_mike` coverage for `.res11` | -| `epanet.res` | EPANET | `Network.from_epanet` coverage | -| `epanet.resx` | EPANET (MIKE+) | the `resx=` companion — extra node quantities merged onto the `.res` network | -| `epanet.inp` | EPANET input | the `inp=` companion — real pipe lengths, which the `.res` does not carry | -| `swmm.out` | SWMM | asserting `.out` is refused — its reach connectivity lives in a companion `.inp` we do not read yet (#689) | - -`epanet.resx` and `epanet.inp` pair with `epanet.res`: same run, same IDs. The -`.resx` node and reach IDs are a strict subset of the `.res` ones, and the `.inp` -`[PIPES]` IDs cover every `.res` reach except the pump. - -`swmm.out` is kept without its `.inp` on purpose. It pins the refusal, so the test -fails the day we add SWMM support or a future mikeio1d starts reporting reach -connectivity for it. diff --git a/tests/testdata/epanet.inp b/tests/testdata/epanet.inp deleted file mode 100644 index 2b42e3a37..000000000 --- a/tests/testdata/epanet.inp +++ /dev/null @@ -1,226 +0,0 @@ -;***************************************************** -;* Generated from MIKE+ * -;***************************************************** - -[TITLE] - -[JUNCTIONS] -;------------------------------------------------------ -;ID Elevation Demand Pattern -;------------------------------------------------------ -10 216.408000 -11 216.408000 -12 213.360000 -13 211.836000 -21 213.360000 -22 211.836000 -23 210.312000 -31 213.360000 -32 216.408000 - -[RESERVOIRS] -;------------------------------------------------------ -;ID Head Pattern -;------------------------------------------------------ -9 243.840000 - -[TANKS] -;--------------------------------------------------------------------------------------------------------- -;ID Elev. Init. Min. Max. Diam. MinVol VolCurve Overflow -; Level Level Level -;--------------------------------------------------------------------------------------------------------- -2 259.080000 36.576000 30.480000 45.720000 15.392400 0.000000 * No - -[PIPES] -;----------------------------------------------------------------- -;ID Head Tail Length Diam Rough. Minor CV -; Node Node -;----------------------------------------------------------------- -10 10 11 3209.544000 457.200000 100.000000 0.000000 -11 11 12 1609.344000 355.600000 100.000000 0.000000 -110 2 12 60.960000 457.200000 100.000000 0.000000 -111 11 21 1609.344000 254.000000 100.000000 0.000000 -112 12 22 1609.344000 304.800000 100.000000 0.000000 -113 13 23 1609.344000 203.200000 100.000000 0.000000 -12 12 13 1609.344000 254.000000 100.000000 0.000000 -121 21 31 1609.344000 203.200000 100.000000 0.000000 -122 22 32 1609.344000 152.400000 100.000000 0.000000 -21 21 22 1609.344000 254.000000 100.000000 0.000000 -22 22 23 1609.344000 304.800000 100.000000 0.000000 -31 31 32 1609.344000 152.400000 100.000000 0.000000 - -[VALVES] -;------------------------------------------------------ -; ID Head Tail Diam Type Setting (Losscoef) -; Node Node -;------------------------------------------------------ - -[PUMPS] -;------------------------------------------------------------------ -;ID Head Tail Properties -; Node Node -;------------------------------------------------------------------ -9 9 10 HEAD 1 - -[VSD_PUMPS] -;------------------------------------------------------------------------------------- -;Pump Node Setpoint Setpoint SetpointType Speed Speed ControlType -;ID ID Value Curve 0/1(pressure/HGL) min max 0/1(downstream node, any node) -;------------------------------------------------------------------------------------- - -[EMITTERS] -;------------------------------------------------------ -; Node Flow Coeff. -; ID -;------------------------------------------------------ - -[DEMANDS] -;------------------------------------------------------------------ -; NodeID Demand Pattern -;------------------------------------------------------------------ -10 0.000000 ;BASE -11 9.463530 ;BASE -12 9.463530 ;BASE -13 6.309020 ;BASE -21 9.463530 ;BASE -22 12.618039 ;BASE -23 9.463530 ;BASE -31 6.309020 ;BASE -32 6.309020 ;BASE - -[PATTERNS] -;ID Multipliers -1 1.000000 -1 1.200000 -1 1.400000 -1 1.600000 -1 1.400000 -1 1.200000 -1 1.000000 -1 0.800000 -1 0.600000 -1 0.400000 -1 0.600000 -1 0.800000 - -[STATUS] -;ID Status/Setting - -[CURVES] -;ID X-Value Y-Value -1 94.635295 76.200000 - -[CONTROLS] -LINK 9 OPEN IF NODE 2 BELOW 33.528000 -LINK 9 CLOSED IF NODE 2 ABOVE 42.672000 - -[RULES] - -[MIXING] -;Tank Model -2 MIXED -9 MIXED - -[QUALITY] -;------------------------------------------------------------------ -;Nodes Initial -;ID quality -;------------------------------------------------------------------ -10 0.500000 -11 0.500000 -12 0.500000 -13 0.500000 -21 0.500000 -22 0.500000 -23 0.500000 -31 0.500000 -32 0.500000 -2 1.000000 -9 1.000000 - -[SOURCES] -;----------------------------------------------- -;NODEID SRCTYPE STRENGTH PATTERN -;----------------------------------------------- - -[REACTIONS] -GLOBAL BULK -0.500000 -GLOBAL WALL -1.000000 -GLOBAL NewBulk 0.000000 0.000000 -ORDER BULK 1.000000 -ORDER WALL 1 -ROUGHNESS CORRELATION 0.000000 - -[ENERGY] -GLOBAL PRICE 0 -GLOBAL EFFIC 75 -DEMAND CHARGE 0 - -[TIMES] -Duration 24:0:0 -Hydraulic Timestep 1:0:0 -Quality Timestep 0:5:0 -Pattern Timestep 2:0:0 -Pattern Start 0:0:0 -Report Timestep 1:0:0 -Report Start 0:0:0 -Start Date 2022:10:13 -Start ClockTime 0:00:00 -STATISTIC NONE - -[REPORT] -;------------------------------------------------------ -STATUS FULL -SUMMARY YES -MESSAGES YES -ENERGY YES -NODES NONE -LINKS NONE - -[OPTIONS] -UNITS LPS -DIFFUSIVITY 1.000000 -HEADLOSS H-W -SPECIFIC GRAVITY 1.000000 -VISCOSITY 1.000000 -TRIALS 40 -TOLERANCE 0.010000 -ACCURACY 0.001000 -Quality NONE -PATTERN 1 -EMITTER EXPONENT 0.500000 -CHECKFREQ 2 -MAXCHECK 10.000000 -DAMPLIMIT 0.000000 -DEMAND MULTIPLIER 1.000000 -UNBALANCED CONTINUE 10 - -[TURBINES] -;------------------------------------------------------ -; ID -;------------------------------------------------------ - -[COORDINATES] -;------------------------------------------------------ -;Node X-coord Y-coord -;ID -;------------------------------------------------------ -10 6.096000 21.336000 -11 9.144000 21.336000 -12 15.240000 21.336000 -13 21.336000 21.336000 -21 9.144000 12.192000 -22 15.240000 12.192000 -23 21.336000 12.192000 -31 9.144000 3.048000 -32 15.240000 3.048000 -2 15.240000 27.432000 -9 3.048000 21.336000 - -[VERTICES] -;------------------------------------------------------ -;Link X-coord Y-coord -;ID -;------------------------------------------------------ - -[END] diff --git a/tests/testdata/epanet.res b/tests/testdata/epanet.res deleted file mode 100644 index dbc080f9a..000000000 Binary files a/tests/testdata/epanet.res and /dev/null differ diff --git a/tests/testdata/epanet.resx b/tests/testdata/epanet.resx deleted file mode 100644 index 327373e02..000000000 Binary files a/tests/testdata/epanet.resx and /dev/null differ diff --git a/tests/testdata/network_cali.res11 b/tests/testdata/network_cali.res11 deleted file mode 100644 index ce51c1db3..000000000 Binary files a/tests/testdata/network_cali.res11 and /dev/null differ diff --git a/tests/testdata/swmm.out b/tests/testdata/swmm.out deleted file mode 100644 index 9801df1b2..000000000 Binary files a/tests/testdata/swmm.out and /dev/null differ