Support explicit_chunk for mode-choice components - #1088
Conversation
Location, destination, and scheduling components already honor the `explicit_chunk` setting (a fixed chunk size, used when chunk_training_mode: explicit), but mode-choice components did not: the setting was never plumbed from the component settings into simple_simulate's chunk loop. This wires it through: add `explicit_chunk` to TemplatedLogitComponentSettings (the base for mode-choice settings), and pass it via mode_choice_simulate -> simple_simulate -> adaptive_chunked_choosers. Defaults to 0 (unchanged behavior). This lets tour_mode_choice and trip_mode_choice run with a fixed, bounded per-chunk memory footprint, which is what makes large samples fit on memory-constrained machines under chunk_training_mode: explicit.
There was a problem hiding this comment.
Pull request overview
This PR extends ActivitySim’s existing explicit_chunk chunking control to mode-choice components, allowing deterministic, bounded chooser chunk sizes during tour/trip mode choice simulation (via chunk_training_mode: explicit).
Changes:
- Add
explicit_chunktoTemplatedLogitComponentSettingsso mode-choice settings can declare fixed chunk sizing. - Add/forward
explicit_chunk_sizethroughmode_choice_simulate(...)andsimple_simulate(...)intochunk.adaptive_chunked_choosers(...). - Wire
model_settings.explicit_chunkinto tour and trip mode choice simulation calls.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| activitysim/core/simulate.py | Adds explicit_chunk_size to simple_simulate and forwards it to the chooser chunking generator. |
| activitysim/core/configuration/logit.py | Adds explicit_chunk to TemplatedLogitComponentSettings so templated/segmented logit components (incl. mode choice) can configure fixed chunking. |
| activitysim/abm/models/util/mode.py | Threads explicit_chunk_size through mode_choice_simulate and passes model_settings.explicit_chunk from tour mode choice runner. |
| activitysim/abm/models/trip_mode_choice.py | Passes model_settings.explicit_chunk into mode_choice_simulate for trip mode choice. |
Comments suppressed due to low confidence (1)
activitysim/abm/models/util/mode.py:43
- The mode_choice_simulate docstring parameter list is now out of sync with the function signature: it still mentions
chunk_size(which is not a parameter) and doesn't documentexplicit_chunk_sizeor the updated optional type forcompute_settings. This can mislead callers when enabling explicit chunking.
explicit_chunk_size: float = 0,
):
"""
common method for both tour_mode_choice and trip_mode_choice
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hi @jpn-- , I wonder if you can check this PR. If everything is good, can it be merged? |
Remove the stale `chunk_size` entry (not a parameter), mark compute_settings as optional, and document the new explicit_chunk_size argument. Addresses the automated review note on PR ActivitySim#1088.
Remove the stale `chunk_size` entry (not a parameter), mark compute_settings as optional, and document the new explicit_chunk_size argument. Addresses the automated review note on PR ActivitySim#1088. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
779fc91 to
706bc39
Compare
|
Synced the |
Description
What
Plumb the existing
explicit_chunksetting through to the mode-choice components(
tour_mode_choice,trip_mode_choice), matching how location, destination, and schedulingcomponents already use it.
Why
explicit_chunk(a fixed per-chunk chooser count, active underchunk_training_mode: explicit) lets acomponent run with a deterministic, bounded memory footprint. Location/destination/scheduling
components honor it via
LocationComponentSettings.explicit_chunk, but mode-choice components neverreceived it — the field didn't exist on their settings class and
mode_choice_simulate/simple_simulateneither accepted nor forwarded it.On memory-constrained machines the adaptive chunker grows each chunk toward its memory budget, so peak
RSS is roughly constant regardless of sample size — which can OOM large models even after subsampling.
Fixed
explicit_chunksizes bound the peak. This change makes that strategy usable for mode choice too;for us it was the last unchunkable core step when fitting a full-sample run (8.1M trips, 7787 zones) onto
a 64 GB node.
What changed (backward-compatible — default
0= unchanged behavior)TemplatedLogitComponentSettings: newexplicit_chunk: float = 0field (the base for mode-choicesettings; mirrors the field already on
LocationComponentSettings).mode_choice_simulate(...): newexplicit_chunk_sizeparameter, forwarded tosimple_simulate.run_tour_mode_choice_simulate(...)andtrip_mode_choice(...): passexplicit_chunk_size=model_settings.explicit_chunk.simple_simulate(...): newexplicit_chunk_sizeparameter, forwarded tochunk.adaptive_chunked_choosers.4 files changed, 16 insertions(+), 1 deletion(-).
Usage
Testing
Verified on a full-sample run (8.1M trips, 7787 zones) on a 64 GB node: tour/trip mode choice ran with a
bounded per-chunk memory footprint under
explicit_chunk: 5000; with the default (0) behavior isidentical to before.
Acknowledgement
This work was supported by the XCarcity project. https://xcarcity.nl