Load-balance, active-box windowing, block-structured AMR - #1628
Open
sbryngelson wants to merge 433 commits into
Open
Load-balance, active-box windowing, block-structured AMR#1628sbryngelson wants to merge 433 commits into
sbryngelson wants to merge 433 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in (“default-off”) family of performance/diagnostic features (load-weight and SFC partition diagnostics, weighted init-time decomposition, rank timing), plus major simulation capabilities (active-box RHS windowing and block-structured AMR) and corresponding post-processing support and documentation/validation updates.
Changes:
- Adds new runtime parameters and toolchain metadata/validation hooks for the experimental performance/AMR feature family.
- Extends the simulation code with new modules for active-box restriction, load-weight diagnostics, SFC partition reporting, rank timing, and AMR integration points (including restart/output plumbing).
- Updates post_process to read/write AMR fine-block overlays and adds/updates golden metadata plus documentation/indexing.
Reviewed changes
Copilot reviewed 82 out of 94 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| toolchain/mfc/params/descriptions.py | Adds user-facing descriptions for new experimental/performance parameters. |
| toolchain/mfc/params/definitions.py | Registers new parameters (AMR, hybrid sensors, load-balance diagnostics) and target applicability. |
| toolchain/mfc/lint_docs.py | Treats new validator checks as non-physics doc checks. |
| tests/F980C769/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/ECABA006/golden-metadata.txt | Adds golden metadata for active-box test coverage. |
| tests/DD4CD8F3/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/CC4213FD/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/BD21A5C0/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/BCBA6E74/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/ACE05393/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/987D9025/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/852CCB81/golden-metadata.txt | Adds golden metadata for AMR-related golden tests. |
| tests/65C375B4/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/4DADE04B/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/454C565F/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/3A474BEE/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/2FC423D3/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/13945217/golden-metadata.txt | Adds golden metadata for moving-IB under AMR test coverage. |
| src/simulation/m_viscous.fpp | Clamps FD coefficient indexing to avoid ghost-region coefficient OOB in IB drag gradient evaluation. |
| src/simulation/m_time_steppers.fpp | Integrates active-box bounds into RK update loops and interleaves AMR fine-stage/subcycle operations. |
| src/simulation/m_start_up.fpp | Wires up new modules (rank timing, active-box, load-weight, SFC partition, AMR) into init/timestep/finalize and restart I/O. |
| src/simulation/m_sfc_partition.fpp | Adds analysis-only SFC tiling + weighted partition prediction and reporting. |
| src/simulation/m_rank_timing.fpp | Adds per-rank wall-time imbalance measurement helpers and reporting. |
| src/simulation/m_load_weight.fpp | Adds per-cell load-weight field construction and rank-level imbalance reporting. |
| src/simulation/m_hypoelastic.fpp | Refactors FD coefficient setup into a callable update routine (supporting AMR grid swaps). |
| src/simulation/m_global_parameters.fpp | Adds AMR working-state mirrors and slot selection helper plus defaults for new parameters. |
| src/simulation/m_data_output.fpp | Adds output/report hooks for load-weight, SFC partition, and rank-time diagnostics. |
| src/simulation/m_checker.fpp | Adds input validation/prohibits for active-box, hybrid sensors, load-balance, and AMR configurations. |
| src/simulation/m_active_box.fpp | Adds active-box initialization/growth and debug envelope checking. |
| src/simulation/m_acoustic_src.fpp | Adds AMR-aware handling of acoustic source support (bounding boxes and overlap abort). |
| src/post_process/m_start_up.fpp | Calls AMR fine-data reader and AMR overlay writer when amr is enabled. |
| src/post_process/m_global_parameters.fpp | Adds default-off amr flag for post_process overlay behavior. |
| src/post_process/m_data_output.fpp | Implements AMR fine-block overlay mesh/variables output (Silo/binary) and multimesh registration. |
| src/common/m_phase_change.fpp | Exposes per-cell Newton iteration count and threads it through relaxation to support load-weighting. |
| src/common/m_global_parameters_common.fpp | Adjusts start_idx lifecycle/allocation and makes load_weight_wrt visible to GPU macros. |
| src/common/m_derived_types.fpp | Introduces a simple t_box type used by new partitioning infrastructure. |
| src/common/m_box.fpp | Adds box/partition arithmetic helpers (equal/weighted splits, box-from-splits). |
| src/common/m_boundary_common.fpp | Skips BC buffer population during AMR fine advance to rely on coarse-driven ghost fill. |
| docs/module_categories.json | Registers new modules under documentation categories. |
| docs/documentation/readme.md | Adds AMR section link to the documentation index. |
| .typos.toml | Adds project-specific abbreviations to the spelling allowlist. |
| D = ((gs_min(lp) - 1.0_wp)*cvs(lp))/((gs_min(vp) - 1.0_wp)*cvs(vp)) | ||
|
|
||
| #ifdef MFC_SIMULATION | ||
| if (relax .and. load_weight_wrt) then |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1628 +/- ##
==========================================
+ Coverage 61.04% 62.13% +1.08%
==========================================
Files 83 93 +10
Lines 20978 25569 +4591
Branches 3099 4186 +1087
==========================================
+ Hits 12807 15888 +3081
- Misses 6126 6967 +841
- Partials 2045 2714 +669 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Member
Author
|
Upstream latent gap found during the MHD+AMR investigation (independent of this PR): Also for the record: MHD+AMR was attempted and re-gated on measured evidence rather than assumption — the coarse/fine seam is a continuous O(1) div(B) source that cleaning spreads but cannot remove (details in the amr.md support matrix row and commit ac203b1). |
…rature side-state coarse<->fine coupling (prolong + restriction, s_amr_prolong_pbmv / s_restrict_pbmv) reads/writes the block owner's LOCAL coarse pb/mv. Under fine-level distribution the SFC-assigned owner need not hold the block's coarse cells, so at np>=2 pb/mv couples to the wrong/stale coarse side-state - a silent wrong answer (q_cons is distributed via the P2P gather/scatter; pb/mv is not yet). Add a checker PROHIBIT (qbmm .and. .not. polytropic .and. num_procs>1) in the amr block of m_checker; remove the now-dead span-only gate in s_amr_compute_isect (it read the mirror isect - overwritten by s_set_amr_fine_geometry under whole-block - and only guarded the initial block, missing regridded ones; the checker gate is stricter and fires first). VALIDATED (isolated worktree, CPU reldebug): np=2 non-poly QBMM AMR aborts with the message; np=1 (8EDFC2F4/BCBA6E74/B0A5D230 physics) runs unchanged. No np>=2 QBMM+AMR golden exists, so nothing regresses. pb/mv gather/scatter = future work when np>=2 QBMM+AMR is needed + testable.
…gression) - s_restrict_fine_to_coarse (owner path) did GPU_UPDATE(host) the fine block -> restrict on HOST (s_amr_restrict_overwrite) -> GPU_UPDATE(device) the WHOLE coarse_tgt%sf. That whole-array push clobbered the device-advanced NON-COVERED coarse cells with the stale host copy: a GPU-only divergence (invisible on CPU where host==device) written straight into the coarse output (cons.1.00). ~1e-10 for stable physics, amplified to O(1e-2) by the IGR Jacobi solve / MHD / acoustic. New s_amr_restrict_overwrite_device (GPU_PARALLEL_LOOP collapse=4) restricts fine(device)->coarse(device) touching ONLY the covered cells - inlines f_amr_restrict_cell EXACTLY (same child-sum order + stp cast) so it is bit-identical to the host path on CPU and matches the coarse restriction. np=1 owner path is now device-native (no fine host-pull, no whole-coarse device push); keeps QBMM + rank_time; num_procs>1 keeps the existing host+P2P path. VALIDATED: gpu-mp -c phoenix full AMR set 42 passed / 2 failed (was ~1/21); the 2 remaining are unrelated (660FFBFE = the separate CPU 1e-9 IGR-regrid regression, now matching CPU; 79B334C7 = np>=2, not touched). CPU reldebug bit-identical: 21C71558/6C20B752(IGR)/73355E90(MHD)/ACE05393(hypoelastic) pass. REMAINING: extend device-native to the np>=2 local restriction (+ the gather) with host only for the cross-rank halo.
…er) - Extend the np=1 fix (d8f1c5c) to the np>=2 restriction scatter. Owner-local covered cells now restrict fine(device)->coarse(device) via s_amr_restrict_overwrite_device instead of s_amr_restrict_overwrite(host)+whole-coarse GPU_UPDATE(device); the receiver pushes ONLY its covered coarse section, not the whole array. Both whole-array device pushes clobbered the device-advanced non-covered coarse cells with a stale host copy - the same GPU-only bug fixed at np=1, invisible on CPU where host==device. Host round-trip remains only for the cross-rank MPI send-slice packing (all-on-gpu-except-halos). VALIDATED (Phoenix V100, gpu mp, clean build): 79B334C7 (np=2 stretched, was FAILING) + 5EFB3277 (np=2 dynamic regrid) + 21C71558 (np=1) all pass; CPU reldebug bit-identical. Also escape %% in amr_fine_distribution.md:133 for the doc-reference lint.
…for tiling - Same class as B7704247: the golden was generated on the mirror decomposition (up/mega 6f518df), which at np=1 CLAMPS a regrid box to amr_maxc_fit (<= half the domain per dim). 660FFBFE's dynamic tag grows to x3:46 (44 coarse cells on the 50-wide grid); the mirror clamps it to x3:27 (25 cells, DIRECTLY OBSERVED: mirror 58c1a0b prints 'block: box x 3:27 (25 coarse cells)') and leaves x28:46 coarse. Block-splitting TILES the box instead into x3:24 + x25:46, refining the FULL tagged region (HEAD prints 4 blocks). So HEAD refines x28:46 the old golden left coarse - the same real, intended more-refinement change, not a bug. EVIDENCE it is benign: (1) the mirror binary (58c1a0b, pre-tiling) PASSES the old golden bit-for-bit; (2) the field diff is spatially CONFINED to x-cells 28:48 = exactly the mirror's clamp-off region, and the left tile x3:24 matches the golden (a solver bug would perturb both tiles); (3) the ~1e-4 conservation defect is INHERENT to this IGR case - the STATIC twin 6C20B752 (single block, no tiling, passes its golden) shows the same-order defect (mass 2.7e-5 vs 7.9e-5), so tiling seams are not breaking conservation; (4) static IGR passes because it never re-tags past amr_maxc. Max diff 3.35e-3 in cons.3 (y-mom). Regenerated release (--no-gpu --no-debug) with the tiled binary; 6C20B752 unchanged.
…roadcast) - The regrid fine-state migration (an old block's stashed fine detail must reach whichever rank now owns a covering new block) was a collective MPI_Bcast of EVERY old block to ALL ranks - O(n_blocks x n_ranks) per regrid, the last global collective in the fine-level coupling. Now POINT-TO-POINT (mirrors s_amr_gather_coarse_patch): each old owner Isends its stashed q_cons_stor only to the DISTINCT new-block owners whose region overlaps that old block (f_amr_boxes_overlap on the replicated coarse boxes); receivers Irecv only the old blocks they need. Moved s_amr_assign_block_owners BEFORE the migration so the new owners are known when choosing destinations. A rank that did not receive old block kk never reads it - the overlap-copy's per-(k,kk) index guard skips every cell of a non-overlapping pair. Fine-level coupling is now fully P2P end to end (gather + restriction + reflux + migration); no per-regrid global collective remains. np=1 bit-identical (migration skipped under num_procs==1; the assign reorder is neutral - nothing between reads amr_block_owner). VALIDATED CPU reldebug -b mpirun: 79B334C7 (stretched np=2 dynamic regrid - the migration-correctness case, reverts to ~1e-4 if a needed block is not delivered) PASSES, plus 5EFB3277 + BD21A5C0 (np=2) and 21C71558 (np=1).
…_amr_restart (parallel_io) required the writing rank count; it now REPARTITIONS. Whole-block ownership makes each block's fine data one contiguous region-sized chunk at a computable offset - the only rank-count dependence in the file layout is the 3*np_old per-block extents record. So: read the writer's np_old from the header, use it for the file-layout strides (blk_base + data offset), re-assign block owners for THIS run via s_amr_assign_block_owners, and each new owner reads its whole blocks (the EXSCAN yields offset 0 for the block's sole current owner). The same-decomposition per-rank layout check is skipped when np_old /= num_procs (it cannot match by design; the end-of-read file-size check still fails closed on truncation/corruption). np_old == num_procs is byte-identical to the previous same-rank path. Serial (non-parallel_io) mode still needs the writing rank count (message now points to parallel_io for repartition). NO write-side change - existing checkpoints are already repartition-readable. VALIDATED CPU reldebug -b mpirun (1D AMR dynamic-regrid, save at step 3, restart 3->6): np=2->np=1 AND np=1->np=2 both print the repartition line, restore the fine level (2 blocks), and conserve (mass 8e-13 / energy 1e-12); 5EFB3277 (np=2 same-rank parallel restart) still passes.
…e_slot (no-op) - Foundation for lazy owned-only slot sizing (right-size a rank's fine memory to ~1/num_procs of the pool instead of all amr_max_blocks slots on every rank). Move the per-slot allocation (coords + the 6 device-resident field vectors + non-poly QBMM side-state) out of the init loop and the matching deallocation out of finalize into s_amr_alloc_slot(islot) / s_amr_free_slot(islot); promote the sizing locals (max_f1/2/3, mbuf1..3_lo/hi) to module scope so the helpers see them; add amr_slot_live(:) to track which slots are allocated. Init/finalize still call the helpers for EVERY slot, and the single QBMM RHS scratch (amr_rhs_pb_f/mv_f) is allocated once outside the per-slot helper - so this commit is bit-identical; the lazy reconcile at regrid/init/restart follows. VALIDATED CPU reldebug -b mpirun: 21C71558 + 6C20B752 (np=1 IGR) + 5EFB3277 (np=2) all pass.
…um_procs - Build on the alloc/free helpers (daa36af): a rank now allocates a slot's per-block field arrays (coords + the 6 device-resident field vectors + non-poly QBMM side-state, all sys_size x block) ONLY for blocks it owns, instead of all amr_max_blocks slots on every rank. s_amr_reconcile_slots allocates owned / frees the rest, called once ownership is known: init (after s_amr_assign_block_owners), restart (serial + parallel, before the owned read), and regrid. The regrid keeps the transient slots live across the critical section - old-owned (for the P2P migration SEND) + newly-owned (for the build) + received-old (alloc'd before the q_cons_stor unpack) - then reconciles to new-owned-only at the end, freeing the rest. Safe because every heavy-array access is ownership-guarded (fill/advance early-return on .not. amr_rank_owns_block), so an unowned/freed slot is never touched; np=1 owns all blocks so it is bit-identical (only the inactive pool slots beyond amr_num_blocks go unallocated). Per-rank fine memory now scales ~1/num_procs instead of num_procs x the data. (IB marker fields are allocated en masse by m_ibm and are per-cell integers, not sys_size x block - left global.) VALIDATED: CPU reldebug np=1 bit-identical (21C71558/6C20B752 IGR), np=2 (79B334C7 stretched regrid+migration, 5EFB3277, BD21A5C0 chem), repartition restart 2->1 conserves (mass 8e-13); GPU mp (Phoenix V100, clean build) 5/5 (21C71558/6C20B752/79B334C7/5EFB3277/660FFBFE) - device present-table consistent through the lazy alloc/free at regrid.
…pped) - The SFC map distributes WHOLE blocks, so with fewer blocks than ranks some ranks own no fine work - the fine-level load balance is capped at min(num_blocks, amr_max_blocks)/num_procs. Surfaced by the np>=4 load-balance demonstration: a concentrated region balanced PERFECTLY (1.00x imbalance, exact conservation) at np=4 (4 blocks) and np=8 (8 blocks with amr_max_blocks=16), but the default amr_max_blocks=4 capped np=8 at 4 blocks -> 2.0x. Add a rank-0 init warning when amr_max_blocks < num_procs pointing the user to raise it. np=1/small runs unaffected (fires only when num_procs > amr_max_blocks).
…-compiler flips The 4 hybrid-sensor golden tests at hybrid_weno_eps=0.5 (BA4340EA, 60739A3E and the AMR 053C5DDA, DDC4BA8A) failed on aggressive-optimizer lanes (intel no-debug, NVHPC 24.1/24.3, Frontier GPU) with percent-level LOCAL diffs. Root cause: the Jameson sensor's per-cell phi = |c+ - 2c0 + c-|/(|c+|+2|c0|+|c-|) is catastrophic cancellation at a shock, so a compiler's FP contraction/reordering jitters phi; where phi sits NEAR eps a cell flips between central and WENO -> a large local change. eps=0.5 sat at the edge of a plateau, ~0.001 from a shock cell at phi~=0.499. Fix is not a tolerance bump but a well-placed threshold. An eps-sweep of each exact case (run at many eps, diff outputs) shows the output is bit-identical across eps in [0.20, 0.40] -> no cell's phi lands in that band on ANY timestep = a clean phi GAP. eps=0.30 is centred there with a +/-0.10 margin: the full cross-compiler SOLUTION drift (~5e-4) moves phi only ~1e-3, ~100x inside the margin, so no compiler can flip a cell. The sensor still fires consequentially (answer moves ~4e-4 vs a dead all-WENO sensor, 7-10x the 5e-5 tol). Same gap holds for weno and riemann modes and on the AMR fine level (all measured). VALIDATED (Phoenix GNU CPU): 4/4 pass vs regenerated goldens under the normal build AND --fastmath (aggressive FP reordering). Robustness is by construction (measured phi gap); the intel/NVHPC-24.x lanes are the final confirmation in CI.
CCE OpenMP-offload never initializes the descriptor of the bare module-scope scalar_field allocatable amr_cg, so allocate(amr_cg) aborts with lib-4425 at startup - crashing every AMR case on gpu-mp/gpu-omp/AMD-flang. Allocate a local (which gets a valid descriptor) and move_alloc it into amr_cg, then map. OpenACC unaffected.
s_find_ghost_points assigns ghost-point slots via GPU atomic-capture (thread-completion order), which is nondeterministic; the order-sensitive surface-force reduction and the discrete image-point stencil then make moving AMR-IB diverge across backends. Sort the list on-device into lexicographic (i,j,k) order - no host round-trip, since a GPU_UPDATE here fails Cray's present-table lookup and this routine runs mid-swap in the AMR fine path. Also drops the routine's dummy argument to use the module-global ghost_points directly. Verified: tests/13945217 fails at 1e-10 without the sort, passes with it.
# Conflicts: # toolchain/mfc/test/cases.py
The AMR fine/coarse ghost-point swap copied the declare-target ghost_points through the host (GPU_UPDATE host -> host park -> GPU_UPDATE device) only because the parking storage ib_fine%gps was a plain host array. An audit confirmed the swap park is the ONLY host access of ghost_points: every consumer (correction kernel, s_find_ghost_points, levelset, image points, interpolation coeffs, moving recompute) runs on-device, the collision-forces ghost_points arg is dead, and there is no cross-rank exchange. Replace ib_fine%gps with a device-resident gp_park(:,:) and swap via on-device kernels, eliminating the host round-trip. gp_park is mapped dynamically with move_alloc + GPU_ENTER_DATA (the amr_cg idiom) so the bare derived-type allocatable gets a valid descriptor and device mapping without the CCE OpenMP-offload lib-4425 uninitialized-descriptor abort; a direct GPU_DECLARE + @:ALLOCATE would corrupt a sibling descriptor. ghost_point is trivially copyable, so the device struct assignment is well-defined. This supersedes the ghost_points(1:num_gps) slice band-aid: removing the whole-array ghost_points GPU_UPDATE also fixes the amdflang IBM+AMR offload crash (per-element custom mapper -> ROCm HSA OUT_OF_RESOURCES) at its source, without keeping the host round-trip. Verified 4/4 IBM-AMR cases (static + moving) on Frontier gpu-mp (CCE OpenMP-offload), gpu-acc (OpenACC), and CPU.
…FlowCode#1636) The AMR moving-IBM-circle test (13945217) ran a wall at vel=0.2 for 10 steps, long/fast enough to develop the spurious-pressure / CFL blow-up inherent to MFC's moving-IB method (no geometric-conservation-law treatment). The marginal instability NaNs on OpenMP-offload (CCE gpu-mp and amdflang) while passing on OpenACC and CPU. The real fix (a GCL/cut-cell treatment) is tracked in MFlowCode#1636 and deferred. Soften the case so it stays sub-marginal on all backends while still exercising the per-substage fine-IB recompute: slow the wall to vel=0.02 and stop at 4 steps (matching the two-body case's short-run pattern). Regenerated goldens. Verified 13945217 passes on gpu-mp (CCE OpenMP-offload), gpu-acc (OpenACC), and CPU.
…sent-table leak) s_amr_free_slot freed each slot's fields with @:DEALLOCATE only, but @:ACC_SETUP_SFs maps two things per scalar field on Cray - the descriptor ([arg]) and the field data ([arg%sf]) via GPU_ENTER_DATA copyin. Since Cray 'exit data delete' decrements the reference counter, the lone @:DEALLOCATE(arg%sf) undoes only the @:ALLOCATE create ref, leaving the descriptor and the ACC_SETUP %sf copyin ref dangling in the present table. Latent at program exit, but s_amr_free_slot frees slots MID-RUN (regrid / restart / reconcile), so the leaked host addresses get reused by the next allocation. On the AMR restart path the reused address is Gs_rs (Riemann init runs after s_read_amr_restart frees the init slot), tripping 'CRAY_ACC_ERROR - Error placing gs_rs: another instance already present' and killing the run (test 5EFB3277, AMR 1D dynamic regrid, 2 MPI ranks, gpu-acc). Add @:ACC_TEARDOWN_SFs (exact inverse of @:ACC_SETUP_SFs) and call it before each field's @:DEALLOCATE in s_amr_free_slot so every ACC_SETUP enter is matched by a delete. Cray-only (#ifdef _CRAYFTN), so non-Cray is unchanged. Verified 5EFB3277 passes on gpu-acc (2 ranks + restart roundtrip) and CPU.
The single-body moving-IBM case (13945217) was slowed to vel=0.02 in 4c877aa to dodge the un-GCL'd blow-up (MFlowCode#1636). The two-body case is pushed on top of it in the CaseGeneratorStack WITHOUT a pop, so it inherited that change - silently dropping body 1's wall speed 0.2 -> 0.02 and invalidating 43AF9F25's golden (which was generated at 0.2). That regressed 43AF9F25 across all backends (the softened setup diverges more on CPU than on GPU, hence the confusing CPU 91% vs gpu-acc 1.4x-tol spread). Restore body 1's vel=0.2 in the two-body push so the softening stays scoped to 13945217. 43AF9F25 is unchanged from its known-good config; no golden regen. Verified 43AF9F25 passes against its existing golden on CPU, gpu-mp (CCE OpenMP-offload), and gpu-acc (OpenACC).
…igns (amdflang shape crash) On the non-case-optimized amdflang path (#:if not MFC_CASE_OPTIMIZATION and USING_AMD), weno reconstruction scratch arrays are sized to the max 0:4 since weno_num_stencils is not a compile constant. The six central-weight assignments 'omega(:) = d_cbL/R(:, sv)' then assign the 0:weno_num_stencils-shaped d_cb (3 elements for WENO5) into the whole 0:4 omega (5), which amdflang's runtime array-shape check aborts on: 'Assign: mismatching element counts, to 5 from 3' (SIGABRT, exit 134). CCE/nvfortran use the exact-sized #:else path so never hit it. Only 0:weno_num_stencils of omega is meaningful, so slice the target. Surfaced by the hybrid_weno sensor's use_central branch: fixes the amdflang gpu-omp crashes on 71E57E55, 053C5DDA, BA4340EA (verified pass on amdflang; CCE/nvfortran/CPU unaffected - the sliced target is identical on the exact path).
master MFlowCode#1572 decomposed the HLLC hot path into shared m_riemann_state device helpers. Reconcile up/mega's hybrid-smooth-flux feature onto that refactor and, per request, extend it from hllc-only to all Riemann solvers. Factor the central (hybrid_smooth_flux=1) / Rusanov (=2) smooth-flux block into one shared device subroutine s_compute_hybrid_smooth_flux in m_riemann_state (no Fypp macro), called by hll/hllc/lf at each WENO-smooth face using the states the solver already computed. hlld reuses its own F_L/F_R/U_L/U_R for the MHD Rusanov override. Relax the hllc-only and MHD hybrid_riemann prohibitions in m_checker. Add hll/lf/hlld hybrid_riemann golden tests.
chi_vw/k_mw/rho_mw were module-scope GPU_DECLARE'd scalars used as per-cell scratch inside the s_compute_bubble_EE_source parallel loop (s_bwproperty writes them; s_vflux/f_bpres_dot read them via host association) but were not private, so GPU threads clobbered each other's values between write and read - a scheduling-dependent error (amdflang 3.6e-7, CCE-OMP 1.6e-9) on the nonpolytropic partial pressures. Thread them through arguments instead (fchi_vw/frho_mw to s_vflux, fk_mw to f_bpres_dot) with per-thread locals, and delete the module scalars. Validated: 454C565F now passes at its 5e-9 tol on amdflang.
…lance-on np=2 test (unification prep)
…s cartesian) (unification #1)
…wners in coexist (unification #2)
…fication #3) s_l0_tiles_init asserted that each tile's SFC compute owner equals its cartesian L0-storage owner, aborting with 'routed initial fill not implemented for this decomposition' otherwise, because the first fill (s_l0_copy_coarse_to_tiles) was a LOCAL L0->tile copy. Whether the two agree depends on how the cartesian split direction lines up with Morton order, so it is a property of the grid SHAPE: a 2:1 grid at np=2 splits in y and agrees, a square grid tie-breaks to x and diverges. Every existing L0 golden uses n=31 against m=63, so the whole suite sits on the passing side and CI cannot see it. Measured on a square 64x64 grid: np=2 aborted at l0_ntile>=2, np=4 aborted at every tile count. s_l0_copy_coarse_to_tiles now routes the seed when compute owner != storage owner: the L0-storage owner device-packs its chunk and sends it, the compute owner unpacks into its tile slot. Exact reverse of s_l0_scatter_tiles_to_coarse, and sound because a tile is built by subdividing ONE rank's cartesian chunk, so it never spans two L0-storage ranks. That alone segfaulted on a null address: s_l0_build_tile_slot ran inside the cartesian loop under 'if (r /= proc_rank) cycle' while the SFC cut overwrites amr_block_owner afterwards, so the compute owner had no slot to unpack into. The abort had been masking it. Slot construction reads only replicated region metadata and the global amr_g?cb, so it moves after the cut and keys on amr_block_owner. When the cut agrees with the cartesian order this allocates exactly the same set as before, just later. The compute==storage path is untouched (same call, same slot-extent arguments), so every previously working configuration is bit-identical. Validated on hpcfund (MI250X, amdflang AFAR 23.2.0, OpenMP offload): square 64x64 np=2 at l0_ntile 2 and 4 now run and are byte-identical to monolithic (restart md5 52a7aab258945073); 56 AMR goldens + 6 L0/coexist goldens pass; precheck clean. New golden 93093920 covers the square-grid np=2 case, which aborts before this change. Cray not exercised locally.
…iling, add batching design note amr.md was last updated 2026-07-04, before the l0-amr arc landed. Three claims had drifted from the code: - 'All amr_max_blocks slots are allocated at init at maximum size' (stated twice) is wrong: amr_slot_live drives lazy owned-only allocation, so a rank's fine memory is roughly 1/num_procs of the pool, not all of it. - 'Level-0 output only ... fine-resolution output is future work' is wrong: post_process overlays the refined fine blocks as separate SILO domains when amr = T in its input (default off). - The load-balance section claimed the coarse split is static after startup with no mention of l0_ntile, which did not appear anywhere on the page. In-run base-grid rebalancing exists: l0_ntile tiles the base grid into rr=1 blocks (byte-identical to untiled) and l0_rebalance_interval periodically re-cuts the SFC partition from measured per-tile cost and migrates tiles. Adds amr_block_batching.md, the design record for the per-block advance cost. Measured on MI250X with OpenMP offload via the l0_ntile sweep (fixed grid, fixed flops, byte-identity verified): cost is linear in block count at ~1.05x a full monolithic step per block, and does not amortize (still 15x at 512^2 with 16 blocks). Per-block cost is independent of block size, so it is fixed overhead, not work. That rules out the lever amr.md implies: per-slot state removes the swap traffic but not the per-block kernel launch count, and per-lane duplication multiplies the O(N^d) working set without reducing launches. The note records the state inventory, the three increments, and the measurement recipe. Also notes why this gates the coexist arc: relaxing the amr_max_level clause of the l0_ntile/amr checker gate fails on 'the nested level-2 block exceeds the per-rank scratch cap', which is the same base-subdomain solver scratch the batching work targets. Docs only; no behavior change. precheck clean.
… not work Traced the hot path: exactly one s_amr_swap_to_fine / s_amr_restore_coarse pair per block per RK stage, wrapping s_compute_rhs in s_amr_fine_stage_rhs. The RK pass and the fill phase do not swap. So a timestep costs 3*nblocks swap round-trips, each carrying s_amr_sync_grid_state_to_device plus s_amr_recompute_weno_coefs on nonuniform grids. Also records the obvious optimization that is NOT available: hoisting the restore so consecutive blocks swap fine->fine fails because m_time_steppers calls s_amr_p2p_reflux_faces and s_amr_apply_reflux between blocks inside the advance loop, both in the coarse frame. Any batching design has to move the reflux out of the per-block loop or make it frame-independent first. Docs only. precheck clean.
…erequisite) The fine stage advanced and refluxed each block in one loop: s_amr_fine_stage_advance runs with the block's grid globals swapped in, then s_amr_p2p_reflux_faces / s_amr_apply_reflux write the coarse rhs_vf at coarse indices. Interleaving the two frames forces a swap/restore round trip per block per RK stage, and it is what prevents consecutive blocks from swapping fine->fine. Phase 3 now advances every block; phase 4 refluxes the level-1 blocks afterwards in the coarse frame. Behavior-preserving: no block's advance reads rhs_vf, the regrid merge invariant keeps blocks >= buff_size apart so their c/f corrections are disjoint, and every rank still visits the same slots in the same order, preserving the collective ordering of s_amr_p2p_reflux_faces. level>=2 blocks still skip L0 reflux (they apply a state reflux into their parent after the stage loop). This does not by itself reduce the measured per-block cost - it removes the structural obstacle to doing so, which docs/documentation/amr_block_batching.md names as the prerequisite for batching the advances. Validated on hpcfund (MI250X, amdflang AFAR 23.2.0, OpenMP offload): the full 56-case AMR golden set passes, including the multi-block, multi-level, np=2 and reflux-exactness cases, plus the 6 L0/coexist goldens. precheck clean. Cray not exercised locally.
…p sites The note claimed the reflux frame was the prerequisite for hoisting the restore. That was incomplete. Traced s_amr_fine_stage_rk: it reads no grid globals itself (slot arrays plus dt and feature flags), but it calls s_amr_pressure_relax_fine, s_amr_ib_correct_fine and s_amr_update_mib_fine, and each opens its OWN s_amr_swap_to_fine / s_amr_restore_coarse pair assuming the coarse frame on entry. With the restore deferred they would swap while already swapped and trip the nested-swap assert. So there are two blockers. The reflux frame is resolved (advance and reflux loops are now split). The nested swap sites are open and are the larger of the two: three of the five swap/restore sites in m_amr.fpp nest inside the RK pass, which makes hoisting the restore a swap-CONTRACT change (make the swap re-entrant, or make those three routines no-op when already on their own slot) rather than a call-site change. Also records that any swap-contract change needs a Cray GPU run before it can be trusted, per the CCE-acc-only stale-device class in .claude/rules/common-pitfalls.md. Docs only. precheck clean.
…e' curve) Measured on hpcfund (MI250X, amdflang, OpenMP offload), 2D 256^2, static single-level AMR, 20 steps: np compact block (32x32) wide block (128x32) 1 0.1009 s (1.00x) 0.1009 s (1.00x) 2 0.0980 s (1.03x) 0.1190 s (0.85x) 4 0.0983 s (1.03x) 0.1267 s (0.80x) AMR does not strong-scale, and wide blocks anti-scale. Both cases hold exactly one fine block (a static block cannot exceed amr_maxc, so tiling never triggers), and single-owner distribution puts all its work on one rank whatever np is. The compact case is flat because extra ranks only split the coarse grid, which is not the bottleneck. The wide case is worse than flat: the block spans more ranks' coarse subdomains, so each added rank buys more coarse<->fine P2P gather/scatter with no fine parallelism to offset it. Caveat recorded in the note: this is the single-block regime and does not test multi-block distribution, which needs dynamic regrid to reach several blocks. That is the curve batching should move; these numbers are the floor it has to beat. Docs only. precheck clean.
s_amr_swap_to_fine asserted .not. amr_swapped, forbidding nesting outright. That assert is what makes hoisting the restore across blocks a contract change rather than a driver change: s_amr_fine_stage_rk calls s_amr_pressure_relax_fine, s_amr_ib_correct_fine and s_amr_update_mib_fine, and each opens its own swap/restore pair, so a deferred restore would trip it three times per block. Replaces the paired-swap logical with a depth counter. Only the outermost swap saves the coarse state into the sw_* bounce buffers (every save is now depth-guarded; the installs are not, since re-installing the same slot is idempotent), and only the matching outermost restore puts it back - an inner restore returns early, leaving the enclosing frame's slot installed. Sound because every nested swap site swaps to the SAME slot, amr_cur. Behavior-preserving by construction: no nesting occurs today (the RK pass's helpers run after the RHS pass has already restored), so the depth never exceeds 1 and every path is exactly as before. The driver is untouched. What this buys is that hoisting the restore is now a change to m_time_steppers alone. Validated on hpcfund (MI250X, amdflang AFAR 23.2.0, OpenMP offload): 56 AMR goldens and 5 L0/coexist goldens pass, precheck clean. Cray not exercised locally - per .claude/rules/common-pitfalls.md the stale-device class around this contract is CCE-acc-specific, so the Frontier lanes are the real gate.
…-and-seed in one routine Correcting an overclaim: with the swap now re-entrant I said hoisting the restore was a driver-level edit. It is not. s_amr_swap_to_fine ends with 'if (igr) call s_amr_igr_swap_sigma()', which both SAVES the coarse jac/jac_old into sw_jac/sw_jac_old and SEEDS the block's sigma from the parent, reading sw_idwbuff for the coarse extent. Once nesting is actually used an inner swap re-runs it and overwrites sw_jac with fine state, so it must be split into a depth-guarded save and a per-block seed first. Marks prerequisite 2 (re-entrant swap) resolved and records this as prerequisite 3. Also records that this routine already caused an OpenACC-only crash - its own comment notes the sw_* host-only state makes OpenACC's present lookup fail while OpenMP's implicit map(to) tolerates it - as a worked example of why an OpenMP-offload pass does not validate this area. Docs only. precheck clean.
…te 3 of 3) s_amr_igr_swap_sigma both saved the coarse jac/jac_old into sw_jac/sw_jac_old and seeded the block's sigma from the parent. Under nesting an inner swap would re-run the save and put FINE state into sw_jac, which both the seed and s_amr_igr_restore_sigma then work from. The save loop is now depth-guarded; the seed loop is not. The seed reads sw_jac, which still holds the coarse state, so every nested block seeds from the correct parent - guarding only the save is both necessary and sufficient. Behavior-preserving today: no nesting occurs, so the depth is always 1 and the path is exactly as before. This was the last of the three obstacles to hoisting the restore across blocks. With the reflux split (advance and reflux loops separated) and the re-entrant depth-counted swap already in, hoisting is now a change to m_time_steppers alone: open a swap frame across the phase-3 loop, let each block's inner swap install its own slot, restore once after. The swap's recomputes need no guard - they are idempotent for whichever grid is installed. Validated on hpcfund (MI250X, amdflang AFAR 23.2.0, OpenMP offload): the two IGR+AMR goldens that exercise this path (6C20B752, 660FFBFE) pass, as does the full 56-case AMR set. precheck clean. Cray not exercised locally, and this routine has a documented OpenACC-only failure history, so the Frontier acc lane is the real gate.
Follow-up to 2bffefd, whose doc hunk did not apply. Records that the IGR sigma save is now depth-guarded while the seed is not, and that guarding only the save is necessary and sufficient because the seed reads sw_jac which still holds the coarse state. All three prerequisites are now resolved, so the note states plainly what is left: hoisting the restore is a change to m_time_steppers alone - open a swap frame across the phase-3 loop, let each block's inner swap install its own slot, restore once after. The swap's recomputes need no guard, being idempotent for whichever grid is installed. Docs only. precheck clean.
…ffect Built the hoist and measured it: added s_amr_swap_hold (save the coarse state without installing a slot) and bracketed the per-tile RHS loop with it, so consecutive tiles ran fine->fine with one coarse restore after the loop instead of one per tile. Output byte-identical. l0_ntile sweep, 2D 256^2, np=1, with vs without: 1.00/1.01/4.32/16.87 vs 1.00/1.01/4.40/16.75. No effect. The coarse round trip between blocks - its sw_* copies, its device sync, its coefficient rebuild - is not a measurable share of the per-block cost even at 16 blocks where the total penalty is ~17x. Reverted rather than carried as unused machinery. This sharpens the note's conclusion: the per-block cost really is kernel launch count inside s_compute_rhs, not swap traffic, so reducing swap traffic in any form should be expected to do nothing on its own. Only the batched-kernel increment addresses the measured cost. Recorded so the hoist is not retried. Docs only. precheck clean.
… cost The note attributed the per-block cost to kernel launch count, but that was reached by elimination, not measurement. Counted it with rocprofv3 --kernel-trace (2D 128^2, 6 steps, np=1, under srun): 381 launches monolithic vs 7619 at l0_ntile=4 (16 blocks), a 20.0x ratio against the 16.75x wall-time ratio. The cost tracks launches directly. Combined with the refuted restore hoist (removing swap traffic changed nothing), this closes the argument: the fixed per-block cost is the launch count, and only batching blocks into single launches addresses it. Also notes each tile advance issues ~450 launches against the monolithic step's 381, since the per-block path adds its own ghost fill and halo work on top of the same RHS kernel sequence. Docs only. precheck clean.
…mented feature Was one combined PROHIBIT covering dynamic regrid, subcycle, and multi-level. Each clause now lifts independently and names the failure it prevents, measured with the gate disabled (2D 64x32 amr_2d_base grid+block, np=2, AMD OMP offload) against the l0_ntile = 0 arm of the same case, which passes in all three modes: static (control) l0_ntile=0 and =2 both 1ae3bfca3521 - byte-identical regrid deadlocks in the first regrid's collectives subcycle runs 6 steps, then NaN at output multi-level segfaults before step 1 Mechanisms: regrid indexes the block pool from slot 1 and sets amr_num_blocks = nboxes, overrunning the level-0 tile prefix in [1..l0_slot_off] (the static path already maps through f_l0_slot; m_amr_regrid uses it zero times). Subcycle applies its Berger-Colella correction as a STATE reflux into L0 with no route to the tile compute-owners, unlike the lock-step rhs reflux. Multi-level is the amr_maxc scratch cap - Track 3's per-block working set, not a coexist bug. Behavior identical: same three conditions, same set of rejected cases.
Four defects, each found by measurement against the l0_ntile = 0 arm of the same case (2D 64x32 amr_2d_base, AMD OMP offload). np=1 coexist regrid is now byte-identical to monolithic over 4 regrids; np>=2 is not yet, and the checker gate is narrowed to it. 1. m_amr_regrid indexed the shared slot pool by BOX index and set amr_num_blocks = nboxes, overrunning the level-0 tile prefix in [1..l0_slot_off]. The static path already maps through f_l0_slot; regrid used it zero times. stash_migrate, rebuild_slots, boxes_unchanged and the block-set commit now work in slot space. Was: deadlock in the first regrid's collectives. 2. s_amr_assign_block_owners cut into amr_owner_cut, then copied it to amr_fine_cut. Harmless at init (it runs before s_l0_tiles_init builds the tile cut) but at REGRID time it clobbers that tile cut, so every level-0 tile then resolves f_amr_owner against the fine cut. Now cuts into amr_fine_cut and mirrors to amr_owner_cut only when there are no tiles. 3. s_l0_tiles_init ZEROED the module-level max_f*/mbuf* slot-extent state before accumulating the tile extents, discarding the fine sizing s_initialize_amr_module had just computed. s_amr_alloc_slot reads those. The static block's slot is allocated before this routine and escapes, but every slot a regrid allocates afterwards is a fine block cut to TILE size - here a block needing 0:23 in y got a tile's 0:15. Was: out-of-range device write in s_l0_copy_block (segfault under __tgt_target_kernel, not a host abort). Now accumulates the max of both. 4. s_amr_regrid both tags off L0 and prolongs each new block's seed from it, but under coexist the tiles own the state and the stage loop refreshes L0 only at the TOP of each stage - so L0 was one stage stale there. Added the same just-in-time s_l0_scatter_tiles_to_coarse that s_save_data already does before it reads L0. Also fixes a latent bug in the parallel-IO AMR restart writer, independent of tiles: the pack loop had no ownership guard (the host-refresh loop in the same routine does), so on a non-owning rank it ran the full extent loop over an unallocated q_cons and overran the 1-element placeholder buffer. Reachable whenever a rank does not own every block; the collective WRITE_AT_ALL still runs everywhere with cnt = 0. Validation: 61/61 buildable AMR + L0 goldens pass (3 chemistry AMR tests fail to execute here - the gpu-mp-chem build variant is not built in this environment - not golden mismatches). New golden 83CC5C6D covers coexist + dynamic regrid at np=1, which no existing test reached: the coexist goldens are all static and every dynamic-regrid golden runs without tiles. np>=2 remains gated. It diverges on the first step after the first regrid (max abs 5.7e-2, total mass off by 1.3e-4) while choosing the SAME boxes as monolithic, so the fault is in the post-regrid cross-rank state, not box selection.
… a strided GPU_UPDATE
s_restrict_fine_to_coarse's np>=2 receiver unpacked the owner's covered-cell
slice on the HOST and then pushed the covered box back with a strided
GPU_UPDATE(device=). AMD flang copies a non-contiguous 3-D array section in an
OpenMP target update as size(section) CONTIGUOUS elements, so only the first
row landed on the cells it named and the remaining cells overwrote their
neighbours with stale host data - silently.
Measured on a 64x32 2D regrid case at np=2: of the 60 covered cells, 10 were
delivered and 50 were not, total mass off by 1.4e-5 and growing per regrid.
The receiver branch only runs when a level-1 block's owner holds NONE of its
covered coarse cells, which no golden produced: static blocks are owned by a
rank that overlaps them, and the one np>=2 regrid golden is 1D, where the
covered box is a single contiguous row. Regrid at np>=2 in 2D creates it.
Unpack on the device instead, via the existing s_l0_pack_unpack_block - its
wire layout (ci fastest, then cj, ck, i) is already s_amr_restrict_pack_device's.
Same defect and same fix in the pb/mv twin s_amr_scatter_pbmv, where the section
is non-contiguous even in 1D (the x sub-range breaks it across nnode/nb).
This was NOT a coexist bug: the monolithic (l0_ntile = 0) arm was equally wrong,
so lift the np>=2 clause of the l0_ntile+amr regrid gate. All four arms of the
probe case - {np=1, np=2} x {monolithic, coexist} - now agree byte-for-byte,
where before np=2 disagreed with np=1 in both arms.
Goldens: 2C46C59A (plain AMR 2D regrid np=2, the routine the defect lives in)
and 33060D84 (coexist 2D regrid np=2, the newly ungated configuration).
Verified: 57 AMR + 5 coexist/regrid goldens pass; the 3 pre-existing coexist
goldens are unchanged, as expected for a fix confined to the cross-rank branch.
The subcycled fine advance needs three things in the L0 frame that coexist did
not maintain, and the run NaN'd at np=1 on the first step:
1. q_cons_ts(stor), its coarse t^n time-lerp bracket, has exactly one writer -
the monolithic RK, which l0_ntile > 0 skips (tiles keep their own per-slot
backup). The fine C/F ghosts were lerping against an unwritten array. Take
the L0-frame backup at stage 1, where the tiles->L0 scatter has just made L0
an exact mirror of the tiles at t^n.
2. q_cons_ts(1) must be t^{n+1} at the subcycle call, but the stage loop
refreshes L0 only at stage TOPS, so it held the stage-3 entry state.
Re-scatter the advanced tiles first.
3. The Berger-Colella correction lands as a STATE reflux on the coarse cells
just OUTSIDE each block (s_amr_apply_reflux_state); the covered-footprint
copy-back (s_l0_restrict_to_tiles) does not carry those cells.
With (2) in place L0 equals the tiles everywhere except the cells the fine fold
deliberately changed, so (3) needs no new delta plumbing: refill every tile
interior from L0 and the restrict and the reflux shell both arrive. That is an
exact copy round-trip on untouched cells. Split the unconditional fill out of
s_l0_copy_coarse_to_tiles, whose seed gate must still fire exactly once.
Lifts the amr_subcycle clause of the coexist gate; only the amr_max_level > 1
clause remains, and that one is a scratch-capacity limit, not a coupling gap.
Verified: coexist is byte-identical to the monolithic (l0_ntile = 0) arm at BOTH
np=1 and np=2 (all four arms 42ea141991e0, where coexist previously NaN'd).
Goldens FD056B71 / 98AA6EDB. 57 AMR + 7 coexist goldens pass.
…t slot 1
s_amr_build_static_multilevel insets its parent to place the nested level-2
box, and read slot 1 directly. Under coexist slot 1 is the first level-0 TILE,
not the level-1 block, so the box was positioned and sized off a tile:
l0_ntile = 1 tile spans the base grid -> box 2x too wide -> the amr_maxc_fit
cap fired, which is what made this look like the scratch-cap
blocker it was documented as
l0_ntile = 2 tile is half the grid -> a plausible-looking box in the wrong
place -> no abort, silent corruption, NaN at a corner cell far
from the block by the final step
Use f_l0_slot(1), and bound the pool-room check by l0_slot_off + amr_max_fine
rather than amr_max_fine (a slot index compared against a fine-only count).
Both are no-ops without tiles, where f_l0_slot is the identity and l0_slot_off
is 0.
Verified byte-identical to the monolithic (l0_ntile = 0) arm at np=1 with
l0_ntile 2 and 4, and at np=2 with l0_ntile 2. Goldens D99F85F8 / 09E0D257.
The coexist multi-level gate is narrowed rather than removed: l0_ntile = 1
still corrupts the heap during init, at any rank count, pool size and block
size. That is NOT the amr_maxc cap (this case clears it) and is undiagnosed, so
it stays gated with a message that says so. l0_ntile >= 2 is now supported.
…uild s_amr_build_static_multilevel ends by restoring amr_cg and the patch frame to 'block 1' and re-gathering its coarse patch. It selected slot 1 - which under coexist is a level-0 TILE, not the first fine block - so the grid globals were left describing tile geometry for the remainder of initialization. s_initialize_weno_module runs after this (m_start_up.fpp:959) and sizes its device-mapped coefficient tables from those globals, so it faulted inside __tgt_target_data_begin_mapper. Why this hid: with several tiles the leftover geometry is a fraction of the subdomain and the tables came out small enough to map; with l0_ntile = 1 the single tile spans the whole base grid, which is the configuration that actually crashed. Fixing the box lookup in e96ca62 removed the amr_maxc_fit abort and exposed this, so the two are one defect class - slot index vs block index - found in two places. Use f_l0_slot(1), the identity without tiles. This was the LAST coexist gate: l0_ntile > 0 .and. amr now has no PROHIBIT at all. Multi-level is byte-identical to the monolithic (l0_ntile = 0) arm at l0_ntile 1, 2 and 4 (np=1, c4f185d15485) and at np=2 (3aaee2b8fed4). Golden 93EFC4F6 covers the single-tile case that failed. Verified: 57 AMR + 10 coexist goldens pass.
The same-rank branch of s_amr_fine_fine_halo routed a purely LOCAL copy through
amr_seambuf_*, a buffer that exists only to cross MPI: four s_amr_fine_slice
calls per seam pair per RK stage, each a kernel plus a blocking device<->host
round trip. Replace them with one fused device kernel doing both directions.
The cross-rank branch is untouched.
Fusing the two directions is safe because all four slabs are disjoint - a
block's seam GHOST lies outside its own interior, and the two reads come from
different slots than the two writes.
Measured, 2D 64x32, l0_ntile = 4 (16 tiles, 24 same-rank seam pairs), AMD MI250X
OMP offload:
kernel launches 10561 -> 7969 (-24.5%, rocprofv3, deterministic)
Time Avg (n=3) 0.8179 -> 0.7121 s (-12.9%; ranges disjoint, 0.8082-0.8348
vs 0.7100-0.7139)
The variance also collapses, 3.3% -> 0.5% spread: the host round trips were a
scatter source, not just a cost.
Byte-identical: it moves the same cells in the same order, and the old wp buffer
only widened and re-narrowed the stp values. All four coexist modes match the
monolithic (l0_ntile = 0) arm at np=1 and np=2 with unchanged checksums, plus
the 16-tile cases; 57 AMR + 10 coexist goldens pass.
Batching further, ACROSS pairs, is not possible without a flat per-slot backing
array: each slot's q_cons(i)%sf is an independent allocation and amr_slots is
not GPU_DECLARE'd, so a runtime slot index in a kernel is a null deref. A pair
has only two slots, which is why they can be dummies here.
s_amr_fill_fine_ghosts issued one GPU_PARALLEL_LOOP per face slab - ns = 2/4/6 in 1D/2D/3D - to prolong the ghost shell. The slabs all belong to ONE block, so q_fine stays a single dummy and they can be fused without the flat per-slot backing store that cross-block batching needs. Fuse with a flat index over the concatenated slabs, decoded per cell (ns <= 6, so a scan beats a stored map). NOT the padded-hull form of s_amr_capture_creg_dense_batch: the x slabs span the full transverse extent, so a hull over all slabs is the whole buffered volume, and masking it would throw away the O(surface) decomposition the slab split exists to get. Safe because s_amr_build_ghost_slabs emits DISJOINT slabs whose union is exactly the ghost shell - verified by construction: x slabs take x outside [0,fm], y slabs restrict x to the interior and take y outside [0,fn], z slabs restrict both. So every ghost cell is still written exactly once and the order does not matter. Measured 4 -> 1 launches per fine block per RK stage (num_fluids = 1; with multiple fluids the alpha-closure loop below is a second, still-unfused 4). On the 16-tile benchmark that is only 10561 -> 7915 vs 7969, because that case has ONE fine block - the 16 tiles are level-0 and get their ghosts from the edge-BC + seam path, not from here. The saving is 3 launches per FINE block per stage, so it scales with fine-block count, which is the regime this targets. Byte-identical: all four coexist modes match the monolithic arm at np=1 with unchanged checksums; 57 AMR + 10 coexist goldens pass.
Same flat-index fusion as the prolongation loop, applied to the sum-to-one closure loop that follows it: ns launches -> 1 per fine block per RK stage. Only runs for num_fluids > 1, so the l0_ntile gate cases (single fluid) cannot reach it - the multi-fluid AMR goldens are the coverage. Two Fypp constraints found the hard way, both reported as a bare FyppFatalError with no location: a Fortran continuation inside a GPU_PARALLEL_LOOP clause only parses when it falls in the LAST argument, and collapse=1 is rejected - a single loop takes no collapse clause. Byte-identical: 57 AMR + 10 coexist goldens pass.
s_amr_build_ghost_slabs has FOUR consumers, all issuing ns launches per fine
block; only s_amr_fill_fine_ghosts was fused. Apply the same flat-index fusion
to the rest:
s_amr_lerp_fine_ghosts the SUBCYCLE time-lerp fill - called per
SUBSTEP, so it runs more often than the
lock-step fill already fused
s_amr_fill_fine_ghosts_pbmv QBMM pb/mv fill
s_amr_lerp_fine_ghosts_pbmv QBMM subcycle lerp
Each goes ns -> 1 launches per fine block (per substep for the lerp pair). The
pbmv two are gated behind qbmm .and. .not. polytropic; goldens DDD79C8B /
BCBA6E74 / B0A5D230 cover them.
Byte-identical: 57 AMR + 10 coexist goldens pass.
s_amr_fine_fine_halo walks the global seam-pair list and exchanges every same-level adjacent pair. The subcycled level-2 child advance needs to reconcile ONLY its own level's seams: it runs inside one of the parent's substeps, when the level-1 blocks are mid-substep and must not be touched. Add an explicit lev_only selector (0 = all levels, as every current caller passes) rather than an optional argument, so each call site states which levels it means. Prerequisite for transposing s_amr_advance_children - today that routine runs each child's ENTIRE subtree before starting the next, so adjacent siblings never see each other, which is why m_amr_regrid clamps subcycle to one capped level-2 child (a wide feature is under-refined, not non-conservative) and s_amr_check_seam_topology aborts on the restart mode-switch route. Byte-identical: 57 AMR + 11 goldens pass, including 00E15144 (multi-level dynamic subcycle), which is the anchor for the transposition to come.
…halo s_amr_advance_children ran each child's ENTIRE subtree before starting the next, so adjacent siblings never saw each other and their shared face carried mismatched fluxes. That is why m_amr_regrid clamps subcycle to ONE capped level-2 child per box (a wide feature is under-refined rather than non-conservative) and s_amr_check_seam_topology aborts on the restart mode-switch route that can produce adjacent L2 blocks anyway. Transpose it into the shape s_amr_advance_fine_subcycle_all already uses at level 1: set every child up, then walk all of them through each substep together - lerp all ghosts, run the level-filtered seam halo, then advance all - recursing into grandchildren per substep and folding back at the end. The halo is filtered to the child level because this runs INSIDE one of the parent's substeps, when the parent's level is mid-substep and must not be touched. s_amr_advance_subtree had no callers left once its body was inlined here, so it is deleted rather than kept as an unused second path. Byte-identical: with one child per parent the transposition is a pure reordering, and 00E15144 (multi-level dynamic subcycle) and B0A5D230 (QBMM nonpolytropic regrid subcycle) both pass unchanged; 57 AMR goldens pass. The multi-child seam this enables is not yet reachable - the regrid clamp still caps it to one child, which is the next commit.
With s_amr_advance_children transposed (2734961), adjacent level-2 siblings exchange their shared face through the level-filtered seam halo, so subcycle no longer needs to keep ONE capped child. Drop the clamp and tile the box exactly as lock-step already does, and drop the s_amr_check_seam_topology clause that aborted on adjacent L2 blocks under subcycle - the halo it warned about now exists. Effect: a level-2 feature wider than amr_maxc_fit/2 was previously UNDER-REFINED under subcycle (the box was truncated); it is now fully covered by adjacent tiled children. Golden A635AA56 exercises it, and was verified to fail without this change: amr_buf = 12 grows the level-2 region past amr_maxc_fit/2 = 16 coarse cells and restoring the clamp moves the answer (a6e3ad6d -> 5c9d3118). Note it is the BUFFER that widens the box, not the tag threshold - a looser amr_tag_eps at amr_buf = 6 leaves the answer bit-identical either way, so an eps-based case would have protected nothing. 58 AMR + 13 goldens pass; every pre-existing golden is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An opt-in, default-off family of performance features and the measurement infrastructure they rest on. With all flags at their defaults the only touched production path is
s_mpi_decompose_computational_domain, refactored through the newm_boxmodule (byte-identical; covered by the existing suite).m_box(partition arithmetic),m_load_weight/load_weight_wrt(per-cell load-weight field + imbalance metric),m_sfc_partition/sfc_partition_wrt(Morton-SFC predicted-imbalance diagnostic),m_load_balance/load_balance(weighted static decomposition at init; AMR-fine-work-aware),m_rank_timing/rank_time_wrt(per-rank compute-time diagnostic).m_active_box/active_box: restricts reconstruction/Riemann/RK windows to a light-cone-grown box around non-ambient flow; strict-subset golden-tested.hybrid_wenoandhybrid_riemann(+hybrid_weno_eps,hybrid_smooth_flux): linear-optimal weights / central-or-Rusanov flux in smooth cells, full WENO/HLLC at flagged discontinuities (Jameson sensor, stencil-dilated, per-level under AMR).m_amr+m_amr_registers: two-level 2:1 refined block hierarchy; conservative restriction and conservative-linear prolongation with physics-specific closures; per-stage flux registers with Berger–Colella refluxing; Berger–Rigoutsos multi-block dynamic regrid; optional dt/2 subcycling; multi-rank (single-owner blocks assigned by Morton-SFC work balancing at each regrid, with migration; blocks may span rank seams via P2P coarse↔fine gather/scatter; same-level seam halo; distributed registers); restart (both IO modes, regridded-layout persistence); AMR-aware post-processing (fine blocks visualizable as Silo overlay domains); GPU-resident fine level on both OpenACC and OpenMP offload.Full algorithm and user documentation:
docs/documentation/amr.md(support matrix enforced at runtime by the checker — unsupported combinations abort with named messages, never silently).AMR physics support matrix (abridged; authoritative table in amr.md)
Supported and golden-tested: single- and multi-fluid (5-eq,
mpp_lim) · 6-eq with per-block pressure relaxation · viscous (refluxed) · phase change (relax) · chemistry incl. species diffusion · Euler–Euler bubbles (polytropic/non-polytropic, mono/polydisperse, QBMM incl. non-polytropic with per-blockpb/mvside-state; dynamic regrid + subcycle) · acoustic sources (coarse-grid support with regrid exclusion) · immersed boundaries (multi-body, static or prescribed-motion, incl. dynamic regrid with body-containment expansion and per-substage guards) · 2D axisymmetric (per-block WENO-coefficient recompute) · stretched grids (exact parent-bisection ghost coordinates + per-swap coefficient recompute) · hybrid WENO/Riemann sensors (per-level) · Lagrangian bubbles (cloud excluded from blocks; two-way coupling on the coarse grid; regrid clips around the moving cloud) ·active_box(blocks contained in the growing window; agrees with plain AMR to ~1e-14) · IGR (restriction-only coupling: fine sigma solve seeded/Dirichlet-bounded by the coarse solve; documented truncation-order seam, exact free-stream) · 1D MHD/RMHD (div(B)=0 by construction in 1D; HLL and HLLD, incl. relativistic).Gated with named aborts (documented rationale): surface tension (seam force imbalance is structural — three fixes attempted and diagnosed in amr.md) · 2D/3D MHD (attempted and measured: the c/f seam is a continuous O(1) div(B) source GLM cleaning cannot remove — needs constrained-transport-class B prolongation/reflux) · hyperelasticity · 3D cylindrical (global azimuthal filter) · force-driven IB (
moving_ibm=2) · STL bodies · Riemann-extrapolation BCs (bc=-4) ·amr_subcycleunder IGR · stretched grids with Lagrangian/IB-regrid (uniform-spacing index conversions).Validation evidence
Known issues (all non-gating or in progress)
continue-on-error): an intermittent post-detected NaN on the two Lagrangian+AMR goldens. Exhaustively unreproducible off GitHub's runners — the exact failing stack (NVHPC 24.3 SDK,-tp=px -Kieee, HPC-X MPI, and the CI docker image itself under apptainer) passes elsewhere, as do native/zen2 builds; 24.5+ green. Documented at the golden definitions.Review guide
The commit history is arc-ordered (active-box → load-weight → SFC → weighted decomposition → rank timing → hybrid → m_box → AMR rungs → physics envelope → CI/GPU hardening); reviewing by arc is much easier than by file. The AMR arc builds stepwise: static hierarchy → restriction/prolongation → fine advance → refluxing → regrid → subcycling → multi-rank → GPU → each physics rung with its own validation. Commit messages carry the validation evidence for their change (measured defects, golden UUIDs, repro details for CI fixes).
All parameters ship default-off with
case_validatorentries, runtime checker gates, andcase.md/amr.mddocumentation.