Avoid freeing failed VMM grow reservations#2237
Conversation
|
xref: #2235 (comment) |
|
Hello @fallintoplace, a heads-up on an interaction between this PR and #2407, since we are both editing this guard. At head expected_ptr = int(buf.handle) + aligned_prev_size
if res != driver.CUresult.CUDA_SUCCESS:
...
return self._grow_allocation_slow_path(...)
if new_ptr != expected_ptr:
(res2,) = driver.cuMemAddressFree(new_ptr, aligned_additional_size)
...
One note on the new tests: |
|
Thank you @aryanputta. Fixed in 08583ca by comparing int(new_ptr) with the expected address. I also updated the reservation mocks to return CUdeviceptr and added coverage confirming that an exactly contiguous reservation reaches the fast path without being freed. |
|
Thank you, that was quick. I checked That makes #2407 redundant on both the fix and the coverage. I will close it once this lands rather than right now, only so defect 2 of #2388 does not sit without an open PR while this one waits on vetter validation. Nothing for you to do about that, and I am not planning to touch that guard again. One piece of #2407 is not duplicated here: a release note bullet, since this changes observable behavior of a stable API. Feel free to lift it if you agree it belongs: - Growing a :class:`VirtualMemoryResource` allocation now preserves the buffer's
base pointer when the driver can extend the reservation contiguously.
Previously an address comparison never matched, so every grow re-reserved and
remapped the entire range.
(`#2237 <https://github.com/NVIDIA/cuda-python/pull/2237>`__)It goes under "Fixes and enhancements" in |
08583ca to
efed4f8
Compare
Summary
cuMemAddressFreewhencuMemAddressReservesucceeded and returned a noncontiguous address.Why
The previous grow path called
cuMemAddressFree(new_ptr, ...)even when the adjacentcuMemAddressReservecall failed with a recoverable error. In that case the returned pointer is not an owned reservation, so cleanup can turn a recoverable fallback into another driver error.Checks
python3 -m py_compile cuda_core/cuda/core/_memory/_virtual_memory_resource.py cuda_core/tests/test_memory.pygit diff --check -- cuda_core/cuda/core/_memory/_virtual_memory_resource.py cuda_core/tests/test_memory.pyNot run locally:
uv run --no-project --with pytest --with cuda-bindings --with cuda-pathfinder --with numpy python -m pytest cuda_core/tests/test_memory.py -k 'vmm_allocator_grow_allocation_does_not_free_failed_adjacent_reservation or vmm_allocator_grow_allocation_frees_noncontiguous_adjacent_reservation'The isolated test environment on this macOS arm64 host cannot resolve
cuda-bindings, because published wheels are available for Linux and Windows but not this platform.