Skip to content

fix: skip CUDA_PATH / HIP_PATH DLL directories that do not exist - #2349

Open
emptyngton wants to merge 1 commit into
abetlen:mainfrom
emptyngton:fix-add-dll-directory-missing-path-upstream
Open

fix: skip CUDA_PATH / HIP_PATH DLL directories that do not exist#2349
emptyngton wants to merge 1 commit into
abetlen:mainfrom
emptyngton:fix-add-dll-directory-missing-path-upstream

Conversation

@emptyngton

Copy link
Copy Markdown

Problem

On Windows, os.add_dll_directory() raises FileNotFoundError: [WinError 3] if the path does not exist. In llama_cpp/_ctypes_extensions.py the CUDA_PATH and HIP_PATH sub-directories are passed to it unconditionally.

So if either variable points at a toolkit the user has since uninstalled, upgraded, or moved, import llama_cpp fails outright — before any user code runs, and regardless of whether that backend is even in use:

FileNotFoundError: [WinError 3] The system cannot find the path specified:
'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\11.2\bin'

Repro on any Windows machine:

import os
os.environ["CUDA_PATH"] = r"C:\does\not\exist"
import llama_cpp  # FileNotFoundError [WinError 3]

A stale toolkit variable is common — CUDA upgrades in particular tend to leave the old CUDA_PATH behind — and because the failure surfaces inside the loader rather than at the offending variable, it reads as a broken wheel or a broken install.

This was reported in #1077: CUDA_PATH pointed at an uninstalled CUDA 11.2 while 12.3 was actually installed. That issue was closed after the reporter corrected their environment, but the loader itself was never changed, so it still happens today.

Solution

Check each candidate directory with os.path.exists() before registering it.

Both variables use the same bin / lib sub-directories, so the two near-identical blocks collapse into one loop. Each directory is tested individually, which also means a partially removed toolkit contributes whichever of bin / lib survive instead of raising.

No behaviour change for a valid toolkit: every directory registered today is still registered.

Notes

  • Verified: stale path (nothing added, no raise), valid toolkit (both dirs added), partially removed (surviving dir added), variable unset, variable set to an empty string.
  • os.environ.get(...) is None is used rather than in os.environ so an empty-string value is handled as well.

os.add_dll_directory() raises FileNotFoundError [WinError 3] when the
directory does not exist, so a CUDA_PATH or HIP_PATH left pointing at an
uninstalled or upgraded toolkit makes "import llama_cpp" fail outright on
Windows, before any user code runs.

Check each candidate directory with os.path.exists() before registering it.
Directories are tested individually, so a partially removed toolkit still
contributes whichever of bin/lib remain rather than raising.

Reported previously in abetlen#1077, where a stale CUDA_PATH pointing at an
uninstalled CUDA 11.2 broke the import on a machine with 12.3 installed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant