feat: unify HTP power-mode control across qairt and llama_cpp - #1447
Draft
Mengsheng Wu (mengshengwu) wants to merge 8 commits into
Draft
feat: unify HTP power-mode control across qairt and llama_cpp#1447Mengsheng Wu (mengshengwu) wants to merge 8 commits into
Mengsheng Wu (mengshengwu) wants to merge 8 commits into
Conversation
geniex_resolve_power_mode maps one alias table (low_power_saver..burst, default) to a shared geniex_PowerMode enum, mirroring how geniex_resolve_device owns the compute-unit aliases. Both plugins will call it to normalize --power-mode instead of each defining its own table. llama_cpp support rides on an unmerged internal PR (ggml/llama.cpp#340): two patches carry it — one mirrors that PR, the other is a permanent GenieX addition exposing a setter proc so the plugin can change the mode per model load instead of only once at process start via an env var. Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
llama_cpp: htp_session gains set_power_mode, which calls the setter proc then releases idle HTP sessions so the next load's reacquire picks up the new mode. A session already in use by another loaded model keeps its old mode until released -- logged, not silently dropped. qairt: sets ModelConfig::perf_profile from the resolved mode on both the LLM and VLM (vision encoder) configs. Model::initialize seeds perf_profile from the caller but then lets a bundle's htp_backend_ext_config.json overwrite it, so the plugin probes the bundle with the same parseHtpConfig and warns when it disagrees with --power-mode, rather than silently losing the caller's choice. Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
Adds the power_mode field to ModelConfig across all three FFI surfaces, plus a Go ResolvePowerMode wrapper for CLI-side fail-fast validation before touching model load (Python and Android pass the string straight through; each plugin validates it on create). Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
infer / run share the --power-mode flag on llmFlags; serve adds it as a per-server default (--power-mode / GENIEX_POWER_MODE) overridable per request via the power_mode JSON field, added to ModelParam so the keep-alive cache reloads the model instead of silently reusing one loaded under a different mode. Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
Threads through to geniex_ModelConfig via the existing fill_model_config, and is recorded in the per-cell JSON params block when set; bumps schema_version to 5. Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
Adds a Power mode section to notes/run.md mirroring the existing Compute-unit aliases section (alias table, default-is-burst, llama_cpp's unmerged-PR caveat, qairt's bundle-override precedence), plus the --power-mode row in the CLI and geniex-bench references. Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
geniex-qairt-plugin#55 fixes Model::initialize to let a caller-supplied perf_profile win over the bundle's htp_backend_ext_config.json, instead of the bundle silently overwriting it. Pulls in 18ee1bfc..277c45c3. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
apply_power_mode() used to probe the bundle's htp_backend_ext_config.json and warn when it disagreed with the caller's --power-mode, because the bundle silently won. geniex-qairt-plugin#55 fixed the precedence itself (caller > bundle > default), so the probe is dead code; drop it and update the docs to describe the corrected precedence. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
Mengsheng Wu (mengshengwu)
force-pushed
the
feat/unified-power-mode
branch
from
September 14, 2026 08:17
b76be1b to
c0b4119
Compare
Contributor
Author
|
Rebased onto latest `main` and bumped `geniex-qairt` to geniex-qairt-plugin#55, which fixes precedence so a caller-supplied `perf_profile` now wins over a bundle's `htp_backend_ext_config.json`. Dropped the now-dead bundle-override warning workaround accordingly. On-device (Snapdragon X Elite), `qualcomm/Qwen3-4B` (bundle pins `perf_profile=burst`):
No override warning fires anymore — the caller's request actually takes effect now. `llama_cpp`, `unsloth/Qwen3-0.6B-GGUF` on NPU: `--power-mode sustained_high_performance` → `ggml-hex: HTP0 power mode: sustained_high_performance`, confirmed applied before session creation. `go mod tidy`: clean. SDK + CLI build green. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
libgeniexalias table (geniex_resolve_power_mode) for HTP DCVS/HMX power/clock-management, shared by theqairtandllama_cppplugins instead of each having its own knob (or none).--power-modeongeniex infer,geniex run,geniex serve(with a per-request JSON override), andgeniex-bench.qairt: setsModelConfig::perf_profile; warns when a bundle's ownhtp_backend_ext_config.jsonoverrides the requested mode.llama_cpp: carried by two patches on top of an unmerged internal PR (ggml/llama.cpp#340) that adds the DCVS/HMX power-mode table toggml-hexagon. This PR stays draft until that upstream PR merges and thethird-party/llama.cpppin can be bumped past it — at that pointllama-hexagon-power-mode.patchshould be deleted;llama-hexagon-power-mode-setter.patchis a permanent GenieX addition.defaultresolves toburston both plugins, matching each plugin's pre-existing behavior (qairt already defaulted toBURST; llama_cpp's HTP corners were hardcoded to max).Test plan
Verified on a Snapdragon X Elite device (Windows ARM64), signed local HTP build:
geniex-bench --plugin llama_cpp --device npu --power-mode {burst,sustained_high_performance,power_saver}— each logsggml-hex: HTP0 power mode: <mode>;power_saverprefill tok/s is measurably lower thanburst.--power-modeproduces byte-identical logging/perf to--power-mode burstonllama_cppandqairt.geniex-bench --plugin qairt --device npu --power-mode balancedagainst a bundle whosehtp_backend_ext_config.jsonpinsperf_profileto burst — the plugin logs a warning and the bundle wins (both LLM and VLM vision-encoder configs).geniex-bench --plugin qairt --vlm --power-mode power_saver(VLM path) applies the mode to both the LLM and vision-encoder configs.--power-mode turbo(invalid alias) is rejected fast on both plugins, before any model load.--device cpu --power-mode balancedonllama_cpplogs a warning and ignores the mode instead of failing.pytest tests -m "api or (llama_cpp and device_cpu)"— 35/35 pass.go mod tidy— no diff.Not independently exercised here (pre-existing behavior, unrelated to this change):
geniex run's HTTP passthrough ofpower_mode, and the keep-alive cache-key change forpower_modeingeniex serve— both follow the exact pattern already used forcompute/spec_type.