Author(s): Yuri Samarin - QVAC Team
Last Update: Sep 7, 2026
Related Documents & Links
- C4 Model Reference
- Agent Integrations - AI SDK provider, OpenCode plugin, CLI serve, models.dev, layer ownership, and release workflow
QVAC SDK is a local-first AI platform for JavaScript, Bare, and Python applications. The architecture is split by responsibility:
@qvac/inferenceis the engine: dispatch, plugins, models, P2P, RAG, schemas, and request lifecycle. Direct Bare apps import this package and call it in process.@qvac/sdkis the TypeScript host for Node.js, Bun, Electron, Expo/React Native, and Pear. It owns the client, worker shell, RPC, bundling, and the default all-plugin distribution. The worker runs@qvac/inferenceinside Bare and re-exports the engine public API.tetherto-qvac-sdkis the generated Python client. It uses the same worker RPC contract as TypeScript.
@qvac/bare-sdk is deprecated. Bare consumers use @qvac/inference.
The core execution model is the same across packages:
- Plugins implement
QvacPlugin, provide Zod schemas, create model instances, and expose unary, server-streaming, or duplex handlers. - Host clients either call the engine in process or send the same request envelope to a Bare worker over the platform transport.
- Models load and execute in the local engine. The runtime model registry contains local model instances only.
- Model distribution uses HTTP, the QVAC Model Registry, Hyperdrive, or local filesystem paths.
Key - Blue: system in scope; grey: external systems; arrows: intent [transport]. Registry catalog and model bytes use Holepunch (Hyperswarm plus a Hypercore blob store). HTTP/CDN is a separate HTTPS path.
The family is one software system at this zoom. Package choice is in the executive summary. Runtime boxes are on the container diagram.
Key - Blue: runtime boxes; grey: external systems; cylinder: data store; arrows: intent [transport]. External I/O leaves @qvac/inference. Holepunch includes the QVAC Model Registry. Host RPC transports are in the deployment table.
Host, worker, and @qvac/inference are the runtime path for @qvac/sdk and Python. Direct Bare skips this diagram's host and worker; it is the right-hand deployment topology.
Key - Host process above, @qvac/inference below. Host to engine is bare-rpc (or in-process on direct Bare). Internal engine arrows are in-process. Python uses the same public API, generated from the wire contract.
Physical process placement is covered in the Deployment Diagram.
Each AI capability is an independent plugin. A QvacPlugin defines its canonical modelType, addon metadata, load-config schema, model factory, optional artifact resolution, and handler set. Handlers declare Zod request/response schemas and whether they are unary, server-streaming, or duplex.
The plugin registry provides uniform dispatch for built-in and custom plugins. Distribution-specific registration rules live in Worker Generation & Bundle System.
Built-in plugins:
| Plugin | Model Type | Wraps |
|---|---|---|
| LLM Completion | llamacpp-completion |
@qvac/llm-llamacpp |
| Embeddings | llamacpp-embedding |
@qvac/embed-llamacpp |
| Whisper | whispercpp-transcription |
@qvac/asr-ggml |
| BCI Whisper | bci-whispercpp-transcription |
@qvac/bci-whispercpp |
| Parakeet | parakeet-transcription |
@qvac/asr-ggml |
| NMT | nmtcpp-translation |
@qvac/translation-nmtcpp |
| TTS | tts-ggml |
@qvac/tts-ggml |
| OCR | ggml-ocr |
@qvac/ocr-ggml |
| Diffusion / Video / Upscale | sdcpp-generation |
@qvac/diffusion-cpp |
| Audio Generation | audiogen-ggml |
@qvac/audiogen-ggml |
| Vision-Language-Action | ggml-vla |
@qvac/vla-ggml |
| Classification | ggml-classification |
@qvac/classification-ggml |
Model types follow an engine-usecase naming convention. Backward-compatible aliases (llm, whisper, bci, embeddings, nmt, parakeet, tts, ocr, diffusion, audiogen, vla, classification) are supported and normalized to canonical types.
Custom plugins ship as npm packages whose plugin manifest is imported through a /plugin subpath.
Plugin Invocation Flow:
Streaming uses invokePluginStream (async generator, newline-delimited JSON). Transport is the same as the RPC flow.
Plugin registration is determined by the package and runtime path:
@qvac/inferenceregisters no plugins by default. Consumers assemble the engine explicitly withplugins([...])orregisterPlugin(...).@qvac/sdkships a default worker that runs@qvac/inferencewith every built-in plugin registered.qvac bundle sdkgenerates an optimized worker entry for the plugin list inqvac.config.{json,js,ts}.
The bundle command emits:
qvac/worker.entry.mjs- standalone worker entry with RPC and lifecycle, used by desktop/Electron packaging.qvac/worker.pear.entry.mjs- Pear worker entry (same process), generated by the Pear pre-hook.qvac/worker.bundle.js- mobile bundle for Expo/React Native BareKit.qvac/addons.manifest.json- native addon manifest derived from the bundle.
TypeScript desktop worker resolution:
| Priority | Source | Description |
|---|---|---|
| 1 | QVAC_WORKER_PATH env var |
Explicit path override |
| 2 | Packaged Electron worker | resources/.../qvac/worker.entry.mjs |
| 3 | qvac/worker.entry.mjs in project root |
Output of npx qvac bundle sdk |
| 4 | Default SDK worker | Fallback that runs @qvac/inference with all built-in plugins |
Key - Nested boxes: deployment environment; blue: runtime instances; cylinder: persistent storage; arrows: intent [transport]. Left is host plus worker: subprocess on Node/Python/Electron, same process on Pear (in-process) and Expo (BareKit). Right is direct Bare: the app imports @qvac/inference and there is no SDK worker.
| Platform | Host | Worker placement | Transport |
|---|---|---|---|
| Node.js / Bun / Electron | @qvac/sdk |
Bare subprocess | Unix socket or named pipe |
| Python | tetherto-qvac-sdk |
Bare subprocess | Loopback TCP |
| Pear | @qvac/sdk |
Same process | In-process |
| Expo / React Native | @qvac/sdk |
BareKit worklet | Native bridge |
| Direct Bare | @qvac/inference |
None | In-process |
Native addon packaging follows the deployment target: Node/Bun use installed prebuilds, Electron and Expo/RN package native addons with the app, and Bare/Pear builds include the addons selected by the authored or generated worker entry.
Worker-backed clients use the same JSON request/response envelopes over different transports:
- TypeScript Node/Bun/Electron clients use
bare-rpcover a Unix socket or Windows named pipe. - Expo clients use
bare-rpcover the BareKit worklet IPC bridge. - Python clients use
bare-rpc-pythonover loopback TCP (127.0.0.1:0) because asyncio has no cross-platform Unix-socket/named-pipe server.
In-process Bare (@qvac/inference) bypasses sockets and calls the dispatch layer directly.
Model Constants: Model constants are rich objects (not plain strings) containing metadata such as name, src, modelId, blobCoreKey, expectedSize, sha256Checksum, and addon. APIs accept string URIs, local paths, descriptor objects, and model constants via the ModelSrcInput union type. Python receives the same constants from packages/sdk/contract/models.json.
Key - Host API calls engine handlers. Operations: chunk, ingest, search, delete, reindex, plus workspace create/open/close.
Workspace Isolation: Each workspace is bound to a specific embedding model at creation. Documents from different workspaces cannot be mixed.
The SDK includes a client for the QVAC Model Registry (@qvac/registry-client), providing catalog-based model discovery. Client APIs: modelRegistryList, modelRegistrySearch, modelRegistryGetModel. Models discovered through the registry can be loaded via loadModel() or pre-downloaded via downloadAsset().
The language-neutral SDK contract lives under packages/sdk/contract/**:
schema.jsoncontains JSON Schema for every request/response wire type and registered public constants.manifest.jsonlists every RPC method and call shape (request-reply,server-stream,duplex).models.jsoncontains the generated model constants catalog.
packages/sdk owns contract generation via bun run contract:export and drift detection via bun run contract:check. packages/sdk-python consumes the contract to generate Pydantic models, typed method stubs, model type maps, error-code registries, model constants, and the pinned SDK version.
The Python package provides:
- A flat public API under
tetherto.qvac_sdk, plustetherto.qvac_sdk.modelsfor model constants. - Ergonomic wrappers for common calls, generated stubs for the full RPC surface, typed errors, logging/profiling helpers, VLA helpers, and a notebook
SyncClientfacade. - Thin PyPI installs that resolve a worker from explicit paths,
QVAC_*env vars, a local@qvac/sdk, a managedinstall-workercache, or global npm. - Self-contained release wheels that bundle the worker and Bare runtime for supported platforms.
| Boundary | Mechanism |
|---|---|
| P2P Model Distribution | Noise protocol encryption (Hyperswarm default) |
| Model Integrity | SHA256 checksum verification (model constants include checksums; optional for custom URLs) |
| Path Security | Path traversal protection for model file resolution |
| Local Worker RPC | Local IPC/loopback only; trusted local process model |
| Local Storage | No encryption at rest; relies on OS-level file permissions |
Not in scope: Authentication/authorization for local API calls (SDK runs as trusted local process).
| Failure | Behavior |
|---|---|
| Download interrupted | Partial file cached; resume on retry (HTTP range requests, Hyperdrive sparse sync) |
| Model load fails (corrupt/incompatible) | Error with cause chain; model not registered |
| Native addon crash | Server process may terminate; client receives RPC error |
| Server process OOM | OS kills subprocess; client receives RPC connection error and must restart SDK |
| Worker crash during an in-flight request | Client life-signal race rejects pending calls instead of hanging |
| Plugin not enabled | Fast-fail with a plugin registration / no-handler error and guidance to configure or register the plugin |
@qvac/inference call before plugin assembly |
Fast-fail with guidance to call plugins([...]) or registerPlugin(...) |
Cancellation: cancel({ requestId }) is the preferred targeted path for migrated long-running operations, including completion/batch completion, embeddings, transcription, translation, fine-tuning, model loading, asset downloads, RAG, and audio generation. Broad cancellation by modelId remains available for shutdown, unload, and admin sweeps.
- Addon C++ Framework
- LLM Completion - llama.cpp
- Embeddings - llama.cpp
- Speech-to-text - ASR GGML
- Translation - nmt.cpp
- Diffusion - stable-diffusion.cpp
- Classification - GGML
Logging: Logs span host client, worker core, and native addons. Addon logs are forwarded to JS via registered callbacks (plugins can configure this via logging.module and logging.namespace). A streaming mechanism (loggingStream / subscribeServerLogs) allows real-time log forwarding from subprocess to client for debugging UIs. Log level and console output are configurable via qvac.config and Python Client(config=...).
Error Handling: All SDK errors expose a numeric code property for programmatic handling, with original errors preserved via cause chain. Errors are structured classes extending QvacErrorBase. Client (50,001-52,000) and server (52,001-54,000) error codes are strictly separated.
Worker Lifecycle: The SDK worker shell frames RPC, acquires the process lock, and calls @qvac/inference send / stream / duplex. Startup registers SIGTERM/SIGINT handlers, registers built-in plugins on the engine, then ensureRPCSetup() creates the IPC client (desktop) or BareKit RPC server (mobile). Direct Bare imports @qvac/inference and skips the worker. On termination, the engine clears registries, unloads models, closes RAG instances, cancels downloads, and closes the registry client; those shutdown paths tear down their P2P resources before the worker lock is released where the runtime owns process exit.
Request Lifecycle: Long-running operations run through request lifecycle primitives (RequestRegistry, RequestContext, DisposableScope) that provide request IDs, cancellation, structured cleanup, concurrency policy, and per-request logging. Client-side completion, loadModel, and downloadAsset expose request IDs synchronously so callers can cancel in-flight work.
Most packages live in this monorepo under packages/. Integration plugins live under plugins/, and the documentation site lives under docs/website.
SDK & CLI
| Directory | Package | Purpose |
|---|---|---|
sdk |
@qvac/sdk |
TypeScript host: public API, worker shell, RPC transports, bundling; re-exports @qvac/inference |
bare-sdk |
@qvac/bare-sdk |
Removed from the repo; last npm release 0.18.2. Bare consumers use @qvac/inference |
inference |
@qvac/inference |
Engine: dispatch, plugins, models, P2P, RAG, schemas; Bare in-process, no worker/RPC |
sdk-python |
tetherto-qvac-sdk |
Generated Python client for the SDK worker contract |
cli |
@qvac/cli |
CLI tooling (qvac bundle sdk, verification, release helpers) |
ai-sdk-provider |
@qvac/ai-sdk-provider |
Vercel AI SDK provider integration |
test-suite |
@qvac/test-suite |
Distributed MQTT test-orchestration framework (qvac-test CLI) driving the SDK e2e suites across desktop, Electron, Snap, Android, and iOS |
plugins/opencode |
@qvac/opencode-plugin |
OpenCode integration |
plugins/openclaw |
@qvac/openclaw-plugin |
OpenClaw integration |
docs/website |
- | Documentation site (Next.js / Fumadocs) |
Inference Addons
| Directory | Package | Purpose |
|---|---|---|
llm-llamacpp |
@qvac/llm-llamacpp |
LLM completion (llama.cpp) |
embed-llamacpp |
@qvac/embed-llamacpp |
Text embeddings (llama.cpp) |
asr-ggml |
@qvac/asr-ggml |
Whisper and Parakeet speech-to-text (GGML) |
bci-whispercpp |
@qvac/bci-whispercpp |
BCI neural-signal transcription |
translation-nmtcpp |
@qvac/translation-nmtcpp |
Translation (nmt.cpp) |
tts-ggml |
@qvac/tts-ggml |
Text-to-speech (GGML) |
ocr-ggml |
@qvac/ocr-ggml |
OCR (GGML) |
diffusion-cpp |
@qvac/diffusion-cpp |
Image/video generation and upscaling |
audiogen-ggml |
@qvac/audiogen-ggml |
Text-conditioned audio generation |
vla-ggml |
@qvac/vla-ggml |
Vision-language-action inference |
classification-ggml |
@qvac/classification-ggml |
Image classification |
Support Libraries
| Directory | Package | Purpose |
|---|---|---|
rag |
@qvac/rag |
RAG with HyperDB |
decoder-audio |
@qvac/decoder-audio |
Audio decoding |
logging |
@qvac/logging |
Logging utilities |
error |
@qvac/error |
Base error types |
langdetect-text |
@qvac/langdetect-text |
Language detection |
registry-server |
process package plus @qvac/registry-client / shared packages |
Distributed model registry service and client/shared contracts |