11# zonnx
22
3- A standalone command-line tool for converting machine learning models to GGUF format. Supports ONNX and SafeTensors inputs, with built-in HuggingFace Hub integration for downloading models.
3+ [ ![ Go Reference] ( https://pkg.go.dev/badge/github.com/zerfoo/zonnx.svg )] ( https://pkg.go.dev/github.com/zerfoo/zonnx )
4+ [ ![ License] ( https://img.shields.io/badge/License-Apache%202.0-blue.svg )] ( https://opensource.org/licenses/Apache-2.0 )
45
5- ## Features
6-
7- - ** ONNX / SafeTensors → GGUF conversion** : Produce portable GGUF files compatible with the ` zerfoo ` runtime and llama.cpp.
8- - ** Model inspection** : Introspect model metadata, IOs, nodes and tensor stats for ONNX and GGUF files. JSON output with ` --pretty ` planned.
9- - ** HuggingFace integration** : Download ONNX models and tokenizer files in one step.
10- - ** Post-conversion quantization** : Quantize weights to Q4_0 or Q8_0 during conversion.
11- - ** CGO-free builds** : Ships as a single static binary. Easy to distribute and run in minimal containers.
12- - ** Architecture-aware mappings** : Tensor name and metadata mappings tuned per model family.
13-
14- ## Supported Models
15-
16- zonnx maps tensor names and metadata to GGUF conventions for each architecture family. The ` --arch ` flag selects the mapping.
6+ Standalone CLI for converting ONNX and SafeTensors models to GGUF format. Ships as a single static binary — zero CGo.
177
18- | Architecture | ` --arch ` value | Input Formats | Tensor Mapping | Notes |
19- | -------------| ----------------| ---------------| ----------------| -------|
20- | Llama | ` llama ` (default) | ONNX | Decoder layers (` model.layers.N.* ` ) | Llama 3, Code Llama, etc. |
21- | Gemma | ` gemma ` | ONNX | Decoder layers (` model.layers.N.* ` ) | Gemma, Gemma 2, Gemma 3 |
22- | BERT | ` bert ` | ONNX, SafeTensors | Encoder layers (` bert.encoder.layer.N.* ` ) | Classification, embeddings, pooler |
23- | RoBERTa | ` roberta ` | ONNX, SafeTensors | Encoder layers (` roberta.encoder.layer.N.* ` ) | Same layer structure as BERT |
8+ Part of the [ Zerfoo] ( https://github.com/zerfoo ) ML ecosystem.
249
25- Any architecture string can be passed via ` --arch ` . The metadata mapping is generic (maps ` hidden_size ` , ` num_hidden_layers ` , etc. to ` {arch}.* ` GGUF keys). However, tensor name mapping currently covers Llama-style decoder models and BERT/RoBERTa encoder models. Unsupported tensor name patterns pass through unchanged.
26-
27- ### Metadata Mapped
28-
29- These HuggingFace ` config.json ` fields are mapped to GGUF metadata for all architectures:
30-
31- | config.json field | GGUF key |
32- | -------------------| ----------|
33- | ` hidden_size ` | ` {arch}.embedding_length ` |
34- | ` num_hidden_layers ` | ` {arch}.block_count ` |
35- | ` num_attention_heads ` | ` {arch}.attention.head_count ` |
36- | ` num_key_value_heads ` | ` {arch}.attention.head_count_kv ` |
37- | ` intermediate_size ` | ` {arch}.feed_forward_length ` |
38- | ` vocab_size ` | ` {arch}.vocab_size ` |
39- | ` max_position_embeddings ` | ` {arch}.context_length ` |
40- | ` rms_norm_eps ` | ` {arch}.attention.layer_norm_rms_epsilon ` |
41- | ` rope_theta ` | ` {arch}.rope.freq_base ` |
42-
43- BERT/RoBERTa additionally map ` layer_norm_eps ` , ` num_labels ` , and ` pooler_type ` .
10+ ## Features
4411
45- ## Usage
12+ - ** ONNX / SafeTensors to GGUF** — produce portable GGUF files compatible with [ zerfoo] ( https://github.com/zerfoo/zerfoo ) and llama.cpp
13+ - ** Post-conversion quantization** — quantize weights to Q4_0 or Q8_0 during conversion
14+ - ** HuggingFace integration** — download ONNX models and tokenizer files in one step
15+ - ** Model inspection** — introspect metadata, IOs, nodes, and tensor stats for ONNX and GGUF files
16+ - ** Architecture-aware mappings** — tensor name and metadata mappings tuned per model family
17+ - ** CGo-free** — single static binary, easy to distribute and run in minimal containers
4618
47- ### Installation
19+ ## Installation
4820
4921``` bash
5022go install github.com/zerfoo/zonnx/cmd/zonnx@latest
@@ -56,92 +28,98 @@ Or build from source:
5628go build -o zonnx ./cmd/zonnx
5729```
5830
59- Requires Go 1.26+. CGO is not required ( ` CGO_ENABLED=0 ` works) .
31+ Requires Go 1.26+. ` CGO_ENABLED=0 ` works.
6032
61- ### Quickstart
33+ ## Quick Start
6234
6335``` bash
64- # 1) Download an ONNX model and tokenizer files from HuggingFace
36+ # Download an ONNX model from HuggingFace
6537zonnx download --model google/gemma-2-2b-it --output ./models
6638
67- # 2) Convert ONNX → GGUF
39+ # Convert ONNX to GGUF
6840zonnx convert --arch gemma --output ./models/model.gguf ./models/model.onnx
6941
70- # 3) Convert SafeTensors → GGUF (pass directory containing config.json + model.safetensors)
42+ # Convert SafeTensors to GGUF
7143zonnx convert --format safetensors --arch bert --output ./models/model.gguf ./models/bert-dir/
7244
73- # 4) Convert with quantization
45+ # Convert with quantization
7446zonnx convert --quantize q4_0 --output ./models/model-q4.gguf ./models/model.onnx
7547
76- # 5) Inspect either format
77- zonnx inspect --pretty ./models/model.onnx
48+ # Inspect a model file
7849zonnx inspect --pretty ./models/model.gguf
7950```
8051
81- ### Commands
52+ ## Supported Architectures
8253
83- #### ` convert `
54+ | Architecture | ` --arch ` | Input Formats | Notes |
55+ | -------------| ----------| ---------------| -------|
56+ | Llama | ` llama ` (default) | ONNX | Llama 3, Code Llama |
57+ | Gemma | ` gemma ` | ONNX | Gemma, Gemma 2, Gemma 3 |
58+ | BERT | ` bert ` | ONNX, SafeTensors | Classification, embeddings |
59+ | RoBERTa | ` roberta ` | ONNX, SafeTensors | Same layer structure as BERT |
8460
85- Convert ONNX or SafeTensors models to GGUF .
61+ Any architecture string can be passed via ` --arch ` . Metadata mapping is generic; tensor name mapping currently covers decoder (Llama-style) and encoder (BERT/RoBERTa) models .
8662
87- ``` bash
63+ ## Commands
64+
65+ ### ` convert `
66+
67+ ```
8868zonnx convert [flags] <input>
8969```
9070
9171| Flag | Default | Description |
9272| ------| ---------| -------------|
93- | ` --output ` | ` <input-dir>/<input-base >.gguf ` | Output GGUF file path |
73+ | ` --output ` | ` <input>.gguf ` | Output GGUF file path |
9474| ` --arch ` | ` llama ` | Model architecture for metadata/tensor mapping |
9575| ` --format ` | ` onnx ` | Input format: ` onnx ` or ` safetensors ` |
9676| ` --quantize ` | (none) | Quantize weights: ` q4_0 ` or ` q8_0 ` |
9777
98- For ONNX input, ` <input> ` is a ` .onnx ` model file. For SafeTensors, ` <input> ` is a directory containing ` config.json ` and ` model.safetensors ` .
99-
100- #### ` download `
78+ ### ` download `
10179
102- Download an ONNX model and tokenizer files from HuggingFace Hub.
103-
104- ``` bash
80+ ```
10581zonnx download --model <huggingface-model-id> [--output <dir>] [--api-key <key>]
10682```
10783
108- | Flag | Default | Description |
109- | ------| ---------| -------------|
110- | ` --model ` | (required) | HuggingFace model ID (e.g., ` google/gemma-2-2b-it ` ) |
111- | ` --output ` | ` . ` | Output directory |
112- | ` --api-key ` | ` $HF_API_KEY ` | HuggingFace API key for authenticated downloads |
113-
11484The ` --api-key ` flag takes precedence over the ` HF_API_KEY ` environment variable.
11585
116- #### ` inspect `
117-
118- Inspect ONNX or GGUF model files.
86+ ### ` inspect `
11987
120- ``` bash
88+ ```
12189zonnx inspect [--type onnx|gguf] [--pretty] <input-file>
12290```
12391
12492Type is inferred from file extension when not specified.
12593
126- #### ` import ` / ` export `
94+ ## Metadata Mapped
12795
128- Future-friendly aliases. ` import ` is an alias for ` convert ` . ` export ` ( GGUF → ONNX) is planned.
96+ These HuggingFace ` config.json ` fields are mapped to GGUF metadata for all architectures:
12997
130- ## Architectural Principles
98+ | config.json field | GGUF key |
99+ | -------------------| ----------|
100+ | ` hidden_size ` | ` {arch}.embedding_length ` |
101+ | ` num_hidden_layers ` | ` {arch}.block_count ` |
102+ | ` num_attention_heads ` | ` {arch}.attention.head_count ` |
103+ | ` num_key_value_heads ` | ` {arch}.attention.head_count_kv ` |
104+ | ` intermediate_size ` | ` {arch}.feed_forward_length ` |
105+ | ` vocab_size ` | ` {arch}.vocab_size ` |
106+ | ` max_position_embeddings ` | ` {arch}.context_length ` |
107+ | ` rms_norm_eps ` | ` {arch}.attention.layer_norm_rms_epsilon ` |
108+ | ` rope_theta ` | ` {arch}.rope.freq_base ` |
109+
110+ BERT/RoBERTa additionally map ` layer_norm_eps ` , ` num_labels ` , and ` pooler_type ` .
131111
132- zonnx is strictly decoupled from the ` zerfoo ` runtime:
112+ ## Design Principles
133113
134- - ** GGUF-only output** : Emits only GGUF files. No runtime code.
135- - ** No ` zerfoo ` imports** : The zonnx codebase does not import ` github.com/zerfoo/zerfoo ` .
136- - ** No ONNX in ` zerfoo ` ** : The zerfoo runtime consumes only GGUF models.
137- - ** Explicit schema** : GGUF output captures all model attributes directly, without relying on ONNX runtime semantics.
114+ - ** GGUF-only output** — emits only GGUF files, no runtime code
115+ - ** No ` zerfoo ` imports** — strictly decoupled from the inference runtime
116+ - ** Explicit schema** — GGUF output captures all model attributes directly
138117
139118## Development
140119
141120``` bash
142121make test # go test ./...
143122make lint # golangci-lint run
144- make lint-fix # golangci-lint run --fix
145123make format # gofmt + goimports
146124```
147125
0 commit comments