DiffGuard

100% offline code review, powered by MLX on Apple Silicon.

Built for the Arm Create: AI Optimization Challenge — Mobile AI track.

The problem

Cloud AI code-review tools mean your proprietary code leaves your machine. For teams under strict confidentiality policies (finance, healthcare, defense, or just companies that don't want their code training someone else's model), that's a hard blocker — not a preference.

DiffGuard reviews your uncommitted local changes entirely on your own Mac. Nothing is ever sent anywhere.

What it does

1. Reads your local uncommitted git diff
2. Builds a lightweight local RAG index over the rest of your codebase, so the review understands your existing conventions, not just the diff in isolation
3. Runs a code-specialized small LLM (Qwen2.5-Coder-3B) on-device via MLX — Apple's own ML framework, built specifically to exploit Apple Silicon's unified memory and Metal GPU
4. Prints a structured, severity-ranked review to your terminal
5. Flags any finding that doesn't actually match text in your real diff as [unverified], with a visible X/Y issues verified badge on every run — a built-in check against the model inventing an issue that isn't really there

The optimization story

benchmark.py runs the exact same model, at the exact same 4-bit quantization, two ways:

• MLX — Metal-accelerated, Arm-native
• llama.cpp, CPU-only — same quantization, no GPU acceleration

This isolates what Apple Silicon's Arm-specific acceleration path actually buys you, rather than comparing different models or quantization levels.

Results (measured on an 8GB Apple Silicon Mac)

Same model (Qwen2.5-Coder-3B-Instruct), same 4-bit quantization, two backends:

Backend	Load time	Generation	Tokens/sec
MLX (Metal, Arm-native)	2.93s	7.52s	26.6
llama.cpp (CPU only, no Metal)	13.5s	11.34s	17.6

MLX is ~1.5x faster at generation and ~4.6x faster to load, on identical model weights and quantization — the only variable is whether the Arm-native Metal acceleration path is used. Raw output saved in examples/benchmark_result.json.

Power draw (measured via macOS powermetrics)

benchmark_power.py measures real system power (CPU + GPU + ANE combined) while each backend runs the same generation task:

Backend	Avg Watts	Watts per 1K tokens
MLX (Metal, Arm-native)	5.77W	230.8
llama.cpp (CPU only, no Metal)	6.35W	283.5

MLX draws ~9.1% less power and is ~18.6% more power-efficient per token generated. This directly speaks to the Mobile AI track's battery-efficiency criterion — the Arm-native Metal path isn't just faster, it costs less energy per unit of work. Raw output saved in examples/power_benchmark_result.json.

Tokens/sec here (25.0) is close to but not identical to the speed benchmark above (26.6) — powermetrics sampling in the background adds a small measurement overhead. Both backends were measured under identical sampling conditions, so the watts and efficiency comparison is still valid.

Model size (measured via Hugging Face Hub metadata, no download needed)

benchmark_size.py compares the unquantized fp16 baseline against the MLX 4-bit variant actually used by DiffGuard:

Variant	Size on disk
fp16 (unquantized)	6.17 GB
MLX 4-bit (Arm-optimized)	1.74 GB

71.9% smaller — same model, same architecture, only numeric precision changed. Raw output saved in examples/size_benchmark_result.json.

Setup

Requires a Mac with Apple Silicon (M1 or later) and Python 3.10+.

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

A note on "100% offline": the first time you run main.py, it needs internet to download the embedding model and the MLX review model (~2GB total, one-time). After that first run, DiffGuard detects both models are already cached and automatically runs in fully offline mode — verified by testing with Wi-Fi disabled after the initial download.

Usage

Review the uncommitted changes in any git repo:

python3 main.py /path/to/some/repo

(defaults to the current directory if no path given)

Run the benchmarks:

python3 benchmark.py          # speed: MLX vs CPU-only
sudo python3 benchmark_power.py   # power draw (requires sudo, macOS only)
python3 benchmark_size.py     # model size (no download needed)

First run of benchmark.py/benchmark_power.py downloads both model variants (a few GB total) — subsequent runs are fast.

Project structure

diffguard/
├── main.py                       # CLI entry point — run this to review a diff
├── benchmark.py                   # MLX vs CPU-only speed benchmark
├── benchmark_power.py              # MLX vs CPU-only power draw benchmark
├── benchmark_size.py                # fp16 vs MLX 4-bit model size comparison
├── requirements.txt
├── src/
│   ├── git_diff.py                # reads local uncommitted changes
│   ├── retrieval.py                # local RAG index over the codebase
│   └── review.py                    # the on-device LLM review step (MLX)
└── examples/
    ├── benchmark_result.json               # generated by benchmark.py
    ├── power_benchmark_result.json          # generated by benchmark_power.py
    ├── size_benchmark_result.json            # generated by benchmark_size.py
    └── sample_review.md                       # a real, unedited review transcript