awq-quantization

Star

Here are 5 public repositories matching this topic...

nicosuter / Qwen3.8-27B-AWQ

Star

AWQ Quantization of Qwen3.8 27B

awq qwen awq-quantization qwen3-8

Updated Aug 24, 2026
Python

G-B-KEVIN-ARJUN / microservices

Star

scaleDown AI: Enterprise Model Quantization Platform Slash inference costs by 70%. Deploy LLMs anywhere. A microservices-based orchestration engine for isolating and automating incompatible AI optimization workflows.

server pytorch cicd quantization onnx fastapi onnxruntime awq-quantization

Updated Jan 28, 2026
Python

ncoder-ai / VibeVoice-awq-engine

Star

Embeddable Python Engine for VibeVoice TTS with AWQ-INT4 quantization (50% VRAM, 2x Faster RTF) of VibeVoice Large 7B model

tts vibevoice-large awq-quantization

Updated May 7, 2026
Python

imanshrajsingh-boost / qwen2.5-coder-finetuning-qlora-dpo

Star

Fine-tuning pipeline for Qwen2.5-Coder-1.5B using QLoRA, DPO alignment, and AWQ quantization on free-tier Colab GPU.

python code python3 lora fine-tuning finetuning sft dpo huggingface huggingface-transformers awq llm lm-evaluation-harness bitsandbytes finetuning-llms awq-quantization qwen2-5-coder lm-eval lm-eval-harness

Updated Aug 19, 2026
Jupyter Notebook

ankur110 / roofline-llama

Star

A from-scratch implementation of Llama-3.2-1B in PyTorch, decode-latency benchmarks on three GPUs (T4, L4, A100), three weight-only quantization methods (RTN, GPTQ, AWQ) measured against both, and a packed int4 format with a fused Triton GEMV so the quantized weights are actually 4 bits in HBM.

python ml inference pytorch triton quantization rtn gqa kv-cache awq llm gptq flash-attention awq-quantization int4-quantization

Updated Sep 3, 2026
Jupyter Notebook

Add this topic to your repo

To associate your repository with the awq-quantization topic, visit your repo's landing page and select "manage topics."

Learn more