- [2026/09] 🦩 FluxServe is announced — a flexible and high-performance inference engine for diffusion language models. [blog]
FluxServe is a lighweight and high-performance serving engine for diffusion langauge models. It is designed and implemented to deliver low-latency and high-throughput inference for autoregressive (AR) diffusion models across different setups, ranging from single GPU batched inference to multi-GPU distributed serving.
Its core features include:
- Native Block-Causal Attention: Provides efficient attention runtime with a block-casual attention mechanism suitable for AR diffusion in real-world scenarios, including varlen prefill and varlen block-deocde with Flashinfer and FA4 support.
- Dynamic Request Scheduler: Provides scheduler with low-overhead C++ control plane and Python execution plane with fine-grained block-level request management suitable for block diffusion models.
- Open Model Support: Provides native support for LLaDA2.X, Diffusion-Gemma, and Nemotron-Labs-Diffusion.
- Efficient Multi-GPU Serving: Provides tensor paralllel (TP), data parallel (DP) and expert parallel (EP) support for large-scale models.
We learned the system design and reused code from the following projects: vLLM, SGLang, TokenSpeed, dInfer, FlashInfer, and Flash-Attention.
@misc{fluxserve2026,
author = {{FluxServe Team}},
title = {FluxServe: A Flexible and High-Performance Inference Engine for Open Diffusion Language Models},
year = {2026},
month = {September},
howpublished = {\url{https://github.com/FLX-OSS/FluxServe}}
}