LAGEN: Are Vision-Language Agents Latency Aware?

Training agents for a world that keeps moving.

Overview

For reuse in another project, install the framework wheel. The default package needs only NumPy and PyYAML.

LAGEN measures how deployment latency affects vision-language agents. It provides evaluation, demonstration collection, latency measurement, and profile replay. Use these stages with your own environment, policy, and training framework.

The repository also provides reference recipes for games, Gymnasium tasks, MIKASA manipulation, and HumanoidBench balance. Sample Factory supplies reference teacher training. The pinned StarVLA source supplies reference VLA training and inference.

Released resources

Resource	Contents
Models	Teachers, VLA weights, model configurations, task contracts
Datasets	Demonstrations, training data, evaluation records, experiment manifests
Profiles	Measured traces, fitted latency profiles, calibration records

The artifact guide identifies fixed revisions, paths, evaluation protocols, and publication status. Browse the resources in the LAGEN collection.

Choose a starting point

Your input	Start here
A published or local model	Select artifacts, then evaluate
Your own environment or policy	Run the CPU example, then replace its factories
A teacher or training dataset	Collect demonstrations or use a task cookbook
A model to measure	Measure and replay latency

The custom CPU example does not require an NVIDIA GPU. Reference VLA and GPU simulator recipes require their selected model and simulator dependencies. See the installation guide for each installation path.

Clone the source:

git lfs install
git clone --filter=blob:none https://github.com/ZihanWang314/latency-sensitive-bench.git
cd latency-sensitive-bench

Install the package in a Python 3.10 environment to run the CPU example:

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[runtime]"
export PYTHONPATH="$PWD/configs/examples/custom_env${PYTHONPATH:+:$PYTHONPATH}"
python -m latency_bench eval --config configs/examples/custom_env/eval.yaml

The installation guide covers reference GPU environments and task dependencies. The workflow guide describes each stage's inputs and local outputs.

Documentation

Start here	Purpose
Documentation index	Find guides, cookbooks, references, and experiment recipes
Installation	Install the selected task and model runtime
Workflow	Start from an existing teacher, dataset, model, or profile
Custom components	Run the complete CPU example with your own environment and policy
Task cookbooks	Train, collect, evaluate, and measure reference tasks
Artifacts	Select published inputs and their fixed source protocols

Citation

If you use LAGEN in your research, cite:

@misc{chafekar2026lagen,
  title={LAGEN: Are Vision-Language Agents Latency Aware?},
  author={Talha Chafekar and Zihan Wang and Zeju Li and Xinyuan Li and Qineng Wang and Jiajun Wu and Yuke Zhu and Yi Dong and Zhiding Yu and Ruohan Zhang and Manling Li},
  year={2026}
}