Discrete-event simulation benchmark for graceful degradation strategies in LLM serving: output capping, CoT disabling, model downgrade, and selective drop under progressive pressure.
performance-engineering benchmark multi-tenant simulation inference systems slo quality-of-service discrete-event-simulation kv-cache llm graceful-degradation llm-serving batch-scheduling machine-learning-infrastructure model-routing serving-systems overload-management
-
Updated
Jul 26, 2026 - Python