cache-eviction

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

SIEVE cache - lockfree, concurrent cache - faster and simpler than LRU

A comprehensive open-source cache trace dataset

Hardware-level security protocol for mitigating cache-based side-channel attacks.

A prefix-cache advisor for LLM serving infrastructure that recommends KV-cache capacity and eviction policies from your request traces/logs.

A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.

Idle but not forgotten: usage-driven agent memory forgets the dormant skills you'll need next quarter; NicheMem's competitive ownership retains them exactly. Mechanism-level benchmark + verified theory (sibling of GAUSE).

CLOCK replacement algorithm, an efficient, compact and concurrent alternative to LRU cache eviction policy.

Trace-driven research harness for KV-cache hierarchy policy evaluation in long-context LLM inference.

REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

An implementation of W-TinyLFU policy

HybridCacheSystem is a modern C++ framework for building high-performance, multi-layered cache systems. It supports both LRU and LFU caching strategies, allows combining them for hybrid setups, and provides a unified interface for cache management and customization.

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

Trace-driven simulator for learned KV Cache eviction on synthetic sparse block-level access traces

Educational caching reverse proxy in Go—TTL-based in-memory HTTP cache, size-aware eviction, gzip negotiation, Cache-Control handling, hit/miss headers, and JSON stats.

Benchmark of KV-cache eviction policies on GPT-2-medium showing two clear regimes: sink-preserving heuristics dominate at tiny budgets, while attention-based eviction becomes near-lossless at moderate budgets. attention_384 matches full-cache quality with ~33% less cache.

Semantic-cache eviction that refuses to trade a cache miss for a silently wrong answer

A small caching library implementing standard caching strategies, starting with LRU, in Go. Part of the multi-language mnemosyne-cache project. Built by hand to develop real, working fluency in caching mechanics rather than just theory.

Neutral, reproducible comparison of Go in-process cache libraries: hit rate and per-op cost across synthetic workloads and real traces, with the provenance to regenerate every number.

A small caching library implementing standard caching strategies, starting with LRU, in Python. Part of the multi-language mnemosyne-cache project. Built by hand to develop real, working fluency in caching mechanics rather than just theory.