distributed-rate-limiter

distributed-rate-limiter is a production-style Spring Boot service that demonstrates how distributed rate limiting is typically implemented in API gateways and microservice platforms. It uses Redis as shared state, supports multiple algorithms, and exposes clean REST APIs for rule management and request admission.

The codebase is modular, practical to run locally, and structured so a backend team can extend it without rewriting the core rate limiting logic.

Project Overview

This service solves a common platform problem: preventing backend overload and abuse when traffic arrives from many clients across many app instances.

It supports:

• per-user limits
• per-IP limits
• per-endpoint limits
• Token Bucket
• Fixed Window Counter
• Sliding Window Log
• Redis-backed shared state for horizontal scaling
• rule creation and rule listing APIs
• request check and request consume APIs

Why Redis

Redis is a strong fit for distributed rate limiting because it offers:

• very fast reads and writes
• shared state across multiple service instances
• atomic primitives such as INCR, sorted sets, hashes, expirations, and Lua scripts
• predictable data structures for counters, windows, and token state

If multiple Spring Boot instances are deployed behind a load balancer, they all talk to the same Redis cluster. That lets the limiter enforce one global view of quota instead of each node maintaining its own local counters.

Architecture

                    +----------------------+
                    | Clients / API Gateway|
                    +----------+-----------+
                               |
                               v
                    +----------------------+
                    | Spring Boot REST API |
                    |  /rules              |
                    |  /check              |
                    |  /consume            |
                    |  /health             |
                    +----------+-----------+
                               |
                 +-------------+--------------+
                 |                            |
                 v                            v
     +-------------------------+   +-------------------------+
     | Rule Service            |   | Rate Limit Service      |
     | - validate rules        |   | - match applicable      |
     | - persist rule configs  |   |   rules by scope        |
     +------------+------------+   | - aggregate decisions   |
                  |                +------------+------------+
                  |                             |
                  v                             v
         +-------------------+       +------------------------+
         | Rule Repository   |       | Strategy Registry      |
         | Redis hash store  |       | Token Bucket           |
         +---------+---------+       | Fixed Window Counter   |
                   |                 | Sliding Window Log     |
                   |                 +------------+-----------+
                   |                              |
                   +--------------+---------------+
                                  |
                                  v
                        +-------------------+
                        | Redis             |
                        | - counters        |
                        | - sorted sets     |
                        | - token buckets   |
                        | - stored rules    |
                        +-------------------+

Package Structure

distributed-rate-limiter
├── Dockerfile
├── docker-compose.yml
├── pom.xml
├── README.md
├── src
│   ├── main
│   │   ├── java/com/example/distributedratelimiter
│   │   │   ├── controller
│   │   │   ├── service
│   │   │   ├── strategy
│   │   │   ├── model
│   │   │   ├── repository
│   │   │   ├── config
│   │   │   ├── exception
│   │   │   └── dto
│   │   └── resources
│   │       ├── application.yml
│   │       └── lua
│   └── test
│       └── java/com/example/distributedratelimiter
│           ├── service
│           ├── controller
│           └── test
└── .gitignore

Key Classes And Responsibilities

• RateLimitRuleController: creates and lists rate limit rules.
• RateLimitController: exposes /check and /consume.
• HealthController: returns application and Redis health.
• DefaultRateLimitRuleService: validates rule definitions and applies defaults.
• DefaultRateLimitService: finds applicable rules for a request and combines strategy results into one final decision.
• RedisRateLimitRuleRepository: stores rule metadata in Redis so all app instances see the same configuration.
• RateLimiterStrategy: common contract for algorithms.
• RateLimiterStrategyRegistry: resolves the correct strategy using the Strategy Pattern.
• TokenBucketRateLimiterStrategy: smooths traffic and supports burst handling.
• FixedWindowCounterRateLimiterStrategy: simple counter-based limiting.
• SlidingWindowLogRateLimiterStrategy: more accurate rolling-window limiting.
• Lua scripts in src/main/resources/lua: guarantee atomic decision logic inside Redis.

APIs

POST /rules

Create a new rule.

GET /rules

List all configured rules.

POST /check

Non-mutating eligibility check. This is useful for diagnostics or advisory decisions.

POST /consume

Authoritative quota consumption call. This is the endpoint to use when enforcing rate limits.

GET /health

Returns service and Redis health.

Example Rules

10 requests per second per user with Token Bucket

{
  "name": "user-burst-control",
  "algorithm": "TOKEN_BUCKET",
  "scopeType": "USER_ID",
  "scopeValue": "*",
  "limit": 10,
  "windowSize": 1,
  "windowUnit": "SECOND",
  "burstCapacity": 20
}

100 requests per minute per IP with Fixed Window Counter

{
  "name": "ip-minute-limit",
  "algorithm": "FIXED_WINDOW_COUNTER",
  "scopeType": "IP_ADDRESS",
  "scopeValue": "*",
  "limit": 100,
  "windowSize": 1,
  "windowUnit": "MINUTE"
}

1000 requests per hour per endpoint with Sliding Window Log

{
  "name": "endpoint-hour-limit",
  "algorithm": "SLIDING_WINDOW_LOG",
  "scopeType": "ENDPOINT",
  "scopeValue": "/api/orders",
  "limit": 1000,
  "windowSize": 1,
  "windowUnit": "HOUR"
}

Sample curl Commands

Create a rule:

curl -X POST http://localhost:8080/rules \
  -H "Content-Type: application/json" \
  -d '{
    "name": "user-burst-control",
    "algorithm": "TOKEN_BUCKET",
    "scopeType": "USER_ID",
    "scopeValue": "*",
    "limit": 10,
    "windowSize": 1,
    "windowUnit": "SECOND",
    "burstCapacity": 20
  }'

List rules:

curl http://localhost:8080/rules

Check a request:

curl -X POST http://localhost:8080/check \
  -H "Content-Type: application/json" \
  -d '{
    "userId": "user-123",
    "ipAddress": "203.0.113.10",
    "endpoint": "/api/orders"
  }'

Consume quota:

curl -X POST http://localhost:8080/consume \
  -H "Content-Type: application/json" \
  -d '{
    "userId": "user-123",
    "ipAddress": "203.0.113.10",
    "endpoint": "/api/orders"
  }'

Health check:

curl http://localhost:8080/health

Algorithms And Tradeoffs

Token Bucket

How it works:

• each identity gets a bucket with capacity and refill rate
• requests consume one token
• tokens refill continuously over time

Pros:

• allows short bursts while still controlling average rate
• good fit for public APIs and user-facing traffic
• smoother than fixed windows

Cons:

• slightly harder to reason about than a plain counter
• token math must be handled carefully to avoid drift

Fixed Window Counter

How it works:

• requests increment a counter in the current time bucket
• when the bucket changes, a new counter starts

Pros:

• simplest algorithm
• lowest operational complexity
• easy to explain and debug

Cons:

• burstiness at window boundaries
• less fair than rolling approaches

Sliding Window Log

How it works:

• each request timestamp is stored in a sorted set
• requests older than the window are removed
• the number of timestamps inside the active window determines allowance

Pros:

• most accurate rolling-window behavior
• avoids boundary spikes common in fixed windows

Cons:

• more Redis memory usage
• more expensive than counter-based approaches

Concurrency And Race Condition Handling

Rate limiting breaks down if multiple app instances read and write shared state non-atomically. This project avoids that by pushing the critical decision logic into Redis Lua scripts.

Each algorithm executes as one atomic Redis operation:

• Fixed Window Counter uses a script around current-window lookup and increment.
• Sliding Window Log uses one script to evict old entries, count active entries, optionally insert the new request, and compute retry timing.
• Token Bucket uses one script to compute refill state, decide admission, optionally consume a token, and update expiry.

This is important because two instances may receive traffic for the same user at the same time. Without atomic execution, both nodes could observe old state and incorrectly allow extra requests.

One practical note: /check is intentionally non-mutating, so it is advisory under high contention. /consume is the authoritative enforcement endpoint because it updates quota atomically.

Validation, Errors, And Logging

• request DTOs use Bean Validation
• invalid inputs return structured 400 responses
• Redis and persistence failures return 503
• unexpected failures return 500
• controllers and services emit structured logs for rule creation and decision flow

Running Locally

Option 1: Docker Compose

docker compose up --build

The app will be available at http://localhost:8080 and Redis at localhost:6379.

Option 2: Run app and Redis separately

Start Redis:

docker run --name local-redis -p 6379:6379 redis:7.2-alpine

Run the app:

mvn spring-boot:run

Running Tests

mvn test

The test suite contains:

• unit tests for service-level behavior
• integration tests using Spring Boot plus Redis, with automatic fallback to local localhost:6379 when available and Testcontainers otherwise
• concurrent integration tests that verify Redis-backed limits do not oversell quota under contention

Horizontal Scalability

This service is stateless at the application layer:

• rules are stored in Redis
• counters and token state are stored in Redis
• no node-local limiter state is required

That means multiple instances can scale behind a load balancer as long as they share the same Redis deployment.

Design Choices

• Strategy Pattern keeps algorithms isolated and easy to extend.
• Redis-backed rule storage makes configuration visible to all nodes.
• DTOs and a global exception handler keep the HTTP contract clean.
• Lua scripts provide atomicity without scattering multi-step Redis logic through Java.
• /check and /consume are separated so the project can demonstrate both advisory and authoritative admission workflows.

Future Improvements

• admin dashboard for rule creation, monitoring, and audits
• dynamic config loading from database or config service
• Prometheus metrics and Grafana dashboards
• Spring Security integration for authenticated user scopes
• API gateway plugin or sidecar integration
• Redis Cluster or Sentinel deployment support
• endpoint pattern matching and tenant-level rules
• persistent audit trail for rule changes

Tradeoff Summary

• If you want the simplest implementation, choose Fixed Window Counter.
• If you want burst tolerance with smooth refill behavior, choose Token Bucket.
• If you want the most accurate rolling-window control, choose Sliding Window Log.

This repository intentionally includes all three to show engineering tradeoffs rather than just one algorithm.