Implementing Rate Limiting

1. Configuring Local Rate Limiting

Property	Description	Default	Notes
local_rate_limit	Enables local rate limiting for a service	false	Applies to all requests to the service
requests_per_second	Maximum number of requests per second	100	Can be overridden per endpoint
burst	Number of requests allowed to exceed the rate limit	10	Used for bursty traffic handling

Example: Local Rate Limit Configuration

rate_limit:
  local_rate_limit: true
  requests_per_second: 50
  burst: 5

Note: Local rate limiting is applied per instance of a service and does not consider traffic across the entire mesh.

2. Implementing Global Rate Limiting

Property	Description	Default Value	Notes
rateLimit.global.requestsPerSecond	Maximum number of requests per second allowed globally	`1000`	Applies to all services in the mesh
rateLimit.global.burst	Number of requests that can be bursted before rate limiting kicks in	`100`	Used to handle temporary spikes in traffic
rateLimit.global.windowSize	Time window over which the rate limit is enforced (in seconds)	`60`	Defines the sliding window for rate limiting

Example: Configuring Global Rate Limiting in Istio

apiVersion: networking.istio.io/v1beta1
kind: RateLimit
metadata:
  name: global-rate-limit
spec:
  rateLimit:
    local:
      requestsPerSecond: 1000
      burst: 100
      windowSize: 60

Note: The global rate limit configuration applies to all services in the mesh and can be overridden per service or route.

3. Setting Rate Limit Quotas

Property	Values	Description
rateLimit.quota.name	`string`	Name of the quota policy
rateLimit.quota.limit	`integer`	Maximum number of requests allowed
rateLimit.quota.duration	`string`	Time window for the quota (e.g., "1m", "1h")
rateLimit.quota.scope	`string`	Scope of the quota (e.g., "global", "per-user")

Example: Configuring a Rate Limit Quota

rateLimit:
  quota:
    name: "user-request-limit"
    limit: 100
    duration: "1m"
    scope: "per-user"

4. Implementing Token Bucket Algorithm

Property	Description	Default Value	Notes
tokens	Current number of tokens in the bucket	0	Updated on each request
rate	Number of tokens added per second	10	Configurable rate limit
capacity	Maximum tokens the bucket can hold	100	Defines burst capacity

Example: Token Bucket Algorithm Implementation

class TokenBucket {
  constructor(rate, capacity) {
    this.rate = rate;
    this.capacity = capacity;
    this.tokens = capacity;
    this.lastRefill = Date.now();
  }

  consume(tokens) {
    this.refill();
    if (this.tokens >= tokens) {
      this.tokens -= tokens;
      return true;
    }
    return false;
  }

  refill() {
    const now = Date.now();
    const timePassed = (now - this.lastRefill) / 1000;
    const tokensToAdd = timePassed * this.rate;
    this.tokens = Math.min(this.capacity, this.tokens + tokensToAdd);
    this.lastRefill = now;
  }
}

const bucket = new TokenBucket(10, 100);
console.log(bucket.consume(5)); // true
console.log(bucket.consume(15)); // true

Note: The token bucket algorithm allows for bursty traffic while maintaining an average rate limit.

5. Configuring Rate Limit Actions

Action	Description	Configuration Example	Notes
deny	Rejects the request with a 429 (Too Many Requests) status code.	`{ "action": "deny", "response": { "status": 429, "message": "Rate limit exceeded" } }`	Default behavior if no action is configured.
redirect	Redirects the request to a specified URL.	`{ "action": "redirect", "url": "https://example.com/rate-limited" }`	Useful for guiding users to a landing page or alternative endpoint.
modify	Modifies the request headers or body before forwarding.	`{ "action": "modify", "headers": { "X-Rate-Limit-Remaining": "5" } }`	Can be used to pass rate limit metadata to downstream services.
log	Logs the request but allows it to proceed.	`{ "action": "log", "level": "info" }`	Useful for monitoring without blocking traffic.

Example: Configuring a Deny Action

rateLimit:
  actions:
    - name: deny
      type: deny
      response:
        status: 429
        message: "Rate limit exceeded"

Example: Redirecting Requests

rateLimit:
  actions:
    - name: redirect
      type: redirect
      url: https://example.com/rate-limited

6. Managing Rate Limit Descriptors

Descriptor Type	Description	Usage Example	Notes
source.cluster	Identifies the source cluster for rate limiting	`"source.cluster": "frontend"`	Used to differentiate traffic sources
destination.cluster	Identifies the destination cluster for rate limiting	`"destination.cluster": "backend"`	Used to apply limits per service
request.header	Applies rate limits based on HTTP headers	`"request.header": "x-api-key"`	Can be used for API key-based rate limiting
request.path	Applies rate limits based on request path	`"request.path": "/api/v1/users"`	Useful for endpoint-specific limits
destination.port	Applies rate limits based on destination port	`"destination.port": 8080`	Useful in multi-port service environments

Example: Rate Limit Descriptor Configuration

rateLimit:
  descriptors:
    - key: source.cluster
      value: frontend
      limit: 100
    - key: destination.cluster
      value: backend
      limit: 500
    - key: request.header
      value: x-api-key
      limit: 1000

Note: Rate limit descriptors are evaluated in order, and the first matching descriptor applies.

7. Setting Rate Limit by Header

Header Name	Value Format	Description
X-RateLimit-Limit	`integer`	Maximum number of requests allowed in the current time window
X-RateLimit-Remaining	`integer`	Number of requests remaining in the current time window
X-RateLimit-Reset	`timestamp`	Time when the current rate limit window resets (Unix timestamp)

Example: Configuring Rate Limiting via Headers in Envoy

rate_limit:
  actions:
    - request_headers:
        header_name: "X-User-ID"
        key: "user_id"
    - request_headers:
        header_name: "X-App-Version"
        key: "app_version"

Note: The X-RateLimit- headers are typically added by the service mesh or proxy to inform clients about their current rate limit status.

8. Implementing Rate Limit by IP

Property	Description	Default	Notes
rateLimit.ip	Rate limit configuration per IP address	None	Enables rate limiting based on client IP
rateLimit.ip.requests	Number of requests allowed per time window	100	Adjust based on expected traffic
rateLimit.ip.window	Time window in seconds for rate limiting	60	Defines the sliding window size

Example: Configuring IP-based rate limiting in Istio

apiVersion: networking.istio.io/v1beta1
kind: RequestRateLimit
metadata:
  name: ip-rate-limit
spec:
  rules:
  - match:
    - source:
        ip: "192.168.1.1"
    rateLimit:
      local:
        requests: 50
        window: 300

Note: The rate limit is enforced per IP address and can be configured to apply to specific services or namespaces.

9. Configuring Rate Limit Overrides

Property	Values	Description
rateLimit.override.enabled	`true` / `false`	Enables or disables rate limit override configuration.
rateLimit.override.key	`string`	Key used to identify the override rule (e.g., `user-id`, `ip-address`).
rateLimit.override.value	`number`	The new rate limit value (requests per second or minute) to apply for the override.
rateLimit.override.ttl	`duration`	Time-to-live for the override rule (e.g., `1h`, `30m`).

Example: Configuring a Rate Limit Override

rateLimit:
  override:
    enabled: true
    key: "user-id"
    value: 100
    ttl: "1h"

Note: Overrides are applied dynamically and take precedence over default rate limits for matching keys.

10. Managing Rate Limit Backends

Backend	Description	Configuration	Notes
Redis	Uses Redis for rate limit storage and tracking	`{ "backend": "redis", "host": "localhost", "port": 6379 }`	Supports distributed rate limiting across services
In-Memory	Stores rate limits in local memory (not persistent)	`{ "backend": "memory" }`	Simple for testing, not suitable for production
Database	Stores rate limits in a SQL or NoSQL database	`{ "backend": "database", "connectionString": "..." }`	Provides persistence and scalability

Example: Redis Backend Configuration

rateLimit:
  backend: redis
  redis:
    host: localhost
    port: 6379
    db: 0

Note: Redis backend is preferred for production due to its performance and support for distributed systems.

Warning: In-memory backend should not be used in multi-instance deployments.

11. Implementing Rate Limit Caching

Property	Description	Default Value	Notes
cache.ttl	Time-to-live for cached rate limit entries	`30s`	Controls how long a rate limit is cached before refresh
cache.size	Maximum number of entries in the rate limit cache	`1000`	Prevents memory exhaustion from excessive caching
cache.key	Template for generating cache keys	`rate_limit:{user_id}:{resource}`	Allows customization of cache key format

Example: Rate Limit Cache Configuration

rate_limit:
  cache:
    ttl: 30s
    size: 1000
    key: rate_limit:{user_id}:{resource}

Note: Cache invalidation is handled automatically based on TTL. Manual cache clearing may be required in edge cases.

12. Troubleshooting Rate Limiting Issues

Example: Identifying Rate Limiting in Logs

2026-04-15T10:23:45Z info: "Rate limit exceeded for client 192.168.1.100"
2026-04-15T10:23:46Z warn: "Request to /api/v1/users blocked due to rate limit"
2026-04-15T10:23:47Z debug: "Client 192.168.1.100 has made 100 requests in last minute"

Issue	Description	Solution
Excessive Requests	Client exceeds configured rate limit (e.g., 100 requests/minute)	Adjust limits or implement client-side caching
Misconfigured Limits	Rate limit policy does not match actual traffic patterns	Review and adjust rate limit rules based on usage analytics
Global vs Per-Client Limits	Global limits may block legitimate traffic from other clients	Use per-client or per-route rate limiting for granular control

Note: Use istioctl or service mesh dashboard to inspect rate limit configurations and logs for troubleshooting.

Warning: Misconfigured rate limits can cause cascading failures or denial of service for legitimate users.

Common Troubleshooting Steps

Check logs for rate limit violation messages
Verify rate limit policies in mesh configuration
Validate client IP or identifier matching
Review traffic patterns to ensure limits are appropriate