Configuring Request Timeouts

1. Setting Route-Level Timeouts

Property	Description	Default Value	Notes
route.timeout	Timeout for individual route requests	`30s`	Applies to all requests matching the route
route.idleTimeout	Idle timeout for route requests	`5s`	Time to wait before closing idle connections
route.perRequestTimeout	Per-request timeout for route-level operations	`10s`	Overrides global timeout settings for this route

Example: Route-Level Timeout Configuration

routes:
  - name: "user-service"
    timeout: "30s"
    idleTimeout: "5s"
    perRequestTimeout: "10s"

2. Configuring Per-Try Timeout

Property	Description	Default Value	Notes
perTryTimeout	Timeout for each individual retry attempt	`1s`	Applies to each retry in a retry policy. Can be overridden per route.
retryPolicy	Retry policy configuration that includes perTryTimeout	`null`	Must be defined to enable retries. Per-try timeout is part of retry policy.

Example: Setting Per-Try Timeout in a Route

routes:
  - name: api-route
    match:
      path: /api/users
    timeout:
      perTryTimeout: 2s
      retryPolicy:
        attempts: 3
        perTryTimeout: 1s

Note: The perTryTimeout is applied to each retry attempt, not the entire request. If a request times out after 3 attempts, and each attempt has a perTryTimeout of 1s, the total timeout can be up to 3 seconds.

3. Setting Idle Connection Timeout

Property	Description	Default Value	Notes
idleTimeout	Maximum idle time for a connection before it is closed	`30s`	Applies to both client and server connections
perRequestIdleTimeout	Idle timeout per individual request	`15s`	Overrides global idleTimeout for individual requests

Example: Configuring Idle Timeout in Istio

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
    timeout: 30s
    idleTimeout: 15s

Note: Setting idleTimeout too low can cause premature connection closures, especially for long-running requests or streaming responses.

4. Configuring Stream Idle Timeout

Property	Description	Default Value	Notes
streamIdleTimeout	Maximum time a stream can be idle before being closed	`30s`	Applies to HTTP/1.1 and HTTP/2 streams
streamIdleTimeout.duration	Duration string for idle timeout (e.g., `30s`, `5m`)	`30s`	Valid units: `s`, `m`, `h`

Example: Setting Stream Idle Timeout in Istio

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
    timeout: 30s
    idleTimeout: 30s

Note: Setting idleTimeout to 0s disables the timeout entirely.

5. Implementing Timeout Overrides

Property	Values	Description
timeout	`30s`	Global default timeout for all requests
per-request.timeout	`10s`	Timeout for individual request operations
retry.timeout	`5s`	Timeout for retry attempts

Example: Setting Per-Request Timeout

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
    timeout: 10s

Example: Using Retry with Timeout

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: resilient-service
spec:
  hosts:
  - resilient-service
  http:
  - route:
    - destination:
        host: resilient-service
    timeout: 5s
    retries:
      attempts: 3
      perTryTimeout: 2s

6. Setting Upstream Connection Timeout

Property	Values	Description
connectTimeout	`30s`	Timeout for establishing a connection to the upstream service
idleTimeout	`1m`	Timeout for idle connections to the upstream service
httpIdleTimeout	`1m`	HTTP-specific idle timeout for upstream connections

Example: Configuring Connection Timeout in Istio

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-destination-rule
spec:
  host: backend-service
  connectionPool:
    http:
      connectTimeout: "30s"
      idleTimeout: "1m"

Note: The connectTimeout setting determines how long the service mesh waits for a new connection to be established before timing out.

7. Configuring TCP Connect Timeout

Property	Value	Description
tcp.connectTimeout	`30s`	Timeout for establishing TCP connections to upstream services
tcp.connectTimeout.unit	`s`	Time unit for connect timeout (e.g., `s`, `ms`)

Example: Setting TCP Connect Timeout

tcp:
  connectTimeout: 30s

Note: The tcp.connectTimeout setting applies to all TCP-based upstream connections in the service mesh.

8. Managing Timeout Inheritance

Property	Description	Default Value	Inheritance Behavior
timeout	Timeout for the request in seconds	`30s`	Can be overridden at service, route, or operation level
idleTimeout	Idle timeout for the connection in seconds	`60s`	Inherited from parent if not explicitly set
requestTimeout	Request timeout including retries	`60s`	Inherited from parent or overridden at route level

Example: Timeout Inheritance in Service Mesh

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
        port:
          number: 8080
    timeout: 15s

Note: Timeout values can be overridden at different levels in the service mesh configuration hierarchy, allowing granular control over request handling.

9. Implementing Timeout Budgets

Property	Values	Description
timeoutBudget	`5%`	Budget percentage of total request time for retries and timeouts
retryBudget	`10%`	Budget percentage of total request time for retries
maxTimeout	`30s`	Maximum timeout value before considering failure

Example: Timeout Budget Configuration

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
    timeout: 5s
    retry:
      attempts: 3
      perTryTimeout: 2s
      retryBudget:
        percent: 5
        minRetriesPerSecond: 10

Note: Timeout budgets help prevent cascading failures by limiting the impact of retries and timeouts on overall system performance.

10. Troubleshooting Timeout Issues

Example: Configuring Timeout with Retry Logic

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-service
spec:
  hosts:
  - example-service
  http:
  - route:
    - destination:
        host: example-service
    timeout: 5s
    retries:
      attempts: 3
      perTryTimeout: 2s

Field	Description	Default	Notes
timeout	HTTP request timeout duration	60s	Applies to entire request lifecycle
retries.attempts	Number of retry attempts	0	Set to 3 or more for resilient services
retries.perTryTimeout	Timeout per retry attempt	Disabled	Must be less than overall timeout

Note: Timeouts can mask underlying issues; ensure logs capture timeout events for debugging.

Workflow: Diagnosing Timeout Root Causes

Check service logs for timeout or connection refused errors
Verify backend service health and resource usage (CPU, memory)
Review VirtualService timeout configurations
Analyze traffic patterns using telemetry (e.g., Prometheus metrics)
Confirm network policies or firewall rules aren't dropping requests

Comparison: Timeout vs Retry Behavior

Timeout: Stops waiting for a response after a set time
Retry: Repeats failed requests up to a configured limit

11. Configuring gRPC Deadline Propagation

Property	Description	Default Value	Notes
grpc.deadline.propagation	Enables or disables gRPC deadline propagation	`false`	When enabled, gRPC deadlines are propagated across service mesh boundaries
grpc.deadline.timeout	Timeout value for gRPC deadline propagation in seconds	`30`	Configures how long a gRPC call can take before it times out

Example: gRPC Deadline Propagation Configuration

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: grpc-service
spec:
  host: grpc-service
  trafficPolicy:
    connectionPool:
      http:
        idleTimeout: "30s"
    outlierDetection:
      consecutive5xxErrors: 5
    grpc:
      deadline:
        propagation: true
        timeout: "30s"

12. Managing Timeout Best Practices

Best Practice	Description	Implementation
Set Appropriate Timeout Values	Timeouts should align with the expected response time of services.	`timeout: 5s`
Use Hierarchical Timeout Configuration	Define timeouts at service, route, and operation levels for granular control.	`routes: - name: user-service timeout: 10s`
Implement Retry Logic with Exponential Backoff	Retry failed requests with increasing delays to prevent overwhelming services.	`retries: attempts: 3 backoff: base: 1s max: 10s`
Monitor and Alert on Timeout Events	Track timeout occurrences to identify performance bottlenecks or misconfigurations.	`{ "metric": "timeout_count", "threshold": 100 }`

Note: Timeout configurations should be tuned based on service behavior and latency patterns in production.