Configuring Outlier Detection

1. Enabling Outlier Detection

Property	Values	Description
outlierDetection.enabled	`true`	Enables outlier detection for the service mesh
outlierDetection.interval	`10s`	Interval between outlier detection evaluations
outlierDetection.baseEjectionTime	`30s`	Minimum ejection time for a host before it can be ejected again

Example: Enabling Outlier Detection in a Service

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: "my-service"
spec:
  host: "my-service"
  outlierDetection:
    enabled: true
    interval: 10s
    baseEjectionTime: 30s

2. Configuring Consecutive 5xx Errors

Property	Description	Default Value	Notes
consecutive5xx	Number of consecutive 5xx errors to trigger outlier detection	5	Used to detect unhealthy hosts in the service mesh
interval	Time window to consider for consecutive 5xx errors (in seconds)	10	Defines how long the error count is tracked
timeout	Duration to wait before marking a host as unhealthy (in seconds)	30	Hosts are ejected from the load balancing pool after timeout

Example: Configuring Consecutive 5xx Errors

outlierDetection:
  consecutive5xx: 5
  interval: 10
  timeout: 30

3. Setting Consecutive Gateway Errors

Property	Values	Description
consecutiveGatewayErrors	`5`	Number of consecutive gateway errors before marking a host as unhealthy.
consecutive5xxErrors	`0`	Number of 5xx errors before marking a host as unhealthy (default is 0, meaning disabled).

Example: Configuring Consecutive Gateway Errors

outlierDetection:
  consecutiveGatewayErrors: 5
  consecutive5xxErrors: 0

Note: Setting consecutiveGatewayErrors to a value greater than 0 enables the outlier detection mechanism for gateway failures.

4. Configuring Consecutive Local Origin Failures

Property	Description	Default Value	Notes
consecutiveLocalOriginFailures	Number of consecutive local origin failures before marking the host as unhealthy.	`5`	Applies to local origin failures (e.g., timeouts, connection errors) in the context of outlier detection.

Example: Configuring Consecutive Local Origin Failures

outlierDetection:
  consecutiveLocalOriginFailures: 3

Note: This setting is part of the outlierDetection configuration and affects how the service mesh handles unhealthy hosts in local origin failure scenarios.

5. Setting Detection Interval

Property	Values	Description
interval	`10s`	Time interval between outlier detection checks
timeout	`5s`	Timeout for each outlier detection check
baseEpsilon	`0.5`	Base epsilon value for outlier detection algorithm

Example: Configuring Detection Interval

outlierDetection:
  interval: 10s
  timeout: 5s
  baseEpsilon: 0.5

6. Configuring Base Ejection Time

Property	Description	Default Value	Notes
baseEjectionTime	Duration for which a host is ejected before it's considered for re-injection	`30s`	Used in conjunction with `consecutiveErrors` to determine ejection duration
baseEjectionTimeMs	Base ejection time in milliseconds	`30000`	Alternative way to configure the same value in milliseconds

Example: Configuring Base Ejection Time in Envoy

outlier_detection:
  base_ejection_time: 60s
  consecutive_errors: 3

Note: The base ejection time is used in exponential backoff calculations when determining how long to keep a host ejected from the load balancing pool.

7. Setting Max Ejection Percent

Property	Values	Description
maxEjectionPercent	`10`	Maximum percentage of hosts in the cluster that can be ejected due to outlier detection.

Example: Configuring maxEjectionPercent in a Service Mesh

outlierDetection:
  maxEjectionPercent: 10

8. Configuring Split External Local Origin Errors

Property	Description	Default Value	Notes
split_external_local_origin_errors	Enables splitting of external and local origin errors for outlier detection.	`false`	When enabled, allows more granular control over how errors from external services and local origin are handled separately in outlier detection logic.

Example: Enabling Split External Local Origin Errors

outlier_detection:
  split_external_local_origin_errors: true

Note: This setting is particularly useful in complex service mesh environments where distinguishing between local and external errors can improve resilience and error handling strategies.

9. Implementing Outlier Detection Metrics

Metric	Description	Configuration	Notes
outlierDetection.metrics.requests	Total number of requests made to the service	`100`	Threshold for outlier detection
outlierDetection.metrics.errors	Number of failed requests (5xx errors)	`10`	Used to identify unhealthy hosts
outlierDetection.metrics.latency	Average response time in milliseconds	`200`	Threshold for latency-based outlier detection
outlierDetection.metrics.consecutive5xx	Consecutive 5xx errors per host	`5`	Triggers immediate host removal

Example: Configuring Outlier Detection Metrics

outlierDetection:
  metrics:
    requests: 100
    errors: 10
    latency: 200
    consecutive5xx: 5

Note: These metrics are used to determine if a host is behaving as an outlier and should be ejected from the load balancing pool.

10. Troubleshooting Outlier Detection

Example: Outlier Detection Misconfiguration

# Incorrect configuration - no outlier detection rules
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: productpage-policy
spec:
  host: productpage
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 100

Issue	Description	Solution
No outlier detection rules	Service mesh does not detect unhealthy hosts	Add `outlierDetection` to `trafficPolicy`
Incorrect thresholds	Too aggressive or lenient detection settings	Adjust `consecutiveErrors`, `interval`, and `baseEjectionTime`
Misconfigured host	Outlier detection applied to wrong service	Verify `host` field matches target service

Note: Use istioctl proxy-config to inspect outlier detection state on a per-pod basis.

Workflow: Diagnose Outlier Detection Issues

Check DestinationRule for outlierDetection configuration
Verify that the service name in host matches the target
Use istioctl proxy-config cluster <pod-name> to inspect cluster settings
Review logs for outlier detection or host ejection messages
Adjust thresholds if needed based on traffic patterns and error rates

Common Misconfigurations

Missing outlierDetection in DestinationRule
Incorrect consecutiveErrors value (e.g., set to 0 or too high)
Not accounting for interval and baseEjectionTime in error rate calculations

11. Configuring Success Rate Outlier Detection

Property	Description	Default Value	Notes
successRateRequestVolume	Minimum number of requests in a window to consider for outlier detection	100	Used to determine if enough data is available for analysis
successRateMinimumHosts	Minimum number of hosts in a cluster to consider for outlier detection	5	Ensures statistical significance in outlier detection
successRateStdevFactor	Standard deviation factor to calculate outlier thresholds	120	Higher values make detection more sensitive to outliers

Example: Success Rate Outlier Detection Configuration

outlierDetection:
  successRateRequestVolume: 100
  successRateMinimumHosts: 5
  successRateStdevFactor: 120

Note: The success rate outlier detection algorithm uses statistical analysis to identify hosts with significantly different success rates compared to the rest of the cluster.

12. Managing Outlier Detection Thresholds

Property	Values	Description
consecutive5xx	`5`	Number of 5xx errors before marking a host as unhealthy
interval	`10s`	Time window to consider for outlier detection
baseEjectionTime	`30s`	Initial ejection time for a host (doubles on each ejection)
maxEjectionPercent	`10%`	Maximum percentage of hosts that can be ejected

Example: Configuring outlier detection thresholds in a service mesh

outlierDetection:
  consecutive5xx: 5
  interval: 10s
  baseEjectionTime: 30s
  maxEjectionPercent: 10%