Skip to main content
Mapagam

Managing Traffic Splitting

1. Implementing Percentage-Based Splits

Field Description Example
route Defines the route for traffic splitting /api/v1/users
weight Percentage of traffic to be routed to a specific version 70
destination Target service version users-v2

Example: Percentage-Based Split Configuration

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: users-route
spec:
  hosts:
    - users
  http:
    - route:
        - destination:
            host: users-v1
            subset: v1
          weight: 30
        - destination:
            host: users-v2
            subset: v2
          weight: 70
Note: The weights must sum to 100 for all destinations in a route.

2. Configuring Weighted Routing

Property Value Description
route.weight 70 Percentage of traffic to route to this version
route.destination service-v1 Name of the service version to route traffic to
route.name primary-route Unique identifier for the route rule

Example: Weighted Route Configuration

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-service-route
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service-v1
        subset: v1
      weight: 80
    - destination:
        host: my-service-v2
        subset: v2
      weight: 20
Note: The weights must sum to 100 for all destinations in a single route rule.
Warning: Misconfiguring weights can lead to uneven traffic distribution or service unavailability.

3. Implementing Header-Based Splits

Header Name Example Value Description Usage
Accept text/html Specifies the content type that the client prefers to receive Used for content negotiation
User-Agent Mozilla/5.0 Identifies the client software making the request Used for browser-specific routing or logging
Authorization Bearer token123 Contains credentials for authentication Used to route requests based on user roles or permissions
X-Forwarded-For 192.168.1.1 Identifies the original IP address of a client connecting through a proxy or load balancer Used for logging and access control

Example: Route based on Accept header

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: api-virtual-service
spec:
  hosts:
  - api.example.com
  http:
  - match:
    - headers:
        accept:
          exact: text/html
    route:
    - destination:
        host: web-api.example.com
        port:
          number: 80
  - match:
    - headers:
        accept:
          exact: application/json
    route:
    - destination:
        host: json-api.example.com
        port:
          number: 80

Example: Route based on User-Agent header

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: mobile-api-virtual-service
spec:
  hosts:
  - api.example.com
  http:
  - match:
    - headers:
        user-agent:
          regex: .*Mobile.*
    route:
    - destination:
        host: mobile-api.example.com
        port:
          number: 80

4. Configuring A/B Testing Routes

Property Description Example
route.name Name of the route for identification "ab-test-route"
route.match Match criteria for traffic routing { "method": "GET", "path": "/api/users" }
route.backends Backend services with weights [{ "name": "service-a", "weight": 80 }, { "name": "service-b", "weight": 20 }]

Example: A/B Testing Route Configuration

routes:
  - name: ab-test-route
    match:
      method: GET
      path: /api/users
    backends:
      - name: service-a
        weight: 80
      - name: service-b
        weight: 20
Note: The weights determine the percentage of traffic sent to each backend.
Warning: Ensure that all services in the A/B test are compatible to avoid unexpected behavior.

5. Implementing Dark Launches

Dark launches allow deploying new features to a subset of users without making them visible in the UI or API. Service meshes like Istio provide traffic splitting capabilities to implement dark launches effectively.

Feature Description Implementation
Traffic Splitting Routes a percentage of traffic to new versions or features Use DestinationRule and VirtualService to define traffic weights
Header-Based Routing Routes traffic based on HTTP headers (e.g., user ID, feature flag) Configure VirtualService with match conditions on headers

Example: Route 10% of Traffic to New Feature

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: api-virtual-service
spec:
  hosts:
  - api.example.com
  http:
  - route:
    - destination:
        host: api-v1.example.com
        port:
          number: 80
      weight: 90
    - destination:
        host: api-v2.example.com
        port:
          number: 80
      weight: 10

Example: Route by User ID Header

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: api-virtual-service
spec:
  hosts:
  - api.example.com
  http:
  - match:
    - headers:
        user-id:
          regex: "user-123.*"
  route:
  - destination:
      host: api-v2.example.com
      port:
        number: 80
Note: Dark launches are useful for testing new features in production without affecting all users. Ensure proper monitoring and rollback strategies.

6. Managing Traffic Shifts

Shift Type Description Configuration Example Notes
Percentage-based shift Routes a percentage of traffic to a new version or service 
{
  "route": [
    {
      "destination": {
        "host": "service-v2.default.svc.cluster.local",
        "subset": "v2"
      },
      "weight": 20
    },
    {
      "destination": {
        "host": "service-v1.default.svc.cluster.local",
        "subset": "v1"
      },
      "weight": 80
    }
  ]
}
 Use for gradual rollouts or A/B testing
Header-based shift Routes traffic based on HTTP header values 
{
  "route": [
    {
      "destination": {
        "host": "service-v2.default.svc.cluster.local",
        "subset": "v2"
      },
      "weight": 100
    }
  ],
  "match": [
    {
      "headers": {
        "user-agent": {
          "prefix": "Mobile"
        }
      }
    }
  ]
}
 Use for targeting specific user groups
Cookie-based shift Routes traffic based on cookie values 
{
  "route": [
    {
      "destination": {
        "host": "service-v2.default.svc.cluster.local",
        "subset": "v2"
      },
      "weight": 50
    },
    {
      "destination": {
        "host": "service-v1.default.svc.cluster.local",
        "subset": "v1"
      },
      "weight": 50
    }
  ],
  "match": [
    {
      "headers": {
        "cookie": {
          "regex": "session=.*?123"
        }
      }
    }
  ]
}
 Use for session-based routing

Example: Percentage-based shift with Istio

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews-v1
        subset: v1
      weight: 80
    - destination:
        host: reviews-v2
        subset: v2
      weight: 20
Note: Traffic shifts can be managed dynamically using tools like istioctl or through configuration updates in your service mesh control plane.
Warning: Ensure that all versions of a service are compatible before shifting traffic to avoid breaking changes.

7. Configuring Multi-Version Routing

Property Description Values Notes
route.version Version identifier for routing rules v1, v2, canary Used to define traffic distribution across versions
route.weight Percentage of traffic to direct to a version 0-100 Sum of all weights must equal 100
route.match Matching criteria for version selection header, cookie, query Can be used for A/B testing or gradual rollouts

Example: Route Configuration

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews-route
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 90
    - destination:
        host: reviews
        subset: v2
      weight: 10
Note: The subset field refers to a defined destination rule in the service mesh.
Warning: Ensure that all versions of a service are compatible with the same client requests to avoid unexpected behavior.

8. Implementing Traffic Mirroring

Property Description Example
trafficMirroring Enables mirroring of traffic to a secondary destination { "trafficMirroring": { "destination": "http://backend-v2:8080" } }
source Specifies the source service for mirroring { "source": "frontend" }
destination Target endpoint to mirror traffic to { "destination": "http://analytics:9090" }

Example: Traffic Mirroring Configuration

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: frontend-mirror
spec:
  hosts:
  - frontend
  http:
  - route:
    - destination:
        host: frontend
    mirror:
      host: analytics
      port:
        number: 9090
Note: Traffic mirroring does not affect the original request flow and is used for observability or testing.

9. Managing Rollback Strategies

Example: Configuring rollback with Istio

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-vs
spec:
  hosts:
  - example.com
  http:
  - route:
    - destination:
        host: example-v1
        port:
          number: 80
      weight: 90
    - destination:
        host: example-v2
        port:
          number: 80
      weight: 10
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: connect-failure,5xx
Strategy Description Use Case Notes
Blue-Green Deployment Route traffic to a new version only after validation. Safe rollouts with minimal downtime Requires two separate deployments
Canary Release Gradually shift traffic to the new version. Controlled risk rollout Use weights to control traffic distribution
Rollback on Failure Automatically switch back to a stable version if errors exceed threshold. Resilient deployments Requires monitoring and alerting integration

Workflow: Implementing Rollback in Service Mesh

  1. Deploy new version with VirtualService
  2. Route small percentage (e.g., 5%) of traffic to new version
  3. Monitor metrics for errors, latency, and availability
  4. If thresholds exceeded, switch traffic back to stable version
  5. Gradually increase traffic if stable

Summary: Rollback Considerations

  • Use circuit breakers to prevent cascading failures
  • Implement health checks before routing traffic
  • Set up automated rollback triggers based on error rates
  • Ensure rollback strategy aligns with business SLAs

10. Monitoring Traffic Split Metrics

Metric Description Unit Notes
traffic_split_ratio Ratio of traffic directed to each service version Percentage Used to monitor distribution of traffic across versions
request_count Total number of requests handled by the service mesh Count Tracks overall traffic volume
error_rate Percentage of failed requests in the traffic split Percentage Indicates reliability of service versions
latency_p50 50th percentile latency for requests Milliseconds Measures average response time under load
latency_p95 95th percentile latency for requests Milliseconds Measures high-latency outliers in traffic
throughput Number of requests processed per second Requests/Second Indicates service mesh performance capacity

Example: Grafana Dashboard Setup

dashboard:
  title: "Traffic Split Metrics"
  panels:
    - title: "Traffic Split Ratio"
      type: "graph"
      targets:
        - expr: "traffic_split_ratio"
          legendFormat: "{{version}}"
    - title: "Error Rate"
      type: "graph"
      targets:
        - expr: "error_rate"
Note: Use Prometheus or similar metrics collection systems to gather these values for real-time monitoring.

11. Implementing User Segment Routing

Property Value Description
route.user.segment userType Header or query parameter used to identify user segment
route.user.value premium Value of the segment identifier to match
route.user.target service-v2 Target service version for matching users

Example: User Segment Route Configuration

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: user-segment-routing
spec:
  hosts:
  - frontend
  http:
  - match:
    - headers:
        user-type:
          exact: premium
    route:
    - destination:
        host: frontend-v2
        port:
          number: 80
  - route:
    - destination:
        host: frontend-v1
        port:
          number: 80
Note: The user-type header is used to route users to different versions of the frontend service based on their segment.

12. Managing Split Configuration Updates

Field Description Example
split.name Name of the traffic split configuration user-service-split
split.matches Match conditions for traffic routing 
[
  {
    "method": "GET",
    "path": "/api/users"
  }
]
split.targets Target services and weights 
[
  {
    "service": "user-service-v1",
    "weight": 80
  },
  {
    "service": "user-service-v2",
    "weight": 20
  }
]

Example: Update Traffic Split Configuration

apiVersion: split.example.io/v1
kind: TrafficSplit
metadata:
  name: user-service-split
spec:
  matches:
  - method: GET
    path: /api/users
  targets:
  - service: user-service-v1
    weight: 70
  - service: user-service-v2
    weight: 30
Note: Updates to traffic split configurations are typically applied via service mesh control plane APIs or configuration management tools like kubectl or istioctl.
Warning: Ensure that all target services are available before updating split weights to avoid traffic disruption.

Was this article helpful?

Related Cheat Sheets