Skip to main content
Mapagam

Implementing Multi-Tenancy

1. Understanding Multi-Tenancy Models

Model Description Use Case Benefits
Isolated Tenants Each tenant has dedicated resources and isolated network paths. High-security environments, strict compliance requirements. Strong security isolation, predictable performance.
Shared Infrastructure Tenants share infrastructure but are logically separated. Cost-effective deployments, moderate security needs. Resource efficiency, reduced operational overhead.
Hybrid Approach Mix of isolated and shared models based on tenant needs. Flexible scaling, varying security/compliance levels. Balance between cost and security; adaptable to changing needs.

Example: Tenant Isolation in Service Mesh

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: tenant-a-service
spec:
  hosts:
  - "tenant-a.example.com"
  http:
  - route:
    - destination:
        host: "tenant-a-app"
        port:
          number: 80
Note: Multi-tenancy in service meshes often involves namespace-level or cluster-level isolation to ensure that traffic and policies for one tenant do not affect another.

2. Configuring Namespace Isolation

Configuration Description Example
istio.io/namespace Label to identify namespace as part of service mesh istio.io/namespace: "true"
istio.io/mesh Mesh name to isolate traffic istio.io/mesh: "production"

Example: Applying Namespace Labels

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-a
  labels:
    istio.io/namespace: "true"
    istio.io/mesh: "production"
Note: Ensure all namespaces intended for multi-tenancy are labeled consistently to enable proper isolation.
Warning: Misconfiguring namespace labels can lead to cross-tenant traffic leakage.

3. Implementing Tenant-Based Policies

Policy Type Description Configuration Example Notes
Traffic Policy Controls traffic routing and load balancing between tenants. 
{
  "trafficPolicy": {
    "loadBalancer": {
      "simple": "LEAST_CONN"
    }
  }
}
 Ensures fair distribution of traffic among tenant services.
Rate Limiting Applies rate limits per tenant to prevent resource exhaustion. 
{
  "rateLimit": {
    "requestsPerSecond": 100,
    "burst": 50
  }
}
 Can be configured per tenant or globally.
Circuit Breaking Prevents cascading failures by breaking circuits under high load. 
{
  "circuitBreaker": {
    "simple": {
      "maxConnections": 1000,
      "maxRequests": 50
    }
  }
}
 Helps maintain service stability across tenants.

Example: Tenant-Based Traffic Policy

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tenant-traffic-policy
spec:
  hosts:
  - app.example.com
  http:
  - match:
    - headers:
        tenant-id:
          exact: "tenant-a"
    route:
    - destination:
        host: app-tenant-a.example.com
        port:
          number: 80
  - match:
    - headers:
        tenant-id:
          exact: "tenant-b"
    route:
    - destination:
        host: app-tenant-b.example.com
        port:
          number: 80

Example: Rate Limiting Configuration

apiVersion: networking.istio.io/v1beta1
kind: RateLimit
metadata:
  name: tenant-rate-limit
spec:
  rateLimits:
    - action:
        requestsPerSecond: 100
      match:
        - headers:
            tenant-id:
              exact: "tenant-a"

4. Managing Resource Quotas per Tenant

Resource Type Quota Name Default Limit Description
CPU cpu.limit 1000m Maximum CPU allocation per tenant
Memory memory.limit 2Gi Maximum memory allocation per tenant
Network Bandwidth network.bandwidth 100Mbps Maximum network bandwidth per tenant
Pods pod.count 50 Maximum number of pods per tenant

Example: Setting Resource Quotas in Kubernetes

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-namespace
spec:
  hard:
    cpu: "1000m"
    memory: "2Gi"
    persistentvolumeclaims: "10"
    pods: "50"
    services.loadbalancers: "5"
Note: Resource quotas are enforced at the namespace level in Kubernetes and must be configured per tenant namespace.

5. Configuring Tenant-Specific Gateways

Gateway Type Description Configuration Example Notes
Tenant Gateway Routes traffic for a specific tenant within the service mesh 
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tenant-gateway
  namespace: tenant-namespace
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    hostname: tenant.example.com
 Each tenant has its own gateway instance for isolation
Shared Gateway Routes traffic for multiple tenants using a single gateway 
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared-gateway
  namespace: mesh-namespace
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    hostname: "*.example.com"
 Requires proper routing and namespace isolation

Example: Tenant Gateway with TLS

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tenant-gateway-tls
  namespace: tenant-namespace
spec:
  gatewayClassName: istio
  listeners:
  - name: https
    port: 443
    protocol: HTTPS
    hostname: tenant.example.com
    tls:
      mode: Terminate
      certificateRef:
        name: tenant-tls-cert
Note: Ensure that each tenant's gateway has appropriate AuthorizationPolicy to restrict access to its own services.
Warning: Using a shared gateway can lead to security risks if tenants are not properly isolated at the service level.

6. Implementing Tenant Authentication

Property Description Example
auth.type Authentication method used for tenant access oidc
auth.issuer OIDC issuer URL for tenant authentication https://keycloak.example.com/realms/myrealm
auth.clientId Client ID registered with the OIDC provider tenant-service
auth.clientSecret Client secret for authenticating with OIDC provider secret-key-here

Example: Configuring OIDC Authentication

auth:
  type: oidc
  issuer: https://keycloak.example.com/realms/myrealm
  clientId: tenant-service
  clientSecret: secret-key-here
Note: Ensure that the OIDC provider supports the required scopes and claims for multi-tenant access control.
Warning: Client secrets should be stored securely and not exposed in configuration files.
Authentication Method Description Use Case
oidc OpenID Connect for centralized identity management Secure, scalable authentication across tenants
jwt JSON Web Token validation for tenant-specific tokens Stateless authentication with custom claims
api-key API key-based authentication for service-to-service Simple, lightweight authentication for internal services

7. Managing Tenant Authorization

Resource Permission Description
tenant.read read Allows reading tenant configuration and metadata
tenant.write write Allows modifying tenant configuration and settings
tenant.delete delete Allows deletion of tenant resources and configurations
tenant.list list Allows listing all tenants accessible to the user

Example: Tenant Authorization Policy

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
rules:
- apiGroups: [""]
  resources: ["tenants"]
  verbs: ["get", "list", "create", "update", "delete"]
Note: Authorization policies should be scoped to specific namespaces or tenants to prevent cross-tenant access.
Warning: Ensure that tenant.delete permissions are restricted to administrators only.

8. Configuring Tenant-Specific Observability

Example: Prometheus Metrics per Tenant

scrape_configs:
- job_name: 'tenant-a-metrics'
  metrics_path: /metrics
  static_configs:
  - targets: ['app-a-tenant-a.mesh:8080']
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_label_tenant]
    target_label: tenant
  - source_labels: [__meta_kubernetes_pod_label_app]
    target_label: app
Configuration Description Default Value Notes
tenant.label Label used to identify tenant in metrics tenant Used for filtering and grouping
metrics.namespace Namespace prefix for tenant metrics tenant_ Ensures uniqueness across tenants
log.level Log level for tenant-specific logs info Can be overridden per tenant
Note: Ensure that metrics and logs are properly labeled to avoid cross-tenant data leakage.
Warning: Tenant-specific observability requires strict access controls to prevent unauthorized data access.

Key Considerations

  • Metrics must be tagged with tenant identifiers for proper isolation
  • Log aggregation should support multi-tenant filtering
  • Alerting rules must be tenant-aware to avoid false positives

9. Managing Tenant Onboarding

Step Description Action Notes
1 Define tenant requirements Create a tenant profile with resource limits, security policies, and access controls. Use a standardized schema to ensure consistency across tenants.
2 Assign unique identifiers Generate a unique tenant ID and namespace for isolation. Ensure the ID is globally unique and follows naming conventions.
3 Configure resource quotas Set CPU, memory, and network limits for the tenant's workloads. Use Kubernetes ResourceQuotas or service mesh-specific limits.
4 Establish network policies Define traffic rules and access control lists (ACLs) for the tenant. Implement mTLS and network segmentation to isolate traffic.
5 Provision monitoring and logging Set up dedicated dashboards, alerts, and log aggregation for the tenant. Use Prometheus and Grafana or similar tools with tenant-specific views.
6 Validate tenant configuration Run tests to ensure the tenant is isolated and secure. Use automated checks or manual validation tools to confirm setup.

Example: Tenant Onboarding Script

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-abc-namespace
  labels:
    tenant.id: "tenant-abc"
    istio.io/rev: "default"

---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-abc-quota
  namespace: tenant-abc-namespace
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: tenant-abc-policy
  namespace: tenant-abc-namespace
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/tenant-abc-namespace/sa/tenant-abc-sa"]
Note: Ensure that all tenant configurations are version-controlled and auditable to support compliance and troubleshooting.

10. Troubleshooting Multi-Tenancy Issues

Issue Potential Causes Solution
Cross-Tenant Traffic Interference serviceA and serviceB share same namespace or use same labels Ensure proper namespace isolation and unique label values per tenant
Rate Limiting Errors Global rate limits applied to tenant-specific services Implement per-tenant rate limiting policies using service mesh controls
Authentication Failures Shared or misconfigured JWT tokens across tenants Use tenant-specific token issuers and validate claims per tenant
Resource Quotas Exhausted Shared resource limits (CPU, memory) without tenant-aware allocation Configure resource quotas per tenant using namespace or service-level limits

Example: Isolating Tenant Traffic with Labels

apiVersion: v1
kind: Service
metadata:
  name: tenant-a-service
  labels:
    tenant: "tenant-a"
    app: "web-app"
spec:
  selector:
    tenant: "tenant-a"
Note: Use consistent labeling strategies across all services to ensure proper tenant isolation.
Warning: Misconfigured traffic policies can lead to cross-tenant data leakage or service disruption.

Common Troubleshooting Steps

  • Verify namespace and label isolation
  • Check for shared or overridden rate limits
  • Validate authentication and authorization policies
  • Review resource allocation per tenant

11. Implementing Tenant Resource Limits

Resource Type Configuration Parameter Default Value Description
cpu limits.cpu 1000m CPU limit in millicores per tenant
memory limits.memory 512Mi Memory limit in mebibytes per tenant
requests.cpu requests.cpu 250m Minimum CPU requested per tenant
requests.memory requests.memory 128Mi Minimum memory requested per tenant

Example: Setting Resource Limits for a Tenant

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-namespace
spec:
  hard:
    requests.cpu: "250m"
    requests.memory: 128Mi
    limits.cpu: "1000m"
    limits.memory: 512Mi
Note: Resource quotas are enforced at the namespace level in Kubernetes and must be configured per tenant namespace.

12. Managing Tenant Data Isolation

Aspect Description Implementation
Data Partitioning Separation of data between tenants to ensure no cross-contamination. Use distinct namespaces or databases per tenant in the service mesh.
Network Policies Control traffic between services and enforce tenant boundaries. Implement strict network policies using Istio or Linkerd to isolate traffic.
Resource Quotas Limit resource consumption per tenant to prevent one from affecting others. Configure quotas in Kubernetes or service mesh control plane for CPU, memory, and network.
Access Control Enforce authentication and authorization per tenant. Use mTLS and RBAC to restrict access to services and data.

Example: Isolating Tenant Data with Namespaces

apiVersion: v1
kind: Namespace
metadata:
  name: tenant-a
---
apiVersion: v1
kind: Namespace
metadata:
  name: tenant-b
Note: Ensure that services within each namespace are configured to only communicate with other services in the same namespace or explicitly allowed ones.

Was this article helpful?

Related Cheat Sheets