Implementing CI/CD for Kubernetes
1. Configuring kubectl Access
| Configuration | Description | Value/Example |
|---|---|---|
| kubectl version | Check kubectl version | v1.28.0 |
| kubectl config view | View current kubeconfig | |
| kubectl get nodes | Verify cluster access | NAME STATUS ROLES AGE VERSION |
Example: Setting up kubectl in GitLab CI
variables:
KUBE_CONFIG: "${CI_PROJECT_DIR}/kubeconfig"
KUBE_NAMESPACE: "default"
before_script:
- mkdir -p ~/.kube
- echo "${KUBE_CONFIG_DATA}" | base64 -d > ~/.kube/config
- export KUBECONFIG=~/.kube/config
- kubectl config set-context --current --namespace=$KUBE_NAMESPACE
test:
script:
- kubectl get nodes
Note: Ensure
KUBE_CONFIG_DATA is securely stored as a GitLab CI/CD variable with protected: true.
2. Using Kubernetes Executor
| Property | Value | Description |
|---|---|---|
| executor | kubernetes |
Specifies the Kubernetes executor for running jobs |
| kubernetes.image | alpine:latest |
Base image for the job pod |
| kubernetes.namespace | default |
Kubernetes namespace to run the job in |
| kubernetes.resources.requests.cpu | 100m |
CPU request for the job pod |
| kubernetes.resources.requests.memory | 128Mi |
Memory request for the job pod |
| kubernetes.resources.limits.cpu | 200m |
CPU limit for the job pod |
| kubernetes.resources.limits.memory | 256Mi |
Memory limit for the job pod |
Example: Basic Kubernetes Executor Configuration
stages:
- build
- test
build_job:
stage: build
image: alpine:latest
executor: kubernetes
kubernetes:
namespace: production
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 200m
memory: 256Mi
script:
- echo "Building the application..."
Note: The Kubernetes executor requires GitLab Runner to be installed and configured with the
kubernetes executor.
Warning: Ensure that the Kubernetes namespace specified has appropriate RBAC permissions for the GitLab Runner to create and manage pods.
3. Deploying with kubectl
| Option | Values | Notes |
|---|---|---|
| kubectl apply | kubectl apply -f <file> |
Apply configuration from a file or stdin |
| kubectl create | kubectl create -f <file> |
Create resources from a file or stdin |
| kubectl set image | kubectl set image deployment/<deployment> <container>=<image> |
Update image of a container in a deployment |
| kubectl rollout | kubectl rollout status deployment/<deployment> |
Check the status of a rollout |
Note: Ensure that the
kubectl context is set to the target Kubernetes cluster before running these commands.
Warning: Always verify the deployment configuration files before applying them to a production environment.
4. Using Helm Charts
| Property | Description | Example |
|---|---|---|
| chart.path | Path to the Helm chart directory | charts/myapp |
| values.files | List of YAML files to override chart values | values/prod.yaml |
| namespace | Kubernetes namespace to deploy the chart | production |
Example: Deploying with Helm in GitLab CI
deploy:
stage: deploy
script:
- helm repo add stable https://charts.helm.sh/stable
- helm upgrade --install myapp ./charts/myapp \
--namespace production \
--values values/prod.yaml
environment:
name: production
Note: Ensure
helm is installed in the GitLab runner image.
Warning: Avoid committing sensitive values to
values.yaml. Use GitLab CI variables or sealed secrets instead.
5. Configuring Agent for Kubernetes
| Property | Description | Value |
|---|---|---|
| agent.kubernetes.namespace | Namespace in Kubernetes where the agent will run | default |
| agent.kubernetes.image | Docker image used for the agent pod | gitlab/gitlab-agent:latest |
| agent.kubernetes.serviceAccountName | Kubernetes ServiceAccount to use for agent permissions | gitlab-agent-sa |
| agent.kubernetes.resources.requests.cpu | Requested CPU for the agent pod | 100m |
| agent.kubernetes.resources.requests.memory | Requested memory for the agent pod | 128Mi |
| agent.kubernetes.resources.limits.cpu | Maximum CPU allowed for the agent pod | 200m |
| agent.kubernetes.resources.limits.memory | Maximum memory allowed for the agent pod | 256Mi |
Example: Kubernetes Agent Configuration
agent:
kubernetes:
namespace: "gitlab-agent"
image: "gitlab/gitlab-agent:latest"
serviceAccountName: "gitlab-agent-sa"
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
Note: Ensure the Kubernetes ServiceAccount has appropriate RBAC permissions to interact with the cluster resources.
Warning: The agent image must be accessible from the Kubernetes cluster where it will run.
6. Using Review Apps on Kubernetes
Example: Enabling Review Apps for Kubernetes
stages:
- build
- test
- deploy
build_job:
stage: build
script:
- echo "Building application..."
artifacts:
paths:
- dist/
test_job:
stage: test
script:
- echo "Running tests..."
environment:
name: review/$CI_COMMIT_REF_NAME
url: https://review-$CI_COMMIT_REF_NAME.$EXTERNAL_URL
deploy_job:
stage: deploy
script:
- echo "Deploying to Kubernetes..."
- kubectl apply -f k8s/deployment.yaml
environment:
name: review/$CI_COMMIT_REF_NAME
url: https://review-$CI_COMMIT_REF_NAME.$EXTERNAL_URL
on_stop: stop_review
only:
- branches
| Property | Description | Default | Notes |
|---|---|---|---|
| environment.name | Name of the environment for review apps | review/$CI_COMMIT_REF_NAME | Must be unique per branch |
| environment.url | URL to access the review app | https://review-$CI_COMMIT_REF_NAME.$EXTERNAL_URL | Must be accessible from the internet |
| environment.on_stop | Job to run when environment is stopped | stop_review | Optional, used to clean up resources |
Note: Review Apps are automatically created for each branch and destroyed when the branch is merged or deleted.
Warning: Ensure proper resource limits to avoid excessive cluster usage.
Review Apps Benefits
- Automated deployment for every branch
- Isolated environments per feature branch
- Easy collaboration and testing
- Automatic cleanup after merge
Review Apps vs Staging Environments
- Review Apps: Per-branch, ephemeral, auto-deployed
- Staging: Centralized, long-lived, manual deployment
7. Implementing Auto Deploy
| Option | Values | Description |
|---|---|---|
| auto_devops_enabled | true |
Enables Auto DevOps for the project |
| auto_devops_artifacts_expiration | 1d |
Expiration time for artifacts (default: 1 day) |
Example: Enabling Auto DevOps
include:
template: Auto-DevOps.gitlab-ci.yml
auto_devops_enabled: true
auto_devops_artifacts_expiration: 30d
Note: Auto DevOps requires GitLab Runner with Kubernetes executor to be configured.
Example: Deploying to Kubernetes
deploy:
stage: deploy
script:
- echo "Deploying to Kubernetes cluster"
- kubectl set image deployment/my-app my-app=registry.gitlab.com/project/image:latest
environment:
name: production
url: https://myapp.example.com
Warning: Ensure proper RBAC permissions for GitLab Runner to interact with Kubernetes cluster.
| Environment Variable | Value | Description |
|---|---|---|
| KUBECONFIG | /home/gitlab-runner/.kube/config |
Kubeconfig file path for Kubernetes access |
| KUBE_NAMESPACE | default |
Kubernetes namespace to deploy to |
8. Using kustomize for Deployments
| Property | Description | Example |
|---|---|---|
| kustomize | Directory containing kustomization.yaml | kustomize/ |
| kustomize.build | Build command with kustomize | kustomize build kustomize/ |
Example: GitLab CI/CD with Kustomize
deploy:
stage: deploy
script:
- echo "Deploying with Kustomize"
- kustomize build kustomize/ | kubectl apply -f -
environment:
name: production
Note: Ensure kustomize is installed in the GitLab CI runner image.
Warning: Kustomize requires a valid
kustomization.yaml file to function.
9. Configuring Namespace Management
| Property | Description | Example |
|---|---|---|
| namespace.name | Name of the Kubernetes namespace to use | production |
| namespace.annotations | Annotations to apply to the namespace | {"app.kubernetes.io/managed-by": "gitlab"} |
| namespace.labels | Labels to apply to the namespace | {"environment": "prod"} |
Example: Creating a namespace in GitLab CI
deploy:
stage: deploy
script:
- |
kubectl create namespace $NAMESPACE_NAME --dry-run=client -o yaml | kubectl apply -f -
- |
kubectl label namespace $NAMESPACE_NAME environment=production
variables:
NAMESPACE_NAME: production
Note: Ensure the GitLab runner has sufficient RBAC permissions to create and label namespaces.
Warning: Avoid using default or generic namespace names like
default in production environments.
10. Using GitOps with ArgoCD
Example: ArgoCD Application Definition
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
namespace: argocd
spec:
project: default
source:
repoURL: https://gitlab.com/my-group/my-repo.git
targetRevision: HEAD
path: k8s/deployment.yaml
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
| Property | Description | Values |
|---|---|---|
| apiVersion | ArgoCD API version | argoproj.io/v1alpha1 |
| kind | Resource kind | Application |
| metadata.name | Application name in ArgoCD | my-app |
| spec.project | Project to which the app belongs | default |
| spec.source.repoURL | Git repository URL | https://gitlab.com/my-group/my-repo.git |
| spec.source.targetRevision | Git revision to sync from | HEAD |
| spec.source.path | Path to Kubernetes manifests in repo | k8s/deiployment.yaml |
| spec.destination.server | Kubernetes API server URL | https://kubernetes.default.svc |
| spec.destination.namespace | Kubernetes namespace to deploy to | production |
| spec.syncPolicy.automated.prune | Whether to delete resources not in Git | true |
| spec.syncPolicy.automated.selfHeal | Whether to auto-heal drift | true |
Note: ArgoCD requires the
argocd namespace to exist in the cluster.Warning: Enabling
prune can lead to accidental deletion of resources if not configured carefully.