Understanding GitLab CI/CD Fundamentals

1. Understanding CI/CD Concepts

Concept	Description	Key Benefits
Continuous Integration (CI)	Automatically builds and tests code changes as they are committed to a shared repository	Early bug detection, improved code quality, faster feedback loops
Continuous Delivery (CD)	Automatically deploys code changes to a staging or production environment after successful testing	Reduced deployment time, consistent release process, faster time-to-market
Continuous Deployment	Automatically deploys every code change that passes testing to production without manual intervention	High deployment frequency, minimal human error, rapid iteration

Example: CI/CD Pipeline Workflow

# Typical GitLab CI/CD pipeline stages
stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - echo "Building project..."

test:
  stage: test
  script:
    - echo "Running tests..."

deploy:
  stage deploy
  script:
    - echo "Deploying to production..."

Key CI/CD Concepts Summary

CI: Integrates code changes frequently to detect issues early
CD: Automates deployment to reduce manual effort and errors
Deployment Types: Delivery (manual) vs. Deployment (automatic)
GitLab CI/CD: Built-in pipeline engine that integrates with GitLab repositories

2. Understanding Pipeline Architecture

Component	Description	Role
Runner	`gitlab-runner`	Executes jobs in isolated environments
Executor	`shell`, `docker`, `kubernetes`	Determines how jobs are executed
Job	Individual task in a pipeline	Runs a set of commands or scripts
Stage	`build`, `test`, `deploy`	Group of jobs that run sequentially
Pipeline	Full CI/CD workflow	Orchestrates stages and jobs

Example: Basic Pipeline Structure

stages:
  - build
  - test
  - deploy

build_job:
  stage: build
  script:
    - echo "Building..."

Key Concepts

Pipelines consist of stages and jobs
Each job runs in a separate environment
Stages run sequentially, jobs within stages in parallel

Pipeline
├── Stage 1 (Build)
│   ├── Job A
│   └── Job B
├── Stage 2 (Test)
│   ├── Job C
│   └── Job D
└── Stage 3 (Deploy)
    └── Job E

3. Understanding GitLab CI/CD Terminology

Term	Definition	Notes
Pipeline	A series of stages and jobs that are executed in response to a GitLab event (e.g., push, merge request)	Each pipeline runs in a separate environment
Job	A set of instructions to be executed by a runner	Jobs run in parallel within a stage unless configured otherwise
Stage	A group of jobs that run sequentially in a pipeline	Each stage runs after the previous one completes successfully
Runner	A process that executes jobs in a pipeline	Can be shared or dedicated; must be registered with GitLab
Trigger	An event that initiates a pipeline (e.g., API call, scheduled job)	Can be configured via GitLab UI or API
Variable	A value that can be used in a pipeline (e.g., environment variables)	Can be set at project, group, or instance level
Artifact	Files produced by a job that can be passed to other jobs in the same pipeline	Useful for sharing build outputs or test reports
Cache	A mechanism to store and reuse files between jobs in a pipeline	Improves performance by reducing redundant downloads
Template	A reusable configuration file that defines a pipeline structure	Can be used to standardize CI/CD across projects
Environment	A named context that represents a deployment target (e.g., staging, production)	Used to manage deployments and monitor services

Example: Pipeline with Multiple Stages

stages:
  - build
  - test
  - deploy

build_job:
  stage: build
  script:
    - echo "Building..."
  artifacts:
    paths:
      - build/

test_job:
  stage: test
  script:
    - echo "Testing..."
  dependencies:
    - build_job

deploy_job:
  stage: deploy
  script:
    - echo "Deploying..."
  environment:
    name: production

4. Understanding Pipeline Types

Type	Description	Use Case	Example
Project Pipeline	Runs for a specific project and its branches.	Standard CI/CD for a single project.	`build_job: script: - echo "Building..."`
Group Pipeline	Runs for all projects within a group.	Shared CI/CD across multiple projects in a group.	`include: - project: 'group/shared-pipeline' file: 'ci-template.yml'`
Instance Pipeline	Runs for all projects in the GitLab instance.	Instance-wide policies or monitoring.	`workflow: rules: - if: '$CI_PIPELINE_SOURCE == "schedule"'`
Multi-Project Pipeline	Runs across multiple projects in a defined hierarchy.	Multi-repo builds or deployments.	`include: - project: 'team/app1' file: 'ci.yml' - project: 'team/app2' file: 'ci.yml'`

5. Understanding Pipeline Triggers

Trigger Type	Description	Usage	Notes
push	Triggers pipeline on commits to a branch or tag	`trigger: push`	Default trigger; can be configured to exclude branches
webhook	Triggers pipeline via external system (e.g., GitHub, Jenkins)	`trigger: webhook`	Requires a webhook URL to be configured in GitLab
schedule	Triggers pipeline at a specific time or interval	`trigger: schedule`	Can be configured in the GitLab UI or via `.gitlab-ci.yml`
manual	Triggers pipeline manually by user from GitLab UI	`trigger: manual`	Useful for approval workflows or testing
external	Triggers pipeline from an external GitLab instance or project	`trigger: external`	Requires a token and project path to be set up

Example: Manual Trigger Configuration

deploy:
  trigger:
    project: "my-org/deploy"
    branch: "main"
  when: manual

Note: Manual triggers can be configured to run only when a specific condition is met, such as a tag being pushed or a merge request being merged.

6. Understanding Pipeline Status

Status	Description	Color Indicator	Notes
created	Pipeline has been created but not yet started	Light Gray	Initial state before execution begins
waiting_for_resource	Pipeline is waiting for a resource to become available	Light Gray	Resource limits or runner availability
preparing	Pipeline is preparing to run jobs	Blue	Setting up environment and dependencies
running	Pipeline is currently executing jobs	Blue	Active execution in progress
success	All jobs in the pipeline completed successfully	Green	Final success state
failed	At least one job in the pipeline failed	Red	Failure occurred in one or more jobs
canceled	Pipeline was manually canceled or stopped by user	Yellow	Explicit cancellation by user or system
skipped	Pipeline was skipped due to conditions (e.g., branch rules)	Light Gray	Conditional skip based on configuration

Note: Pipeline status can be accessed via the GitLab UI or API endpoints like /projects/:id/pipelines.

7. Understanding Job States

State	Description	Notes
created	Job has been created and is waiting to be picked up by a runner.	Initial state when a job is scheduled.
running	Job is currently executing on a runner.	Active execution phase.
success	Job completed successfully without errors.	Final state for successful jobs.
failed	Job failed due to an error or script failure.	Can be retried if configured.
canceled	Job was manually canceled or preempted by a newer pipeline.	Can occur when a new pipeline starts before this job finishes.
skipped	Job was skipped due to conditions (e.g., when `only:` or `except:` filters do not match).	Does not run any scripts.

Example: Job state transition in a pipeline

job1:
  script:
    - echo "Running job"
  status: success

Note: Job states can be monitored via the GitLab UI or API to track pipeline progress and troubleshoot failures.

8. Understanding Runner Types

Type	Description	Use Case	Notes
Shared Runners	Runners provided by GitLab.com or a self-hosted GitLab instance	For projects that don't require specific infrastructure or security controls	May be shared across multiple projects; less control over environment
Group Runners	Runners assigned to a specific group of projects	For projects within the same team or organization that share similar requirements	Can be configured with group-specific settings and access controls
Project Runners	Runners assigned to a specific project	For projects requiring custom environments or strict isolation	Offer full control over configuration and resources
Specific Runners	Runners tied to a specific project or group and can be configured with custom tags	For projects that require specific hardware or software configurations	Can be tagged to match job requirements using `tags:`

Example: Using Tags to Match Runners

job_name:
  script:
    - echo "Running on a tagged runner"
  tags:
    - linux
    - docker

Note: Runners must be registered with appropriate tags to match jobs that specify those tags.

9. Understanding Executor Types

Executor	Description	Use Case	Notes
shell	Runs jobs in a shell environment on the runner machine.	Simple scripts, basic automation tasks.	Default executor for most runners. Supports all shell commands.
docker	Runs jobs inside a Docker container using the specified image.	Isolated environments, multi-language projects.	Requires Docker to be installed on the runner. Supports custom images.
docker-compose	Runs jobs using a Docker Compose configuration file.	Complex setups with multiple services (e.g., DBs, caches).	Uses Docker Compose to define and run multi-container applications.
kubernetes	Runs jobs in a Kubernetes pod using a specified image.	Scalable, cloud-native environments.	Requires Kubernetes cluster access. Supports dynamic resource allocation.
ssh	Runs jobs on a remote server via SSH.	Remote deployment or testing in specific environments.	Requires SSH access to the target host. Not suitable for CI/CD pipelines with high security requirements.
parallels	Runs jobs inside a Parallels VM on macOS runners.	macOS-specific builds or testing in virtualized environments.	Only available on macOS runners. Requires Parallels Desktop installed.
virtualbox	Runs jobs inside a VirtualBox VM on macOS runners.	macOS-specific builds or testing in virtualized environments.	Only available on macOS runners. Requires VirtualBox installed.

Example: Using Docker Executor

job_name:
  image: node:18
  script:
    - npm install
    - npm test

Note: The docker executor is ideal for projects that require consistent environments across different stages of CI/CD.

Warning: When using the kubernetes executor, ensure that your Kubernetes cluster has sufficient resources to handle concurrent job execution.

10. Understanding Artifacts vs Cache

Aspect	Artifacts	Cache
Purpose	Store files generated during a job for use in other jobs	Store files between jobs to speed up builds
Scope	Shared across jobs in the same pipeline	Shared within a project or group
Retention	Stored until pipeline is deleted or manually removed	Retained for a specified time or until manually cleared
Usage	Used for output files like binaries, reports, or documentation	Used to store dependencies or build tools for faster builds
Storage Location	GitLab server or external storage (e.g., S3)	GitLab server or external storage (e.g., S3)
Access	Can be downloaded by other jobs in the pipeline	Available to subsequent jobs in the same project

Example: Using Artifacts

build_job:
  script:
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

test_job:
  script:
    - npm run test
  dependencies:
    - build_job

Example: Using Cache

build_job:
  script:
    - npm install
  cache:
    key: "node-modules"
    paths:
      - node_modules/

Note: Artifacts are useful for storing build outputs, while cache is ideal for speeding up repeated steps like dependency installation.