Skip to main content
Mapagam

Understanding Pipeline Concepts

1. Understanding Pipeline as Code

Jenkins Pipeline as Code allows you to define your CI/CD workflows in a Jenkinsfile, which is a text file written in Groovy syntax. This approach enables version control, collaboration, and better maintainability of your build processes.

Feature Description Usage
Jenkinsfile Text file containing Pipeline definition pipeline { agent any; stages { stage('Build') { steps { echo 'Building...' } } } }
Declarative Pipeline Structured, opinionated syntax for defining Pipelines pipeline { agent any; stages { stage('Build') { steps { sh 'make' } } } }
Scripted Pipeline Flexible, Groovy-based syntax for complex logic node { stage('Build') { sh 'make' } }

Example: Declarative Pipeline

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'make'
            }
        }
        stage('Test') {
            steps {
                sh 'make test'
            }
        }
    }
}
Note: The Jenkinsfile should be committed to your source control system alongside your code.
Warning: Scripted pipelines can be more error-prone due to their flexibility and lack of structure.

2. Comparing Declarative vs Scripted Pipelines

Feature Declarative Pipeline Scripted Pipeline
Syntax Structured, DSL-based syntax with predefined blocks Full Groovy scripting with flexible syntax
Structure Uses stages, steps, and agents with strict structure Uses nodes, stages, and steps with more freedom
Readability Highly readable and intuitive for simple pipelines Less readable due to Groovy syntax complexity
Flexibility Less flexible, limited to predefined structures Highly flexible, allows complex logic and control flow
Error Handling Automatic error handling within stages Manual error handling required
Use Case Best for standard CI/CD workflows with clear steps Best for complex logic, conditional execution, and custom scripts

Example: Declarative Pipeline

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}

Example: Scripted Pipeline

node {
    stage('Build') {
        echo 'Building...'
    }
}

3. Understanding Jenkinsfile Structure

Element Description Usage
pipeline Top-level directive defining a Jenkins Pipeline pipeline { ... }
agent Defines where the Pipeline will run agent any or agent { docker 'node:16'}
stages Contains all the stages to be executed in the Pipeline stages { ... }
post Defines actions to be executed after the Pipeline completes post { always { ... } }
environment Defines environment variables for the Pipeline environment { NODE_ENV = 'production' }
options Defines Pipeline-specific options options { buildDiscarder(logRotator: daysToKeep(7)) }

Example: Basic Jenkinsfile Structure

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}
Note: The Jenkinsfile is a text file that contains the entire definition of your Pipeline in a single location.

4. Working with Pipeline Stages and Steps

Stage/Step Description Example
stage Defines a distinct phase of the pipeline execution 
stage('Build') {
    steps {
        sh 'mvn clean package'
    }
}
steps Contains the actual actions to be executed within a stage 
steps {
    sh 'echo "Starting build"'
    sh 'npm install'
}
sh Executes shell commands in the build environment sh 'ls -la'
script Allows Groovy code execution inside a stage 
script {
    def buildNumber = "BUILD-${env.BUILD_NUMBER}"
    echo "Build number is ${buildNumber}"
}
when Conditionally executes a stage based on criteria 
when {
    branch 'main'
} 
stage('Deploy') { ... }

Example: Defining a Multi-Stage Pipeline

pipeline {
    agent any
    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/example/repo.git'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }
        stage('Test') {
            steps {
                sh 'mvn test'
            }
        }
        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh 'echo "Deploying to production"'
            }
        }
    }
}
Note: Stages are executed sequentially by default, but can be configured to run in parallel using parallel blocks.

Key Concepts

  • Stages define logical phases of a pipeline (e.g., build, test, deploy)
  • Steps are the actual commands or actions within a stage
  • sh is used for executing shell commands in the build environment
  • when allows conditional execution of stages based on criteria like branch name or environment

5. Configuring Pipeline Agents

Property Description Values Notes
agent Defines where the pipeline will run any, none, node, docker, dockerfile Default is any if not specified
agent.node Specifies the node name to run on label, name Can specify a label or node name
agent.docker Runs the pipeline in a Docker container image, registryUrl, args Requires Docker to be installed on the Jenkins node

Example: Using Docker Agent

pipeline {
    agent {
        docker {
            image 'maven:3.8.1-jdk-8'
            args '-v /tmp:/tmp'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
    }
}

Example: Using Node Agent

pipeline {
    agent {
        node {
            label 'master'
        }
    }
    stages {
        stage('Build') {
            steps {
                sh 'echo "Building on master node"'
            }
        }
    }
}
Note: The agent directive must be defined at the top level of the pipeline or within a stage. It determines where Jenkins runs the steps in that part of the pipeline.

6. Managing Pipeline Workspace

Property Description Default Value Notes
workspace Defines the working directory for a Jenkins Pipeline job /var/lib/jenkins/workspace/<job_name> Can be customized using the workspace directive in a Pipeline script
workspace.path Specifies the path to the workspace directory /var/lib/jenkins/workspace/<job_name> Can be overridden to use a custom directory
workspace.clean Determines whether to clean the workspace before running a build false Set to true to ensure a clean environment for each build

Example: Setting Custom Workspace Path

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
    post {
        always {
            cleanWs()
        }
    }
}

Example: Cleaning Workspace

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'echo "Building..."'
            }
        }
    }
    post {
        always {
            cleanWs()
        }
    }
}
Note: The cleanWs() step removes all files from the workspace before starting a new build. This ensures that builds are isolated and do not depend on previous build artifacts.

7. Learning Pipeline Syntax

Jenkins Pipeline syntax is based on Groovy, with a domain-specific language (DSL) that allows defining CI/CD workflows as code. The syntax supports both declarative and scripted pipelines.

Feature Description Example
Declarative Pipeline Structured, opinionated syntax using a Jenkinsfile. Easier to read and maintain. 
pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}
Scripted Pipeline More flexible, allows complex logic using Groovy. Less structured but more powerful. 
node {
    stage('Build') {
        echo 'Building...'
    }
}

Example: Declarative Pipeline with Parameters

pipeline {
    agent any
    parameters {
        string(name: 'BUILD_NAME', defaultValue: 'default-build', description: 'Name of the build')
    }
    stages {
        stage('Build') {
            steps {
                echo "Building ${params.BUILD_NAME}"
            }
        }
    }
}

Example: Scripted Pipeline with Conditional Logic

node {
    stage('Check Branch') {
        if (env.BRANCH_NAME == 'main') {
            echo 'Running on main branch'
        } else {
            echo 'Running on feature branch'
        }
    }
}
Note: Declarative pipelines are preferred for most use cases due to their simplicity and readability.

Pipeline Syntax Overview

  • agent: Defines where the pipeline will run.
  • stages: Contains one or more stages to execute.
  • stage: A logical grouping of steps in a pipeline.
  • steps: The actions to perform within a stage.
  • parameters: Allows passing input values to the pipeline.

8. Applying Pipeline Best Practices

Key Best Practices for Jenkins Pipelines

  • Use Declarative Pipeline syntax for better readability and maintainability
  • Implement proper error handling with catch blocks
  • Utilize parallel stages to optimize build times
  • Keep Jenkinsfiles in version control with the source code
  • Use parameterized pipelines for flexibility
  • Implement proper logging and monitoring
  • Follow security best practices for pipeline configuration
Best Practice Description Implementation
Declarative Syntax Use Declarative Pipeline over Scripted Pipeline for better structure and readability 
pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}
Error Handling Implement error handling to prevent pipeline failures from stopping the entire process 
pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                script {
                    try {
                        sh 'build-command'
                    } catch (e) {
                        echo "Build failed: ${e}"
                    }
                }
            }
        }
    }
}
Parallel Stages Use parallel stages to run independent tasks concurrently 
pipeline {
    agent any
    stages {
        stage('Build') {
            parallel {
                stage('Frontend') {
                    steps {
                        echo 'Building frontend...'
                    }
                }
                stage('Backend') {
                    steps {
                        echo 'Building backend...'
                    }
                }
            }
        }
    }
}

Example: Parameterized Pipeline

pipeline {
    agent any
    parameters {
        string(name: 'ENVIRONMENT', defaultValue: 'dev', description: 'Target environment')
        booleanParam(name: 'DEPLOY', defaultValue: true, description: 'Deploy after build')
    }
    stages {
        stage('Build') {
            steps {
                echo "Building for environment: ${params.ENVIRONMENT}"
            }
        }
    }
}
Note: Always validate parameters in pipelines to prevent unexpected behavior or security issues.

Declarative vs Scripted Pipeline

  • Declarative: Easier to read, structured, and less error-prone
  • Scripted: More flexible but harder to maintain and debug

9. Storing Jenkinsfile in SCM

Option Values Description
scm git Source Control Management system to use for storing Jenkinsfile
url https://github.com/example/repo.git URL of the Git repository containing Jenkinsfile
branch main Branch name to check out Jenkinsfile from

Example: Jenkinsfile in Git Repository

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}
Note: Jenkinsfile must be stored in the root directory of your SCM repository or in a subdirectory specified by the scriptPath parameter.
Warning: Ensure that Jenkins has appropriate permissions to read from the SCM repository.

10. Using Pipeline Snippet Generator

Example: Using Pipeline Snippet Generator

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'echo "Building..."'
            }
        }
    }
}
Field Description Usage
pipeline Top-level directive defining a Jenkins Pipeline Required to define a pipeline job
agent Specifies where the Pipeline will be executed Must be defined at least once in a pipeline
stages Contains all the stages to be executed in the pipeline Required to define steps in a pipeline
steps Defines the actions to be performed in a stage Required inside each stage
Note: The Pipeline Snippet Generator helps users generate code snippets for various Jenkins plugins and actions without needing to write the full syntax manually.
Warning: Ensure that the Jenkins Pipeline plugin is installed and enabled to use the Snippet Generator.

Key Features of Pipeline Snippet Generator

  • Generates code snippets for Jenkins plugins
  • Supports various pipeline steps like sh, bat, and others
  • Helps in reducing syntax errors during pipeline creation
  • Provides a user-friendly interface for complex configurations

Was this article helpful?

Related Cheat Sheets