Working with Pipeline Stages

1. Defining Sequential Stages

Stage	Description	Example
stage("Build")	Executes build steps in sequence	`stage('Build') { steps { sh 'mvn clean compile' } }`
stage("Test")	Runs tests after build stage	`stage('Test') { steps { sh 'mvn test' } }`
stage("Deploy")	Deploys application to target environment	`stage('Deploy') { steps { sh 'mvn deploy' } }`

Example: Basic Pipeline with Sequential Stages

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        sh 'echo "Building..."'
      }
    }
    stage('Test') {
      steps {
        sh 'echo "Testing..."'
      }
    }
    stage('Deploy') {
      steps {
        sh 'echo "Deploying..."'
      }
    }
  }
}

2. Using Parallel Stages

Parallel stages allow you to run multiple stages concurrently within a Jenkins Pipeline, which can significantly reduce build times when stages are independent of each other.

Property	Description	Example
parallel	Defines a block where multiple stages can run in parallel.	`parallel`

Example: Basic Parallel Stage Usage

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        echo 'Running unit tests...'
                    }
                }
                stage('Integration Tests') {
                    steps {
                        echo 'Running integration tests...'
                    }
                }
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

Note: All stages within the parallel block must complete before moving to the next stage in the pipeline.

Key Points About Parallel Stages

Each parallel stage runs independently.
Stages inside parallel must be defined as stages, not steps.
Parallel execution is managed by Jenkins and does not require additional configuration.

3. Naming Stages Effectively

Stage Name	Description	Best Practices
checkout	Checks out the source code from version control	Use descriptive names like `checkout-source` for clarity
build	Compiles and builds the project	Include sub-stages like `build-backend` or `build-frontend`
test	Runs unit and integration tests	Break down into `test-unit`, `test-integration`
deploy	Deploys the application to a target environment	Use `deploy-staging` or `deploy-production`

Example: Stage Naming in Jenkins Pipeline

pipeline {
    agent any
    stages {
        stage('checkout-source') {
            steps {
                checkout scm
            }
        }
        stage('build-backend') {
            steps {
                sh 'make build'
            }
        }
        stage('test-unit') {
            steps {
                sh 'npm test'
            }
        }
    }
}

Note: Stage names should be unique within a pipeline and follow the convention of using lowercase letters and hyphens for readability.

4. Using Nested Stages

Nested stages in Jenkins Pipeline allow you to organize complex workflows into logical groups. They are useful for structuring builds with shared setup/teardown logic or grouping related steps.

Property	Description	Example
stage("Outer Stage")	Top-level stage containing nested stages	`stage("Build") { ... }`
stage("Inner Stage")	Stage defined inside another stage	`stage("Unit Tests") { ... }`

Example: Nested Stages in Jenkins Pipeline

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
            stages {
                stage('Compile') {
                    steps {
                        echo 'Compiling sources...'
                    }
                }
                stage('Test') {
                    steps {
                        echo 'Running tests...'
                    }
                }
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

Note: Nested stages must be defined within the stages block of a parent stage. The outer stage can contain both steps and nested stages.

Best Practices for Nested Stages

Use nested stages to group logically related steps under a common parent stage
Keep nesting levels shallow (max 2-3 levels) to maintain readability
Ensure each nested stage has a descriptive name that reflects its purpose
Use shared libraries or functions to reduce repetition in nested stages

5. Skipping Stages Conditionally

Property	Description	Usage
when	Condition to evaluate before running a stage	`when { branch 'main' }`
skip	Skips stage if condition is met	`when { environment name: 'ENV', value: 'prod' } skip true`

Example: Skip stage based on environment

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Deploy to Staging') {
            when {
                environment name: 'ENV', value: 'staging'
            }
            steps {
                echo 'Deploying to staging...'
            }
        }
        stage('Deploy to Production') {
            when {
                environment name: 'ENV', value: 'prod'
            }
            steps {
                echo 'Deploying to production...'
            }
        }
    }
}

Note: The when directive can be used to conditionally skip stages based on environment variables, branches, or other conditions.

6. Using Stage Timeout

Stage timeouts in Jenkins Pipeline help prevent long-running stages from consuming resources indefinitely. You can configure a timeout for individual stages using the timeout directive.

Directive	Description	Example
timeout	Defines a maximum time for a stage to complete. If exceeded, the stage is terminated.	`stage('Build') { timeout(time: 10, unit: 'MINUTES') { sh 'mvn clean compile' } }`

Example: Setting a Timeout on a Stage

pipeline {
  agent any
  stages {
    stage('Build') {
      timeout(time: 5, unit: 'MINUTES') {
        sh 'echo "Building project..." && sleep 300'
      }
    }
  }
}

Note: The unit parameter supports values like SECONDS, MINUTES, HOURS, and DAYS.

Warning: If a stage exceeds its timeout, Jenkins will mark it as failed and stop execution of that stage.

7. Implementing Stage Retry Logic

Property	Description	Default Value	Notes
retry	Number of times to retry a failed stage	0	Can be used with `stage` or `parallel` blocks
maxFailures	Maximum number of failures before aborting the pipeline	Unlimited	Useful for preventing infinite retries in unstable environments

Example: Retry a Stage on Failure

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
                sh 'exit 1' // Simulate failure
            }
            post {
                always {
                    echo 'Post-build actions'
                }
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
                sh 'exit 1' // Simulate failure
            }
            retry(3) {
                echo 'Retrying deployment...'
                sh 'exit 1'
            }
        }
    }
}

Example: Retry with Max Failures

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                echo 'Running tests...'
                sh 'exit 1'
            }
            post {
                always {
                    echo 'Test stage completed'
                }
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
                sh 'exit 1'
            }
            retry(maxFailures: 2) {
                echo 'Retrying deployment...'
                sh 'exit 1'
            }
        }
    }
}

Note: The retry directive will retry the entire stage if any step fails, including steps in post blocks.

Warning: Be cautious with retry logic in production pipelines as it can lead to resource exhaustion or inconsistent states.

8. Using Stage Input Approvals

Property	Description	Example
input	Defines the input step for stage approval	`input 'Approve deployment'`
message	Message displayed to the user during input	`message 'Please confirm deployment'`
ok	Text for the "OK" button	`ok 'Deploy'`
submitter	Whitelist of users allowed to approve	`submitter 'admin, deployer'`

Example: Stage with Input Approval

pipeline {
    agent any
    stages {
        stage('Deploy') {
            input {
                message 'Approve deployment to production'
                ok 'Deploy'
                submitter 'admin, deployer'
            }
            steps {
                echo 'Deploying to production...'
            }
        }
    }
}

Note: The input step pauses the pipeline and waits for user input before proceeding.

Warning: Ensure that the submitter list is properly configured to restrict access.

9. Visualizing Stage View

Example: Pipeline Stage View Configuration

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing...'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }
    }
}

Property	Description	Default Value	Notes
stage.name	Name of the stage	None	Required field for each stage
stage.steps	Steps to execute in the stage	None	Can include multiple steps
stage.parallel	Parallel execution of stages	false	Allows running stages concurrently

Note: The Stage View in Jenkins provides a visual representation of the pipeline execution flow, making it easier to understand and debug complex pipelines.

Warning: When using parallel stages, ensure that each stage has sufficient resources to avoid performance bottlenecks.

10. Using Stage Options

Option	Description	Values	Notes
stage.name	Name of the stage	`string`	Used for display in UI and logs
stage.parallel	Run steps in parallel	`true` or `false`	Default: `false`
stage.timeout	Timeout for stage execution	`duration`	Example: `10 min`
stage.retry	Number of retries on failure	`integer`	Default: `0`
stage.agent	Agent to run the stage on	`agent specification`	Can be `none`, `any`, or a specific agent
stage.options	Additional options for stage	`map`	Supports `buildDiscarder`, `timestamps`

Example: Using Stage Options

pipeline {
    agent any
    stages {
        stage('Build') {
            options {
                buildDiscarder logRotator(daysToKeepStr: '7', numToKeepStr: '5')
                timestamps()
            }
            steps {
                echo 'Building...'
            }
        }
    }
}

Note: Stage options can be applied globally or per stage. timestamps() adds timestamps to logs.