Implementing Parallel Execution
1. Defining Parallel Stages
| Property | Description | Values | Notes |
|---|---|---|---|
| parallel | Defines a block where multiple stages run in parallel | parallel |
Used inside a stage block to define concurrent execution |
| stage | Represents a single stage within the pipeline | stage('Name') |
Each parallel stage must be defined as a stage block |
Example: Basic Parallel Stages
pipeline {
agent any
stages {
stage('Build') {
parallel {
stage('Frontend') {
steps {
echo 'Building frontend...'
}
}
stage('Backend') {
steps {
echo 'Building backend...'
}
}
}
}
}
}
Note: All parallel stages within a
parallel block must complete before the next stage in the pipeline begins.
2. Using FailFast Option
| Option | Values | Description |
|---|---|---|
| failFast | true | false |
When set to true, stops execution of the pipeline if any stage fails. Defaults to false. |
Example: Setting failFast in a Pipeline
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
}
}
stage('Test') {
steps {
sh 'npm test'
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
}
}
}
options {
failFast true
}
}
Note: Setting
failFast to true ensures that the pipeline stops immediately upon failure, preventing unnecessary resource consumption.
3. Implementing Parallel Steps
| Step | Description | Example |
|---|---|---|
| parallel | Executes multiple stages in parallel | |
Example: Parallel Build Steps
pipeline {
agent any
stages {
stage('Build') {
parallel {
stage('Backend') {
steps {
echo 'Building backend...'
}
}
stage('Frontend') {
steps {
echo 'Building frontend...'
}
}
}
}
}
}
Note: All parallel stages must complete before the pipeline continues.
Warning: Avoid resource contention in parallel stages.
4. Using Matrix Builds
Matrix builds in Jenkins allow you to run tests or builds across multiple combinations of axes, such as different operating systems, JDK versions, or other parameters.
| Axis | Values | Description |
|---|---|---|
| os | linux, windows, macos |
Operating system to run the build on |
| jdk | openjdk11, openjdk17, oracle8 |
Java Development Kit version to use |
Example: Matrix Build Configuration
pipeline {
agent any
stages {
stage('Build') {
matrix {
axes {
os ['linux', 'windows']
jdk ['openjdk11', 'oracle8']
}
stage('Test') {
steps {
sh "echo 'Running on ${os} with JDK ${jdk}'"
}
}
}
}
}
}
Note: Matrix builds are particularly useful for testing compatibility across multiple environments. Each combination of axis values creates a separate build.
5. Configuring Parallel Axis
| Property | Description | Values | Notes |
|---|---|---|---|
| parallel.axis | Defines the axis along which parallel execution occurs | node, environment |
Default is node |
| parallel.axis.value | Value to use for the parallel axis | String or variable reference | Used to define how jobs are distributed across nodes or environments |
Example: Configuring Parallel Axis on Nodes
parallel axis: 'node',
branches: [
master: {
echo "Running on master node"
},
worker: {
echo "Running on worker node"
}
]
Note: The
parallel.axis property allows you to define how Jenkins distributes jobs across different execution contexts, such as nodes or environments.
6. Excluding Matrix Combinations
| Property | Values | Description |
|---|---|---|
| excludes | string[] |
Array of matrix combinations to exclude from execution |
Example: Excluding Matrix Combinations
pipeline {
agent any
stages {
stage('Build') {
matrix {
axes {
platform(['linux', 'windows'])
version(['1.0', '2.0'])
}
excludes [
[platform: 'windows', version: '1.0']
]
}
}
}
}7. Using Parallel with Dynamic Stages
Example: Parallel Execution with Dynamic Stages
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
}
}
stage('Test') {
parallel {
stage('Unit Tests') {
steps {
echo 'Running unit tests'
}
}
stage('Integration Tests') {
steps {
echo 'Running integration tests'
}
}
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
}
}
}
}| Property | Description | Values | Notes |
|---|---|---|---|
| parallel | Defines a set of stages to run in parallel | stage('name') { steps { ... } } |
Each stage runs concurrently |
| stage | Defines a single stage within the pipeline | stage('name') { steps { ... } } |
Can be nested inside parallel blocks |
Note: Parallel stages must be defined within a
parallel block inside a stage declaration.Warning: Ensure all parallel stages complete before proceeding to subsequent stages.
Key Points
- Parallel stages allow concurrent execution of multiple tasks.
- Each stage inside
parallelruns independently. - Useful for running tests or builds in parallel to save time.
8. Handling Parallel Failures
| Failure Type | Description | Handling Strategy | Configuration |
|---|---|---|---|
| Build Failure | When one or more parallel stages fail during execution | Continue other stages, fail the overall build | failFast true to stop all stages on failure |
| Node Failure | When a node running a parallel stage becomes unresponsive | Retry or skip the failed node | retryCount and nodeProperties |
| Timeout | When a stage exceeds its configured timeout | Fail the stage and continue others | timeout(time: 10, unit: 'MINUTES') |
Example: Fail Fast Configuration
pipeline {
agent any
stages {
stage('Parallel Build') {
parallel {
stage('Build A') {
failFast true
steps {
echo 'Building A'
}
}
stage('Build B') {
failFast true
steps {
echo 'Building B'
}
}
}
}
}
}Example: Timeout Handling
pipeline {
agent any
stages {
stage('Parallel With Timeout') {
parallel {
stage('Slow Task') {
steps {
timeout(time: 5, unit: 'MINUTES') {
sh 'sleep 300'
}
}
}
}
}
}
}
Note: Use
failFast false to allow other stages to continue even if one fails.
Warning: Setting
failFast true can cause cascading failures in complex pipelines.
9. Using Lock Resources in Parallel
Lock resources in Jenkins allow you to control access to shared resources when running parallel builds. This ensures that only one build can use a specific resource at a time, preventing conflicts or race conditions.
| Property | Description | Example |
|---|---|---|
| lock(resource: 'my-resource') | Acquires a lock on the specified resource | |
| lock(resource: 'my-resource', maxLeases: 1) | Acquires a lock with a maximum number of leases | |
| lock(resource: 'my-resource', quantity: 1) | Acquires a lock with a specific quantity | |
Example: Using Lock Resources in Parallel
pipeline {
agent any
stages {
stage('Parallel with Lock') {
parallel {
stage('Build 1') {
steps {
lock(resource: 'database') {
sh 'echo "Build 1 using database"'
}
}
}
stage('Build 2') {
steps {
lock(resource: 'database') {
sh 'echo "Build 2 using database"'
}
}
}
}
}
}
}
Note: The
lock step ensures that only one build can access the resource at a time. If multiple builds attempt to acquire the lock simultaneously, they will queue until the lock is released.
10. Visualizing Parallel Execution
Example: Pipeline with Parallel Stages
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
}
}
stage('Test') {
parallel {
stage('Unit Tests') {
steps {
echo 'Running unit tests...'
}
}
stage('Integration Tests') {
steps {
echo 'Running integration tests...'
}
}
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
}
}
}
}| Stage | Description | Parallel Execution | Visualization |
|---|---|---|---|
| Build | Initial build stage | No | Sequential view |
| Test | Parallel execution of unit and integration tests | Yes | Parallel view with sub-stages |
| Deploy | Deployment stage | No | Sequential view |
Note: Jenkins Pipeline visualization shows parallel stages as horizontal bars with sub-stages stacked vertically.