Skip to main content

Tracking Pipeline Metrics and Analytics

1. Using Pipeline Metrics Plugin

Property Description Default Value Notes
pipeline.metrics.enabled Enables or disables the collection of pipeline metrics false Set to true to enable metrics collection
pipeline.metrics.exporter Specifies the exporter to use for publishing metrics prometheus Currently supports prometheus and influxdb
pipeline.metrics.prometheus.port Port on which Prometheus metrics are exposed 9090 Only used when exporter is set to prometheus

Example: Enabling Pipeline Metrics

pipeline {
    agent any
    environment {
        METRICS_ENABLED = 'true'
    }
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}

Example: Configuring Prometheus Exporter

pipeline {
    agent any
    environment {
        PROMETHEUS_PORT = '9091'
    }
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
}
Note: Ensure that the Pipeline Metrics Plugin is installed and configured in Jenkins before enabling metrics collection.
Warning: Enabling metrics can introduce performance overhead, especially in high-volume pipelines.
Metric Description Collection Method Notes
build.duration Time taken to complete a build Jenkins API or UI Can be used to identify performance bottlenecks
build.duration.avg Average build duration over a period Analytics plugin or custom script Helps in trend analysis and capacity planning
build.duration.median Median build duration over a period Analytics plugin or custom script Less affected by outliers than average

Example: Fetching Build Duration Data via Jenkins API

curl -s "http://jenkins-server/job/my-job/lastBuild/api/json?tree=duration" | jq .duration

Example: Calculating Average Build Duration

const builds = [30000, 45000, 25000, 35000]; // in milliseconds
const avgDuration = builds.reduce((a, b) => a + b, 0) / builds.length;
console.log(`Average build duration: ${avgDuration}ms`);
Note: Build duration trends can be visualized using plugins like the Build Time Trend or Plot Plugin.

3. Analyzing Stage Performance

Property Description Values Notes
stage.name Name of the stage Build, Test, Deploy Used for grouping and filtering in analytics dashboards
stage.duration Duration of the stage in milliseconds 120000 Can be used to identify bottlenecks in pipeline execution
stage.status Execution status of the stage success, failure, aborted Used to calculate pipeline success rate and failure trends
stage.startTime Start time of the stage in ISO 8601 format 2025-04-05T10:00:00Z Helps in tracking temporal performance patterns

Example: Stage Duration Analysis

def stageDuration = currentBuild.getBuild().getAction(org.jenkinsci.plugins.workflow.job.WorkflowRun$StageAction).getStages().find { it.name == "Build" }.duration
println "Build stage took ${stageDuration}ms"

Example: Stage Status Tracking

def stageStatus = currentBuild.getBuild().getAction(org.jenkinsci.plugins.workflow.job.WorkflowRun$StageAction).getStages().find { it.name == "Deploy" }.status
if (stageStatus == "SUCCESS") {
    echo "Deployment stage completed successfully"
} else {
    echo "Deployment stage failed with status: ${stageStatus}"
}
Note: The stage.duration and stage.status properties are available only in Jenkins Pipeline (Groovy) scripts.

Key Metrics for Stage Performance

  • Duration - Time taken to complete a stage
  • Status - Execution outcome (success, failure, aborted)
  • Start Time - When the stage began execution

4. Monitoring Queue Times

Property Description Default Value Notes
queue.time Time a build spends in the queue before starting execution N/A Measured in milliseconds
queue.wait.time Time a build waits in the queue before it can start N/A Includes resource allocation and scheduling delays

Example: Queue Time Metrics in Jenkins Pipeline

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo "Queue time: ${env.BUILD_QUEUE_TIME}ms"
            }
        }
    }
}
Note: Queue times can be influenced by resource constraints, build dependencies, and concurrent job limits.

Key Metrics for Queue Time Analysis

  • Average Queue Time - Helps identify bottlenecks in job scheduling
  • Max Queue Time - Highlights jobs that are consistently delayed
  • Queue Time Distribution - Shows how queue times are spread across builds

5. Using Datadog Integration

Feature Description Configuration Notes
Datadog Plugin Enables Jenkins to send pipeline metrics and logs to Datadog. Install via Jenkins Plugin Manager Requires Datadog API key
Metric Collection Collects build duration, success rate, and resource usage. Enabled via plugin settings Supports custom metric tags
Log Forwarding Forwards Jenkins logs to Datadog for centralized monitoring. Configured in Jenkins global settings Supports log filtering and enrichment

Example: Datadog Plugin Configuration

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building...'
            }
        }
    }
    post {
        always {
            datadog {
                service 'jenkins-pipeline'
                tags ['env:prod', 'team:devops']
            }
        }
    }
}
Note: Ensure the Datadog API key is securely stored in Jenkins credentials.
Warning: Misconfigured tags can lead to high cardinality in Datadog metrics.

6. Exporting Metrics to Prometheus

Property Value Description
jenkins.metrics.prometheus.enabled true Enables Prometheus metrics export
jenkins.metrics.prometheus.endpoint /prometheus Endpoint path for Prometheus scraping
jenkins.metrics.prometheus.port 9090 Port to expose metrics (default: 9090)

Example: Configuring Prometheus Exporter

import jenkins.metrics.api.Metrics
import org.jenkinsci.plugins.prometheus.config.PrometheusConfiguration

def config = PrometheusConfiguration.get()
config.setEnabled(true)
config.setEndpoint("/prometheus")
config.setPort(9090)
config.save()
Note: Ensure the prometheus-plugin is installed and enabled in Jenkins.
Warning: Prometheus metrics are exposed on the Jenkins root URL; ensure proper access control or network isolation.

7. Creating Custom Dashboards

Example: Adding a New Dashboard

import jenkins.model.Jenkins
import hudson.model.*
import org.jenkinsci.plugins.dashboard_view.DashboardView

def jenkins = Jenkins.instance
def viewName = "CustomAnalyticsDashboard"
def dashboardView = new DashboardView(viewName, "Custom analytics dashboard")
jenkins.addView(dashboardView)
jenkins.save()
Property Description Default Value
viewName Name of the new dashboard view None
description Description for the dashboard Empty string
Note: Custom dashboards require the Dashboard View plugin to be installed.
Warning: Ensure that the user has appropriate permissions to create and manage dashboards in Jenkins.

Dashboard Configuration Options

  • View Name: Unique identifier for the dashboard
  • Description: Brief explanation of the dashboard's purpose
  • Widgets: Components displayed on the dashboard (e.g., build status, job trends)
  • Permissions: Access control for dashboard visibility

Dashboard Types in Jenkins

  • Default View: Standard Jenkins view with job list
  • Dashboard View: Customizable dashboard with widgets and analytics
  • Folder View: Organized view for jobs within folders

8. Analyzing Pipeline Failure Rates

Metric Description Formula Notes
Failure Rate Percentage of failed builds in a given period (Failed Builds / Total Builds) * 100 Used to identify trends in pipeline reliability
Build Success Rate Percentage of successful builds in a given period (Successful Builds / Total Builds) * 100 Complementary to failure rate for overall health
Failure Count Total number of failed builds in a period Count of builds with status = 'FAILURE' Used for tracking absolute failure volume

Example: Calculating Failure Rate in Jenkins Pipeline

def totalBuilds = 100
def failedBuilds = 5
def failureRate = (failedBuilds / totalBuilds) * 100
echo "Failure Rate: ${failureRate}%"
Note: Jenkins provides built-in analytics and plugins like the Build Time Trend plugin to visualize failure rates over time.

Key Metrics for Failure Rate Analysis

  • Failure Rate - Percentage of failed builds
  • Build Success Rate - Percentage of successful builds
  • Failure Count - Absolute number of failed builds

9. Tracking Resource Utilization

Resource Type Metric Name Description Collection Method
CPU Usage cpu.utilization Percentage of CPU time used by Jenkins processes System-level monitoring via JMX or OS tools
Memory Usage memory.used Total memory consumed by Jenkins instance Java Memory MBean or jstat
Disk I/O disk.io.read.bytes Total bytes read from disk by Jenkins OS-level I/O counters or iostat
Network Traffic network.bytes.sent Total bytes sent over network by Jenkins System-level network monitoring tools

Example: Monitoring CPU Usage via JMX

curl -u admin:password http://jenkins.example.com/jmx/threads/cpu.utilization
Note: Resource utilization metrics can be collected using external tools like Prometheus or Grafana, which integrate with Jenkins via JMX or custom plugins.

10. Using Build Cause Analytics

Build Cause Description Analytics Type Usage
SCM Triggered by changes in source control management system Change Frequency Tracks how often code changes trigger builds
Timer Triggered by a scheduled time Schedule Adherence Measures adherence to scheduled builds
Manual Triggered by user action User Activity Monitors manual build frequency and patterns
Upstream Triggered by completion of another build Dependency Tracking Measures impact of upstream builds on downstreams
Remote Triggered by external system or API call Integration Effectiveness Analyzes effectiveness of external triggers

Example: Build Cause Analysis Script

def buildCauses = [:]
pipeline {
    agent any
    stages {
        stage('Analyze') {
            steps {
                script {
                    def builds = Jenkins.instance.getItem("my-project").builds
                    builds.each { build ->
                        def cause = build.getCause(hudson.triggers.SCMTrigger$SCMTriggerCause)
                        if (cause) {
                            buildCauses["SCM"] = (buildCauses["SCM"] ?: 0) + 1
                        }
                        cause = build.getCause(hudson.triggers.TimerTrigger$TimerTriggerCause)
                        if (cause) {
                            buildCauses["Timer"] = (buildCauses["Timer"] ?: 0) + 1
                        }
                    }
                }
            }
        }
    }
}
Note: Build cause analytics help in identifying patterns of build triggers and can be used to optimize pipeline scheduling and resource allocation.

Was this article helpful?