Optimizing Build Performance

1. Using Parallelism Effectively

Property	Description	Values
parallelism	Number of parallel jobs to run in a workflow	`4`

Example: Setting parallelism in a workflow

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
      - run: npm test

workflows:
  version: 2
  build-and-test:
    jobs:
      - build:
          parallelism: 4

Note: Parallelism is set per job and affects how many containers run simultaneously.

Warning: Setting parallelism too high can lead to resource contention and slower builds.

2. Choosing Optimal Resource Classes

Resource Class	CPUs	Memory	Notes
circleci/python:3.9	`1`	`2GB`	Standard Python environment
circleci/python:3.9-node	`1`	`2GB`	Python with Node.js support
circleci/python:3.9-node-browsers	`1`	`2GB`	Python with Node.js and browser support
circleci/python:3.9-node-browsers-2x	`2`	`4GB`	Double resource class for larger builds
circleci/python:3.9-node-browsers-4x	`4`	`8GB`	Quad resource class for heavy builds

Example: Setting Resource Class in Config

version: 2.1
jobs:
  build:
    resource_class: circleci/python:3.9-node-browsers-2x
    steps:
      - checkout
      - run: python -m pytest

Note: Resource classes are defined in the CircleCI configuration file using the resource_class key within a job definition.

3. Running Jobs in Parallel

Property	Description	Values
parallelism	Number of parallel jobs to run	`4`
matrix	Define job variations	`{ version: [12, 14, 16] }`

Example: Parallel Jobs Configuration

jobs:
  build:
    parallelism: 4
    steps:
      - checkout
      - run: echo "Running on parallel job"

Example: Matrix Configuration

jobs:
  test:
    matrix:
      parameters:
        version: [12, 14, 16]
    steps:
      - run: echo "Testing with Node.js ${{ parameters.version }}"

4. Enabling Docker Layer Caching

Docker layer caching in CircleCI can significantly speed up your builds by reusing previously downloaded or built layers. This feature is particularly useful when building Docker images in your CI pipeline.

Setting	Description	Default	Notes
docker_layer_caching	Enables or disables Docker layer caching for the job.	`false`	Set to `true` to enable caching.

Example: Enabling Docker Layer Caching

jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm install
      - save_cache:
          key: v1-dependencies-{{ checksum "package-lock.json" }}
          paths:
            - node_modules
      - restore_cache:
          key: v1-dependencies- {{ checksum "package-lock.json" }}

Note: Docker layer caching is supported in CircleCI's Docker executor. It does not work with the macOS or Windows executors.

5. Optimizing Cache Usage

Example: Using CircleCI's Built-in Caching

version: 2.1
jobs:
  build:
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package-lock.json" }}
            - v1-dependencies-
      - run: npm install
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package-lock.json" }}

Cache Key	Description	Notes
v1-dependencies-{{ checksum "package-lock.json" }}	Cache key based on checksum of package-lock.json	Ensures cache is invalidated when dependencies change
v1-dependencies-	Fallback cache key without checksum	Used if checksum fails or for initial cache restore

Note: Cache keys must be unique per project and should include a version prefix to avoid conflicts.

Example: Cache with Multiple Paths

version: 2.1
jobs:
  build:
    steps:
      - restore_cache:
          keys:
            - v1-build-{{ arch }}-{{ checksum "build.config" }}
      - run:
          name: Build Project
          command: |
            npm run build
      - save_cache:
          paths:
            - ./dist
            - ./node_modules
          key: v1-build-{{ arch }}-{{ checksum "build.config" }}

Best Practices for Cache Optimization

Use checksums to invalidate cache on dependency changes
Include architecture-specific keys for cross-platform builds
Save cache after successful builds to reduce re-computation
Avoid caching large files that change frequently

6. Using Workspaces Efficiently

Property	Description	Usage
save	Saves files or directories to the workspace	`save: /path/to/files`
restore	Restores files or directories from the workspace	`restore: /path/to/files`
paths	Defines paths to save or restore	`paths: ["/tmp/cache"]`

Example: Save and Restore Workspace

jobs:
  build:
    steps:
      - run:
          name: Save cache
          command: |
            mkdir -p /tmp/cache
            cp -r node_modules /tmp/cache/
      - persist_to_workspace:
          root: /tmp/cache
          paths:
            - node_modules

  test:
    steps:
      - attach_workspace:
          at: /tmp/cache

Note: Workspaces are scoped to the build and are not shared between different builds or branches.

Warning: Large workspace sizes can slow down builds. Keep workspace content minimal.

7. Splitting Tests Intelligently

Option	Values	Notes
parallelism	`4`	Number of parallel jobs to run
shard	`1/4`	Shard index and total shards (e.g., 1/4)
test_split_strategy	`size`	How to split tests: `size`, `time`, or `random`

Example: Splitting Tests by Size

jobs:
  test:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run:
          name: Run Tests in Parallel
          command: |
            npm test -- --shard=1/4

Note: The --shard option is provided by CircleCI's test splitting feature. It allows you to run a subset of tests in parallel.

Best Practices for Test Splitting

Use test_split_strategy to balance test execution times.
Ensure each shard has a roughly equal number of tests to avoid bottlenecks.
Monitor test execution times to optimize shard distribution.

8. Optimizing Dependency Installation

Option	Values	Notes
circleci/cli	`circleci/cli`	Install CircleCI CLI for local testing and debugging
cache key	`cache-key`	Use consistent cache keys to avoid unnecessary reinstallation
restore cache	`restore-cache`	Restore dependencies from previous builds to speed up installation
save cache	`save-cache`	Save dependencies after installation to reuse in future builds

Example: Caching Dependencies in CircleCI

steps:
  - restore_cache:
      key: v1-dependencies-{{ checksum "package-lock.json" }}
  - run:
      name: Install Dependencies
      command: npm ci --prefer-offline
  - save_cache:
      key: v1-dependencies-{{ checksum "package-lock.json" }}
      paths:
        - node_modules

Note: Using npm ci instead of npm install ensures deterministic builds and faster dependency installation.

Warning: Avoid caching node_modules directly without checksums, as it can lead to inconsistent builds.

Cache Strategy	Description	Best For
`checksum "package-lock.json"`	Caches based on the content of `package-lock.json`	Stable dependency versions
`checksum "package.json"`	Caches based on `package.json` changes	Quick dependency updates
`branch-{{ branch }}`	Caches per branch	Branch-specific dependencies

9. Reducing Checkout Time

Option	Values	Notes
checkout	`true`	Default behavior; checks out code
checkout	`false`	Skips checkout; useful for jobs that don't need source code

Example: Skip checkout for non-source jobs

jobs:
  lint:
    docker:
      - image: circleci/node:18
    steps:
      - checkout: false
      - run: npm run lint

Note: When checkout: false, the working directory is not populated with source code, so any steps that require files from the repo must be skipped or handled differently.

Option	Values	Notes
checkout	`true`	Default behavior; checks out code
checkout	`false`	Skips checkout; useful for jobs that don't need source code

10. Using Orbs for Common Tasks

Orb	Description	Use Case	Version
circleci/node	Pre-configured Node.js environment with caching and dependency installation	JavaScript/TypeScript projects	`5.0.0`
circleci/python	Python environment with virtualenv support and pip caching	Python projects	`5.2.0`
circleci/aws-cli	Pre-configured AWS CLI with common tools and credentials handling	AWS deployment or data operations	`1.4.0`
circleci/docker	Docker build and push utilities with image caching	Containerized applications	`4.1.0`

Example: Using Node.js Orb

version: 2.1
orbs:
  node: circleci/node@5.0.0
jobs:
  build:
    executor: node/default
    steps:
      - checkout
      - node/restore-cache
      - run: npm install
      - node/save-cache
      - run: npm test

Note: Orbs can significantly reduce configuration time by providing pre-built workflows and executors.

Warning: Ensure that the orb version is compatible with your CircleCI configuration syntax.