Optimizing Resource Usage and Costs

1. Choosing Resource Classes

Resource Class	CPUs	Memory	Notes
medium	`2`	`4GB`	Default for most jobs
large	`4`	`8GB`	Good for CPU-intensive tasks
xlarge	`8`	`16GB`	For memory-heavy builds
2xlarge	`16`	`32GB`	High-performance builds
arm.medium	`2`	`4GB`	ARM-based builds (Apple Silicon)
arm.large	`4`	`8GB`	ARM-based builds with more resources

Note: Resource classes are defined in your config.yml under resource_class key.

Example: Setting Resource Class

jobs:
  build:
    resource_class: large
    steps:
      - checkout
      - run: npm install

2. Using ARM Resources

Resource Type	Description	Availability	Notes
arm	`arm` executor type for running jobs on ARM64 architecture	NEW CircleCI 2.1+	Enables running builds on ARM-based runners
arm64	`arm64` resource class for ARM-based compute instances	NEW CircleCI 2.1+	Provides ARM64 compute resources for faster builds

Example: Using ARM Resources in a Job

version: 2.1
jobs:
  build-arm:
    machine:
      image: ubuntu-2004:2023.07.1
      resource_class: arm64
    steps:
      - checkout
      - run:
          name: Run ARM-compatible build
          command: |
            echo "Running on ARM64 architecture"
            # Your ARM-specific build commands here

Note: ARM resources are available in CircleCI's machine executor with the arm64 resource class.

Warning: Not all tools or dependencies may be compatible with ARM64 architecture. Ensure your build environment supports ARM.

3. Optimizing for Cost

Resource	Description	Cost Optimization Tips
Compute Resources	CircleCI provides various instance types for running jobs.	Choose smaller instance types (e.g., `medium`) for lightweight jobs. Use `large` or `xlarge` only when necessary.
Parallelism	Running multiple jobs in parallel can speed up builds but increases cost.	Use `parallelism` strategically to reduce build time without overusing resources. Monitor resource usage to avoid unnecessary parallelism.
Caching	CircleCI's caching mechanism helps reduce build times by reusing dependencies.	Implement efficient caching strategies to avoid redundant downloads. Use `save_cache` and `restore_cache` effectively.
Orbs	Reusable configuration templates that can simplify workflows.	Use orbs to reduce configuration overhead and avoid reinventing the wheel. Ensure orbs are well-maintained to prevent performance issues.
Workflows	CircleCI workflows allow you to define complex job dependencies and execution patterns.	Optimize workflow structure to minimize redundant jobs. Use `depends_on` to ensure efficient job sequencing.

Example: Using Smaller Instance Types

jobs:
  build:
    machine:
      image: ubuntu-2004:202107-02
    resource_class: medium
    steps:
      - checkout
      - run: echo "Running on a medium instance"

Example: Caching Dependencies

jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package-lock.json" }}
      - run: npm install
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package-lock.json" }}

Note: CircleCI offers a free tier with limited resources. For larger projects, consider upgrading to a paid plan for better performance and cost control.

Warning: Avoid using large or xlarge instances unless necessary, as they can significantly increase costs.

4. Monitoring Credit Usage

Resource	Description	Usage Metric	Notes
credits	Credit consumption for builds	`build.credits`	Each job consumes credits based on runtime and resource type
build_minutes	Build minutes consumed per project	`project.build_minutes`	Used to calculate cost in CircleCI's credit system
credit_balance	Current credit balance for organization	`org.credit_balance`	Updated daily, reflects available credits for use

Example: Querying Credit Usage via API

curl -H "Circle-Token: <token>" \
  https://circleci.com/api/v2/credit-usage?org-slug=org-name

Note: CircleCI credits are consumed based on job runtime and resource type (e.g., medium, large, xlarge). Each job consumes credits proportional to its resource allocation.

Credit Usage Summary

Build Minutes: Used to calculate total credit consumption.
Credit Balance: Available credits for the organization to use.
API Access: Use CircleCI API to query credit usage for projects or organizations.

5. Implementing Concurrency Limits

Setting	Description	Default Value	Notes
concurrent_jobs	Maximum number of jobs that can run concurrently in a pipeline	10	Can be adjusted per project or organization
max_parallelism	Maximum parallelism allowed for a job	1	Used to limit resource usage per job

Example: Setting Concurrency Limits in config.yml

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
      - run: npm test

workflows:
  version: 2
  build-and-test:
    jobs:
      - build:
          concurrency: 5
          max_parallelism: 2

Note: Concurrency limits can be set at the job or workflow level to control resource consumption and cost.

6. Reducing Build Minutes

Example: Using CircleCI's Resource Class

version: 2.1
jobs:
  build:
    resource_class: small
    steps:
      - checkout
      - run: echo "Building..."

Resource Class	CPU	Memory	Build Minutes Cost
small	`1`	`2GB`	1 minute
medium	`2`	`4GB`	2 minutes
large	`4`	`8GB`	4 minutes
xlarge	`8`	`16GB`	8 minutes

Note: Resource classes are available in CircleCI's paid plans. Free plan users get small by default.

Best Practices for Reducing Build Minutes

Use small resource class for lightweight jobs.
Optimize build steps to reduce overall runtime.
Use CircleCI's save_cache and restore_cache to avoid re-downloading dependencies.
Parallelize jobs using parallelism to reduce total build time.

7. Using Free Tier Efficiently

Resource	Free Tier Limit	Notes
Build Minutes	`1,200 minutes/month`	Includes 20 build minutes per job
Parallelism	`1`	Only 1 job can run at a time
Workflows	`1`	Only 1 workflow can run at a time
Projects	`10`	Up to 10 projects can be configured
Storage	`5GB`	Includes build artifacts and caches

Example: Optimizing Build Minutes

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:18
    steps:
      - checkout
      - run:
          name: Install Dependencies
          command: npm install
      - run:
          name: Run Tests
          command: npm test
workflows:
  version: 2
  build-and-test:
    jobs:
      - build

Note: Free tier users can only run one job at a time. To optimize, combine steps into fewer jobs or use parallelism: 1 in job configuration.

Best Practices for Free Tier Efficiency

Use parallelism: 1 to avoid exceeding limits
Combine steps in jobs to reduce build minutes
Use caching strategies to avoid re-downloading dependencies
Limit the number of projects to 10 to stay within storage limits

8. Analyzing Resource Bottlenecks

Bottleneck Type	Description	Impact	Detection Method
CPU Usage	High CPU utilization across jobs or workflows	Slower job execution, potential timeouts	Use CircleCI's resource monitoring dashboard or `circleci-cli` to view job metrics
Memory Usage	Jobs exceeding memory limits or causing swap usage	Job failures, reduced performance	Monitor via CircleCI's job logs or `top` command in build environment
I/O Bottlenecks	High disk read/write activity or network latency	Slower builds, increased build times	Check logs for I/O wait times or use `iostat` in custom scripts
Concurrency Limits	Jobs waiting due to resource or parallelism limits	Longer build queue times, inefficient use of resources	Review job status in CircleCI UI or `circleci-cli` for queued jobs

Example: Identifying High CPU Jobs

circleci-cli monitor --job-id <job-id>

Note: CircleCI provides resource usage metrics in the job logs under Resource Usage section. These can be used to identify bottlenecks.

Common Resource Bottleneck Indicators

Jobs taking longer than expected to complete
Builds failing due to resource limits (e.g., OOMKilled)
High number of queued jobs in a workflow
Repeated use of circleci-cli to check job metrics

9. Balancing Speed vs Cost

Configuration	Description	Impact
jobs.<job_name>.resource_class	Defines the compute resources for a job (e.g., `medium`, `large`, `xlarge`)	Higher resource classes increase speed but also cost. NEW in v2.1
parallelism	Number of parallel containers to run a job	Improves speed but increases cost per job
executor	Defines the environment (e.g., `docker`, `machine`, `macos`)	Machine executor is faster but more expensive than Docker

Example: Resource Class Configuration

jobs:
  build:
    resource_class: large
    steps:
      - checkout
      - run: npm install

Speed vs Cost Trade-offs

Small resource class: Lower cost, slower builds
Large resource class: Higher cost, faster builds
Parallelism > 1: Faster execution, higher cost
Machine executor: Fastest, most expensive

Note: Use resource_class: small for quick checks or PRs to reduce cost.

Warning: Setting parallelism too high can lead to resource contention and increased cost.

10. Canceling Redundant Builds

Example: Configuring Build Cancellation

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
      - run: npm test

workflows:
  version: 2
  main:
    jobs:
      - build
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - main

Option	Values	Notes
workflows.triggers.schedule.filters.branches.only	`main`	Only run on specified branches
workflows.triggers.schedule.cron	`0 0 * * *`	Cron expression for scheduled runs

Note: Scheduled builds can be canceled if new commits arrive on the branch before the scheduled time.

Build Cancellation Rules

If a new commit is pushed to a branch with a scheduled workflow, the previous build will be canceled.
Builds are canceled only for workflows that have a schedule trigger.
Canceling builds helps reduce resource usage and costs.