Managing Workflows

1. Defining Workflows

Property	Description	Values	Notes
workflows.name	Name of the workflow	`string`	Required field
workflows.jobs	List of jobs to run in the workflow	`array`	Each job must be defined in the `jobs` section
workflows.triggers	Conditions to trigger the workflow	`array`	Can include `scheduled`, `push`, `pull_request`

Example: Basic Workflow Definition

workflows:
  version: 2
  build-and-test:
    jobs:
      - build
      - test

Example: Scheduled Workflow

workflows:
  version: 2
  nightly-build:
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - main
    jobs:
      - build

2. Adding Jobs to Workflows

Property	Description	Values
jobs	List of jobs to run in the workflow	`[job1, job2, ...]`
triggers	Conditions that trigger the workflow	`schedule, push, pull_request`

Example: Adding Jobs to Workflow

workflows:
  version: 2
  build-and-test:
    jobs:
      - build
      - test
      - deploy

3. Sequencing Jobs

Property	Description	Values
requires	Specifies which jobs must complete before this job runs	`job-name` or `[job-name, job-name]`
type	Defines the execution type of the job	`build`, `test`, `deploy`
matrix	Enables running the same job with different parameters	`{ version: [12, 14, 16] }`

Example: Basic Job Dependencies

jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - run: echo "Building..."
  test:
    requires:
      - build
    docker:
      - image: circleci/node:16
    steps:
      - run: echo "Testing..."
  deploy:
    requires:
      - test
    docker:
      - image: circleci/node:16
    steps:
      - run: echo "Deploying..."

Example: Matrix Job Configuration

jobs:
  build:
    machine:
      image: ubuntu-2004:202107-01
    parameters:
      node-version:
        type: string
        default: "16"
    steps:
      - run: echo "Building with Node ${{ parameters.node-version }}"

4. Running Jobs in Parallel

CircleCI allows you to run jobs in parallel to speed up your build process. This is particularly useful when you have multiple independent tasks that can be executed simultaneously.

Property	Description	Values
parallelism	Defines the number of parallel jobs to run	`1` to `20`

Example: Configuring Parallel Jobs

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:14
    parallelism: 4
    steps:
      - checkout
      - run: npm install
      - run: npm test

Note: Parallelism is only supported in CircleCI 2.0 and later versions.

Warning: Using too many parallel jobs can lead to resource contention and slower overall build times.

5. Configuring Workflow Parameters

Parameter	Description	Values	Default
name	Unique identifier for the workflow	`string`	None
triggers	Conditions that initiate the workflow	`array`	`["push"]`
parameters	Input parameters for the workflow	`object`	None

Example: Defining Workflow Parameters

workflows:
  build-and-test:
    parameters:
      environment:
        type: string
        default: production
      debug:
        type: boolean
        default: false
    jobs:
      - build:
          environment: <<parameters.environment>>
          debug: <<parameters.debug>>

Note: Parameters can be used to customize workflows for different environments or test scenarios.

6. Using Conditional Workflows

Property	Description	Values	Notes
when	Condition to evaluate for workflow execution	`failed`, `passed`, `always`, `not`	Can be used in job or workflow level
branches	Filter jobs by branch name	`only`, `ignore`	Supports regex patterns
tags	Filter jobs by tag name	`only`, `ignore`	Supports regex patterns

Example: Conditional Job Execution

jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - run: echo "Building..."
  test:
    docker:
      - image: circleci/node:16
    steps:
      - run: echo "Testing..."
workflows:
  version: 2
  build-and-test:
    jobs:
      - build
      - test:
          requires:
            - build
          when: passed

Example: Branch-Based Conditional Workflow

workflows:
  version: 2
  deploy-to-staging:
    jobs:
      - build
      - deploy-staging:
          when: passed
          branches:
            only: staging

Note: The when condition can be used at both job and workflow levels to control execution based on previous job status or other criteria.

7. Canceling Redundant Workflows

Option	Values	Notes
cancel	`true`	Cancel redundant workflows when a new build is created

Example: Canceling Redundant Workflows

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
      - run: npm test

workflows:
  version: 2
  main:
    jobs:
      - build:
          cancel: true

8. Using Multiple Workflows

CircleCI allows you to define multiple workflows in a single configuration file, enabling complex CI/CD processes with parallel and sequential execution.

Workflow	Description	Use Case
name	`string`	Unique identifier for the workflow
triggers	`object`	Defines when the workflow runs (e.g., `schedule`, `webhook`)
jobs	`array`	List of jobs to execute in this workflow

Example: Defining Multiple Workflows

version: 2.1
workflows:
  build-and-test:
    jobs:
      - build
      - test
  deploy:
    jobs:
      - deploy

Note: Workflows can be triggered manually or via scheduled events.

Key Features of Multiple Workflows

Parallel Execution: Jobs within a workflow can run in parallel.
Sequential Execution: Jobs can depend on each other in a sequence.
Triggering: Workflows can be triggered by branches, tags, or schedules.

9. Optimizing Workflow Graphs

Example: Basic Workflow with Parallel Jobs

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
  test:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm test
  deploy:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm run deploy

workflows:
  version: 2
  build-and-deploy:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test

Optimization Strategy	Description	Benefits
Parallel Jobs	Run independent jobs simultaneously using `requires` for dependencies	Reduces overall workflow time by running jobs concurrently
Job Dependencies	Use `requires` to define job execution order	Ensures proper sequence and avoids race conditions
Resource Class	Use `resource_class` to scale compute power	Improves performance for CPU-intensive tasks

Note: Parallel execution is limited by CircleCI's resource allocation. Jobs must be truly independent to run in parallel.

Example: Optimized Workflow with Resource Class

version: 2.1
jobs:
  build:
    resource_class: large
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
  test:
    resource_class: large
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm test

workflows:
  version: 2
  optimized-build:
    jobs:
      - build
      - test:
          requires:
            - build

Optimization Tips

Use resource_class: large for compute-intensive jobs
Minimize job dependencies to enable parallel execution
Avoid unnecessary steps in jobs to reduce runtime

10. Naming Workflows

Property	Value	Description
name	`string`	The name of the workflow as it appears in the CircleCI UI

Example: Defining a Workflow Name

workflows:
  version: 2.1
  build-and-test:
    jobs:
      - build
      - test