Troubleshooting Common Issues

1. Resolving Config Validation Errors

Error Type	Description	Resolution Steps
Invalid YAML syntax	CircleCI config fails to parse due to malformed YAML structure.	Validate YAML using an online validator or `circleci config validate`. Fix indentation and nesting issues. Ensure all keys are properly aligned and closed.
Missing or incorrect job name	Job names must be unique and follow naming conventions.	Verify job name exists in `jobs` section. Ensure no special characters or spaces in job names. Use lowercase and hyphens for naming (e.g., `build-and-test`).
Invalid orb reference	Orb version or name not found in CircleCI registry.	Check orb name and version for typos. Verify the orb exists using `circleci orb list`. Use a valid version (e.g., `1.2.3`) or latest tag.
Missing required keys	Config missing required fields like `version`, `jobs`, or `workflows`.	Add missing keys to config file. Ensure `version: 2.1` is present at root. Define at least one job and workflow to run.

Example: Valid Config Snippet

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout
      - run: npm install
workflows:
  version: 2
  main:
    jobs:
      - build

Note: Use circleci config validate locally to catch errors before pushing to CircleCI.

2. Fixing Checkout Failures

Issue	Solution	Notes
Checkout fails with "No such file or directory"	`checkout: path: ~/project`	Ensure the checkout path is correctly set in config
Checkout fails with "Permission denied"	`checkout: user: circleci`	Set the user to circleci for proper permissions
Checkout fails with "Timeout"	`checkout: timeout: 300`	Increase the checkout timeout to allow more time

Example: Setting Checkout Path

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:16
    steps:
      - checkout:
          path: ~/project

Note: Ensure the checkout path matches the expected directory structure in your project

Warning: Avoid using checkout: path: /tmp as it may cause permission issues

3. Debugging Cache Misses

Cache Key	Match Behavior	Notes
key	`cache key pattern`	Used to identify and retrieve cached artifacts
paths	`list of paths to cache`	Directories or files to include in the cache

Example: Cache Configuration

steps:
  - save_cache:
      key: v1-dependencies-{{ checksum "package-lock.json" }}
      paths: 
        - node_modules

Note: Cache keys are case-sensitive and must match exactly to retrieve a cached artifact.

Debugging Steps for Cache Misses

Verify cache key pattern matches between builds
Check that paths are correctly specified and exist
Confirm cache key includes all relevant files (e.g., package-lock.json)
Use circleci debug to inspect cache state in the job environment
Ensure no changes in dependencies or build environment affect cache key

Common Cache Key Patterns

v1-dependencies-{{ checksum "package-lock.json" }} - Caches based on lockfile
v1-{{ arch }}-{{ checksum "package-lock.json" }} - Includes architecture
v1-{{ .Branch }}-{{ checksum "package-lock.json" }} - Branch-specific caching

4. Resolving Dependency Issues

Issue	Description	Solution
Missing dependencies	Build fails due to missing packages or libraries.	Ensure all required dependencies are installed in the job using `apt-get`, `yum`, or package managers.
Conflicting versions	Multiple versions of the same dependency cause build errors.	Use `npm ls` or `yarn list` to identify conflicts and resolve via lock files or version pinning.
Cache invalidation	Dependency cache is outdated and causes build failures.	Clear the CircleCI dependency cache or use `circleci step cache` to manage it explicitly.

Example: Install missing dependencies

steps:
  - run:
      name: Install dependencies
      command: |
        sudo apt-get update
        sudo apt-get install -y libpq-dev build-essential

Example: Resolve version conflicts

dependencies:
  override:
    - npm install --prefer-distributed

Note: Use circleci config validate to check for configuration issues before running builds.

5. Fixing Docker Errors

Error Type	Description	Solution
Cannot connect to the Docker daemon	Docker service not running or permissions issue	Ensure Docker is running and user has `docker` group access. Run `sudo usermod -aG docker $USER` and relogin.
Image not found	CircleCI cannot pull the specified Docker image	Verify image name and tag. Use `docker pull <image>` locally to test. Ensure image is public or properly authenticated.
Build timeout due to Docker	Long-running Docker operations cause job timeouts	Optimize Dockerfile with multi-stage builds. Use `circleci/cli` to set `executors` with longer timeouts.
Permission denied in container	Container lacks permissions to access files or directories	Add `user: <uid>` to config or run container with `--user` flag. Ensure file permissions are correct in build context.

Example: Fixing Docker Image Pull Issues

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/node:18.17.0
        auth:
          username: $DOCKERHUB_USERNAME
          password: $DOCKERHUB_PASSWORD

Note: If using private Docker images, ensure credentials are stored securely in CircleCI context.

Warning: Avoid using latest tags in production. Pin to specific image versions for consistency.

6. Handling Out of Memory Errors

Error Type	Description	Common Causes	Solutions
OOMKilled	Process killed due to exceeding memory limits	Large build artifacts Infinite loops in scripts Memory leaks in tests	Use `circleci config process` to validate config Optimize build steps with `circleci local execute` Add memory limits to jobs using `resource_class`
Node.js Memory Limit	Node.js process exceeds memory limit set by CircleCI	Large Node.js project with many dependencies High memory usage in build scripts	Use `--max_old_space_size` in Node.js commands Increase job resource class to `large` or `xlarge` Implement caching to reduce recompilation

Example: Increasing Memory Limit in Node.js

node --max_old_space_size=4096 /path/to/script.js

Note: Memory limits in CircleCI are based on the resource class assigned to your job. The large resource class provides up to 4GB of memory, while xlarge offers up to 8GB.

Memory Management Best Practices

Use resource_class: large or xlarge for memory-intensive jobs
Monitor build logs for OOMKilled errors
Implement caching to avoid recompiling large dependencies
Avoid storing large artifacts in the workspace

7. Resolving Network Timeouts

Issue	Solution	Notes
Timeout during build	`circleci config validate`	Validate config for network issues
DNS resolution failure	Use `circleci local execute`	Test locally to isolate network issues
HTTP 504 Gateway Timeout	Retry with `circleci build`	May be transient; retry after 1-2 minutes

Example: Configuring Retry Logic

jobs:
  build:
    steps:
      - run:
          name: "Retry on Timeout"
          command: |
            for i in {1..3}; do
              if curl -f https://example.com/api; then
                break
              else
                echo "Attempt $i failed, retrying..."
                sleep 5
              fi
            done

Note: Network timeouts can be caused by external API rate limits or temporary connectivity issues.

Warning: Avoid excessive retries in build steps as they may lead to resource exhaustion.

8. Fixing Flaky Tests

Issue	Description	Solution	Notes
Non-deterministic test results	Tests pass or fail inconsistently across runs	Ensure tests are isolated and do not depend on external resources or timing	Use deterministic random seeds and mock external dependencies
Race conditions in tests	Tests fail due to concurrent execution or shared state	Use thread-safe operations and proper synchronization mechanisms	Implement retry logic or use CircleCI's parallelism features
Resource contention	Tests fail due to limited resources (e.g., DB connections, memory)	Optimize resource usage or increase resource limits	Use CircleCI's resource class settings to allocate more resources

Example: Adding retry logic to a test

const retry = require('async-retry');

test('flaky test with retry', async () => {
  await retry(
    async () => {
      // Test logic that may fail occasionally
      const result = await someAsyncOperation();
      expect(result).toBe('expected-value');
    },
    {
      retries: 3,
      minTimeout: 1000,
      maxTimeout: 5000
    }
  );
});

Note: Use CircleCI's retries configuration to allow builds to retry on failure.

Common Causes of Flaky Tests

External dependencies (APIs, databases) with inconsistent behavior
Timing issues due to asynchronous operations
Shared state between tests causing interference
Resource constraints in the test environment

Flaky Test Fix Approaches

Retry Logic: Retry failed tests up to a specified number of times
Isolation: Ensure each test runs in complete isolation
Mocking: Replace external dependencies with mocks or stubs
Resource Management: Allocate sufficient resources for tests to complete reliably

9. Debugging Workflow Filters

Example: Debugging a workflow filter

workflows:
  build-and-test:
    jobs:
      - build
      - test:
          filters:
            branches:
              only: main
            tags:
              only: /^v\d+\.\d+\.\d+$/

Filter Type	Configuration	Description
branches	`only: main`	Run job only on the `main` branch
branches	`ignore: develop`	Exclude the `develop` branch from running
tags	`only: /^v\d+\.\d+\.\d+$/`	Run job only on tags matching semantic versioning pattern
tags	`ignore: /^v\d+\.\d+\.\d+(-rc\.\d+)$/`	Exclude release candidate tags from running

Note: Filters are evaluated in order, and the first matching filter determines whether a job runs.

Warning: If no filter matches, the job will not run. Ensure that your filters cover all intended branches or tags.

Common Debugging Steps

Verify that the branch or tag name matches the filter pattern exactly.
Check for typos in the filter configuration (e.g., only: vs only:).
Use the CircleCI UI to inspect which branches and tags are triggering workflows.

10. Resolving Permission Errors

Issue	Description	Solution
Pipeline fails with "Permission denied"	CircleCI job fails due to insufficient permissions when accessing files or directories	Ensure the user running the job has appropriate file system permissions. Use `chmod` to adjust file permissions if needed.
SSH key access denied	CircleCI cannot access SSH keys due to incorrect permissions or key format	Verify that the private SSH key has `600` permissions and is in the correct format (e.g., PEM). Use `chmod 600 ~/.ssh/id_rsa`.
Read-only file system error	Job attempts to write to a read-only filesystem or directory	Check that the workspace or cache paths are not set to read-only. Adjust configuration in `config.yml` if necessary.

Example: Setting SSH key permissions

chmod 600 ~/.ssh/id_rsa

Note: Ensure that the SSH key is added to the CircleCI project settings under Project Settings > SSH Keys.

Warning: Never commit SSH keys or sensitive files to version control.