Managing Self-Hosted Runners

1. Understanding Self-Hosted Runners

Self-hosted runners in CircleCI allow you to run builds on your own infrastructure instead of CircleCI's managed cloud runners. This provides greater control over the environment, security, and customization options.

Feature	Description	Notes
Runner Types	CircleCI supports Linux, macOS, and Windows runners.	Each OS requires specific runner setup and configuration.
Runner Registration	Runners must be registered with a CircleCI project or organization.	Registration includes generating a unique token for authentication.
Resource Allocation	Runners can be configured with specific CPU, memory, and disk resources.	Resource limits can be adjusted to match your infrastructure capabilities.
Security	Runners can be isolated in secure environments or VPCs.	Supports integration with CircleCI's security features like secrets management.
Scalability	Self-hosted runners can be scaled up or down based on demand.	Can be used in conjunction with CircleCI's cloud runners for hybrid workflows.

Example: Runner Registration

circleci runner register --name my-runner --token <your-token>

Note: Ensure that the runner has access to necessary dependencies and tools required for your builds.

Warning: Self-hosted runners must be maintained and updated to ensure compatibility with CircleCI's latest features.

2. Installing Runner Agent

Example: Install Runner Agent on Linux

# Download the CircleCI Runner binary
curl -o circleci-runner https://circleci.com/runner/download/latest/linux/amd64

# Make it executable
chmod +x circleci-runner

# Install the runner agent
./circleci-runner install --config-file /etc/circleci/runner-config.yaml

Option	Values	Description
--config-file	`/etc/circleci/runner-config.yaml`	Path to the configuration file for the runner agent
--log-level	`info`, `debug`, `warn`	Set the logging level for the runner agent
--runner-name	`my-runner`	Name of the runner instance (used in logs and UI)

Note: Ensure the runner agent has sufficient permissions to access required resources and network endpoints.

Warning: The --config-file must point to a valid YAML configuration file or the installation will fail.

3. Configuring Runner Agent

Property	Value	Description
runner.name	`"my-runner"`	Name of the runner as it appears in the CircleCI UI
runner.resource_class	`"medium"`	Resource class to use for jobs (e.g., `medium`, `large`, `xlarge`)
runner.tags	`["linux", "docker"]`	List of tags to filter jobs that can run on this runner
runner.capacity	`2`	Number of concurrent jobs this runner can execute

Example: Runner Configuration File

runner:
  name: "my-runner"
  resource_class: "medium"
  tags:
    - "linux"
    - "docker"
  capacity: 2

Note: The capacity setting determines how many jobs can run in parallel on this runner. Adjust based on available system resources.

4. Creating Custom Resource Classes

Property	Description	Example
resource_class.name	Name of the resource class	`custom-medium`
resource_class.cpu	Number of CPU cores	`2`
resource_class.memory	Memory in GB	`4`
resource_class.gpu	Boolean indicating GPU support	`false`

Example: Define Custom Resource Class

resource_classes:
  custom-medium:
    cpu: 2
    memory: 4
    gpu: false

Note: Resource classes must be defined at the project or organization level in CircleCI configuration.

5. Assigning Jobs to Runners

Property	Description	Values
runner.tags	Tags to match runners	`["linux", "ubuntu"]`
runner.name	Runner name to match	`"my-runner"`
runner.capacity	Number of jobs this runner can handle	`2`

Example: Assigning Jobs to Runners

jobs:
  build:
    machine:
      image: ubuntu-2004:2023.07.1
    resource_class: large
    steps:
      - checkout
      - run: echo "Building..."

Example: Using Tags to Select Runners

jobs:
  build:
    machine:
      image: ubuntu-2004:2023.07.1
      tags:
        - linux
        - ubuntu
    steps:
      - checkout
      - run: echo "Building on tagged runner..."

6. Using Docker on Runners

Property	Description	Example
docker.image	Specifies the Docker image to use for the job	`circleci/node:14`
docker.ports	Maps host ports to container ports	`[8080:8080]`
docker.volumes	Mounts volumes from the host into the container	`[/host/path:/container/path]`
docker.privileged	Enables privileged mode for the container	`true`

Example: Using Docker Image

jobs:
  build:
    docker:
      - image: circleci/node:14
        environment:
          NODE_ENV: test

Example: Mounting Volumes

jobs:
  build:
    docker:
      - image: circleci/python:3.9
        volumes:
          - /host/data:/container/data

Note: When using Docker on self-hosted runners, ensure that the runner has sufficient resources (CPU, memory) to handle containerized builds.

7. Monitoring Runner Health

Example: Health Check Endpoint

curl -X GET http://runner-host:8080/health

Endpoint	Description	Response
/health	Checks the overall health status of the runner	`{"status": "healthy"}`
/metrics	Exposes Prometheus-compatible metrics for monitoring	`{"runner": "active", "jobs": 5}`

Note: The /metrics endpoint requires Prometheus integration to be enabled.

Warning: If the runner is unhealthy, it will be automatically marked as DEPRECATED in the CircleCI UI.

Key Metrics to Monitor

Job Queue Length - Indicates runner load
CPU Usage - Should be under 80% for optimal performance
Memory Usage - Monitor for memory leaks or overuse
Job Completion Rate - Tracks success/failure ratio

8. Scaling Runner Infrastructure

Example: Auto-scaling Configuration

version: 2.1
orbs:
  circleci-runner: circleci/runner@1.0

jobs:
  build:
    machine:
      image: ubuntu-2004:2023.07.1
    steps:
      - checkout
      - run: echo "Building..."

workflows:
  version: 2
  build-workflow:
    jobs:
      - build

Configuration	Description	Values	Notes
runner.scale	Auto-scaling policy	`auto`, `manual`	Default: `manual`
runner.min_instances	Minimum runner instances	`1` to `100`	Default: `1`
runner.max_instances	Maximum runner instances	`1` to `1000`	Default: `10`
runner.cpu_cores	CPU cores per runner	`1`, `2`, `4`	Default: `2`
runner.memory_gb	Memory per runner (GB)	`2`, `4`, `8`	Default: `4`

Note: Scaling is managed via the CircleCI Runner API or CLI tools.

Warning: Ensure sufficient resource allocation to avoid overloading the infrastructure.

Scaling Strategies

Manual Scaling: Requires direct API or CLI interaction to add/remove runners.
Auto Scaling: Automatically adjusts based on queue length and resource usage.

9. Implementing Runner Security

Example: Configuring Runner Security Settings

setup:
  security:
    allowed_images:
      - "circleci/node:16"
      - "circleci/python:3.9"
    allowed_registries:
      - "docker.io"
      - "gcr.io"
    allowed_volumes:
      - "/tmp"
      - "/var/run/docker.sock"

Setting	Description	Default	Notes
allowed_images	List of Docker images allowed to run on the runner	None	Required for security compliance
allowed_registries	Allowed container registries for image pulls	All registries allowed	Restricts image sources to known locations
allowed_volumes	Mountable volumes for runner containers	None	Prevents access to sensitive host paths
disable_host_networking	Disables host network access for runners	false	Enforces isolated network environments

Note: Runner security settings must be defined in the config.yml file under the setup.security section.

Warning: Disabling host networking can break builds that require direct host access.

Security Best Practices

Always restrict allowed_images to official or verified images
Limit allowed_volumes to necessary paths only
Enable disable_host_networking for enhanced isolation
Use allowed_registries to prevent pulling from untrusted sources

10. Updating Runner Agent

Method	Description	Notes
circleci runner update	Updates the CircleCI Runner agent to the latest version	Requires runner to be running and accessible via CLI
circleci runner update --version <version>	Updates the CircleCI Runner agent to a specific version	Useful for maintaining consistency across environments

Example: Update Runner Agent to Latest Version

circleci runner update

Example: Update Runner Agent to Specific Version

circleci runner update --version 1.2.3

Note: Ensure that the target version is compatible with your CircleCI server version to avoid unexpected behavior.