Building with BuildKit
1. Enabling BuildKit
Example: Enable BuildKit in Dockerfile
# syntax=docker/dockerfile:1
FROM alpine:latest
RUN echo "BuildKit is enabled"
| Method | Description | Notes |
|---|---|---|
| DOCKER_BUILDKIT=1 | Environment variable to enable BuildKit | Set before running docker build |
| buildkit.toml | Configuration file for BuildKit | Located at ~/.docker/buildkit.toml |
Note: BuildKit is enabled by default in Docker 20.10 and later.
2. Using BuildKit Syntax
| Directive | Description | Example |
|---|---|---|
| FROM | Sets the base image for the build | FROM --platform=linux/amd64 ubuntu:20.04 |
| ARG | Defines a build-time variable | ARG VERSION=latest |
| ENV | Sets environment variables | ENV NODE_ENV=production |
| RUN | Executes commands during build | RUN --mount=type=cache,target=/var/cache/apt apt-get update |
| COPY | Copies files from host to container | COPY --chown=1000:1000 . /app |
| ADD | Similar to COPY but supports remote URLs and tar extraction | ADD https://example.com/file.tar.gz /tmp/ |
| WORKDIR | Sets the working directory for subsequent instructions | WORKDIR /app |
| EXPOSE | Documents which ports the application listens on | EXPOSE 8080 |
| CMD | Provides default command to run when container starts | CMD ["npm", "start"] |
| ENTRYPOINT | Sets the command to run when container starts | ENTRYPOINT ["./run.sh"] |
Example: Multi-stage build with BuildKit features
FROM --platform=linux/amd64 golang:1.21 AS builder
WORKDIR /app
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build go build -o main .
FROM --platform=linux/amd64 alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/main .
CMD ["./main"]
Note: BuildKit syntax supports advanced features like
--mount for caching and multi-platform builds using --platform.
Warning: Ensure that all
--platform values match the target architecture to avoid runtime issues.
3. Using Build Secrets
| Secret Source | Description | Usage |
|---|---|---|
| --secret | Pass a secret from the host filesystem to the build | --secret id=secret-name,src=/path/to/secret |
| --secret id=secret-name,env=ENV_VAR_NAME | Pass a secret from an environment variable to the build | --secret id=api-key,env=API_KEY |
Example: Using a file-based secret
docker build --secret id=ssh-key,src=/home/user/.ssh/id_rsa -t myapp .
Example: Using an environment variable as secret
export API_KEY=abc123
docker build --secret id=api-key,env=API_KEY -t myapp .
Note: Secrets are not stored in the image layer and are only available during the build process.
Warning: Ensure that secret files have appropriate permissions to prevent unauthorized access.
4. Mounting SSH Agent
| Option | Values | Description |
|---|---|---|
| --mount=type=ssh | --mount=type=ssh |
Mounts the SSH agent socket into the build container |
Note: The SSH agent must be running and accessible for this to work.
5. Implementing Build Mounts
| Mount Type | Description | Usage | Notes |
|---|---|---|---|
| bind | Mounts a file or directory from the host machine into the build context | --mount=type=bind,source=/host/path,target=/container/path |
Useful for sharing source code or configuration files |
| tmpfs | Mounts a temporary filesystem in memory for the build context | --mount=type=tmpfs,target=/tmp |
Fast, ephemeral storage; not persisted in final image |
| cache | Mounts a cache directory to speed up builds | --mount=type=cache,source=cache-dir,target=/app/cache |
Improves build performance by reusing cached data |
| secret | Mounts a secret file into the build context (e.g., API keys) | --mount=type=secret,source=secret-file,target=/run/secrets/my-secret |
Securely injects sensitive data without exposing it in the image |
| ssh | Mounts SSH agent socket to allow access to private repositories | --mount=type=ssh,target=/run/ssh-agent |
Enables git operations using SSH keys during build |
Example: Using bind mount to share source code
docker build --mount=type=bind,source=./src,target=/app/src -t my-app .
Example: Using cache mount for faster builds
docker build --mount=type=cache,source=npm-cache,target=/root/.npm -t my-app .
Note: Build mounts are only available when using BuildKit. Ensure
BUILDKIT=1 is set or use --mount flag in build commands.
6. Caching Build Layers
| Layer Type | Description | Caching Behavior | Notes |
|---|---|---|---|
| RUN | Executes commands in a new layer | Cache invalidated if command or files change | Use --mount=cache for persistent caches |
| COPY/ADD | Copies files from host to image | Cache invalidated if source files change | Use --from=... to reuse layers from other stages |
| ENV | Sets environment variables | Cache invalidated if value changes | Use ARG for build-time variables |
Example: Caching RUN Commands
FROM alpine:latest
RUN apk update && apk add curl
RUN curl -O https://example.com/file.txt
Note: BuildKit caches layers based on the command and file contents. Changing any part of the command or files will invalidate the cache.
7. Using Inline Cache
| Option | Values | Description |
|---|---|---|
| --cache-from | type=registry,ref=... |
Specifies external cache sources to use for build steps |
| --cache-to | type=registry,ref=... |
Specifies where to store cache after build |
Example: Using inline cache with Dockerfile
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["npm", "start"]
Example: Build with inline cache
docker build --cache-from type=registry,ref=myapp:latest-cache \
--cache-to type=registry,ref=myapp:latest-cache \
-t myapp .
Note: Inline cache requires a registry to store and retrieve cached layers.
8. Configuring Build Output
| Option | Values | Description |
|---|---|---|
| --output | type=local,dest=<path> |
Specify the output destination for the build artifacts |
| --output | type=docker |
Push the built image directly to Docker daemon |
| --output | type=image,name=<image-name> |
Save the image to the local registry |
| --output | type=registry,ref=<image-ref> |
Push the image to a remote registry |
| --output | type=oci,dest=<path> |
Save the image in OCI format to a directory |
Example: Push to remote registry
docker buildx build --output type=registry,ref=my-registry.com/myapp:latest .
Note: The
--output option supports multiple formats and destinations. You can combine them using comma-separated values.
Warning: When pushing to a registry, ensure you have proper authentication configured for the target registry.
9. Using Build Contexts
| Context | Description | Usage |
|---|---|---|
| build context | Directory containing files needed for building an image | Specified with --build-context or default . |
| context path | Path to the build context directory | Can be local or remote (e.g., Git repo) |
Example: Specifying a build context
docker build --build-context mycontext=/path/to/context -t myimage .
Note: Build contexts can be specified using the
--build-context flag to define named contexts with custom paths or sources.
Key Points
- Build context includes all files needed for the build process
- Can be a local directory or remote source (e.g., Git repository)
- Use
--build-contextto define custom context paths
10. Building for Multiple Platforms
| Platform | Architecture | OS | Notes |
|---|---|---|---|
| linux/amd64 | x86_64 |
Linux | Most common for servers |
| linux/arm64 | aarch64 |
Linux | Used in ARM-based systems like Raspberry Pi |
| linux/386 | i386 |
Linux | Legacy 32-bit systems |
| windows/amd64 | x86_64 |
Windows | For Windows containers |
| darwin/amd64 | x86_64 |
macOS | For macOS native builds |
| darwin/arm64 | aarch64 |
macOS | Apple Silicon (M1/M2) builds |
Example: Building for Multiple Platforms
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest .
Note: BuildKit must be enabled to use
--platform flag.
Warning: Ensure your application and dependencies are compatible with target platforms.
Key Points
- Use
--platformto build for multiple architectures - BuildKit is required for multi-platform builds
- Supports cross-compilation for ARM, x86, and more
11. Optimizing Layer Caching
| Directive | Description | Best Practice |
|---|---|---|
| FROM | Sets the base image for subsequent instructions | Pin to specific tags (e.g., node:18-alpine) to ensure reproducibility |
| COPY | Copies files from the host into the image | Use COPY --from=... to reuse layers from other builds |
| ADD | Similar to COPY but supports remote URLs and tar extraction | Avoid ADD unless needed; prefer COPY for clarity and control |
| RUN | Executes commands in a new layer | Chain multiple RUN commands with && to reduce layer count |
| WORKDIR | Sets the working directory for subsequent instructions | Use WORKDIR to avoid long paths in commands |
| ENV | Sets environment variables | Use ENV for values that change infrequently or are reused across layers |
Example: Layer Caching Optimization
FROM node:18-alpine
# Set working directory
WORKDIR /app
# Copy package files first to leverage layer caching
COPY package*.json ./
# Install dependencies (only re-run if package files change)
RUN npm ci --only=production
# Copy application code after installing dependencies
COPY . .
# Expose port and start app
EXPOSE 3000
CMD ["npm", "start"]Note: BuildKit uses a content-based hash to determine if a layer can be reused. Changes in any file or command will invalidate the cache and force rebuilding from that point.
12. Debugging Builds with Progress Output
| Flag | Description | Values |
|---|---|---|
| --progress | Controls the output format of build progress | auto, plain, tty |
Note: Using
--progress=plain provides verbose output useful for debugging build issues.