Docker BuildKit: Advanced Image Building Techniques

BuildKit is Docker's next-generation build engine, replacing the legacy builder with a dramatically faster, more secure, and more capable system. If you are still writing Dockerfiles without leveraging BuildKit features, you are leaving significant performance and security gains on the table. Builds that used to take minutes can finish in seconds with proper cache mounts. Secrets that used to leak into image layers can be handled safely. And multi-stage builds that used to execute sequentially can now run in parallel.

This guide covers every major BuildKit feature with practical examples you can start using today.

Enabling BuildKit

As of Docker 23.0, BuildKit is the default builder. For older versions, enable it explicitly:

# Option 1: Environment variable (per-command)
DOCKER_BUILDKIT=1 docker build -t myapp .

# Option 2: Docker daemon configuration (permanent)
# /etc/docker/daemon.json
{
  "features": {
    "buildkit": true
  }
}

# Option 3: Use docker buildx (recommended)
docker buildx build -t myapp .

# Verify BuildKit is active (look for "moby/buildkit" in output)
docker buildx version

Build Secrets

One of the most important BuildKit features is secure handling of build-time secrets. Before BuildKit, developers would COPY secrets into the image and then try to rm them—but they remained in previous layers. BuildKit secrets are never stored in the image:

# Dockerfile
# syntax=docker/dockerfile:1
FROM node:20-alpine

WORKDIR /app
COPY package*.json ./

# Mount the secret at build time - it's never stored in a layer
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci --production

COPY . .
CMD ["node", "server.js"]

# Build with the secret
docker buildx build \
  --secret id=npmrc,src=$HOME/.npmrc \
  -t myapp .

# Multiple secrets
docker buildx build \
  --secret id=npmrc,src=$HOME/.npmrc \
  --secret id=aws,src=$HOME/.aws/credentials \
  -t myapp .

The secret is mounted as a temporary file during the RUN instruction and is automatically removed afterward. It never appears in docker history or in any layer of the final image.

SSH Forwarding

For cloning private Git repositories during the build, BuildKit can forward your SSH agent instead of copying private keys:

# Dockerfile
# syntax=docker/dockerfile:1
FROM golang:1.22-alpine

RUN apk add --no-cache git openssh-client
RUN mkdir -p -m 0700 ~/.ssh && \
    ssh-keyscan github.com >> ~/.ssh/known_hosts

# Use SSH agent forwarding for private repos
RUN --mount=type=ssh \
    git clone [email protected]:myorg/private-lib.git /src/lib

WORKDIR /src/app
COPY . .
RUN go build -o /app ./cmd/server

FROM alpine:3.19
COPY --from=0 /app /usr/local/bin/app
CMD ["app"]

# Build with SSH forwarding
eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519
docker buildx build --ssh default -t myapp .

Cache Mounts

Cache mounts are arguably the single most impactful BuildKit feature for build performance. They persist a directory across builds, so package managers do not re-download everything on every build:

# Go application with module cache
FROM golang:1.22-alpine AS builder

WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -o /app ./cmd/server

# Node.js application with npm cache
FROM node:20-alpine AS builder

WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci

COPY . .
RUN npm run build

# Python application with pip cache
FROM python:3.12-slim AS builder

WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-cache-dir -r requirements.txt

COPY . .
RUN python -m compileall .

# apt-get with cache mount (avoid re-downloading packages)
FROM ubuntu:22.04

RUN --mount=type=cache,target=/var/cache/apt \
    --mount=type=cache,target=/var/lib/apt/lists \
    apt-get update && apt-get install -y \
    curl \
    git \
    build-essential

All Mount Types

BuildKit supports several mount types beyond secrets and caches:

Parallel Multi-Stage Builds

BuildKit analyzes the dependency graph of your Dockerfile stages and executes independent stages in parallel. This is a free performance win if you structure your Dockerfile correctly:

# syntax=docker/dockerfile:1
# These three stages run IN PARALLEL because they are independent

FROM node:20-alpine AS frontend
WORKDIR /app
COPY frontend/package*.json ./
RUN --mount=type=cache,target=/root/.npm npm ci
COPY frontend/ .
RUN npm run build

FROM golang:1.22-alpine AS backend
WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -o /server ./cmd/server

FROM golang:1.22-alpine AS migrations
WORKDIR /src
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -o /migrate ./cmd/migrate

# Final stage depends on all three - they run in parallel above
FROM alpine:3.19
RUN apk add --no-cache ca-certificates
COPY --from=frontend /app/dist /static
COPY --from=backend /server /usr/local/bin/server
COPY --from=migrations /migrate /usr/local/bin/migrate
EXPOSE 8080
CMD ["server"]

BuildKit detects that frontend, backend, and migrations stages have no dependency on each other and builds them simultaneously. The final stage only starts after all three complete.

Heredocs in Dockerfiles

BuildKit enables heredoc syntax, making multi-line scripts and configuration files much cleaner:

# syntax=docker/dockerfile:1
FROM python:3.12-slim

# Multi-line script without backslash line continuations
RUN <<EOF
apt-get update
apt-get install -y --no-install-recommends curl git
rm -rf /var/lib/apt/lists/*
EOF

# Inline file creation
COPY <<EOF /etc/nginx/conf.d/default.conf
server {
    listen 80;
    server_name _;
    location / {
        proxy_pass http://app:8080;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
    }
}
EOF

# Multiple heredocs in one RUN
RUN <<INSTALL && <<CONFIGURE
apt-get update
apt-get install -y nginx
INSTALL
nginx -t
systemctl enable nginx
CONFIGURE

Build Arguments and Conditional Logic

Combine build arguments with BuildKit for flexible, parameterized builds:

# syntax=docker/dockerfile:1
ARG GO_VERSION=1.22
ARG ALPINE_VERSION=3.19

FROM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS builder

ARG TARGETOS
ARG TARGETARCH
ARG VERSION=dev
ARG COMMIT=unknown

WORKDIR /src
COPY . .

RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build \
    -ldflags "-s -w -X main.version=${VERSION} -X main.commit=${COMMIT}" \
    -o /app ./cmd/server

FROM alpine:${ALPINE_VERSION}
COPY --from=builder /app /usr/local/bin/app
CMD ["app"]

# Build with custom arguments
docker buildx build \
  --build-arg GO_VERSION=1.22 \
  --build-arg VERSION=$(git describe --tags) \
  --build-arg COMMIT=$(git rev-parse --short HEAD) \
  -t myapp:$(git describe --tags) .

Build Cache Import/Export

One of BuildKit's killer features for CI/CD is the ability to export and import build caches. This makes CI builds as fast as local builds:

# Export cache to a registry (inline mode - stored in the image)
docker buildx build \
  --cache-to type=inline \
  -t registry.example.com/myapp:latest \
  --push .

# Import cache from the registry
docker buildx build \
  --cache-from type=registry,ref=registry.example.com/myapp:latest \
  -t registry.example.com/myapp:new-version \
  --push .

# Export/import with separate cache image (more flexible)
docker buildx build \
  --cache-to type=registry,ref=registry.example.com/myapp:cache,mode=max \
  --cache-from type=registry,ref=registry.example.com/myapp:cache \
  -t registry.example.com/myapp:latest \
  --push .

# Local directory cache (useful for CI systems with persistent storage)
docker buildx build \
  --cache-to type=local,dest=/tmp/buildcache \
  --cache-from type=local,src=/tmp/buildcache \
  -t myapp .

The mode=max option exports all layers (including intermediate stages), not just the final image layers. This maximizes cache hits for subsequent builds.

GitHub Actions CI Example

# .github/workflows/build.yml
name: Build and Push
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max
          build-args: |
            VERSION=${{ github.ref_name }}
            COMMIT=${{ github.sha }}

BuildKit Output Types

BuildKit can produce various output formats beyond just Docker images:

# Output as local files (extract build artifacts)
docker buildx build --output type=local,dest=./output .

# Output as a tar archive
docker buildx build --output type=tar,dest=./image.tar .

# Output as an OCI image layout
docker buildx build --output type=oci,dest=./oci-image.tar .

# Load into Docker daemon (default for single-platform)
docker buildx build --load -t myapp .

# Push directly to registry
docker buildx build --push -t registry.example.com/myapp:latest .

BuildKit Front-end Features

The # syntax= directive at the top of your Dockerfile selects the BuildKit frontend. Different frontends unlock different features:

# Use the latest stable Dockerfile frontend
# syntax=docker/dockerfile:1

# Use a specific version
# syntax=docker/dockerfile:1.7

# Use the labs channel for experimental features
# syntax=docker/dockerfile:1-labs

The labs channel includes experimental features that may change. Currently, this includes:

• Here-documents (now stable in 1.4+)
• COPY --parents for preserving directory structure
• COPY --chmod for setting permissions during copy

# syntax=docker/dockerfile:1
FROM alpine:3.19

# Copy with permissions (no separate chmod RUN needed)
COPY --chmod=755 scripts/ /usr/local/bin/

# Copy preserving parent directory structure
COPY --parents src/config/ /app/

Debugging BuildKit Builds

When builds fail, BuildKit provides better debugging tools than the legacy builder:

# Show build progress in plain text (see each step)
docker buildx build --progress=plain -t myapp .

# Export the build graph as JSON
docker buildx build --metadata-file metadata.json -t myapp .

# Interactive debugging with buildx debug (experimental)
docker buildx debug build -t myapp .

# Show detailed build timing
docker buildx build --progress=plain -t myapp . 2>&1 | grep "DONE"

For CI/CD pipelines, usulnet can track image builds and their associated metadata, helping teams understand build performance trends and cache hit rates across their Docker infrastructure.

Performance Comparison

Conclusion

BuildKit transforms Docker builds from a slow, sequential, and insecure process into a fast, parallel, and security-conscious workflow. The three features with the highest impact are cache mounts (for dependency caching), build secrets (for secure credential handling), and parallel multi-stage builds (for faster compilation). Add remote cache export/import for CI/CD, and your Docker builds will be both faster and more reliable across every environment.

Start by adding # syntax=docker/dockerfile:1 to the top of your Dockerfiles and converting your RUN instructions to use cache mounts. The improvement is immediate and requires minimal refactoring.