CI/CD with GitHub Actions: From Testing to Docker Deployment

GitHub Actions has become the default CI/CD platform for open-source and many commercial projects. Its tight integration with GitHub repositories, generous free tier, and marketplace of pre-built actions make it the fastest path from "code pushed" to "container deployed." But the gap between a working Hello World workflow and a production-grade pipeline with Docker builds, caching, secrets management, and multi-environment deployments is significant.

This guide covers the full journey: workflow syntax fundamentals, multi-stage pipelines, Docker image building and pushing, secrets management, matrix builds for multi-platform testing, caching strategies, self-hosted runners, and deployment to Docker-based infrastructure.

Workflow Syntax Fundamentals

Every GitHub Actions workflow lives in .github/workflows/ as a YAML file. Here is the anatomy of a complete workflow:

# .github/workflows/ci.yml
name: CI Pipeline

# Triggers
on:
  push:
    branches: [main, develop]
    paths-ignore:
      - '**.md'
      - 'docs/**'
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 6 * * 1'  # Weekly Monday 6 AM UTC
  workflow_dispatch:       # Manual trigger via UI
    inputs:
      environment:
        description: 'Target environment'
        required: true
        default: 'staging'
        type: choice
        options: [staging, production]

# Environment variables available to all jobs
env:
  GO_VERSION: '1.22'
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

# Permissions for GITHUB_TOKEN
permissions:
  contents: read
  packages: write
  security-events: write

# Jobs run in parallel by default
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - run: go test -race -coverprofile=coverage.out ./...
      - uses: actions/upload-artifact@v4
        with:
          name: coverage
          path: coverage.out

Multi-Stage Pipeline

A production pipeline typically has stages: lint, test, build, scan, and deploy. Use job dependencies to create the correct execution order:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - uses: golangci/golangci-lint-action@v4
        with:
          version: v1.57

  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: testpass
          POSTGRES_DB: testdb
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      redis:
        image: redis:7
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    env:
      DATABASE_URL: postgres://postgres:testpass@localhost:5432/testdb?sslmode=disable
      REDIS_URL: redis://localhost:6379
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: ${{ env.GO_VERSION }}
      - run: go test -race -cover -coverprofile=coverage.out ./...
      - run: go tool cover -func=coverage.out | tail -1

  build:
    needs: [lint, test]  # Only runs after lint and test pass
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/build-push-action@v5
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: |
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy-staging:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - name: Deploy to staging
        run: |
          echo "Deploying ${{ github.sha }} to staging..."
          # SSH deploy, Docker Swarm update, or API call

The services block in the test job spins up PostgreSQL and Redis as sidecar containers, giving tests access to real databases without mocking.

Docker Build and Push

The docker/build-push-action is the standard for building Docker images in GitHub Actions. Here is a complete configuration with multi-platform builds:

  build-and-push:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-qemu-action@v3  # For multi-platform builds
      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha,prefix=

      - uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
          build-args: |
            VERSION=${{ github.ref_name }}
            COMMIT=${{ github.sha }}

The docker/metadata-action automatically generates image tags based on the Git context: branch names, PR numbers, semver tags, and commit SHAs. This eliminates manual tag management.

Secrets Management

GitHub Actions provides several layers for managing secrets:

# Using secrets in workflows
steps:
  - name: Deploy
    env:
      SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
      API_TOKEN: ${{ secrets.API_TOKEN }}
    run: |
      # Secrets are masked in logs automatically
      echo "$SSH_PRIVATE_KEY" > /tmp/deploy_key
      chmod 600 /tmp/deploy_key
      ssh -i /tmp/deploy_key user@server "docker pull myapp:latest && docker compose up -d"

  - name: Deploy to production
    # Environment secrets with required reviewers
    environment:
      name: production
      url: https://myapp.com
    run: |
      # This step only runs after manual approval
      echo "Deploying to production..."

Matrix Builds

Matrix strategies run the same job across multiple configurations simultaneously:

  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false  # Don't cancel other jobs if one fails
      matrix:
        go-version: ['1.21', '1.22']
        postgres-version: ['15', '16']
        include:
          - go-version: '1.22'
            postgres-version: '16'
            coverage: true  # Only collect coverage for latest versions
        exclude:
          - go-version: '1.21'
            postgres-version: '15'  # Skip this combination
    services:
      postgres:
        image: postgres:${{ matrix.postgres-version }}
        env:
          POSTGRES_PASSWORD: test
        ports: ['5432:5432']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: ${{ matrix.go-version }}
      - run: go test -race ./...
      - if: matrix.coverage
        run: go test -coverprofile=coverage.out ./...

Caching Strategies

Caching dramatically reduces CI time. Different ecosystems need different caching approaches:

# Go dependency caching (automatic with setup-go v5)
- uses: actions/setup-go@v5
  with:
    go-version: '1.22'
    cache: true  # Default: caches ~/go/pkg/mod and ~/.cache/go-build

# Node.js caching
- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'

# Generic caching (for anything else)
- uses: actions/cache@v4
  with:
    path: |
      ~/.cache/pip
      ~/.local/share/virtualenvs
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

# Docker layer caching via Buildx
- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

Self-Hosted Runners

For builds that need more resources, specific hardware, or private network access, self-hosted runners are the answer:

# Run the GitHub Actions runner as a Docker container
docker run -d --restart always \
  --name github-runner \
  -e RUNNER_NAME="docker-builder" \
  -e GITHUB_PERSONAL_TOKEN="${GH_TOKEN}" \
  -e GITHUB_OWNER="myorg" \
  -e GITHUB_REPOSITORY="myrepo" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  myoung34/github-runner:latest

# Use self-hosted runner in workflow
jobs:
  build:
    runs-on: self-hosted
    # Or with labels:
    runs-on: [self-hosted, linux, x64, docker]
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t myapp .

Security note: Never use self-hosted runners for public repositories. Anyone who can submit a pull request can run arbitrary code on your runner. Self-hosted runners are for private repositories only.

Deployment to Docker Swarm

Deploy to Docker Swarm by updating the service image after a successful build:

  deploy:
    needs: build
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    environment: production
    steps:
      - name: Deploy to Docker Swarm
        uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.DEPLOY_HOST }}
          username: deploy
          key: ${{ secrets.DEPLOY_SSH_KEY }}
          script: |
            docker service update \
              --image ghcr.io/${{ github.repository }}:${{ github.sha }} \
              --update-parallelism 1 \
              --update-delay 30s \
              --update-failure-action rollback \
              myapp_web

            # Verify deployment
            sleep 10
            docker service ls | grep myapp_web
            curl -sf http://localhost:8080/health || exit 1

Reusable Workflows

Avoid duplicating workflow logic across repositories by creating reusable workflows:

# .github/workflows/reusable-docker-build.yml
name: Reusable Docker Build
on:
  workflow_call:
    inputs:
      image-name:
        required: true
        type: string
      dockerfile:
        required: false
        type: string
        default: './Dockerfile'
    secrets:
      registry-token:
        required: true

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.registry-token }}
      - uses: docker/build-push-action@v5
        with:
          context: .
          file: ${{ inputs.dockerfile }}
          push: true
          tags: ghcr.io/${{ inputs.image-name }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

# Consuming the reusable workflow from another repo
name: CI
on: push
jobs:
  build:
    uses: myorg/.github/.github/workflows/reusable-docker-build.yml@main
    with:
      image-name: myorg/myapp
    secrets:
      registry-token: ${{ secrets.GITHUB_TOKEN }}

Complete Production Pipeline

Here is a full pipeline combining everything: lint, test with services, build multi-platform Docker images, scan for vulnerabilities, and deploy with rollback:

name: Production Pipeline
on:
  push:
    branches: [main]
    tags: ['v*']
  pull_request:
    branches: [main]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with: { go-version: '1.22' }
      - run: go vet ./...
      - uses: golangci/golangci-lint-action@v4

  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env: { POSTGRES_PASSWORD: test, POSTGRES_DB: testdb }
        ports: ['5432:5432']
        options: --health-cmd pg_isready --health-interval 10s --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with: { go-version: '1.22' }
      - run: go test -race -coverprofile=coverage.out ./...
      - run: |
          coverage=$(go tool cover -func=coverage.out | tail -1 | awk '{print $3}' | tr -d '%')
          echo "Coverage: ${coverage}%"
          if (( $(echo "$coverage < 40" | bc -l) )); then
            echo "Coverage below 40% threshold"
            exit 1
          fi

  build:
    needs: [quality, test]
    runs-on: ubuntu-latest
    permissions: { contents: read, packages: write }
    outputs:
      image-digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=sha
            type=semver,pattern={{version}}
            type=raw,value=latest,enable={{is_default_branch}}
      - uses: docker/build-push-action@v5
        id: build
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Deploy and verify
        uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.DEPLOY_HOST }}
          username: deploy
          key: ${{ secrets.DEPLOY_SSH_KEY }}
          script: |
            docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
            docker compose -f /opt/app/docker-compose.yml up -d --no-deps app
            sleep 15
            curl -sf http://localhost:8080/health || {
              echo "Health check failed, rolling back..."
              docker compose -f /opt/app/docker-compose.yml rollback app
              exit 1
            }

When managing Docker deployments triggered by GitHub Actions, platforms like usulnet provide real-time visibility into which image version each container is running, making it easy to verify that deployments completed successfully and to track rollbacks across your infrastructure.

Pipeline Optimization Tips

1. Use paths-ignore filters - Do not run the full pipeline for README changes
2. Enable concurrency - Cancel in-progress runs when a new commit is pushed to the same branch
3. Cache aggressively - Dependencies, build artifacts, Docker layers
4. Use fail-fast: false in matrices - See all failures at once instead of aborting early
5. Split long jobs into parallel jobs - Run lint and test in parallel, then build after both pass
6. Pin action versions with SHA - Use actions/checkout@abcdef123 instead of @v4 for supply chain security

# Concurrency: cancel superseded runs
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true