Docker Engine API: Automating Container Management Programmatically

Every docker CLI command you run is just an HTTP request to the Docker Engine API. The CLI is a convenience wrapper, but the real power lies in the API itself. By using the API directly—through curl, Python, Go, or any HTTP client—you can build custom automation, monitoring tools, deployment pipelines, and management dashboards that go far beyond what the CLI offers.

This guide covers the Docker Engine API from first principles: how to enable and secure it, the key REST endpoints, and practical examples using both curl and the Python SDK.

Understanding the API Architecture

The Docker Engine API is a RESTful HTTP API that uses JSON for request and response bodies. By default, it listens on a Unix socket:

# The default Docker socket
/var/run/docker.sock

# The CLI is just an HTTP client
# This docker command:
docker ps

# Is equivalent to this API call:
curl --unix-socket /var/run/docker.sock http://localhost/v1.45/containers/json

The API is versioned. Always specify the API version in your requests to avoid breaking changes:

# Check the API version your daemon supports
curl --unix-socket /var/run/docker.sock http://localhost/version | jq
# {
#   "ApiVersion": "1.45",
#   "MinAPIVersion": "1.24",
#   "Version": "26.1.0",
#   ...
# }

Enabling the TCP API

To access the API remotely (not just via Unix socket), you need to enable TCP listening:

# /etc/docker/daemon.json
{
  "hosts": ["unix:///var/run/docker.sock", "tcp://0.0.0.0:2376"],
  "tls": true,
  "tlscacert": "/etc/docker/certs/ca.pem",
  "tlscert": "/etc/docker/certs/server-cert.pem",
  "tlskey": "/etc/docker/certs/server-key.pem",
  "tlsverify": true
}

Generating TLS Certificates

#!/bin/bash
# generate-docker-certs.sh
set -euo pipefail

CERT_DIR="/etc/docker/certs"
HOST="docker-host.example.com"
mkdir -p "$CERT_DIR"

# Generate CA key and certificate
openssl genrsa -out "$CERT_DIR/ca-key.pem" 4096
openssl req -new -x509 -days 365 -key "$CERT_DIR/ca-key.pem" \
  -sha256 -out "$CERT_DIR/ca.pem" -subj "/CN=Docker CA"

# Generate server key and certificate
openssl genrsa -out "$CERT_DIR/server-key.pem" 4096
openssl req -new -sha256 -key "$CERT_DIR/server-key.pem" \
  -subj "/CN=$HOST" -out "$CERT_DIR/server.csr"

echo "subjectAltName = DNS:$HOST,IP:192.168.1.10,IP:127.0.0.1" > extfile.cnf
echo "extendedKeyUsage = serverAuth" >> extfile.cnf

openssl x509 -req -days 365 -sha256 -in "$CERT_DIR/server.csr" \
  -CA "$CERT_DIR/ca.pem" -CAkey "$CERT_DIR/ca-key.pem" -CAcreateserial \
  -out "$CERT_DIR/server-cert.pem" -extfile extfile.cnf

# Generate client key and certificate
openssl genrsa -out "$CERT_DIR/client-key.pem" 4096
openssl req -new -key "$CERT_DIR/client-key.pem" \
  -subj "/CN=client" -out "$CERT_DIR/client.csr"

echo "extendedKeyUsage = clientAuth" > client-extfile.cnf
openssl x509 -req -days 365 -sha256 -in "$CERT_DIR/client.csr" \
  -CA "$CERT_DIR/ca.pem" -CAkey "$CERT_DIR/ca-key.pem" -CAcreateserial \
  -out "$CERT_DIR/client-cert.pem" -extfile client-extfile.cnf

# Set permissions
chmod 400 "$CERT_DIR"/*-key.pem
chmod 444 "$CERT_DIR"/ca.pem "$CERT_DIR"/*-cert.pem

rm -f extfile.cnf client-extfile.cnf *.csr *.srl

Core API Endpoints with curl

Container Operations

# List all running containers
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/containers/json" | jq

# List all containers (including stopped)
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/containers/json?all=true" | jq

# Create a container
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/containers/create?name=my-nginx" \
  -H "Content-Type: application/json" \
  -d '{
    "Image": "nginx:alpine",
    "ExposedPorts": {"80/tcp": {}},
    "HostConfig": {
      "PortBindings": {"80/tcp": [{"HostPort": "8080"}]},
      "Memory": 268435456,
      "CpuQuota": 50000
    }
  }' | jq
# Returns: {"Id": "abc123...", "Warnings": []}

# Start a container
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/containers/my-nginx/start"

# Stop a container (with 10s timeout)
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/containers/my-nginx/stop?t=10"

# Remove a container
curl -s --unix-socket /var/run/docker.sock \
  -X DELETE "http://localhost/v1.45/containers/my-nginx?force=true"

# Get container logs
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/containers/my-nginx/logs?stdout=true&stderr=true&tail=100"

# Inspect a container
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/containers/my-nginx/json" | jq

# Container stats (streaming)
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/containers/my-nginx/stats?stream=false" | jq

Image Operations

# List images
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/images/json" | jq

# Pull an image (streaming response)
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/images/create?fromImage=alpine&tag=latest"

# Inspect an image
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/images/nginx:alpine/json" | jq

# Remove an image
curl -s --unix-socket /var/run/docker.sock \
  -X DELETE "http://localhost/v1.45/images/nginx:alpine"

# Search Docker Hub
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/images/search?term=nginx&limit=5" | jq

Network and Volume Operations

# List networks
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/networks" | jq

# Create a network
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/networks/create" \
  -H "Content-Type: application/json" \
  -d '{"Name": "my-network", "Driver": "bridge"}' | jq

# List volumes
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/volumes" | jq

# Create a volume
curl -s --unix-socket /var/run/docker.sock \
  -X POST "http://localhost/v1.45/volumes/create" \
  -H "Content-Type: application/json" \
  -d '{"Name": "my-data", "Driver": "local"}' | jq

Event Streaming

The events endpoint provides a real-time stream of everything happening in Docker:

# Stream all events
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/events"

# Filter events by type and action
curl -s --unix-socket /var/run/docker.sock \
  "http://localhost/v1.45/events?filters=%7B%22type%22%3A%5B%22container%22%5D%2C%22event%22%3A%5B%22start%22%2C%22stop%22%2C%22die%22%5D%7D"

# Same with Python (much cleaner for filters)
# See Python section below

Python SDK Examples

The Docker SDK for Python provides a high-level interface that is much more practical than raw HTTP calls:

# Install the SDK
pip install docker

Basic Container Management

import docker

# Connect to Docker (uses DOCKER_HOST or default socket)
client = docker.from_env()

# List running containers
for container in client.containers.list():
    print(f"{container.name}: {container.status} ({container.image.tags})")

# Run a container
container = client.containers.run(
    "nginx:alpine",
    name="web-server",
    ports={"80/tcp": 8080},
    detach=True,
    mem_limit="256m",
    cpu_quota=50000,
    restart_policy={"Name": "unless-stopped"},
    labels={"app": "web", "env": "production"}
)
print(f"Started: {container.id[:12]}")

# Get container logs
logs = container.logs(tail=50, timestamps=True)
print(logs.decode("utf-8"))

# Get real-time stats
stats = container.stats(stream=False)
cpu_percent = calculate_cpu_percent(stats)
mem_usage = stats["memory_stats"]["usage"]
print(f"CPU: {cpu_percent:.1f}%, Memory: {mem_usage / 1024 / 1024:.1f}MB")

# Stop and remove
container.stop(timeout=10)
container.remove()


def calculate_cpu_percent(stats):
    """Calculate CPU usage percentage from Docker stats."""
    cpu_delta = (
        stats["cpu_stats"]["cpu_usage"]["total_usage"]
        - stats["precpu_stats"]["cpu_usage"]["total_usage"]
    )
    system_delta = (
        stats["cpu_stats"]["system_cpu_usage"]
        - stats["precpu_stats"]["system_cpu_usage"]
    )
    num_cpus = stats["cpu_stats"]["online_cpus"]
    if system_delta > 0 and cpu_delta > 0:
        return (cpu_delta / system_delta) * num_cpus * 100.0
    return 0.0

Image Management

import docker

client = docker.from_env()

# Pull an image
image = client.images.pull("python", tag="3.12-slim")
print(f"Pulled: {image.tags}")

# Build an image from a Dockerfile
image, build_logs = client.images.build(
    path="./app",
    tag="myapp:latest",
    rm=True,
    buildargs={"VERSION": "2.1.0"}
)
for chunk in build_logs:
    if "stream" in chunk:
        print(chunk["stream"], end="")

# Push to a registry
client.images.push("registry.example.com/myapp", tag="latest")

# Clean up unused images
pruned = client.images.prune(filters={"dangling": True})
print(f"Reclaimed: {pruned['SpaceReclaimed'] / 1024 / 1024:.1f}MB")

Event Monitoring

import docker
import json

client = docker.from_env()

# Real-time event monitoring
print("Watching Docker events (Ctrl+C to stop)...")
for event in client.events(decode=True):
    event_type = event.get("Type", "unknown")
    action = event.get("Action", "unknown")
    actor = event.get("Actor", {}).get("Attributes", {}).get("name", "unknown")
    time = event.get("time", 0)

    print(f"[{time}] {event_type}/{action}: {actor}")

# Filter for specific events
for event in client.events(
    decode=True,
    filters={
        "type": ["container"],
        "event": ["start", "stop", "die", "oom"]
    }
):
    name = event["Actor"]["Attributes"].get("name", "unknown")
    action = event["Action"]

    if action == "oom":
        print(f"WARNING: Container {name} was OOM killed!")
    elif action == "die":
        exit_code = event["Actor"]["Attributes"].get("exitCode", "?")
        print(f"Container {name} died with exit code {exit_code}")

Custom Health Check Dashboard

import docker
import time

client = docker.from_env()

def get_container_health():
    """Get health status of all containers."""
    containers = client.containers.list()
    report = []

    for c in containers:
        # Get health check status
        health = c.attrs.get("State", {}).get("Health", {})
        health_status = health.get("Status", "no healthcheck")

        # Get resource usage
        try:
            stats = c.stats(stream=False)
            mem_usage = stats["memory_stats"].get("usage", 0)
            mem_limit = stats["memory_stats"].get("limit", 1)
            mem_percent = (mem_usage / mem_limit) * 100
        except Exception:
            mem_percent = 0

        report.append({
            "name": c.name,
            "status": c.status,
            "health": health_status,
            "image": c.image.tags[0] if c.image.tags else "untagged",
            "memory_percent": round(mem_percent, 1),
            "restart_count": c.attrs["RestartCount"]
        })

    return report

# Print health report
while True:
    print("\n" + "=" * 60)
    print(f"{'Container':<20} {'Status':<12} {'Health':<12} {'Mem%':<8}")
    print("=" * 60)

    for c in get_container_health():
        print(f"{c['name']:<20} {c['status']:<12} {c['health']:<12} {c['memory_percent']:<8}")

    time.sleep(10)

Security Considerations

1. Never expose the Docker socket or API without authentication—the Docker API provides root-equivalent access
2. Use TLS with client certificates for any remote API access
3. Restrict socket access—only users in the docker group can access the Unix socket
4. Use API proxies like docker-socket-proxy to limit which API endpoints are accessible:

   # docker-socket-proxy limits API access
   docker run -d \
     --name socket-proxy \
     -v /var/run/docker.sock:/var/run/docker.sock:ro \
     -e CONTAINERS=1 \
     -e IMAGES=1 \
     -e NETWORKS=0 \
     -e VOLUMES=0 \
     -e POST=0 \
     -p 2375:2375 \
     tecnativa/docker-socket-proxy

5. Audit API access—log all API calls in production environments
6. Use read-only socket mounts (:ro) when containers only need to read Docker state

Conclusion

The Docker Engine API is the true interface to Docker. Mastering it enables you to build automation that perfectly fits your operational needs, from simple health check scripts to full-featured management platforms. Start with curl for exploration, graduate to the Python SDK for production automation, and always remember that the API provides the same level of access as running Docker commands as root—secure it accordingly.