Docker Fundamentals: Complete Container Guide
Table of Contents
- Introduction to Docker
- Docker Architecture
- Images and Containers
- Dockerfile Best Practices
- Docker Networking
- Volume Management
- Docker Compose
- Security Considerations
- Performance Optimization
- Production Deployment
Introduction to Docker {#introduction}
Docker is a containerization platform that enables developers to package applications and their dependencies into lightweight, portable containers. These containers can run consistently across different environments, from development to production.
Why Docker?
- Consistency: "It works on my machine" problems eliminated
- Isolation: Applications run in isolated environments
- Portability: Containers run anywhere Docker is installed
- Scalability: Easy horizontal scaling of applications
- Resource Efficiency: Containers share OS kernel, using fewer resources than VMs
Key Concepts
# Container: Running instance of an image
docker run hello-world
# Image: Read-only template for creating containers
docker images
# Registry: Repository for storing and distributing images
docker push myapp:latest
# Volume: Persistent data storage
docker volume create mydata
Docker Architecture {#architecture}
Docker Engine Components
- Docker Daemon: Background service managing containers
- Docker CLI: Command-line interface for user interaction
- REST API: Interface between CLI and daemon
- containerd: High-level container runtime
- runc: Low-level container runtime
Container vs VM Architecture
Traditional VMs:
┌─────────────────┐ ┌─────────────────┐
│ Application │ │ Application │
├─────────────────┤ ├─────────────────┤
│ Guest OS │ │ Guest OS │
├─────────────────┤ ├─────────────────┤
│ Hypervisor │ │ Hypervisor │
├─────────────────┴─┴─────────────────┤
│ Host OS │
└─────────────────────────────────────┘
Docker Containers:
┌─────────────────┐ ┌─────────────────┐
│ Application │ │ Application │
├─────────────────┤ ├─────────────────┤
│ Docker Engine │ │ Docker Engine │
├─────────────────┴─┴─────────────────┤
│ Host OS │
└─────────────────────────────────────┘
Container Lifecycle
# Create container (without starting)
docker create --name myapp nginx:alpine
# Start existing container
docker start myapp
# Run (create and start in one command)
docker run -d --name webapp nginx:alpine
# Stop container
docker stop webapp
# Restart container
docker restart webapp
# Remove container
docker rm webapp
# Force remove running container
docker rm -f webapp
Images and Containers {#images-containers}
Working with Images
# Pull image from registry
docker pull nginx:1.21-alpine
# List local images
docker images
# Inspect image details
docker inspect nginx:1.21-alpine
# View image history
docker history nginx:1.21-alpine
# Remove image
docker rmi nginx:1.21-alpine
# Build image from Dockerfile
docker build -t myapp:v1.0 .
# Tag image
docker tag myapp:v1.0 myregistry.com/myapp:v1.0
# Push image to registry
docker push myregistry.com/myapp:v1.0
Container Operations
# Run container in detached mode
docker run -d --name webapp -p 8080:80 -e NODE_ENV=production -v /host/data:/app/data myapp:v1.0
# Execute command in running container
docker exec -it webapp bash
# View container logs
docker logs webapp
# Follow log output
docker logs -f webapp
# Copy files to/from container
docker cp ./config.json webapp:/app/config.json
docker cp webapp:/app/logs ./logs
# View container processes
docker top webapp
# Monitor container stats
docker stats webapp
Image Layers and Optimization
# View image layers
docker history --no-trunc myapp:v1.0
# Analyze image size
docker system df
# Remove unused images
docker image prune
# Remove all unused resources
docker system prune -a
Dockerfile Best Practices {#dockerfile}
Multi-stage Build Example
# Build stage
FROM node:16-alpine AS builder
WORKDIR /app
# Copy package files first (better caching)
COPY package*.json ./
RUN npm ci --only=production
# Copy source code
COPY src/ ./src/
RUN npm run build
# Production stage
FROM nginx:1.21-alpine AS production
# Create non-root user
RUN addgroup -g 1001 -S nodejs && adduser -S nextjs -u 1001
# Copy built application
COPY --from=builder --chown=nextjs:nodejs /app/dist /usr/share/nginx/html
# Copy custom nginx config
COPY nginx.conf /etc/nginx/nginx.conf
# Switch to non-root user
USER nextjs
# Expose port
EXPOSE 3000
# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 CMD curl -f http://localhost:3000/health || exit 1
# Start application
CMD ["nginx", "-g", "daemon off;"]
Python Application Dockerfile
FROM python:3.9-slim AS base
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
# Install system dependencies
RUN apt-get update && apt-get install -y build-essential curl && rm -rf /var/lib/apt/lists/*
# Create app user
RUN useradd --create-home --shell /bin/bash app
# Set work directory
WORKDIR /app
# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY --chown=app:app . .
# Switch to app user
USER app
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 CMD curl -f http://localhost:8000/health || exit 1
# Start application
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "app:app"]
Dockerfile Optimization Tips
# 1. Use specific base image tags
FROM node:16.14.2-alpine3.15
# 2. Minimize layers by combining RUN commands
RUN apt-get update && apt-get install -y curl git && apt-get clean && rm -rf /var/lib/apt/lists/*
# 3. Order layers by frequency of change
COPY package*.json ./ # Changes less frequently
RUN npm install
COPY . . # Changes more frequently
# 4. Use .dockerignore
# Create .dockerignore file:
node_modules
.git
.gitignore
README.md
Dockerfile
.dockerignore
# 5. Use multi-stage builds for smaller images
FROM golang:1.19-alpine AS builder
# ... build steps ...
FROM alpine:3.16
COPY --from=builder /app/binary /usr/local/bin/
Docker Networking {#networking}
Network Types
# List networks
docker network ls
# Create custom bridge network
docker network create --driver bridge --subnet=172.20.0.0/16 --ip-range=172.20.240.0/20 mynetwork
# Create overlay network (for swarm)
docker network create --driver overlay --attachable my-overlay
# Inspect network
docker network inspect mynetwork
# Connect container to network
docker network connect mynetwork webapp
# Disconnect container from network
docker network disconnect mynetwork webapp
Container Communication
# Run containers on same network
docker run -d --name database --network mynetwork postgres:13
docker run -d --name backend --network mynetwork -e DATABASE_URL=postgresql://user:pass@database:5432/db myapi:v1.0
# Containers can communicate using container names as hostnames
# backend can reach database at hostname "database"
Port Mapping
# Map container port to host port
docker run -p 8080:80 nginx # Host:Container
# Map to specific interface
docker run -p 127.0.0.1:8080:80 nginx
# Map random port
docker run -P nginx
# Multiple port mappings
docker run -p 80:80 -p 443:443 nginx
Network Security
# Create isolated network
docker network create --internal secure-network
# Run container with no network
docker run --network none alpine
# Limit container resources
docker run --memory=512m --cpus=1.5 myapp
Volume Management {#volumes}
Volume Types
# Named volumes (managed by Docker)
docker volume create mydata
docker run -v mydata:/app/data myapp
# Bind mounts (host directory)
docker run -v /host/path:/container/path myapp
# tmpfs mounts (temporary, in-memory)
docker run --tmpfs /tmp myapp
Volume Operations
# List volumes
docker volume ls
# Inspect volume
docker volume inspect mydata
# Remove volume
docker volume rm mydata
# Remove unused volumes
docker volume prune
# Backup volume data
docker run --rm -v mydata:/data -v $(pwd):/backup alpine tar czf /backup/backup.tar.gz -C /data .
# Restore volume data
docker run --rm -v mydata:/data -v $(pwd):/backup alpine tar xzf /backup/backup.tar.gz -C /data
Database Volume Example
# PostgreSQL with persistent data
docker run -d --name postgres -e POSTGRES_PASSWORD=secretpassword -e POSTGRES_DB=myapp -v postgres_data:/var/lib/postgresql/data -p 5432:5432 postgres:13
# MySQL with persistent data
docker run -d --name mysql -e MYSQL_ROOT_PASSWORD=rootpassword -e MYSQL_DATABASE=myapp -v mysql_data:/var/lib/mysql -p 3306:3306 mysql:8.0
Docker Compose {#compose}
Basic docker-compose.yml
version: '3.8'
services:
# Web application
web:
build:
context: .
dockerfile: Dockerfile
target: production
ports:
- "3000:3000"
environment:
- NODE_ENV=production
- DATABASE_URL=postgresql://user:password@db:5432/myapp
depends_on:
- db
- redis
networks:
- app-network
volumes:
- ./uploads:/app/uploads
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
interval: 30s
timeout: 10s
retries: 3
# Database
db:
image: postgres:13-alpine
environment:
- POSTGRES_DB=myapp
- POSTGRES_USER=user
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
volumes:
- postgres_data:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
networks:
- app-network
secrets:
- db_password
restart: unless-stopped
# Redis cache
redis:
image: redis:6-alpine
command: redis-server --requirepass ${REDIS_PASSWORD}
networks:
- app-network
volumes:
- redis_data:/data
restart: unless-stopped
# Nginx reverse proxy
nginx:
image: nginx:1.21-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./ssl:/etc/nginx/ssl:ro
depends_on:
- web
networks:
- app-network
restart: unless-stopped
volumes:
postgres_data:
redis_data:
networks:
app-network:
driver: bridge
secrets:
db_password:
file: ./secrets/db_password.txt
Development docker-compose.override.yml
version: '3.8'
services:
web:
build:
target: development
environment:
- NODE_ENV=development
- DEBUG=app:*
volumes:
- .:/app
- /app/node_modules
ports:
- "3000:3000"
- "9229:9229" # Debug port
command: npm run dev
db:
ports:
- "5432:5432" # Expose for development tools
redis:
ports:
- "6379:6379"
Compose Commands
# Start services
docker-compose up -d
# View logs
docker-compose logs -f web
# Scale services
docker-compose up -d --scale web=3
# Execute command in service
docker-compose exec web bash
# Stop services
docker-compose stop
# Remove services and networks
docker-compose down
# Remove with volumes
docker-compose down -v
# Build images
docker-compose build
# Pull latest images
docker-compose pull
Security Considerations {#security}
Image Security
# Use official base images
FROM node:16-alpine
# Keep base images updated
FROM ubuntu:20.04
RUN apt-get update && apt-get upgrade -y
# Don't run as root
RUN useradd -m -u 1001 appuser
USER appuser
# Use specific versions
FROM nginx:1.21.6-alpine
# Remove unnecessary packages
RUN apt-get remove -y build-essential && apt-get autoremove -y && apt-get clean
Runtime Security
# Run with limited privileges
docker run --user 1001:1001 myapp
# Limit resources
docker run --memory=512m --cpus=1.0 --pids-limit=100 myapp
# Read-only root filesystem
docker run --read-only myapp
# No new privileges
docker run --security-opt=no-new-privileges myapp
# Drop capabilities
docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE nginx
# Use security profiles
docker run --security-opt apparmor:docker-default myapp
Secrets Management
# Docker secrets (Swarm mode)
echo "mysecretpassword" | docker secret create db_password -
# Environment files
docker run --env-file .env myapp
# External secret management
docker run -e VAULT_ADDR=https://vault.company.com -e VAULT_TOKEN_FILE=/run/secrets/vault_token myapp
Network Security
# Custom bridge network
docker network create --driver bridge isolated-network
# Internal network (no external access)
docker network create --internal internal-network
# Encrypted overlay network
docker network create --driver overlay --opt encrypted secure-overlay
Performance Optimization {#performance}
Image Optimization
# Multi-stage build to reduce size
FROM node:16-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
FROM node:16-alpine AS runtime
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["node", "index.js"]
Container Resource Limits
# Memory limits
docker run -m 512m myapp
# CPU limits
docker run --cpus=1.5 myapp
# Combined limits
docker run --memory=1g --memory-swap=2g --cpus=2.0 --oom-kill-disable=false myapp
Monitoring and Logging
# Container stats
docker stats
# System information
docker system df
docker system events
# Log management
docker run --log-driver=json-file --log-opt max-size=10m --log-opt max-file=3 myapp
# Send logs to external system
docker run --log-driver=syslog --log-opt syslog-address=tcp://logserver:514 myapp
Performance Monitoring
# docker-compose.yml with monitoring
version: '3.8'
services:
app:
image: myapp:latest
prometheus:
image: prom/prometheus
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
image: grafana/grafana
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
Production Deployment {#production}
Docker Swarm Setup
# Initialize swarm
docker swarm init
# Add worker nodes
docker swarm join --token <token> <manager-ip>:2377
# Deploy stack
docker stack deploy -c docker-compose.yml myapp
# List services
docker service ls
# Scale service
docker service scale myapp_web=5
# Update service
docker service update --image myapp:v2.0 myapp_web
# Remove stack
docker stack rm myapp
Production docker-compose.yml
version: '3.8'
services:
web:
image: myregistry.com/myapp:${VERSION}
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
failure_action: rollback
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
resources:
limits:
memory: 512M
cpus: '0.5'
reservations:
memory: 256M
cpus: '0.25'
networks:
- web-network
secrets:
- app_secret
configs:
- source: app_config
target: /app/config.json
nginx:
image: nginx:1.21-alpine
deploy:
replicas: 2
placement:
constraints:
- node.role == manager
ports:
- "80:80"
- "443:443"
volumes:
- /etc/ssl/certs:/etc/ssl/certs:ro
configs:
- source: nginx_config
target: /etc/nginx/nginx.conf
networks:
web-network:
driver: overlay
attachable: true
secrets:
app_secret:
external: true
configs:
app_config:
external: true
nginx_config:
external: true
CI/CD Pipeline Example
# .github/workflows/deploy.yml
name: Deploy to Production
on:
push:
tags:
- 'v*'
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build Docker image
run: |
docker build -t myregistry.com/myapp:${{ github.ref_name }} .
- name: Run tests
run: |
docker run --rm myregistry.com/myapp:${{ github.ref_name }} npm test
- name: Push to registry
run: |
echo ${{ secrets.REGISTRY_PASSWORD }} | docker login myregistry.com -u ${{ secrets.REGISTRY_USERNAME }} --password-stdin
docker push myregistry.com/myapp:${{ github.ref_name }}
- name: Deploy to production
run: |
ssh production-server "
export VERSION=${{ github.ref_name }}
docker stack deploy -c docker-compose.yml myapp
"
Health Checks and Monitoring
# Container health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 CMD curl -f http://localhost:3000/health || exit 1
# Service health monitoring
docker service ps myapp_web
docker service logs myapp_web
# System monitoring
docker system df
docker system prune -a
Backup and Recovery
# Backup volumes
docker run --rm -v myapp_data:/data -v $(pwd):/backup alpine tar czf /backup/data-$(date +%Y%m%d).tar.gz -C /data .
# Database backup
docker exec postgres pg_dump -U user myapp > backup.sql
# Automated backup script
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
docker run --rm -v postgres_data:/data -v /backups:/backup alpine tar czf /backup/postgres_$DATE.tar.gz -C /data .
# Keep only last 7 days of backups
find /backups -name "postgres_*.tar.gz" -mtime +7 -delete
This comprehensive Docker guide covers everything from basic concepts to production deployment. Docker containers provide a powerful way to package, distribute, and run applications consistently across different environments. By following these best practices, you'll be able to build secure, efficient, and scalable containerized applications.
Remember to keep your images updated, follow security best practices, monitor your containers in production, and always test your deployment processes before rolling them out to production environments.