Turning Your App into Separate Containers for Better CI/CD

This article is inspired by a post originally published on blog.codeship.com. The analysis and explanations below are independently written for a DevOps engineering audience.

The Build Time Problem

Twenty-minute CI pipelines are not uncommon. A project with thousands of unit tests, heavy dependency installation, and multiple compilation steps can easily push past that mark. Multiply that by dozens of code pushes per day across a team, and your developers spend a significant portion of their working hours waiting — waiting to know whether their change is safe to ship.

The traditional approach of treating an application as a single, monolithic build unit is often the root cause. When every push rebuilds and retests everything, you lose the ability to run things in parallel, reuse cached layers intelligently, or isolate failures quickly. Containerization — and more specifically, decomposing your application into separate, purpose-built containers — is one of the most effective architectural changes you can make to reclaim that lost time.

Why Separate Containers Improve CI/CD

When your entire application lives in a single container (or no container at all), the CI system must rebuild and retest the whole thing on every push. Splitting concerns into distinct containers gives you several concrete advantages:

• Parallel execution: A CI orchestrator can build and test your API container, your worker container, and your frontend container simultaneously rather than sequentially.
• Layer caching: Docker caches image layers. If your dependencies haven’t changed, the layer containing npm install or pip install is reused instantly. Separate containers mean smaller, more focused layer trees that cache more reliably.
• Faster feedback: A failure in one container doesn’t block feedback on the others. Developers learn exactly which service broke without waiting for the full suite.
• Independent deployments: When only the worker service changes, only the worker image needs to be built, tested, and pushed — not the entire application.
• Reproducibility: Each container encapsulates its own runtime, dependencies, and configuration. What passes in CI is exactly what runs in production.

How to Decompose Your Application into Containers

Effective decomposition follows the same principles as good service design. Each container should have a single, well-defined responsibility. Here is a practical approach:

1. Identify Natural Service Boundaries

Start by listing the distinct runtime roles in your application. Common examples include:

• Web server / API layer (e.g., Rails, Django, Express)
• Background job workers (e.g., Sidekiq, Celery)
• Scheduled tasks / cron jobs
• Frontend asset build (e.g., webpack, Vite)
• Database (Postgres, MySQL)
• Cache / message broker (Redis, RabbitMQ)

Each of these is a candidate for its own container. You do not have to move all of them at once — start with the service that takes the longest to build or test and containerize that first.

2. Write Focused Dockerfiles

A common mistake is writing one large Dockerfile that installs every dependency the application could ever need. Instead, write targeted Dockerfiles for each service. Order instructions from least-frequently-changed to most-frequently-changed to maximise layer cache hits.

# Example: API service Dockerfile
FROM node:20-alpine

WORKDIR /app

# Copy dependency manifests first — this layer is cached until package.json changes
COPY package.json package-lock.json ./
RUN npm ci --production

# Copy application source last — this layer changes on every push
COPY src/ ./src/

CMD ["node", "src/server.js"]Code language: PHP (php)

3. Use Docker Compose to Orchestrate Locally and in CI

Docker Compose (or its CI equivalents) lets you declare all of your services, their build contexts, and their dependencies in a single docker-compose.yml file. Your CI pipeline can then bring up the full environment, run tests against each service, and tear it all down — with confidence that the environment matches production.

version: "3.9"
services:
  api:
    build:
      context: .
      dockerfile: docker/api/Dockerfile
    depends_on:
      - db
      - cache
    environment:
      DATABASE_URL: postgres://user:pass@db:5432/myapp

  worker:
    build:
      context: .
      dockerfile: docker/worker/Dockerfile
    depends_on:
      - db
      - cache

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: pass

  cache:
    image: redis:7-alpineCode language: JavaScript (javascript)

4. Run Tests per Container in Parallel

Most modern CI platforms (GitHub Actions, GitLab CI, CircleCI, and others) support parallel job execution. Structure your pipeline so that each container’s test suite runs as a separate, independent job. Your integration tests that require all services can run as a final stage once all unit test jobs have passed.

# Example GitHub Actions strategy matrix
jobs:
  test:
    strategy:
      matrix:
        service: [api, worker, frontend]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build and test ${{ matrix.service }}
        run: |
          docker build -f docker/${{ matrix.service }}/Dockerfile -t myapp-${{ matrix.service }} .
          docker run --rm myapp-${{ matrix.service }} npm testCode language: PHP (php)

Common Pitfalls to Avoid

• Shared code duplication: If multiple services share library code, consider a base image or a private package rather than copying code into each container’s build context.
• Bloated images: Install only what each service needs. Use multi-stage builds to separate build-time tools from runtime images.
• Environment parity gaps: Ensure the Docker Compose configuration used in CI matches the production deployment manifests as closely as possible. Drift between environments is a common source of “works on CI, fails in prod” bugs.
• Ignoring the .dockerignore file: Failing to exclude node_modules, .git, and other large directories from the build context slows every build unnecessarily.

The Compounding Returns of Container-Based CI/CD

The initial investment in restructuring your pipeline pays dividends over time. As your team grows, parallel container builds scale naturally. New services slot into the existing Compose and CI patterns without requiring rearchitecture. Developers can spin up isolated service environments locally with a single command, reducing the “it works on my machine” class of bugs.

Most importantly, faster feedback loops change developer behaviour positively. When a test run takes two minutes instead of twenty, developers run it more often, catch regressions earlier, and maintain higher confidence in the codebase.

Further Reading

The original article that inspired this post was published on blog.codeship.com. For deeper guidance on Docker layer caching strategies, multi-stage builds, and CI platform-specific configuration, consult the official Docker documentation and the documentation for your CI platform of choice.