Mastering GitLab CI/CD: Hands-On Pipeline Tutorial for Beginners

DevOps

YOUR COSMETIC CARE STARTS HERE

Find the Best Cosmetic Hospitals

Trusted • Curated • Easy

Looking for the right place for a cosmetic procedure? Explore top cosmetic hospitals in one place and choose with confidence.

“Small steps lead to big changes — today is a perfect day to begin.”

Explore Cosmetic Hospitals Compare hospitals, services & options quickly.

✓ Shortlist providers • ✓ Review options • ✓ Take the next step with confidence

1. Introduction

Modern software delivery demands speed, consistency, and absolute reliability, making automated build, test, and deployment cycles essential for engineering teams. For developers looking to master these critical competencies, structured learning paths from providers like DevOpsSchool offer valuable practical insights into industry-standard practices. This hands-on tutorial guides you through the complete process of building a robust CI/CD workflow using GitLab Pipelines—from configuring your first .gitlab-ci.yml file and executing automated tests to containerizing applications with Docker and deploying them securely.

2. Understand the GitLab CI/CD Building Blocks

Before writing any configuration files, it is essential to understand the core architectural components that make up GitLab CI/CD:

  • GitLab Repository: The central version control system storing your application code and configuration.
  • .gitlab-ci.yml: A YAML file placed at the root of your repository that defines the pipeline structure, stages, and jobs.
  • Pipeline: The top-level component that coordinates jobs. A pipeline consists of multiple sequential or parallel stages.
  • Stages: Logical groupings of operations (e.g., build, test, deploy) that execute in a defined sequence.
  • Jobs: Individual tasks executed within a stage (e.g., running unit tests or building a container).
  • GitLab Runner: An isolated application that picks up pipeline jobs, executes them, and sends the results back to GitLab.
  • Artifacts and Cache: Files generated by jobs that can be passed to subsequent stages (artifacts) or stored to speed up future runs (cache).
  • Variables and Secrets: Environment parameters and credentials injected securely into pipeline jobs.
  • Environments: Designated targets (such as staging or production) where applications are deployed and tracked.

3. Prepare the Development Environment

To successfully follow this tutorial, ensure you have the following prerequisites ready:

  • A active GitLab account and a project repository.
  • Git installed locally on your workstation.
  • A basic sample application (Node.js and Express).
  • Node.js and npm installed locally for initial development and verification.
  • Docker installed locally if you intend to test container builds.
  • A functioning GitLab Runner assigned to your project or instance.

GitLab-Hosted vs. Self-Managed Runners

GitLab provides shared, cloud-hosted runners (GitLab-hosted runners) out of the box, which require zero maintenance and work immediately for standard workloads. Alternatively, organizations often deploy self-managed runners on private infrastructure to gain control over hardware resources, network access, and specialized security requirements.

To check if your local environment and Git configuration are ready, open your terminal and verify your Git status:

Bash

git --version
node --version
npm --version

Expected result: Version numbers returned for Git, Node, and npm without errors.

4. Create the Application Repository

Let’s initialize our sample project structure. We will build a lightweight Node.js web application that serves a simple JSON API response and includes a basic test suite.

Project Directory Structure

Plaintext

my-node-app/
├── src/
│   └── app.js
├── test/
│   └── app.test.js
├── package.json
├── Dockerfile
└── .gitlab-ci.yml

Initializing and Pushing to GitLab

Initialize your local repository, commit the files, and push them to your newly created GitLab project repository using the following Git commands:

Bash

git init
git remote add origin https://gitlab.com/your-username/my-node-app.git
git add .
git commit -m "Initial commit with sample application structure"
git branch -M main
git push -u origin main
Code language: JavaScript (javascript)

Expected result: Code successfully uploaded to your remote GitLab repository.

5. Write Your First .gitlab-ci.yml File

The pipeline configuration file controls everything GitLab does when code is pushed. Create a file named .gitlab-ci.yml at the root of your repository.

Here is a beginner-friendly pipeline configuration:

YAML

stages:
  - validate
  - build
  - test

validate_job:
  stage: validate
  script:
    echo "Validating project files..."
    node -v

build_job:
  stage: build
  image: node:18-alpine
  script:
    - echo "Installing dependencies..."
    - npm install

test_job:
  stage: test
  image: node:18-alpine
  script:
    - echo "Running application tests..."
    - npm test
Code language: PHP (php)

Understanding the YAML Structure

  • stages: Defines the global execution order. Stages run sequentially; jobs within the same stage run in parallel.
  • job_name: Custom identifier for a specific task.
  • image: Specifies the Docker container image used to execute the job environment.
  • script: A list of shell commands executed sequentially by the runner.

Commit and push this file to GitLab, then navigate to your project’s Build > Pipelines menu in the GitLab UI to observe your first pipeline execution.

6. Expand the Pipeline Into a Complete CI Workflow

Now let’s expand our configuration into a production-grade Continuous Integration workflow. We will incorporate code validation, dependency installation, unit testing, and artifact management.

Enhanced Configuration

Update your .gitlab-ci.yml file with the following content:

YAML

stages:
  - validate
  - build
  - test

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/

lint_code:
  stage: validate
  image: node:18-alpine
  script:
    - echo "Checking code quality..."
    - npm run lint || echo "No linter configured, skipping."

build_application:
  stage: build
  image: node:18-alpine
  script:
    - echo "Installing production dependencies..."
    - npm ci
  artifacts:
    name: build-artifact-${CI_COMMIT_SHORT_SHA}
    expire_in: 7 days
    paths:
      - node_modules/
      - package-lock.json

run_unit_tests:
  stage: test
  image: node:18-alpine
  dependencies:
    - build_application
  script:
    - echo "Executing test suite..."
    - npm test
Code language: PHP (php)

Caching vs. Artifacts

  • Cache: Used to store downloaded dependencies (like node_modules/) between pipeline runs to speed up installation times. Caches are tied to branches or keys.
  • Artifacts: Files generated during a job (like compiled assets or build outputs) that are stored by GitLab and can be passed to subsequent stages or downloaded directly from the UI.

7. Configure and Understand GitLab Runners

Runners are the worker engines executing your .gitlab-ci.yml scripts. Understanding how they pick up jobs is vital for managing larger teams and complex infrastructures.

Runner Tags and Job Assignment

Runners can be configured with specific tags (e.g., linux, docker, gpu). Jobs can target specific runners using the tags keyword:

YAML

specialized_test_job:
  stage: test
  tags:
    - docker-high-memory
  script:
    - npm run heavy-test

Common Runner Failures and Investigation

  • Stuck Pipeline (“This job is stuck…”): Occurs when no available runner matches the required tags or project visibility settings. Check Settings > CI/CD > Runners in GitLab to ensure active runners are online.
  • Environment Discrepancies: If system commands fail inside a job image, verify that the runner executor (Docker, Shell, or Kubernetes) supports the required tools and user permissions.

8. Add Docker Image Building to the Pipeline

Containerizing applications ensures consistency across development, testing, and production environments. We can build and push Docker images directly inside our pipeline using the GitLab Container Registry.

Writing a Dockerfile

Create a Dockerfile at your project root:

Dockerfile

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY src/ ./src/
EXPOSE 3000
CMD ["node", "src/app.js"]
Code language: JavaScript (javascript)

Adding the Docker Build Stage

Update your .gitlab-ci.yml to include container building and pushing:

YAML

variables:
  DOCKER_DRIVER: overlay2

stages:
  - build
  - containerize

build_container:
  stage: containerize
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  script:
    - echo "Logging into GitLab Container Registry..."
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - echo "Building Docker image..."
    - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" -t "$CI_REGISTRY_IMAGE:latest" .
    - echo "Pushing image to registry..."
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
    - docker push "$CI_REGISTRY_IMAGE:latest"
Code language: PHP (php)

Security Considerations and Docker-in-Docker (DinD)

Using docker:dind requires privileged runner execution. In hardened production environments, alternative container build tools like Kaniko or Buildah are often preferred because they do not require a Docker daemon running in privileged mode. Always use built-in CI/CD variables (CI_REGISTRY_USER, CI_REGISTRY_PASSWORD) rather than hardcoding credentials.

9. Deploy the Application to a Test Environment

Once container images are safely stored in the registry, the next step is deployment. For this hands-on tutorial, we will simulate deployment to a staging server via a deployment script.

Adding a Deployment Job

Add the deployment stage to your .gitlab-ci.yml:

YAML

stages:
  - containerize
  - deploy

deploy_staging:
  stage: deploy
  image: alpine:latest
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - echo "Connecting to staging server..."
    - echo "Deploying image tag ${CI_COMMIT_SHA} to environment..."
    - echo "Deployment complete successfully."
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
Code language: PHP (php)

Secure Credential Handling

Never store SSH private keys or server passwords in plaintext files. Instead, navigate to Settings > CI/CD > Variables in your GitLab project and add sensitive parameters as masked and protected variables (e.g., STAGING_SSH_KEY, SERVER_IP).

10. Control Pipeline Execution With Rules and Conditions

Executing every job on every git push wastes time and computing resources. GitLab rules offer precise control over when jobs run.

Rule Examples

  • Run tests on all branches, but deploy only on main:

YAML

deploy_production:
  stage: deploy
  script:
    - echo "Deploying to production..."
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: manual
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: never
Code language: PHP (php)
  • when: manual: Pauses the pipeline and requires an authorized engineer to click a button in the GitLab UI before deployment proceeds.

11. Make the Pipeline More Efficient and Reliable

Optimize your pipeline execution speed and reliability with these proven techniques:

  • Parallelization: Split extensive test suites across multiple parallel jobs using parallel:.
  • Timeouts: Define explicit job timeouts to prevent stalled jobs from consuming runner limits indefinitely (timeout: 30m).
  • Retries: Handle transient network failures gracefully by configuring automatic retries:

YAML

 flaky_network_job:
   stage: test
   script:
     - npm run flaky-test
   retry:
     max: 2
     when:
       - runner_system_failure
  • Job Exclusions: Use workflow:rules to prevent duplicate pipelines from triggering simultaneously on branch pushes and merge requests.

12. Troubleshooting: Fix Common GitLab Pipeline Failures

Even experienced practitioners encounter pipeline issues. Here is how to diagnose and resolve common failures:

Issue SymptomRoot CausePractical Fix
Pipeline Stuck (“Job is stuck”)No available runners match the job’s tags or project settings.Register a runner or assign matching tags in project settings.
YAML Syntax ErrorIncorrect indentation or invalid syntax structure.Validate your file using the built-in CI/CD Linter in GitLab under Build > Pipelines > CI Lint.
Dependency Install FailureNetwork timeout or outdated package registry.Use a cached node_modules directory or check network access on self-hosted runners.
Docker Build Error (DinD)Docker daemon not running or missing service declaration.Ensure services: [docker:dind] is properly defined in the job configuration.
Registry Authentication ErrorExpired tokens or incorrect variable references.Use built-in CI_REGISTRY_USER and CI_REGISTRY_PASSWORD variables instead of custom tokens.
Deployment Permission DeniedMissing SSH keys or incorrect environment variables.Verify that required secret variables are added under project CI/CD settings.

13. Secure Your GitLab CI/CD Workflow

Security must be built into every layer of your CI/CD pipeline:

  • Protected Variables: Mark sensitive variables as protected so they are only exposed to pipelines running on protected branches or tags.
  • Masked Variables: Prevent sensitive tokens and passwords from appearing in plaintext pipeline logs.
  • Least-Privilege Access: Restrict production deployment permissions to senior engineers and maintainers.
  • Dependency Scanning: Integrate automated scanning tools to check for vulnerabilities in third-party packages before building images.
  • Review Third-Party Components: Carefully vet any community-provided CI/CD templates or include files before importing them into your workflow.

14. Final Project Review: Verify the Complete Pipeline

Let’s review the complete .gitlab-ci.yml configuration combining everything we have built:

YAML

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH'

stages:
  - validate
  - build
  - test
  - containerize
  - deploy

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/

lint_code:
  stage: validate
  image: node:18-alpine
  script:
    - echo "Validating code syntax..."

build_application:
  stage: build
  image: node:18-alpine
  script:
    - npm ci
  artifacts:
    expire_in: 3 days
    paths:
      - node_modules/

run_unit_tests:
  stage: test
  image: node:18-alpine
  dependencies:
    - build_application
  script:
    - npm test

build_container:
  stage: containerize
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'

deploy_staging:
  stage: deploy
  image: alpine:latest
  environment:
    name: staging
  script:
    - echo "Deploying container image to staging environment..."
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: manual
Code language: PHP (php)

Completion Checklist

  • [ ] Repository initialized and pushed to GitLab.
  • [ ] .gitlab-ci.yml validated via CI Lint.
  • [ ] Stages execute in logical sequential order.
  • [ ] Unit tests pass successfully.
  • [ ] Container image successfully pushed to the GitLab registry.
  • [ ] Deployment job executes correctly upon manual approval.

15. Next-Level Exercises for Learners

To further sharpen your GitLab CI/CD expertise, tackle these three practical exercises:

  1. Add Security Scanning: Integrate an automated static application security testing (SAST) stage into your pipeline to detect vulnerabilities prior to containerization.
  2. Multi-Environment Promotion: Configure separate staging and production deployment jobs with strict approval rules and environment protection tiers.
  3. Automated Rollback: Write a conditional rollback job that triggers automatically if downstream health checks fail immediately following a production deployment.

16. Conclusion: Turn Pipeline Knowledge Into Practical Skills

Building and maintaining robust CI/CD pipelines is a foundational skill for modern software delivery. Throughout this tutorial, you have learned how to initialize a repository, structure a .gitlab-ci.yml configuration, manage caching and artifacts, containerize applications with Docker, deploy to target environments, and troubleshoot common execution failures.

By continuing to practice these concepts in controlled environments, you can design fast, secure, and reliable DevOps pipelines tailored to any software project.

0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x