Example .gitlab-ci.yml with environment:

YAML

# .gitlab-ci.yml

stages:
  - build
  - deploy_review
  - stop_review
  - deploy_staging
  - deploy_production
  - cleanup_staging

default:
  image: alpine:latest

# A simple build job (not directly using 'environment')
build_app:
  stage: build
  script:
    - echo "--- Building Application ---"
    - mkdir -p my_app_build/
    - echo "version_1.0" > my_app_build/app.txt
    - echo "Build complete."
  artifacts:
    paths:
      - my_app_build/
    expire_in: 1 hour

# Job 1: Deploy to a dynamic Review App environment
deploy_review_app:
  stage: deploy_review
  script:
    - echo "--- Deploying Review App for branch: $CI_COMMIT_REF_SLUG ---"
    - echo "App version from build: $(cat my_app_build/app.txt)"
    - echo "Deployed to https://$CI_COMMIT_REF_SLUG.example.com" # Simulate deployment
    - sleep 10
  environment:
    name: review/$CI_COMMIT_REF_SLUG # Dynamic environment name based on branch
    url: https://$CI_COMMIT_REF_SLUG.example.com # Dynamic URL for the review app
    on_stop: stop_review_app # Specifies a job to run when this environment is "stopped"
    auto_stop_in: 2 hours # Automatically triggers 'stop_review_app' after 2 hours of inactivity/age
  rules:
    - if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH != "staging"' # Run for feature branches

# Job 2: Stops the Review App environment
stop_review_app:
  stage: stop_review
  script:
    - echo "--- Stopping Review App for branch: $CI_COMMIT_REF_SLUG ---"
    - echo "Cleaning up resources for https://$CI_COMMIT_REF_SLUG.example.com"
  environment:
    name: review/$CI_COMMIT_REF_SLUG # Must match the environment name to stop
    action: stop # This action marks the environment as stopped
  when: manual # Usually, stop jobs are manual or triggered by 'on_stop' or 'auto_stop_in'
  rules:
    - if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH && $CI_COMMIT_BRANCH != "staging"'

# Job 3: Deploy to Staging environment
deploy_to_staging:
  stage: deploy_staging
  script:
    - echo "--- Deploying to Staging Environment ---"
    - echo "App version from build: $(cat my_app_build/app.txt)"
    - echo "Deployed to https://staging.example.com"
    - sleep 10
  environment:
    name: staging
    url: https://staging.example.com
    # on_stop: stop_staging # You could define a 'stop_staging' job similarly if needed
  rules:
    - if: '$CI_COMMIT_BRANCH == "staging"' # Example: run only for 'staging' branch

# Job 4: Deploy to Production environment (often manual)
deploy_to_production:
  stage: deploy_production
  script:
    - echo "--- Deploying to Production Environment ---"
    - echo "App version from build: $(cat my_app_build/app.txt)"
    - echo "Deployed to https://production.example.com"
    - sleep 15
  environment:
    name: production
    url: https://production.example.com
  when: manual # Production deployments are often manual for control
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH' # Run only for the default branch (e.g., main)

Explanation:

1. environment: Keyword:
   • Used within a job’s definition to declare that this job performs a deployment to a specific environment.
   • It takes a map of sub-keys.
2. deploy_review_app Job (Dynamic Environment):
   • environment::
     • name: review/$CI_COMMIT_REF_SLUG:
       • (Required) This is the unique name of the environment. Using predefined CI/CD variables like $CI_COMMIT_REF_SLUG (the branch or tag name in a URL-friendly format) allows you to create dynamic environment names. This is very common for “Review Apps” where each feature branch gets its own temporary deployment environment.
     • url: https://$CI_COMMIT_REF_SLUG.example.com:
       • (Optional) A publicly accessible URL for this environment. GitLab will display this URL on the Environments page, merge requests, etc., making it easy to access the deployed application.
     • on_stop: stop_review_app:
       • (Optional) Specifies the name of another job (stop_review_app in this case) that should be run when this environment is “stopped.” An environment can be stopped manually from the GitLab UI, or automatically (e.g., when a branch is deleted, or via auto_stop_in).
     • auto_stop_in: 2 hours:
       • (Optional) If this environment is not re-deployed within “2 hours”, GitLab will automatically trigger the job specified in on_stop (if defined) or simply mark the environment as stopped. This is great for cleaning up temporary review environments.
   • rules:: This job is configured to run only for feature branches (not the default branch or staging), which is typical for review apps.
3. stop_review_app Job (Stopping an Environment):
   • environment::
     • name: review/$CI_COMMIT_REF_SLUG: This must match the name of the environment that this job is intended to stop.
     • action: stop: This tells GitLab that this job’s purpose is to stop the specified environment. When this job runs successfully, GitLab updates the environment’s status to “stopped.”
   • when: manual: Stop jobs are often manual, allowing users to decide when to tear down an environment. However, they can also be triggered automatically by on_stop from the deployment job or by branch deletion if configured.
4. deploy_to_staging and deploy_to_production Jobs (Static Environments):
   • These jobs deploy to well-known, static environments (staging and production).
   • name: staging / name: production: Static names.
   • url: Static URLs for these environments.
   • when: manual (for production): It’s a common safety measure to make production deployments a manual step, requiring explicit user action.

Key Concepts and Benefits:

• Deployment Tracking: GitLab keeps a record of all deployments to each environment. You can see this under Operate > Environments in your project. This includes when each deployment happened, by whom, and the commit SHA.
• Rollbacks and Re-deploys: From the Environments page, you can easily re-deploy a previous successful deployment to an environment (effectively a rollback if the previous deployment was an older version) or re-run the latest deployment job.
• Environment-Specific Variables: You can define CI/CD variables in GitLab (Settings > CI/CD > Variables) and scope them to specific environments. This allows you to have different configurations (e.g., database URLs, API keys) for staging vs. production without hardcoding them in your .gitlab-ci.yml.
• Protected Environments: Production environments can be “protected,” limiting who can deploy to them and which branches/tags can be deployed.
• Review Apps: Dynamic environments (like review/$CI_COMMIT_REF_SLUG) are a powerful feature for previewing changes from feature branches before merging them.
• Cleanup (on_stop, action: stop, auto_stop_in): These features help manage the lifecycle of temporary environments, ensuring resources are cleaned up.
• Visualizations: The environment URL is shown in merge requests if a review app is deployed, making it easy for reviewers to access and test the changes.

Using the environment keyword provides significant benefits for managing and visualizing your deployment processes, making your CI/CD pipeline more robust and transparent.

This allows you to:

• Designate specific runners for specific types of jobs (e.g., runners with Docker installed, runners on a particular OS like Windows or macOS, runners with access to specific hardware like GPUs, or runners dedicated to deployment tasks).
• Control resource allocation and job execution environments.

Example .gitlab-ci.yml with tags:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - deploy

default:
  image: alpine:latest # A global default image

# Job 1: Requires a runner with the 'docker' tag
build_with_docker_runner:
  stage: build
  tags:
    - docker # This job will only be picked up by a runner that has the 'docker' tag.
  script:
    - echo "--- Building with a Docker-enabled runner ---"
    - echo "This job expects the runner to have Docker capabilities."
    - docker info # Example command that would require Docker
    - echo "Build complete."

# Job 2: Requires a runner with BOTH 'linux' AND 'gpu' tags
gpu_accelerated_tests:
  stage: test
  tags:
    - linux   # Runner must have 'linux' tag
    - gpu     # AND runner must also have 'gpu' tag
  script:
    - echo "--- Running GPU Accelerated Tests ---"
    - echo "This job needs a Linux runner with GPU access."
    - echo "Simulating GPU-intensive computations..."
    - sleep 10
    - echo "GPU tests complete."

# Job 3: No specific tags defined at the job level
# This job can be picked up by:
# 1. Runners that are configured to run untagged jobs.
# 2. Shared runners (if available and configured for the project) that might not require specific tags.
# 3. If 'default:tags:' were defined, it would use those.
generic_script_job:
  stage: test
  # No 'tags:' keyword here
  script:
    - echo "--- Running Generic Script Job ---"
    - echo "This job has no specific tag requirements from the job definition."
    - echo "It can run on any available runner that is allowed to pick up untagged jobs for this project."
    - date

# Job 4: Deployment job, might target specific deployment runners
deploy_to_production_server:
  stage: deploy
  tags:
    - deploy   # Runner must have 'deploy' tag
    - production # AND runner must also have 'production' tag
  script:
    - echo "--- Deploying to Production ---"
    - echo "This job must run on a secure runner designated for production deployments."
    - echo "Deployment scripts would execute here."
    - sleep 5
    - echo "Deployment to production successful."

Explanation:

1. tags: Keyword:
   • Used within a job’s definition (e.g., build_with_docker_runner).
   • It takes an array of strings, where each string is a tag.
2. How Runners are Tagged (Background):
   • When a GitLab Runner is registered (by a GitLab administrator or a user setting up a specific runner), it can be assigned one or more tags (e.g., docker, linux, windows, macos, ruby-2.7, gpu, deploy-production).
   • These tags describe the capabilities or purpose of that runner.
3. build_with_docker_runner Job:
   • tags: - docker:
     • This job specifies that it requires a runner which has the tag docker.
     • Only runners that have been configured with the docker tag will be eligible to pick up and execute this job. This is useful if the job needs Docker to build images, run containers, etc.
4. gpu_accelerated_tests Job:
   • tags: - linux - gpu:
     • This job lists multiple tags. For a runner to pick up this job, it must possess ALL the tags listed by the job.
     • So, this job requires a runner that is tagged with linux AND also tagged with gpu. A runner tagged only with linux (but not gpu) would not pick it up.
5. generic_script_job Job:
   • This job does not have a tags: keyword. Its behavior depends on the runner configurations for your project/instance:
     • Untagged Runners: Runners can be configured to pick up jobs that don’t specify any tags (i.e., “run untagged jobs”). If such runners are available to your project, one of them might pick up this job.
     • Shared Runners: If your project has access to shared runners provided by your GitLab instance, their configuration determines if they pick up untagged jobs.
     • default:tags:: If you had a default:tags: section at the top level of your .gitlab-ci.yml, this job would inherit those default tags.
6. deploy_to_production_server Job:
   • tags: - deploy - production:
     • This illustrates a common use case for deploying to a specific environment. You would tag your production deployment runners with deploy and production to ensure only those trusted and specifically configured runners handle such critical tasks.

Key Concepts and Behavior:

• Purpose: To ensure that jobs are executed on runners that have the necessary capabilities, environment, or permissions.
• Matching Logic:
  • A job will only be executed by a runner if the runner has all the tags specified in the job’s tags list.
  • A runner can have more tags than a job specifies. As long as it has the ones the job requires, it’s a match. For example, a runner tagged [docker, linux, fast] can run a job with tags: [docker, linux].
• No Tags in Job: If a job has no tags defined, it can only be picked up by runners that are configured to run untagged jobs.
• Runner Configuration:
  • When registering a runner, you specify its tags.
  • You also configure whether a runner can pick up untagged jobs.
  • Runners can be “shared” (available to multiple projects), “group” (available to projects in a group), or “specific” (registered for a single project).
• default:tags:: You can set default tags for all jobs using the default: top-level keyword: YAMLdefault: tags: - common-runner Job-level tags will override these defaults.
• Order of Tags: The order of tags in the tags list in your job definition, or on the runner, does not matter.

Using tags effectively is crucial for managing a diverse set of CI/CD tasks that may require different operating systems, software dependencies, or security contexts. It gives you precise control over where your jobs run.

“Hidden” jobs are typically used as templates. Their names start with a dot (.), and they are not processed as regular jobs by GitLab CI unless explicitly extended.

Example .gitlab-ci.yml with extends:

# .gitlab-ci.yml

stages:
  - setup
  - test
  - build

default:
  image: alpine:latest # A global default image

# 1. Define a hidden job template for common test configurations
.base_test_config:
  stage: test
  image: ruby:3.0 # Specific image for test jobs
  tags:
    - docker # Default tags for these test jobs
  variables:
    TEST_SUITE_VERSION: "v1.2"
    RAILS_ENV: "test"
  before_script:
    - echo "--- Base Test Config: before_script ---"
    - echo "Setting up common test environment..."
    - bundle install --jobs $(nproc) --path vendor/bundle # Example common setup
  script: # This is a base script; jobs extending this can add more commands
    - echo "--- Base Test Config: script ---"
    - echo "Running common pre-test checks..."
  after_script:
    - echo "--- Base Test Config: after_script ---"
    - echo "Cleaning up common test resources."

# 2. Define another hidden job template, perhaps for a specific type of test
.rspec_test_template:
  extends: .base_test_config # This template itself extends another one
  variables:
    TEST_FRAMEWORK: "RSpec" # Overrides or adds to variables from .base_test_config
  before_script: # Commands here will run AFTER commands from .base_test_config's before_script
    - echo "--- RSpec Template: before_script ---"
    - echo "Setting up RSpec specific environment..."
  script: # Commands here will run AFTER commands from .base_test_config's script
    - echo "--- RSpec Template: script ---"
    - echo "Executing RSpec tests with framework $TEST_FRAMEWORK on Rails env $RAILS_ENV"
    # Placeholder for actual rspec command

# --- Regular Jobs using 'extends' ---

# Job 1: Extends the base test configuration
unit_tests:
  extends: .base_test_config
  # This job inherits: stage, image, tags, variables (TEST_SUITE_VERSION, RAILS_ENV),
  # before_script, script, and after_script from .base_test_config.
  script: # Commands here run AFTER the script block from .base_test_config
    - echo "--- Unit Tests: script ---"
    - echo "Running unit tests for version $TEST_SUITE_VERSION..."
    - echo "Actual unit test commands would go here."

# Job 2: Extends the RSpec template (which itself extends .base_test_config)
# It also overrides a variable and adds specific script commands.
feature_tests_rspec:
  extends: .rspec_test_template
  variables:
    RAILS_ENV: "test_feature" # Overrides RAILS_ENV from .base_test_config (via .rspec_test_template)
    SPEC_FILES: "spec/features"
  script: # Commands here run AFTER script blocks from .base_test_config AND .rspec_test_template
    - echo "--- Feature Tests (RSpec): script ---"
    - echo "Running RSpec feature tests from $SPEC_FILES on Rails env $RAILS_ENV"
    - echo "Test Suite version: $TEST_SUITE_VERSION, Framework: $TEST_FRAMEWORK"
    - bundle exec rspec $SPEC_FILES # Example actual command

# Job 3: Extends the base but completely overrides its 'script' and 'before_script'
# while keeping other things like image and variables.
custom_setup_test:
  extends: .base_test_config
  before_script: # This completely replaces .base_test_config's before_script
    - echo "--- Custom Setup Test: entirely new before_script ---"
    - echo "Performing a very different setup."
  script: # This completely replaces .base_test_config's script
    - echo "--- Custom Setup Test: entirely new script ---"
    - echo "Running tests with custom setup. RAILS_ENV is $RAILS_ENV."

# Job 4: Example of a non-test job, not using extends
# This job will use the global default image 'alpine:latest'
build_documentation:
  stage: build
  script:
    - echo "Building documentation..."
    - mkdir public
    - echo "Docs built" > public/index.html
  artifacts:
    paths:
      - public

Explanation:

1. Hidden Job Templates (. prefix):
   • Jobs like .base_test_config and .rspec_test_template start with a dot (.). This makes them “hidden” jobs.
   • Hidden jobs are not run directly as part of your pipeline. They serve as templates that other regular jobs can inherit from using the extends keyword.
2. extends: Keyword:
   • Used within a job definition to specify one or more template jobs whose configurations should be inherited.
   • extends: .base_test_config means the unit_tests job will inherit all configurations from .base_test_config.
   • extends: .rspec_test_template means feature_tests_rspec inherits from .rspec_test_template. Since .rspec_test_template itself extends .base_test_config, feature_tests_rspec effectively inherits from both, with .rspec_test_template‘s definitions taking precedence over .base_test_config where there are overlaps, and then feature_tests_rspec‘s own definitions taking precedence over both.
3. Merge Strategy (How Inheritance Works):
   GitLab performs a “deep merge” of the configurations. The specific job’s configurations always take precedence over what’s inherited.
   • Scalars (single values like image, stage): The value from the extending job (the most specific one) is used. If the extending job doesn’t define it, the value from the template is used.
   • Hashes (key-value pairs like variables): They are merged. If both the template and the extending job define the same variable key, the value from the extending job (the most specific one) wins. New variables from either are included.
     • Example: .base_test_config defines RAILS_ENV: "test". feature_tests_rspec defines RAILS_ENV: "test_feature". So, feature_tests_rspec will use test_feature.
   • Arrays (lists like script, before_script, after_script, tags):
     • script: Commands from the extended template’s script are executed before the commands in the extending job’s script. They are effectively prepended.
     • before_script: Commands from the extended template’s before_script are executed before the commands in the extending job’s before_script. They are prepended.
     • after_script: Commands from the extended template’s after_script are executed before the commands in the extending job’s after_script. They are prepended.
     • tags: The tags array from the extending job replaces the tags array from the template.
     • Important Exception: If an extending job redefines script, before_script, or after_script completely (as shown in custom_setup_test), the template’s corresponding script block is not prepended; the job’s definition entirely replaces it for that specific script block. To achieve prepending, the job must also include the script keywords from the template if it wants to re-define only parts, which extends handles implicitly for these script arrays if the job also has a script section. Correction based on typical CI behavior: For script, before_script, and after_script, GitLab CI prepends the inherited script content to the specific job’s script content. If you want to completely override, you’d typically need to ensure the template doesn’t have that script section, or you are actually replacing a more deeply nested script. However, the common understanding and use of extends with these script arrays is that the inherited commands run, then the specific job’s commands run. My example custom_setup_test reflects this prepending for before_script and script.
     • Self-correction for clarity: The most accurate way to describe script, before_script, and after_script merging with extends is: the commands from the extended configuration are executed, then the commands from the current job’s configuration for that same key are executed. They are effectively combined in sequence.
4. unit_tests Job:
   • Inherits stage: test, image: ruby:3.0, tags, variables, the before_script, the initial script part, and after_script from .base_test_config.
   • Its own script block adds more commands that run after the script block from .base_test_config.
5. feature_tests_rspec Job:
   • Inherits from .rspec_test_template.
   • .rspec_test_template first inherits from .base_test_config.
   • Then, .rspec_test_template‘s own definitions (like TEST_FRAMEWORK: "RSpec" and its before_script and script) are merged.
   • Finally, feature_tests_rspec‘s own definitions (like RAILS_ENV: "test_feature" and its script) are merged, taking highest precedence.
   • The before_script commands will run in order: .base_test_config‘s -> .rspec_test_template‘s.
   • The script commands will run in order: .base_test_config‘s -> .rspec_test_template‘s -> feature_tests_rspec‘s.
6. custom_setup_test Job:
   • This job extends .base_test_config but provides its own complete before_script and script definitions. In this case, for these specific keys, the definitions from .base_test_config are prepended to the definitions in custom_setup_test.
   • Correction/Clarification: If a job defines script and extends a template that also defines script, the inherited script runs, then the job’s script runs. The example above where I said it “replaces” was slightly off for these script arrays; the behavior is more like chaining.
7. Extending Multiple Templates:
   • You can extend multiple templates: extends: [.template_A, .template_B].
   • Configurations are merged from left to right. .template_B‘s values would override .template_A‘s if there are conflicts, and then the current job’s values override anything from the templates.

Benefits of extends:

• DRY (Don’t Repeat Yourself): Reduces a lot of boilerplate and repeated configuration.
• Consistency: Ensures common settings (like Docker images, setup scripts, tags) are applied consistently across similar jobs.
• Maintainability: If you need to change a common setting (e.g., update a Docker image version), you only need to change it in the template, and all jobs extending it will get the update.
• Readability: Job definitions become shorter and focus on what’s unique to them.

extends is a very powerful tool for organizing your GitLab CI/CD configurations, especially as your pipelines grow in complexity. It’s preferred over older YAML anchor/alias methods for CI configurations because extends performs a more intelligent “deep merge” suitable for CI job structures.

The trigger keyword allows a job in your main .gitlab-ci.yml (the parent pipeline) to initiate a new, separate pipeline (the child pipeline) based on another YAML file within the same project or even in a different project (multi-project pipelines).

Example Files:

You’ll need two YAML files in your repository for this example:

1. .gitlab-ci.yml (The main/parent pipeline configuration)
2. .child-pipeline.yml (The configuration for the child pipeline)

File 1: .gitlab-ci.yml (Parent Pipeline)

YAML

# .gitlab-ci.yml

stages:
  - setup
  - build_and_trigger
  - deploy # This stage might run jobs from the parent or child

default:
  image: alpine:latest

# A regular job in the parent pipeline
prepare_environment:
  stage: setup
  script:
    - echo "--- Parent Pipeline: Preparing Environment ---"
    - export PARENT_VARIABLE="Hello from Parent"
    - echo "Parent variable set."

# This job triggers a child pipeline defined in another YAML file
trigger_build_services_pipeline:
  stage: build_and_trigger
  # 'trigger' keyword starts the definition for a downstream pipeline
  trigger:
    include: '.child-pipeline.yml' # Path to the child pipeline's YAML configuration in THIS project
    strategy: depend # The trigger job will mirror the status of the child pipeline.
                     # If the child pipeline succeeds, this job succeeds. If it fails, this job fails.
    forward_pipeline_variables: true # Forwards predefined CI variables (like CI_COMMIT_REF_NAME) to the child.
    forward_yaml_variables: true # Forwards variables defined in this parent YAML (like PARENT_VARIABLE if it was global or job level)
                                  # Note: For variables defined in the 'script' like PARENT_VARIABLE above,
                                  # you'd typically pass them explicitly using 'trigger:variables:'.
  variables: # Explicitly pass variables to the child pipeline
    PASSED_FROM_PARENT: "This data comes from the trigger job"
    ANOTHER_PARENT_VAR: "Value_XYZ"
  script: # This script runs BEFORE the trigger actually happens.
    - echo "--- Parent Pipeline: About to trigger child pipeline for building services ---"
    - echo "This job will now trigger and wait for .child-pipeline.yml"

# Another job in the parent pipeline that might run after the trigger job (if strategy:depend is used)
# or in parallel if it's in the same stage and doesn't depend on the trigger job.
parent_final_step:
  stage: deploy
  needs: ["trigger_build_services_pipeline"] # Make this explicitly depend on the trigger job
  script:
    - echo "--- Parent Pipeline: Final Step ---"
    - echo "This runs after the child pipeline (triggered by 'trigger_build_services_pipeline') completes successfully."

File 2: .child-pipeline.yml (Child Pipeline Configuration)

YAML

# .child-pipeline.yml

# This file defines the jobs that will run in the child pipeline.
# It can have its own stages, jobs, variables, etc.

default:
  image: alpine:latest

stages:
  - build_child
  - test_child

variables:
  CHILD_PIPELINE_VAR: "Specific to Child Pipeline"

build_service_a:
  stage: build_child
  script:
    - echo "--- Child Pipeline: Building Service A ---"
    - echo "Accessing variable passed from parent: $PASSED_FROM_PARENT"
    - echo "Accessing another parent var: $ANOTHER_PARENT_VAR"
    - echo "Accessing child-specific variable: $CHILD_PIPELINE_VAR"
    - echo "Accessing forwarded predefined variable (e.g., branch): $CI_COMMIT_REF_NAME"
    - sleep 10
    - echo "Service A built."
  artifacts:
    paths:
      - service_a.txt
    expire_in: 1 hour

test_service_a:
  stage: test_child
  needs: ["build_service_a"] # Child pipeline can also use 'needs'
  script:
    - echo "--- Child Pipeline: Testing Service A ---"
    - if [ -f "service_a.txt" ]; then echo "Artifact from child build_service_a found."; else echo "Artifact NOT found!"; exit 1; fi
    - sleep 5
    - echo "Service A tests passed."

Explanation:

1. trigger: Keyword (in .gitlab-ci.yml):
   • Used within a job (trigger_build_services_pipeline in this case) to initiate a downstream (child) pipeline.
   • include: '.child-pipeline.yml':
     • This specifies that the child pipeline’s configuration is defined in the .child-pipeline.yml file located in the same project and same Git ref (branch/commit) as the parent pipeline. This is the core of a “parent-child pipeline.”
   • strategy: depend:
     • (Optional, but very common) This makes the parent trigger job (trigger_build_services_pipeline) wait for the entire child pipeline to complete.
     • The status of the trigger job will then mirror the status of the child pipeline. If the child pipeline succeeds, the trigger job succeeds. If the child pipeline fails, the trigger job fails.
     • Without strategy: depend (or if set to strategy: none), the trigger job would be marked as successful as soon as it successfully initiates the child pipeline, without waiting for it to finish.
   • forward_pipeline_variables: true:
     • (Optional) Forwards most of the predefined CI/CD variables (like CI_COMMIT_SHA, CI_COMMIT_REF_NAME, CI_PROJECT_DIR, etc.) from the parent pipeline to the child pipeline.
   • forward_yaml_variables: true:
     • (Optional) Forwards variables defined in the parent YAML (global variables: block or job-level variables: block of the trigger job) to the child pipeline.
     • Note: Variables set dynamically within a script block of the trigger job (like PARENT_VARIABLE in prepare_environment) are not automatically forwarded by this. For those, you need to pass them explicitly.
   • variables: (under trigger:):
     • This block allows you to explicitly pass new variables or override existing ones to the child pipeline.
     • PASSED_FROM_PARENT: "This data comes from the trigger job": This variable will be available in all jobs of the triggered child pipeline.
   • script: (in the trigger job):
     • Any commands in the script section of the trigger job are executed before the downstream pipeline is actually triggered. This can be used for setup or logging.
2. .child-pipeline.yml (Child Pipeline):
   • This is a standard GitLab CI/CD YAML file. It defines its own stages, jobs, variables, etc.
   • It runs in the context of the same project, commit, and Git ref as the parent pipeline that triggered it.
   • Accessing Variables: Jobs in the child pipeline (like build_service_a) can access:
     • Variables explicitly passed via trigger:variables: (e.g., $PASSED_FROM_PARENT).
     • Forwarded predefined variables (if forward_pipeline_variables: true).
     • Forwarded YAML variables from the parent (if forward_yaml_variables: true).
     • Its own globally or job-level defined variables (e.g., $CHILD_PIPELINE_VAR).
   • Artifacts in Child Pipeline: Jobs in the child pipeline can produce artifacts. These artifacts are scoped to the child pipeline. If the parent pipeline (or other pipelines) needs these, you would typically use needs: with project and job attributes to fetch artifacts from a completed child pipeline job (this is more of an advanced multi-project artifact passing scenario). For parent-child, if strategy: depend is used, the trigger job itself can have artifacts from the child if the child pipeline is configured to produce artifacts that are then “passed up” (this is a more nuanced behavior that depends on GitLab version and configuration). Usually, artifacts from the child are used within the child pipeline itself or by downstream multi-project pipelines.

Key Concepts and Benefits:

• Modularity: Break down a very large, monolithic .gitlab-ci.yml into smaller, more focused YAML files. This improves readability and maintainability.
• Reusability: Child pipeline configurations can potentially be reused or triggered by multiple parent pipelines or different trigger jobs.
• Dynamic Pipeline Generation: You can dynamically choose which child pipeline to trigger based on conditions in the parent.
• Isolation: Child pipelines can have their own stages and flow, providing some level of isolation.
• Multi-Project Pipelines (Brief Mention):
  • Instead of trigger:include:, you can use trigger:project: 'namespace/another-project' and trigger:branch: 'some-branch' to trigger a pipeline in a completely different GitLab project. This is useful for microservice architectures or when coordinating deployments across multiple repositories.

How it Looks in the UI:

• The trigger job in the parent pipeline will show a link to the downstream (child) pipeline.
• If strategy: depend is used, the trigger job will remain in a “running” state until the child pipeline completes, and then its status will reflect the child’s status.

Using trigger for parent-child pipelines is a great way to manage complexity as your CI/CD processes grow.

• Speeding up test suites: By splitting tests across several parallel jobs (test sharding/splitting).
• Building for multiple targets/architectures: Running the same build script with different configuration variables.
• Processing data in chunks: Dividing a large dataset or a list of items among parallel jobs.

GitLab provides two main ways to use parallel:

1. Fixed number of parallel jobs: parallel: <number>
2. Matrix of parallel jobs: parallel:matrix: (using variable combinations)

Example .gitlab-ci.yml with parallel:

YAML

# .gitlab-ci.yml

stages:
  - test
  - build

default:
  image: alpine:latest

# Job 1: Demonstrates a fixed number of parallel jobs for sharding tests
parallel_test_execution:
  stage: test
  parallel: 3 # Run this job 3 times in parallel
  script:
    - echo "--- Parallel Test Execution ---"
    # CI_NODE_INDEX is 1-based (e.g., 1, 2, 3)
    # CI_NODE_TOTAL is the total number of parallel jobs (e.g., 3)
    - echo "This is test job instance $CI_NODE_INDEX of $CI_NODE_TOTAL."
    - echo "Simulating test execution for a subset of tests..."
    # Example: Logic to select a subset of tests based on CI_NODE_INDEX
    - |
      if [ "$CI_NODE_INDEX" -eq 1 ]; then
        echo "Running test group 1 (e.g., unit tests part 1)"
        sleep 5
      elif [ "$CI_NODE_INDEX" -eq 2 ]; then
        echo "Running test group 2 (e.g., unit tests part 2)"
        sleep 7
      elif [ "$CI_NODE_INDEX" -eq 3 ]; then
        echo "Running test group 3 (e.g., integration tests part 1)"
        sleep 6
      fi
    - echo "Test instance $CI_NODE_INDEX finished."

# Job 2: Demonstrates a matrix of parallel jobs for different configurations
build_multi_target:
  stage: build
  script:
    - echo "--- Building for Target: $TARGET_OS, Arch: $TARGET_ARCH ---"
    - echo "This is job instance $CI_NODE_INDEX of $CI_NODE_TOTAL in the matrix."
    - echo "Fetching dependencies for $TARGET_OS-$TARGET_ARCH..."
    - sleep 10 # Simulate build time
    - echo "Build complete for $TARGET_OS-$TARGET_ARCH."
    - mkdir -p "build_output/${TARGET_OS}_${TARGET_ARCH}"
    - echo "binary_for_${TARGET_OS}_${TARGET_ARCH}" > "build_output/${TARGET_OS}_${TARGET_ARCH}/app"
  parallel:
    matrix:
      # First set of combinations
      - TARGET_OS: ["linux", "windows"]
        TARGET_ARCH: ["amd64"]
      # Second set of combinations (can have different variables or values)
      - TARGET_OS: ["darwin"]
        TARGET_ARCH: ["amd64", "arm64"]
      # A single specific combination
      - TARGET_OS: ["linux"]
        TARGET_ARCH: ["arm64"]
        EXTRA_FLAG: ["--optimized"] # You can add other variables
  artifacts:
    paths:
      - build_output/
    expire_in: 1 hour

Explanation:

1. parallel: Keyword:
   • Used within a job’s definition to specify that the job should be run multiple times in parallel.
2. parallel_test_execution Job (Fixed Number):
   • parallel: 3:
     • This tells GitLab to create and run 3 instances of the parallel_test_execution job concurrently (assuming enough runners are available).
     • In the GitLab UI, these jobs will appear as parallel_test_execution 1/3, parallel_test_execution 2/3, and parallel_test_execution 3/3.
   • Predefined CI/CD Variables:
     • $CI_NODE_INDEX: A 1-based index for the current parallel job instance (e.g., 1, 2, or 3 in this case).
     • $CI_NODE_TOTAL: The total number of parallel jobs in this set (e.g., 3 in this case).
   • Script Logic: The script uses $CI_NODE_INDEX to simulate running different groups of tests. This is a common pattern for test sharding, where a large test suite is divided among parallel jobs to reduce overall execution time.
3. build_multi_target Job (Matrix):
   • parallel:matrix:: This allows you to define a list of variable combinations. GitLab will create a separate parallel job instance for each unique combination.
   • Variable Combinations:
     • The first matrix entry: YAML- TARGET_OS: ["linux", "windows"] TARGET_ARCH: ["amd64"] This will generate 2 jobs:
       1. TARGET_OS=linux, TARGET_ARCH=amd64
       2. TARGET_OS=windows, TARGET_ARCH=amd64
     • The second matrix entry: YAML- TARGET_OS: ["darwin"] TARGET_ARCH: ["amd64", "arm64"] This will generate 2 more jobs:
       1. TARGET_OS=darwin, TARGET_ARCH=amd64
       2. TARGET_OS=darwin, TARGET_ARCH=arm64
     • The third matrix entry: YAML- TARGET_OS: ["linux"] TARGET_ARCH: ["arm64"] EXTRA_FLAG: ["--optimized"] This generates 1 job:
       1. TARGET_OS=linux, TARGET_ARCH=arm64, EXTRA_FLAG=--optimized
     • Total Jobs: In this example, the build_multi_target job with this matrix configuration would generate 2 + 2 + 1 = 5 parallel job instances. Each instance will have the specified environment variables (TARGET_OS, TARGET_ARCH, EXTRA_FLAG) set accordingly.
   • Script Logic: The script uses these environment variables (e.g., $TARGET_OS, $TARGET_ARCH) to perform actions specific to that combination, like building for a particular operating system and architecture.
   • Artifacts: Each parallel job instance can produce its own artifacts. The example shows creating artifacts specific to the build target.

Key Concepts and Behavior:

• Purpose: To significantly speed up pipelines by running identical or similar tasks concurrently or to build/test across a range of configurations.
• Runner Availability: The actual level of parallelism depends on the number of available GitLab Runners that can pick up these jobs. If you define parallel: 10 but only have 2 runners available, only 2 jobs will run at a time.
• Job Naming:
  • For parallel: <number>, jobs are named job_name X/Y.
  • For parallel:matrix:, jobs are typically named job_name: [VAR1:value1, VAR2:value2, ...] and also include the X/Y suffix based on their order in the expanded matrix.
• CI_NODE_INDEX and CI_NODE_TOTAL: These are crucial for dividing work among the parallel instances, especially when using parallel: <number>. For parallel:matrix:, you typically rely on the matrix variables themselves to differentiate the work.
• Resource Consumption: Using parallel can consume more CI/CD minutes and runner resources, so use it judiciously where the speed benefits outweigh the costs.
• Dependencies and Artifacts: Each parallel job instance is treated as a separate job. If other jobs need a parallelized job, they would typically need all instances to complete (unless you specifically need a particular instance, which is more advanced). Artifacts from parallel jobs are usually named uniquely or aggregated in a later stage.

Using parallel effectively can drastically improve the efficiency of your CI/CD pipelines, especially for time-consuming tasks like extensive test suites or multi-platform builds.

The allow_failure keyword in a GitLab CI/CD job definition specifies whether a pipeline should continue running and potentially be marked as successful even if that particular job fails. By default, if any job fails, the entire pipeline is marked as failed, and subsequent stages are typically not executed.

This is useful for:

• Non-critical jobs like code quality checks, linters, or experimental tests whose failure shouldn’t block the main deployment flow.
• Jobs that deploy to non-critical environments.
• Informational scripts.

Example .gitlab-ci.yml with allow_failure:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - deploy

default:
  image: alpine:latest

# Job 1: A standard build job that should succeed
build_application:
  stage: build
  script:
    - echo "--- Building Application ---"
    - sleep 5
    - echo "Application build successful."
    - mkdir -p app_binaries
    - echo "my_app_v1" > app_binaries/app

# Job 2: A critical test job. If this fails, the pipeline should reflect a problem.
critical_unit_tests:
  stage: test
  script:
    - echo "--- Running Critical Unit Tests ---"
    - sleep 8
    - echo "Critical unit tests PASSED."
    # To see a pipeline failure, uncomment the next line:
    # - echo "Simulating critical test failure!"; exit 1

# Job 3: A non-critical or "flaky" test job that is allowed to fail
# without stopping the pipeline or marking the pipeline as failed overall (if other critical jobs pass).
non_critical_lint_check:
  stage: test
  allow_failure: true # This is the key configuration
  script:
    - echo "--- Running Non-Critical Lint Check ---"
    - echo "Checking code style..."
    - sleep 3
    - echo "Lint check found some non-critical style issues and will 'fail'."
    - exit 1 # Simulate a failure for this non-critical job

# Job 4: Another non-critical job (e.g., an optional test suite)
# This also uses allow_failure and will demonstrate that the pipeline continues.
experimental_feature_test:
  stage: test
  allow_failure: true
  script:
    - echo "--- Testing Experimental Feature (allowed to fail) ---"
    - sleep 4
    - echo "Experimental feature test completed (simulating success for this one, but could fail)."
    - exit 0

# Job 5: Deployment job.
# This job will run if 'build_application' and 'critical_unit_tests' succeed,
# even if 'non_critical_lint_check' failed, because it was allowed to fail.
deploy_to_staging:
  stage: deploy
  script:
    - echo "--- Deploying to Staging ---"
    - echo "Checking if critical tests passed before deploying..."
    - if [ -f "app_binaries/app" ]; then echo "App binary found."; else echo "App binary NOT found! Build might have issues."; exit 1; fi
    - echo "Deploying application to staging environment..."
    - sleep 10
    - echo "Deployment to staging successful."
  # This job depends on the success of prior stages' critical jobs.
  # If 'critical_unit_tests' failed (and didn't have allow_failure: true),
  # this 'deploy_to_staging' job would not run.

Explanation:

1. allow_failure: Keyword:
   • Used within a job’s definition.
   • It accepts a boolean value: true or false.
   • allow_failure: true: If this job fails, the job itself will be marked as “failed” (often with a warning icon like orange/yellow in the UI), but it will not cause the entire pipeline to be marked as “failed”. Subsequent jobs in later stages (or parallel jobs that don’t depend on it) can still proceed as if this job had succeeded for the purpose of pipeline progression.
   • allow_failure: false (or omitting the keyword): This is the default behavior. If the job fails, the pipeline is marked as “failed”, and typically, jobs in subsequent stages will not run.
2. build_application Job:
   • A standard job that is expected to succeed. It does not use allow_failure, so if it failed, the pipeline would fail.
3. critical_unit_tests Job:
   • This job also does not use allow_failure: true. If this job were to fail (e.g., by uncommenting exit 1), the deploy_to_staging job would not run, and the pipeline would be marked as failed. This represents a critical part of your CI.
4. non_critical_lint_check Job:
   • allow_failure: true: This is the key demonstration.
   • The script is designed to exit 1, which means this job will fail.
   • However, because allow_failure is true:
     • The non_critical_lint_check job itself will show a failed status (often with a warning icon).
     • The overall pipeline can still proceed to the deploy stage and potentially be marked as “passed” or “passed with warnings” if all other critical jobs (like build_application and critical_unit_tests) succeed.
5. experimental_feature_test Job:
   • Also uses allow_failure: true. This job is written to succeed in this example, but if it were to fail, it wouldn’t stop the pipeline. This is typical for tests of features that are under development and not yet stable.
6. deploy_to_staging Job:
   • This job is in the deploy stage, which runs after the test stage.
   • It will execute if build_application and critical_unit_tests (the critical jobs in preceding stages) have passed. The failure of non_critical_lint_check will not prevent deploy_to_staging from running because of allow_failure: true on the lint check job.

Key Concepts and Behavior:

• Pipeline Status: allow_failure: true primarily affects the overall pipeline status. The job that fails but is allowed to fail will still be marked as failed individually.
• Visual Indication: In the GitLab UI, jobs that fail but have allow_failure: true are often displayed with a distinct icon (e.g., orange or yellow warning) rather than a red failure icon that would stop the pipeline. The pipeline itself might show as “passed with warnings.”
• Use Cases:
  • Linters/Style Checks: Useful if you want to be informed of style issues but not block deployments for minor infractions.
  • Non-Critical Tests: For test suites that are known to be flaky or cover less critical aspects of the application.
  • Experimental Features: When testing new, unstable features.
  • Informational Scripts: Scripts that gather data or generate reports where failure isn’t catastrophic.
  • Manual Jobs: Often, manual jobs (like deployments to optional environments) might have allow_failure: true if their failure shouldn’t affect the primary outcome of the CI run.
• Within rules:: The allow_failure attribute can also be used within a rules: block, allowing you to make the failure allowance conditional. YAMLmy_job_with_rules: script: - "exit 1" rules: - if: '$CI_COMMIT_BRANCH == "main"' allow_failure: false # Failure on main is critical - if: '$CI_COMMIT_BRANCH =~ /^feature\//' allow_failure: true # Allow failures on feature branches

Using allow_failure: true provides flexibility in defining which job failures are considered critical for your pipeline’s success and which are merely warnings or informational. It helps maintain a balance between ensuring quality and not being overly restrictive for non-essential checks.

Note: While when can be used as a top-level keyword within a job, it is also a crucial component of the more powerful and flexible rules: syntax. The rules: keyword is now the recommended way to control job execution for most complex scenarios. This example will focus on when as a top-level job attribute.

Example .gitlab-ci.yml with when:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - deploy
  - notify
  - cleanup

default:
  image: alpine:latest

# Job 1: Standard job, runs on success of previous stage (default behavior)
build_project:
  stage: build
  # when: on_success # This is the default, so it's often omitted
  script:
    - echo "--- Building Project ---"
    - sleep 5
    - echo "Build successful!"

# Job 2: Another standard job
test_feature_A:
  stage: test
  script:
    - echo "--- Testing Feature A ---"
    - sleep 3
    - echo "Feature A tests PASSED."

# Job 3: This job is designed to fail, to demonstrate 'on_failure' and 'always'
test_feature_B_fails:
  stage: test
  script:
    - echo "--- Testing Feature B (designed to fail) ---"
    - sleep 2
    - echo "Feature B encountered an error!"
    - exit 1 # Explicitly fail the job
  allow_failure: true # Allow this job to fail without failing the whole pipeline,
                      # so we can see 'on_failure' and 'always' jobs run.
                      # In a real scenario, you might not allow failure if it's a critical test.

# Job 4: Runs only if a previous job in the same pipeline (from an earlier stage) failed
cleanup_on_failure:
  stage: cleanup # Placed in a later stage to observe effect
  when: on_failure
  script:
    - echo "--- Cleanup on Failure ---"
    - echo "A previous critical job failed. Performing cleanup tasks."
    - echo "CI_PIPELINE_ID: $CI_PIPELINE_ID, Job that might have failed is in an earlier stage."

# Job 5: Runs regardless of the success or failure of jobs in previous stages
notify_always:
  stage: notify # Placed in a later stage
  when: always
  script:
    - echo "--- Notify Always ---"
    - echo "This notification runs regardless of previous job statuses (success or failure)."
    - echo "Pipeline finished processing up to this point."

# Job 6: Requires manual intervention to run
manual_deploy_to_staging:
  stage: deploy
  when: manual
  script:
    - echo "--- Manual Deploy to Staging ---"
    - echo "Deploying to staging environment after manual trigger..."
    - sleep 10
    - echo "Staging deployment complete."
  allow_failure: false # Optional: specify if this manual job failing should fail the pipeline

# Job 7: Runs after a delay once its stage is ready
delayed_report_generation:
  stage: notify
  when: delayed
  start_in: '1 minute' # Job will start 1 minute after the 'notify' stage is reached
                      # (and previous jobs in this stage, if any, are done)
  script:
    - echo "--- Delayed Report Generation ---"
    - echo "Generating report after a 1-minute delay..."
    - sleep 5
    - echo "Report generated."

# Job 8: Default 'on_success' behavior
final_production_deploy:
  stage: deploy
  # Relies on the default 'when: on_success'
  # This job will only run if 'build_project', 'test_feature_A', AND 'test_feature_B_fails'
  # (if 'allow_failure' was false for it) all succeeded.
  # Given 'test_feature_B_fails' has 'allow_failure: true', this job's execution
  # depends on whether the overall pipeline is considered successful up to its stage.
  # To make it clearly depend on 'test_feature_A' for this example, we could use 'needs'.
  # For simplicity here, we'll assume it runs if prior stages are "green enough".
  script:
    - echo "--- Final Production Deploy ---"
    - echo "All critical tests passed (or allowed to fail). Deploying to production."

Explanation:

1. when: Keyword:
   • Used directly within a job’s definition to specify the condition for its execution.
2. Common when: Values:
   • on_success (Default):
     • If when is not specified, this is the default behavior.
     • The job runs only if all jobs from previous stages in the pipeline have completed successfully (or are marked with allow_failure: true and failed).
     • build_project and test_feature_A implicitly use this. final_production_deploy also implicitly uses this.
   • on_failure:
     • The job runs only if at least one job from a previous stage in the pipeline has failed (and that failing job was not marked with allow_failure: true to the extent that it stops the pipeline progression for on_success jobs).
     • cleanup_on_failure will run because test_feature_B_fails is designed to fail. Even though test_feature_B_fails has allow_failure: true, an on_failure job in a later stage will still trigger if a failure occurred.
   • always:
     • The job runs regardless of the success or failure status of jobs in previous stages, as long as the pipeline itself is running and the job’s stage is reached (or its needs dependencies are met).
     • notify_always will run whether test_feature_B_fails fails or if all test jobs had passed.
   • manual:
     • The job is added to the pipeline but is not executed automatically. It appears with a “play” icon in the GitLab UI, and a user with appropriate permissions must manually trigger it.
     • manual_deploy_to_staging demonstrates this. It’s common for deployment to sensitive environments or for optional long-running tasks.
     • You can optionally add allow_failure: true to manual jobs if their failure shouldn’t block the pipeline.
   • delayed:
     • The job is executed automatically after a specified duration once its stage is ready (all previous stage jobs or needs dependencies are complete).
     • delayed_report_generation uses start_in: '1 minute' to specify the delay. The start_in value can be a human-readable duration like '5 seconds', '10 minutes', '1 hour'.
3. allow_failure: in test_feature_B_fails:
   • I’ve added allow_failure: true to test_feature_B_fails. This means even though this job fails:
     • The pipeline will still be considered “successful” overall if all other critical jobs pass.
     • It allows subsequent stages and jobs (like notify_always and cleanup_on_failure) to run and demonstrate their when conditions.
     • on_failure jobs will still trigger based on the fact that a job did fail, even if that failure was “allowed.”

How when Affects Pipeline Flow:

• on_success jobs create a sequence where success in one stage gates entry into the next.
• on_failure jobs provide a way to run specific tasks (like cleanup or notifications) only when something goes wrong.
• always jobs ensure certain actions (like logging or final notifications) happen no matter what.
• manual jobs introduce decision points or optional steps in the pipeline.
• delayed jobs allow for non-critical tasks to be deferred slightly.

when within rules:

It’s important to note that when is also a key clause within the more modern rules: syntax. For example:

YAML

job_with_rules:
  script: echo "Running with rules"
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: always # This 'when' is part of the rule
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: manual
    - when: never # Default to not running if other rules don't match

In this context, when (along with allow_failure, variables, etc.) is applied if the corresponding if, changes, or exists condition in that specific rule entry evaluates to true.

Understanding when is crucial for building pipelines that can react appropriately to different situations, manage optional steps, and handle failures gracefully.

Example .gitlab-ci.yml with timeout:

This example includes two jobs: one that will likely exceed its timeout and another that should complete within its timeout.

YAML

# .gitlab-ci.yml

stages:
  - process
  - quick_task

default:
  image: alpine:latest

# Job 1: This job is designed to exceed its specified timeout
long_running_job_with_timeout:
  stage: process
  timeout: 30 seconds # Set a timeout of 30 seconds for this job
  script:
    - echo "--- Long Running Job with Timeout ---"
    - echo "This job has a 30-second timeout."
    - echo "Starting a task that will take 45 seconds..."
    - sleep 45 # This command will cause the job to exceed its 30s timeout
    - echo "This line will likely NOT be reached because the job will time out."
  after_script: # after_script also has its own timeout, separate from the job's main script timeout
    - echo "After_script: This runs even if the main script timed out or failed."
    - echo "Job status at this point: $CI_JOB_STATUS"

# Job 2: This job should complete within its timeout
quick_job_with_timeout:
  stage: quick_task
  timeout: 1 minute # Set a timeout of 1 minute for this job
  script:
    - echo "--- Quick Job with Timeout ---"
    - echo "This job has a 1-minute timeout and should complete successfully."
    - echo "Performing a quick task..."
    - sleep 10 # This task takes only 10 seconds
    - echo "Quick task finished well within the timeout."
  after_script:
    - echo "After_script: Quick job finished. Status: $CI_JOB_STATUS"

Explanation:

1. timeout: Keyword:
   • Used within a job definition to specify the maximum allowed execution time for that job’s script section.
   • If the job’s script execution time exceeds this value, GitLab CI will stop the job, and it will be marked as failed with a “job execution timeout” reason.
2. long_running_job_with_timeout Job:
   • timeout: 30 seconds:
     • This job is configured to time out after 30 seconds.
   • script::
     • sleep 45: This command will pause execution for 45 seconds. Since this is longer than the 30-second timeout, this job is expected to be terminated by GitLab CI before the sleep command completes.
     • The final echo statement in the script will likely not be executed.
   • after_script::
     • Commands in after_script are executed after the main script finishes, regardless of whether the main script succeeded, failed, or timed out.
     • after_script has its own, separate timeout (usually shorter, configured by the GitLab administrator at the instance or runner level). It does not inherit the job’s timeout.
     • The $CI_JOB_STATUS variable within the after_script will reflect the status of the main script (e.g., failed if it timed out).
3. quick_job_with_timeout Job:
   • timeout: 1 minute: This job has a more generous timeout of 1 minute.
   • script::
     • sleep 10: This command only pauses for 10 seconds, which is well within the 1-minute timeout.
     • This job should complete successfully.

Duration Format:

The timeout value is specified as a human-readable duration. GitLab supports various units and combinations:

• s or second or seconds
• m or minute or minutes
• h or hour or hours
• Examples:
  • timeout: 5s (5 seconds)
  • timeout: 10m (10 minutes)
  • timeout: 1h 30m (1 hour and 30 minutes)
  • timeout: 2 hours
  • timeout: 300 (Interpreted as 300 seconds, but using units is clearer)

Key Concepts and Behavior:

• Purpose: To prevent runaway jobs from consuming excessive CI/CD runner time and resources. It acts as a safeguard.
• Job Failure: When a job times out, it’s marked as failed in the pipeline, and the failure reason will indicate a timeout.
• Scope:
  • Job-level timeout: As shown in the example, defined per job.
  • Project-level default timeout: You can set a default timeout for all jobs in your project under Settings > CI/CD > General pipelines > Timeout. Job-level timeouts will override this project default.
  • Runner-level timeout: GitLab Runner configurations might also have maximum timeout settings. The most restrictive timeout (job, project, or runner) will apply. For example, if a job specifies timeout: 2 hours but the runner is configured with a maximum of 1 hour, the job will time out after 1 hour.
• after_script Timeout: As mentioned, after_script has its own independent timeout, which is generally not configurable via the .gitlab-ci.yml job timeout keyword but at the runner or instance level.
• Use Cases:
  • Preventing stuck deployment scripts.
  • Limiting the time for long-running test suites that might have performance regressions.
  • Ensuring resource-intensive jobs don’t block runners for too long.
  • Setting realistic expectations for how long a particular automated task should take.

Setting appropriate timeouts is an important part of maintaining a healthy and efficient CI/CD pipeline. It helps in quickly identifying and addressing jobs that are taking longer than expected.

Example .gitlab-ci.yml with retry:

This example simulates a job that might fail on its first attempt but succeed on a retry.

YAML

# .gitlab-ci.yml

stages:
  - test
  - deploy

default:
  image: alpine:latest

# Job 1: A job that might fail due to a transient issue (simulated)
flaky_test_job:
  stage: test
  retry: 2 # Allow up to 2 retries (total 3 attempts: initial + 2 retries)
  script:
    - echo "--- Running Flaky Test Job (Attempt: $((CI_JOB_MANUAL ? 0 : ${CI_JOB_ATTEMPT:-1}))) ---" # CI_JOB_ATTEMPT is available from GitLab 15.5
    - echo "Simulating a test that sometimes fails..."
    # Create a counter file to simulate success on a later attempt
    - ATTEMPT_FILE=".job_attempt_counter"
    - |
      if [ ! -f "$ATTEMPT_FILE" ]; then
        echo "First attempt: Simulating failure."
        echo "1" > "$ATTEMPT_FILE" # Record first attempt
        exit 1 # Fail the job
      elif [ "$(cat $ATTEMPT_FILE)" = "1" ]; then
        echo "Second attempt: Simulating another failure for demonstration."
        echo "2" > "$ATTEMPT_FILE" # Record second attempt
        exit 1 # Fail the job again
      else
        echo "Third attempt (or later if manual retry): Test passes!"
        rm "$ATTEMPT_FILE" # Clean up
        exit 0 # Succeed the job
      fi

# Job 2: Another job with more specific retry conditions
deploy_to_staging:
  stage: deploy
  retry:
    max: 1 # Allow up to 1 retry (total 2 attempts)
    when: # Specify conditions under which to retry
      - runner_system_failure # Retry if the runner had a system issue
      - stuck_or_timeout_failure # Retry if the job got stuck or timed out
      - script_failure # Also retry on general script failures for this example
  script:
    - echo "--- Deploying to Staging (Attempt: $((CI_JOB_MANUAL ? 0 : ${CI_JOB_ATTEMPT:-1}))) ---"
    - RANDOM_NUMBER=$((1 + RANDOM % 3)) # Generate a random number between 1 and 3
    - |
      if [ "$RANDOM_NUMBER" -ne "3" ] && [ "${CI_JOB_ATTEMPT:-1}" -lt "2" ] ; then # Fail on 1 or 2 on first attempt
        echo "Deployment to staging failed due to a simulated transient issue (e.g., network timeout)."
        exit 1
      else
        echo "Deployment to staging successful!"
        exit 0
      fi

Explanation:

1. retry: Keyword:
   • Used within a job definition to configure its retry behavior.
   • It tells the GitLab Runner to re-run the job automatically if it fails, up to a specified number of times and under certain conditions.
2. flaky_test_job (Simple Retry Count):
   • retry: 2:
     • This is the simpler way to define retries. The number 2 means the job will be retried up to 2 times after its initial failure.
     • This results in a maximum of 3 attempts in total (1 initial attempt + 2 retries).
     • GitLab allows a maximum of 2 retries (so values of 0, 1, or 2).
     • If when: is not specified, it defaults to retrying on most failure reasons (effectively retry:when:always in older terms, or a broad set of common failure types now).
   • Script Logic:
     • The script uses a file (.job_attempt_counter) to simulate a scenario where the first two attempts fail, and the third attempt succeeds.
     • CI_JOB_ATTEMPT (available GitLab 15.5+) is a predefined variable that indicates the current attempt number for a job, starting from 1. The script uses a fallback for older versions or manual retries.
3. deploy_to_staging (Detailed Retry Configuration):
   • retry:: This starts the retry configuration block.
   • max: 1: Specifies the maximum number of retries (so, up to 1 retry, meaning 2 attempts in total).
   • when:: This sub-key allows you to specify a list of failure reasons for which the job should be retried. The job will only be retried if its failure reason is in this list.
     • runner_system_failure: A problem with the GitLab Runner itself (e.g., crash, unexpected shutdown).
     • stuck_or_timeout_failure: The job was stuck or exceeded its execution timeout.
     • script_failure: The job failed because a command in its script section exited with a non-zero status code.
     • Other common when values include: unknown_failure, api_failure, missing_dependency_failure, job_execution_timeout, etc. You can list multiple reasons. If any of the listed reasons occur, the job is retried.
   • Script Logic:
     • This script uses a random number to simulate a deployment that might fail on the first attempt but succeed on the retry. It’s designed to likely fail if RANDOM_NUMBER isn’t 3 on the first try but will succeed on the second attempt (as CI_JOB_ATTEMPT would be 2).

Key Concepts and Behavior:

• Purpose: To handle intermittent or transient failures automatically, improving pipeline reliability without manual intervention for minor, non-deterministic issues.
• Number of Retries: The integer value for retry or retry:max: specifies the number of additional attempts after the first failure. So retry: 1 means one initial attempt and one retry (total 2 attempts). The maximum allowed value is 2 (total 3 attempts).
• Retry Conditions (when):
  • If only retry: <count> is used, it retries on a broad range of failures.
  • retry:when: gives you fine-grained control over why a job should be retried. This is useful to avoid retrying jobs that fail due to genuine code errors (where a retry won’t help).
• Job Logs: Each attempt of a retried job is logged separately in the GitLab UI, allowing you to see the output and failure reason for each attempt.
• Final Status: If all allowed attempts (initial + retries) fail, the job is marked as failed in the pipeline. If any attempt succeeds, the job is marked as successful.
• Idempotency: Jobs that are configured to retry should ideally be idempotent. This means running the job multiple times should have the same effect as running it once successfully (e.g., it shouldn’t create duplicate resources or have unintended side effects on repeated runs).
• CI_JOB_ATTEMPT Variable: Starting from GitLab 15.5, this predefined variable can be used within your script to know which attempt number is currently running. This can be useful for implementing conditional logic within your script for retries.

When to Use retry:

• Flaky Tests: Tests that occasionally fail due to timing issues, external service flakiness, or unstable test environments.
• Network Dependent Operations: Jobs that interact with external services over a network that might experience temporary outages or glitches (e.g., fetching dependencies, deploying to a cloud provider).
• Resource Contention: Situations where a job might fail if a shared resource (like a database connection pool on a test server) is temporarily exhausted.

Do not overuse retry to mask fundamental problems in your code or scripts. If a job consistently fails, it needs to be fixed, not just retried. retry is for genuinely transient issues.

This is essential for:

• Passing build outputs (like compiled binaries, distributable packages, or static website files) to subsequent jobs (e.g., for testing or deployment).
• Storing reports (test reports, code coverage reports, etc.).
• Making build results available for users to download.

Example .gitlab-ci.yml with artifacts:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - package

default:
  image: alpine:latest

# Job 1: Builds the application and creates artifacts
build_application:
  stage: build
  script:
    - echo "--- Building Application ---"
    - mkdir -p dist/app  # Create a directory for distributable files
    - echo "MyApplication_v1.0_binary" > dist/app/my_app
    - echo "Configuration file content" > dist/app/config.json
    - echo "Build successful!"

    - echo "Generating a build report..."
    - echo "Build report for commit $CI_COMMIT_SHORT_SHA" > build_report.txt
    - echo "Timestamp: $(date)" >> build_report.txt

  artifacts:
    name: "${CI_JOB_NAME}_${CI_COMMIT_REF_SLUG}" # Optional: Custom name for the artifact archive
    paths:
      - dist/ # Save the entire 'dist' directory
      - build_report.txt # Save an individual file
    expire_in: '1 week' # Optional: How long to keep the artifacts (e.g., 1 day, 1 week, 3 months)
    when: on_success # Optional: When to create artifacts (on_success, on_failure, always). Default is on_success.
    # untracked: true # Optional: Set to true to also include all Git untracked files

# Job 2: Tests the application using artifacts from the 'build_application' job
test_application:
  stage: test
  # This job will automatically download artifacts from 'build_application'
  # because it's in a later stage and 'dependencies' is not set to [].
  script:
    - echo "--- Testing Application ---"
    - echo "Checking for build artifacts..."
    - ls -R dist/ # List contents of the 'dist' directory (should contain 'app/my_app' and 'app/config.json')
    - if [ -f "dist/app/my_app" ]; then echo "Application binary found."; else echo "ERROR: Application binary NOT found!"; exit 1; fi
    - if [ -f "build_report.txt" ]; then cat build_report.txt; else echo "ERROR: Build report NOT found!"; exit 1; fi
    - echo "Simulating tests on my_app..."
    - sleep 5
    - echo "Tests passed!"

# Job 3: Packages the application, also using artifacts from 'build_application'
# This demonstrates another job in a later stage accessing the same artifacts.
package_release:
  stage: package
  script:
    - echo "--- Packaging Release ---"
    - echo "Using build artifacts for packaging..."
    - if [ -f "dist/app/my_app" ]; then echo "Packaging $(cat dist/app/my_app)..."; else echo "ERROR: Application binary for packaging NOT found!"; exit 1; fi
    - tar -czvf "release_package_${CI_COMMIT_SHORT_SHA}.tar.gz" dist/ build_report.txt
    - echo "Release package created: release_package_${CI_COMMIT_SHORT_SHA}.tar.gz"
  artifacts:
    paths:
      - "*.tar.gz" # Save the created tarball as an artifact of this job
    expire_in: '30 days'

Explanation:

1. artifacts: Keyword:
   • Used within a job definition to specify which files and directories should be preserved after the job completes.
   • GitLab collects these files, archives them (usually as a ZIP file), and uploads them.
2. build_application Job (Artifact Producer):
   • script:: This section creates a dist/app/ directory with some dummy application files and a build_report.txt.
   • artifacts::
     • name: "${CI_JOB_NAME}_${CI_COMMIT_REF_SLUG}":
       • (Optional) Allows you to define a custom name for the downloadable artifact archive. Here, it uses predefined CI/CD variables to make the name dynamic (e.g., build_application_main.zip). If omitted, GitLab generates a default name.
     • paths::
       • (Required if artifacts is used) An array of file paths or directory paths that should be included in the artifact archive. Paths are relative to the project’s root directory ($CI_PROJECT_DIR).
       • - dist/: Includes the entire dist directory and its contents.
       • - build_report.txt: Includes the individual build_report.txt file.
     • expire_in: '1 week':
       • (Optional) Specifies how long the artifacts for this job should be stored in GitLab. After this period, they might be deleted to save space (unless “Keep artifacts from most recent successful_jobs jobs” is enabled in project settings). The format can be natural language (e.g., '1 day', '2 hrs 30 mins', '6 mos') or a specific duration.
     • when: on_success:
       • (Optional) Determines when artifacts are created.
         • on_success (Default): Artifacts are saved only if the job completes successfully.
         • on_failure: Artifacts are saved only if the job fails (useful for logs or debug info).
         • always: Artifacts are saved regardless of the job’s success or failure.
     • untracked: true:
       • (Optional) If set to true, GitLab will also include all files that are untracked by Git in the artifact archive, in addition to the paths specified. Use with caution.
3. test_application Job (Artifact Consumer):
   • This job is in the test stage, which is later than the build stage where build_application ran.
   • Automatic Artifact Download: By default, jobs in later stages automatically download artifacts from all jobs in all previous stages if those jobs completed successfully and produced artifacts. This happens unless you use dependencies: [] in the consuming job or manage dependencies with needs: in a way that excludes certain artifacts.
   • The script section in test_application can therefore access the files from dist/ and build_report.txt as if they were part of its own workspace.
4. package_release Job (Another Artifact Consumer & Producer):
   • This job also benefits from the default artifact download from build_application.
   • It then creates its own artifact (release_package_${CI_COMMIT_SHORT_SHA}.tar.gz) by packaging the received artifacts and the report.

Key Concepts and Behavior:

• Purpose: To persist files and directories created by a job for use in subsequent jobs, for manual download, or for features like GitLab Pages or test reporting.
• Storage: Artifacts are stored on the GitLab server (or external storage if configured) and are associated with the job and pipeline run.
• Downloading Artifacts:
  • From UI: You can download artifacts as a ZIP file from the job’s page in the GitLab UI, or from the pipeline view.
  • By Other Jobs: As shown, later jobs often get them automatically. needs: provides more explicit control over which artifacts are downloaded from specified prerequisite jobs. dependencies: can also be used for more fine-grained control when not using needs.
• Artifacts vs. Cache:
  • Artifacts: Intended for passing outputs between stages, for final build products, or for reports. They are generally meant to be kept (at least until expire_in). They are uploaded to GitLab.
  • Cache: Intended to speed up subsequent runs of the same job or related jobs (e.g., node_modules, compiler caches). Caches are best-effort and might be deleted by runners to save space. They are typically stored on the runner or a shared cache storage like S3.
• artifacts:reports: (Advanced):
  • GitLab has special support for certain types of reports (e.g., JUnit XML for test results, Cobertura for code coverage). Using artifacts:reports:junit: report.xml allows GitLab to parse these reports and display information directly in the UI (e.g., in Merge Requests).

Using artifacts is fundamental to building effective CI/CD pipelines in GitLab, as it enables the flow of work and results between different stages of your development lifecycle.

• dependencies traditionally controlled which artifacts from jobs in previous stages were downloaded before a job starts. It does not control the order of execution (that’s handled by stages or needs).
• needs: is a more modern feature that allows you to create a Directed Acyclic Graph (DAG) of job execution. When a job uses needs: [job_A], it implies an execution dependency on job_A AND, by default, it will download artifacts from job_A.
• Current Practice:
  • If you are using needs: to define relationships between jobs, you typically do not need to use dependencies to download artifacts from those needs-specified jobs, as needs: handles it.
  • The most common and clear use of dependencies in modern GitLab CI is dependencies: [] (an empty array), which means the job should not download artifacts from any jobs in previous stages.
  • If you are not using needs for a specific job relationship and want to selectively download artifacts from previous stages, then dependencies can be used.

This example will illustrate both the selective use of dependencies (when not using needs for that specific artifact flow) and the common dependencies: [] pattern.

Example .gitlab-ci.yml with dependencies:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - package

default:
  image: alpine:latest

# --- Build Stage Jobs (Producers) ---
build_code:
  stage: build
  script:
    - echo "--- Building Main Code ---"
    - mkdir -p build_output/code
    - echo "main_application_binary" > build_output/code/app.bin
    - echo "Build Code complete."
  artifacts:
    paths:
      - build_output/code/
    expire_in: 1 hour

build_assets:
  stage: build
  script:
    - echo "--- Building Assets ---"
    - mkdir -p build_output/assets
    - echo "image.png" > build_output/assets/image.png
    - echo "style.css" > build_output/assets/style.css
    - echo "Build Assets complete."
  artifacts:
    paths:
      - build_output/assets/
    expire_in: 1 hour

# --- Test Stage Jobs (Consumers) ---

# Job 1: Selectively downloads artifacts ONLY from 'build_code'
# This is the "classic" use of dependencies when 'needs' isn't used for this relationship.
test_main_app:
  stage: test
  dependencies:
    - build_code # Only download artifacts from 'build_code' job
  script:
    - echo "--- Testing Main Application ---"
    - echo "Checking for 'build_code' artifacts:"
    - ls -R build_output/
    - if [ -f "build_output/code/app.bin" ]; then echo "app.bin found."; else echo "ERROR: app.bin NOT found!"; exit 1; fi
    - if [ -f "build_output/assets/image.png" ]; then echo "WARNING: image.png found, but was not explicitly depended upon (might indicate default download if 'dependencies' was misconfigured or absent)."; else echo "image.png correctly NOT found (as not depended upon)."; fi
    - echo "Main application tests complete."

# Job 2: Downloads NO artifacts from previous stages
# This is a very common modern use case for 'dependencies'.
test_assets_isolated:
  stage: test
  dependencies: [] # Explicitly download no artifacts
  script:
    - echo "--- Testing Assets in Isolation ---"
    - echo "This job should start with a clean workspace (no artifacts from 'build' stage)."
    - echo "Checking for any unexpected artifacts in 'build_output':"
    - if [ -d "build_output" ]; then ls -R build_output/; echo "ERROR: 'build_output' directory exists! Artifacts were downloaded unexpectedly."; exit 1; else echo "'build_output' directory does not exist. Correct."; fi
    # In a real scenario, this job might fetch assets differently or test their generation.
    - echo "Asset tests (isolated) complete."

# Job 3: Default behavior - downloads artifacts from ALL successful jobs in previous stages
# This happens if 'dependencies' is omitted AND 'needs' is not used to link to these specific jobs.
test_with_all_artifacts_default:
  stage: test
  # No 'dependencies' or 'needs' keyword here
  script:
    - echo "--- Testing with All Default Artifacts ---"
    - echo "This job should have artifacts from ALL successful jobs in the 'build' stage."
    - ls -R build_output/
    - if [ -f "build_output/code/app.bin" ] && [ -f "build_output/assets/image.png" ]; then echo "Both app.bin and image.png found. Correct default behavior."; else echo "ERROR: Expected artifacts not found!"; exit 1; fi
    - echo "Tests with all artifacts complete."

# --- Package Stage Job ---
# This job uses 'needs' which handles artifact downloads from 'test_main_app'.
# 'dependencies' would typically not be used here in conjunction with 'needs' for 'test_main_app'.
package_application:
  stage: package
  needs: ["test_main_app"] # 'needs' implies artifacts from 'test_main_app' will be downloaded if 'test_main_app' produced them.
                           # 'test_main_app' itself only got artifacts from 'build_code'.
                           # If 'package_application' also needed artifacts directly from 'build_assets',
                           # you'd add 'build_assets' to 'needs' or use a different strategy.
  script:
    - echo "--- Packaging Application ---"
    - echo "This job uses 'needs'. Artifacts from 'test_main_app' (which are originally from 'build_code') should be available."
    - ls -R build_output/
    - if [ -f "build_output/code/app.bin" ]; then echo "app.bin found for packaging."; else echo "ERROR: app.bin NOT found for packaging!"; exit 1; fi
    - echo "Application packaging complete."

Explanation:

1. dependencies: Keyword:
   • Used within a job definition to control which artifacts from jobs in previous stages are downloaded into the job’s workspace before its before_script and script sections run.
   • It does NOT define execution order. The stages keyword defines the general order, and needs: defines specific job execution dependencies.
2. test_main_app Job (Selective Dependencies):
   • dependencies: - build_code:
     • This tells GitLab CI to download artifacts only from the build_code job (which ran in the preceding build stage).
     • Artifacts from build_assets (also in the build stage) will not be downloaded for this test_main_app job.
     • This is useful when a job only requires a subset of artifacts from previous stages.
3. test_assets_isolated Job (Empty Dependencies):
   • dependencies: [] (an empty array):
     • This is a special and very common instruction. It tells the job not to download any artifacts from any jobs in previous stages.
     • The job will start with a clean workspace concerning artifacts from prior stages. This is useful for jobs that should not be influenced by previous outputs or for jobs that manage their dependencies in a different way (e.g., pulling from a package registry).
4. test_with_all_artifacts_default Job (Default Behavior):
   • This job omits the dependencies keyword and also does not use needs to link to specific jobs in the build stage.
   • In this scenario (no dependencies and no relevant needs), a job will, by default, download artifacts from all jobs in all previous stages that completed successfully and produced artifacts.
   • So, test_with_all_artifacts_default will get artifacts from both build_code AND build_assets.
5. package_application Job (Interaction with needs):
   • This job uses needs: ["test_main_app"].
   • When needs is used, it implies that artifacts from the specified needs jobs will be downloaded by default (equivalent to needs: [{ job: "job_name", artifacts: true }]).
   • Therefore, package_application will download artifacts produced by test_main_app. Since test_main_app itself only downloaded artifacts from build_code, those are the artifacts that package_application will ultimately see from the build stage via this chain.
   • You generally do not specify dependencies for jobs already listed in needs because needs handles the artifact download. Using both can sometimes lead to downloading artifacts twice or having less clear intent.

Key Concepts:

• Artifact Control (Pre-needs Era / Specific Cases): dependencies was the primary way to manage which artifacts get downloaded. If a job needed artifacts from job A and job B in a previous stage, but not job C, you’d list dependencies: [job_A, job_B].
• Clean Workspace: dependencies: [] is the standard way to ensure a job starts clean without inheriting artifacts from previous stages.
• Default Download: If dependencies is absent and needs isn’t defining the artifact flow from previous stages, artifacts from all previous stage jobs are downloaded. This can be resource-intensive if not intended.
• No Impact on Order: dependencies only affects artifact downloading. It does not change the execution order of jobs or stages.
• Superseded by needs for Linked Jobs: For jobs that have an explicit execution dependency defined by needs, the artifact flow from those needed jobs is also managed by needs (artifacts are downloaded by default).

In modern GitLab CI, you’ll often see needs used to define both the execution flow and the artifact flow for direct dependencies, and dependencies: [] used when a job needs a completely clean slate. The classic use of dependencies: [job_A, job_B] is less common if you’re heavily leveraging needs for your pipeline’s structure.

Example .gitlab-ci.yml with needs:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - deploy

default:
  image: alpine:latest
  # For 'needs' to automatically download artifacts from prerequisite jobs,
  # those jobs must define 'artifacts'. We'll add simple artifacts for demonstration.

# --- Build Stage Jobs ---
build_backend:
  stage: build
  script:
    - echo "--- Building Backend ---"
    - sleep 10 # Simulate build time
    - mkdir -p backend_build/
    - echo "backend_app_v1" > backend_build/version.txt
    - echo "Backend built successfully."
  artifacts:
    paths:
      - backend_build/
    expire_in: 1 hour

build_frontend:
  stage: build
  script:
    - echo "--- Building Frontend ---"
    - sleep 15 # Simulate build time
    - mkdir -p frontend_build/
    - echo "frontend_app_v1" > frontend_build/ui_bundle.js
    - echo "Frontend built successfully."
  artifacts:
    paths:
      - frontend_build/
    expire_in: 1 hour

# --- Test Stage Jobs ---
# This job will start as soon as 'build_backend' finishes, without waiting for 'build_frontend'.
test_backend_api:
  stage: test
  needs: ["build_backend"] # Explicitly depends on 'build_backend'
  script:
    - echo "--- Testing Backend API ---"
    - echo "Checking backend build artifacts..."
    - if [ -f "backend_build/version.txt" ]; then cat backend_build/version.txt; else echo "Backend version file not found!"; exit 1; fi
    - sleep 8  # Simulate test time
    - echo "Backend API tests PASSED."

# This job will start as soon as 'build_frontend' finishes, without waiting for 'build_backend'.
test_frontend_ui:
  stage: test
  needs: ["build_frontend"] # Explicitly depends on 'build_frontend'
  script:
    - echo "--- Testing Frontend UI ---"
    - echo "Checking frontend build artifacts..."
    - if [ -f "frontend_build/ui_bundle.js" ]; then echo "Frontend bundle found."; else echo "Frontend bundle not found!"; exit 1; fi
    - sleep 10 # Simulate test time
    - echo "Frontend UI tests PASSED."

# This job needs both build jobs to complete before it can start.
# It will start as soon as BOTH 'build_backend' AND 'build_frontend' are done.
integration_tests:
  stage: test
  needs: ["build_backend", "build_frontend"] # Depends on multiple jobs
  script:
    - echo "--- Running Integration Tests ---"
    - echo "Verifying backend and frontend integration..."
    - if [ -f "backend_build/version.txt" ] && [ -f "frontend_build/ui_bundle.js" ]; then echo "All necessary artifacts found."; else echo "Missing artifacts for integration!"; exit 1; fi
    - sleep 12 # Simulate integration test time
    - echo "Integration tests PASSED."

# --- Deploy Stage Job ---
# This job will start as soon as 'integration_tests' finishes.
# It also implicitly benefits from artifacts if integration_tests produced any relevant for deployment.
deploy_application:
  stage: deploy
  needs: ["integration_tests"]
  script:
    - echo "--- Deploying Application ---"
    - echo "Deploying based on successful integration tests."
    - sleep 10 # Simulate deployment time
    - echo "Application deployed successfully!"
  when: on_success # Default, but good for deploy jobs

Explanation:

1. needs: Keyword:
   • Used within a job definition to specify an array of other jobs that must complete successfully before this job can start.
   • It allows you to create a more granular dependency chain than what stages provide.
2. stages: Still Define the Overall Flow (and Visualization):
   • Jobs still belong to stages. Stages are used for:
     • Visual grouping in the GitLab CI/CD pipeline graph.
     • A fallback execution order if needs: is not used.
     • Determining the order if needs: creates parallel paths that eventually converge on a stage.
   • However, needs: can make a job start earlier than its defined stage if its needs dependencies are met.
3. test_backend_api Job:
   • stage: test
   • needs: ["build_backend"]: This is the key. test_backend_api will not wait for build_frontend (which is also in the build stage) to complete. As soon as build_backend finishes successfully, test_backend_api can start, even if build_frontend is still running.
4. test_frontend_ui Job:
   • stage: test
   • needs: ["build_frontend"]: Similar to the backend test, this job starts as soon as build_frontend is done, independently of build_backend‘s completion or test_backend_api‘s status (unless test_backend_api was also in its needs list).
5. integration_tests Job:
   • stage: test
   • needs: ["build_backend", "build_frontend"]: This job depends on both build_backend and build_frontend. It will only start after both of these prerequisite jobs have completed successfully.
6. deploy_application Job:
   • stage: deploy
   • needs: ["integration_tests"]: The deployment will only start after integration_tests is successful.
7. Artifacts and needs:
   • When you use needs:, by default, GitLab will attempt to download artifacts from all the jobs listed in the needs array. This is equivalent to needs: [{ job: "job_name", artifacts: true }].
   • In the example, build_backend produces an artifact in backend_build/. test_backend_api (which needs: ["build_backend"]) will have access to this backend_build/ directory and its contents.
   • If you don’t want artifacts from a needed job, you can specify needs: [{ job: "job_name", artifacts: false }].
8. Impact of Failure:
   • If any job listed in a needs array fails (or is skipped, or canceled), the dependent job will be skipped by default and will not run.

How needs: Speeds Up Pipelines:

Imagine the build stage without needs:

• build_backend (10s) and build_frontend (15s) run in parallel.
• The build stage finishes after the longest job, so after 15s.
• Only then would the test stage begin.

With needs::

• build_backend finishes in 10s. test_backend_api can start immediately.
• build_frontend finishes in 15s. test_frontend_ui can start immediately.
• integration_tests starts when both build_backend and build_frontend are done (i.e., after 15s in this example).
• The overall pipeline can proceed much faster because testing for completed build components starts earlier.

Key Concepts and Benefits:

• DAG (Directed Acyclic Graph): needs: allows you to define your pipeline as a graph of dependencies, rather than a strict linear sequence of stages.
• Reduced Latency: Jobs start as soon as their true dependencies are met, leading to shorter overall pipeline durations.
• Increased Efficiency: Better utilization of runners as jobs don’t wait unnecessarily.
• Implicit Artifact Download: Simplifies passing artifacts between dependent jobs.
• Optional Dependencies: You can also use needs: [{ job: "optional_job", optional: true }]. If optional_job fails or is skipped, the current job will still run (but it won’t download artifacts from the optional job if it didn’t produce them).

Using needs: is a powerful way to optimize your GitLab CI/CD pipelines, especially for projects with multiple independent build components or test suites that can run as soon as their specific dependencies are ready.

A rules section is an array of individual rule objects. These rules are evaluated in order from top to bottom for each job. The first rule that matches determines the job’s behavior.

Example .gitlab-ci.yml with rules:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - security
  - deploy
  - report

default:
  image: alpine:latest

# Job 1: Runs only on pushes to the default branch (e.g., 'main' or 'master')
build_default_branch:
  stage: build
  script:
    - echo "--- Building for default branch: $CI_DEFAULT_BRANCH ---"
    - echo "Commit on branch: $CI_COMMIT_BRANCH"
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "push"'
      # 'when: on_success' is implied if 'when' is not specified for an 'if' rule

# Job 2: Runs only when a Git tag is pushed
deploy_on_tag:
  stage: deploy
  script:
    - echo "--- Deploying for tag: $CI_COMMIT_TAG ---"
  rules:
    - if: '$CI_COMMIT_TAG' # This condition is true if a tag is present
      when: on_success # Explicitly stating the default when job should run

# Job 3: Manual job for merge requests, allows failure
test_on_merge_request:
  stage: test
  script:
    - echo "--- Testing for Merge Request: $CI_MERGE_REQUEST_IID ---"
    - echo "Source branch: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"
    - echo "Target branch: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
    # - exit 1 # Uncomment to simulate a failure
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: manual # Job will be created but needs to be started manually from the UI
      allow_failure: true # If this job fails, it won't cause the whole pipeline to fail

# Job 4: Runs only if files in the 'docs/' directory have changed on a push event
build_docs:
  stage: build
  script:
    - echo "--- Building documentation because files in 'docs/' changed ---"
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push"' # Only consider push events for changes
      changes:
        - docs/**/* # Glob pattern for any file within the docs directory or its subdirectories
        - README.md
      # If 'changes' are detected and the 'if' condition is met, the job runs.

# Job 5: Runs only if a specific file (e.g., a Dockerfile) exists in the repository
docker_build_if_exists:
  stage: build
  script:
    - echo "--- Dockerfile found, proceeding with Docker build ---"
    # - docker build -t my-image . # Actual docker build command
  rules:
    - if: '$CI_COMMIT_BRANCH' # Ensure it runs for branch commits, not other events unless specified
      exists:
        - Dockerfile # Path to the file to check, relative to project root
        # - another-required-file.txt
      # If 'Dockerfile' exists and the 'if' condition is met, the job runs.

# Job 6: A job that is explicitly prevented from running in any 'push' pipeline, but could run for other types if rules were added
never_run_on_push:
  stage: report
  script:
    - echo "This job should not run on push events."
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never # Explicitly prevents the job from being added to the pipeline for push events
    # - if: '$CI_PIPELINE_SOURCE == "schedule"' # Example: but it could run for schedules
    #   when: always

# Job 7: A job with a delayed execution for non-critical tasks
delayed_notification:
  stage: report
  script:
    - echo "--- Sending a delayed notification ---"
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "push"'
      when: delayed
      start_in: '5 minutes' # Job will start 5 minutes after the pipeline completes this stage successfully

# Job 8: Catch-all rule if no other rules in this job match (optional)
# If a job has rules, and none of them match, the job is NOT added to the pipeline by default.
# To change this, you can add a final rule without an 'if' clause.
optional_job_always_manual:
  stage: report
  script:
    - echo "This is an optional manual job."
  rules:
    - if: '$CI_COMMIT_BRANCH == "feature/special-feature"' # Runs normally for this specific branch
    - when: manual # For all other cases where the above 'if' didn't match, make it manual

Explanation:

1. rules: Keyword:
   • Used within a job definition. It takes an array of rule objects.
   • Rules are evaluated in the order they are listed. The first rule that matches determines the job’s behavior.
   • If no rules match for a job, the job is not added to the pipeline by default.
2. Structure of a Rule:Each item in the rules array is an object that can contain several clauses:
   • if:: An expression that is evaluated. If true, this rule matches (unless other conditions like changes or exists are also present and don’t match). You use CI/CD predefined variables here.
     • Operators: == (equals), != (not equals), =~ (regex match), !~ (regex no match), && (AND), || (OR – though multiple if clauses in separate rules act like OR for the job). Parentheses () can be used for grouping.
   • when:: Defines what to do if the rule matches.
     • on_success (default if if is present and when is omitted): Run the job automatically if the stage is reached.
     • manual: Add the job to the pipeline, but it requires manual action (clicking a play button) to start.
     • delayed: Run the job automatically after a specified duration once its stage is reached and preceding jobs are complete. Requires start_in.
     • never: Do not add the job to the pipeline if this rule matches.
     • always: Run the job regardless of the status of earlier stage jobs (use with caution, often combined with allow_failure: true).
   • changes:: An array of file paths or glob patterns. The rule matches only if files matching these patterns have changed in the latest push (compared to the previous commit on the same branch, or for MRs, compared to the target branch).
   • exists:: An array of file paths or glob patterns. The rule matches only if all specified files/paths exist in the repository at the commit being processed.
   • allow_failure:: (boolean, defaults to false) If true, the pipeline will continue to the next stage even if this job fails. This can be set per rule.
   • variables:: Define job-specific variables if this rule matches.
3. build_default_branch (Job 1):
   • if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "push"': This job runs only if the commit is on the default branch AND the pipeline was triggered by a push event. The when: on_success is implied.
4. deploy_on_tag (Job 2):
   • if: '$CI_COMMIT_TAG': This job runs if the $CI_COMMIT_TAG variable is set (meaning a Git tag was pushed).
5. test_on_merge_request (Job 3):
   • if: '$CI_PIPELINE_SOURCE == "merge_request_event"': Runs for merge request events.
   • when: manual: It will appear in the pipeline but won’t run automatically.
   • allow_failure: true: If it fails after being run manually, the overall pipeline status won’t be marked as failed.
6. build_docs (Job 4):
   • if: '$CI_PIPELINE_SOURCE == "push"': Condition based on pipeline source.
   • changes: - docs/**/* - README.md: The job only runs if there are changes in the docs directory or the README.md file for that push event.
7. docker_build_if_exists (Job 5):
   • if: '$CI_COMMIT_BRANCH': A general condition to ensure it runs for branches.
   • exists: - Dockerfile: The job runs only if a Dockerfile exists at the root of the repository for the given commit.
8. never_run_on_push (Job 6):
   • if: '$CI_PIPELINE_SOURCE == "push"' combined with when: never: This effectively blocks the job from running on any push event. You could add other rules below it to make it run for other events (e.g., schedules).
9. delayed_notification (Job 7):
   • when: delayed and start_in: '5 minutes': This job will automatically start 5 minutes after its stage (report) is reached and preceding jobs in that stage (if any) are complete, but only for pushes to the default branch.
10. optional_job_always_manual (Job 8):
    • The first rule if: '$CI_COMMIT_BRANCH == "feature/special-feature"' makes the job run normally (auto) for this specific branch.
    • The second rule when: manual has no if. This acts as a catch-all. If the first rule didn’t match, this second rule will match, and the job will be added as a manual job.

Benefits of rules::

• Clarity: Conditions are often easier to read and understand than complex only/except combinations.
• Power: Allows more complex conditions, including checks for file changes (changes), file existence (exists), and variable comparisons with regex.
• Control: Provides fine-grained control over when a job runs (manual, delayed) and allow_failure on a per-rule basis.
• Single Source of Truth: Replaces the need for both only and except.

rules: is a fundamental concept for modern GitLab CI/CD, allowing you to build highly efficient and precise pipelines.

Important Note: Just like with only, while except still functions, GitLab’s current best practice and strong recommendation is to use the rules: keyword instead. rules: offer significantly more power, flexibility, and clarity for controlling when jobs run or don’t run. This example is provided because you specifically asked about except. Please consider using rules: for your future and more complex pipeline configurations.

Example .gitlab-ci.yml with except:

YAML

# .gitlab-ci.yml

stages:
  - build
  - test
  - deploy
  - cleanup

default:
  image: alpine:latest

# Job 1: Runs on all branches EXCEPT 'main'
build_on_feature_branches:
  stage: build
  except:
    - main # Can also be written as refs: - main
  script:
    - echo "--- Building on a feature branch (not main) ---"
    - echo "Current branch: $CI_COMMIT_REF_NAME"

# Job 2: Runs for all pipeline triggers EXCEPT when Git tags are pushed
test_everything_but_tags:
  stage: test
  except:
    - tags # Special keyword for any tag
  script:
    - echo "--- Running tests (not for a tag) ---"
    - echo "Pipeline source: $CI_PIPELINE_SOURCE"
    - echo "This job does NOT run when a tag is pushed."

# Job 3: Runs for all pipeline triggers EXCEPT for merge request events
deploy_to_staging_not_for_mr:
  stage: deploy
  except:
    - merge_requests # Special keyword for merge request pipelines
  script:
    - echo "--- Deploying to Staging (not for an MR) ---"
    - echo "This job runs for direct pushes, tags, schedules, etc., but NOT for merge requests."

# Job 4: Runs for all pipeline triggers EXCEPT for scheduled pipelines
manual_cleanup_only:
  stage: cleanup
  except:
    - schedules # Special keyword for scheduled pipelines
  script:
    - echo "--- Manual Cleanup Task (not for schedules) ---"
    - echo "This job does NOT run for scheduled pipelines."

# Job 5: Excludes multiple specific branches
# This job runs unless the branch is 'develop' OR 'release/v1.0'
build_except_specific_branches:
  stage: build
  except:
    - develop
    - release/v1.0
  script:
    - echo "--- Building (excluding develop and release/v1.0) ---"
    - echo "Current branch is $CI_COMMIT_REF_NAME."

# Job 6: Runs always (no 'only' or 'except' clause)
# This job is included to show the default behavior.
always_run_job_default:
  stage: test
  script:
    - echo "--- Always Run Job (Default Behavior) ---"
    - echo "This job has no 'only' or 'except', so it runs for any pipeline trigger by default"
    - echo "(unless workflow:rules prevent the pipeline itself, or a more specific 'only' restricts it)."

# Job 7: Using 'refs' with 'except' (though 'rules' is better for this)
# This job does NOT run for branches starting with 'legacy-' OR for the 'archive' branch.
# Note: For complex logic, 'rules' is much clearer.
advanced_except_example:
  stage: cleanup
  except:
    refs:
      - /^legacy-.*$/ # Regex for branches starting with legacy-
      - archive
  script:
    - echo "--- Advanced Except Example ---"
    - echo "Running because this is not a legacy or archive branch."
    - echo "Branch: $CI_COMMIT_BRANCH"

Explanation:

1. except: Keyword:
   • Used within a job definition to specify conditions under which the job should not run. It acts as a blacklist.
   • The job will be skipped if any of the conditions listed under except are met for the event that triggered the pipeline.
2. build_on_feature_branches Job:
   • except: - main: This job will be created and run for any branch push, unless the push is to the main branch.
3. test_everything_but_tags Job:
   • except: - tags: This job will run for any pipeline trigger (branch pushes, merge requests, schedules, etc.) except when a Git tag is pushed.
4. deploy_to_staging_not_for_mr Job:
   • except: - merge_requests: This job will run for any pipeline trigger except for pipelines associated with merge request events.
5. manual_cleanup_only Job:
   • except: - schedules: This job will run for any pipeline trigger except when the pipeline is triggered by a predefined schedule.
6. build_except_specific_branches Job:
   • except: - develop - release/v1.0: This job will be skipped if the pipeline is for the develop branch OR for the release/v1.0 branch. For all other branches/events, it will run.
7. always_run_job_default Job:
   • This job has no except (or only or rules) clause. By default, such a job is included in all pipelines (unless a global workflow:rules configuration prevents the pipeline itself from running).
8. advanced_except_example Job:
   • This demonstrates the refs keyword within except.
   • refs: - /^legacy-.*$/ - archive: The job will be skipped if the ref is a branch starting with “legacy-” (regex match) OR if it’s the archive branch.
   • Complexity Note: As with only, when you start combining conditions or using regex with except, the rules: syntax often provides a clearer and more maintainable way to express the logic.

Key Concepts and Behavior:

• Blacklist: except defines when a job should not run. If any condition in except is met, the job is skipped.
• Multiple Conditions: If you list multiple items in the except array (e.g., except: [main, develop]), the job is skipped if the pipeline event matches any of those items (logical OR).
• Precedence: Similar to only, job-level except is evaluated after global workflow:rules.
• Counterpart only: only is a whitelist. You can use only and except together in the same job definition, but this is strongly discouraged as it can lead to very complex and hard-to-understand logic. rules: handles these scenarios much better.
• Special Keywords: The same special keywords available for only (like tags, merge_requests, schedules, branches, web, api, etc.) can also be used with except.

Recommendation: Use rules: Instead

It’s worth reiterating: GitLab now strongly recommends using rules: for controlling job execution due to its superior clarity and power.

Here’s a quick example of how build_on_feature_branches might look with rules:

YAML

build_on_feature_branches_with_rules:
  stage: build
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: never # Don't run if it's the main branch
    - when: on_success # Run in all other cases (e.g., feature branches)
  script:
    - echo "Building on a feature branch (using rules)"
    - echo "Current branch: $CI_COMMIT_REF_NAME"

rules: allow for more complex if conditions using CI/CD variables, support when: manual, when: delayed, allow_failure, exists (for checking file existence), and generally make your pipeline’s conditional logic much more explicit and maintainable.

While understanding except is useful, especially if you encounter it in older GitLab CI configurations, aim to use rules: for any new CI/CD pipeline development or when refactoring existing ones.

• stages (Global): A top-level keyword that defines all possible stages in your pipeline and their order of execution.
• stage (Job-level): A keyword used within a job definition to assign that job to one of the globally defined stages.

Example .gitlab-ci.yml with stages and stage:

YAML

# .gitlab-ci.yml

# 1. Global 'stages' definition:
# Defines the sequence of stages in the pipeline.
# Jobs in earlier stages must complete successfully before jobs in later stages can start.
stages:
  - build_assets  # First stage
  - run_tests     # Second stage
  - deploy_app    # Third stage

default:
  image: alpine:latest # Using a lightweight default image for our jobs

# --- Jobs for the 'build_assets' stage ---
compile_code:
  stage: build_assets # 2. Assigning this job to the 'build_assets' stage
  script:
    - echo "--- Compiling Code (Stage: build_assets) ---"
    - sleep 5 # Simulate work
    - echo "Code compilation complete."

package_application:
  stage: build_assets # This job is also in the 'build_assets' stage
  script:
    - echo "--- Packaging Application (Stage: build_assets) ---"
    - sleep 7 # Simulate work
    - echo "Application packaging complete."

# --- Jobs for the 'run_tests' stage ---
unit_tests:
  stage: run_tests # Assigning this job to the 'run_tests' stage
  script:
    - echo "--- Running Unit Tests (Stage: run_tests) ---"
    - sleep 10 # Simulate work
    - echo "Unit tests PASSED."

integration_tests:
  stage: run_tests # This job is also in the 'run_tests' stage
  script:
    - echo "--- Running Integration Tests (Stage: run_tests) ---"
    - sleep 12 # Simulate work
    - echo "Integration tests PASSED."
    # - exit 1 # Uncomment this line to simulate a test failure and see subsequent stages not run

# --- Job for the 'deploy_app' stage ---
deploy_to_production:
  stage: deploy_app # Assigning this job to the 'deploy_app' stage
  script:
    - echo "--- Deploying to Production (Stage: deploy_app) ---"
    - sleep 8 # Simulate work
    - echo "Deployment to production successful!"
  when: on_success # Default, but good to be explicit for deploy jobs

Explanation:

1. Global stages Definition:
   • stages: at the top level of your .gitlab-ci.yml file lists all the stages your pipeline will have.
   • Order Matters: The order in which you list the stages here dictates the sequence of execution for the entire pipeline.
     • In this example: build_assets will run first, then run_tests, and finally deploy_app.
   • If this global stages list is not defined, GitLab CI defaults to a set of predefined stages (build, test, deploy), and if a job doesn’t specify a stage, it defaults to test. However, it’s a strong best practice to always define your stages explicitly for clarity and control.
2. Job-Level stage: Keyword:
   • Inside each job definition (e.g., compile_code, unit_tests), the stage: keyword assigns that job to one of the stages previously listed in the global stages definition.
   • A job must belong to a stage that is defined in the global stages list. If you assign a job to a stage not listed globally, you’ll get a linting error.
3. Execution Flow:
   • Within a Stage (Parallel Execution):
     • All jobs belonging to the same stage can run in parallel if enough GitLab Runners are available.
     • For example, compile_code and package_application are both in the build_assets stage. They can start simultaneously.
     • Similarly, unit_tests and integration_tests (both in run_tests) can run in parallel.
   • Between Stages (Sequential Execution):
     • A stage will only start after all jobs in the preceding stage have completed successfully.
     • The run_tests stage (containing unit_tests and integration_tests) will only begin after both compile_code AND package_application from the build_assets stage have finished successfully.
     • The deploy_app stage (with deploy_to_production) will only begin after both unit_tests AND integration_tests from the run_tests stage have finished successfully.
   • Job Failure Impact:
     • If any job in a stage fails, by default:
       • No new jobs will start in that stage (though currently running parallel jobs in the same stage will continue unless interruptible is used or they depend on the failed job via needs).
       • All subsequent stages will be skipped, and the pipeline will be marked as failed.
       • In the example, if integration_tests fails (e.g., by uncommenting exit 1), the deploy_to_production job in the deploy_app stage will not run.
4. default: Section:
   • default: image: alpine:latest is just setting a default Docker image for all jobs to keep the example concise. It’s not directly related to staging but is good practice.

Benefits of Using Stages:

• Control Flow: Clearly defines the order of operations in your CI/CD process.
• Logical Grouping: Groups related tasks together (e.g., all build activities, all testing activities).
• Efficiency: Allows for parallel execution of independent tasks within the same phase of your pipeline.
• Failure Management: Prevents later stages (like deployment) from running if earlier critical stages (like testing) fail.
• Readability: Makes the pipeline structure easier to understand.

By defining stages and assigning jobs to them using stage:, you create a structured, ordered, and manageable workflow for your GitLab CI/CD pipelines.

Example .gitlab-ci.yml with script:

YAML

# .gitlab-ci.yml

default:
  image: alpine:latest # Using a lightweight default image for our jobs

stages:
  - build
  - test
  - report

# Job 1: Demonstrates basic single-line commands
build_project:
  stage: build
  script:
    - echo "--- Starting Project Build ---" # Simple command to print a message
    - echo "Current directory:"
    - ls -la # Lists files in the current directory
    - mkdir build_output # Creates a directory
    - echo "Build files would be generated here." > build_output/info.txt
    - echo "Project build process finished."

# Job 2: Demonstrates a sequence of commands and a multi-line command
run_tests:
  stage: test
  script:
    - echo "--- Running Tests ---"
    - echo "Setting up test environment..."
    - sleep 2 # Simulate some setup time
    - echo "Running test suite 1..."
    - echo "Test 1.1: PASSED"
    - echo "Test 1.2: PASSED"
    - | # This is a YAML block scalar, allowing for a multi-line script block
      echo "Running a multi-line command or a series of commands as one script step:"
      echo "  Detail 1: Checking component A"
      echo "  Detail 2: Verifying integration B"
      echo "Test suite 2 completed."
    - echo "All tests finished."

# Job 3: Demonstrates a script that might fail
generate_report:
  stage: report
  script:
    - echo "--- Generating Report ---"
    - echo "Creating report data..."
    - echo "This is a report" > report.txt
    - cat report.txt
    # - cat non_existent_file.txt # Uncomment this line to see the job fail
    - echo "Report generation successful (unless a command above failed)."

Explanation:

1. script: Keyword:
   • The script keyword is required for any job that needs to execute commands. If a job definition doesn’t have a script section, it might be considered a “no-op” job or a configuration error depending on other settings.
   • It takes an array of strings, where each string is a shell command.
2. Execution Environment:
   • The commands in the script section are executed by a shell (like sh, bash, or PowerShell depending on the GitLab Runner’s configuration and the specified Docker image’s default shell).
   • In this example, default:image: alpine:latest means all jobs will run inside a Docker container based on the alpine:latest image, and the commands will be run by Alpine’s sh.
3. build_project Job:
   • echo "--- Starting Project Build ---": A simple command to print a message to the job log.
   • ls -la: Lists files and directories in the job’s working directory.
   • mkdir build_output: Creates a new directory.
   • echo "Build files..." > build_output/info.txt: Creates a file with some content.
   • Sequential Execution: The commands are executed one after the other, in the order they are listed.
4. run_tests Job:
   • This job also shows a sequence of echo commands.
   • | (YAML Literal Block Scalar):
     • The | character at the start of the script block allows you to write multi-line strings more easily. Each line under it (preserving indentation) is treated as part of the same script command or a sequence of commands passed to the shell.
     • In this case, it’s used to group several echo statements that logically belong together for readability. Each of these echo commands is still executed individually by the shell.
     • You could also write a long single command that spans multiple lines in your YAML editor, and as long as it’s a valid shell command, it will work.
5. generate_report Job:
   • This job demonstrates a typical sequence.
   • Failure Handling: If any command in a script block exits with a non-zero status code (which typically indicates an error), GitLab CI will, by default:
     • Stop executing subsequent commands in that script block.
     • Mark the job as failed.
     • The pipeline might stop at that point, depending on its configuration (e.g., allow_failure settings on the job or stage configurations).
   • If you uncomment - cat non_existent_file.txt, the cat command will fail because the file doesn’t exist, causing the generate_report job to fail.

Key Concepts and Behavior:

• Heart of the Job: The script section defines the primary actions the job performs.
• Array of Commands: Always an array, even if there’s only one command. YAMLmy_job: script: "echo 'Single command still needs to be in an array or be the only item'" # Or more commonly: my_job: script: - "echo 'Single command'"
• Working Directory: Commands are executed in the context of the cloned repository, specifically $CI_PROJECT_DIR.
• Shell Interpreter: The specific shell used depends on the Docker image and the GitLab Runner configuration. You can often find out which shell is used by adding echo $SHELL or ps -p $$ to your script.
• Exit Codes: The success or failure of each command (and thus the job) is determined by its exit code. 0 usually means success, and any non-zero value means failure.
• Environment Variables: You can use predefined CI/CD variables (like $CI_PROJECT_DIR, $CI_COMMIT_SHA) and user-defined variables within your script commands.

The script section is where you’ll spend most of your time defining the logic for your CI/CD jobs, whether it’s building, testing, deploying, or automating any other task.

Example .gitlab-ci.yml for GitLab Pages:

This example will create a very simple static HTML page and deploy it.

YAML

# .gitlab-ci.yml

stages:
  - deploy # A stage for the pages deployment job

# This is the special job that GitLab Pages looks for.
# It must be named 'pages'.
pages:
  stage: deploy
  image: alpine:latest # A lightweight image is sufficient for this simple example

  script:
    - echo "--- Generating static site for GitLab Pages ---"
    # 1. Create the directory GitLab Pages expects for site content
    #    This directory MUST be named 'public'.
    - mkdir -p public

    # 2. Create some static content.
    #    In a real project, this step would involve running a static site generator
    #    (like Jekyll, Hugo, MkDocs, VuePress, Gatsby, or just copying HTML/CSS/JS files).
    - echo '<!DOCTYPE html>' > public/index.html
    - echo '<html lang="en">' >> public/index.html
    - echo '  <head>' >> public/index.html
    - echo '    <meta charset="UTF-8">' >> public/index.html
    - echo '    <meta name="viewport" content="width=device-width, initial-scale=1.0">' >> public/index.html
    - echo '    <title>My GitLab Pages Site</title>' >> public/index.html
    - echo '    <style>' >> public/index.html
    - echo '      body { font-family: sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; background-color: #f0f0f0; }' >> public/index.html
    - echo '      .container { text-align: center; padding: 20px; background-color: white; border-radius: 8px; box-shadow: 0 4px 8px rgba(0,0,0,0.1); }' >> public/index.html
    - echo '      h1 { color: #333; }' >> public/index.html
    - echo '      p { color: #555; }' >> public/index.html
    - echo '    </style>' >> public/index.html
    - echo '  </head>' >> public/index.html
    - echo '  <body>' >> public/index.html
    - echo '    <div class="container">' >> public/index.html
    - echo '      <h1>Hello from GitLab Pages!</h1>' >> public/index.html
    - echo "      <p>This site was deployed on: $(date)</p>" >> public/index.html
    - echo '      <p>Deployed from commit: <a href="${CI_PROJECT_URL}/-/commit/${CI_COMMIT_SHA}">${CI_COMMIT_SHORT_SHA}</a> on branch <em>${CI_COMMIT_REF_NAME}</em>.</p>' >> public/index.html
    - echo '    </div>' >> public/index.html
    - echo '  </body>' >> public/index.html
    - echo '</html>' >> public/index.html

    # You can create more files and subdirectories within 'public/'
    - mkdir -p public/about
    - echo "<h1>About Us</h1><p>This is the about page.</p><a href='../'>Home</a>" > public/about/index.html

    - echo "Static site content generated in 'public/' directory."

  artifacts:
    paths:
      # 3. This path MUST be 'public'. GitLab Pages serves files from this directory.
      - public
    expire_in: 1 week # Optional: How long to keep the job artifacts

  rules:
    # 4. Deploy GitLab Pages only when pushing to the default branch (e.g., main or master)
    #    Adjust $CI_DEFAULT_BRANCH if your default branch has a different name.
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      when: on_success # Run only if the script succeeds

Explanation:

1. Job Naming (pages):
   • The job responsible for deploying to GitLab Pages must be named pages. This is a special convention that GitLab recognizes.
2. stage: deploy:
   • While you can name your stage anything, deploy is a common and logical choice for a job that deploys a website.
3. image: alpine:latest:
   • For this very simple example of echoing HTML content, a lightweight image like alpine is sufficient.
   • In real-world scenarios, you would use an image that contains your static site generator (e.g., ruby for Jekyll, an image with Node.js for Gatsby/VuePress, python for MkDocs, or a specific image like klakegg/hugo).
4. script: Section:
   • mkdir -p public: This command creates a directory named public. All the static content (HTML, CSS, JavaScript, images, etc.) that you want to publish must be placed inside this public directory.
   • echo ... > public/index.html: These lines are a simple way to create a basic index.html file inside the public directory. In a real project, this section would contain commands to:
     • Install dependencies for your static site generator (e.g., npm install, bundle install, pip install -r requirements.txt).
     • Run the build command of your static site generator (e.g., hugo, jekyll build, npm run build, mkdocs build). This command typically outputs the static site into a specific folder (like _site, dist, build, site), which you would then need to copy or move into the public directory.
     • Example for a hypothetical generator: YAML# script: # - npm install # - npm run build # Assume this outputs to a './dist' folder # - rm -rf public # Clean up previous content if any # - mv dist public # Move the generated site to 'public'
5. artifacts: Section:
   • This is a critical part of the pages job.
   • paths: - public: This tells GitLab to take the entire contents of the public directory and save it as job artifacts. GitLab Pages then uses these specific artifacts from the latest successful pages job on the appropriate branch to serve your website.
   • expire_in: 1 week: (Optional) Specifies how long the job artifacts should be kept. For GitLab Pages, the artifacts are used for deployment and are not typically needed for long-term storage in the job artifacts section itself, as the deployed site persists.
6. rules: Section:
   • - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH': This rule ensures that the pages job (and thus the website deployment) only runs when changes are pushed or merged to the default branch of your repository (commonly main or master). This prevents deploying every commit from feature branches.
   • when: on_success: The job will only proceed if the script commands are successful.

How GitLab Pages Works with This Job:

1. When you commit changes to your default branch (as defined in the rules), the GitLab CI pipeline triggers.
2. The pages job runs.
3. The script section generates your static site content and places it in the public directory.
4. The artifacts section uploads the public directory.
5. GitLab takes these artifacts and deploys them to a web server.
6. Your static site becomes available at a URL typically structured like:
   • For projects under your username: https://<username>.gitlab.io/<projectname>
   • For projects within a group: https://<groupname>.gitlab.io/<projectname>
   • If you’ve configured a custom domain, it will be available at that domain.

You can find the exact URL for your GitLab Pages site in your project’s Settings > Pages section after the first successful deployment.

This basic example provides the fundamental structure. You would replace the echo commands with the actual build process for your chosen static site generator.

This example will demonstrate caching node_modules for a Node.js project, which is a very common use case.

Example .gitlab-ci.yml with cache:

YAML

# .gitlab-ci.yml

# 1. Global Cache Definition (applied to all jobs by default unless overridden)
#    Alternatively, you can place this under 'default:cache:'
cache:
  key: "$CI_COMMIT_REF_SLUG" # Creates a cache per branch/tag
  paths:
    - node_modules/ # Cache the node_modules directory
  policy: pull-push # Default policy: pull the cache at the start, push (update) at the end
  when: on_success # Default: save cache only if the job succeeds

stages:
  - setup
  - build
  - test

# Job 1: Installs dependencies and builds the project
# This job will create and push the cache on its first successful run.
# Subsequent runs on the same branch (if node_modules is still valid) will pull the cache.
install_and_build:
  stage: setup
  image: node:18-alpine
  script:
    - echo "--- Install and Build Job ---"
    - echo "CI_COMMIT_REF_SLUG is: $CI_COMMIT_REF_SLUG, this determines cache scope if not overridden by file key."
    - echo "Checking if node_modules directory exists from a previous cache restore..."
    - if [ -d "node_modules" ]; then echo "node_modules found, restored from cache."; else echo "node_modules not found, will run full npm install."; fi
    - npm install # This will be faster if node_modules is restored from cache
    - echo "Finished npm install."
    - echo "Running build command..."
    - npm run build # Assuming you have a 'build' script in your package.json
    - echo "Build complete."

# Job 2: Uses the cache created by the 'install_and_build' job (if on the same branch)
# This job might run tests and would benefit from not re-installing all dependencies.
run_tests:
  stage: test
  image: node:18-alpine
  # This job will inherit the global cache settings by default.
  # It will attempt to 'pull' the cache (including node_modules/) before running.
  script:
    - echo "--- Test Job ---"
    - echo "Checking if node_modules directory exists from cache..."
    - if [ -d "node_modules" ]; then echo "node_modules found, restored from cache for testing."; else echo "node_modules not found for testing. This might happen if 'install_and_build' failed or cache key differs."; fi
    - echo "Running tests..."
    - npm test # Assuming you have a 'test' script in your package.json
    - echo "Tests complete."

# Job 3: Demonstrates a more specific cache key based on a dependency file
# This is often a better strategy for dependency caching.
lint_code:
  stage: test
  image: node:18-alpine
  cache: # Job-level cache overrides the global cache setting for this job
    key:
      files:
        - package-lock.json # Cache is specific to the content of package-lock.json
      prefix: "$CI_PROJECT_NAME" # Optional prefix for the key
    paths:
      - node_modules/
    policy: pull # This job only pulls the cache, doesn't update it (useful for linters/tests)
  script:
    - echo "--- Lint Job ---"
    - echo "Cache key for this job is based on package-lock.json."
    - if [ ! -d "node_modules" ]; then npm install; else echo "node_modules found from cache."; fi
    - echo "Running linter..."
    - npm run lint # Assuming you have a 'lint' script

Explanation:

1. Global cache: Block:
   • This top-level cache: definition (or default:cache:) sets default caching behavior for all jobs in the pipeline.
   • key: "$CI_COMMIT_REF_SLUG":
     • The key is crucial. It identifies the cache. Jobs with the same cache key will share the same cache.
     • $CI_COMMIT_REF_SLUG is a predefined GitLab variable representing the branch or tag name in a URL-friendly format. This means each branch will have its own independent cache.
     • While simple, this key might not be optimal if dependencies don’t change often within a branch.
   • paths::
     • An array of files or directories to be cached.
     • - node_modules/: This tells GitLab to cache the contents of the node_modules directory.
   • policy: pull-push: (Default behavior if not specified)
     • pull: At the beginning of the job, GitLab attempts to download and extract the cache.
     • push: At the end of a successful job, GitLab creates a new cache archive with the specified paths and uploads it.
   • when: on_success: (Default behavior if not specified)
     • The cache will only be saved (pushed) if the job finishes successfully. Other options include on_failure and always.
2. install_and_build Job:
   • This job inherits the global cache settings.
   • First Run (or if cache doesn’t exist/key changes):
     • npm install will download all dependencies and create node_modules/.
     • After the script successfully completes, the node_modules/ directory will be archived and uploaded as a cache associated with the key (e.g., main if on the main branch).
   • Subsequent Runs (on the same branch, with the same cache key):
     • Before the script starts, GitLab will download and extract the node_modules/ cache.
     • npm install will then run. It should be much faster because most, if not all, dependencies are already present. It will only fetch updates or new packages.
3. run_tests Job:
   • This job also inherits the global cache settings.
   • If it runs on the same branch (and thus uses the same cache key) as a previously successful install_and_build job, it will pull the node_modules/ cache. This means it doesn’t need to run npm install from scratch (though the example does it for demonstration, in a real scenario you might skip npm install if node_modules exists and is valid).
4. lint_code Job (Job-Level Cache):
   • This job defines its own cache: block, which overrides the global cache settings specifically for this job.
   • key::
     • files: - package-lock.json: This is a more robust way to key your dependency cache. The cache key will be generated based on a checksum of the package-lock.json file. This means the cache is only reused if the exact dependencies haven’t changed. If package-lock.json changes, a new cache is created.
     • prefix: "$CI_PROJECT_NAME": (Optional) Adds a prefix to the generated key, which can help avoid key collisions if multiple projects share the same cache storage and use similar file names for keys.
   • paths: - node_modules/: Still caching node_modules.
   • policy: pull:
     • This job will only attempt to download (pull) an existing cache that matches its key.
     • It will not upload (push) or update the cache, even if its node_modules directory changes. This is often useful for jobs like linters or tests that consume dependencies but don’t modify them in a way that should update the shared cache.

Key Concepts and Behavior:

• Purpose: To speed up pipeline execution by reducing the need to re-download dependencies or re-generate files on every job run.
• How it Works:
  1. Restore (Pull): At the start of a job, GitLab Runner checks for a cache matching the job’s cache key. If found, it’s downloaded and extracted into the job’s working directory before the before_script runs.
  2. Save (Push): If the job’s cache policy includes push and the when condition is met (e.g., on_success), after the after_script completes, the runner archives the directories/files specified in paths and uploads them to the cache storage, associated with the cache key.
• Cache key is Critical:
  • A good key ensures cache validity and effectiveness.
  • key: files: is highly recommended for dependency caches as it ties the cache to the state of your dependency lock files.
  • You can also use predefined variables like $CI_COMMIT_REF_SLUG, $CI_JOB_IMAGE, or create composite keys.
• paths: Defines what gets cached. These paths are relative to the project directory ($CI_PROJECT_DIR).
• policy:
  • pull-push: Downloads at start, uploads at end.
  • pull: Only downloads at start.
  • push: Only uploads at end (useful for jobs that only produce a cache but don’t need to download one).
• when:
  • on_success: (Default) Cache is saved only if the job succeeds.
  • on_failure: Cache is saved only if the job fails.
  • always: Cache is saved regardless of job success or failure.
• Cache vs. Artifacts:
  • Cache: Meant for temporary storage to speed up jobs (e.g., node_modules, downloaded libraries). Not guaranteed to be available (runners might clear caches). Shared between subsequent runs of the same job or different jobs sharing the same cache key.
  • Artifacts: Meant for passing build outputs between stages/jobs or for downloading after a pipeline completes (e.g., compiled binaries, test reports). Guaranteed to be available for the duration defined by artifact expiry settings.
• Cache Scope:
  • By default, caches are stored per GitLab Runner.
  • For distributed caching (e.g., using S3), an administrator needs to configure it. This allows different runners to share the same cache.
• Clearing Cache: You can clear pipeline caches manually via the GitLab UI (CI/CD settings or on the pipeline view). Changing the cache key also effectively starts a new cache.

Using cache effectively can significantly reduce the runtime of your GitLab CI/CD pipelines, especially for projects with many dependencies.

This makes after_script ideal for tasks like:

• Cleaning up temporary files or resources.
• Logging final status or collecting diagnostic information.
• De-authenticating from services.
• Uploading artifacts even if the job failed.

You can define after_script at two main levels:

1. Globally (as a default): Using default:after_script:. These commands will run after every job unless a job overrides it.
2. Job-level: Defined within a specific job. This overrides any global after_script for that particular job.

Example .gitlab-ci.yml with after_script:

YAML

# .gitlab-ci.yml

# 1. Default after_script: Applied to all jobs unless overridden
default:
  image: alpine:latest # Using a common default image
  after_script:
    - echo "--- Default after_script (Global) ---"
    - echo "Job finished at: $(date)"
    - echo "Job status was: $CI_JOB_STATUS" # CI_JOB_STATUS reflects the script's outcome
    - echo "Cleaning up default resources..."

stages:
  - build
  - test
  - cleanup_test

# Job 1: Successful job that uses the default after_script
successful_job:
  stage: build
  before_script:
    - echo "Before script for successful_job"
  script:
    - echo "--- Main script for successful_job ---"
    - echo "This job will succeed."
    - exit 0 # Explicitly succeed

# Job 2: Failing job that also uses the default after_script
failing_job:
  stage: test
  before_script:
    - echo "Before script for failing_job"
  script:
    - echo "--- Main script for failing_job ---"
    - echo "This job will intentionally fail."
    - exit 1 # Explicitly fail

# Job 3: Job with a custom after_script (overrides the default)
job_with_custom_after_script:
  stage: cleanup_test
  after_script: # Job-level after_script overrides the default one
    - echo "--- Custom after_script for job_with_custom_after_script ---"
    - echo "Performing specific cleanup for this job."
    - echo "Job status was: $CI_JOB_STATUS"
  script:
    - echo "--- Main script for job_with_custom_after_script ---"
    - echo "This job uses its own custom after_script."

# Job 4: Job that disables after_script (even the default one)
job_without_any_after_script:
  stage: cleanup_test
  after_script: [] # Explicitly set to an empty array to disable
  # You could also use `after_script: null` in some GitLab versions.
  script:
    - echo "--- Main script for job_without_any_after_script ---"
    - echo "No after_script (neither default nor custom) will be executed for this job."

Explanation:

1. default:after_script:
   • The default: block allows setting default configurations.
   • after_script: within default: defines commands that execute after every job, regardless of whether the job passed or failed (unless the job has its own after_script or disables it).
   • In this example:
     • It prints introductory lines and the finish time.
     • It prints the CI_JOB_STATUS (a predefined variable indicating if the job’s main script succeeded, failed, etc.).
     • It simulates a default cleanup.
2. successful_job:
   • This job completes its script section successfully (exit 0).
   • It does not have its own after_script.
   • Therefore, it inherits and executes the commands from the default:after_script: section after its script block. The CI_JOB_STATUS will likely be success.
3. failing_job:
   • This job intentionally fails in its script section (exit 1).
   • It also does not have its own after_script.
   • Despite the failure, it will still inherit and execute the commands from the default:after_script: section. This demonstrates the key behavior of after_script. The CI_JOB_STATUS will likely be failed.
4. job_with_custom_after_script:
   • This job defines its own after_script: block.
   • This job-level after_script completely overrides (replaces) the default:after_script:. The commands from default:after_script: will not run for this job.
   • Only the commands listed in this job’s after_script will execute after its main script, regardless of the main script’s outcome.
5. job_without_any_after_script:
   • This job explicitly sets after_script: [] (an empty array).
   • This tells GitLab CI not to run any after_script commands for this job, not even the ones defined in the default:after_script: section.

Key Concepts and Behavior:

• Purpose: after_script is intended for cleanup, final logging, or any actions that must occur regardless of the job’s outcome.
• Execution Order: For any given job:
  1. before_script (if any) runs.
  2. Main script runs.
  3. after_script (if any) runs.
• Runs on Success or Failure: This is the most important characteristic. Commands in after_script are executed even if the before_script or script sections fail.
• Failure Handling in after_script:
  • The exit code of commands in after_script does not affect the overall job status. The job’s status is determined by the before_script and script sections.
  • However, if a command in after_script fails, you might see errors in the job log for the after_script phase.
• Array of Commands: after_script takes an array of strings, where each string is a shell command executed sequentially.
• Inheritance and Overriding:
  • Job-level after_script overrides default:after_script.
  • To prevent any after_script (including a global one) from running for a specific job, set after_script: [] or after_script: null.
• Execution Context:
  • after_script commands run on the same GitLab Runner that executed the job.
  • They generally have access to the same environment variables, files in the workspace (unless cleaned up by the job’s script), and services as the main script.
  • The CI_JOB_STATUS predefined variable is particularly useful in after_script to know the outcome of the job’s main execution.
• Timeout: after_script has its own separate timeout, which is configured by the GitLab administrator (usually shorter than the main job timeout).
• Common Use Cases:
  • Removing temporary files or directories.
  • Stopping services started during the job.
  • Logging out from authenticated sessions.
  • Uploading test reports or logs, especially if the job failed and artifacts wouldn’t be uploaded otherwise (though artifacts:when:always is often preferred for artifact uploading).
  • Sending notifications.

Using after_script ensures that critical cleanup and finalization steps are performed consistently, contributing to a more robust and manageable CI/CD pipeline.

• Logging information.
• Setting up environment variables.
• Installing dependencies.
• Authenticating to services.

You can define before_script at two main levels:

1. Globally (as a default): Using default:before_script:. These commands will run before every job unless a job overrides it.
2. Job-level: Defined within a specific job. This overrides any global before_script for that particular job.

Example .gitlab-ci.yml with before_script:

YAML

# .gitlab-ci.yml

# 1. Default before_script: Applied to all jobs unless overridden
default:
  image: alpine:latest # Using a common default image
  before_script:
    - echo "--- Default before_script (Global) ---"
    - echo "Current date: $(date)"
    - echo "Setting up common environment..."
    - export COMMON_SETUP_VAR="Set by default before_script"

stages:
  - setup
  - build
  - test

# Job 1: Uses the default before_script
job_with_default_before_script:
  stage: setup
  script:
    - echo "--- Main script for job_with_default_before_script ---"
    - echo "COMMON_SETUP_VAR is: $COMMON_SETUP_VAR"
    - echo "This job used the global before_script."

# Job 2: Overrides the default before_script with its own
job_with_custom_before_script:
  stage: build
  before_script: # Job-level before_script overrides the default one
    - echo "--- Custom before_script for job_with_custom_before_script ---"
    - echo "Performing specific setup for this job..."
    - apk add --no-cache curl # Example: installing a tool
    - curl --version
  script:
    - echo "--- Main script for job_with_custom_before_script ---"
    - echo "This job used its own custom before_script."
    - echo "COMMON_SETUP_VAR is: $COMMON_SETUP_VAR (This will be empty as default before_script didn't run)"

# Job 3: Disables before_script (even the default one)
job_without_any_before_script:
  stage: test
  before_script: [] # Explicitly set to an empty array to disable
  # You could also use `before_script: null` in some GitLab versions, but `[]` is safer.
  script:
    - echo "--- Main script for job_without_any_before_script ---"
    - echo "No before_script (neither default nor custom) was executed for this job."
    - echo "COMMON_SETUP_VAR is: $COMMON_SETUP_VAR (This will be empty)"

Explanation:

1. default:before_script:
   • The default: block allows you to set default configurations for all jobs.
   • before_script: within default: defines a list of commands that will be executed before the main script of every job in the pipeline, provided the job doesn’t have its own before_script or explicitly disables it.
   • In this example:
     • It prints some introductory lines.
     • It prints the current date.
     • It simulates setting up a common environment variable COMMON_SETUP_VAR.
2. job_with_default_before_script:
   • This job does not have its own before_script keyword.
   • Therefore, it inherits and executes the commands from the default:before_script: section before its own script block runs.
   • You’ll see the output from the default before_script, and the COMMON_SETUP_VAR will be available in its main script.
3. job_with_custom_before_script:
   • This job defines its own before_script: block.
   • This job-level before_script completely overrides (replaces) the default:before_script:. The commands from default:before_script: will not run for this job.
   • Only the commands listed in this job’s before_script (e.g., installing curl) will execute before its main script.
   • Consequently, COMMON_SETUP_VAR (set in the default before_script) will not be defined or will be empty when this job’s main script runs.
4. job_without_any_before_script:
   • This job explicitly sets before_script: [] (an empty array).
   • This tells GitLab CI not to run any before_script commands for this job, not even the ones defined in the default:before_script: section.
   • The job will proceed directly to its script block.
   • COMMON_SETUP_VAR will be undefined or empty.

Key Concepts and Behavior:

• Purpose: before_script is intended for setup tasks that prepare the environment for the main script. It’s not for the core logic of your job.
• Execution Order: For any given job, if a before_script is active (either default or job-specific), its commands run first. If all commands in before_script succeed, then the commands in the job’s script section are executed.
• Failure Handling: If any command in before_script fails (exits with a non-zero status), the job is typically marked as failed immediately. The main script section for that job will not be executed.
• Array of Commands: before_script takes an array of strings, where each string is a shell command to be executed sequentially.
• Inheritance and Overriding:
  • Job-level before_script overrides default:before_script.
  • To prevent any before_script (including a global one) from running for a specific job, set before_script: [] or before_script: null (though [] is generally preferred for clarity and consistency).
• Common Use Cases:
  • Printing debug information (versions of tools, environment variables).
  • Logging into private repositories or services.
  • Installing required software packages or dependencies that are not part of the Docker image.
  • Setting dynamic environment variables needed by the main script.
  • Fetching configuration files.

Using before_script helps keep your main script section clean and focused on the primary task of the job, while common or job-specific setup steps are neatly organized.