Gitlab Pipeline – dependencies – What is dependencies in GitLab CI/CD?

DevOps

MOTOSHARE 🚗🏍️
Turning Idle Vehicles into Shared Rides & Earnings

From Idle to Income. From Parked to Purpose.
Earn by Sharing, Ride by Renting.
Where Owners Earn, Riders Move.
Owners Earn. Riders Move. Motoshare Connects.

With Motoshare, every parked vehicle finds a purpose. Owners earn. Renters ride.
🚀 Everyone wins.

Start Your Journey with Motoshare

Important Context on dependencies vs. needs:

  • dependencies traditionally controlled which artifacts from jobs in previous stages were downloaded before a job starts. It does not control the order of execution (that’s handled by stages or needs).
  • needs: is a more modern feature that allows you to create a Directed Acyclic Graph (DAG) of job execution. When a job uses needs: [job_A], it implies an execution dependency on job_A AND, by default, it will download artifacts from job_A.
  • Current Practice:
    • If you are using needs: to define relationships between jobs, you typically do not need to use dependencies to download artifacts from those needs-specified jobs, as needs: handles it.
    • The most common and clear use of dependencies in modern GitLab CI is dependencies: [] (an empty array), which means the job should not download artifacts from any jobs in previous stages.
    • If you are not using needs for a specific job relationship and want to selectively download artifacts from previous stages, then dependencies can be used.

This example will illustrate both the selective use of dependencies (when not using needs for that specific artifact flow) and the common dependencies: [] pattern.


Example .gitlab-ci.yml with dependencies:

YAML

# .gitlab-ci.yml
stages:
- build
- test
- package
default:
image: alpine:latest
# --- Build Stage Jobs (Producers) ---
build_code:
stage: build
script:
- echo "--- Building Main Code ---"
- mkdir -p build_output/code
- echo "main_application_binary" > build_output/code/app.bin
- echo "Build Code complete."
artifacts:
paths:
- build_output/code/
expire_in: 1 hour
build_assets:
stage: build
script:
- echo "--- Building Assets ---"
- mkdir -p build_output/assets
- echo "image.png" > build_output/assets/image.png
- echo "style.css" > build_output/assets/style.css
- echo "Build Assets complete."
artifacts:
paths:
- build_output/assets/
expire_in: 1 hour
# --- Test Stage Jobs (Consumers) ---
# Job 1: Selectively downloads artifacts ONLY from 'build_code'
# This is the "classic" use of dependencies when 'needs' isn't used for this relationship.
test_main_app:
stage: test
dependencies:
- build_code # Only download artifacts from 'build_code' job
script:
- echo "--- Testing Main Application ---"
- echo "Checking for 'build_code' artifacts:"
- ls -R build_output/
- if [ -f "build_output/code/app.bin" ]; then echo "app.bin found."; else echo "ERROR: app.bin NOT found!"; exit 1; fi
- if [ -f "build_output/assets/image.png" ]; then echo "WARNING: image.png found, but was not explicitly depended upon (might indicate default download if 'dependencies' was misconfigured or absent)."; else echo "image.png correctly NOT found (as not depended upon)."; fi
- echo "Main application tests complete."
# Job 2: Downloads NO artifacts from previous stages
# This is a very common modern use case for 'dependencies'.
test_assets_isolated:
stage: test
dependencies: [] # Explicitly download no artifacts
script:
- echo "--- Testing Assets in Isolation ---"
- echo "This job should start with a clean workspace (no artifacts from 'build' stage)."
- echo "Checking for any unexpected artifacts in 'build_output':"
- if [ -d "build_output" ]; then ls -R build_output/; echo "ERROR: 'build_output' directory exists! Artifacts were downloaded unexpectedly."; exit 1; else echo "'build_output' directory does not exist. Correct."; fi
# In a real scenario, this job might fetch assets differently or test their generation.
- echo "Asset tests (isolated) complete."
# Job 3: Default behavior - downloads artifacts from ALL successful jobs in previous stages
# This happens if 'dependencies' is omitted AND 'needs' is not used to link to these specific jobs.
test_with_all_artifacts_default:
stage: test
# No 'dependencies' or 'needs' keyword here
script:
- echo "--- Testing with All Default Artifacts ---"
- echo "This job should have artifacts from ALL successful jobs in the 'build' stage."
- ls -R build_output/
- if [ -f "build_output/code/app.bin" ] && [ -f "build_output/assets/image.png" ]; then echo "Both app.bin and image.png found. Correct default behavior."; else echo "ERROR: Expected artifacts not found!"; exit 1; fi
- echo "Tests with all artifacts complete."
# --- Package Stage Job ---
# This job uses 'needs' which handles artifact downloads from 'test_main_app'.
# 'dependencies' would typically not be used here in conjunction with 'needs' for 'test_main_app'.
package_application:
stage: package
needs: ["test_main_app"] # 'needs' implies artifacts from 'test_main_app' will be downloaded if 'test_main_app' produced them.
# 'test_main_app' itself only got artifacts from 'build_code'.
# If 'package_application' also needed artifacts directly from 'build_assets',
# you'd add 'build_assets' to 'needs' or use a different strategy.
script:
- echo "--- Packaging Application ---"
- echo "This job uses 'needs'. Artifacts from 'test_main_app' (which are originally from 'build_code') should be available."
- ls -R build_output/
- if [ -f "build_output/code/app.bin" ]; then echo "app.bin found for packaging."; else echo "ERROR: app.bin NOT found for packaging!"; exit 1; fi
- echo "Application packaging complete."
Code language: PHP (php)

Explanation:

  1. dependencies: Keyword:
    • Used within a job definition to control which artifacts from jobs in previous stages are downloaded into the job’s workspace before its before_script and script sections run.
    • It does NOT define execution order. The stages keyword defines the general order, and needs: defines specific job execution dependencies.
  2. test_main_app Job (Selective Dependencies):
    • dependencies: - build_code:
      • This tells GitLab CI to download artifacts only from the build_code job (which ran in the preceding build stage).
      • Artifacts from build_assets (also in the build stage) will not be downloaded for this test_main_app job.
      • This is useful when a job only requires a subset of artifacts from previous stages.
  3. test_assets_isolated Job (Empty Dependencies):
    • dependencies: [] (an empty array):
      • This is a special and very common instruction. It tells the job not to download any artifacts from any jobs in previous stages.
      • The job will start with a clean workspace concerning artifacts from prior stages. This is useful for jobs that should not be influenced by previous outputs or for jobs that manage their dependencies in a different way (e.g., pulling from a package registry).
  4. test_with_all_artifacts_default Job (Default Behavior):
    • This job omits the dependencies keyword and also does not use needs to link to specific jobs in the build stage.
    • In this scenario (no dependencies and no relevant needs), a job will, by default, download artifacts from all jobs in all previous stages that completed successfully and produced artifacts.
    • So, test_with_all_artifacts_default will get artifacts from both build_code AND build_assets.
  5. package_application Job (Interaction with needs):
    • This job uses needs: ["test_main_app"].
    • When needs is used, it implies that artifacts from the specified needs jobs will be downloaded by default (equivalent to needs: [{ job: "job_name", artifacts: true }]).
    • Therefore, package_application will download artifacts produced by test_main_app. Since test_main_app itself only downloaded artifacts from build_code, those are the artifacts that package_application will ultimately see from the build stage via this chain.
    • You generally do not specify dependencies for jobs already listed in needs because needs handles the artifact download. Using both can sometimes lead to downloading artifacts twice or having less clear intent.

Key Concepts:

  • Artifact Control (Pre-needs Era / Specific Cases): dependencies was the primary way to manage which artifacts get downloaded. If a job needed artifacts from job A and job B in a previous stage, but not job C, you’d list dependencies: [job_A, job_B].
  • Clean Workspace: dependencies: [] is the standard way to ensure a job starts clean without inheriting artifacts from previous stages.
  • Default Download: If dependencies is absent and needs isn’t defining the artifact flow from previous stages, artifacts from all previous stage jobs are downloaded. This can be resource-intensive if not intended.
  • No Impact on Order: dependencies only affects artifact downloading. It does not change the execution order of jobs or stages.
  • Superseded by needs for Linked Jobs: For jobs that have an explicit execution dependency defined by needs, the artifact flow from those needed jobs is also managed by needs (artifacts are downloaded by default).

In modern GitLab CI, you’ll often see needs used to define both the execution flow and the artifact flow for direct dependencies, and dependencies: [] used when a job needs a completely clean slate. The classic use of dependencies: [job_A, job_B] is less common if you’re heavily leveraging needs for your pipeline’s structure.

Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x