Why a Patch-Upload Test Passes But the Same Merge-Request Test Fails on Drupal.org

This article is based on a bug report originally filed in the Drupal.org issue queue. The source attribution is drupal.org.

The Problem

A confusing class of CI failures can occur on Drupal.org: the exact same code change passes automated tests when submitted as a patch file, but fails when the same change is pushed to an issue fork and a merge request is opened. Because the diff is identical, the failure is not obviously a code problem — it points instead to a difference in the environment or dependency-resolution process used by each testing path.

Complicating Factors That Trigger This Behaviour

This issue most commonly surfaces when the module under test has one or more of the following characteristics:

• A sub-module with additional Composer dependencies, including dependencies on other contributed modules.
• A test class in that sub-module that has a compile-time (class-loading) dependency on a contributed module — for example, using a PHP trait or extending a class defined in the dependency.

The canonical example mentioned in the original report is the Salesforce module, which contains a sub-module called salesforce_jwt. That sub-module defines a test class that uses a PHP trait provided by the Key contributed module. When the Key module is not present in the test environment, PHP cannot load the test class at all, causing the test run to fail — even before any test assertions are evaluated.

Why the Two Paths Produce Different Results

Drupal.org’s automated testing infrastructure handles patch-based submissions and merge-request-based submissions through related but not necessarily identical build pipelines. The key differences that can explain the divergence are:

• Composer dependency resolution: A patch is applied on top of a fully installed project. Depending on when and how the patch pipeline runs composer install, the cross-contrib dependency (e.g., Key module) may already be present. A merge-request pipeline may clone the issue fork and resolve dependencies differently, potentially omitting a contributed module that is listed only in a sub-module’s composer.json or as a require-dev entry.
• Working directory and project root: The two pipelines may set up the Drupal root differently, affecting how sub-module paths are resolved during the Composer install step.
• Test discovery scope: If the contributed dependency is absent, PHP will throw a fatal error when the test runner tries to load the test class, causing the entire test suite to fail rather than skipping the specific test.

Steps to Reproduce

The following sequence reliably demonstrates the discrepancy for modules with cross-contrib test dependencies:

1. Choose a Drupal contributed module that has on-commit tests and a sub-module whose tests depend on another contributed module at the PHP class level.
2. Create an issue fork on Drupal.org for an open issue against that module.
3. Push a new commit to the issue fork branch.
4. Open a merge request targeting the module’s main branch. Observe that the automated tests fail.
5. Export the same commit as a patch file (e.g., via git diff or git format-patch) and upload it to the same issue. Observe that the automated tests pass.

Diagnosing Whether This Is a Local or Systemic Issue

Before filing a bug against the Drupal.org CI infrastructure, it is worth ruling out configuration problems in the module itself:

• Check your composer.json files at every level. Ensure that cross-contrib dependencies needed by sub-module tests are listed in require-dev (or require if needed at runtime) in the sub-module’s own composer.json, and that the root module’s composer.json references the sub-module path correctly.
• Review the CI pipeline logs for both runs. Compare the Composer install output between the patch-triggered job and the merge-request-triggered job. Look for missing packages or version mismatches.
• Check for PHP fatal errors in the test output. A class-not-found fatal during test discovery will look like a test failure but is actually a dependency error. The stack trace will reference the missing trait or class.
• Reproduce locally by simulating both build processes: a clean composer install from scratch versus applying a patch to an already-installed project.

Proposed Resolution

The ideal fix is for Drupal.org’s CI infrastructure to use the same build and dependency-resolution process for both patch-upload tests and merge-request tests, so that identical code always produces identical results. Until that parity exists, maintainers can work around the issue by:

• Ensuring all cross-contrib dependencies are explicitly declared and discoverable by Composer in both pipeline contexts.
• Adding a .gitlab-ci.yml override (if the project uses GitLab CI on Drupal.org) that explicitly installs required contributed modules before running tests.
• Conditionally skipping tests at runtime (using $this->markTestSkipped()) if an optional dependency is not available, rather than relying on the presence of the class at PHP parse time — this prevents fatal load errors from failing unrelated tests.

Scope of Changes

This is a CI environment and tooling issue. It does not require any changes to:

• User interface
• Public API
• Data model

Remaining Work

The primary open question is whether this is a systemic issue affecting all modules with cross-contrib sub-module test dependencies, or a local configuration problem specific to how a particular module declares its dependencies. Gathering data from other maintainers who observe the same patch-passes / merge-request-fails discrepancy would help confirm which it is.