Skip to content
GitLab

CI configuration internals

Workflow rules

Pipelines for the GitLab project are created using the workflow:rules keyword feature of the GitLab CI/CD.

Pipelines are always created for the following scenarios:

  • main branch, including on schedules, pushes, merges, and so on.
  • Merge requests.
  • Tags.
  • Stable, auto-deploy, and security branches.

Pipeline creation is also affected by the following CI/CD variables:

  • If $FORCE_GITLAB_CI is set, pipelines are created.
  • If $GITLAB_INTERNAL is not set, pipelines are not created.

No pipeline is created in any other cases (for example, when pushing a branch with no MR for it).

The source of truth for these workflow rules is defined in .gitlab-ci.yml.

Default image

The default image is defined in .gitlab-ci.yml.

It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick.

The images used in our pipelines are configured in the gitlab-org/gitlab-build-images project, which is push-mirrored to gitlab/gitlab-build-images for redundancy.

The current version of the build images can be found in the "Used by GitLab section".

Default variables

In addition to the predefined CI/CD variables, each pipeline includes default variables defined in .gitlab-ci.yml.

Stages

The current stages are:

  • sync: This stage is used to synchronize changes from gitlab-org/gitlab to gitlab-org/gitlab-foss.
  • prepare: This stage includes jobs that prepare artifacts that are needed by jobs in subsequent stages.
  • build-images: This stage includes jobs that prepare Docker images that are needed by jobs in subsequent stages or downstream pipelines.
  • fixtures: This stage includes jobs that prepare fixtures needed by frontend tests.
  • lint: This stage includes linting and static analysis jobs.
  • test: This stage includes most of the tests, and DB/migration jobs.
  • post-test: This stage includes jobs that build reports or gather data from the test stage's jobs (for example, coverage, Knapsack metadata, and so on).
  • review: This stage includes jobs that build the CNG images, deploy them, and run end-to-end tests against Review Apps (see Review Apps for details). It also includes Docs Review App jobs.
  • qa: This stage includes jobs that perform QA tasks against the Review App that is deployed in stage review.
  • post-qa: This stage includes jobs that build reports or gather data from the qa stage's jobs (for example, Review App performance report).
  • pages: This stage includes a job that deploys the various reports as GitLab Pages (for example, coverage-ruby, and webpack-report (found at https://gitlab-org.gitlab.io/gitlab/webpack-report/, but there is an issue with the deployment).
  • notify: This stage includes jobs that notify various failures to Slack.

Dependency Proxy

Some of the jobs are using images from Docker Hub, where we also use ${GITLAB_DEPENDENCY_PROXY_ADDRESS} as a prefix to the image path, so that we pull images from our Dependency Proxy. By default, this variable is set from the value of ${GITLAB_DEPENDENCY_PROXY}.

${GITLAB_DEPENDENCY_PROXY} is a group CI/CD variable defined in gitlab-org as ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/. This means when we use an image defined as:

image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:edge

Projects in the gitlab-org group pull from the Dependency Proxy, while forks that reside on any other personal namespaces or groups fall back to Docker Hub unless ${GITLAB_DEPENDENCY_PROXY} is also defined there.

Work around for when a pipeline is started by a Project access token user

When a pipeline is started by a Project access token user (e.g. the release-tools approver bot user which automatically updates the Gitaly version used in the main project), the Dependency proxy isn't accessible and the job fails at the Preparing the "docker+machine" executor step. To work around that, we have a special workflow rule, that overrides the ${GITLAB_DEPENDENCY_PROXY_ADDRESS} variable so that Dependency proxy isn't used in that case:

- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $GITLAB_USER_LOGIN =~ /project_\d+_bot\d*/'
  variables:
    GITLAB_DEPENDENCY_PROXY_ADDRESS: ""

NOTE: We don't directly override the ${GITLAB_DEPENDENCY_PROXY} variable because group-level variables have higher precedence over .gitlab-ci.yml variables.

Common job definitions

Most of the jobs extend from a few CI definitions defined in .gitlab/ci/global.gitlab-ci.yml that are scoped to a single configuration keyword.

Job definitions Description
.default-retry Allows a job to retry upon unknown_failure, api_failure, runner_system_failure, job_execution_timeout, or stuck_or_timeout_failure.
.default-before_script Allows a job to use a default before_script definition suitable for Ruby/Rails tasks that may need a database running (for example, tests).
.setup-test-env-cache Allows a job to use a default cache definition suitable for setting up test environment for subsequent Ruby/Rails tasks.
.rails-cache Allows a job to use a default cache definition suitable for Ruby/Rails tasks.
.static-analysis-cache Allows a job to use a default cache definition suitable for static analysis tasks.
.coverage-cache Allows a job to use a default cache definition suitable for coverage tasks.
.qa-cache Allows a job to use a default cache definition suitable for QA tasks.
.yarn-cache Allows a job to use a default cache definition suitable for frontend jobs that do a yarn install.
.assets-compile-cache Allows a job to use a default cache definition suitable for frontend jobs that compile assets.
.use-pg13 Allows a job to use the postgres 13 and redis services (see .gitlab/ci/global.gitlab-ci.yml for the specific versions of the services).
.use-pg13-ee Same as .use-pg13 but also use an elasticsearch service (see .gitlab/ci/global.gitlab-ci.yml for the specific version of the service).
.use-pg14 Allows a job to use the postgres 14 and redis services (see .gitlab/ci/global.gitlab-ci.yml for the specific versions of the services).
.use-pg14-ee Same as .use-pg14 but also use an elasticsearch service (see .gitlab/ci/global.gitlab-ci.yml for the specific version of the service).
.use-kaniko Allows a job to use the kaniko tool to build Docker images.
.as-if-foss Simulate the FOSS project by setting the FOSS_ONLY='1' CI/CD variable.
.use-docker-in-docker Allows a job to use Docker in Docker.

rules, if: conditions and changes: patterns

We're using the rules keyword extensively.

All rules definitions are defined in rules.gitlab-ci.yml, then included in individual jobs via extends.

The rules definitions are composed of if: conditions and changes: patterns, which are also defined in rules.gitlab-ci.yml and included in rules definitions via YAML anchors

if: conditions

if: conditions Description Notes
if-not-canonical-namespace Matches if the project isn't in the canonical (gitlab-org/) or security (gitlab-org/security) namespace. Use to create a job for forks (by using when: on_success or when: manual), or not create a job for forks (by using when: never).
if-not-ee Matches if the project isn't EE (that is, project name isn't gitlab or gitlab-ee). Use to create a job only in the FOSS project (by using when: on_success or when: manual), or not create a job if the project is EE (by using when: never).
if-not-foss Matches if the project isn't FOSS (that is, project name isn't gitlab-foss, gitlab-ce, or gitlabhq). Use to create a job only in the EE project (by using when: on_success or when: manual), or not create a job if the project is FOSS (by using when: never).
if-default-refs Matches if the pipeline is for master, main, /^[\d-]+-stable(-ee)?$/ (stable branches), /^\d+-\d+-auto-deploy-\d+$/ (auto-deploy branches), /^security\// (security branches), merge requests, and tags. Note that jobs aren't created for branches with this default configuration.
if-master-refs Matches if the current branch is master or main.
if-master-push Matches if the current branch is master or main and pipeline source is push.
if-master-schedule-maintenance Matches if the current branch is master or main and pipeline runs on a 2-hourly schedule.
if-master-schedule-nightly Matches if the current branch is master or main and pipeline runs on a nightly schedule.
if-auto-deploy-branches Matches if the current branch is an auto-deploy one.
if-master-or-tag Matches if the pipeline is for the master or main branch or for a tag.
if-merge-request Matches if the pipeline is for a merge request.
if-merge-request-title-as-if-foss Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-as-if-foss"
if-merge-request-title-update-caches Matches if the pipeline is for a merge request and the MR has label ~"pipeline:update-cache".
if-merge-request-title-run-all-rspec Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-all-rspec".
if-security-merge-request Matches if the pipeline is for a security merge request.
if-security-schedule Matches if the pipeline is for a security scheduled pipeline.
if-nightly-master-schedule Matches if the pipeline is for a master scheduled pipeline with $NIGHTLY set.
if-dot-com-gitlab-org-schedule Limits jobs creation to scheduled pipelines for the gitlab-org group on GitLab.com.
if-dot-com-gitlab-org-master Limits jobs creation to the master or main branch for the gitlab-org group on GitLab.com.
if-dot-com-gitlab-org-merge-request Limits jobs creation to merge requests for the gitlab-org group on GitLab.com.
if-dot-com-gitlab-org-and-security-tag Limits job creation to tags for the gitlab-org and gitlab-org/security groups on GitLab.com.
if-dot-com-gitlab-org-and-security-merge-request Limit jobs creation to merge requests for the gitlab-org and gitlab-org/security groups on GitLab.com.
if-dot-com-gitlab-org-and-security-tag Limit jobs creation to tags for the gitlab-org and gitlab-org/security groups on GitLab.com.
if-dot-com-ee-schedule Limits jobs to scheduled pipelines for the gitlab-org/gitlab project on GitLab.com.

changes: patterns

changes: patterns Description
ci-patterns Only create job for CI configuration-related changes.
ci-build-images-patterns Only create job for CI configuration-related changes related to the build-images stage.
ci-review-patterns Only create job for CI configuration-related changes related to the review stage.
ci-qa-patterns Only create job for CI configuration-related changes related to the qa stage.
yaml-lint-patterns Only create job for YAML-related changes.
docs-patterns Only create job for docs-related changes.
frontend-dependency-patterns Only create job when frontend dependencies are updated (for example, package.json, and yarn.lock) changes.
frontend-patterns-for-as-if-foss Only create job for frontend-related changes that have impact on FOSS.
backend-patterns Only create job for backend-related changes.
db-patterns Only create job for DB-related changes.
backstage-patterns Only create job for backstage-related changes (that is, Danger, fixtures, RuboCop, specs).
code-patterns Only create job for code-related changes.
qa-patterns Only create job for QA-related changes.
code-backstage-patterns Combination of code-patterns and backstage-patterns.
code-qa-patterns Combination of code-patterns and qa-patterns.
code-backstage-qa-patterns Combination of code-patterns, backstage-patterns, and qa-patterns.
static-analysis-patterns Only create jobs for Static Analytics configuration-related changes.

Best Practices

When to use extends:, <<: *xyz (YAML anchors), or !reference

Reference

Key takeaways

  • If you need to extend a hash, you should use extends
  • If you need to extend an array, you'll need to use !reference, or YAML anchors as last resort
  • For more complex cases (e.g. extend hash inside array, extend array inside hash, ...), you'll have to use !reference or YAML anchors

What can extends and YAML anchors do?

extends
  • Deep merge for hashes
  • NO merge for arrays. It overwrites (source)
YAML anchors
  • NO deep merge for hashes, BUT it can be used to extend a hash (see the example below)
  • NO merge for arrays, BUT it can be used to extend an array (see the example below)

A great example

This example shows how to extend complex YAML data structures with !reference and YAML anchors:

.strict-ee-only-rules:
  # `rules` is an array of hashes
  rules:
    - if: '$CI_PROJECT_NAME !~ /^gitlab(-ee)?$/ '
      when: never

# `if-security-merge-request` is a hash
.if-security-merge-request: &if-security-merge-request
  if: '$CI_PROJECT_NAMESPACE == "gitlab-org/security"'

# `code-qa-patterns` is an array
.code-qa-patterns: &code-qa-patterns
  - "{package.json,yarn.lock}"
  - ".browserslistrc"
  - "babel.config.js"
  - "jest.config.{base,integration,unit}.js"

.qa:rules:as-if-foss:
  rules:
    # We extend the `rules` array with an array of hashes directly
    - !reference [".strict-ee-only-rules", rules]
    # We extend a single array entry with a hash
    - <<: *if-security-merge-request
      # `changes` is an array, so we pass it an entire array
      changes: *code-qa-patterns

qa:selectors-as-if-foss:
  # We include the rules from .qa:rules:as-if-foss in this job
  extends:
    - .qa:rules:as-if-foss