Skip to content

GitLab

Secret Detection (FREE)

A recurring problem when developing applications is that developers may unintentionally commit secrets and credentials to their remote repositories. If other people have access to the source, or if the project is public, the sensitive information is then exposed and can be leveraged by malicious users to gain access to resources like deployment environments.

GitLab 11.9 includes a new check called Secret Detection. It scans the content of the repository to find API keys and other information that should not be there.

GitLab displays identified secrets visibly in a few places:

Secret Detection in merge request widget

Use cases

  • Detecting unintentional commit of secrets like keys, passwords, and API tokens.
  • Performing a single or recurring scan of the full history of your repository for secrets.

Supported secrets

Secret Detection detects a variety of common secrets by default. You can also customize the secret detection patterns using custom rulesets.

The default ruleset provided by TruffleHog and Gitleaks includes the following key types:

  • Cloud services:
    • Amazon Web Services (AWS)
    • Google Cloud Platform (GCP)
    • Heroku API
  • Encryption keys:
    • PKCS8
    • RSA
    • SSH
    • PGP
    • DSA
    • EC
  • Social media platforms:
    • Facebook API
    • Twitter API
  • Cloud SaaS vendors:
    • GitHub API
    • Shopify API
    • Slack Token
    • Slack Webhook
    • Stripe API
    • Twilio API
    • Generic API key strings starting with api-
  • Password in URL
  • U.S. Social Security Number

WARNING: Gitleaks does not support scanning binary files.

Requirements

To run Secret Detection jobs, by default, you need GitLab Runner with the docker or kubernetes executor. If you're using the shared runners on GitLab.com, this is enabled by default.

WARNING: Our Secret Detection jobs expect a Linux container type. Windows containers are not supported.

WARNING: If you use your own runners, make sure the Docker version installed is not 19.03.0. See troubleshooting information for details.

Making Secret Detection available to all GitLab tiers

To make Secret Detection available to as many customers as possible, we have enabled it for all GitLab tiers. However not all features are available on every tier. See the breakdown below for more details.

Summary of features per tier

Different features are available in different GitLab tiers, as shown in the following table:

Capability In Free & Premium In Ultimate
Configure Secret Detection Scanners {check-circle} {check-circle}
Customize Secret Detection Settings {check-circle} {check-circle}
View JSON Report {check-circle} {check-circle}
Presentation of JSON Report in Merge Request {dotted-circle} {check-circle}
View identified secrets in the pipelines' Security tab {dotted-circle} {check-circle}
Interaction with Vulnerabilities {dotted-circle} {check-circle}
Access to Security Dashboard {dotted-circle} {check-circle}

Configuration

  • In GitLab 13.1, Secret Detection was split from the SAST configuration into its own CI/CD template. If you're using GitLab 13.0 or earlier and SAST is enabled, then Secret Detection is already enabled.
  • In GitLab 14.0, Secret Detection jobs secret_detection_default_branch and secret_detection were consolidated into one job, secret_detection.

Secret Detection is performed by a specific analyzer during the secret-detection job. It runs regardless of your app's programming language.

The Secret Detection analyzer includes Gitleaks checks.

Note that the Secret Detection analyzer ignores Password-in-URL vulnerabilities if the password begins with a dollar sign ($), as this likely indicates the password is an environment variable. For example, https://username:$password@example.com/path/to/repo isn't detected, while https://username:password@example.com/path/to/repo is.

NOTE: You don't have to configure Secret Detection manually as shown in this section if you're using Auto Secret Detection, provided by Auto DevOps.

To enable Secret Detection for GitLab 13.1 and later, you must include the Secret-Detection.gitlab-ci.yml template that's provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined in that template.

Ensure your .gitlab-ci.yml file has a stage called test, and add the following to your .gitlab-ci.yml file:

include:
  - template: Security/Secret-Detection.gitlab-ci.yml

The included template creates Secret Detection jobs in your CI/CD pipeline and scans your project's source code for secrets.

The results are saved as a Secret Detection report artifact that you can later download and analyze. Due to implementation limitations, we always take the latest Secret Detection artifact available.

Enable Secret Detection via an automatic merge request (ULTIMATE SELF)

To enable Secret Detection in a project, you can create a merge request from the Security Configuration page.

  1. In the project where you want to enable Secret Detection, go to Security & Compliance > Configuration.
  2. In the Secret Detection row, select Configure via Merge Request.

This automatically creates a merge request with the changes necessary to enable Secret Detection that you can review and merge to complete the configuration.

Customizing settings

The Secret Detection scan settings can be changed through CI/CD variables by using the variables parameter in .gitlab-ci.yml.

To override a job definition, (for example, change properties like variables or dependencies), declare a job with the same name as the SAST job to override. Place this new job after the template inclusion and specify any additional keys under it.

WARNING: Beginning in GitLab 13.0, the use of only and except is no longer supported. When overriding the template, you must use rules instead.

GIT_DEPTH

The GIT_DEPTH CI/CD variable affects Secret Detection. The Secret Detection analyzer relies on generating patches between commits to scan content for secrets. If you override the default, ensure the value is greater than 1. If the number of commits in an MR is greater than the GIT_DEPTH value, Secret Detection will fail to detect secrets.

Custom settings example

In the following example, we include the Secret Detection template and at the same time we override the secret_detection job with the SECRET_DETECTION_HISTORIC_SCAN CI/CD variable to true:

include:
  - template: Security/Secret-Detection.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_HISTORIC_SCAN: "true"

Because the template is evaluated before the pipeline configuration, the last mention of the variable takes precedence.

Available CI/CD variables

Secret Detection can be customized by defining available CI/CD variables:

CI/CD variable Default value Description
SECRET_DETECTION_COMMIT_FROM - The commit a Gitleaks scan starts at.
SECRET_DETECTION_COMMIT_TO - The commit a Gitleaks scan ends at.
SECRET_DETECTION_EXCLUDED_PATHS "" Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, doc,spec ). Parent directories also match patterns. Introduced in GitLab 13.3.
SECRET_DETECTION_HISTORIC_SCAN false Flag to enable a historic Gitleaks scan.

Custom rulesets (ULTIMATE)

Introduced in GitLab 13.5.

You can customize the default secret detection rules provided with GitLab. Customization allows replace the default secret detection rules with rules that you define.

To create a custom ruleset:

  1. Create a .gitlab directory at the root of your project, if one doesn't already exist.

  2. Create a custom ruleset file named secret-detection-ruleset.toml in the .gitlab directory.

  3. In the secret-detection-ruleset.toml file, do one of the following:

    • Define a custom ruleset:

      [secrets]
  description = 'secrets custom rules configuration'

  [[secrets.passthrough]]
    type  = "raw"
    target = "gitleaks.toml"
    value = """\
title = "gitleaks config"
# add regexes to the regex table
[[rules]]
description = "Test for Raw Custom Rulesets"
regex = '''Custom Raw Ruleset T[est]{3}'''
"""

    • Provide the name of the file containing a custom ruleset:

      [secrets]
  description = 'secrets custom rules configuration'

  [[secrets.passthrough]]
    type  = "file"
    target = "gitleaks.toml"
    value = "config/gitleaks.toml"

Logging level

To control the verbosity of logs set the SECURE_LOG_LEVEL CI/CD variable. Messages of this logging level or higher are output. Introduced in GitLab 13.1.

From highest to lowest severity, the logging levels are:

  • fatal
  • error
  • warn
  • info (default)
  • debug

Post-processing and revocation

Introduced in GitLab 13.6.

Upon detection of a secret, GitLab supports post-processing hooks. These can be used to take actions like notifying the cloud service who issued the secret. The cloud provider can confirm the credentials and take remediation actions like revoking or reissuing a new secret and notifying the creator of the secret. Post-processing workflows vary by supported cloud providers.

GitLab currently supports post-processing for following service providers:

  • Amazon Web Services (AWS)

Third party cloud and SaaS providers can express integration interest by filling out this form. Learn more about the technical details of post-processing secrets.

NOTE: Post-processing is currently limited to a project's default branch, see the above epic for future efforts to support additional branches.

sequenceDiagram
    autonumber
    Rails->>+Sidekiq: gl-secret-detection-report.json
    Sidekiq-->+Sidekiq: Ci::BuildFinishedWorker
    Sidekiq-->+RevocationAPI: GET revocable keys types
    RevocationAPI-->>-Sidekiq: OK
    Sidekiq->>+RevocationAPI: POST revoke revocable keys
    RevocationAPI-->>-Sidekiq: ACCEPTED
    RevocationAPI-->>+Cloud Vendor: revoke revocable keys
    Cloud Vendor-->>+RevocationAPI: ACCEPTED

Full History Secret Scan

GitLab 12.11 introduced support for scanning the full history of a repository. This new functionality is particularly useful when you are enabling Secret Detection in a repository for the first time and you want to perform a full secret scan. Running a secret scan on the full history can take a long time, especially for larger repositories with lengthy Git histories. We recommend not setting this CI/CD variable as part of your normal job definition.

A new configuration variable (SECRET_DETECTION_HISTORIC_SCAN) can be set to change the behavior of the GitLab Secret Detection scan to run on the entire Git history of a repository.

We have created a short video walkthrough showcasing how you can perform a full history secret scan.

Running Secret Detection in an offline environment

For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required for the Secret Detection job to run successfully. For more information, see Offline environments.

Requirements for offline Secret Detection

To use Secret Detection in an offline environment, you need:

  • GitLab Runner with the docker or kubernetes executor.
  • A Docker Container Registry with locally available copy of Secret Detection analyzer images.
  • Configure certificate checking of packages (optional).

GitLab Runner has a default pull policy of always, meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner pull_policy can be set to if-not-present in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to always if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines.

Make GitLab Secret Detection analyzer image available inside your Docker registry

Import the following default Secret Detection analyzer images from registry.gitlab.com into your local Docker container registry:

registry.gitlab.com/security-products/secret-detection:3

The process for importing Docker images into a local offline Docker registry depends on your network security policy. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. These scanners are periodically updated with new definitions, and you may be able to make occasional updates on your own.

For details on saving and transporting Docker images as a file, see Docker's documentation on docker save, docker load, docker export, and docker import.

If support for Custom Certificate Authorities are needed

Support for custom certificate authorities was introduced in the following versions.

Analyzer Version
secrets v3.0.0

To trust a custom Certificate Authority, set the ADDITIONAL_CA_CERT_BUNDLE variable to the bundle of CA certs that you want to trust in the SAST environment. The ADDITIONAL_CA_CERT_BUNDLE value should contain the text representation of the X.509 PEM public-key certificate. For example, to configure this value in the .gitlab-ci.yml file, use the following:

variables:
  ADDITIONAL_CA_CERT_BUNDLE: |
      -----BEGIN CERTIFICATE-----
      MIIGqTCCBJGgAwIBAgIQI7AVxxVwg2kch4d56XNdDjANBgkqhkiG9w0BAQsFADCB
      ...
      jWgmPqF3vUbZE0EyScetPJquRFRKIesyJuBFMAs=
      -----END CERTIFICATE-----

The ADDITIONAL_CA_CERT_BUNDLE value can also be configured as a custom variable in the UI, either as a file, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate.

Set Secret Detection CI/CD variables to use local Secret Detection analyzer

Add the following configuration to your .gitlab-ci.yml file. You must replace SECURE_ANALYZERS_PREFIX to refer to your local Docker container registry:

include:
  - template: Security/Secret-Detection.gitlab-ci.yml

variables:
  SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers"

The Secret Detection job should now use local copies of the Secret Detection analyzer to scan your code and generate security reports without requiring internet access.

Troubleshooting

Getting warning message gl-secret-detection-report.json: no matching files

For information on this, see the general Application Security troubleshooting section.

Error: Couldn't run the gitleaks command: exit status 2

If a pipeline is triggered from a Merge Request containing 60 commits while the GIT_DEPTH variable is set to 50 (a project default), the Secret Detection job fails as the clone is not deep enough to contain all of the relevant commits.

To confirm this as the cause of the error, set the logging level to debug, then rerun the pipeline. The logs should look similar to the following example. The text "object not found" is a symptom of this error.

ERRO[2020-11-18T18:05:52Z] object not found
[ERRO] [secrets] [2020-11-18T18:05:52Z] ▶ Couldn't run the gitleaks command: exit status 2
[ERRO] [secrets] [2020-11-18T18:05:52Z] ▶ Gitleaks analysis failed: exit status 2

To resolve the issue, set the GIT_DEPTH CI/CD variable to a higher value. To apply this only to the Secret Detection job, the following can be added to your .gitlab-ci.yml file:

secret_detection:
  variables:
    GIT_DEPTH: 100