Skip to main content

Remediate

GitGuardian knows how damaging a potential data breach can be. Therefore if you face a secret incident, there is a remediation process that needs to be followed for any incident. The outline of this process is displayed in the right-hand section of the incident page as a reminder.

Remediation guidelines

Prelude#

Make sure assignee is declared and document your investigation#

Working in teams, the incident should be assigned to a member of the dashboard. This should help avoid duplication of work as the other members of the team would not run the same investigation in parallel. Plus, it is always best to get one person to follow the incident through to the end to ensure that it has been resolved correctly.

Incident assignee

Your investigation should be documented by team notes on the incident page. In fact, during your remediation process, you will come across internal information, obviously unknown by GitGuardian, that is important for the resolution of the incident. Keeping track of that information is crucial for you and your teammates, especially if you are tracking many incidents at the same time.

Put the developer in the loop#

The first remediation step after leaking a secret is to consult the developer responsible for the leak. In most cases, the developer that was handling the secret has the most knowledge about what that secret provides access to. Understanding what the secret provides access to will help establish the severity of the situation and guide you during the next stages.

Check out the Collaborate section to see how GitGuardian can help you achieve this easily.

The following steps are relevant for secret incidents:

1. Understand the implications of revoking the secret#

Of course, all leaked secrets should be revoked and this is generally the immediate impulse and reaction developers have when faced with such a mistake. You need to exercise control in this situation and consult senior developers before you take action. A bad situation can be made worse if a key is revoked without understanding how that key is currently being used.

2. Revoke the secret#

Revoking the secret is essential since once a secret has been leaked into a git repository you should consider it compromised, even if it was deleted immediately. This is because git, especially public git, can be easily monitored (eg: GitHub has a public API for all commits) and there is no guarantee that the secret has not already been picked up. It will also be of interest that even if your git repositories' history has been rewritten to remove a file, sensitive information can still exist in your git ref logs and will also still be stored on your VCS for some time.

2.1 Verify the secret's validity#

With the validity check, it is possible to verify if the exposed secret is still valid (thus exploitable) or not. For each incident, GitGuardian performs the least intrusive call possible to the associated service (host). The validity check can return 4 possible values:

  • valid: the secret can still be exploited and needs to be revoked and rotated ASAP ⚠️.
  • invalid: the secret has been likely revoked.
  • failed to check: GitGuardian was not able to determine the validity of the secret. It happens when it is not possible to distinguish the error caused by an invalid secret and an IP allowlist or another mechanism preventing the validity check. It can also mean the service provider is temporarily unavailable.
  • cannot check: the service provider or the secret type do not allow for checks by GitGuardian. This status is also displayed when secret validity checks are manually toggled off by the workspace admin (secret validity check settings).
  • unknown: GitGuardian has recently introduced a validity checker, this secret’s validity has not been verified yet.

Example of a valid secret

GitGuardian will regularly check for a secret's validity and display the date and time at which the last verification was performed. You can also manually run the validity check from the incident page should you need to force update the status.

GitGuardian also checks validity of secrets when running historical scans, unless the repository has not been modified since the last scan.

3. Delete evidence of the secret - Optional#

Whenever possible, all the occurrences of the secret incident shall be removed.

To achieve this, you can:

  • either delete your repository entirely.
  • or rewrite your git history if you have the possibility to do so (using git push --force or the BFG Repo-Cleaner).

3.1 Verify the secret's presence in repositories#

With the presence check, it is possible to verify the presence (and thus reachability) of every occurrence of the secret in your repositories. For each incident, GitGuardian will display the dates at which each occurrence was first and last seen and if it is still present in the repository.

Occurrences no longer visible in git

Once you have fully deleted the repository or rewritten the git history, return to your incident's page to obtain the secret's occurrence proof of deletion (see screenshot below). Once an occurrence is no longer visible in the git history, GitGuardian will stop checking for its presence.

Occurrences no longer visible in git

GitGuardian will regularly check for the presence of occurrences, to the best of its ability and considering existing VCS API rate limits, to mirror the state of your git repository. You can also manually run the presence check should you need to force update the status of the occurrences of an incident.

The frequency of these automated checks depends on your plan and the status and age of secret incidents:

PlanIncident statusIncident ageFrequency
BusinessOpenLess than a year oldDaily
FreeOpenLess than a year oldWeekly
BusinessOpenMore than a year oldWeekly
FreeOpenMore than a year oldMonthly
BusinessIgnoredLess than a year oldHalf-yearly
FreeIgnoredLess than a year oldNever
BusinessIgnoredMore than a year oldNever
FreeIgnoredMore than a year oldNever
BusinessResolvedLess than a year oldMonthly
FreeResolvedLess than a year oldHalf-yearly
BusinessResolvedMore than a year oldHalf-yearly
FreeResolvedMore than a year oldNever

If you host GitGuardian on-premise, you can change these frequencies in your admin area.

⚠️ Presence checks and orphan commits

When you delete a repository, all its commits are permanently deleted. However, using git push --force to remove a specific commit in a repository does not permanently delete it. It becomes an orphan commit, still existing but no longer referenced by any branch. It's impossible to see orphan commits in a pull request or in an existing branch unless you have access to the unique id (sha) of the commit. GitGuardian considers a secret's occurrence in an orphan commit as present. VCSs provide garbage collection, completely emptying the bin of orphan commits. In the case of:

  • GitHub, contacting the support to run the garbage collection is necessary
  • GitLab, the frequency of garbage collection is decided by the VCS admin

When you decide to unmonitor a repository, GitGuardian consequently loses access to that repository, which in turn prompts the presence check to lose its ability to update values with regards to its corresponding occurrences. GitGuardian will display an icon to indicate that the repository is no longer monitored.

For incidents containing occurrences related to more than one repository, the presence check will continue to work for the rest of the occurrences linked to the monitored sources until the last monitored source gets removed from the perimeter.

4. Review your access logs#

For secret incidents, it is important to review the log data and to check for suspicious activity:

  • Some secrets can lead to other secrets. For example, Slack tokens can give access to messages and shared files generally containing other secrets. GitHub tokens can give access to private repositories also containing secrets.
  • Depending on your findings, if part of your infrastructure or data has been further exposed, you may need to take additional mitigation actions.

✅ Resolve the incident on GitGuardian#

The final step is to mark the incident as resolved. GitGuardian gives you two options here:

  • This is sensitive and I revoked the secret
  • This is sensitive and I chose not to revoke the secret

If you choose not to revoke the secret, you can leave a note on the incident timeline to explain the reasons behind this choice.

tip

When trying to mark an incident as resolved, GitGuardian will fetch the status of the last validity check on the secret. If the secret is found to be valid, you will not be able to select the option "This is sensitive and I revoked the secret".

If you really want to mark the incident as resolved even if the secret is still valid, select the "This is sensitive and I chose not to revoke the secret" option.

For a deep-dive into secret leaks remediation, you can refer to this blog post illustrating remediation on GitHub.