Skip to main content

Remediate

GitGuardian knows how damaging a potential data breach can be. Therefore if you face a secret incident, there is a remediation process that needs to be followed for any incident. The outline of this process is displayed in the right-hand section of the incident page as a reminder.

Incident assignee and remediation outline

Prelude#

Make sure assignee is declared and document your investigation#

Working in teams, the incident should be assigned to a member of the dashboard. This should help avoid duplication of work as the other members of the team would not run the same investigation in parallel. Plus, it is always best to get one person to follow the incident through to the end to ensure that it has been resolved correctly.

Incident assignee and remediation outline

Your investigation should be documented by team notes on the incident page. In fact, during your remediation process, you will come across internal information, obviously unknown by GitGuardian, that is important for the resolution of the incident. Keeping track of that information is crucial for you and your teammates, especially if you are tracking many incidents at the same time.

Put the developer in the loop#

The first remediation step after leaking a secret is to consult the developer responsible for the leak. In most cases, the developer that was handling the secret has the most knowledge about what that secret provides access to. Understanding what the secret provides access to will help establish the severity of the situation and guide you during the next stages.

Check out the Collaborate section to see how GitGuardian can help you achieve this easily.

The following steps are relevant for secret incidents:

1. Understand the implications of revoking the secret#

Of course, all leaked secrets should be revoked and this is generally the immediate impulse and reaction developers have when faced with such a mistake. You need to exercise control in this situation and consult senior developers before you take action. A bad situation can be made worse if a key is revoked without understanding how that key is currently being used.

2. Revoke the secret#

Revoking the secret is essential since once a secret has been leaked into a git repository you should consider it compromised, even if it was deleted immediately. This is because git, especially public git, can be easily monitored (eg: GitHub has a public API for all commits) and there is no guarantee that the secret has not already been picked up. It will also be of interest that even if your git repositories' history has been rewritten to remove a file, sensitive information can still exist in your git ref logs and will also still be stored on your VCS for some time.

2.1 Verify the secret's validity#

With the validity check, it is possible to verify if the exposed secret is still valid (thus exploitable) or not. For each incident, GitGuardian performs the least intrusive call possible to the associated service (host). The validity check can return 4 possible values:

  • valid: the secret can still be exploited and needs to be revoked and rotated ASAP ⚠️.
  • invalid: the secret has been likely revoked.
  • cannot check: the service provider or the secret type do not allow for checks by GitGuardian.
  • unknown: GitGuardian has recently introduced a validity checker, this secret’s validity has not been verified yet.

Example of a valid secret

GitGuardian will regularly check for a secret's validity and display the date and time at which the last verification was performed. You can also manually run the validity check from the incident page should you need to force update the status.

3. Delete evidence of the secret#

Whenever possible, all the occurrences of the secret incident shall be removed.

To achieve this, you can:

  • either delete your repository entirely.
  • or rewrite your git history if you have the possibility to do so (using git push --force or the BFG Repo-Cleaner).

3.1 Verify the secret's presence in git repositories#

With the presence check, it is possible to verify the presence (and thus reachability) of every occurrence of the secret in your git repositories. For each incident, GitGuardian will display the dates at which each occurrence was first and last seen and if it is still present in the git repository.

Occurrences no longer visible in git

GitGuardian will regularly check for the presence of occurrences, to the best of its ability and considering existing VCS API rate limits, to mirror the state of your git repository. You can also manually run the presence check should you need to force update the status of the occurrences of an incident.

Once you have fully deleted the repository or rewritten the git history, return to your incident's page to obtain the secret's occurrence proof of deletion (see screenshot below). Once an occurrence is no longer visible in the git history, GitGuardian will stop checking for its presence.

Occurrences no longer visible in git

⚠️ Presence checks and orphan commits

When you delete a repository, all its commits are permanently deleted. However, using git push --force to remove a specific commit in a repository does not permanently delete it. It becomes an orphan commit, still existing but no longer referenced by any branch. It's impossible to see orphan commits in a pull request or in an existing branch unless you have access to the unique id (sha) of the commit. GitGuardian considers a secret's occurrence in an orphan commit as present. VCSs provide garbage collection, completely emptying the bin of orphan commits. In the case of:

  • GitHub, contacting the support to run the garbage collection is necessary
  • GitLab, the frequency of garbage collection is decided by the VCS admin

4. Review your access logs#

For secret incidents, it is important to review the log data and to check for suspicious activity:

  • Some secrets can lead to other secrets. For example, Slack tokens can give access to messages and shared files generally containing other secrets. GitHub tokens can give access to private repositories also containing secrets.
  • Depending on your findings, if part of your infrastructure or data has been further exposed, you may need to take additional mitigation actions.

✅ Resolve the incident on GitGuardian#

The final step is to mark the incident as resolved. GitGuardian lets you indicate if the secret could be revoked or not during the remediation process. This is important information that your team needs to be aware of.

For a deep-dive into secret leaks remediation, you can refer to this blog post illustrating remediation on GitHub.