Skip to main content

Backup

Strategy#

Backup#

To fully backup the GitGuardian application, you need three things:

  • To backup the postgres
  • To backup the redis
  • To backup the Kots config

Backuping the embedded postgres and redis is possible (see below), but it is not the recommended way to run GitGuardian in production. We recommend using externally managed databases and datastores, and use the associated strategy for snapshots and backups.

Then, you will need to backup the KOTS config files. You can use these commands:

kubectl kots download -n <YOUR_NAMESPACE> gitguardian-seal --dest .
cp -r gitguardian-seal <BACKUP_LOCATION>

Do not edit this files manually.

Restoration#

To restore a GitGuardian instance, first restore the database and datastore.

Then, install a new KOTS instance, and configure it the most basic setup: embedded database, basic hostname... All these parameters will be replaced in the next step.

Finally, you can restore the manifests:

kubectl kots upload -n <YOUR_NAMESPACE> <SOURCE_FOLDER>

Connect to the admin console and deploy the uploaded version. You may have to edit the database's hosts.

KOTS Snapshots#

KOTS offers a mechanism to simplify the backups of your GitGuardian application.

This procedure has been tested on embedded cluster only.

Management#

Backups can be managed through the admin console. Backups can be launched manually or be scheduled to run daily, weekly, etc.

Backups are only used for the embedded postgres and the embedded redis. External databases are not backed up or managed.

Backups can be restored through the admin console for rollback purposes, or via CLI in case of a disaster recovery. See below for more information.

Types: full vs partial#

There are 2 types of snapshots: full and partial.

  • Partial snapshots only save the GitGuardian application data. It includes the embedded databases if you use them, your config set in the config page, and a copy of the Kubernetes' manifests for the GitGuardian application. They cannot be used on another instance.
  • Full snapshots include everything in the partial snapshots, but also the licence, the KOTS data (databases, manifests, support bundles...). They can be used on another instance (for example for a disaster recovery).

We recommend doing full snapshots, since they allow for both full and partial restores.

Storage#

By default, snapshots are only stored on the cluster, and they need to be exported. You can also choose to upload these to an AWS S3 bucket or another S3 compatible storage. More information on how to configure it is available here.

Schedule#

There is no default automatic backup schedule. We recommend to configure a schedule.

Usage#

These backups will mainly be used in two cases:

  • A cluster crash/wipe: The persistent volumes containing the data do not exist anymore, only backups have a recent copy of the data.
  • Rolling back the application if all else failed.

Postgres backups#

Postgres writes changes almost immediately to disk. In case of a crash, it should restart with all the data. Backups will be used in cases mentioned above. Postgres is backed up using the pg_dumpall command. The SQL file is then compressed and sent to the location of your choice. During restoration, using the "Restore" button on the admin console, an empty database is created, and the SQL dump file is applied to it.

Redis backups#

We are using the RDB persistence for Redis. Snapshots are done every 30s if there is at least 1 change over this period. In the event of a Redis crash, this snapshot will be used at Redis restart. During backups, the latest snapshot is exported to the location of your choice. During restoration, the backed up snapshot is copied, and Redis restarted.

KOTS Restore#

Via admin console#

The admin console allows you to perform a partial restore. It will only restore the GitGuardian application, not the KOTS/kURL dependencies. This is useful when the application is in a bad state after an update.

To do this, go to the "Snapshots" tab. Then you can use either the full snapshots or the partial snapshots. Go to the "Full Snapshots" or "Partial Snapshots" tab, depending on how you configured your backups. Choose the backup you want to restore. You can use the timestamp (underline in red in the screenshots below) to choose the correct one. Then click on the blue rotating arrows. A dialog will open up with instructions. With a full snapshot, you have 2 options, but only the partial restore can be done from the web interface, for the full restore, you will need to use the given command line.

Via CLI#

With the CLI, you can perform a full instance restore. This is useful to transfer GitGuardian to another instance, or for a disaster recovery.

The easiest way to get instructions for a restore is to go the snapshot tab, select the correct snapshot, and get the instructions by clicking on the two blue rotating arrows. Then, go to your instance, and run the command.

Example: Recreate a full embedded instance#

Here, we will describe an situation where we do a backup of GitGuardian on the instance A to restore it on the instance B.

First, on instance A, you need to create a directory owner by user 1001 and group 1001:

mkdir ${HOME}/kots-backupschown -R 1001:1001 ${HOME}/kots-backups

Then configure Velero to use this directory for snapshots with kots cli:

kubectl kots -n default velero configure-hostpath --with-minio=false --hostpath ${HOME}/kots-backups

And create a snapshot. Copy the backup name once it's finished:

kubectl kots backup

You will now need to copy the whole backup folder (${HOME}/kots-backups here) to instance B. To do it, we advise to create an archive (zip, tar...), and copy the archive:

# Instance Acd ${HOME}zip -r kots-backups.zip kots-backups
# Your laptop/workstationscp instance_a:kots-backups.zip .scp kots-backups.zip instance_b:.
# Instance Bcd ${HOME}unzip kots-backups.zipchown -R 1001:1001 ${HOME}/kots-backups

If it's not already done, install the embedded kubernetes cluster on instance B:

curl -sSL https://k8s.kurl.sh/gitguardian-seal-prod | sudo bash

Now you can configure Velero on this new kubernetes cluster:

kubectl kots -n default velero configure-hostpath --with-minio=false --hostpath ${HOME}/kots-backups

And finally restore the backup created on instance A:

kubectl kots restore --from-backup=<backup-name>

You can now change your DNS/reverse proxy/load balancer, to redirect traffic to the new instance.