Aller au contenu principal

ClickHouse

GitGuardian uses ClickHouse, an open-source column-oriented database, to power features that require fast analytics over large volumes of data (for example, machine and endpoint secret occurrence analytics). PostgreSQL remains the primary datastore for the rest of the application; ClickHouse is only involved for the specific features that need it.

Mandatory from 2027.1

Starting with release 2027.1.0, ClickHouse and its object storage backend become mandatory for every self-hosted deployment. See the customer notice for what changes and how to prepare.

Deployment model

ClickHouse runs as a dedicated, in-cluster stateful component, and relies on external object storage (S3-compatible, Azure Blob Storage, or Google Cloud Storage) for durable data storage. It is not a fully externally managed database like PostgreSQL or Redis. See ClickHouse storage for details.

Default topology

ClickHouse is deployed as a single node. This is the simplest and lightest-footprint option, and is sufficient for the vast majority of deployments, though it is not highly available: there is no other replica to fail over to.

Supported version

GitGuardian pins the ClickHouse version shipped with each release to the same version running our SaaS offering, and validates it against our test suite before release. Do not attempt to run a different ClickHouse version than the one shipped with your release.

Storage

ClickHouse splits its storage into three parts: local metadata and filesystem cache volumes, and object storage holding the actual table data, where the bulk of the storage footprint lives and where clickhouse-backup writes to when backing up.

See ClickHouse storage for the local volumes and provider-specific object storage configuration, Sizing and hardening for hardware sizing, Backup and restore for the backup runbook, and Monitoring and alerting to catch a failing backup that pod health won't show you.

Required configuration

A production-ready ClickHouse deployment requires all of the following to be configured:

What to configureDetailsWhere
Compute sizing and hardeningNode & instance resources, scheduling, node stabilitySizing and hardening
ClickHouse credentialsInstance passwordPrerequisites
Object storage configurationS3 / Azure Blob / GCS backend, disk authenticationStorage → Configuration
In-cluster volumes configurationMetadata & cache volumes, storage classStorage → Local volumes
Backup configurationDestination bucket, credentials secret, schedule & retentionBackup and restore

Prerequisites

Before installing, the ClickHouse admin password and the backup sidecar's local API password should be passed via a Kubernetes secret rather than set inline in values, the same pattern used throughout GitGuardian self-hosted. It can be created directly or synced from your own secret store. See Helm sensitive information management for how to populate it.

If you create it directly:

kubectl create secret generic clickhouse-credentials \
--namespace <namespace> \
--from-literal=admin-password=<a-strong-password> \
--from-literal=api-password=<a-strong-password> \
--from-literal=api-username=<a-username>
All three keys are required: don't rename or omit any of them

clickhouse-credentials, admin-password, api-password, and api-username are wired into the chart's defaults under those exact names: admin-password via clickhouse-server.auth.existingSecret/existingSecretKey (GitGuardian derives its own CLICKHOUSE_PASSWORD from the same secret), and api-password/api-username consumed directly by the backup sidecar and its ServiceMonitor. Missing or renaming any of the three breaks ClickHouse's own authentication, the backup sidecar, or GitGuardian's connection to it, unless you also override the corresponding values.

With the secret in place, enable ClickHouse itself:

clickhouse:
enabled: true

This alone isn't enough for a working deployment: ClickHouse's object storage backend has no default and must be configured for your provider (see ClickHouse storage), and the backup destination needs the same before backups can run (see Backup and restore). Follow both pages before considering ClickHouse installed.