Skip to main content

Bring Your Own Sources

Bring Your Own Sources lets you scan and monitor secrets from any custom data source that isn’t natively supported by GitGuardian’s integrations. Use it to extend GitGuardian’s secret detection to CI logs, legacy systems, local filesystems, FTP servers, and any other place secrets might appear.

Prerequisites

Bring Your Own Sources is available on Business and Enterprise plans.
Only users with the Owner or Manager role can create custom sources.

How It Works​

With Bring Your Own Sources, connect any data source to GitGuardian in minutes:

  1. Set up a custom source in the GitGuardian dashboard or via the GitGuardian API.
  2. Scan your custom sources using ggshield, simple scripts, or an automation tool.
    Incidents are automatically created and linked to your custom integration.
  3. Filter, view, and manage incidents from your custom sources just like with native integrations in the GitGuardian dashboard.

Setting Up a Custom Source​

Step 1: Create a Custom Integration​

  1. Navigate to Settings > Integrations > Sources in your GitGuardian dashboard.
  2. From the Secrets scanning tab, in the Custom Sources section, click Add Custom Source. Find Custom Sources
  3. Give your source a clear name (required) and description (optional). Create Custom Source
  4. Click Create Integration. Once saved, note the generated UUID; you’ll need it shortly.

Alternatively, you can create your custom source using the custom-sources endpoint of our Public API. You will need a dedicated Service Account Token with the scan and scan:create-incidents permissions to proceed.

Upon creation, you'll receive:

  • A unique integration ID to attach findings and declare incidents to your custom integration. You can view it at any time after creation.
  • Instructions for setting up a service account.

Custom Source Created

Step 2: Configure a Service Account​

  1. Create a dedicated Service Account with the following permissions: scan and scan:create-incidents.
    This allows scanning and creating incidents from your custom sources.
  2. Configure your environment with the necessary credentials, depending on your internal processes (for example, environment variables or a direct connection to your secrets manager, etc.):
export GITGUARDIAN_API_KEY="your-service-account-token"
export GITGUARDIAN_INSTANCE="https://api.gitguardian.com"  # For SaaS US
# export GITGUARDIAN_INSTANCE="https://api.eu1.gitguardian.com"  # For SaaS EU
# export GITGUARDIAN_INSTANCE="https://your-instance.com/exposed"  # For self-hosted
Security note

We recommend creating a dedicated service account for each custom source integration to maintain proper access control and audit trails.

Scanning Custom Sources using ggshield​

You can use our CLI, ggshield, to scan for exposed secrets in files, directories, build logs, and CI/CD artifacts.

Create incidents by providing your integration’s unique ID so findings are attached to your custom data source in the GitGuardian dashboard.
Here are a few examples to get you started.
Use ggshield secret scan --help to get more details.

Scan files​

ggshield secret scan path /path/to/file.txt --source-uuid YOUR_INTEGRATION_ID

Scan directories​

ggshield secret scan path -r /path/to/directory --source-uuid YOUR_INTEGRATION_ID

Scan Jenkins build logs​

ggshield secret scan path /var/jenkins_home/jobs/*/builds/*/log --source-uuid YOUR_INTEGRATION_ID

Scan GitLab CI artifacts​

find /builds -name "*.log" -exec ggshield secret scan path {} --source-uuid YOUR_INTEGRATION_ID \+

Scan application logs​

ggshield secret scan path /var/log/legacy-app.log --source-uuid YOUR_INTEGRATION_ID

Scan Docker images​

ggshield secret scan docker ubuntu:22.04 --source-uuid YOUR_INTEGRATION_ID

Scanning Custom Sources using an automation workflow (for example, n8n)​

You can also use your existing automation tool or simple scripts to scan content from any of your data sources.
Here is a sample scenario using n8n to send GitHub Issues content to GitGuardian via a GitHub webhook.

Step 1: Set Up GitGuardian Authentication​

  1. Once your custom source is created, set up a Header Auth credential in n8n.
  2. Provide both values:
  • Header Name: Authorization
  • Value: Bearer <GG_TOKEN_VALUE> (replace with your GitGuardian token)

Step 2: Trigger the Workflow with a GitHub Webhook​

  1. Add a Webhook node as the trigger.
  2. Copy its test URL, then go to your GitHub repository’s settings.
  3. Under Webhooks, add the n8n URL and set it to trigger on “Issues” events.

Step 3: Send Issues to GitGuardian’s API​

  1. Add an HTTP node in n8n.
  2. Configure it to send a POST request to https://api.gitguardian.com/v1/scan/create-incidents using your Header Auth credential.
  3. For the request body (JSON), use:
{
  "source_uuid": "my_source",
  "documents": [
    {
      "filename": "{{ $json.body.issue.url }}",
      "document": {{ JSON.stringify($json.body.issue.body) }}
    }
  ]
}
  1. Replace "my_source" with your actual source UUID.
  2. Test your workflow by clicking Execute workflow in n8n, then create an Issue in your GitHub repository—try including a mock secret. Seconds later, you should see your first security incident appear in your GitGuardian dashboard.
  3. Go live: activate the workflow.

Voilà! You now have real-time secrets detection running on your GitHub Issues.

Filter, view, and manage incidents​

Custom sources and their incidents appear in the GitGuardian dashboard just like native integrations.

Filtering and Organization​

Use dashboard filters to organize incidents:

  • Integration: Select the custom source integration.
  • Source: Filter by the source name provided within your integration.
  • Source type: Select Custom Source.
  • Standard filters: Apply date, severity, and status filters as usual.

Filter per Custom Source

Viewing Custom Source Incidents​

All incidents created from custom sources appear in the Incidents and Incident Details pages:

  • Secret preview in the list. Custom Source Incident Preview
  • Full incident details and context.
  • Source identification showing your custom integration name.
  • Standard incident management capabilities (assign, resolve, ignore, etc.).

Custom Source Incident Detail

Perimeter Visibility​

Custom sources appear in the Monitored Perimeter with:

  • Disabled status (cannot be scanned from the dashboard).
  • Integration name and description.

Custom Source in Perimeter

Considerations​

Current Limitations​

  • File size: Individual files are limited to 50 MB due to API limits.
  • Sources per integration: Each custom integration supports one source.
  • Batch scanning: Large datasets should be scanned in batches for optimal performance.

Performance Considerations​

  • API rate limits: The GitGuardian API limits batches to 20 documents per call.
  • Large directories: Scanning directories with more than 20 files will result in multiple API calls.
  • Network timeouts: Consider network stability for large scanning operations.

Future Enhancements​

  • Support for multiple sources per integration.
  • Increased file size limits.
  • Enhanced batch processing capabilities.

Pro Tips and Best Practices​

Security​

  • Use a dedicated Service Account Token for each custom source.
  • Rotate Service Account Tokens regularly.
  • Apply the principle of least privilege for token permissions.
  • Avoid storing tokens in plaintext files.

Performance​

  • Scan files in batches rather than individually when possible.
  • Use the --recursive flag judiciously with large directory trees.
  • Monitor API usage to avoid rate limiting.
  • Consider scheduling scans during off-peak hours for large datasets.

Organization​

  • Use descriptive names for custom integrations.
  • Add meaningful descriptions to provide context for team members.
  • Implement consistent naming conventions across custom sources.
  • Document your custom scanning workflows for team knowledge sharing.

Troubleshooting​

Common Issues​

Authentication Errors​

  • Verify your Service Account Token has the scan and scan:create-incidents permissions.
  • Ensure the GitGuardian API key has been properly configured for ggshield or your custom script.
  • Check that your instance URL is correct for SaaS or self-hosted deployments.

Integration ID Not Found​

  • Confirm the integration ID is correctly set in your ggshield command or custom script.
  • Verify the integration is active and not deleted.

File Size Errors​

  • Files larger than 50 MB will be skipped.
  • Use the --verbose flag to see which files are skipped.
  • Consider splitting large files into smaller chunks.

Network Connectivity​

  • Verify network connectivity to your GitGuardian instance.
  • Check firewall rules and proxy configurations.
  • Refer to the connectivity troubleshooting guide.

Getting Help​

For additional support:

Last updated on

Something we didn’t cover?

See our Roadmap

Subscribe on GitHub

Submit a request

API status