Generic database assignment

Description

General

The generic database assignment detector aims at catching any quadruple host, port, username, and password that are database credentials for which it wasn't possible to infer the database type.

This statement is pretty wide, therefore to avoid raising many false alerts, GitGuardian has come up with a range of validation steps and specifications to refine the perimeter to look at.

Specifications

The four components of the quadruple that the detector catches are referred as host, port, username, and password. The detector keeps only the combination of matched element that form a quadruple which are the closest matches inside the document. Another version of this detector exists for cases where the port is attached to the host name.

For this detector, each element must follow a specific set of rules to be considered as sensitive and therefore valid:

For all:

• The document must contain the string db or database (see whitelist hereunder).
• Must be an assigned value except for the port, namely of the form {assigned_variable} {assignment_token} {value}, where {value} is either host, username, or password. The port can be either an assigned value or present in the host (for example my_host:some_port).

host

• The document must contain the string host (see whitelist hereunder).
• Caught hosts should be sensitive ones as well as their corresponding {assigned_variable}. Therefore, a set of common hosts are banned such as localhost, test/example hosts, or dummy IPs such as 1.2.3.4 and host assigned variables such as proxy (see banlist hereunder).

port

• The document must contain the string port (see whitelist hereunder).
• Caught ports should be sensitive ones as well as their corresponding {assigned_variable}. Therefore, a set of common ports are banned such as 8080 and port assigned variables such as support (see banlist hereunder).

username

• The document must contain the string user (see whitelist hereunder).
• Caught usernames should be sensitive ones as well as their corresponding {assigned_variable}. Therefore, a set of common usernames are banned such as db_user and username assigned variables such as user-agent (see banlist hereunder).

password

• The document must contain the string pwd or pass (see whitelist hereunder).
• Caught passwords should be sensitive ones as well as their corresponding {assigned_variable}. Therefore, a set of common password are banned such as encrypted or hashed ones and password assigned variables such as getpass (see banlist hereunder).

Revoke the secret

This detector catches generic database credentials, hence GitGuardian cannot infer the type of database concerned. To properly revoke the secret:

1. Understand what type of database is concerned.
2. Refer to the corresponding database documentation to know how to revoke and rotate the credentials.

Example

Examples that WILL be caught

- text: >
    DB CONTEXT
    host=mongo.com
    port=5434
    username=root
    password=m42ploz2wd
  host: mongo.com
  port: '5434'
  username: root
  password: m42ploz2wd

- text: >
    db_host=mongo.com
    db_port=5434
    db_username=root
    db_password=m42ploz2wd
  host: mongo.com
  port: '5434'
  username: root
  password: m42ploz2wd

- text: >
    dbhost=real.database.com
    dbport=5434
    dbuser=pilal
    dbpass=yourock93
  host: real.database.com
  port: '5434'
  username: pilal
  password: yourock93

- text: >
    DB CONTEXT
    host=my.mongo.com:27017
    username=root
    password=m42ploz2wd
  host: my.mongo.com
  port: '27017'
  username: root
  password: m42ploz2wd

- text: >
    dbhost=my.mongo.com:27017
    dbuser=root
    dbpwd=m42ploz2wd
  host: my.mongo.com
  port: '27017'
  username: root
  password: m42ploz2wd

Examples that WILL NOT be caught

• Host name is not a sensitive one.

- text: >
    db_host=localhost   # host not sensitive
    db_port=5434
    db_username=root
    db_password=m42ploz2wd

• The IP is not sensitive.

- text: >
    DB CONTEXT
    host=mongo.com
    port=1.1.1.1        # dummy IP
    username=root
    password=m42ploz2wd

• The username is not a sensitive one.

- text: >
    dbhost=real.database.com
    dbport=5434
    dbuser=db_user      # wrong username
    dbpass=yourock93

• The password is hashed.

- text: >
    dbhost=my.mongo.com:27017
    dbuser=root
    dbpwd=sha256_mypassword-hashed # hashed password

Details for Generic database assignment

• High Recall: False

• Validity Check: False

• Minimum Number of Matches: 3

• Occurrences found for one million commits: 244

• Prefixed: False

• PreValidators:

- type: FilenameBanlistPreValidator
- type: ContentWhitelistPreValidator
  patterns: ['db', 'database']
- type: ContentWhitelistPreValidator
  patterns: ['pwd', 'pass']
- type: ContentWhitelistPreValidator
  patterns:
    - host
- type: ContentWhitelistPreValidator
  patterns:
    - user
- type: ContentWhitelistPreValidator
  patterns:
    - port

• PostValidators

host:
  - type: CommonValueBanlistPostValidator
  - type: CommonHostBanlistPostValidator
  - type: ValueBanlistPostValidator
    patterns:
      - 'smtp\.'
      - localhost
      - 'this\.'
      - 'example\.com$'
      - 'mail\.'
      - 'self\.'
      - '\.java'
      - 'local\.'
      - 'process\.env'
      - 'config'
      - 'test'
      - '\.hostname'
      - 'host\.'
      - '\.host$'
      - '\.env'
      - 'env\.'
      - 'settings'
      - 'string'
      - 'default'
      - 'args\.'
      - '^com\.'
      - 'error'
      - 'request'
      - '(\d{1,3}).\1.\1.\1' # Rejects dummy IPs like 1.1.1.1
      - '\.ip$'
      - 'grafana'
      - '^api.weixin'
      - 'foobar'
      - 'x{1,3}\.x{1,3}\.x{1,3}\.x{1,3}'
      - '1\.2\.3\.4'
      - 'www\.google\.com'
      - 'bing\.com'
  - type: AssignmentBanlistPostValidator
    patterns:
      - 'allowed_hosts'
      - '\.localhost'
      - '^localhost$'
      - 'trusted[_.-]?host'
      - 'http'
      - 'proxy'
      - 'redis'
      - 'mongo'
      - 'm[sy]sql'
      - 'postgres'
      - 'ftp'
      - 'smtp'
      - 'zookeeper'
      - 'ldap'
      - 'mail'

password:
  - type: CommonValueBanlistPostValidator
  - type: CommonPasswordBanlistPostValidator
  - type: ValueBanlistPostValidator
    patterns:
      - 'encrypted'
      - 'false'
      - 'true'
      - 'self'
      - '__vault__'
      - 'test1234'
      - 'abcd1234'
      - 'nil'
      - 'hidden'
      - 'string'
      - '(\d)\1{4,}' #repeating digit 5 times or more
      - 'get_env'
      - '\.env'
      - 'env[.(]'
      - '^test$'
      - 'args\.'
      - 'error'
      - 'request'
      - '\.pem$'
      - '^buf$'
      - 'pg[_.-]?pass'
      - 'fs\.read'
      - 'required'
      - '^masked$'
      - '^hashed$'
      - '^secured'
      - 'removed$'
      - '^None'
      - '^The$'
      - '^\.\.\.$'
      - 'models\.'
      - 'sha256'
      - 'md5'
      - '^some-?pass$'
      - '^getpass\.'
      - 'password'
      - '^array$'
      - 'crypted'
      - 'credential'
      - '^_?pwd,?$'
      - '^null,?$'
      - '^isnull'
      - 'username'
      - '^user$'
      - '^host[,=]'
      - 'dbhost'
      - 'config'
      - 'noreply'
      - '\*\*\*\*'
      - 'optional'
      - 'database'
      - 'await'
      - 'function'
      - 'encode'
      - '[,:\(\)]$'
      - '\);$'
      - '^,'
      - '(?-i:^[A-Z_]*$)'
  - type: HeuristicPostValidator
    filters:
      - file_path
      - file_name
  - type: AssignmentBanlistPostValidator
    patterns:
      - 'proxy'
      - 'redis'
      - 'mongo'
      - 'm[sy]sql'
      - 'postgres'
      - 'ftp'
      - 'smtp'
      - 'zookeeper'
      - 'ldap'
      - 'mail'
      - 'getpass\.'

username:
  - type: CommonValueBanlistPostValidator
  - type: CommonUsernameBanlistPostValidator
  - type: ValueBanlistPostValidator
    patterns:
      - 'db_user'
      - 'self'
      - 'true'
      - 'false'
      - '__vault__'
      - '^[\*x]+$'
      - '^null$'
      - 'userinfo'
      - 'test'
      - 'nil'
      - 'string'
      - '^str$'
      - 'args\.'
      - 'error'
      - 'request'
      - 'pg[_.-]?user'
      - 'fs\.read'
      - '^masked$'
      - '^blank$'
      - '^flask_user$'
      - '^someone$'
      - '^some-?user$'
      - '^return$'
      - '^grafana$'
      - '^err$'
      - '^choose$'
      - '^pwd$'
      - '^Mozilla$'
  - type: AssignmentBanlistPostValidator
    patterns:
      - 'user[_-]?agent'
      - 'proxy'
      - 'redis'
      - 'mongo'
      - 'm[sy]sql'
      - 'postgres'
      - 'ftp'
      - 'smtp'
      - 'zookeeper'
      - 'ldap'
      - 'mail'

port:
  - type: ValueBanlistPostValidator
    patterns:
      - '^25$'
      - '^465$'
      - '^587$'
      - '^80$'
      - '^8080$'
      - '^443$'
      - '^2[012]$'
      - '^389$'
  - type: AssignmentBanlistPostValidator
    patterns:
      - 'imap'
      - 'report'
      - 'support'
      - 'args\.'
      - 'http'
      - 'proxy'
      - 'redis'
      - 'mongo'
      - 'm[sy]sql'
      - 'postgres'
      - 'ftp'
      - 'smtp'
      - 'zookeeper'
      - 'ldap'
      - 'mail'
      - 'portal'