# Scrubbing and privacy controls

Bucker scrubs every event **before it is stored** and again **before anything reaches an AI
agent**. These are two tiers with different rule sets. The agent tier applies everything the
stored tier applies, and then more. So an agent can never see more than was stored, only less.

This page covers what each tier removes by default, and the three controls an organization
owner can change under **Settings › Compliance › Privacy controls**. It also covers what those
controls deliberately cannot do.

## What is removed by default

| Removed from | What |
| --- | --- |
| Stored data **and** agents | Authorization headers and cookies; values under credential-shaped keys (`password`, `api_key`, `accessToken`, …); private-key blocks; JWTs; AWS key ids; vendor tokens (GitHub, Slack, Stripe, OpenAI-style); credentials inside URLs, query strings and `key=value` text; card numbers that pass Luhn; high-entropy opaque tokens |
| Agents only | Email addresses; public IPv4 and IPv6 addresses; dashed US Social Security numbers that pass the SSA validity rules; IBANs that pass the ISO 13616 checksum; `user.id`, `user.username`, `user.email` and `user.ip_address`, which are hashed so an agent can still tell that two events came from the same user |

People on your team can still see the personal identifiers in the second row, because
debugging often needs them. Your organization can move them into stored data too, as
described below.

Your own **scrub rules** (Settings › Compliance › Scrub rules) are added on top of these
defaults. A scrub rule can only add redaction. It can never switch a built-in rule off.

## Privacy controls

Only an organization **owner** can change these controls or preview them. Every member can read
them. Each change writes a `privacy_settings.update` row to the organization's audit log, with
the before and after values of each field it changed. A change applies on the machine that
handled it at once, and on every other machine within five seconds.

### Do not store end-user IP addresses

When **Store end-user IP addresses** is off, the following happens before an event is stored:

- `user.ip_address`, `request.env.REMOTE_ADDR`, the forwarding headers (`X-Forwarded-For`,
  `X-Real-IP`, `True-Client-IP`, `CF-Connecting-IP`, `Forwarded`) and the OpenTelemetry address
  attributes (`client.address`, `http.client_ip`, `net.peer.ip`, `net.sock.peer.addr`,
  `network.peer.address`) are removed, whatever their value. This includes the Sentry
  placeholder `{{auto}}`.
- `user.geo` is removed, because a location derived from an address discloses the same thing
  the address did. Bucker does not derive location from IP addresses itself. This rule covers a
  location sent by an SDK or by a relay in front of Bucker.
- Public IPv4 and IPv6 addresses inside any text are replaced with `[Filtered]`. Loopback,
  private, link-local and unique-local addresses are kept, because they describe your own
  infrastructure and are often the most useful part of a connection error.
- An IP address is no longer used as the user identity for user counts. An event whose only
  identifier was the IP address counts no user.

The safe-field list below cannot override this setting.

### Also redact from stored data

These detectors can be added to the **stored** tier. Each one is opt-in, for the reason given.

| Entry | Detects | Why it is opt-in |
| --- | --- | --- |
| US Social Security numbers | `123-45-6789`, dashed only, valid area, group and serial | Already hidden from agents by default. Whether your own engineers may read it is your decision. Nine undashed digits are never matched, because they are usually zip codes and order numbers. |
| Phone numbers | International notation with a leading `+` and 8–15 digits | Local formats cannot be told apart from ports, durations and ids, so they are not detected at all. |
| IBANs | Upper-case IBANs, grouped or contiguous, that pass the mod-97 checksum | Already hidden from agents by default. |
| PEM certificates | `BEGIN CERTIFICATE` and `BEGIN CERTIFICATE REQUEST` blocks | A certificate is public material. Private keys are always redacted, whether or not this entry is chosen. |
| Usernames in file paths | The username in `/Users/<name>/`, `/home/<name>/` and `C:\Users\<name>\`, replaced with `[user]` | It rewrites stack-frame paths, which grouping and code links read. The rewrite is consistent, but it changes those strings from the next event onward. |

### Safe fields

Safe fields are dotted paths, one per line, up to 50, for example `request.data.contact` or
`extra.customer.**`. A listed path covers everything under it. Paths use the same syntax as
scrub-rule paths: `*` matches one segment, `**` matches any depth, and `{a,b}` matches either
name. A path made only of wildcards, such as `**`, is refused, because it would switch personal
data detection off everywhere.

On a safe field, only the **personal-data detectors** are skipped: email, IP address, SSN,
IBAN, the `user.*` hashing, and the entries you chose above. The following still apply on every
field, listed or not:

- every credential rule in the first table row above, including the high-entropy token
  detector;
- the IP-address setting;
- your organization's own scrub rules.

This is a design decision, not a missing feature. The safe-field list exists for fields that are
*supposed* to hold personal data, such as a support ticket's contact block, and that a detector
would otherwise blank. It must never become a way to store a password or a token. If a
credential detector is redacting a value that is not a secret, contact support with the field
and an example. Do not try to allow-list it.

### Preview

**Preview** runs the settings on screen through the same rule stack that ingest uses: the
defaults, your scrub rules and the candidate privacy controls. It shows what would be stored for
a pasted JSON event or text, and saves nothing. The API is
`POST /orgs/:orgSlug/compliance/privacy/preview`.

## Runtime captures

A state slice, a live-evidence probe capture, a Python frame's local variables and the
process attributes on an OTLP resource are text from inside your process, and they are held
to a stricter standard than an error message. Four layers apply, and the server applies the
newest rules to whatever the SDK ran, because an SDK is as old as its owner left it.

- **The deny list.** A binding, a member path, a map key or a command-line flag whose name looks
  like a credential (`password`, `pwd`, `api_key`, `authorization`, `cookie`, a bare `auth`,
  `database_url`, and the camelCase `basicAuth`) is never captured. The JavaScript SDK, the
  Python SDK, probe authoring and the server are generated from one table
  (`shared/src/scrub/value-shapes.json`), so a name a human can approve a probe for is never
  one the SDK then refuses. Default entries cannot be removed; an org's own rules only add.
- **Bytes keys.** `b'authorization'`, a Node `Buffer` key, a `(key, value)` pair in a list
  (an ASGI `scope['headers']`) and the same spelling inside a repr string are matched as text.
  The key stays so the shape of the request is legible; the value is withheld.
- **Typed denial and containers.** A value whose declared type is on the typed deny list
  (`SecretStr`, `KeyObject`, `CryptoKey`, a cookie jar, `os.environ`) is withheld whole. The
  Python SDK captures named leaves, not the `repr` of a container: a dict, list or plain
  object is walked field by field, and one that is too deep, too wide or opaque is refused
  (`container_refused`) rather than printed. Frames in credential-handling modules (`secrets`,
  `hmac`, `jwt`, `cryptography`, a `.env` loader) leave no locals at all. A probe cannot target
  such a file or module.
- **Value shapes.** Every string passes through one table: JWTs, `Bearer`/`Basic`/`Token`
  values, AWS key ids, GitHub tokens (`ghp_…`, `github_pat_…`), Stripe live keys, PEM private-key
  headers (even without an END line), and any 20-character-or-longer token over 4.5 bits per
  character. The scan runs on the whole string before it is shortened, so a secret past the
  preview limit cannot leave a recognisable head.

Every capture returns `redactions`, a list of `{path, reason}` naming where something was
withheld and why, never what it was. Reasons are `bytes_key`, `denied_type`, `denied_path`,
`value_shape`, `container_refused` and `resource_attribute`; the server folds the SDK's list
into the stored redaction manifest.

OTLP resource attributes that describe the process get their own rules at the stored tier:
`process.command_args`, `process.command_line` and `container.command_args` have the value of
every credential flag (`--password=…`, `--api-key …`, `API_KEY=…`) removed, `process.executable.path`
keeps its shape but loses the user name in a home directory, `k8s.pod.annotation.*` is scanned
(and `kubectl.kubernetes.io/last-applied-configuration` is withheld whole), and
`pprof.label.*` / `profile.label.*` are reserved and withheld until a profile ingest path defines
what may appear in one.

## Infrastructure text

Manifests, machine and service configs, process lists, cloud audit events and provider status
text carry credentials under neutral names and prose written by someone else. A keyword list
(`password`, `api_key`) misses both, so Bucker fences them with a typed field policy
(`INFRA_FIELD_POLICY`, `shared/src/scrub/infra-field-policy.ts`, versioned) instead: for each
data class it says, per path, what happens to the value.

| Action | Meaning |
|---|---|
| `drop` | replaced by `[Dropped]`, never inspected again |
| `hash8` | an 8-character SHA-256 prefix, salted with the project and written `hash8:1f2e3d4c`; names stay |
| `scan` | the value-shape scanner, URL userinfo, and for a command the flag-aware word scrubber |
| `keep` | an identifier that cannot hold a secret (`userIdentity.type`, `accountId`) |

A path no rule names is `scan`, never `keep`. The classes are `k8s_manifest`, `machine_config`,
`service_config`, `provider_status_text`, `cloud_audit_event`, `process_list` and `env_var`.
What the default rules do:

- Environment variable **values** are hashed (list, map and `NAME=value` forms); the names stay,
  so a changed value shows as a changed hash. An empty value stays empty.
- `kubectl.kubernetes.io/last-applied-configuration`, `managedFields`, and the `data`,
  `stringData`, `binaryData` and `encryptedData` of a Secret, ConfigMap or SealedSecret are
  dropped. Secret and ConfigMap data is dropped, **never hashed**; a `data` field on an object
  that names no kind is dropped too.
- Container `args` and `command`, probe and lifecycle `exec.command`, annotations, labels and
  image references are scanned. `--db-password=x`, `--api-token x`, `NAME=x`, `-u user:pass`,
  `-H "Authorization: …"`, `postgres://user:pass@host` and a script inside `sh -c` are all read.
  A label or annotation whose key names a credential has its value withheld.
- Caller identity (names, emails, IP addresses, user agents, session context) is removed from
  CloudTrail, Event Grid, GCP Admin Activity and Heroku collaborator events; what happened stays.
- A key that is itself a secret shape is renamed `[filtered-key-N]`: keys are attacker text too.

It fails closed in three ways. An **unknown class** withholds the whole value
(`[withheld:unknown_infra_class]`). A structured class given text that is not JSON (a YAML
manifest) is withheld (`[withheld:unparsed_infra_value]`); send the parsed object. A walk past
32 levels or 50,000 nodes replaces what it did not reach.

At the stored tier the same rules apply to the OTLP resource attributes that carry these fields:
`process.environment_variable.*`, `process.env.*`, `container.env.*` and `k8s.container.env.*`
are hashed, `k8s.<kind>.label.*`, `container.label.*` and every kind's `annotation.*` are scanned,
and `container.command_line` / `container.command` are read like `process.command_line`.

At the agent tier `sanitizeInfraField` (`mcp-server/src/sanitize.ts`) runs the policy, the
agent-tier scrub with the org's rules, and the injection scan over every string and key, and
returns the value labelled `untrusted` with a `scrubbed` count (what was dropped, hashed or
masked) and a `quarantined` count. Every infrastructure read reaches an agent through it.

**Adding your own paths** is the existing scrub-rules setting (`infra_scrub_rule.update` is the
same write, `org:admin`): a rule with `path` and `remove` or `hash` applies after the policy, on
the same path the manifest has (`metadata.annotations.**`; an annotation key contains dots, so
use `**`, not `*`). Customer rules only add. Nothing in the settings, and no safe-field entry,
removes or relaxes a default rule; changing a default is a reviewed code change that moves
`INFRA_FIELD_POLICY_VERSION`.

Two limits to know. A short free-form word under a **neutral** key in a scanned position
(`labels.team: hunter2`) cannot be told from a team name and is not found. And the injection
scanner is pattern-based: on the 200 annotation and status-text payloads in
`api/test/otlp/fixtures/infra` an adaptive attacker with 25 tries got a payload past it in every
campaign, which is why these fields are also labelled untrusted, rendered as data and counted
toward session taint. The measured rates are recorded per class in the transparency register.

## What was redacted, and where

The redaction summary on a stored row (`3x jwt`) says how many and what shape, not which rule or field.
For the person who has to fix the leak, Bucker also keeps, per project, **the rule and a path template**
of every credential or personal-data redaction, per service, release, environment and hour, with the
first trace and event: the **scrub hit rollup** (30 days). It never holds a value, and a path segment that
looks like a credential, an e-mail address or a card number is replaced by `*`. The detector over it
files [secret and personal-data findings](./security-findings.md#secrets-and-personal-data-in-telemetry-secret_exposure-pii_exposure).

E-mail, public IP, SSN and IBAN are stored by design, so they never appear in that rollup from the
stored tier. A worker therefore **samples** stored events and log records (1 % an hour, at least 100 per
table while that many exist), runs the agent-tier rules over them and counts the rule and path. It reads
and counts; it never changes or removes anything it read, and its counts are of the sampled rows.

The pre-scrub bytes live in the raw envelope for 7 days after processing. A finding shows the date they
expire, and an owner can purge them early (a human-only act with a 10-minute cancel window, a signed
deletion receipt and an audit row).

An agent can propose a redaction rule with `scrub_rule_propose`. It is always asked: a person approves it,
it is checked with the same refusals as the dashboard (including the ReDoS probe) before anyone is asked,
and it can be switched off. It only ever **adds** a rule; the built-in credential rules always run first.

## API


| Method | Path | Gate |
| --- | --- | --- |
| `GET` | `/orgs/:orgSlug/compliance/privacy` | any member |
| `PATCH` | `/orgs/:orgSlug/compliance/privacy` (`storeIpAddresses`, `scrubVocabulary`, `safeFields`; any subset) | owner |
| `POST` | `/orgs/:orgSlug/compliance/privacy/preview` (`settings`, `sample`, optional `tier`) | owner |
| `GET` | `/orgs/:orgSlug/projects/:projectSlug/findings/:id/raw-envelopes` | `security:read` |
| `POST` | `.../findings/:id/raw-envelopes/purge` (`reason`, `acknowledgeEnvelopes`), then `.../purge/:purgeId/cancel` | owner, a person's own session |

## Not built yet

There is no **scrubbing relay** yet, meaning a process you run inside your own network that
scrubs events before they leave it, as Sentry's Relay does. When it is built, it will reuse
the same rule engine (`shared/src/scrub`) unchanged. Until then, scrubbing happens on Bucker's
servers, before storage, and the SDKs' own `beforeSend` hooks remain the place to remove data
that must never leave your network at all.
