Skip to content
bucker

Alert routing, correlation groups, SLOs and what nothing watches

Alert rules decide what fires. This page covers the four things that decide who hears about it, how often, and whether anything is watching at all: routing policies, correlation groups, SLO budgets and burn alerts, and the coverage and quality reads that find rules nobody would be told about. It also says which of these are off until you turn them on, and which acts only a person can perform.

Routing policies: who is told

A rule may carry up to 20 tags and a priority from P1 to P5. An organization's routing policy matches on those and names who is told:

  • A policy is a list of up to 50 entries. Each entry matches on any of tags (team: payments, or team: "*" for any value), projects, environments, priorities and transitions (fire, warning, recovery, no_data), and names up to 50 recipients (channels).
  • A policy has a required fallback channel: where a fire goes when a matched entry's recipients are gone, and when a prioritised rule matches no entry. The fallback is a safety net, not coverage.
  • An entry may carry quiet hours (an IANA time zone and day and time windows). They hold only P3 to P5 deliveries. P1 and P2 are never held.
  • A policy may carry a page budget (perHour, perDay, in fixed UTC buckets per recipient). Past it, a P3 to P5 delivery is queued for the next bucket with room. A page is never dropped, and P1 and P2 are never held by it.

P1 and P2 rules nobody would be told about are refused. A save that would leave a live P1 or P2 rule with no recipient (its own, or one a policy entry reaches) is refused with routing_policy_coverage, naming the rules, before anyone is asked.

A policy is born disabled, and a disabled policy reaches nothing. With no enabled policy routing behaves as it always did.

What Route Who
List the organization's policies GET /orgs/:org/routing-policies org read
Policies covering one project's rules GET /orgs/:org/projects/:project/routing-policies project read
See which rules a draft would cover, and any refusal POST …/routing-policies/preview configure_alerts; writes nothing
Create or replace a policy POST …/routing-policies goes through approval; asks a person, always
Delete a policy DELETE /orgs/:org/routing-policies/:policyId a person, org admin
Set a rule's tags and priority PUT …/alert-rules/:ruleId/routing a person

The preview is its own route, never a flag on the write, so a retried preview cannot become the act. A policy write is always asked: no policy row, review mode or autonomy rung makes it automatic. It becomes critical, which no agent is allowed by default, when more than 20 rules are affected or a paging recipient or the fallback is removed. Who is paged stays a person's decision through rule edits too, which is why tags and priority are set by a person.

For an agent: routing_policy_get answers "is this rule routed, and to whom?" (with the live P1 and P2 rules that have no recipient), and routing_policy_propose raises the request.

Correlation groups: one notification per burst

When a deploy breaks things, a dozen rules fire inside minutes. Correlation folds the later chat and mail notifications of one burst into the first. Fires of one project that share an environment and at least one of release, service, or the same nearest change event (within 30 minutes) inside the window (10 minutes by default, 1 to 60) become one group. The first fire opens it and is delivered as it always was; a later fire's chat or mail delivery to a destination that already heard about the group is recorded and not sent.

  • Off by default, per project. The policy (on or off, which keys, the window) is stored per project and ships enabled: false; with it off, delivery is exactly as without the feature. The weekly learning pass may propose a change to it, and a person decides.
  • A page is never folded. Only chat and mail channels fold, only to a destination the group has already notified, never a recovery, and never a fire whose priority is higher than anything the group has notified so far.
  • A person's removals stick. Removing a rule from a group marks the member (it stays in the list, with who removed it) and that rule does not rejoin that group.
  • correlation_group_get lists a project's recent groups, or one in full (keys, the change event it attached, members), and with stats: true the seven-day collapse and removal rates. correlation_groups_update removes or restores members, and combines two open groups or splits one. Removing a member (the group notifies more) applies without asking; restoring or combining (it notifies less) asks a person. The policy itself has no tool.

SLOs: budgets, burn, OpenSLO

An SLO says how much failure is acceptable over a window (7 to 365 days, 30 by default). Three kinds are measured: span_success (good over total requests by status class), span_latency (the share of hourly slices whose p95 stays under a bound) and release_stability (crash-free sessions). uptime, journey and issue_free_sessions are not measured through this door, and a request for one is refused.

  • The budget is over observed time. Hours with no data are excluded and counted (emptyHours, missingHours), never counted as good, and a window below its minimum volume reads not_enough_data. The grain is one hour, and a latency slice is judged good only when the upper bound of the bucket holding its p95 is within the bound, so a slice is never called good on a midpoint.
  • Maintenance windows are a correction at read. An hour a window touches is left out of the corrected figure, and the read returns both the raw and the corrected budget.
  • Read them: GET /orgs/:org/projects/:project/slos and …/slos/:slo (id or name; ?include=openslo adds the OpenSLO v1 document), or slos_list and slo_get (include: ["openslo"]). The routes only read the hourly slices a worker wrote, so a read cannot make the database scan spans.
  • Authoring asks a person. slo_propose creates, re-targets, edits or archives an SLO; every act asks, because an SLO moves the error budget that caps what agents may do in the project, and an agent that could re-target one unasked could lift its own cap. There is no route that creates an SLO, and deleting one has no tool at all.
  • Alert on the budget with the slo_burn condition (paired long and short windows; a pair fires only when both burn above its threshold; defaults 14.4 over 1 hour and 5 minutes, 6 over 6 hours and 30 minutes, 1 over 3 days and 6 hours) or slo_budget (fires when the remaining budget falls below a percentage). Slices are hourly, so a 5-minute short window reads one hour and the reading says so. A window below the SLO's minimum volume is held as no data, never read as recovery.

When your own PagerDuty, incident.io or Rootly opens, acknowledges, updates or resolves an incident, Bucker keeps a link to it from the issue, correlation group or alert: provider, the vendor's id and URL, state, severity label, commander as text, timestamps, and the seconds to acknowledge and resolve. incident_links_list reads them, filtered by issue, group, alert event or state. The pager's acknowledgement is mirrored read-only, and it may stop a re-fire only when the pager itself reported it: an agent that attaches an already acknowledged incident has acknowledged nothing. This is a link, not an incident product. Connecting the tools is in Incident integrations.

Finding what nothing watches

  • Backtest. alert_rules_backtest (and POST …/alert-rules/backtest) replays a rule over up to seven days of stored telemetry with the cooldown and silences applied: how many times it would have fired. See Backtest.
  • Coverage. alert_rules_coverage finds five kinds of gap: endpoints with traffic and no rule on error rate, latency or volume; issues above a volume threshold that no rule would page on; ownership rules whose owners nothing routes to; cron and uptime monitors whose issues no rule routes; and status components with an SLO and no burn rule. Each gap lists its evidence, and the first gap of a group carries a ready shadow draft (the arguments of alert_rules_create). It creates and schedules nothing; making the call and promoting are separate acts of someone with alerts:write. With the request-rollup pillar on, traffic is the pre-sampling rollup; otherwise it is the newest 100,000 stored server spans, marked stored_sampled and truncated, and never extrapolated.
  • Quality. monitor_quality_get returns the daily score of each rule, worst first, with the keys that set it and the blocking subset (no resolvable owner, no reachable route, broken) that stops an agent-made rule being promoted. See Monitor Quality.

The shadow lifecycle of an agent's rule

A rule an agent creates is born SHADOW: it evaluates and records what it would have done, and delivers nothing. A person's own rule is live. A shadow rule nobody promotes lapses after 14 days. Promoting asks a person and is gated on the Monitor Quality blocking keys above, so a rule with no owner or no route cannot go live by accident. The full lifecycle, the cap of 20 shadow rules per project per agent, and demotion are in Alert rules; the governance of agent acts is in Agent governance.

What needs what

Feature Off or on by default What turns it on
Routing policy A new policy is disabled; none enabled means no change A person approves the write, with enabled: true
Correlation Off per project The weekly learning pass proposes enabled: true and a person approves it; no route or tool asks for it directly
SLOs None exist A person approves slo_propose; the hourly slices need span or session data
slo_burn and slo_budget rules None exist A rule naming the SLO
Incident links None A connected incident tool whose webhooks reach Bucker
Coverage with the unsampled basis Stored-span basis The operator's request-rollup flag (pillar.apm_red_rollups)

MCP access to these tools is under the scopes in MCP setup: alerts:write is inside the live ceiling. The holds on the wider MCP surface were lifted by owner decision on 2026-11-16 without a signed-off security review of the OAuth server.

See also: Alert rules · Alert webhooks · Personal notifications · Incident integrations · Maintenance windows and mutes