Skip to content
bucker

CI setup runbook

This page is for a person or a coding agent connecting a repository's CI to Bucker. It is written to be read by a program: every table cell is filled, and a support matrix says yes, no or n/a, never nothing. A docs test fails on an empty cell in this file.

What you may change

Thing An agent may change it How
Files in the repository (workflow YAML, test-runner config) yes a pull request that a person merges
Merging the pull request no a person
Branch protection and required checks no a repository admin
Bucker project settings, including turning CI ingest on no an organization admin, in the dashboard
The GitHub App's permissions (consent) no an organization admin
Tokens and the DSN no a person creates them and stores them as CI secrets or variables

Doctor reads; it never writes. It makes GET requests only and prints a token's kind and scopes, never the token.

The loop

  1. Run bucker ci doctor --json in the repository root, after the tests have written their reports.
  2. Read the checks array. Each finding has id, status, evidence and fix.
  3. For each fail or warn whose fix is a file in the repository, propose the change as a pull request.
  4. For each whose fix names a person (consent, ingest switch, branch protection), say so in the pull request and stop.
  5. After a person merges, run doctor again.

Exit codes

Exit code Meaning What to do
0 every check passed or warned warnings are advice, not blockers
1 at least one check failed fix the failed checks; a failure beats a could-not-check
2 nothing failed but a check that matters could not be made retry; the server may be older than the CLI, or the token may not reach the project

A not_checked finding carries blocking. Only a blocking one moves the exit code to 2. Branch protection that needs admin consent to read is not_checked with blocking: false.

The checks

Check id Fails when Warns when Needs the server
action_version no workflow runs a bucker action or @bucker/cli a reference is a branch, a major tag or an unversioned CLI no
token no token, a 401, or release:write missing the token is personal or an agent's, not an org CI token yes
dsn no DSN, or one that does not parse never no
connectivity the API or the ingest host is unreachable or answers 5xx never no
github_consent no connection, or checks:write or actions:read not granted never yes
webhook_deliveries no delivery at all in 24 hours one of workflow_job, check_run, merge_group, deployment_status is missing yes
junit_files no JUnit file, or none usable some unusable, or older than 71 hours no
coverage_files never no coverage file found no
identity_stability never more than 20% of test identities appear in only one of the last 30 runs yes
traceparent never TRACEPARENT is absent or malformed no
merge_queue never merge_group events arrive and no workflow triggers on them, or neither exists yes
gate_setup never the gate check is not required, or production's only protection is Bucker's yes

Uploading results

bucker ci upload "reports/*.xml" --dsn "$BUCKER_DSN"

The command reads the commit, branch, run id, attempt, workflow, job and pull request from the CI system's own variables, and TRACEPARENT and BAGGAGE (cicd.pipeline.run.id) when the pipeline exports them. A flag wins over the environment. It sends each file to POST /api/:projectId/ci/reports on the DSN's ingest host with the DSN public key, and exits 0 only when every file was accepted.

Flag Meaning Default
--format junit or coverage junit
--sha full commit id the CI variable, then git rev-parse HEAD
--branch branch name the CI variable
--run-id the CI system's run id the CI variable, then BAGGAGE
--attempt attempt number, 1 to 100 the CI variable, else 1
--workflow workflow name the CI variable
--job job name the CI variable
--config a suffix for test identity: operating system, runtime, matrix key none
--pr comma-separated pull request numbers the CI variable
--dry-run find the files and print what would be sent off

CI systems read

CI system Commit Run id Job Pull request
GitHub Actions yes yes yes yes
GitLab CI yes yes yes yes
CircleCI yes yes yes yes
Buildkite yes yes yes yes
Jenkins yes yes yes no
Anything else no no no no

On a system marked no, pass the flag. The commit always falls back to the git checkout.

Formats the door accepts

Format Accepted today Notes
JUnit XML yes surefire reruns and flaky markers are understood; a DOCTYPE is refused
Coverage no the door answers with a reason until coverage ingest ships
TAP no convert to JUnit
TRX no convert to JUnit

Limits: 8 MiB per file. A refused file prints the door's reason (ci_ingest_disabled, report_too_large, report_format_unsupported, and the parser's own). A 429 or 5xx is retried three times; a refusal is never retried.

The GitHub Actions step

- uses: bucker/release-action/ci-upload@<full commit sha>
  if: always()
  with:
    dsn: ${{ vars.BUCKER_DSN }}
    files: reports/*.xml

Pin the action to a full commit SHA. The cli-version input is an exact version.

What doctor reads

Doctor reads two routes through the control plane with the caller's token. A server that does not serve them yields not_checked findings and exit 2, not a failure.

Route Used by
GET /orgs/:org/projects/:project/ci/setup token scopes, github_consent, webhook_deliveries, merge_queue, gate_setup
GET /orgs/:org/projects/:project/ci/runs?limit=30 identity_stability

DORA metrics

dora_get (MCP, delivery toolset) and GET /orgs/:org/projects/:project/ci/dora return DORA's five metrics for one project and environment: deployment frequency, change lead time, change failure rate, failed deployment recovery time and deployment rework rate. The window defaults to 30 days and is at most 90, and the environment defaults to production.

Rule What it means
Honest denominators every rate prints its numerator, its denominator and the deploys it could not measure, for example 48 deploys, 41 measured, 7 without a final status
Minimum sample a rate over fewer than 5 measured deploys is null with a reason
Publishable sample a rate over fewer than 30 measured deploys exists but is marked publishable: false, and a shared report leaves it out
No deploys a project with none reads not_enough_data as a whole
Evidence every failed or rework deploy lists its evidence (a rollback, a guardian halt, a failed release verdict, a status incident, a manual mark) and whether the basis is observed, heuristic, manual or rework
Heuristics revert and hotfix wording in commit messages counts only in its own heuristic column, never in the rate
Rule set the rule set (dora-rules@1) is printed with every answer

A person can mark a deploy's outcome (POST .../ci/dora/outcome/preview, then .../ci/dora/outcome). The mark is stored beside the observed evidence and labelled manual; it never overwrites a deploy's own status. Cohorts by authorship are not built, and the answer says not_available rather than returning an empty list.

Scorecards

A scorecard is a set of rules over data Bucker already holds, evaluated daily for every project you can read. Each rule is a closed predicate over named facts (an owner is defined, an alert rule routes, release health is reported, no overdue reversal debt, an instrumentation score, a gate is enforced). It has a level ladder from 1 to 3. A scorecard describes and never enforces: nothing it reports blocks anything.

Question Answer
Who may read anyone who can read the projects; only projects the caller can read contribute a row, a count or a level
A fact that could not be read counted as skipped, never as a pass and never as a fail
Who proposes a rule an agent or a person, through scorecards_update or POST .../scorecards/rules; a proposal is always a draft
What a draft shows organization admins see how many projects it would fail (shadow); nobody else learns a draft exists
Who publishes a person with org:admin, on POST /orgs/:org/scorecards/rules/:ruleSlug/publish or in the dashboard; there is no tool for publishing, unpublishing or archiving a published rule
Limits at most 100 rules per organization and 5 leaves in one predicate

Read them with scorecards_list or GET /orgs/:org/scorecards. With scorecard: <slug> and include: ["outcomes"] each project's outcome shows the value read, the rows read behind it and the fix each failing leaf points at. Campaigns are not built, and asking for them answers not_available.

Gates

A gate is a named set of rules checked against a pull request, a commit or a deploy. The rule types are tests_failed, new_flaky_test, patch_coverage_below, live_issues_in_touched_files, release_health_regression, overdue_reversal_debt and synthetic_journeys_failing. A project has at most 20 gates and a gate at most 12 rules. A rule that cannot be decided reads not_enough_data, never passed.

A gate moves through three stages, and every move is a person's act.

Stage What it does Who moves it up
shadow evaluates and records; posts to no pull request and blocks nothing a person with org:admin publishes it (POST .../gates/:gateId/publish)
warn posts a check with a neutral conclusion: it reports and does not block a person enforces it (POST .../gates/:gateId/enforce)
enforced the check can block a merge the step above
  • Enforcement is earned. Enforcing needs at least 14 days in shadow and at least 30 recorded check evaluations, and a person confirms with a security key (the step-up route is .../enforce/challenge), bound to the gate, its revision, its fail mode and the reason.
  • Agents propose, people publish. gates_update creates a gate in shadow, revises a shadow gate, or archives one. A new gate carries a backtest over the last 200 pull requests once applied. A published or enforced gate is refused (not_shadow), and publish, enforce and disable are not inputs of the tool.
  • Reading. gates_list shows each gate's stage, fail mode (closed or open), revision and shadow record. With include: ["evaluations"] it shows the stored evaluations: the verdict that was posted, each rule's outcome with its evidence ids and watermark, and the signed entry id. gate_evaluate asks what a gate would say about a commit, pull request or environment now. It is stored as adhoc, never posts a check, never blocks and is never signed.
  • Overrides are a person's. A person other than the pull request's author, holding fix:approve, may override an enforced gate that is failing the commit, with a reason, an expiry and a step-up bound to the evaluation. There is no tool for it. Revoking an override restores the failing verdict, so it is allowed to ci:write.
  • Signed. An enforced verdict is signed, so a result can be checked later.
  • Undecided checks. A non-shadow check that cannot be decided within 10 minutes is settled by the gate's fail mode.

Flaky tests

flaky_tests_list shows the tests whose outcome depends on something other than the code: the verdict, the executions behind it, the same-commit flips, an age-weighted flake rate, the CODEOWNERS owner, the issue the detector opened, and any quarantine. Minutes lost to flakes are not shown, and the field is absent rather than zero.

flaky_test_quarantine_create keeps a flaky test running and reporting while its failures stop failing the gate, for at most 14 days (the default is 7). It is a governed write that asks a person, and the approval card carries the evidence: the verdict, executions, same-commit flips, the flake rate and how much of the project's cap is in use. It is refused before anyone is asked when:

Refusal Cause
not_flaky the test is not judged flaky, or was never seen to pass and fail on one commit
too_few_samples fewer than 10 executions
protected_path the test is under a protected path
blamed_production_issue the test file changed in a commit blamed for an unresolved production issue
cap_reached the project already has 25 active quarantines

A test cannot be quarantined twice. The default protected paths are **/auth/**, **/authz/**, **/migrations/** and **/security/**. Restoring is operation_revert on the quarantine's operation, and the sweeper restores it at the end of its lifetime. A renewal is a new call.

Reruns and cancels

ci_runs_update reruns a failed job, reruns the failed set of a run, or cancels a run. The approver sees the jobs that would run, an estimate of minutes and dollars with the price's source and date, and the reruns GitHub still allows. It is refused before anyone is asked if the run or the restart set holds a deploy job (holds_deploy_job), the run is still executing, a cancel targets a finished run, nothing failed, or GitHub's 30-day or 50-rerun limits are passed. It also pauses when the caller's CI-minutes budget is spent. GitHub's Actions write permission is an opt-in on the GitHub App that an organization admin grants; without it, nothing can rerun.

What turns it on

Capability Needs
Any CI data to read at all the ci_ingest pillar on the deployment (off by default; a refused upload says ci_ingest_disabled) and CI ingest turned on for the project by an organization admin
Test, coverage and run data the upload step above, and the GitHub App consent that bucker ci doctor checks
Agent writes (gates_update, scorecards_update, flaky_test_quarantine_create, ci_runs_update, ci_ingest_filter_update) a token with ci:write, granted at consent
Blocking a merge a gate a person has published and enforced after the shadow period

The SG-059 holds on these tools were lifted by owner decision on 2026-11-16, with no signed-off security review of the authorization server recorded. Related: Releases, Release attestations, Staged rollouts, Production assertions, Application performance.