Verification

Nothing posts on a hunch.

AI review is cheap to generate and expensive to trust. Baz splits every review into proposing and proving: agents survey the change for concerns, then a separate verifier must confirm each one with evidence before it reaches your pull request. Requirements from the ticket and design get the same treatment, checked against the code and the running application.

Every layer of verification in one review

Bugs and requirements are verified on separate tracks, each with its own verdicts, and both land in one session you can inspect.

  1. SurveyAgents read the change in parallel and propose concerns, without posting anything
  2. ConsolidateOverlapping concerns fold into one, so each is proven once
  3. ProveOne verifier per concern traces the path and confirms or refutes it
  4. Check the specEach ticket and design requirement gets a verdict with evidence
  5. RecordVerdicts, evidence, and screenshots stored in the session

Survey

Agents for each reviewer read the diff and the code around it, and propose concerns. A concern your team already ruled out in a past discussion or a memory is skipped here.

Concern verification

A dedicated verifier investigates each concern across files, history, and related repositories. It must confirm it with evidence or close it as refuted, suppressed, or incomplete.

Proof collection

A confirmed finding states the entry point, the runtime condition, the failure, how far it reaches, and how certain the verifier is. Severity is scored from that evidence, not from tone.

Speculation filtered out

A finding posts only when the trigger is shown to be reachable on the changed path. Claims that rest on what might happen are dropped before anyone reads them.

Spec validation from the ticket

The Verification Agent pulls requirements from the linked ticket in Jira, Linear, Azure DevOps Boards, and other trackers, and marks each one met, partially met, not met, or unverified.

UI verification

With a sandbox or a preview environment, the agent signs in, opens the app in a real browser, and compares it to the Figma design or ticket images.

Evidence you can see

Annotated screenshots mark where each requirement lives on screen, in the Spec Reviewer Report on the pull request and in the session.

Fits beside your proofs and tests

Your test suites, fuzzers, and formal proofs keep running in CI. Baz verifies what they cannot express: intent, reachability, and how the app actually looks. Security Review applies the same proof bar to exploitability.

What verification looks like in the product

One pull request, followed from the first concern to the sandbox that ran it.

Survey · Verify

Fourteen concerns proposed. Two posted.

Survey agents propose concerns without commenting. Overlaps fold into one, then a verifier investigates each and returns a verdict. The session keeps every verdict and its reason, including the ones that never reached the PR.

How sessions record a review
Session · Code reviewPR #1731 · 9f3c2a1
  1. 14Proposed by survey
  2. 9After consolidation
  3. 2Confirmed and posted
  • ConfirmedformatCredits() return type changedConsumer in billing-service adds it to a numberusage/format.ts:31
  • ConfirmedExport runs without a workspace scopeReachable from the authenticated handlerroutes/export.ts:18
  • Refutedrows may be undefined on first loadEvery caller passes the loader resultUsageTable.tsx:44
  • RefutedDate range off by one dayRange is end-exclusive by designusage/range.ts:12
  • SuppressedPrefer named export for UsageTableTeam memory: default exports allowed hereUsageTable.tsx:1
  • IncompleteCSV header may be localizedLocale source not reachable from this repousage/csv.ts:9

Only confirmed concerns become PR comments. The rest stay in the session with the reason.

Proof collection

A comment that carries its proof

A confirmed finding states where it enters, when it fails, what breaks, and how far it reaches, with the commit it was verified at. Here the consumer that breaks lives in another repository.

What each review agent covers
github.com/acme/frontend/pull/1731
Baz Code Correctness bot commentednow
High Breaking change · cross-repo✓ Verified

formatCredits() now returns a formatted string. billing-service still sums its result, so invoice totals concatenate instead of add.

EntryGET /api/usage/export calls buildUsageRows()
RuntimeAny workspace with credits above zero
FailureformatCredits() now returns a string; invoice.ts:88 adds it to a number
ReachRead by billing-service at its pinned commit, outside this PR
CertaintyHigh: traced through both repositories
src/usage/format.ts
31- return Math.round(credits * 100) / 100;
31+ return credits.toLocaleString(locale);
Verified at 9f3c2a1. 2 refuted concerns not posted.
Spec validation

Every requirement in the ticket gets a verdict

The Verification Agent reads the linked ticket and design, then checks each requirement against the code and the running app. When it cannot check one, it says unverified instead of guessing.

Connect a ticketing integration
github.com/acme/frontend/pull/1731
Baz Verification Agent bot commentednow
Spec Reviewer Report
Ticket requirements not met & unverified
  • MetExport button on the Billing usage pageScreenshot 1 · /settings/billing
  • MetCSV has seats, credits, and agent runsusage/csv.ts:9-27
  • Partially metExport as CSV and PDFPDF route returns 404 · routes/export.ts:40
  • Not metExport sits top right, as in the designScreenshot 2 vs Figma frame Usage / Header
  • UnverifiedEmpty state for workspaces with no usageNo workspace without usage in the sandbox
Used resources:9f3c2a1CR-4611Figma · UsageCheckout in Baz
UI verification

The running app, held up to the design

The agent opens the change in a real browser, signs in, and compares what renders with the Figma frame. Screenshots come back annotated, so the reviewer sees the drift instead of reading about it.

Connect Figma
Figma Usage / Header
Browser sandbox · /settings/billing
  1. 1Columns match the ticket. Seats, credits, and agent runs render with data.Met
  2. 2Export placement. The design puts it top right; the running app shows it top left.Not met
Sandbox · Runtime

The app runs before it is judged

Each repository defines how to set up, start, and sign in to the app. Baz builds a short-lived sandbox from the branch, waits for the health check, verifies, stores the evidence, and deletes the sandbox.

Configure sandboxed application review
Settings · Sandboxacme/frontend
Runtime
Node 22 · Baz runtime image
Setup command
pnpm install --frozen-lockfile
Start command
pnpm dev
App port
3000
Health check
/healthz
Environment
API_URL SESSION_TOKEN ••••••
Authentication
Session token
Dependency cache
On
sandbox · PR #1731
sandboxcreated for PR #1731 at 9f3c2a1
gitchecked out cr-4611-usage-export
cacherestored pnpm store in 6.1s
$pnpm install --frozen-lockfile · done in 18.4s
$pnpm dev · ready on :3000
healthGET /healthz → 200
browsersigned in with session token
browser/settings/billing → screenshot 1, 2 captured
browser/api/usage/export?format=pdf → 404
verify5 requirements · 2 met · 1 partial · 1 not met · 1 unverified
sandboxdeleted

What verification changes on a real PR

Each example is a concern or a requirement, and the verdict Baz returns instead of a guess.

A null that never arrives

Proposed The survey flags a missing null check on a new parameter.

Verified Every caller passes a validated value. The concern is refuted and never posts.

A breaking change across repos

Proposed A response field is renamed in a shared API.

Verified The verifier finds a consumer in another repository still reading the old name, and confirms it with the file and line.

Already decided by the team

Proposed A style concern your reviewers rejected last month.

Verified Skipped during the survey, because a memory says the team chose this pattern.

A requirement half done

Proposed The ticket asks for export to CSV and PDF.

Verified Partially met: CSV is wired, PDF is not. The verdict points to the lines.

A design that drifted

Proposed The Figma frame shows the primary action on the right.

Verified The browser screenshot shows it on the left. Not met, with the annotated screenshot attached.

A requirement code cannot show

Proposed The empty state should appear for new workspaces.

Verified Checked in the running app with a fresh account, or marked unverified when no environment is available.

Where each kind of verification lives

Verification is a stack. Baz takes the layers a test runner or a prover cannot, and leaves the rest where it already works.

Where each kind of verification lives
Where it runsWhat Baz does
Unit and integration testsMerge Automation Your CIWeighs CI results in the merge verdict; review agents do not run your suite
Fuzzing and property testsYour CI, with cargo-fuzz, libFuzzer, proptest, and similarRecognizes the fuzz and property tooling a repository already uses
Formal verificationYour toolchain: TLA+, Coq, Lean, Dafny, KaniRecognizes the formal tooling a repository already uses; Baz does not run proofs
Bug claims from AI reviewReview accuracy Usually posted as generatedSurveyed, then confirmed or refuted by a separate verifier with evidence
Speculative findingsPosted with a hedgeDropped unless the trigger is shown to be reachable
Requirements from the ticketSpec review A human reads the ticket next to the diffA verdict per requirement: met, partially met, not met, or unverified
UI against the designApplication review Manual QA on a previewOpens the app in a browser and compares it with Figma, with annotated screenshots

Keep your tests, fuzzers, and provers. Baz verifies the change against intent and the running app.

From comments you check to verdicts you can trust

Review as generationReview as verification
Every plausible concern becomes a commentConcerns are proposed first and posted only once proven
"This might fail if..." left for you to chaseSpeculative claims are dropped before they post
The ticket read once, by whoever remembers itEach requirement checked and given a verdict
UI checked by hand on a preview, if at allThe running app compared with the design, screenshots attached
Trust the bot or reread the codeA session that shows what was checked and why

Questions teams ask first

Does Baz run our tests, fuzzers, or formal proofs?

No. Those stay in CI. Baz recognizes the tooling a repository uses and verifies what it cannot cover: intent, reachability, and the UI.

What does the Verification Agent need?

A ticketing integration. Add Figma for visual checks, and a sandbox or preview environment to open the running app.

What does unverified mean?

The requirement could not be checked from the code or the app available. Baz says so instead of guessing.

Can I see why a concern was dropped?

Yes. The session records what ran and what each agent read.

Review that proves its findings

Connect a repository and a ticketing integration, and see verified findings and requirement verdicts on your next PR. Want UI checks in a sandbox? Talk to us.