// docs

Reporting & exports

Read back the verdicts a human already recorded: trends, run comparison, CI report recipes, and results export. Nothing here invents a pass or fail.

Everything on this page reads back verdicts a human already recorded while stepping through a run. Nothing here executes a check, invents a pass/fail, or decides anything on its own — it slices, compares, and ships out the tally a tester made by eye.

If you're looking for how a run gets those verdicts in the first place, that's the Runner, not this page.

Feature	Where	Who it's for
Quality trends	`/packages/:packageId/trends`	A QA lead watching one deck's health over time
Run comparison (A vs B)	`/runs/compare` → `/runs/[runId]/compare/[otherId]`	Regression triage after a re-run
Milestones	`/milestones`	Rolling several runs up under one release label
Run history — filter, search, saved views	`/runs`	Finding a specific run fast
Export a run	`GET /api/qa/runs/:runId/export?format=…`	Feeding a run into an existing CI/Allure dashboard
CI report → PR	recipe — see `ci-reporting.md`	Getting a run's verdict onto the PR/commit that triggered it

All six sit on the same two tables — qa_runs and qa_results — and the same entitlement check every other authenticated QA read in the app uses: a run outside your visibility 404s, it doesn't leak a hint that the run exists.

flowchart LR
    R["A human runs a deck\n(Runner: pass / fail / skip)"] --> S[("qa_runs +\nqa_results")]
    S --> T[Quality trends]
    S --> C[Run comparison]
    S --> M[Milestones]
    S --> H[Run history]
    S --> E[Export: JUnit / Cucumber / Allure]
    E --> CI[CI report → PR comment + commit status]

None of the pages below are reachable from the ⌘K command palette yet — each has its own in-page link (noted per section) or a direct URL. If you're navigating by keyboard shortcut and can't find one of these, that's why.

Quality trends (per deck)

Route: /packages/:packageId/trends, reached via the "View trends" link on the Package Picker's selected-deck panel. Gated the same as /runs — any signed-in user, admin or member — scoped to whichever of the deck's runs you're entitled to see.

Five things, computed only from that deck's completed runs:

Metric	How it's actually computed
Pass-rate series	`pass / (pass + fail)` per completed run, oldest → newest. Skips are excluded from the denominator — a skip says nothing about whether the behaviour behind it actually works.
Fail-count series	Raw fail count per completed run, same order.
Average run duration	Mean of `updatedAt − createdAt` across completed runs. `null` (rendered as "no data yet") if there are none — never a fake `0m 0s`.
Most-failing cases	Top 5 by raw fail count, ties broken by fail rate then case title. A case that's never failed simply isn't on the list.
Flaky candidates	Top 5 cases by number of verdict flips between adjacent completed runs (pass↔fail, either side against skip — all count as a flip). A case that always fails isn't flaky, it's just failing — that's the row above.

Charts are inline SVG bars in brand, not a charting library — a <title> per bar and a screen-reader description carry the same numbers non-visually.

Honest note: "flaky" here means the verdict changed run to run, full stop. There's no clustering, no root-cause guess, nothing inferred about why — that kind of analysis is spec-only and unbuilt (see the AI-advisory backlog in the roadmap).

Run comparison (A vs B)

Entry points: the "⇄ Compare two runs" link on Run History (/runs), or go straight to /runs/compare.

It's a two-click, zero-JS picker: pick a baseline run, then a candidate run, and you land on the actual diff at /runs/[runId]/compare/[otherId]. Baseline and candidate are then re-ordered by wall-clock createdAt, not by which one you clicked first or which id sits in which URL segment — so "regressed" and "fixed" stay meaningful no matter which run you picked as A and which as B.

Cases are matched between the two runs by identity, not by row order:

First pass — by testCaseId (survives a case being renamed later in the Case Editor).
Second pass — by caseKey, for whatever's left unmatched (covers a case deleted and re-created under the same key).

Every case lands in exactly one bucket:

Status	Meaning
`regressed`	Passed on the baseline, fails on the candidate.
`fixed`	Failed on the baseline, passes on the candidate.
`still-failing`	Failed on both.
`still-passing`	Passed on both.
`still-skipped`	Skipped on either side. A skip carries no pass/fail signal, so it's never folded into a regression or a fix.
`new`	Only present on the candidate run.
`removed`	Only present on the baseline run.

Rows render in triage order — regressed, still-failing, new, removed first (what you'd scan for), fixed/still-skipped/still-passing after (the reassuring part).

A comparison across two different decks still renders — matching is by case identity, not by deck — but it carries an inline warning rather than being blocked outright, since it's usually a mis-click rather than a real intent.

Milestones / release grouping

Route: /milestones. Same gate as everywhere else on this page — any signed-in user, admin or member.

A milestone is a name (required, ≤255 characters), an optional description (≤2,000 characters), an optional single-deck scope, and an optional due date. Status is a plain open / closed.

You don't assign a run to a milestone from this page — you do it from that run's own Run Summary page, where a small assignment control sits alongside the summary (deliberately not baked into the summary component itself). Back on /milestones, each milestone card shows a rolled-up pass/fail/skip tally across every run assigned to it that you're entitled to see — never the raw total, so a rollup can't imply visibility into a run you can't actually open.

Nothing here schedules or gates a release automatically — a milestone is a label you attach to runs you've already recorded, and the rollup is a readback of that.

Run history: filter, search, saved views

Route: /runs — every run across every deck, newest first.

The filter bar narrows what's on the page along four axes, all AND'ed together:

Filter	Values
Deck	Exact match against a deck name present on the current page
Status	`All statuses` / `Not started` / `In progress` / `Complete`
Outcome	`Any outcome` / `Has fails` / `All pass`
Date range	Inclusive calendar-day bounds

Plus a free-text Search box matching against the run's label and deck name, case-insensitive substring. An empty search always matches everything — it's a narrowing tool, never an accidental "show nothing."

Has fails catches a still-in-progress run with an early fail already logged — it doesn't wait for the run to finish. All pass needs at least one recorded pass and zero fails, so a pending run or an all-skip run doesn't count as "clean" — it just hasn't vouched for anything yet.

You can name the current filter combination, save it, and re-apply or delete it later — up to 20 per user. Saving under a name you've already used overwrites that view in place rather than piling up "Failing today", "Failing today (2)"…

Honest note: saved views are an MVP — they're stored per-user, in your browser's localStorage, not a server-side table. They don't follow you to another browser or device, and a teammate can't see the views you've saved. Filtering and search also operate over the page of runs already fetched, not the whole run history at once — reaching further back is a noted follow-up, not a silent limitation nobody knew about.

Exporting a run (JUnit / Cucumber / Allure)

Any run marked complete can be downloaded in three standard CI-report shapes:

GET /api/qa/runs/:runId/export?format=junit|cucumber|allure

curl -b cookies.txt \
  "https://qa.example.com/api/qa/runs/8f14e45f-ceea-467e-9de1-0e9c69cc43a1/export?format=junit" \
  -o teasynaer-checkout-regression-junit.xml

This is a normal authenticated route, not a token API — you need a logged-in session (-b cookies.txt above stands in for whatever your HTTP client uses to carry one). There's no bearer-token read path for run results today; the only token-authenticated surface in Teasynaer is machine import, and that's write-only (it creates decks, it doesn't read runs back). More on working around that in the CI report section below.

`format=`	Content-Type	Filename	Good for
`junit`	`application/xml`	`teasynaer-<run-slug>-junit.xml`	Forgejo Actions, GitLab, Jenkins, GitHub Actions test-reporter actions — anything that speaks JUnit
`cucumber`	`application/json`	`teasynaer-<run-slug>-cucumber.json`	Cucumber-JSON-aware dashboards (cucumber-html-reporter, Jenkins' Cucumber plugin)
`allure`	`application/json`	`teasynaer-<run-slug>-allure-results.json`	Allure — with one caveat, below

A few things worth knowing before you wire this in:

Only complete runs export. A pending or in-progress run 400s — exporting mid-pass could read as a false-clean or false-failing report before the human QA pass is actually done.
A run outside your visibility 404s, same as opening it in the UI directly — no separate export-specific ACL to worry about.
Teasynaer records one verdict per case, never per-step. Every export format still has a step-level shape (JUnit's <testcase>, Cucumber's per-step results, Allure's steps), but every synthesized step under a case carries that case's single verdict. There's no per-case duration column either — the elapsed time attributed to each case is derived from the gap between successive verdict timestamps, not measured.
The Allure output is a single JSON array of result objects — the individual pieces an allure-results/ directory is made of, not the directory or a zip itself. To feed allure generate, split the array into one <uuid>-result.json file per entry (using each object's own uuid) first. A literal directory/zip export is on the backlog, not shipped.

CI report → PR comment + commit status

This is a documented recipe, not a shipped integration. Nothing in Teasynaer posts a PR comment or sets a commit status by itself. src/lib/qa/ci-report.ts only formats a run's stored verdicts into a comment body and a commit-status object — pure functions, no network call, no database access. The HTTP call that actually posts either of them is a step in your own CI job, using your own CI token. An in-app "post automatically" feature is deliberately out of scope — it overlaps the (also not-yet-built) Notifier seam, and this repo's rule that Teasynaer never invents a verdict, it only reads one back.

Two outputs, built from the exact same run data the export formats above use:

A PR-comment Markdown body — a pass/fail/skip tally, a link back to the run summary, and, only when there are any, a table of failing cases (case key + title, a preview of the tester's note, any filed-issue link):

<!-- teasynaer-ci-report:8f14e45f-ceea-467e-9de1-0e9c69cc43a1 -->

## Teasynaer QA — Checkout regression, build #482

**11** passed · **1** failed · **0** skipped — [full run summary](<https://qa.example.com/runs/8f14e45f-ceea-467e-9de1-0e9c69cc43a1/summary>)

### Failing cases

| Case | Notes | Issue |
| --- | --- | --- |
| **CHK-07** — Apply a 100%-off coupon at checkout | Coupon accepts the code but the total never drops to £0.00 | [#412](<https://git.example.com/org/repo/issues/412>) |

_Package: Checkout — regression_

The invisible marker on the first line lets a poster script find and update the same PR comment on a later push instead of piling up a new one every time. Every free-text field (case titles, tester notes, tracker refs/URLs) is Markdown- and HTML-escaped, and only an http(s) URL is ever rendered as a live link — a malformed or hostile tracker response degrades to plain text, never an executable link.

A commit status, derived conservatively from the same tally — never an invented decision:

Run state	`state`	Example `description`
pending / in progress	`pending`	`Teasynaer run "Checkout regression, build #482" is still pending`
complete, zero cases recorded	`error`	`Teasynaer run "…" recorded no cases`
complete, ≥1 fail	`failure`	`Teasynaer: 1 failed, 11 passed`
complete, 0 fails	`success`	`Teasynaer: 11 passed, 0 failed`

There's no "run completed" webhook to trigger this — you (or your pipeline) dispatch the report job manually once the run reads complete, the same low-ceremony workflow_dispatch trigger the evidence-retention GC job uses.

On the auth gap above: since there's no bearer-token read endpoint for run results yet, you have two real options for getting getRunSummary's output into a CI job:

Recommended — run the fetch-and-format step wherever the app's own database access already exists (a maintenance step in the same deploy pipeline, or docker compose run against the same Postgres). Call getRunSummary() directly — no HTTP hop, and you get the full shape including filed-issue links.
External CI with no DB access — authenticate with a session cookie for a dedicated, least-privilege "CI reporter" account, stored as a masked secret. Two things to remember if you do this: Better-Auth sessions idle-expire, so the cookie needs periodic refresh; and the three /export formats don't carry issueRef, so a cookie-only integration gets the tally and failing-case names but not filed-issue links unless it also reads the run summary page.

The full Forgejo Actions and GitHub Actions YAML — plus the small glue script that prints the JSON both recipes consume — lives in [`/docs/integrations). This page gives you the shape; that one gives you the pipeline.

What this isn't (yet)

No bearer-token read API for run results. Export and the CI-report data path are session-authenticated only — see above.
No in-app auto-poster, no "run completed" webhook. The CI-report path is a recipe you wire up with your own CI token; nothing fires on its own.
Allure export is a JSON array, not a ready allure-results/ directory or zip. You split it yourself before allure generate.
Saved run views are per-user localStorage, not server-side. No cross-device or cross-teammate sharing yet.
Run history filtering and search operate over the currently-loaded page, not the full run history at once.
No AI anywhere on this page. Flaky detection is a flip count. There's no clustering, dedup, or root-cause guess for a failure — that's spec/backlog only, nothing shipped.