Formatters API

The executable-stories-formatters package provides a programmatic API to turn test results into reports. It supports Cucumber JSON, HTML, JUnit XML, and Markdown. Framework reporters (Vitest, Jest, Playwright) use this package under the hood; you can also use it directly in custom scripts or CI pipelines.

Installation

Add the formatters package as a dependency (it is typically used alongside a framework package):

pnpm add -D executable-stories-formatters

If you only need adapters in a separate build, you can use the /adapters subpath:

import { adaptJestRun, adaptVitestRun, adaptPlaywrightRun } from "executable-stories-formatters/adapters";

Architecture

Three layers:

1. Adapters — Convert framework-specific results to a raw run (RawRun).
2. Anti-Corruption Layer (ACL) — Normalize to a canonical TestRunResult via canonicalizeRun.
3. Formatters — Turn TestRunResult into Cucumber JSON, HTML, JUnit, or Markdown.

The ReportGenerator class combines adapters + ACL + formatters: you feed it a canonical TestRunResult and options, and it writes files.

Quick start

Normalize framework results, then generate reports:

import {
  normalizeVitestResults,
  ReportGenerator,
} from "executable-stories-formatters";

// After a Vitest run, you have testModules (from the reporter or custom harvest).
const run = normalizeVitestResults(testModules);

const generator = new ReportGenerator({
  formats: ["markdown", "cucumber-json"],
  outputDir: "reports",
  output: { mode: "aggregated" },
});

const written = await generator.generate(run);
// written.get("markdown") → ["reports/test-results.md"]
// written.get("cucumber-json") → ["reports/test-results.cucumber.json"]

Same idea for Jest or Playwright: use normalizeJestResults or normalizePlaywrightResults with the appropriate result shape, then ReportGenerator.

Adapters

Adapters turn framework output into RawRun (input to the ACL).

Adapter	Input	Usage
adaptJestRun	Jest aggregated result + story reports	adaptJestRun(jestResults, storyReports, adapterOptions?)
adaptVitestRun	Vitest test modules	adaptVitestRun(testModules, adapterOptions?)
adaptPlaywrightRun	Playwright test results	adaptPlaywrightRun(testResults, adapterOptions?)

Adapter options are framework-specific (e.g. projectRoot, startedAtMs). See the package types for JestAdapterOptions, VitestAdapterOptions, PlaywrightAdapterOptions.

Normalizers

Convenience functions that run adapter + canonicalizeRun in one step:

• normalizeJestResults(jestResults, storyReports, adapterOptions?, canonicalizeOptions?) → TestRunResult
• normalizeVitestResults(testModules, adapterOptions?, canonicalizeOptions?) → TestRunResult
• normalizePlaywrightResults(testResults, adapterOptions?, canonicalizeOptions?) → TestRunResult

Use these when you have framework results and want a canonical run for ReportGenerator or individual formatters.

ReportGenerator

ReportGenerator accepts only a canonical TestRunResult (create it with normalizers or canonicalizeRun(rawRun, options)).

Options

Option	Type	Default	Description
formats	OutputFormat[]	["cucumber-json"]	Output formats: "cucumber-json", "html", "junit", "markdown".
outputDir	string	"reports"	Base directory for output files.
outputName	string	"test-results"	Base filename (without extension) for aggregated output.
output	OutputConfig	see below	Output routing (mode, colocated style, rules).
cucumberJson	{ pretty?: boolean }	{ pretty: false }	Cucumber JSON options.
html	HtmlOptions	—	Title, darkMode, searchable, startCollapsed, embedScreenshots.
junit	JUnitOptions	—	suiteName, includeOutput.
markdown	MarkdownFormatterOptions	—	title, includeStatusIcons, includeMetadata, includeErrors, scenarioHeadingLevel, stepStyle, groupBy, sortScenarios, includeFrontMatter, includeSummaryTable, permalinkBaseUrl, ticketUrlTemplate, includeSourceLinks, customRenderers.

OutputConfig:

Field	Type	Default	Description
mode	"aggregated" | "colocated"	"aggregated"	Single file vs one file per source.
colocatedStyle	"mirrored" | "adjacent"	"mirrored"	Colocated: mirrored under outputDir or next to source file.
rules	OutputRule[]	[]	Pattern-based overrides (first match wins).
outputName	string	—	Override base filename for rules.

OutputRule: match (glob), mode, colocatedStyle, outputDir, outputName, formats.

Output routing

• Aggregated — All test cases in one file per format under outputDir (e.g. reports/test-results.md).
• Colocated mirrored — One file per source file, directory structure mirrored under outputDir.
• Colocated adjacent — One file per source file, written next to the test file (ignores outputDir for that rule).

Rules allow different routing per path (e.g. src/** colocated, e2e/** aggregated).

Generate

const generator = new ReportGenerator(options);
const result: Map<OutputFormat, string[]> = await generator.generate(run);

result maps each requested format to the list of written file paths.

Individual formatters

You can use formatters without ReportGenerator if you already have a TestRunResult:

• CucumberJsonFormatter — formatToString(run) → string
• HtmlFormatter — format(run) → string
• JUnitFormatter — format(run) → string
• MarkdownFormatter — format(run) → string

Instantiate with the same options as in ReportGenerator (e.g. MarkdownFormatterOptions for Markdown).

ACL and validation

• canonicalizeRun(rawRun, options?) — Normalize RawRun to TestRunResult. Options: attachments, cucumber, defaults.
• validateCanonicalRun(run) — Returns validation result; assertValidRun(run) throws if invalid.

Utilities: normalizeStatus, generateTestCaseId, generateRunId, slugify, deriveStepResults, mergeStepResults, resolveAttachment, resolveAttachments.

Types

Key types exported:

• Canonical: TestRunResult, TestCaseResult, TestCaseAttempt, StepResult, Attachment, TestStatus, CIInfo, CoverageSummary
• Raw: RawRun, RawStatus, RawAttachment, RawStepEvent, RawTestCase, RawCIInfo
• Cucumber JSON: IJsonFeature, IJsonScenario, IJsonStep, IJsonStepResult, etc.
• Options: FormatterOptions, ResolvedFormatterOptions, OutputFormat, OutputMode, ColocatedStyle, OutputRule, CanonicalizeOptions, MarkdownFormatterOptions, MarkdownRenderers

When to use

• Framework reporters — Vitest/Jest/Playwright reporters use this package to produce Markdown (and optionally other formats). You configure them in the framework config; no need to call the formatters API directly.
• Custom scripts — Harvest test results (e.g. from a framework API or JSON output), then call normalize*Results and ReportGenerator to produce HTML, JUnit, or Cucumber JSON in addition to (or instead of) the built-in reporter.
• CI / tooling — Generate multiple formats from one run, or merge runs from multiple projects and then format once.

For reporter options (title, output path, front-matter, etc.) when using the framework reporter, see Vitest reporter options, Jest reporter options, and Playwright reporter options.

CLI

The formatters package provides an executable-stories CLI for generating reports from JSON test results.

Subcommands:

• executable-stories format <file> — Read raw (or canonical) test results and generate reports. Use --format to choose one or more of: html, cucumber-html, markdown, junit, cucumber-json, cucumber-messages. Default format is html.
• executable-stories validate <file> — Validate a JSON file against the schema (no output generated).

Filtering by source file:

• --include <globs> — Comma-separated globs; only test cases whose sourceFile matches at least one pattern are included.
• --exclude <globs> — Comma-separated globs; test cases whose sourceFile matches any pattern are excluded (applied after include).

HTML report options (all enabled by default):

• Step text in the HTML report highlights quoted strings and standalone numbers (step parameter highlighting) for readability.
• --html-no-syntax-highlighting — Disable syntax highlighting in HTML.
• --html-no-mermaid — Disable Mermaid diagram rendering in HTML.
• --html-no-markdown — Disable Markdown parsing in HTML.

CI detection: When the CLI runs in a CI environment, it auto-detects the provider (GitHub Actions, GitLab, CircleCI, Azure DevOps, Buildkite, Jenkins, Travis) from environment variables and attaches branch, commit SHA, PR number, and build URL to the run. The HTML report shows this in a CI meta block. No flags required.

Notifications: After generating reports, the CLI can send a summary to Slack, Microsoft Teams, or a generic webhook. Use --slack-webhook or --teams-webhook (or SLACK_WEBHOOK_URL / TEAMS_WEBHOOK_URL env), or --webhook-url (repeatable) for a generic HTTP endpoint. --notify controls when: always, on-failure (default), or never. --report-url supplies a link to the report in notification messages. Optional HMAC signing: --webhook-hmac-secret, --webhook-hmac-header, --webhook-hmac-timestamp.

Run history: Use --history-file <path> to persist run history to a JSON file. The CLI updates it after each run and uses it to show flakiness, stability grade (A–F), and performance trend in the HTML report. --max-history-runs <n> (default 10) caps how many runs are kept per test. Omit --history-file to disable history.

Standalone binary: From the formatters package directory, run bun run compile to build a single executable-stories binary. CI builds produce platform-specific binaries (e.g. executable-stories-linux-x64); the release workflow uploads multi-platform binaries (linux-x64, linux-arm64, darwin-x64, darwin-arm64, windows-x64) as the formatters-binaries artifact.

format flags reference

Flag	Type	Default	Description
--format	string	html	Output format(s): html, cucumber-html, markdown, junit, cucumber-json, cucumber-messages
--output-dir	string	reports	Directory to write output files
--output-name	string	test-results	Base filename (without extension) for aggregated output
--input-type	string	raw	Input type: raw, canonical, or ndjson
--sort-test-cases	string	none	Sort scenarios: id, source, or none
--include-tags	string	—	Comma-separated tags to include (any match)
--exclude-tags	string	—	Comma-separated tags to exclude (any match)
--include	string	—	Glob patterns to include by source file
--exclude	string	—	Glob patterns to exclude by source file
--synthesize-stories	boolean	true	Synthesize story metadata for plain tests
--no-synthesize-stories	boolean	—	Disable story synthesis (strict mode)
--html-no-syntax-highlighting	boolean	false	Disable syntax highlighting in HTML
--html-no-mermaid	boolean	false	Disable Mermaid diagram rendering in HTML
--html-no-markdown	boolean	false	Disable Markdown parsing in HTML
--html-permalink-base-url	string	—	Base URL for source file permalinks (e.g. https://github.com/org/repo/blob/main)
--html-ticket-url-template	string	—	URL template for ticket links (use {ticket} placeholder)
--html-no-toc	boolean	false	Disable table of contents sidebar
--html-theme-picker	boolean	false	Embed all themes with a picker UI
--asset-mode	string	none	Asset bundling: none or copy
--allow-missing-assets	boolean	false	Warn instead of fail on missing assets
--output-name-timestamp	boolean	false	Append UTC timestamp to output filename
--emit-canonical	string	—	Write canonical JSON to given path
--json-summary	boolean	false	Print machine-parsable JSON summary
--history-file	string	—	Path to run history JSON file
--max-history-runs	number	10	Maximum runs to keep per test in history
--slack-webhook	string	—	Slack webhook URL for notifications
--teams-webhook	string	—	Microsoft Teams webhook URL for notifications
--webhook-url	string	—	Generic webhook URL (repeatable)
--notify	string	on-failure	When to send notifications: always, on-failure, or never
--report-url	string	—	Link to the report included in notification messages
--webhook-hmac-secret	string	—	HMAC secret for webhook signing
--webhook-hmac-header	string	—	Header name for HMAC signature
--webhook-hmac-timestamp	boolean	false	Include timestamp in HMAC signing

compare

Compare two test runs and generate a diff report showing regressions, fixes, and changes.

executable-stories compare current.json --baseline baseline.json --format html

Flag	Type	Default	Description
--baseline	string	—	Baseline JSON file, or auto to pick the most recent prior run
--baseline-dir	string	—	Directory to scan when using --baseline auto
--pr-summary	boolean	false	Print PR-friendly markdown summary to stdout
--pr-summary-file	string	—	Write the PR summary to a file

Inherits all format flags. Only html and markdown formats are supported for diff reports.

Auto-baseline:

executable-stories compare current.json \
  --baseline auto \
  --baseline-dir .executable-stories/history/ \
  --format html

PR summary for CI:

executable-stories compare current.json \
  --baseline baseline.json \
  --pr-summary-file pr-comment.md

list

List all scenarios from a test run.

executable-stories list raw-run.json

Flag	Type	Default	Description
--include-tags	string	—	Comma-separated tags to include
--exclude-tags	string	—	Comma-separated tags to exclude
--json-summary	boolean	false	Output as JSON instead of text table
--input-type	string	raw	Input type: raw, canonical, or ndjson
--stdin	boolean	false	Read from stdin