Conformance | GistUI

The five conformance classes and three profiles of GistUI Format 1.0, and how to run the 149-case suite against an implementation.

A claim of conformance names a class, the version and a profile, for example: “GistUI Format 1.0 processor, Core profile”.

Classes

Class	Conforms when
Generator	Every program it writes is clean: a conforming processor reports no diagnostic. It must write strict text and must not rely on a recovery rule.
Processor	It accepts every strict program, applies the recovery rules to text that is not strict and reports each one, produces exactly the tree and diagnostics the specification defines, gives the same result however the input is chunked, and never fails or runs without bound.
Runtime	It meets the requirements for state, expressions, tools, actions and limits, for the profiles it claims.
Renderer	It draws the abstract tree, applies the security policies, meets the accessibility requirements, and reports interaction with the action envelope. The specification does not define pixels.
Catalog	Its manifest is valid against `schemas/catalog.schema.json` and meets the catalog constraints.

Profiles

Profiles are cumulative.

Profile	Adds
Core	Statements, components, tables, references, edits, diagnostics, streaming. The tree is static.
Interactive	State, expressions, built-in functions, `@each`, actions, bindings.
Tools	`@query`, `@mutation`, `@run`.

A Core processor still parses the whole grammar. It keeps a runtime expression as an expression and does not evaluate it. A runtime with Interactive but not Tools evaluates a @query to its default and calls no tool.

The suite

The suite is in spec/conformance/.

File	What it is
`NN-name.gistui`	The input: a program, often an imperfect one
`NN-name.expected.json`	The expected tree, diagnostics in order, and validity flags
`library.json`	The catalog the cases use
`manifest.json`	The list of cases with their profiles

There are 149 cases. They cover every recovery rule, every diagnostic code, tables, edits, limits, cycles, Unicode, line endings and the order of diagnostics.

What passing means

A processor passes a case when it gives the expected tree, the expected diagnostics in order, and the expected validity flags, in two runs:

with the whole text at once;
with the text fed one character at a time.

Diagnostic messages are for people and are not compared. Codes, severities, statement ids, lines and the fixed flag are.

Run it

With the reference implementation:

cd packages/core
bun test conformance

Checking the schemas and the conformance files against them (Node or Bun, no dependencies):

bun spec/schemas/check.mjs

With your own implementation, the loop is short in any language:

for each case in manifest.json:
    catalog  = load library.json
    text     = read NN-name.gistui
    expected = read NN-name.expected.json

    result_whole = process(text, catalog)
    result_chars = process_streaming(each character of text, catalog)

    assert result_whole == expected
    assert result_chars == expected

spec/conformance/README.md describes the files precisely enough to write a runner.

What has no suite yet

Runtime and renderer claims are made against the text of the specification. A suite for them is future work, and a good place to contribute.