Benchmarks

GistUI measured against OpenUI: rendering speed in a real browser, accuracy and tokens with real models, and bundle size. Every screen, script and answer is in this repository. The same results, with charts, are at gistui.com/benchmarks; the full report is benchmark.md.

Last updated 2026-10-05.

1. Summary

• Rendering: 2–6× less main-thread work to render a streaming answer, across OpenUI’s own sample screens (section 2).
• Accuracy: with gpt-oss-120b, 3.3 points more valid answers as written; 100% after the code-only repair (section 4). With the small gpt-5-nano, the two are even as written (22.8% against 22.3%) and the repair takes GistUI to 99.5%.
• Tokens: 21% fewer tokens per screen, prompt and answer together.

2. Rendering a streaming answer (bench/render)

How much work does the page do while an answer streams in?

• Setup. One React 19 page in Chrome (Playwright), production builds. OpenUI renders with its default library (@openuidev/react-lang 0.3.0, @openuidev/react-ui 0.16.3, its documented setup); GistUI with its own (@gistui/react, this repository). Each run loads only one of them, in a fresh page.
• Screens. OpenUI’s seven sample screens from its own repository (benchmarks/samples), updated to its current syntax because the current library rejects parts of the originals, and matching GistUI versions. Both parse with zero errors. The eighth screen puts all seven in one answer.
• Stream. The text grows by 4 characters every 10 ms (about 100 tokens a second), or every 1 ms (about 1,000), on a fixed clock: a page that falls behind receives several chunks at once, as it would from the network.
• Measured. Chrome’s own main-thread counters (script, layout, style, all tasks) from the first chunk until the screen stops changing; also first content, long tasks and frame times. Median of 3 runs.

Normal CPU, about 100 tokens a second

CPU slowed 4× (a mid-range phone), about 100 tokens a second

Normal CPU, about 1,000 tokens a second

CPU slowed 4×, about 1,000 tokens a second

Reading these numbers.

• GistUI needs 2–6× less main-thread work on most screens; the gap grows with the length of the answer. On a tiny screen the two are about the same.
• Neither library made the page janky: no task blocked it for more than a few milliseconds, and the worst frame was under 70 ms for both.
• GistUI shows first content sooner on every screen.
• GistUI’s version of each screen is about a third shorter, so part of the difference is the format: less text to process.
• With a slowed CPU some totals go down: a page that falls behind receives bigger batches and renders fewer times. Compare the two libraries within a table, not across tables.

3. The parsers alone (bench/render/parse.ts)

The parser inside that work, without rendering. Each parser reads the same screens in chunks and, after every chunk, produces an up-to-date result, as a renderer needs. OpenUI’s streaming parser (@openuidev/lang-core 0.3.0) returns one from push(); GistUI’s stream builds one on flush(). Times are OpenUI / GistUI for the whole screen, median of 21 runs (5 for the largest), Bun.

Read at once, the two parsers are about as fast. Streamed, OpenUI re-reads the whole answer on every chunk, so its time grows with the square of the answer’s length; GistUI reads each new character once. Parsing is a small part of rendering, though: section 2 is the number that reaches a person.

An earlier version of this report quoted “130× faster streaming parse”. That measurement let GistUI skip building its result for the line still being written, so it was not like for like; it is replaced by this table and section 2.

4. Accuracy and tokens with a real model (bench/genui)

The benchmark. thesysdev/generative-ui-bench at commit fcca05a, the benchmark behind openui.com/benchmarks, built by Thesys, the company that makes OpenUI. GistUI was added as a format next to OpenUI Lang; nothing else was changed.

The task. 46 prompts in 5 difficulty levels, from 2–3 required components (an invoice view) to 16–18 (an emergency response center). Each runs 4 times: 184 answers per format. Two GistUI requests were rate-limited by the provider, so both formats are compared on the same 182 answers.

The same for both formats: the 70-component catalog, the two worked examples, the rules and the per-group notes, and the model settings (temperature 0.7, minimal reasoning, 16,384-token limit). Only the language differs.

What “valid” means (the benchmark’s own definition): the answer parses, has a root, every reference resolves and every statement is reachable, required props are present, enum values are valid, it was not truncated, and it has at least as many components as the prompt requires. OpenUI is scored by its own library (@openuidev/lang-core 0.2.16); GistUI by bench/genui/validator.ts, which applies the same rules as strictly.

gpt-oss-120b, through the Vercel AI Gateway

By screen complexity:

• GistUI failures, as written: wrong argument shape 13, missing required prop 9, wrong enum value 6, references left dangling or unused 6, syntax 1. An answer can have more than one.
• OpenUI failures: invalid enum 17, signature mismatch 13, reference errors 7, unknown components 2, missing required prop 1.
• The OpenUI answers come from an earlier run through the same gateway, with the same prompts and settings (this model has no published answers on the benchmark’s site). Cost per 46 screens is at the gateway’s list price ($0.10 per million input tokens, $0.50 per million output), answer tokens only; the four GistUI passes, hidden reasoning included, were billed $0.18.

gpt-5-nano, through the Vercel AI Gateway

A small, cheap model. Both formats were generated in the same session, through the same gateway, with each format’s own prompt from the benchmark (BENCH_FORMAT=openui sends OpenUI’s).

By screen complexity:

• As written, the two formats are even, and both are weak: this model often defines a section and never places it, or refers to one it never wrote (GistUI 117 answers, OpenUI 129).
• GistUI answers are 29% shorter (605 tokens against 850), and no GistUI screen came out blank.
• The repair takes GistUI to 99.5%. The one answer it cannot fix is a single root line that refers to 15 sections the model never wrote: code cannot add what is not there.
• Cost per 46 screens is at the gateway’s list price ($0.05 per million input tokens, $0.40 per million output), answer tokens only. The run was billed $0.05 (GistUI) and $0.07 (OpenUI). The gateway’s free tier allows 5 requests a minute per provider; BENCH_RPM and BENCH_GATEWAY_PROVIDERS pace the run.

The repair (@gistui/core, autofix)

Plain code with no model call. It fixes each error where it points and parses again, for up to 6 rounds, and lists every change. OpenUI also repairs, with a second model call in OpenUI Cloud; its benchmark does not include that step, so the OpenUI numbers here are as the model wrote them.

With gpt-oss-120b, no repaired screen lost a component, and the repair placed 35 components the model wrote but never put on the screen. Checking an answer takes about 0.3 ms (median); checking plus repair 1–2 ms (median), 7 ms at the slowest. It runs once, when the stream ends. Only code is changed: text in strings, table cells and chat around a fenced program stays as the model wrote it.

Across every GistUI answer generated for this benchmark so far (1,159, from nine runs of five models; the earlier runs are in this repository’s history), the repair makes 99.4% valid. Six of the seven it cannot fix are empty answers; the seventh is the single root line above.

5. Caveats

• Two models. The model test uses gpt-oss-120b and gpt-5-nano; more will follow.
• The rendering test uses each library’s own components, which is what you get by default. A different component library changes the numbers.
• Randomness. Temperature 0.7 adds a few points of noise to validity, and each figure here is one run of 46 screens × 4. With gpt-oss-120b, three GistUI runs with slightly different prompts scored 86.7%, 86.8% and 87.9% (the last is the one reported); OpenUI’s one run scored 84.6%. Read a gap of about 3 points as “slightly ahead”, not as settled.
• The benchmark prompt has three rules the library’s default prompt does not, written after reading failures on these same screens.
• The model test checks structure, not appearance. Its answers use the benchmark’s 70-component catalog, not GistUI’s own components.

6. Reproduce

# Rendering and parsers (no API key)
cd bench/render && bun install && bunx vite build && bunx vite preview --port 5310 &
node run.mjs && bun parse.ts && node report.mjs    # results/render.json, parse.json, RESULTS.md

# The model test (calls a paid model; BENCH_BUDGET_USD is a hard stop)
bun run bench:genui:setup
BENCH_PROVIDER=vercel AI_GATEWAY_API_KEY=… BENCH_MODEL=openai/gpt-oss-120b BENCH_LABEL=gptoss120b-v3 \
BENCH_BUDGET_USD=0.50 bun run bench:genui:run
bun run bench:genui:score gptoss120b-v3

# A model the benchmark has no OpenUI answers for: generate both formats, then score
BENCH_PROVIDER=vercel BENCH_MODEL=openai/gpt-5-nano BENCH_LABEL=gpt5nano BENCH_RPM=4 \
BENCH_GATEWAY_PROVIDERS=azure,openai BENCH_BUDGET_USD=0.30 bun run bench:genui:run
BENCH_FORMAT=openui BENCH_PROVIDER=vercel BENCH_MODEL=openai/gpt-5-nano BENCH_LABEL=gpt5nano BENCH_RPM=4 \
BENCH_GATEWAY_PROVIDERS=azure,openai BENCH_BUDGET_USD=0.30 bun run bench:genui:run
bun run bench:genui:score gpt5nano

# Scoring alone needs no key: the answers of both formats are in bench/genui/raw/<label>
bun run bench:genui:setup && bun run bench:genui:score gptoss120b-v3 gpt5nano