Benchmark methodology before results

A benchmark result is not publishable evidence until the model, runtime, hardware, workload, procedure, statistics, raw data, and limitations are recorded.

Performance numbers become misleading when environment and method are omitted. Define the benchmark record before collecting data.

Minimum record

Capture exact model and revision, runtime and commit, backend, precision, quantization, device, CPU, GPU, memory, operating system, browser when relevant, input, warm-up, sample count, median, P95 when available, throughput unit, peak memory, UTC date, methodology version, raw data, limitations, and reproduction steps.

Comparison rules

Do not compare results from incompatible hardware, context, model variants, precision, or workloads as if only one variable changed. Preserve the accessible data table even when an SVG chart is added.