How correctness is checked
Warm64's tests are proven on real Arm silicon before they judge the emulator, with QEMU as a second reference.
An emulator is only as good as its tests, so Warm64's tests are checked against hardware before they're used to check Warm64.
The M3 oracle
The instruction-set suites are assembly programs that check their own results. Each is built twice:
- Natively for an Apple M3, where it runs on the real CPU and must pass. This proves the expected results, which come from Arm silicon, not from Warm64.
- For Warm64, where it runs under the interpreter and under WebAssembly, and must pass again.
Suites cover the integer set, floating point, Advanced SIMD, crypto, pointer authentication, system registers, the hypervisor and secure-monitor levels, and 32-bit code.
Randomized suites go further: thousands of floating-point and SIMD operations on random inputs, whose results and status flags are hashed and compared with the M3's.
The QEMU oracle
The M3 lacks some features: SVE, SME, SM3 and SM4, MOPS, 128-bit atomics, and the GIC's virtualization. Those suites run bare-metal under qemu-system-aarch64 -M virt -cpu max as an independent reference, and must agree. Comparing the two found five places where Warm64 differed from QEMU, all since fixed, and one bug in QEMU itself.
The Just-In-Time engine against the interpreter
The Just-In-Time engine must do exactly what the interpreter does. Every suite runs with every block compiled and with none, and must produce the same results and the same instruction count. During development, whole operating-system boots run with the compiled machine and the interpreted one in lockstep, compared as they go, which finds differences at the instruction where they happen.
The whole machine
System tests boot each supported operating system to its prompt, check snapshots by saving a booted machine and resuming it in a new one, and exercise the disks, shared folders, network, sound, USB and 3D graphics from inside the guest.
The QEMU-compatible interface is checked against QEMU 11.1's own sources: every command-line option, QMP command and monitor command must be known.