Three claims in the Test Manual were true enough to pass and loose enough to mislead.
The headline said the two assemblers produce identical bytes and the two filesystems
produce identical disks, with no qualifier. That reads as universal and is not: it is
evidence about the corpus assembled and the operations performed. The detailed sections
already said so; the headline now says so too, because a document arguing about what
evidence is worth cannot overstate its own.
"Compiles under strict ISO C" omitted -D_XOPEN_SOURCE=700. The check is strict C11 with
the POSIX interfaces the code uses explicitly selected - realpath, strdup, dirname and
getopt - not freestanding ISO C, and the flag is part of the check rather than a hole
in it.
And the sanitizer section leaned on AddressSanitizer's junk fill, which is a toolchain
default this build does not configure and which buys almost nothing here anyway: there
are six heap allocations in the whole repository, all in the assembler, the largest a
deliberate calloc, and the machine's own memories are static arrays the sanitizers
neither fill nor bound-check. That last part is the overrun blind spot seen from the
other side, so it now points at it. The same claim is corrected in the makefile, where
it originated.
Found by ChatGPT reviewing the manual.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
The test system had grown to seven scripts making five genuinely different
kinds of claim, and nothing said which was which. A recorded transcript and a
byte-for-byte comparison against a second implementation both print [ok ] and
are worth wildly different amounts, so the fourth manual exists to say so: what
each script can and cannot answer, why every determinism rule is there, how to
add a test, and - the part written nowhere else - where the suite is blind.
That last section is the reason for the document. Three buffer overruns into
adjacent variables were all found by a person using the machine and none by the
suite, the sanitizers cannot see them because emulated Data Memory is one
legitimate host array, and there is no second opinion about the CPU at all. A
document listing only strengths teaches the wrong lesson.
The bullets describing each script move out of the README, so docs.sh now reads
the manual for them, and five more numbers in it are settled from the source
rather than trusted: the shape of the manifest, the xfail count, how many
fixture disks makedisks.sh builds, how large the lint baseline is, and the tool
count in either document. Each of the new checks was broken on purpose and
watched to report before being kept, which is the discipline the manual itself
argues for.
Also drops the stale "70 instructions" from instructiontable.py's docstring.
There are 72, and a number that carries no meaning is better removed than
corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
AnachronautandClaude Opus 5