Say what the strict build and the sanitizers actually establish

Three claims in the Test Manual were true enough to pass and loose enough to mislead.

The headline said the two assemblers produce identical bytes and the two filesystems
produce identical disks, with no qualifier. That reads as universal and is not: it is
evidence about the corpus assembled and the operations performed. The detailed sections
already said so; the headline now says so too, because a document arguing about what
evidence is worth cannot overstate its own.

"Compiles under strict ISO C" omitted -D_XOPEN_SOURCE=700. The check is strict C11 with
the POSIX interfaces the code uses explicitly selected - realpath, strdup, dirname and
getopt - not freestanding ISO C, and the flag is part of the check rather than a hole
in it.

And the sanitizer section leaned on AddressSanitizer's junk fill, which is a toolchain
default this build does not configure and which buys almost nothing here anyway: there
are six heap allocations in the whole repository, all in the assembler, the largest a
deliberate calloc, and the machine's own memories are static arrays the sanitizers
neither fill nor bound-check. That last part is the overrun blind spot seen from the
other side, so it now points at it. The same claim is corrected in the makefile, where
it originated.

Found by ChatGPT reviewing the manual.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW

SplitBit Test Manual.md

@@ -14,11 +14,16 @@ knowing which claim you are relying on is the whole point of this document. A re
 transcript and a byte-for-byte comparison against a second implementation both print
 `[ok  ]`, and they are worth wildly different amounts.
 
-The strongest thing the suite says is this: **the assembler that runs on SplitBit and the
+The strongest thing the suite says is this: **on the source it is given, the assembler that
-assembler that runs on the host produce identical bytes, and the filesystem written by the
+runs on SplitBit and the one that runs on the host produce identical bytes; and on the
-tool and the filesystem written by the machine are identical disks.** Those are two
+operations it performs, the filesystem written by the tool and the filesystem written by the
-programs written from one specification, sharing no code, checking each other. Nothing else
+machine are identical disks.** Those are two programs written from one specification,
-here is that strong.
+sharing no code, checking each other.
+
+Note the qualifiers, because they are the whole difference between a strong claim and an
+untrue one. This is evidence about a corpus and a sequence of operations, not a proof about
+every program that could be assembled or every disk that could be built. No suite says more
+than that. But nothing else here is even that strong.
 
 The weakest thing it says is that a program prints what it printed last time. That is worth
 having and it is worth having a lot of, but it only ever catches change. It cannot catch a
@@ -354,10 +359,16 @@ make sanitize
 ```
 
 Rebuilds all four tools with `-fsanitize=address,undefined` and runs **the whole suite**
-under them. It catches reads and writes off the end of an array, use after free, leaks, and
+under them. What it reliably catches is invalid access: reads and writes off the end of an
-undefined arithmetic. It also fills fresh allocations with a junk pattern, which is what
+array, use after free, leaks, and arithmetic the standard does not define.
-turns a read of uninitialised memory from something that quietly works into something the
+
-tests notice.
+AddressSanitizer also fills fresh heap allocations with a junk pattern, and it is worth
+knowing why that buys almost nothing here. It is a default of the toolchain rather than
+anything this build configures, so it is not something to rely on; and there are **six heap
+allocations in the whole repository**, all of them in the assembler, the largest a
+deliberate `calloc`. The machine's Program and Data memories are static arrays, which the
+sanitizers neither fill nor bound-check - which is the same fact, seen from a
+different side, as the overrun blind spot below.
 
 It runs everything because it used to not. It built all four tools sanitized and then ran
 only `run.sh` and `terminal.sh`, so SplitDisk was compiled with the sanitizers and never
@@ -374,7 +385,14 @@ Worth running before a release, and after anything that touches memory handling.
 ## The Strict Build:
 
 `make test` depends on `make strict`, which compiles every source file with
-`-std=c11 -pedantic -Wall -Wextra -Werror` and throws away the object.
+`-std=c11 -pedantic -Wall -Wextra -Werror` **and `-D_XOPEN_SOURCE=700`**, then throws away
+the object.
+
+That last flag is part of the check rather than a hole in it, and the distinction matters:
+this is strict C11 with the POSIX interfaces the code actually uses explicitly selected, not
+freestanding ISO C. The sources call `realpath`, `strdup`, `dirname` and `getopt`, and
+asking for POSIX.1-2008 by name is what makes a strict C11 build declare them rather than
+guess.
 
 This exists because the README says "the sources are ISO C and build clean under
 `-std=c11 -pedantic` with `-Wall -Wextra`", and that is a claim somebody may check by

makefile

@@ -164,9 +164,10 @@ test: $(EMU_TARGET) $(ASM_TARGET) $(DSK_TARGET) $(LINT_TARGET) strict
 # are for. Adding the three of them costs about six seconds.
 #
 # The sanitizers catch reads and writes off the end of an array, use after free,
-# leaks, and undefined arithmetic. They also fill fresh allocations with a junk
+# leaks, and undefined arithmetic. What they do NOT usefully catch here is uninitialised
-# pattern, which is what turns a read of uninitialised memory from something that
+# memory: AddressSanitizer's junk fill is a toolchain default this build does not
-# quietly works into something the tests notice.
+# configure, there are six heap allocations in the repository and the largest is a
+# deliberate calloc, and the machine's own memories are static arrays it never touches.
 #
 # If the suite fails, the sanitizer binaries are deliberately left in place so
 # that the failing case can be run again by hand. 'make' puts the normal ones back.