diff --git a/SplitBit Test Manual.md b/SplitBit Test Manual.md index ce07a4e..1d5ea76 100644 --- a/SplitBit Test Manual.md +++ b/SplitBit Test Manual.md @@ -14,11 +14,16 @@ knowing which claim you are relying on is the whole point of this document. A re transcript and a byte-for-byte comparison against a second implementation both print `[ok ]`, and they are worth wildly different amounts. -The strongest thing the suite says is this: **the assembler that runs on SplitBit and the -assembler that runs on the host produce identical bytes, and the filesystem written by the -tool and the filesystem written by the machine are identical disks.** Those are two -programs written from one specification, sharing no code, checking each other. Nothing else -here is that strong. +The strongest thing the suite says is this: **on the source it is given, the assembler that +runs on SplitBit and the one that runs on the host produce identical bytes; and on the +operations it performs, the filesystem written by the tool and the filesystem written by the +machine are identical disks.** Those are two programs written from one specification, +sharing no code, checking each other. + +Note the qualifiers, because they are the whole difference between a strong claim and an +untrue one. This is evidence about a corpus and a sequence of operations, not a proof about +every program that could be assembled or every disk that could be built. No suite says more +than that. But nothing else here is even that strong. The weakest thing it says is that a program prints what it printed last time. That is worth having and it is worth having a lot of, but it only ever catches change. It cannot catch a @@ -354,10 +359,16 @@ make sanitize ``` Rebuilds all four tools with `-fsanitize=address,undefined` and runs **the whole suite** -under them. It catches reads and writes off the end of an array, use after free, leaks, and -undefined arithmetic. It also fills fresh allocations with a junk pattern, which is what -turns a read of uninitialised memory from something that quietly works into something the -tests notice. +under them. What it reliably catches is invalid access: reads and writes off the end of an +array, use after free, leaks, and arithmetic the standard does not define. + +AddressSanitizer also fills fresh heap allocations with a junk pattern, and it is worth +knowing why that buys almost nothing here. It is a default of the toolchain rather than +anything this build configures, so it is not something to rely on; and there are **six heap +allocations in the whole repository**, all of them in the assembler, the largest a +deliberate `calloc`. The machine's Program and Data memories are static arrays, which the +sanitizers neither fill nor bound-check - which is the same fact, seen from a +different side, as the overrun blind spot below. It runs everything because it used to not. It built all four tools sanitized and then ran only `run.sh` and `terminal.sh`, so SplitDisk was compiled with the sanitizers and never @@ -374,7 +385,14 @@ Worth running before a release, and after anything that touches memory handling. ## The Strict Build: `make test` depends on `make strict`, which compiles every source file with -`-std=c11 -pedantic -Wall -Wextra -Werror` and throws away the object. +`-std=c11 -pedantic -Wall -Wextra -Werror` **and `-D_XOPEN_SOURCE=700`**, then throws away +the object. + +That last flag is part of the check rather than a hole in it, and the distinction matters: +this is strict C11 with the POSIX interfaces the code actually uses explicitly selected, not +freestanding ISO C. The sources call `realpath`, `strdup`, `dirname` and `getopt`, and +asking for POSIX.1-2008 by name is what makes a strict C11 build declare them rather than +guess. This exists because the README says "the sources are ISO C and build clean under `-std=c11 -pedantic` with `-Wall -Wextra`", and that is a claim somebody may check by diff --git a/makefile b/makefile index 29ca3e7..0250433 100644 --- a/makefile +++ b/makefile @@ -164,9 +164,10 @@ test: $(EMU_TARGET) $(ASM_TARGET) $(DSK_TARGET) $(LINT_TARGET) strict # are for. Adding the three of them costs about six seconds. # # The sanitizers catch reads and writes off the end of an array, use after free, -# leaks, and undefined arithmetic. They also fill fresh allocations with a junk -# pattern, which is what turns a read of uninitialised memory from something that -# quietly works into something the tests notice. +# leaks, and undefined arithmetic. What they do NOT usefully catch here is uninitialised +# memory: AddressSanitizer's junk fill is a toolchain default this build does not +# configure, there are six heap allocations in the repository and the largest is a +# deliberate calloc, and the machine's own memories are static arrays it never touches. # # If the suite fails, the sanitizer binaries are deliberately left in place so # that the failing case can be run again by hand. 'make' puts the normal ones back.