diff --git a/SplitBit Test Manual.md b/SplitBit Test Manual.md index dbbf339..0ee7f1a 100644 --- a/SplitBit Test Manual.md +++ b/SplitBit Test Manual.md @@ -350,7 +350,7 @@ them. Four forms: | Form | Meaning | | --- | --- | | `name.img` | Scratch. Removed before the run, so the emulator makes a blank one | -| `disks/name.img` | A fixture built by `makedisks.sh`, used as it stands | +| `disks/name.img` | A fixture built by `makedisks.sh`, copied fresh for the run | | `...:ro` | Attached write protected, so a test can check the *device* bars writes | | `...@N` | Given a latency of N cycles, so a test can check the filesystem waits | @@ -358,6 +358,15 @@ The `@N` form deserves a note. Every other test runs with the disk's answer avai before the next instruction, which is the one condition under which failing to wait looks exactly like working. +**Both kinds of disk are fresh for every test**, and the fixture was not always. It used to +be handed to each test where it lay, and twenty four tests name `disks/cosmos.img` while +several of them write to one. `romBoot` was the test that found it: its recorded output +described a directory that `selfBoot` had made earlier in the same run, so it passed in a +full run and failed on its own. **That is the worst way round for a test to be wrong** - the +form nobody runs is the one that tells the truth - and the shape of it is worth naming, +because it is the same shape as a program that works only because of what ran before it. +The whole suite was checked one test at a time afterwards, and that was the only one. + **keys** names a file in `Tests/input` to be fed to the console as a *keyboard* rather than as standard input, and the difference between those is the whole reason the field exists. diff --git a/Tests/expected/romBoot.out b/Tests/expected/romBoot.out index bc027b8..bf05c04 100644 --- a/Tests/expected/romBoot.out +++ b/Tests/expected/romBoot.out @@ -9,7 +9,7 @@ deleted it and it is gone finished > mkdir Notes -cannot make that: check the path, the name, and whether it is taken +made > cd Notes /Notes> dir 0 files diff --git a/Tests/input/cosmosEditKeys.in b/Tests/input/cosmosEditKeys.in index 8a8a00e..d223682 100644 --- a/Tests/input/cosmosEditKeys.in +++ b/Tests/input/cosmosEditKeys.in @@ -1,7 +1,7 @@ -eco ok„ƒƒh -Xecho ok„† -echo o„…k +eco ok„ƒƒh +Xecho ok„† +echo o„…k echo okX -echo k‚o -Xecho aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa„†…! +echo k‚o +Xecho aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa„†…! exit diff --git a/Tests/run.sh b/Tests/run.sh index 493ce74..1afd53f 100755 --- a/Tests/run.sh +++ b/Tests/run.sh @@ -248,9 +248,9 @@ while IFS='|' read -r name src mode stdin limit disk keys; do EMUARGS+=(--keyboard "$INPUT/$keys") fi [ "$limit" != "-" ] && EMUARGS+=(--cycles "$limit") - # A disk starts fresh for every run, so a test cannot pass because of what a - # previous one left lying on it. The emulator makes the image if it is - # missing, which is what removing it first arranges for. + # A disk starts fresh for every test, so a test cannot pass because of what a + # previous one left lying on it. How that is arranged differs between a scratch + # image and a built fixture, and both are below. # ---- More than one, separated by a plus ---- # # The machine has four drives, so a test may name up to four images and they @@ -277,12 +277,24 @@ while IFS='|' read -r name src mode stdin limit disk keys; do case "$DISKFILE" in *@*) DISKWAIT="${DISKFILE##*@}"; DISKFILE="${DISKFILE%@*}" ;; esac - # A name with a directory in it is one of the images makedisks.sh built, - # and is used as it stands. A bare name is scratch: it is removed first so - # that nothing a test writes can be seen by the next one, and the emulator - # makes a blank image in its place. + # ---- A fresh disk for every test, whichever kind it is ---- + # + # A bare name is scratch: it is removed, and the emulator makes a blank image + # in its place. + # + # A name with a directory in it is one of the images makedisks.sh built, and + # it is COPIED rather than used where it lies. Twenty four tests name + # disks/cosmos.img and several of them write to it, so a test used to be able + # to hand the next one a disk with its leavings on. That is not hypothetical: + # romBoot's recorded output described a directory that selfBoot had made, so + # it passed in a full run and failed on its own, which is the worst way round + # for a test to be wrong. case "$DISKFILE" in - */*) ;; + */*) + mkdir -p "$BUILD/.fixture" + cp "$BUILD/$DISKFILE" "$BUILD/.fixture/${DISKFILE##*/}" + DISKFILE=".fixture/${DISKFILE##*/}" + ;; *) rm -f "$BUILD/$DISKFILE" ;; esac EMUARGS+=(--disk "$BUILD/$DISKFILE")