Repair a table cut in half, and measure the numbers that had drifted
The README's emulator options table was split by forty lines of prose: two rows, then the whole discussion of the cost model, then five more rows with no header above them. Markdown renders that second half as something other than a table, so three of the seven options were not being shown as options. The rows are back together and the prose follows them. Four numbers had gone stale, in three different ways, and none was noticed: - "Five more scripts run alongside it" - there are six, and lint.sh had no bullet saying what it was for. - "rebuild all three tools" - there are four. - Files.asm quoted at 645 bytes in two places; it is 665. - Edit quoted at 1,983; it is 1,996. The last two are the most quotable sentences in the CosmOS README and the least likely to be rechecked by hand: the programs kept being made better and the sentences about how small they are stayed where they were. So docs.sh measures all four now. It counts the scripts in Tests/ that are not the driver or the disk builder and checks the README says that many and explains each one; it counts what the makefile's all target builds and checks the tool count in both phrasings, which took two attempts because one sentence says "the four tools" and the other "all four tools"; and it assembles every app the CosmOS README quotes a size for and compares. Each check was confirmed by making the fact wrong and watching it fail. WAIT also added a second kind of cycle this morning and the cost model section still described only one. It now says what an idle cycle is, why the two are counted apart, and what that distinction is FOR - a machine that slept through a slow disk and one that spun on it take the same elapsed time and print the same characters, and only the split tells them apart. The duplicated sentence about pipelining is gone; it was said twice, eleven lines apart, in nearly the same words.
This commit is contained in:
Anachronaut
@@ -361,7 +361,7 @@ from every assembly file in it. Several are old programs written for the bare ma
|
||||
| Snake | A game. Draws a whole screen with cursor addressing and steers with single keys, asking the console once a frame and never waiting. |
|
||||
| Keys | The console interrupting rather than being asked. The only one that brings a vector of its own, which is what the version two format exists for. |
|
||||
| Say | Prints whatever it was told, which is the shortest thing that shows osArgument working. |
|
||||
| Files | Writes a file, reads it back, renames it and deletes it, in 645 bytes, including nothing but the service names. It is what says a program does not need a filesystem inside it. |
|
||||
| Files | Writes a file, reads it back, renames it and deletes it, in 665 bytes, including nothing but the service names. It is what says a program does not need a filesystem inside it. |
|
||||
| Break | Stops itself twice with SWI osBreak, so that the registers can be seen changing between one stop and the next. |
|
||||
| Edit | A line editor. |
|
||||
| Stream | Reads an 84,000 byte file through a buffer of 256, which is what says a file bigger than Data Memory can be read at all. |
|
||||
@@ -462,7 +462,7 @@ Typed in as bytes, checked by disassembling it back, and run. It ends with `SWI
|
||||
|
||||
`Edit` is the first program on this machine that makes a file a person typed - every byte on every disk before it was put there by the host tool. It is line oriented in the manner of `ed`: `l` lists, `a` adds at the end, `i` and `c` and `d` take a line number, `w` writes and `q` stops.
|
||||
|
||||
It includes nothing but `services.asm` and `text.asm`: the filesystem and the console are the system's, asked for rather than carried. That is what took it from 4,941 bytes to 1,983 without a line of its own logic changing - and the way that was checked is worth knowing, because the recorded output of the `cosmosEdit` test did not move by a single byte across the rewrite.
|
||||
It includes nothing but `services.asm` and `text.asm`: the filesystem and the console are the system's, asked for rather than carried. That is what took it from 4,941 bytes to 1,996 without a line of its own logic changing - and the way that was checked is worth knowing, because the recorded output of the `cosmosEdit` test did not move by a single byte across the rewrite.
|
||||
|
||||
It keeps the document as a **linked list of lines** rather than one buffer with newlines in it. Each line says where the next one is, how long it is, and then its bytes. Inserting is two pointers changed and nothing moved; with a flat buffer it would mean shifting every byte after the edit, on a machine whose only block move is a device asked politely. The price is that deleted lines are not reused, so a heavy session uses more room than the document needs and writing it out is what tidies up.
|
||||
|
||||
@@ -646,7 +646,7 @@ Running off the end is how a reader finds out it has finished, so it gets an ans
|
||||
|
||||
`Programs/CosmOS/Apps/Stream.asm` reads an 84,000 byte file through a 256 byte buffer, then reads a small file both ways - whole with `osFileRead` and streamed - and checks that the two agree.
|
||||
|
||||
`Programs/CosmOS/Apps/Files.asm` does the whole round trip - write, read, report, rename, delete - in 645 bytes, and includes nothing but the service names.
|
||||
`Programs/CosmOS/Apps/Files.asm` does the whole round trip - write, read, report, rename, delete - in 665 bytes, and includes nothing but the service names.
|
||||
|
||||
`osArgument` is how a program is told what it is for. Everything written before it did the same thing however it was started, which is fine for a program that greets you and no use to one that edits a named document. What arrives is the whole rest of the line, spaces and all, rather than a list of words: what counts as an argument is the program's business, and handing over what was typed is the system's.
|
||||
|
||||
|
||||
@@ -88,6 +88,11 @@ Then `dir` to see what is there, `load Snake.sbx` and `run` to play something, o
|
||||
| --- | --- |
|
||||
| `-d`, `--debug` | Single step through cycles. Each key press advances one instruction. |
|
||||
| `-c`, `--cycles N` | Stop after N cycles rather than running until the program halts. Useful for programs that never halt, and for getting the same output from a run every time. |
|
||||
| `-f`, `--fast` | Run as fast as the host allows, ignoring the emulated cycle rate. |
|
||||
| `-D`, `--disk <file>` | Attach a disk image, creating a 128K one if the file is not there. |
|
||||
| `-L`, `--disk-cycles N` | How many cycles a block read or write takes. Zero, the default, finishes before the next instruction starts. |
|
||||
| `-W`, `--write-protect` | Attach the disk read only. A disk whose image the host will not let you write is read only whether you ask for this or not. |
|
||||
| `-h`, `--help` | Show help and usage information. |
|
||||
|
||||
**A cycle is one access to memory**, not one instruction. Fetching an opcode is a cycle,
|
||||
fetching each byte after it is another, reading or writing Data Memory is one, every byte a
|
||||
@@ -124,15 +129,26 @@ trusting the answer to be there. Zero is the default and is how the machine has
|
||||
|
||||
The waiting is one small routine, and it is reached with `RCAL` rather than `CALL` because
|
||||
what it hands back is the settled status in A, and an ordinary call would put A back the way
|
||||
it found it. Whether real hardware would overlap a fetch with the end of the
|
||||
previous instruction is left open, and deliberately: this is the conservative model, and
|
||||
pipelining is a decision to make while drawing the hardware rather than one to inherit from
|
||||
an emulator.
|
||||
| `-f`, `--fast` | Run as fast as the host allows, ignoring the emulated cycle rate. |
|
||||
| `-D`, `--disk <file>` | Attach a disk image, creating a 128K one if the file is not there. |
|
||||
| `-L`, `--disk-cycles N` | How many cycles a block read or write takes. Zero, the default, finishes before the next instruction starts. |
|
||||
| `-W`, `--write-protect` | Attach the disk read only. A disk whose image the host will not let you write is read only whether you ask for this or not. |
|
||||
| `-h`, `--help` | Show help and usage information. |
|
||||
it found it.
|
||||
|
||||
**And waiting is not the same kind of cycle as working.** A machine stopped in a `WAIT` is
|
||||
clocked but is not using the bus, so those cycles are counted apart from the rest and the
|
||||
halt line says so when there are any:
|
||||
|
||||
```
|
||||
Execution halted after 1042474 cycles, 119772 of them waiting.
|
||||
```
|
||||
|
||||
Added together they are elapsed time, which is what `--cycles` measures. Told apart they
|
||||
say whether a program was working or waiting - and that distinction is the only thing that
|
||||
separates a machine which slept through a slow disk from one which spun on it. The two take
|
||||
the same wall clock time and print the same characters. When the filesystem's wait was
|
||||
first written, taking the line-clearing out of `WAIT` moved the total by a single cycle,
|
||||
20,100 against 20,099, while the idle half halved.
|
||||
|
||||
Whether real hardware would overlap a fetch with the end of the previous instruction is
|
||||
left open, and deliberately: this is the conservative model, and pipelining is a decision
|
||||
to make while drawing the hardware rather than one to inherit from an emulator.
|
||||
|
||||
If the CPU reads a byte that is not an instruction, it goes to the fault handler the program installed. If it installed none, it raises the Fault Flag and halts, and the emulator reports the byte and the address it was found at and exits with a non zero status. The same happens if a program or a device asks for a handler that was never installed.
|
||||
|
||||
@@ -266,18 +282,19 @@ The disk images tests read from are built first by `Tests/makedisks.sh`, using S
|
||||
test that reads one is therefore checked against a filesystem written by different code from
|
||||
the same written specification, rather than against itself.
|
||||
|
||||
`Tests/run.sh` drives that comparison. Five more scripts run alongside it, and each exists
|
||||
`Tests/run.sh` drives that comparison. Six more scripts run alongside it, and each exists
|
||||
because a recorded file cannot answer its question:
|
||||
|
||||
- **`Tests/disk.sh`** checks the disk tool on its own: files of every awkward size onto an image and off again, and the things the format says cannot happen refused rather than half done.
|
||||
- **`Tests/terminal.sh`** checks what a recorded file cannot see. Piped output is buffered and flushed at exit, so a prompt shown before its answer is asked for and one shown an hour late produce identical files; and key mode only touches a terminal when there is one. Both have gone wrong here, and both were found by a person whose terminal stopped working rather than by anything in this suite. So it runs the emulator under a pseudo-terminal and asks directly: that a prompt arrives before input is read, that a keystroke arrives without Return, that the terminal is handed back however the machine dies, and that suspending and resuming leave it as they found it.
|
||||
- **`Tests/terminal.sh`** checks what a recorded file cannot see. Piped output is buffered and flushed at exit, so a prompt shown before its answer is asked for and one shown an hour late produce identical files; and key mode only touches a terminal when there is one. Both have gone wrong here, and both were found by a person whose terminal stopped working rather than by anything in this suite. So it runs the emulator under a pseudo-terminal and asks directly: that a prompt arrives before input is read, that a keystroke arrives without Return, that the terminal is handed back however the machine dies, and that suspending and resuming leave it as they found it. It also asks the one question about cycles that a recorded file cannot, since the count is stripped from every one: whether a program on a slow disk slept through the wait or spun on it. Both print the same characters and take the same elapsed time, and only the split between idle and bus cycles tells them apart.
|
||||
- **`Tests/native.sh`** checks the assembler that runs on SplitBit against the one that runs on the host, byte for byte, on a boot image and four loadable programs, and then on CosmOS and on itself, and then on the CosmOS that CosmOS built.
|
||||
- **`Tests/agree.sh`** checks the two implementations of SBFS against each other rather than each against itself, by building the same disk with SplitDisk and with CosmOS and comparing the images byte for byte. Every field one of them writes and the other only reads is checked there and nowhere else.
|
||||
- **`Tests/lint.sh`** checks SplitLint against a fixture written so that every line of it trips exactly one rule. It compares which warning came out and at which line rather than how many came out in total: a count stays right while the thing behind it goes wrong, and breaking one rule's message left the total untouched at twenty three.
|
||||
- **`Tests/docs.sh`** checks the manuals against the code: that every instruction has a row and every row is an instruction, that the counts in the headings are right, that every directive is written down, that every service the system implements is described and every service described is implemented, that every routine the manuals promise exists, that CosmOS still fits in the half of the machine its memory map gives it, and that the worked examples still assemble to the bytes printed beside them.
|
||||
|
||||
A cycle count is deliberately **not** part of a recorded result. The last line of the emulator's output has the number taken out before anything is compared, keeping only whether the program stopped on its own or ran into its limit, which is behaviour. Two instructions added to CosmOS used to move that number in six unrelated files at once, so a real difference would have arrived in a crowd of meaningless ones. Anything that wants to measure cycles should say so in a test of its own.
|
||||
|
||||
To rebuild all three tools with the address and undefined behaviour sanitizers and run the suite under them:
|
||||
To rebuild all four tools with the address and undefined behaviour sanitizers and run the suite under them:
|
||||
|
||||
```
|
||||
make sanitize
|
||||
|
||||
@@ -13,6 +13,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT" || exit 1
|
||||
|
||||
python3 - <<'PY'
|
||||
import glob
|
||||
import re
|
||||
import sys
|
||||
|
||||
@@ -580,6 +581,75 @@ else:
|
||||
" 0x%s - the buffers are on top of the variables"
|
||||
% (ends - 1, first.group(1).upper()))
|
||||
|
||||
# ---- Every test script the suite runs has a bullet saying why it exists ----
|
||||
#
|
||||
# Two claims in the README went stale at once and neither was noticed: it said FIVE more
|
||||
# scripts run alongside run.sh when there were six, and "all three tools" when there were
|
||||
# four. Both are the kind of number that is written once, is true for months, and is then
|
||||
# quietly wrong - which is the entire subject of this file.
|
||||
#
|
||||
# run.sh is the driver rather than one of the others, and makedisks.sh makes the images
|
||||
# rather than checking anything, so neither is counted.
|
||||
# The repository README rather than CosmOS's, which is what `readme` above holds.
|
||||
rootReadme = open("README.md").read()
|
||||
scripts = sorted(os.path.basename(p) for p in glob.glob("Tests/*.sh")
|
||||
if os.path.basename(p) not in ("run.sh", "makedisks.sh"))
|
||||
words = {"three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9}
|
||||
said = re.search(r"([A-Za-z]+) more scripts run alongside it", rootReadme)
|
||||
if not said:
|
||||
problems.append("the README no longer says how many scripts run alongside run.sh")
|
||||
elif words.get(said.group(1).lower()) != len(scripts):
|
||||
problems.append("the README says %s scripts run alongside run.sh, and there are %d: %s"
|
||||
% (said.group(1), len(scripts), ", ".join(scripts)))
|
||||
for name in scripts:
|
||||
if ("`Tests/%s`" % name) not in rootReadme:
|
||||
problems.append("Tests/%s runs in the suite and the README does not say what it is"
|
||||
" for" % name)
|
||||
|
||||
# ---- And the tool count is the number of things the makefile builds ----
|
||||
makefile = open("makefile").read()
|
||||
built = re.search(r"^all:(.*)$", makefile, re.M)
|
||||
if not built:
|
||||
problems.append("the makefile no longer has an all target this can count")
|
||||
else:
|
||||
tools = len(built.group(1).split())
|
||||
for said in re.findall(r"(?:build|rebuild) (?:all )?(?:the )?([a-z]+) tools", rootReadme):
|
||||
if words.get(said) != tools:
|
||||
problems.append("the README says the %s tools and the makefile builds %d"
|
||||
% (said, tools))
|
||||
|
||||
# ---- The sizes the CosmOS README quotes for its own programs ----
|
||||
#
|
||||
# It said Files was 645 bytes in two places and Edit 1,983. They were 665 and 1,996: the
|
||||
# programs kept being improved and the sentences about how small they are did not move.
|
||||
# These are the most quotable numbers in the document and the least likely to be rechecked
|
||||
# by hand, so they are measured.
|
||||
#
|
||||
# A claim is "<something naming an app> ... in N bytes" on one line. Only lines that name
|
||||
# an app are looked at, so ordinary prose about bytes is left alone.
|
||||
apps = {}
|
||||
for source in glob.glob("Programs/CosmOS/Apps/*.asm"):
|
||||
apps[os.path.basename(source)[:-4]] = source
|
||||
|
||||
for line in readme.split("\n"):
|
||||
said = re.search(r"\b(?:in|to) ([\d,]+) bytes", line)
|
||||
if not said:
|
||||
continue
|
||||
named = [name for name in apps
|
||||
if re.search(r"(?:`|\| )%s(?:\.asm)?\b" % re.escape(name), line)]
|
||||
if len(named) != 1:
|
||||
continue
|
||||
built = subprocess.run(["./Assembler", "-I", "Programs/CosmOS/Source",
|
||||
apps[named[0]], "-o", os.devnull],
|
||||
capture_output=True, text=True)
|
||||
real = re.search(r"Total size: (\d+) bytes", built.stdout)
|
||||
if not real:
|
||||
problems.append("could not measure %s, which the CosmOS README quotes a size for"
|
||||
% named[0])
|
||||
elif int(said.group(1).replace(",", "")) != int(real.group(1)):
|
||||
problems.append("the CosmOS README says %s is %s bytes and it is %s"
|
||||
% (named[0], said.group(1), real.group(1)))
|
||||
|
||||
if problems:
|
||||
print("The manuals and the code disagree:")
|
||||
for p in problems:
|
||||
|
||||
Reference in New Issue
Block a user