Repair a table cut in half, and measure the numbers that had drifted
The README's emulator options table was split by forty lines of prose: two rows, then the whole discussion of the cost model, then five more rows with no header above them. Markdown renders that second half as something other than a table, so three of the seven options were not being shown as options. The rows are back together and the prose follows them. Four numbers had gone stale, in three different ways, and none was noticed: - "Five more scripts run alongside it" - there are six, and lint.sh had no bullet saying what it was for. - "rebuild all three tools" - there are four. - Files.asm quoted at 645 bytes in two places; it is 665. - Edit quoted at 1,983; it is 1,996. The last two are the most quotable sentences in the CosmOS README and the least likely to be rechecked by hand: the programs kept being made better and the sentences about how small they are stayed where they were. So docs.sh measures all four now. It counts the scripts in Tests/ that are not the driver or the disk builder and checks the README says that many and explains each one; it counts what the makefile's all target builds and checks the tool count in both phrasings, which took two attempts because one sentence says "the four tools" and the other "all four tools"; and it assembles every app the CosmOS README quotes a size for and compares. Each check was confirmed by making the fact wrong and watching it fail. WAIT also added a second kind of cycle this morning and the cost model section still described only one. It now says what an idle cycle is, why the two are counted apart, and what that distinction is FOR - a machine that slept through a slow disk and one that spun on it take the same elapsed time and print the same characters, and only the split tells them apart. The duplicated sentence about pipelining is gone; it was said twice, eleven lines apart, in nearly the same words.
This commit is contained in:
@@ -13,6 +13,7 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT" || exit 1
|
||||
|
||||
python3 - <<'PY'
|
||||
import glob
|
||||
import re
|
||||
import sys
|
||||
|
||||
@@ -580,6 +581,75 @@ else:
|
||||
" 0x%s - the buffers are on top of the variables"
|
||||
% (ends - 1, first.group(1).upper()))
|
||||
|
||||
# ---- Every test script the suite runs has a bullet saying why it exists ----
|
||||
#
|
||||
# Two claims in the README went stale at once and neither was noticed: it said FIVE more
|
||||
# scripts run alongside run.sh when there were six, and "all three tools" when there were
|
||||
# four. Both are the kind of number that is written once, is true for months, and is then
|
||||
# quietly wrong - which is the entire subject of this file.
|
||||
#
|
||||
# run.sh is the driver rather than one of the others, and makedisks.sh makes the images
|
||||
# rather than checking anything, so neither is counted.
|
||||
# The repository README rather than CosmOS's, which is what `readme` above holds.
|
||||
rootReadme = open("README.md").read()
|
||||
scripts = sorted(os.path.basename(p) for p in glob.glob("Tests/*.sh")
|
||||
if os.path.basename(p) not in ("run.sh", "makedisks.sh"))
|
||||
words = {"three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9}
|
||||
said = re.search(r"([A-Za-z]+) more scripts run alongside it", rootReadme)
|
||||
if not said:
|
||||
problems.append("the README no longer says how many scripts run alongside run.sh")
|
||||
elif words.get(said.group(1).lower()) != len(scripts):
|
||||
problems.append("the README says %s scripts run alongside run.sh, and there are %d: %s"
|
||||
% (said.group(1), len(scripts), ", ".join(scripts)))
|
||||
for name in scripts:
|
||||
if ("`Tests/%s`" % name) not in rootReadme:
|
||||
problems.append("Tests/%s runs in the suite and the README does not say what it is"
|
||||
" for" % name)
|
||||
|
||||
# ---- And the tool count is the number of things the makefile builds ----
|
||||
makefile = open("makefile").read()
|
||||
built = re.search(r"^all:(.*)$", makefile, re.M)
|
||||
if not built:
|
||||
problems.append("the makefile no longer has an all target this can count")
|
||||
else:
|
||||
tools = len(built.group(1).split())
|
||||
for said in re.findall(r"(?:build|rebuild) (?:all )?(?:the )?([a-z]+) tools", rootReadme):
|
||||
if words.get(said) != tools:
|
||||
problems.append("the README says the %s tools and the makefile builds %d"
|
||||
% (said, tools))
|
||||
|
||||
# ---- The sizes the CosmOS README quotes for its own programs ----
|
||||
#
|
||||
# It said Files was 645 bytes in two places and Edit 1,983. They were 665 and 1,996: the
|
||||
# programs kept being improved and the sentences about how small they are did not move.
|
||||
# These are the most quotable numbers in the document and the least likely to be rechecked
|
||||
# by hand, so they are measured.
|
||||
#
|
||||
# A claim is "<something naming an app> ... in N bytes" on one line. Only lines that name
|
||||
# an app are looked at, so ordinary prose about bytes is left alone.
|
||||
apps = {}
|
||||
for source in glob.glob("Programs/CosmOS/Apps/*.asm"):
|
||||
apps[os.path.basename(source)[:-4]] = source
|
||||
|
||||
for line in readme.split("\n"):
|
||||
said = re.search(r"\b(?:in|to) ([\d,]+) bytes", line)
|
||||
if not said:
|
||||
continue
|
||||
named = [name for name in apps
|
||||
if re.search(r"(?:`|\| )%s(?:\.asm)?\b" % re.escape(name), line)]
|
||||
if len(named) != 1:
|
||||
continue
|
||||
built = subprocess.run(["./Assembler", "-I", "Programs/CosmOS/Source",
|
||||
apps[named[0]], "-o", os.devnull],
|
||||
capture_output=True, text=True)
|
||||
real = re.search(r"Total size: (\d+) bytes", built.stdout)
|
||||
if not real:
|
||||
problems.append("could not measure %s, which the CosmOS README quotes a size for"
|
||||
% named[0])
|
||||
elif int(said.group(1).replace(",", "")) != int(real.group(1)):
|
||||
problems.append("the CosmOS README says %s is %s bytes and it is %s"
|
||||
% (named[0], said.group(1), real.group(1)))
|
||||
|
||||
if problems:
|
||||
print("The manuals and the code disagree:")
|
||||
for p in problems:
|
||||
|
||||
Reference in New Issue
Block a user