Make the manuals plain ASCII, and check that they stay that way

"All files must be plain ASCII, the user's tooling doesn't support Unicode"
is a standing rule of this repository. Nothing enforced it, so it drifted:
39 em dashes and an ellipsis had collected in the two manuals, every one of
them typed by something that helpfully substituted a nicer character. The
spaced em dash becomes a spaced hyphen, which is what the source comments
and both READMEs use for the same job.

Tests/docs.sh now checks every tracked file and says which line and which
character. Verified that it bites.

THE CHECK READS git ls-files NUL SEPARATED, and that is the whole reason
this went unnoticed. I ran the obvious shell version of this audit two
commits ago - a loop over $(git ls-files) - and reported the repository
clean. It splits on whitespace, so it looked for a file called "SplitBit",
failed into /dev/null, and found nothing wrong with either manual because it
never opened them. Both have spaces in their names.

A check that cannot see the files with spaces in their names is worse than
no check at all, because it answers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
Anachronaut
2026-08-21 14:16:54 -04:00
co-authored by Claude Opus 5
parent b2945e41c4
commit 460a687939
3 changed files with 63 additions and 35 deletions
+29 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Checks the manuals against the code.
# Checks the manuals against the code, and the repository against its own rules.
#
# Documentation goes stale quietly. An instruction added without a table row, or a count
# in a heading that nobody updated, is wrong in a way nothing notices until somebody
@@ -28,6 +28,34 @@ am = read("SplitBit Assembler Manual.md")
asmc = read("Source/Assembler/assembly.c")
util = read("Source/Assembler/Assm-util.c")
# ---- Every tracked file is plain ASCII ----
#
# A standing rule of this repository, and nothing enforced it, so it drifted: 39 em dashes
# and an ellipsis had collected in the two manuals, all of them typed by something that
# helpfully substituted a nicer character.
#
# GIT LS-FILES IS READ NUL SEPARATED, and that is not fussiness. The obvious shell version
# of this check - looping over $(git ls-files) - splits on whitespace, so it looked for a
# file called "SplitBit" and reported the repository clean while both manuals had drifted.
# A check that cannot see the files with spaces in their names is worse than no check.
import subprocess
tracked = subprocess.run(["git", "ls-files", "-z"], capture_output=True).stdout
for name in tracked.split(b"\0"):
if not name:
continue
path = name.decode()
try:
text = open(path, encoding="utf-8").read()
except (UnicodeDecodeError, OSError):
continue
for number, line in enumerate(text.split("\n"), 1):
odd = sorted({c for c in line if ord(c) > 127})
if odd:
problems.append("%s line %d is not plain ASCII: %s"
% (path, number, ", ".join("%r (U+%04X)" % (c, ord(c)) for c in odd)))
break
# ---- Every instruction has a row, and every row is an instruction ----
#
# A mnemonic begins with a letter, which is what keeps the offset and size columns of the