AnachronautandClaude Opus 5
2e9cb9af7e
A comment used to walk through tokGet and srcNext and be classified,
character by character, on the way to being discarded. Comments are most
of what this assembler reads: 87 per cent of what Say.asm pulls in once
services.asm is counted, 59 per cent of CosmOS. The house style here is
dense commentary, so the assembler is penalised more than most by its own
sources.
The saving is not the classifying, it is the BOOKKEEPING. srcNext loads
and stores the walking pointer through memory for every character and
asks numCompare whether the buffer is used up. srcSkipComment keeps the
pointer in a data pointer for a whole run and the newline in B, so a
comment byte costs a load, a compare and two steps. A run is capped at
255 so one byte can count it, which is the only reason it loops.
Measured, each version with its own rebuilt images:
without with
hello 791,957 586,184
Say 9,924,401 4,506,702 2.20x
Files 13,197,710 7,188,351 1.84x
Keys 23,091,447 15,069,880 1.53x
cosmos 886,498,996 789,982,899 1.12x
Which tracks the comment ratios: Say gains most and cosmos least, in
proportion to how much of each is prose.
---- And two mistakes worth keeping ----
The scratch went among the READER'S STATE, which is a block copied whole
by a count written down somewhere else - so every saved file lost the
last four bytes of itself and an include came back with its pointer
wrong. The comment above that block says not to do this, in capitals.
That is twice this week: scriptCopyState had the same shape this morning.
And a file that ends inside a comment has to put back the file that
included it, exactly as srcAtEnd does for a character. NOTHING IN THIS
REPOSITORY ENDS THAT WAY - every source here ends on a line of code with
a newline after it - so break.sh could not catch an error in that path
because nothing reached it. tail.asm is generated with no newline on its
last line for that reason, and usestail.asm names a label after the
include, which is what goes missing when the include never returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
222 lines
10 KiB
Bash
Executable File
222 lines
10 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Checks the assembler that runs on SplitBit against the one that runs on the host.
|
|
#
|
|
# THE ONLY HONEST TEST OF AN ASSEMBLER IS THE BYTES IT PRODUCES. "It ran" and "the sizes
|
|
# look right" both pass for a binary with a label one byte out, which is a program that
|
|
# jumps into the middle of an instruction. So this assembles the same source both ways and
|
|
# compares the two files byte for byte, the same discipline SplitDisk and sbfs.asm work
|
|
# under: two implementations of one written specification, each one checking the other.
|
|
#
|
|
# Then it runs what the machine built, because a file that matches and does not work would
|
|
# mean both assemblers were wrong together.
|
|
#
|
|
# It checks a boot image and four loadable programs. The loadable ones are the harder case:
|
|
# they include another file, they are based somewhere other than zero, and every service
|
|
# they call is a name declared in that included file. The last of them also brings a vector,
|
|
# which is the only thing here that asks anything of its loader.
|
|
#
|
|
# Written by Anachronaut
|
|
|
|
set -u
|
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
WORK="$ROOT/Tests/build/native"
|
|
ASM="$ROOT/Assembler"
|
|
TOOL="$ROOT/SplitDisk"
|
|
EMU="$ROOT/SplitBit"
|
|
|
|
PASS=0
|
|
FAIL=0
|
|
FAILED_NAMES=()
|
|
|
|
check() {
|
|
local name="$1"
|
|
shift
|
|
if "$@"; then
|
|
printf ' [ok ] %-22s %s\n' "$name" "${REPORT:-}"
|
|
PASS=$((PASS + 1))
|
|
else
|
|
printf ' [FAIL] %-22s %s\n' "$name" "${REPORT:-}"
|
|
FAIL=$((FAIL + 1))
|
|
FAILED_NAMES+=("$name")
|
|
fi
|
|
REPORT=""
|
|
}
|
|
|
|
for tool in "$ASM" "$TOOL" "$EMU"; do
|
|
[ -x "$tool" ] || { echo "$(basename "$tool") is not built."; exit 1; }
|
|
done
|
|
|
|
rm -rf "$WORK"
|
|
mkdir -p "$WORK"
|
|
|
|
# The system it runs under, and the assembler itself, both built by the host assembler.
|
|
# Bootstrapping has to start somewhere, and this is the last place it does.
|
|
"$ASM" -I "$ROOT/Programs/Libraries" -I "$ROOT/Programs/CosmOS/Source" \
|
|
-o "$WORK/cosmos.bin" "$ROOT/Programs/CosmOS/Source/cosmos.asm" >/dev/null || exit 1
|
|
"$ASM" -I "$ROOT/Programs/CosmOS/Source" -I "$ROOT/Programs/CosmOS/Assembler" \
|
|
-o "$WORK/Asm.sbx" "$ROOT/Programs/CosmOS/Assembler/Asm.asm" >/dev/null || exit 1
|
|
|
|
"$TOOL" format "$WORK/native.img" 2048 4 >/dev/null
|
|
"$TOOL" put "$WORK/native.img" "$ROOT/Programs/Examples/hello.asm" hello.asm >/dev/null
|
|
"$TOOL" put "$WORK/native.img" "$WORK/Asm.sbx" Asm.sbx >/dev/null
|
|
|
|
# The applications, and the file of service names they all include. These are the reason
|
|
# the second milestone exists: an assembler that cannot follow an #Include cannot build
|
|
# anything that asks the system for anything.
|
|
#
|
|
# Keys is the last of them and the hardest: it brings a vector of its own, so it needs a
|
|
# Vector Segment in the output and the version two header that says so. Nothing else here
|
|
# asks anything of its loader beyond code and data.
|
|
APPS="Say greet Files Keys"
|
|
"$TOOL" put "$WORK/native.img" "$ROOT/Programs/CosmOS/Source/services.asm" services.asm >/dev/null
|
|
"$TOOL" put "$WORK/native.img" "$ROOT/Programs/CosmOS/Source/console.asm" console.asm >/dev/null
|
|
for app in $APPS; do
|
|
[ -f "$ROOT/Programs/CosmOS/Apps/$app.asm" ] || continue
|
|
"$TOOL" put "$WORK/native.img" "$ROOT/Programs/CosmOS/Apps/$app.asm" "$app.asm" >/dev/null
|
|
done
|
|
|
|
# ---- A file that ends in the middle of a comment ----
|
|
#
|
|
# Generated rather than committed, because what makes it work is the ABSENCE of a newline on
|
|
# the last line and no editor can be trusted to leave that alone.
|
|
#
|
|
# An included file running out inside a comment is the case the reader has to pop the include
|
|
# stack for, and it is the one shape no source in this repository happens to have: every file
|
|
# here ends on a line of code with a newline after it. Nothing exercised that path, so an
|
|
# error in it could not be caught - and comment skipping is exactly where such an error would
|
|
# live, since a comment is the only thing that reads past the end of a line without looking.
|
|
#
|
|
# usestail.asm names a label AFTER the include, which is what goes missing if the file that
|
|
# did the including is not put back: the include never returns, the rest of the program is
|
|
# never read, and the label is never defined.
|
|
printf '; nothing but a comment, and no newline after it' > "$WORK/tail.asm"
|
|
printf '#Include tail.asm\n#Program\n #Base 0x5000\nstart:\n SETD.0 Later\n HALT\n#Data\n #Base 0x3000\nLater:\n 0x00\n' > "$WORK/usestail.asm"
|
|
"$TOOL" put "$WORK/native.img" "$WORK/tail.asm" tail.asm >/dev/null
|
|
"$TOOL" put "$WORK/native.img" "$WORK/usestail.asm" usestail.asm >/dev/null
|
|
|
|
{
|
|
echo "load Asm.sbx"
|
|
echo "run hello.asm"
|
|
echo "run usestail.asm"
|
|
for app in $APPS; do
|
|
[ -f "$ROOT/Programs/CosmOS/Apps/$app.asm" ] || continue
|
|
echo "run $app.asm"
|
|
done
|
|
echo "exit"
|
|
} | "$EMU" --fast --cycles 600000000 -D "$WORK/native.img" "$WORK/cosmos.bin" \
|
|
> "$WORK/session.txt" 2>&1
|
|
|
|
# ---- It got as far as writing something ----
|
|
check "it wrote a file" grep -q "wrote hello.bin" "$WORK/session.txt"
|
|
|
|
# ---- And a file that ends inside a comment did not swallow the one that included it ----
|
|
#
|
|
# The label is defined after the #Include, so it exists only if the reader came back.
|
|
check "an include that ends in a comment comes back" \
|
|
grep -q "wrote usestail.sbx" "$WORK/session.txt"
|
|
|
|
# ---- And the file is the one the host assembler makes ----
|
|
"$ASM" -o "$WORK/reference.bin" "$ROOT/Programs/Examples/hello.asm" >/dev/null
|
|
"$TOOL" get "$WORK/native.img" hello.bin "$WORK/native.bin" >/dev/null 2>&1
|
|
if [ -f "$WORK/native.bin" ]; then
|
|
REPORT="$(wc -c < "$WORK/native.bin" | tr -d ' ') bytes"
|
|
fi
|
|
check "byte for byte" cmp -s "$WORK/native.bin" "$WORK/reference.bin"
|
|
|
|
# ---- And what it built actually runs ----
|
|
"$EMU" --fast --cycles 100000 "$WORK/native.bin" > "$WORK/ran.txt" 2>&1
|
|
check "and it runs" grep -q "^Hello, World!$" "$WORK/ran.txt"
|
|
|
|
# ---- The report it printed says what it did ----
|
|
REPORT="$(grep -o 'program [0-9]*, data [0-9]*, labels [0-9]*' "$WORK/session.txt" | head -1 || true)"
|
|
check "it counted right" grep -q "program 17, data 14, labels 2" "$WORK/session.txt"
|
|
|
|
# ---- And the applications, which need an include, a base and the service names ----
|
|
#
|
|
# A loadable program is the harder case and the one that matters: #Include splices another
|
|
# file in, #Base moves every label to where the program will really live, and every SWI in
|
|
# them names a vector declared in a file the source never mentions by number.
|
|
for app in $APPS; do
|
|
"$ASM" -I "$ROOT/Programs/Libraries" -I "$ROOT/Programs/CosmOS/Source" \
|
|
-o "$WORK/ref-$app.sbx" "$ROOT/Programs/CosmOS/Apps/$app.asm" >/dev/null
|
|
rm -f "$WORK/got-$app.sbx"
|
|
"$TOOL" get "$WORK/native.img" "$app.sbx" "$WORK/got-$app.sbx" >/dev/null 2>&1
|
|
if [ -f "$WORK/got-$app.sbx" ]; then
|
|
REPORT="$(wc -c < "$WORK/got-$app.sbx" | tr -d ' ') bytes"
|
|
fi
|
|
check "$app byte for byte" cmp -s "$WORK/got-$app.sbx" "$WORK/ref-$app.sbx"
|
|
done
|
|
|
|
# ---- Self hosting ----
|
|
#
|
|
# The two things that make this more than a demonstration: the machine assembling the
|
|
# operating system it is running on, and the assembler assembling itself. Both have to come
|
|
# out byte for byte identical to what the host tool builds from the same source.
|
|
#
|
|
# THE FIXED POINT IS THE THIRD CHECK AND THE ONE THAT MATTERS MOST. A binary that matches
|
|
# could still have been built by an assembler that is wrong in some way the source happens
|
|
# not to exercise; a binary that BUILDS ITSELF AGAIN and comes out the same has been through
|
|
# its own machinery. The host is no longer necessary after this point - it is a convenience.
|
|
|
|
# ---- Read out of cosmos.asm rather than listed here ----
|
|
#
|
|
# Everything the system includes has to be on the disk it is going to be assembled on. This
|
|
# was a hand written list and went stale the moment script.asm was added: the self build
|
|
# stopped, saying it could not find a file nobody had thought to put there. That is the
|
|
# fourth time a list beside a thing has drifted from the thing, so it is now the thing that
|
|
# is asked.
|
|
SOURCES="cosmos $(sed -n 's/^#Include \(.*\)\.asm$/\1/p' "$ROOT/Programs/CosmOS/Source/cosmos.asm" | tr '\n' ' ')"
|
|
PARTS="Asm numbers source token classify labels vectors table"
|
|
|
|
"$TOOL" format "$WORK/self.img" 4096 8 >/dev/null
|
|
for f in $SOURCES; do
|
|
"$TOOL" put "$WORK/self.img" "$ROOT/Programs/CosmOS/Source/$f.asm" "$f.asm" >/dev/null
|
|
done
|
|
for f in $PARTS; do
|
|
"$TOOL" put "$WORK/self.img" "$ROOT/Programs/CosmOS/Assembler/$f.asm" "$f.asm" >/dev/null
|
|
done
|
|
"$TOOL" put "$WORK/self.img" "$WORK/Asm.sbx" Asm.sbx >/dev/null
|
|
|
|
printf 'load Asm.sbx\nrun cosmos.asm\nrun Asm.asm\nexit\n' \
|
|
| "$EMU" --fast --cycles 4000000000 -D "$WORK/self.img" "$WORK/cosmos.bin" \
|
|
> "$WORK/self.txt" 2>&1
|
|
|
|
rm -f "$WORK/built-cosmos.bin" "$WORK/built-Asm.sbx"
|
|
"$TOOL" get "$WORK/self.img" cosmos.bin "$WORK/built-cosmos.bin" >/dev/null 2>&1
|
|
"$TOOL" get "$WORK/self.img" Asm.sbx "$WORK/built-Asm.sbx" >/dev/null 2>&1
|
|
|
|
REPORT="$(grep -o 'program 7036, data 2448, labels [0-9]*' "$WORK/self.txt" || true)"
|
|
check "CosmOS builds CosmOS" cmp -s "$WORK/built-cosmos.bin" "$WORK/cosmos.bin"
|
|
REPORT="$(grep -o 'program 7533, data [0-9]*, labels [0-9]*' "$WORK/self.txt" || true)"
|
|
check "and builds itself" cmp -s "$WORK/built-Asm.sbx" "$WORK/Asm.sbx"
|
|
|
|
# ---- And the second generation is the first ----
|
|
#
|
|
# Boot the CosmOS that CosmOS built, and have that assemble CosmOS again.
|
|
if [ -s "$WORK/built-cosmos.bin" ]; then
|
|
"$TOOL" delete "$WORK/self.img" cosmos.bin >/dev/null 2>&1
|
|
printf 'load Asm.sbx\nrun cosmos.asm\nexit\n' \
|
|
| "$EMU" --fast --cycles 4000000000 -D "$WORK/self.img" "$WORK/built-cosmos.bin" \
|
|
> "$WORK/second.txt" 2>&1
|
|
rm -f "$WORK/second-cosmos.bin"
|
|
"$TOOL" get "$WORK/self.img" cosmos.bin "$WORK/second-cosmos.bin" >/dev/null 2>&1
|
|
fi
|
|
REPORT="a fixed point"
|
|
check "second generation" cmp -s "$WORK/second-cosmos.bin" "$WORK/built-cosmos.bin"
|
|
|
|
echo
|
|
if [ "$FAIL" -eq 0 ]; then
|
|
echo "All $PASS native assembler checks passed."
|
|
else
|
|
echo "$PASS passed, $FAIL failed: ${FAILED_NAMES[*]}"
|
|
echo
|
|
echo "The session was:"
|
|
sed 's/^/ /' "$WORK/session.txt"
|
|
if [ -f "$WORK/self.txt" ]; then
|
|
echo
|
|
echo "And the self hosting run:"
|
|
sed 's/^/ /' "$WORK/self.txt"
|
|
fi
|
|
exit 1
|
|
fi
|