M4: SplitBit assembles SplitBit, and then assembles itself

> load Asm.sbx
    > run cosmos.asm
    wrote cosmos.bin: program 7036, data 2448, labels 475
    > run Asm.asm
    wrote Asm.sbx: program 7533, data 4099, labels 555

Both byte for byte identical to what the host assembler builds from the
same source. The machine now builds the operating system it is running on,
and builds the thing that built it.

THE CHECK THAT MATTERS MOST IS THE THIRD ONE. A binary that matches could
still have come from an assembler wrong in some way this particular source
happens not to exercise. So Tests/native.sh boots the CosmOS that CosmOS
built and has THAT assemble CosmOS again - and the second generation is
identical to the first, down to the cycle count. It is a fixed point: the
machinery has been through itself. After this the host is a convenience
rather than a necessity.

WHAT STOOD IN THE WAY was not the assembler. It loaded, faulted at 7,780
cycles, and the fault was in CosmOS: a loaded program is staged at 0x8000
before being blitted into place, so the whole FILE has to fit in the 32,768
bytes above it. The assembler's file was 33,983, and 22K of that was
zeroed scratch buffers - because #Reserve emits what it reserves.

None of that is initialised data. It is scratch, wanted only while the
assembler runs, and while it runs everything above its own data is free.
So the buffers are a MAP now rather than declarations - Assembler/scratch.asm
writes down six addresses and the file carries none of it. 33,983 bytes
became 11,648, and the assembler could load itself.

The map has a file of its own because the reader and the label table both
need addresses out of it while neither includes the other.

The sizes are cut to the largest thing it is asked to build, and that turns
out not to be the operating system: the assembler is 555 labels and 11,648
bytes of output against CosmOS's 475 and 9,564. The hardest thing this
assembles is itself.

Also: sizing it for CosmOS meant raising the label table, and raising the
label table is what pushed the file over the staging limit. The two facts
only met because the first one was tried.

Speed, measured rather than guessed: CosmOS takes 80,168,646 cycles, which
is eighty seconds of emulated time and under a second under --fast. Most of
it is a straight walk of 475 label names, several thousand times. Sorting
or bucketing that is easy and was deliberately not written before there was
something to measure.

make run-cosmos now puts every source file on the disk, so the whole thing
can be done rather than read about.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
Anachronaut
2026-08-21 12:03:31 -04:00
co-authored by Claude Opus 5
parent 20989c3439
commit fb335681d2
12 changed files with 180 additions and 55 deletions
+18 -17
View File
@@ -1358,9 +1358,9 @@ layOutImage:
CALL checkImageRoom
BNQ layOutNo
SETD.0 Image
SETD.1 ImgWalk
STD.0.1
SETD.0 ImgWalk
SETD.2 ScratchImage
CALL numSet
SETD.0 MagicSPBT
INIA 0d4
CALL putBytes
@@ -1412,9 +1412,9 @@ layOutLoadable:
CALL checkImageRoom
BNQ layOutNo
SETD.0 Image
SETD.1 ImgWalk
STD.0.1
SETD.0 ImgWalk
SETD.2 ScratchImage
CALL numSet
SETD.0 MagicSBEX
INIA 0d4
CALL putBytes
@@ -1674,9 +1674,9 @@ writeVectorsHeader:
SETD.0 Loadable
LDA.0
BRA writeVectorsNone
SETD.0 Image
SETD.1 ImgWalk
STD.0.1
SETD.0 ImgWalk
SETD.2 ScratchImage
CALL numSet
SETD.0 ImgWalk
INIA 0d4
CALL numAddByte
@@ -1693,9 +1693,9 @@ writeVectorsNone:
SETD.0 Loadable
LDA.0
BRA writeVectorsOut
SETD.0 Image
SETD.1 ImgWalk
STD.0.1
SETD.0 ImgWalk
SETD.2 ScratchImage
CALL numSet
SETD.0 ImgWalk
INIA 0d8
CALL numAddByte
@@ -1773,7 +1773,8 @@ vectorNotEntry:
writeImage:
SETD.0 OutName
SETD.1 Image
SETD.1 ScratchImage
LDD.1.1
SETD.2 ImgTotal
LDA.2
INCD.2
@@ -2046,9 +2047,10 @@ DropWalk:
0x00 0x00
; How big a binary this can build. Everything the assembler makes has to fit here at once,
; because a file is written in one call and there is nowhere to put half of one.
; because a file is written in one call and there is nowhere to put half of one. CosmOS
; itself comes to 9,564 bytes.
ImgRoom:
0x10 0x00
0x34 0x00
MagicSPBT:
"SPBT"
@@ -2169,9 +2171,8 @@ LabelsEnd:
"
"
Image:
#Reserve 0d4096
#Include scratch.asm
#Include numbers.asm
#Include source.asm
#Include token.asm
+14 -15
View File
@@ -25,12 +25,12 @@ labReset:
CALL numZero
SETD.0 LabUsed
CALL numZero
SETD.0 LabArena
SETD.1 LabNext
STD.0.1
SETD.0 LabIndex
SETD.1 LabBase
STD.0.1
SETD.0 LabNext
SETD.2 ScratchLabArena
CALL numSet
SETD.0 LabBase
SETD.2 ScratchLabIndex
CALL numSet
RET
; Adds the name at DP0, meaning the address in A and B. Q is zero if it went in.
@@ -294,13 +294,16 @@ LabLenWalk:
LabEnd:
0x00 0x00
; How many labels there may be, and how many bytes of name between them. Sized for a
; single program rather than for CosmOS: raising either is changing a number here, and
; running into one says so rather than writing past the end of the table.
; How many labels there may be, and how many bytes of name between them.
;
; SIZED FOR THE ASSEMBLER ITSELF, which turns out to be the largest thing it is asked to
; build: 555 labels and about 6,800 bytes of name, against CosmOS's 475 and 5,881. Running into
; either limit says so rather than writing past the end of the table. The buffers live
; above the program rather than inside it - see the scratch map in Asm.asm.
LabLimit:
0x01 0x00
0x03 0x00
LabRoom:
0x08 0x00
0x20 0x00
LabNamed:
": "
@@ -316,7 +319,3 @@ LabFull:
LabNoRoom:
"no room left for label names"
LabIndex:
#Reserve 0d1024
LabArena:
#Reserve 0d2048
+1
View File
@@ -80,5 +80,6 @@ NoFileText:
; The libraries go last, after both segments have been based. An included file that carries
; code brings its own #Program and #Data lines with it, and a #Base has to come before
; anything is in the segment it bases - so the bases are set here and the code arrives after.
#Include scratch.asm
#Include numbers.asm
#Include source.asm
+44
View File
@@ -0,0 +1,44 @@
; Where the assembler's big buffers live.
;
; A map rather than a set of declarations, and it has a file of its own because the reader
; and the label table both need addresses out of it while neither includes the other.
;
#Data
; NOT #Reserve, AND THAT IS THE WHOLE POINT. Reserved space in a segment is written into
; the file as zeroes and copied at load, so 22K of scratch made a 34K file - and a loaded
; program is staged at 0x8000 before being put in place, which leaves exactly 32,768 bytes
; for the whole of it. The assembler could not load itself.
;
; None of this is initialised data. It is scratch, wanted only while the assembler is
; running, and while it is running everything above its own data is free: the system keeps
; below 0x1000, the staging area is only in use during a load, and the Stack comes down
; from the top. So the addresses are written down here and the file carries none of it.
;
; 0x8000 3072 the label index, 768 entries of four
; 0x8C00 8192 the label names, packed end to end
; 0xAC00 13312 the binary being built
; 0xE000 1792 the vector names, 64 entries of twenty eight
; 0xE700 1758 the reader's stack, six levels of 293
; 0xEE00 368 which files have been included, sixteen names of 23
;
; That ends at 0xEF70, with the Stack coming down from 0xFFFF above it - about four
; kilobytes, against the tens of bytes of CALL frames this ever nests.
;
; THE TWO THINGS THAT DECIDE THESE SIZES are the largest program it will be asked to build
; and the largest one it will be asked to read. CosmOS is 475 labels and 9,564 bytes of
; output; the assembler itself is 555 labels, about 6,800 bytes of name and 11,648 of output. The
; second is bigger than the first, which is worth knowing: the hardest thing this assembles
; is not the operating system, it is itself.
ScratchLabIndex:
0x80 0x00
ScratchLabArena:
0x8C 0x00
ScratchImage:
0xAC 0x00
ScratchVecNames:
0xE0 0x00
ScratchSrcStack:
0xE7 0x00
ScratchIncNames:
0xEE 0x00
+6 -12
View File
@@ -278,9 +278,9 @@ srcStepBackAt:
; Where the slot for the current depth is, into SrcSlot.
srcSlot:
SETD.0 SrcStack
SETD.1 SrcSlot
STD.0.1 ; WHERE the stack is, not what is in it.
SETD.0 SrcSlot
SETD.2 ScratchSrcStack
CALL numSet ; WHERE the stack is, not what is in it.
SETD.0 SrcSlotLeft
SETD.2 SrcDepth
LDA.2
@@ -388,9 +388,9 @@ srcRemember:
; Where name number IncLeft sits, into IncSlot. Fixed fields of 23 bytes: there are few of
; these and they are short, so an arena would cost more code than it saved.
srcSeenSlot:
SETD.0 IncNames
SETD.1 IncSlot
STD.0.1
SETD.0 IncSlot
SETD.2 ScratchIncNames
CALL numSet
SETD.0 IncSlotLeft
SETD.2 IncLeft
LDA.2
@@ -549,10 +549,4 @@ TooDeepText:
"included files are nested deeper than this assembler will follow
"
; Six levels of nesting, at 293 bytes each. CosmOS itself nests three deep.
SrcStack:
#Reserve 0d1758
; Sixteen names of 23 bytes, which is more separate files than anything here includes.
IncNames:
#Reserve 0d368
+1
View File
@@ -169,6 +169,7 @@ NoFileText:
"no such file
"
#Include scratch.asm
#Include numbers.asm
#Include source.asm
#Include token.asm
+3 -5
View File
@@ -225,9 +225,9 @@ vecWriteHandler:
; Where entry number VecWhich sits, into VecSlot. Twenty eight bytes an entry.
vecSlotAt:
SETD.0 VecNames
SETD.1 VecSlot
STD.0.1
SETD.0 VecSlot
SETD.2 ScratchVecNames
CALL numSet
SETD.0 VecSlotLeft
SETD.2 VecWhich
CALL numSet
@@ -293,5 +293,3 @@ VecFull:
VecName:
#Reserve 0d23
VecNames:
#Reserve 0d1792
+13 -2
View File
@@ -86,10 +86,11 @@ cosmos: $(COSMOS) $(APPS) $(NATIVE_ASM)
# also whatever used to be.
$(COSMOS_DISK): $(APPS) $(NATIVE_ASM) testPrograms/stringKeyword.asm \
CosmOS/Apps/hello.asm CosmOS/Apps/Say.asm CosmOS/Apps/Keys.asm \
CosmOS/Source/services.asm CosmOS/Source/console.asm
CosmOS/Source/services.asm CosmOS/Source/console.asm \
$(wildcard CosmOS/Source/*.asm) $(wildcard CosmOS/Assembler/*.asm)
@mkdir -p $(@D)
rm -f $@
$(DISKTOOL) format $@ 2048 4
$(DISKTOOL) format $@ 4096 8
@for app in $(APPS); do $(DISKTOOL) put $@ $$app; done
$(DISKTOOL) put $@ $(NATIVE_ASM)
@# SOURCE goes on as well, because an assembler with nothing to assemble is a
@@ -112,6 +113,16 @@ $(COSMOS_DISK): $(APPS) $(NATIVE_ASM) testPrograms/stringKeyword.asm \
$(DISKTOOL) put $@ CosmOS/Source/services.asm
$(DISKTOOL) put $@ CosmOS/Source/console.asm
$(DISKTOOL) put $@ testPrograms/stringKeyword.asm strings.asm
@# And the whole of CosmOS, and the whole of the assembler, so that the machine can
@# build the system it is running on and then build the thing that built it:
@#
@# > load Asm.sbx
@# > run cosmos.asm
@# > run Asm.asm
@#
@# Both come out byte for byte what the host tool makes from the same source.
@for f in CosmOS/Source/*.asm CosmOS/Assembler/*.asm; do \
$(DISKTOOL) put $@ $$f >/dev/null; done
# The system as well as the disk. Building only the image leaves whatever cosmos.bin was
# there before, or none at all, and then the disk is booted with a system that does not
+1 -1
View File
@@ -16,7 +16,7 @@ SplitBit is a custom 8 bit system designed for hobbyist projects and experimenta
- Loadable Programs: A program that was not booted from carries a header saying where it belongs, and Programs/loader.asm reads one off a disk, puts it there, and runs it.
- An Operating System: CosmOS boots the machine, mounts a disk, lists what is on it, loads a program and runs it, and takes the machine back when it finishes. It comes with a library of programs to run, including a game and a line editor that writes files a person typed.
- System Services: A loaded program reaches the console and the disk through numbered software interrupts rather than carrying a copy of the code that drives them. The numbers are written down in one file that both sides include, so neither ever types one. It took the editor from 4941 bytes to 1983 without changing a line of what it does.
- A Native Assembler: SplitBit assembles SplitBit. Programs/CosmOS/Assembler/ is an assembler written in SplitBit assembly that runs under CosmOS, reads source off a SplitBit disk, and writes a binary back to it with no host involved. It builds boot images and loadable applications, following every directive the language has - including #Vectors, so a program that brings its own interrupt handlers can be assembled on the machine, loaded by it, and have its vectors installed and taken back again. Its output has to be byte for byte identical to what the C assembler produces from the same source, which is what Tests/native.sh checks.
- A Native Assembler: SplitBit assembles SplitBit. Programs/CosmOS/Assembler/ is an assembler written in SplitBit assembly that runs under CosmOS, reads source off a SplitBit disk, and writes a binary back to it with no host involved. It builds boot images and loadable applications, following every directive the language has. IT ASSEMBLES COSMOS, AND IT ASSEMBLES ITSELF, both byte for byte identical to what the C assembler produces from the same source. Tests/native.sh then boots the CosmOS that CosmOS built and has that one assemble CosmOS again, so the machinery has been through itself: after that the host is a convenience rather than a necessity.
- Streaming Reads: A file bigger than the machine's memory is read a block at a time, through services that keep nothing open between calls. CosmOS's own source is 104K against 64K of Data Memory, so this is what a self-hosted assembler will stand on.
- Storage: A block device with 256 byte blocks and 16 megabytes of them, backed by an image file on the host. It knows blocks and not files, because a filesystem is meant to be software SplitBit runs.
- Memory Controller: Reads and writes Program Memory, moves blocks between memory banks, reaches memory that devices bring with them, and guards a range against being written by accident. It is how a SplitBit machine loads a program.
+23 -3
View File
@@ -537,8 +537,28 @@ A `#Vectors` line that names a **handler** says this program implements that vec
**A declaration and an implementation are the same entry.** `services.asm` says a service is called `osPrintString` and has number 16; `cosmos.asm` says `osPrintString` is handled by `handlePrintString`. Both sides include the first file, so the name is met twice and the second time fills in the handler. That is what lets one shared file serve both a program that calls a service and the system that implements it — and it is why the first pass declares and the second implements, a handler being an address and no address being known until every label has been placed.
### Self Hosting:
It assembles CosmOS, and it assembles itself.
```
> load Asm.sbx
> run cosmos.asm
wrote cosmos.bin: program 7036, data 2448, labels 475
> run Asm.asm
wrote Asm.sbx: program 7533, data 4099, labels 555
```
Both come out **byte for byte identical** to what the host assembler builds from the same source. `make run-cosmos` puts every source file on the disk, so this can be done rather than read about.
**The check that matters most is the third one.** A binary that matches could still have been built by an assembler wrong in some way this particular source happens not to exercise. So `Tests/native.sh` boots the CosmOS that CosmOS built and has *that* assemble CosmOS again — and the second generation is identical to the first, down to the cycle count. It is a fixed point, which means the machinery has been through itself.
After that the host is a convenience rather than a necessity.
CosmOS takes about 80 million cycles, which is eighty seconds of emulated time and under a second under `--fast`. Most of that is the label table: a straight walk of 475 names, several thousand times. Sorting it or bucketing it on the first character are both easy, and neither was worth writing before there was something to measure.
**The hardest thing it assembles is not the operating system, it is itself** — 555 labels and 6,770 bytes of name against CosmOS's 475 and 5,881, and a larger output. That is what the buffer sizes are cut to.
### What It Does Not Do Yet:
Nothing in the language. Every directive this manual describes is understood, and `Tests/native.sh` checks a boot image and four loadable programs — the last of them bringing a vector — against the host assembler byte for byte on every run.
What is left is scale rather than grammar: whether it can assemble CosmOS itself, which is 104,142 bytes of source across several files and 453 labels.
Nothing in the language. Every directive this manual describes is understood, and every one of them is checked against the host assembler byte for byte on every test run.
+1
View File
@@ -345,6 +345,7 @@ app-Stream | CosmOS/Apps/Stream.asm | assemble | -
asm-Asm | CosmOS/Assembler/Asm.asm | assemble | - | -
asm-readTest | CosmOS/Assembler/readTest.asm | assemble | - | -
asm-tokenTest | CosmOS/Assembler/tokenTest.asm | assemble | - | -
asm-scratch | CosmOS/Assembler/scratch.asm | assemble | - | -
# ---- Programs driven by console input ----
inputTest | inputTest.asm | run | inputTest.in | -
+55
View File
@@ -117,6 +117,56 @@ for app in $APPS; do
check "$app byte for byte" cmp -s "$WORK/got-$app.sbx" "$WORK/ref-$app.sbx"
done
# ---- Self hosting ----
#
# The two things that make this more than a demonstration: the machine assembling the
# operating system it is running on, and the assembler assembling itself. Both have to come
# out byte for byte identical to what the host tool builds from the same source.
#
# THE FIXED POINT IS THE THIRD CHECK AND THE ONE THAT MATTERS MOST. A binary that matches
# could still have been built by an assembler that is wrong in some way the source happens
# not to exercise; a binary that BUILDS ITSELF AGAIN and comes out the same has been through
# its own machinery. The host is no longer necessary after this point - it is a convenience.
SOURCES="cosmos console sbfs services text"
PARTS="Asm numbers source token classify labels vectors table"
"$TOOL" format "$WORK/self.img" 4096 8 >/dev/null
for f in $SOURCES; do
"$TOOL" put "$WORK/self.img" "$ROOT/Programs/CosmOS/Source/$f.asm" "$f.asm" >/dev/null
done
for f in $PARTS; do
"$TOOL" put "$WORK/self.img" "$ROOT/Programs/CosmOS/Assembler/$f.asm" "$f.asm" >/dev/null
done
"$TOOL" put "$WORK/self.img" "$WORK/Asm.sbx" Asm.sbx >/dev/null
printf 'load Asm.sbx\nrun cosmos.asm\nrun Asm.asm\nexit\n' \
| "$EMU" --fast --cycles 4000000000 -D "$WORK/self.img" "$WORK/cosmos.bin" \
> "$WORK/self.txt" 2>&1
rm -f "$WORK/built-cosmos.bin" "$WORK/built-Asm.sbx"
"$TOOL" get "$WORK/self.img" cosmos.bin "$WORK/built-cosmos.bin" >/dev/null 2>&1
"$TOOL" get "$WORK/self.img" Asm.sbx "$WORK/built-Asm.sbx" >/dev/null 2>&1
REPORT="$(grep -o 'program 7036, data 2448, labels [0-9]*' "$WORK/self.txt" || true)"
check "CosmOS builds CosmOS" cmp -s "$WORK/built-cosmos.bin" "$WORK/cosmos.bin"
REPORT="$(grep -o 'program 7533, data [0-9]*, labels [0-9]*' "$WORK/self.txt" || true)"
check "and builds itself" cmp -s "$WORK/built-Asm.sbx" "$WORK/Asm.sbx"
# ---- And the second generation is the first ----
#
# Boot the CosmOS that CosmOS built, and have that assemble CosmOS again.
if [ -s "$WORK/built-cosmos.bin" ]; then
"$TOOL" delete "$WORK/self.img" cosmos.bin >/dev/null 2>&1
printf 'load Asm.sbx\nrun cosmos.asm\nexit\n' \
| "$EMU" --fast --cycles 4000000000 -D "$WORK/self.img" "$WORK/built-cosmos.bin" \
> "$WORK/second.txt" 2>&1
rm -f "$WORK/second-cosmos.bin"
"$TOOL" get "$WORK/self.img" cosmos.bin "$WORK/second-cosmos.bin" >/dev/null 2>&1
fi
REPORT="a fixed point"
check "second generation" cmp -s "$WORK/second-cosmos.bin" "$WORK/built-cosmos.bin"
echo
if [ "$FAIL" -eq 0 ]; then
echo "All $PASS native assembler checks passed."
@@ -125,5 +175,10 @@ else
echo
echo "The session was:"
sed 's/^/ /' "$WORK/session.txt"
if [ -f "$WORK/self.txt" ]; then
echo
echo "And the self hosting run:"
sed 's/^/ /' "$WORK/self.txt"
fi
exit 1
fi