Files
SplitBit-Emulator/Programs/CosmOS/Assembler/source.asm
T
AnachronautandClaude Opus 5 2e9cb9af7e Comments are thrown away by the reader, not read one byte at a time
A comment used to walk through tokGet and srcNext and be classified,
character by character, on the way to being discarded. Comments are most
of what this assembler reads: 87 per cent of what Say.asm pulls in once
services.asm is counted, 59 per cent of CosmOS. The house style here is
dense commentary, so the assembler is penalised more than most by its own
sources.

The saving is not the classifying, it is the BOOKKEEPING. srcNext loads
and stores the walking pointer through memory for every character and
asks numCompare whether the buffer is used up. srcSkipComment keeps the
pointer in a data pointer for a whole run and the newline in B, so a
comment byte costs a load, a compare and two steps. A run is capped at
255 so one byte can count it, which is the only reason it loops.

Measured, each version with its own rebuilt images:

              without        with
  hello       791,957     586,184
  Say       9,924,401   4,506,702   2.20x
  Files    13,197,710   7,188,351   1.84x
  Keys     23,091,447  15,069,880   1.53x
  cosmos  886,498,996 789,982,899   1.12x

Which tracks the comment ratios: Say gains most and cosmos least, in
proportion to how much of each is prose.

---- And two mistakes worth keeping ----

The scratch went among the READER'S STATE, which is a block copied whole
by a count written down somewhere else - so every saved file lost the
last four bytes of itself and an include came back with its pointer
wrong. The comment above that block says not to do this, in capitals.
That is twice this week: scriptCopyState had the same shape this morning.

And a file that ends inside a comment has to put back the file that
included it, exactly as srcAtEnd does for a character. NOTHING IN THIS
REPOSITORY ENDS THAT WAY - every source here ends on a line of code with
a newline after it - so break.sh could not catch an error in that path
because nothing reached it. tail.asm is generated with no newline on its
last line for that reason, and usestail.asm names a label after the
include, which is what goes missing when the include never returns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
2026-09-06 21:33:15 -04:00

785 lines
18 KiB
NASM

; The source reader: characters out of a file of any size.
;
; Everything else in the assembler sits on this, so it is the first thing built and the
; thing most worth getting right. It hands out one character at a time and keeps a line
; number, which is what lets an error say where it happened rather than only what it was.
;
; A FILE IS NEVER HELD WHOLE. It arrives a block at a time through osFileBlock, into one
; buffer of 256 bytes, and is fetched again when the buffer runs out. That is why the
; assembler can read a source file bigger than the memory it runs in - which cosmos.asm,
; at 56,047 bytes, already is.
;
; The file is read TWICE, once per pass, and srcRewind is how the second pass starts over.
; Nothing is kept between the passes but the label table.
;
; ---- A stack of readers ----
;
; #Include splices another file in where it stands, so the reader is a stack: srcInclude
; puts the current file's whole state aside, opens the new one, and the end of that file
; pops the old one back and carries on where it left off. Everything above works on "the
; current file" and does not know the stack is there.
;
; THE WHOLE STATE GOES ASIDE, buffer and all, 293 bytes of it. Keeping only the position
; and re-reading the block on the way back would be cheaper in memory and would cost a disk
; read per pop; at six levels of nesting the copy costs less than the arithmetic to avoid it.
; The buffer pointer survives the trip because it points into the buffer, which is always at
; the same address - the state is saved from and restored to the same variables.
;
; A FILE IS INCLUDED ONCE. Including it twice is not an error, it just does nothing, which
; is what lets two libraries depend on a third. The names are remembered for the length of
; one pass and forgotten between them, because the second pass has to walk exactly the same
; tree the first one did.
;
; Written by Anachronaut
#Program
; Opens the file DP0 names. Q is zero if it is there.
;
; The name is copied rather than pointed at, because the caller's copy is in the caller's
; memory and every later block read has to name the file again - there being no such thing
; as an open file to hold on to.
srcOpen:
SETD.1 SrcTopName
CALL srcKeepName ; Kept apart, so that each pass can open it again.
CALL srcRestart
RET
; Back to the top of the tree: the first file, no includes taken yet, nothing on the stack.
; This is what starts each pass.
srcRestart:
RSTA
SETD.0 SrcDepth
STA.0
SETD.0 IncCount
STA.0
SETD.0 SrcTopName
SETD.1 SrcName
CALL srcKeepName
CALL srcRewind
RET
; Back to the first character, for the second pass.
srcRewind:
SETD.0 SrcIndex
CALL numZero
SETD.0 SrcAt
CALL numZero
SETD.0 SrcCount
CALL numZero
RSTA
SETD.0 SrcEnded
STA.0
; The line number counts from one, the way an editor does.
SETD.0 SrcLine
CALL numZero
SETD.0 SrcLine
CALL numStep
; Ask how big it is, which is both the answer to "is it there" and the thing that says
; when to stop asking for blocks.
SETD.0 SrcName
SWI osFileInfo
BNQ srcRewindNo
PSHD.3
POPB
POPA
SETD.0 SrcBlocks
STA.0
INCD.0
STB.0
RSTA
RSTB
CCF
ADD ; Q is zero: it is there.
RET
srcRewindNo:
INIA 0d1
SETD.0 SrcEnded
STA.0
RSTA
INIB 0d1
CCF
ADD ; Q is not zero: it is not.
RET
; ---- Everything up to and including the next newline, thrown away ----
;
; Q is zero if a newline was found, and something else if the file ended first.
;
; THIS IS MOST OF WHAT THE ASSEMBLER READS. Measured on this repository: 87 per cent of what
; Say.asm pulls in is comment and blank line, and 59 per cent of CosmOS - the house style is
; dense commentary, so the assembler is penalised more than most by its own sources. Every one
; of those bytes used to go through tokGet and srcNext and be classified and thrown away.
;
; The saving is not the classifying, it is the BOOKKEEPING. srcNext loads and stores the
; walking pointer through memory for every single character, and asks numCompare whether the
; buffer is used up. Here the pointer lives in DP0 for a whole run and the newline lives in B,
; so a comment byte costs a load, a compare and two steps.
;
; A run is capped at 255 so that one byte can count it, which is why the outer loop exists at
; all: a full block is 256, one more than a byte holds.
srcSkipComment:
SETD.0 SrcEnded
LDA.0
BNA srcSkipEnded
srcSkipBlock:
; What is left of the buffer.
SETD.0 SrcSpan
SETD.2 SrcCount
CALL numSet
SETD.0 SrcSpan
SETD.2 SrcAt
CALL numTake
SETD.0 SrcSpan
LDA.0
INCD.0
LDB.0
OR
BRQ srcSkipRefill ; Used up, so the next block or the end.
SETD.0 SrcSpan
LDA.0
BNA srcSkipCapped ; A high half at all means 256, which is one too many to count.
INCD.0
LDA.0
BRI srcSkipCounted
srcSkipCapped:
INIA 0xFF
srcSkipCounted:
SETD.0 SrcRun
STA.0
SETD.0 SrcTaken
STA.0 ; Where the run began, to work out what it swallowed.
SETD.2 SrcPointer
LDD.0.2
SETD.1 SrcRun
INIB 0x0A ; The newline, held in B for the whole run.
srcSkipChar:
LDA.0
XOR
BRQ srcSkipNewline
INCD.0
LDA.1
DECA
STA.1
BNA srcSkipChar
; The run ended without a newline in it, so the rest of the block is comment too.
SETD.2 SrcPointer
STD.0.2
CALL srcSkipAccount
BRI srcSkipBlock
srcSkipNewline:
; Past the newline itself, which belongs to the comment.
INCD.0
SETD.1 SrcRun
LDA.1
DECA
STA.1
SETD.2 SrcPointer
STD.0.2
CALL srcSkipAccount
; A newline is what makes the next character part of the next line, counted as it goes past
; exactly as srcNext counts it.
SETD.0 SrcLine
CALL numStep
RSTA
RSTB
CCF
ADD
RET
srcSkipRefill:
CALL srcLoad
BRQ srcSkipBlock ; Another block of the same file.
; ---- A file that runs out inside a comment ----
;
; The same thing srcAtEnd does for a character, and it has to be done the same way here: an
; included file ending puts the one that included it back on, and the comment carries on
; into it exactly as it did when this was read a character at a time.
;
; NOT DOING THIS WAS THE WHOLE OF THE FIRST VERSION'S BUG. services.asm ends inside a
; comment, so an include of it never came back, and every label after the #Include line -
; the whole of the program doing the including - was quietly never defined. It surfaced as
; "no label of that name is defined anywhere" pointing at a line whose label was three
; lines from the end of the file.
INIA 0d1
SETD.0 SrcEnded
STA.0
SETD.0 SrcDepth
LDA.0
BRA srcSkipEnded ; Nothing included this, so there is no more source anywhere.
CALL srcPop
BRI srcSkipComment
srcSkipEnded:
INIA 0x01
RSTB
CCF
ADD
RET
; How much of the run went by, added to how far into the buffer we have read.
srcSkipAccount:
SETD.0 SrcTaken
LDA.0
SETD.0 SrcRun
LDB.0
CCF
SUB
MVQA
SETD.0 SrcAt
CALL numAddByte
RET
; The next character of the file, into SrcChar. Q is zero if there was one, and something
; else at the end of the file.
srcNext:
SETD.0 SrcEnded
LDA.0
BNA srcAtEnd
; Is the buffer used up? SrcAt counts how far into it we have read and SrcCount how many
; of its bytes are the file's, which is 256 for every block but a short last one.
SETD.0 SrcAt
SETD.2 SrcCount
CALL numCompare
BNQ srcHaveByte
CALL srcLoad
BNQ srcAtEnd
srcHaveByte:
SETD.1 SrcPointer
LDD.0.1
LDA.0
INCD.0
STD.0.1
SETD.0 SrcChar
STA.0
SETD.0 SrcAt
CALL numStep
; A newline is what makes the next character part of the next line. Counting it here,
; as it is handed out, means the line number always describes the character just given.
SETD.0 SrcChar
LDA.0
INIB 0x0A
XOR
BNQ srcNextDone
SETD.0 SrcLine
CALL numStep
srcNextDone:
RSTA
RSTB
CCF
ADD ; Q is zero: there was a character.
RET
srcAtEnd:
; This file is finished. If it was included by another, that one is not: it goes back on
; and the next character comes from where it left off, which is what makes an include
; read as though the text had been written there.
INIA 0d1
SETD.0 SrcEnded
STA.0
SETD.0 SrcDepth
LDA.0
BRA srcNothingLeft
CALL srcPop
BRI srcNext
srcNothingLeft:
RSTA
INIB 0d1
CCF
ADD ; Q is not zero: there is no more source anywhere.
RET
; ---- The stack ----
; Opens the file named at DP0 as though its text were written here. Q is zero if the
; reader is now inside it, or if it had already been included and there is nothing to do.
srcInclude:
SETD.1 IncWanted
CALL srcKeepName
CALL srcSeenAlready
BRQ srcIncludeSkip
SETD.0 SrcDepth
LDA.0
SETD.2 SrcDepthLimit
LDB.2
CCF
SUB
BNC srcTooDeep
CALL srcRemember
CALL srcPush
; TWO PLACES, TRIED IN ORDER: where you are, and then /Lib. The same rule the shell uses
; for a program it does not recognise, which is where it came from - a name means the
; one beside you if there is one, and the system's otherwise.
;
; It is what a search path is for, and the host assembler has had one since before there
; was a machine to run this on. Without it every source that calls a service has to sit
; in the same directory as services.asm, and the disk cannot be organised at all.
SETD.0 IncWanted
SETD.1 SrcName
CALL srcKeepName
CALL srcRewind
BRQ srcIncludeIn
CALL srcInLibrary
CALL srcRewind
BNQ srcIncludeGone
srcIncludeIn:
RET ; Q is zero, out of srcRewind.
; SrcName becomes the same name inside the library directory. Built here rather than kept
; as a second buffer, because what has to survive is the name the file was ASKED for -
; that is what the include-once list holds, and a file found in the library on one line
; and beside you on another is still the same include.
srcInLibrary:
SETD.0 SrcLibrary
SETD.1 SrcName
srcLibraryPrefix:
LDA.0
BRA srcLibraryName
STA.1
INCD.0
INCD.1
BRI srcLibraryPrefix
srcLibraryName:
SETD.0 IncWanted
srcLibraryCopy:
LDA.0
STA.1
BRA srcLibraryDone
INCD.0
INCD.1
BRI srcLibraryCopy
srcLibraryDone:
RET
srcIncludeSkip:
RSTA
RSTB
CCF
ADD ; Already in, so this line does nothing at all.
RET
srcIncludeGone:
; ---- Say which one ----
;
; It said nothing at all, and the assembler's catch-all then said "nothing was written" -
; which is true and useless. Three programs on the disk could be read and not assembled,
; and finding out why meant reading the assembler rather than the message.
;
; SrcName holds the library path by now, because that was the second place tried, so this
; says where it looked as well as what for. Both are worth having: a name that is not in
; /Lib and not beside you is usually one or the other spelt wrong.
SETD.0 GoneText
SWI osPrintString
SETD.0 IncWanted
SWI osPrintString
SETD.0 GoneWhereText
SWI osPrintString
SETD.0 SrcName
SWI osPrintString
SETD.0 GoneEndText
SWI osPrintString
; The file is not there. The stack is left as it is: the caller stops the assembly, and
; unwinding for the sake of tidiness would only hide where it happened.
RSTA
INIB 0d1
CCF
ADD
RET
srcTooDeep:
SETD.0 TooDeepText
SWI osPrintString
RSTA
INIB 0d1
CCF
ADD
RET
; Puts the current file aside and makes room for another.
srcPush:
CALL srcSlot
SETD.0 SrcState
SETD.1 SrcSlot
LDD.1.1
CALL srcCopyState
SETD.0 SrcDepth
LDA.0
INCA
STA.0
RET
; And takes it back.
srcPop:
SETD.0 SrcDepth
LDA.0
DECA
STA.0
CALL srcSlot
SETD.1 SrcSlot
LDD.0.1
SETD.1 SrcState
CALL srcCopyState
RET
; Puts one character back, so that the next read produces it again. The character itself
; is in A, because whether it was a newline decides whether a line goes back too.
;
; The tokenizer holds one character of lookahead, and at an #Include that character belongs
; to the file about to be put aside. Undoing the read is how it stays with that file: when
; the file is opened again the character is simply still there, and nothing has to be
; carried across the include or handed back at some moment chosen by the reader.
;
; CARRYING IT ACROSS WAS THE OBVIOUS THING AND IT WAS WRONG. A file runs out in the middle
; of whatever the tokenizer happens to be doing, so handing the character back then injects
; it into the middle of a word: `start:` came back as `s` and then `tart:`, which assembles
; into a file that looks entirely reasonable.
srcStepBack:
INIB 0x0A
XOR
BNQ srcStepBackAt
SETD.0 SrcLine
SETD.2 SrcOne
CALL numTake ; A newline not yet read has not started a line either.
srcStepBackAt:
SETD.0 SrcAt
SETD.2 SrcOne
CALL numTake
SETD.0 SrcPointer
SETD.2 SrcOne
CALL numTake
RET
; Where the slot for the current depth is, into SrcSlot.
srcSlot:
SETD.0 SrcSlot
SETD.2 ScratchSrcStack
CALL numSet ; WHERE the stack is, not what is in it.
SETD.0 SrcSlotLeft
SETD.2 SrcDepth
LDA.2
STA.0
srcSlotLoop:
SETD.0 SrcSlotLeft
LDA.0
BRA srcSlotDone
DECA
STA.0
SETD.0 SrcSlot
SETD.2 SrcStateBytes
CALL numAdd
BRI srcSlotLoop
srcSlotDone:
RET
; The whole reader state, from DP0 to DP1.
srcCopyState:
SETD.2 SrcCopyFrom
STD.0.2
SETD.2 SrcCopyTo
STD.1.2
SETD.0 SrcCopyLeft
SETD.2 SrcStateBytes
CALL numSet
srcCopyLoop:
SETD.1 SrcCopyFrom
LDD.0.1
LDA.0
SETD.1 SrcCopyTo
LDD.0.1
STA.0
SETD.0 SrcCopyFrom
CALL numStep
SETD.0 SrcCopyTo
CALL numStep
SETD.0 SrcCopyLeft
SETD.2 SrcOne
CALL numTake
SETD.0 SrcCopyLeft
LDA.0
INCD.0
LDB.0
OR
BNQ srcCopyLoop
RET
; ---- Which files have been in ----
; Q is zero if IncWanted has already been included in this pass.
srcSeenAlready:
RSTA
SETD.0 IncLeft
STA.0
srcSeenLoop:
SETD.0 IncLeft
LDA.0
SETD.2 IncCount
LDB.2
CCF
SUB
BRQ srcSeenNo
CALL srcSeenSlot
SETD.1 IncSlot
LDD.0.1
SETD.1 IncWanted
CALL sameText
BRQ srcSeenYes
SETD.0 IncLeft
LDA.0
INCA
STA.0
BRI srcSeenLoop
srcSeenYes:
RSTA
RSTB
CCF
ADD
RET
srcSeenNo:
RSTA
INIB 0d1
CCF
ADD
RET
; Writes IncWanted down as having been included.
srcRemember:
SETD.0 IncLeft
SETD.2 IncCount
LDA.2
STA.0
CALL srcSeenSlot
SETD.0 IncWanted
SETD.1 IncSlot
LDD.1.1
CALL srcKeepName
SETD.0 IncCount
LDA.0
INCA
STA.0
RET
; Where name number IncLeft sits, into IncSlot. Fixed fields of 23 bytes: there are few of
; these and they are short, so an arena would cost more code than it saved.
srcSeenSlot:
SETD.0 IncSlot
SETD.2 ScratchIncNames
CALL numSet
SETD.0 IncSlotLeft
SETD.2 IncLeft
LDA.2
STA.0
srcSeenSlotLoop:
SETD.0 IncSlotLeft
LDA.0
BRA srcSeenSlotDone
DECA
STA.0
INIA 0d23
SETD.0 IncSlot
CALL numAddByte
BRI srcSeenSlotLoop
srcSeenSlotDone:
RET
; Fetches the block SrcIndex names, and steps SrcIndex past it. Q is zero if there was one.
;
; Running off the end is not a failure here: osFileBlock answers three for a block past the
; end of the file, which is how a reader finds out it has finished. Any other refusal is a
; real one, and both come back the same way because there is nothing useful to do about
; either except stop.
srcLoad:
SETD.0 SrcName
SETD.1 SrcBuffer
SETD.2 SrcIndex
LDA.2
INCD.2
LDB.2
SWI osFileBlock
BNQ srcLoadNo
; DP3 says how many of the block's bytes belong to the file: a whole 256 except in a
; short last one, which is why it comes back in a pointer and not a register.
PSHD.3
POPB
POPA
SETD.0 SrcCount
STA.0
INCD.0
STB.0
SETD.0 SrcAt
CALL numZero
SETD.0 SrcIndex
CALL numStep
; The walking pointer starts at the front of the buffer again.
SETD.0 SrcBuffer
SETD.1 SrcPointer
STD.0.1
RSTA
RSTB
CCF
ADD
RET
srcLoadNo:
RSTA
INIB 0d1
CCF
ADD
RET
; Copies the name at DP0 into DP1, up to 22 characters of it and the zero after them,
; which is as long as a name on this filesystem may be.
srcKeepName:
INIA 0d22
SETD.2 SrcLeft
STA.2
srcKeepLoop:
LDA.0
BRA srcKeepEnd
STA.1
INCD.0
INCD.1
LDA.2
DECA
STA.2
BNA srcKeepLoop
srcKeepEnd:
RSTA
STA.1 ; The zero that makes it a string.
RET
#Data
; ---- The current file, as one block so that it can be put aside in one piece ----
;
SrcState:
; THIRTY TWO RATHER THAN THE TWENTY THREE A NAME NEEDS, because what goes here is a PATH:
; an include not found beside you is looked for in the library, and "/Lib/" plus a name of
; twenty two plus the zero that ends it is twenty eight. Every block of the file is asked
; for by this name, so it has to be the one that resolves, not the one that was typed.
SrcName:
#Reserve 0d32
SrcBlocks:
0x00 0x00
SrcIndex:
0x00 0x00
SrcCount:
0x00 0x00
SrcAt:
0x00 0x00
SrcLine:
0x00 0x00
SrcPointer:
0x00 0x00
SrcEnded:
0x00
; One block, which is the whole of what a source file costs in memory however big it is.
SrcBuffer:
#Reserve 0d256
; 301 bytes: a name of 32, six numbers of two, one single byte, and the buffer. NOTHING MAY
; BE ADDED IN THE MIDDLE OF THE BLOCK ABOVE without changing this to match, and the room
; set aside for six of them in scratch.asm has to be at least six times it.
SrcStateBytes:
0x01 0x2D
SrcDepthLimit:
0d6
; ---- OUTSIDE the state block above, and that is not an accident ----
;
; What is left of the buffer, how much of it one run may take, and where that run began.
; They live for the length of one call to srcSkipComment and never across an include, so they
; are not part of what a file remembers while another one is being read.
;
; PUT AMONG THE STATE FIRST, which added four bytes to a block that is copied whole by a
; count written down somewhere else - so every saved file lost the last four bytes of itself
; and an include came back with its buffer pointer wrong. The comment two paragraphs up says
; not to do it, and says it in capitals.
SrcSpan:
0x00 0x00
SrcRun:
0x00
SrcTaken:
0x00
; Where an include is looked for when it is not beside you. One fixed place rather than a
; list somebody sets, for the same reason the shell has one fixed place for programs: a
; list would need somewhere to live between one boot and the next.
SrcLibrary:
"/Lib/"
SrcOne:
0x00 0x01
SrcChar:
0x00
SrcLeft:
0x00
SrcDepth:
0x00
SrcSlot:
0x00 0x00
SrcSlotLeft:
0x00
SrcCopyFrom:
0x00 0x00
SrcCopyTo:
0x00 0x00
SrcCopyLeft:
0x00 0x00
; The file the assembly started from, so that each pass can open it again.
SrcTopName:
#Reserve 0d23
IncWanted:
#Reserve 0d23
IncCount:
0x00
IncLeft:
0x00
IncSlot:
0x00 0x00
IncSlotLeft:
0x00
GoneText:
"cannot find "
GoneWhereText:
", not beside the file that asked and not as "
GoneEndText:
"
"
TooDeepText:
"included files are nested deeper than this assembler will follow
"