AnachronautandClaude Opus 5
c5e4ec3455
> load Asm.sbx
> run Say.asm
wrote Say.sbx: program 46, data 93, labels 7
> load Say.sbx
> run built by the machine itself
it says: built by the machine itself
The machine assembles an application and then runs what it built. Say,
greet and Files all come out byte for byte identical to the C assembler's,
and Tests/native.sh checks all three on every run alongside the boot image
M1 already covered.
WHAT IT TOOK, and it was more than #Include and #Base:
#Include The reader is a stack of readers. The current file's whole
state goes aside - buffer and all, 292 bytes - the new one
opens, and the end of it pops the old one back. A file goes in
once; including it twice does nothing, which is what lets two
libraries depend on a third. The list is forgotten between the
passes, because the second has to walk the same tree.
#Base Cursors start there, so labels hold the addresses the program
will really have. A program that says where it goes gets the
SBEX header and a .sbx name; one that says nothing gets SPBT
and .bin. A program that bases one segment and leaves the
other unbased with content in it is refused.
#Reserve Runs of zeroes, moved over in the first pass and written in
#Align the second. How many an #Align comes to depends on where the
cursor has reached, which is why both passes keep a cursor.
#Vectors Names are read and numbered, pinned where the source pins
them, so SWI osPrintString resolves. Every application needs
this - a program that calls a service names a vector declared
in a file it includes.
THE TWO PASSES ARE NOW ONE LOOP, walked twice, with Emitting the only
difference. They have to agree about the length of every token, and the way
they stop agreeing is by being two pieces of code that drifted apart -
which is the exact shape of the bug this assembler found in the C one.
Sharing the body means there is nothing to drift. What is left is checked
anyway: the second pass compares its own totals against the first's and
refuses to write the file if they differ.
THE BUG WORTH RECORDING. The tokenizer holds one character of lookahead,
and at an #Include that character belongs to the file being put aside. It
was carried across and handed back on the way out, which is wrong: a file
runs out in the middle of whatever the tokenizer happens to be doing, so
the character arrives in the middle of a word. `start:` came back as `s`
and then `tart:` - and the result assembled into a perfectly plausible
file. The fix is to undo the read instead, so the character is simply still
there when the file is opened again and the question of when to hand it
back never arises.
Three smaller ones, all old friends: three places took the CONTENTS of a
buffer where they wanted its ADDRESS; vecTakeAuto returned its answer in A,
which a RET puts back; and pass two re-declared every vector because only
the label table was being skipped on the second walk.
sameText moved down into numbers.asm from the label table - four parts want
it now, and a reader test that includes neither labels nor tokens has to
build on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
294 lines
6.2 KiB
NASM
294 lines
6.2 KiB
NASM
; Tokens out of characters.
|
|
;
|
|
; A token is a run of characters with whitespace or a comment on either side, or anything
|
|
; between a pair of quotes. That is the whole of the lexical grammar: SplitBit assembly has
|
|
; no operators, no punctuation and no line continuation, so there is nothing here that has
|
|
; to look ahead more than one character.
|
|
;
|
|
; ONE CHARACTER OF LOOKAHEAD, and it is held here rather than in the reader. A word ends
|
|
; when something that is not part of it turns up, and that something has already been read
|
|
; by the time anyone knows - so it is put in TokPending and taken again next time. Keeping
|
|
; it at this level rather than pushing it back into the reader means the line number needs
|
|
; no arithmetic: srcNext counted the newline when it handed it out, and it stays counted.
|
|
;
|
|
; Zero means "nothing held", which is safe because a source file is text and a text file
|
|
; has no zero bytes in it. A file that did would be rejected as unassemblable long before
|
|
; the difference showed.
|
|
;
|
|
; Written by Anachronaut
|
|
|
|
#Program
|
|
|
|
; The next token, into TokText with a zero after it. Q is zero if there was one.
|
|
;
|
|
; TokLength is how long it is, TokString says whether it arrived in quotes, and TokLine is
|
|
; the line it STARTED on - captured before the token is read, because a token ending in a
|
|
; newline has already moved the reader on to the next line by the time it is finished.
|
|
tokNext:
|
|
SETD.0 TokHeld
|
|
LDA.0
|
|
BRA tokFresh
|
|
RSTA
|
|
STA.0
|
|
RSTA
|
|
RSTB
|
|
CCF
|
|
ADD ; The one that was handed back, exactly as it was.
|
|
RET
|
|
|
|
tokFresh:
|
|
RSTA
|
|
SETD.0 TokString
|
|
STA.0
|
|
|
|
tokSkip:
|
|
CALL tokGet
|
|
BNQ tokEnded
|
|
CALL tokIsSpace
|
|
BRQ tokSkip
|
|
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x3B ; A semicolon starts a comment.
|
|
XOR
|
|
BNQ tokBegin
|
|
|
|
tokComment:
|
|
CALL tokGet
|
|
BNQ tokEnded
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x0A
|
|
XOR
|
|
BNQ tokComment ; Everything up to the newline belongs to the comment.
|
|
BRI tokSkip
|
|
|
|
tokBegin:
|
|
; Where it starts, for anything that has to complain about it later.
|
|
SETD.0 TokLine
|
|
SETD.2 SrcLine
|
|
CALL numSet
|
|
|
|
RSTA
|
|
SETD.0 TokLength
|
|
STA.0
|
|
SETD.0 TokText
|
|
SETD.1 TokPointer
|
|
STD.0.1
|
|
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x22 ; A quote starts a string.
|
|
XOR
|
|
BNQ tokWord
|
|
|
|
INIA 0d1
|
|
SETD.0 TokString
|
|
STA.0
|
|
|
|
tokStringLoop:
|
|
CALL tokGet
|
|
BNQ tokDone ; The file ended inside a string. Take what there is; the
|
|
; classifier will have something to complain about.
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x22
|
|
XOR
|
|
BRQ tokDone
|
|
CALL tokAppend
|
|
BRI tokStringLoop
|
|
|
|
tokWord:
|
|
CALL tokAppend
|
|
tokWordLoop:
|
|
CALL tokGet
|
|
BNQ tokDone
|
|
CALL tokIsSpace
|
|
BRQ tokHoldDone
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x3B
|
|
XOR
|
|
BRQ tokHoldDone ; A comment butting straight up against a word ends it.
|
|
CALL tokAppend
|
|
BRI tokWordLoop
|
|
|
|
tokHoldDone:
|
|
; Whatever ended the word was not part of it, so it goes back to be looked at again.
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
SETD.0 TokPending
|
|
STA.0
|
|
|
|
tokDone:
|
|
SETD.1 TokPointer
|
|
LDD.0.1
|
|
RSTA
|
|
STA.0 ; The zero that makes it a string.
|
|
RSTA
|
|
RSTB
|
|
CCF
|
|
ADD ; Q is zero: there was a token.
|
|
RET
|
|
|
|
tokEnded:
|
|
RSTA
|
|
SETD.0 TokLength
|
|
STA.0
|
|
SETD.0 TokText
|
|
STA.0
|
|
RSTA
|
|
INIB 0d1
|
|
CCF
|
|
ADD ; Q is not zero: the source is finished.
|
|
RET
|
|
|
|
; Puts the held character back into the file it came from, leaving nothing in hand.
|
|
;
|
|
; This is what #Include calls before the reader puts the current file aside. The character
|
|
; the tokenizer is holding was read from that file and has not been used, so it goes back
|
|
; into it; there is then nothing to carry across the include and nothing to hand back at a
|
|
; moment that might land in the middle of a word.
|
|
tokUnread:
|
|
SETD.0 TokPending
|
|
LDA.0
|
|
BRA tokUnreadDone
|
|
CALL srcStepBack
|
|
RSTA
|
|
SETD.0 TokPending
|
|
STA.0
|
|
tokUnreadDone:
|
|
RET
|
|
|
|
; Hands the token just read back, so that the next tokNext produces it again.
|
|
;
|
|
; ONE TOKEN, and only where nothing has changed it since. The Vector Segment needs it: a
|
|
; name there may be followed by a number, by a handler, or by the next line's name, and
|
|
; which it is cannot be known without looking. Do NOT use it after anything that alters
|
|
; TokText - a label definition with its colon written over would come back as a use of the
|
|
; name rather than as a definition of it.
|
|
tokBack:
|
|
INIA 0d1
|
|
SETD.0 TokHeld
|
|
STA.0
|
|
RET
|
|
|
|
; The next character, into TokChar. Q is zero if there was one. Takes the held one first.
|
|
tokGet:
|
|
SETD.0 TokPending
|
|
LDA.0
|
|
BRA tokGetFresh
|
|
SETD.0 TokChar
|
|
STA.0
|
|
RSTA
|
|
SETD.0 TokPending
|
|
STA.0
|
|
RSTA
|
|
RSTB
|
|
CCF
|
|
ADD
|
|
RET
|
|
|
|
tokGetFresh:
|
|
CALL srcNext
|
|
BNQ tokGetNone
|
|
SETD.0 SrcChar
|
|
LDA.0
|
|
SETD.0 TokChar
|
|
STA.0
|
|
RSTA
|
|
RSTB
|
|
CCF
|
|
ADD
|
|
RET
|
|
|
|
tokGetNone:
|
|
RSTA
|
|
INIB 0d1
|
|
CCF
|
|
ADD
|
|
RET
|
|
|
|
; Adds TokChar to the token being built, unless it is already as long as one may be.
|
|
;
|
|
; A token that runs over is truncated rather than refused, and the classifier refuses it
|
|
; afterwards: nothing 255 characters long is a valid mnemonic, literal or label, so the
|
|
; error that comes out names what was wrong with it rather than only how long it was.
|
|
tokAppend:
|
|
SETD.0 TokLength
|
|
LDA.0
|
|
INIB 0xFF
|
|
XOR
|
|
BRQ tokAppendFull
|
|
SETD.1 TokPointer
|
|
LDD.0.1
|
|
SETD.2 TokChar
|
|
LDA.2
|
|
STA.0
|
|
INCD.0
|
|
STD.0.1
|
|
SETD.0 TokLength
|
|
LDA.0
|
|
INCA
|
|
STA.0
|
|
tokAppendFull:
|
|
RET
|
|
|
|
; Q is zero if TokChar is whitespace: a space, or anything in the run from tab to carriage
|
|
; return, which is what the C library calls a space and what the other assembler uses.
|
|
tokIsSpace:
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x20
|
|
XOR
|
|
BRQ tokSpaceYes
|
|
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x09
|
|
CCF
|
|
SUB
|
|
BRC tokSpaceNo ; Below a tab.
|
|
SETD.0 TokChar
|
|
LDA.0
|
|
INIB 0x0E
|
|
CCF
|
|
SUB
|
|
BNC tokSpaceNo ; Past a carriage return.
|
|
|
|
tokSpaceYes:
|
|
RSTA
|
|
RSTB
|
|
CCF
|
|
ADD
|
|
RET
|
|
|
|
tokSpaceNo:
|
|
RSTA
|
|
INIB 0d1
|
|
CCF
|
|
ADD
|
|
RET
|
|
|
|
#Data
|
|
|
|
TokLine:
|
|
0x00 0x00
|
|
TokPointer:
|
|
0x00 0x00
|
|
TokLength:
|
|
0x00
|
|
TokString:
|
|
0x00
|
|
TokChar:
|
|
0x00
|
|
TokPending:
|
|
0x00
|
|
TokHeld:
|
|
0x00
|
|
|
|
; As long as a token may be, and one more for the zero. The other assembler stops at the
|
|
; same 255, and the limit is worth matching rather than choosing again.
|
|
TokText:
|
|
#Reserve 0d256
|