Two calls per character of every pass: numCompare to ask whether the
buffer is used up, and numStep to move one further into it. Inlined, the
compare settles almost every time on the high bytes and settles them
FIRST - a full block is 256, so SrcCount's high byte is one and SrcAt's
is nought until the last sixteenth of the block. Six instructions where
the call was twenty.
And the newline test uses A, which still holds the character, where it
used to fetch it back out of the SrcChar it had just been put in.
before after
hello 586,184 574,737
Say 4,506,702 4,333,689
Files 7,188,351 6,863,744
Keys 15,069,880 14,399,801
cosmos 789,982,899 772,826,841
Two to four and a half per cent, which is worth having and is much less
than expected - AND THAT IS THE INTERESTING PART. It says the per
character cost is not in the reading. Now that comments no longer come
through here at all, what is left is the tokenizer above it, and the
remaining time is there rather than in getting the bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
AnachronautandClaude Opus 5
09c72e4951