Stop the allocator starting the directory again for every file in its way

Placing one file on a disk with the source tree on it cost 9.4 million cycles. It costs 1.4
million now, and assembling colours.asm went from 13.9 to 6.0 seconds.

sbfsAllocate gave up the moment it found anything in the candidate's way: it moved the
candidate past that one entry and STARTED THE DIRECTORY AGAIN FROM THE FIRST BLOCK. With
files laid down one after another that is a restart per file, and every restart reads
directory blocks off the disk until it reaches the next thing in the way - which is further
in each time. Placing one file among 183 of them cost thousands of block reads.

The candidate moves along DURING the pass now, and the pass carries on from where it is, so
entries later in the directory are tested against where the candidate has got to. On a disk
that has been appended to - which is what a disk mostly is - one pass walks it past
everything and a second confirms nothing is left. Two passes rather than one per file.

IT IS STILL FIRST FIT, and Tests/agree.sh is what says so: the machine and SplitDisk build
the same tree and the images still match byte for byte, which they could not if allocation
had started choosing differently. The argument is that the candidate only ever moves past
something that genuinely overlaps it, and when it does there is nowhere below to go - the
entry in the way covers everything up to its end and begins before the candidate ends.

The first attempt at this was slower than what it replaced, by three times. It finished the
pass and jumped to the FURTHEST overlap, which sounds better and is worse: with files laid
contiguously only one entry ever overlaps, so the old early exit was the fast path and
reading the whole directory to find the one thing was pure loss. The number of passes was
never the thing to fix - restarting them was.

The boot slot in the test fixtures goes from 32 blocks to 40, which is what a shipped disk
has. Stage two is 8,231 bytes and 32 blocks is 8,192: a fixture tighter than the thing it
stands in for fails on a change the real disk would have taken, and says "the boot slot is
too small" rather than what actually grew.

WHAT THIS DOES NOT FIX is assembling CosmOS, and that is worth saying plainly. It takes 654
million cycles on the mirrored disk and 653 million on a flat test disk with a sixth as many
files, so it is not a filesystem problem at all. Cycles per byte of source climb with the
size of it - 1,383 for colours.asm, about 3,000 for Edit.asm, 6,290 for cosmos.asm - which
says the native assembler is superlinear in what it reads. That is a separate thing to go
and look at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW

Programs/CosmOS/Source/sbfs.asm

@@ -1413,10 +1413,37 @@ sbfsBoundsDone:
 ; First fit, walking the directory, because with files laid down contiguously the
 ; directory already says which blocks are spoken for. There is no allocation table to
 ; consult and none to keep right.
+;
+; ---- Moving the candidate along without starting again ----
+;
+; This used to give up the moment it found something in the way: it moved the candidate past
+; that one entry and STARTED THE DIRECTORY AGAIN FROM THE FIRST BLOCK. With files laid down
+; one after another that is a restart per file, and every restart reads directory blocks off
+; the disk until it reaches the next one in the way - which is further in each time. A disk
+; of 183 files cost thousands of block reads to place one file, and assembling onto a disk
+; with the whole source tree on it took eleven minutes with nearly all of it spent here.
+;
+; The candidate moves along DURING the pass now, and the pass carries on from where it is.
+; Entries later in the directory are then tested against where the candidate has got to, so
+; on a disk that has been appended to - which is what a disk mostly is - one pass walks it
+; past everything and the next confirms there is nothing left in the way. Two passes rather
+; than one per file.
+;
+; IT IS STILL FIRST FIT. The candidate only ever moves past something that genuinely
+; overlaps it, and when it does there is nowhere below that could have held the run: the
+; entry in the way covers everything up to its end, and it begins before the candidate ends.
+; So nothing is skipped that first fit would have found - and because entries earlier in the
+; directory were tested against an earlier candidate, the pass repeats until one goes by with
+; the candidate standing still.
 sbfsAllocate:
   CALL sbfsFirstData
 
 sbfsAllocTry:
+  ; The candidate has not moved yet this time round.
+  SETD.0 SbfsAllocMoved
+  RSTA
+  STA.0
+
   SETD.0 SbfsCandEnd
   SETD.2 SbfsCandidate
   CALL sbfsSetWord
@@ -1491,12 +1518,26 @@ sbfsAllocEntry:
   CALL sbfsCompareWord
   BNC sbfsAllocClear
 
-  ; They do overlap, so try again from the far end of whatever is in the way.
+  ; They do overlap, so the candidate moves to the far end of what is in the way - and the
-  POPD.2
+  ; scan carries on from here rather than beginning again, so whatever comes next is measured
+  ; against where the candidate has got to.
+  ;
+  ; WHERE THIS ENTRY IS STAYS ON THE STACK THROUGHOUT. The work below wants DP2 for its own
+  ; purposes and the scan needs it back on the entry to step to the next one, which is what
+  ; sbfsAllocClear is for.
   SETD.0 SbfsCandidate
   SETD.2 SbfsEntryEnd
   CALL sbfsSetWord
-  BRI sbfsAllocTry
+  SETD.0 SbfsCandEnd
+  SETD.2 SbfsCandidate
+  CALL sbfsSetWord
+  SETD.0 SbfsCandEnd
+  SETD.2 SbfsWantBlocks
+  CALL sbfsAddWord
+  INIA 0x01
+  SETD.0 SbfsAllocMoved
+  STA.0
+  ; And on into sbfsAllocClear, which puts the entry back in DP2 and steps to the next one.
 
 sbfsAllocClear:
   POPD.2
@@ -1517,6 +1558,16 @@ sbfsAllocNext:
   STA.1
   BNA sbfsAllocBlock
 
+  ; ---- The pass is over ----
+  ;
+  ; Something was in the way, so start again from the far end of the furthest of them. That
+  ; is one jump for however many files the candidate ran into, rather than one jump each.
+  SETD.0 SbfsAllocMoved
+  LDA.0
+  BRA sbfsAllocRoom             ; It never moved, so nothing is in the way of where it is.
+  BRI sbfsAllocTry
+
+sbfsAllocRoom:
   ; Nothing was in the way, so this is where it goes.
   SETD.0 SbfsFileStart
   SETD.2 SbfsCandidate
@@ -3087,6 +3138,9 @@ SbfsCandidate:
   0x00 0x00
 SbfsCandEnd:
   0x00 0x00
+; Whether the candidate had to move at all during the pass just finished. See sbfsAllocate.
+SbfsAllocMoved:
+  0x00
 SbfsEntryStart:
   0x00 0x00
 SbfsEntryEnd:

Tests/makedisks.sh

@@ -393,7 +393,13 @@ tail -c +17 "$WORK/slotData.sbx" > "$WORK/slotData.raw"
 # ordinary file. Nothing here is a boot-specific format: /System/cosmos.bin is the same
 # SPBT image the emulator has always been handed directly, which is what makes a program
 # that wants no operating system startable the same way.
-"$TOOL" format "$DISKS/selfboot.img" 512 4 32 >/dev/null
+# ---- Forty blocks a slot, the same as a disk that ships ----
+#
+# It was thirty-two, which was ten thousand bytes of headroom when stage two was four
+# thousand bytes and none at all when it reached 8,234. A fixture tighter than the thing it
+# stands in for is a fixture that fails on a change the real disk would have taken, and the
+# failure says "the boot slot is too small" rather than what actually grew.
+"$TOOL" format "$DISKS/selfboot.img" 512 4 40 >/dev/null
 "$ROOT/Assembler" -I "$ROOT/Programs/CosmOS/Source" \
     "$ROOT/Programs/Boot/stage2.asm" -o "$WORK/stage2.sbx" >/dev/null
 tail -c +17 "$WORK/stage2.sbx" > "$WORK/stage2.raw"
@@ -413,7 +419,7 @@ tail -c +17 "$WORK/stage2.sbx" > "$WORK/stage2.raw"
 
 # And one with no system on it, so that a second stage which cannot find what to start
 # says so rather than jumping somewhere.
-"$TOOL" format "$DISKS/nosystem.img" 512 4 32 >/dev/null
+"$TOOL" format "$DISKS/nosystem.img" 512 4 40 >/dev/null
 "$TOOL" boot "$DISKS/nosystem.img" "$WORK/stage2.raw" 0 >/dev/null
 
 # ---- Configuration choosing what starts ----