Streaming: read a file bigger than the machine's memory

osFileRead hands over a whole file, which settles anything under 64K and
settles nothing above it. CosmOS's own source is above it - the sources
together are 104K against 64K of Data Memory - so a machine that is going
to assemble itself needs another way to ask.

osFileInfo (0d26) says how many blocks a file occupies. osFileBlock (0d27)
hands over one of them and says how many of its bytes belong to the file.
Between them a program reads a file of any size through a buffer of 256.

Blocks rather than bytes from osFileInfo is forced, not chosen: a file on a
sixteen megabyte disk is up to twenty four bits long and a pointer holds
sixteen. osFileBlock's count answers in DP3 for the same kind of reason -
a whole block is 256 bytes, which does not fit in a register, and a count
that reported it as zero would make every reader special-case the end.

Nothing is kept open. Every call names the file, so there is no handle to
leak and nothing left behind by a program that stops halfway. Taken at its
word that means searching the directory once per block, so the system
remembers where the last file it was asked about lives; every path that can
change what a name means calls fileForget, including the shell's own delete
and rename, which do not go through the services. Correctness never depends
on the cache - a cache thrown away is indistinguishable from one never
filled. Measured on a 329 block file: 7% of the run saved when the file is
the first directory entry, 11% when it is the sixteenth.

These two say WHY when the answer is no, which the others do not. Elsewhere
the only useful response to a failure is to give up, so one value suffices.
These are asked questions, and running off the end is how a reader learns it
has finished, so it gets an answer of its own: 1 no disk, 2 no such file,
3 past the end, 4 the disk refused.

Apps/Stream.asm reads an 84,000 byte file through 256 bytes. The check that
matters is the second one: a small file read BOTH ways - whole with
osFileRead and streamed - with the two checksums compared, so streaming is
measured against the path already known to work rather than against a number
someone wrote down. The checksum is Fletcher's rather than a sum, because a
sum is the same whatever order the bytes arrived in and the order is exactly
what streaming has to get right. Both checksums were also confirmed against
the same arithmetic run on the host.

The rest of the test is the cache: two files read alternately catch a memory
that missed the name changing, and a rename catches one that missed the file
moving - and that one would otherwise pass, since the blocks are still there
holding the same bytes.

The test file is generated rather than taken from the repository. The CosmOS
sources would be a truer picture and would move the recorded checksum every
time a line of CosmOS was edited, putting a real difference in a crowd of
meaningless ones - the same trap the cycle counts used to set.

cosmosBreak's recorded output moves by two bytes in two pointers: SbfsIndex
added two bytes to the filesystem's data and Break prints the system
addresses the registers happened to hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
Anachronaut
2026-08-20 21:19:36 -04:00
co-authored by Claude Opus 5
parent 3b800a69e8
commit 3d2ab34229
12 changed files with 940 additions and 2 deletions
+44
View File
@@ -473,6 +473,48 @@ sbfsReadDone:
ADD ; Q is zero: read.
RET
; Reads one block of the file sbfsFind found, the one SbfsIndex names counting from zero,
; into Data Memory at DP1. Q is zero if it worked.
;
; This is the whole of streaming: a file too big to hold is read a block at a time by
; asking for each in turn, and nothing has to be kept between the calls but the number.
; sbfsRead is what this would be if it were called in a loop, which is why the two look
; alike; it stays as it is because reading a whole small file is what most callers want
; and doing it in one call is both shorter and faster.
;
; The whole block comes across, including a last one that the file only partly fills, so
; the bytes past the end of it are whatever else was on the disk there. SbfsFileTail says
; where the file stops and it is the caller's business to respect it, the same bargain
; sbfsRead offers.
;
; An index past the end of the file is not caught here. It reads whatever block that
; works out to, which is somebody else's file or free space; the caller knows how many
; blocks there are, because sbfsFileExtent tells it.
sbfsReadOne:
PSHD.1
POPD.3 ; Where it goes. DP3 is the one pointer a CALL does not put back.
SETD.0 SbfsBlock
SETD.2 SbfsFileStart
CALL sbfsSetWord
SETD.0 SbfsBlock
SETD.2 SbfsIndex
CALL sbfsAddWord
CALL sbfsReadBlock
BNQ sbfsReadOneDone ; The read failed, and Q says so.
PSHD.3
POPD.1
CALL sbfsBufferOut
RSTA
RSTB
CCF
ADD ; Q is zero: read.
sbfsReadOneDone:
RET
; ---- Writing ----
; Where the first block that can hold a file is: past the superblock and the directory.
@@ -1239,6 +1281,8 @@ SbfsMatchLeft:
0x00
SbfsLeft:
0x00
SbfsIndex:
0x00 0x00
; ---- What a walk through the directory keeps between calls ----