Streaming: read a file bigger than the machine's memory

osFileRead hands over a whole file, which settles anything under 64K and
settles nothing above it. CosmOS's own source is above it - the sources
together are 104K against 64K of Data Memory - so a machine that is going
to assemble itself needs another way to ask.

osFileInfo (0d26) says how many blocks a file occupies. osFileBlock (0d27)
hands over one of them and says how many of its bytes belong to the file.
Between them a program reads a file of any size through a buffer of 256.

Blocks rather than bytes from osFileInfo is forced, not chosen: a file on a
sixteen megabyte disk is up to twenty four bits long and a pointer holds
sixteen. osFileBlock's count answers in DP3 for the same kind of reason -
a whole block is 256 bytes, which does not fit in a register, and a count
that reported it as zero would make every reader special-case the end.

Nothing is kept open. Every call names the file, so there is no handle to
leak and nothing left behind by a program that stops halfway. Taken at its
word that means searching the directory once per block, so the system
remembers where the last file it was asked about lives; every path that can
change what a name means calls fileForget, including the shell's own delete
and rename, which do not go through the services. Correctness never depends
on the cache - a cache thrown away is indistinguishable from one never
filled. Measured on a 329 block file: 7% of the run saved when the file is
the first directory entry, 11% when it is the sixteenth.

These two say WHY when the answer is no, which the others do not. Elsewhere
the only useful response to a failure is to give up, so one value suffices.
These are asked questions, and running off the end is how a reader learns it
has finished, so it gets an answer of its own: 1 no disk, 2 no such file,
3 past the end, 4 the disk refused.

Apps/Stream.asm reads an 84,000 byte file through 256 bytes. The check that
matters is the second one: a small file read BOTH ways - whole with
osFileRead and streamed - with the two checksums compared, so streaming is
measured against the path already known to work rather than against a number
someone wrote down. The checksum is Fletcher's rather than a sum, because a
sum is the same whatever order the bytes arrived in and the order is exactly
what streaming has to get right. Both checksums were also confirmed against
the same arithmetic run on the host.

The rest of the test is the cache: two files read alternately catch a memory
that missed the name changing, and a rename catches one that missed the file
moving - and that one would otherwise pass, since the blocks are still there
holding the same bytes.

The test file is generated rather than taken from the repository. The CosmOS
sources would be a truer picture and would move the recorded checksum every
time a line of CosmOS was edited, putting a real difference in a crowd of
meaningless ones - the same trap the cycle counts used to set.

cosmosBreak's recorded output moves by two bytes in two pointers: SbfsIndex
added two bytes to the filesystem's data and Break prints the system
addresses the registers happened to hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
Anachronaut
2026-08-20 21:19:36 -04:00
co-authored by Claude Opus 5
parent 3b800a69e8
commit 3d2ab34229
12 changed files with 940 additions and 2 deletions
+491
View File
@@ -0,0 +1,491 @@
; Reading a file the machine cannot hold.
;
; Every other program here asks for a file and is handed the whole of it, which settles the
; question for anything under 64K and settles nothing above. CosmOS's own source is above:
; the sources together are a hundred kilobytes, and Data Memory is sixty four. A machine
; that is one day going to assemble itself has to be able to read a file bigger than its
; memory, and this is the program that proves it can.
;
; It uses osFileInfo and osFileBlock, and nothing else knows how a filesystem works. There
; is no open and no close - every call names the file and says which block it wants, so a
; program that stops halfway leaves nothing behind for anybody to clean up.
;
; ---- What it checks, and why each one is here ----
;
; 1. A file of four hundred odd blocks is read from end to end, a block at a time, into a
; buffer of one block. That is the feature.
; 2. A small file is read BOTH WAYS - whole with osFileRead, and streamed - and the two
; have to agree. This is the real proof: it compares streaming against the path that
; was already known to work, so a fault in the block count or the order of the blocks
; shows up as a difference rather than as a plausible wrong answer.
; 3. Two files are read alternately. The system remembers where the last file it was
; asked about lives, and this is the case that catches a memory that does not notice
; the name has changed.
; 4. A rename in the middle. Same reason, from the other side: the file the system
; remembers has moved out from under the name it remembered it by.
; 5. The three ways of being told no, each with its own number.
;
; THE CHECKSUM IS FLETCHER'S, not a sum. A plain total is the same whatever order the bytes
; arrived in, and the order is exactly what streaming has to get right; carrying a second
; accumulator that adds the first one in each time makes a block delivered out of turn
; change the answer.
;
; Written by Anachronaut
#Include services.asm
#Program
#Base 0x2000
start:
; ---- 1. How big is something that will not fit ----
;
; In blocks, not bytes, and that is forced rather than chosen: a file on a sixteen
; megabyte disk can be twenty four bits long and a pointer holds sixteen.
SETD.0 BigName
SWI osFileInfo
BNQ noBig
SETD.0 BigIs
SWI osPrintString
PSHD.3
POPB
POPA
SWI osPrintNumber
SETD.0 BlocksText
SWI osPrintString
; ---- 2. Read the whole of it through a hole one block wide ----
CALL clearChecksum
CALL clearIndex
bigLoop:
SETD.0 BigName
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2 ; Which block, most significant first.
SWI osFileBlock
BNQ bigDone
CALL takeCount
SETD.1 Block
CALL checksum
CALL stepIndex
BRI bigLoop
bigDone:
; The loop ends because a block past the end was asked for, which is answer three. Any
; other answer stopped it early and would otherwise look exactly like success, so what
; ended it is printed rather than assumed.
CALL keepWhy
SETD.0 ReadText
SWI osPrintString
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osPrintNumber
SETD.0 BlocksSumText
SWI osPrintString
CALL printChecksum
SETD.0 StoppedText
SWI osPrintString
CALL printWhy
; ---- 3. The same file both ways ----
;
; osFileRead is the path that already worked, so it is what streaming is measured
; against. If the two checksums agree, every byte arrived and they arrived in order.
SETD.0 SmallName
SETD.1 Whole
SWI osFileRead
BNQ noSmall
CALL takeCount
CALL clearChecksum
SETD.1 Whole
CALL checksum
CALL keepChecksum
CALL clearChecksum
CALL clearIndex
smallLoop:
SETD.0 SmallName
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osFileBlock
BNQ smallDone
CALL takeCount
SETD.1 Block
CALL checksum
CALL stepIndex
BRI smallLoop
smallDone:
SETD.0 BothText
SWI osPrintString
CALL printChecksum
SETD.0 AgainstText
SWI osPrintString
CALL printKept
SETD.0 NewLine
SWI osPrintString
CALL sameAsKept
BNQ differ
SETD.0 SameText
SWI osPrintString
BRI interleave
differ:
SETD.0 DifferText
SWI osPrintString
; ---- 4. Two files, alternately ----
;
; Block zero of the big file, then a block of the small one, then block zero of the big
; file again. The two readings of the same block have to match. A system that remembered
; the first file and did not notice the name had changed would hand back a block of the
; wrong file in the middle, and then the right one again, so only the middle call would
; be wrong - which is why this asks for the same block twice rather than once.
interleave:
CALL clearChecksum
CALL readFirstBig
CALL keepChecksum
SETD.0 SmallName
SETD.1 Block
RSTA
RSTB
SWI osFileBlock
CALL clearChecksum
CALL readFirstBig
CALL sameAsKept
BNQ mixedUp
SETD.0 InterleaveOk
SWI osPrintString
BRI moved
mixedUp:
SETD.0 InterleaveBad
SWI osPrintString
; ---- 5. A file that moves out from under the name ----
;
; The system has just been asked about the small file, so it is the one being remembered.
; Renaming it has to throw that away: the blocks are still there and still hold the same
; bytes, so a stale answer would work perfectly and be wrong.
moved:
SETD.0 SmallName
SETD.1 OtherName
SWI osFileRename
BNQ noRename
SETD.0 MovedText
SWI osPrintString
SETD.0 SmallName
SWI osFileInfo
CALL keepWhy
SETD.0 OldNameText
SWI osPrintString
CALL printWhy
SETD.0 NewNameText
SWI osPrintString
SETD.0 OtherName
SWI osFileInfo
CALL keepWhy
CALL printWhy
; ---- 6. The three ways of being told no ----
missing:
SETD.0 MissingName
SWI osFileInfo
CALL keepWhy
SETD.0 MissingText
SWI osPrintString
CALL printWhy
SETD.0 OtherName
SETD.1 Block
INIA 0xFF
INIB 0xFF
SWI osFileBlock
CALL keepWhy
SETD.0 PastText
SWI osPrintString
CALL printWhy
SWI osExit
noBig:
CALL keepWhy
SETD.0 NoBigText
SWI osPrintString
CALL printWhy
SWI osExit
noSmall:
SETD.0 NoSmallText
SWI osPrintString
SWI osExit
noRename:
SETD.0 NoRenameText
SWI osPrintString
SWI osExit
; ---- Routines ----
; Block zero of the big file, into the running checksum.
readFirstBig:
SETD.0 BigName
SETD.1 Block
RSTA
RSTB
SWI osFileBlock
BNQ readFirstDone
CALL takeCount
SETD.1 Block
CALL checksum
readFirstDone:
RET
; What the service just answered in DP3 becomes Left, which is what the checksum counts
; down. Kept in memory rather than in a pointer because a CALL does not preserve one.
takeCount:
PSHD.3
POPB
POPA
SETD.2 Left
STA.2
INCD.2
STB.2
RET
; Adds the bytes at DP1 into the running checksum, as many of them as Left says.
;
; Two accumulators, each a byte wide, each throwing away what carries off the top. The
; first is the sum of the bytes and the second is the sum of the first, so a byte that
; arrives late counts for less than one that arrived early - which is what makes this
; notice a block delivered out of turn.
checksum:
checksumLoop:
LDA.1
SETD.2 Fletch1
LDB.2
CCF
ADD
MVQA
STA.2
SETD.2 Fletch2
LDB.2
CCF
ADD
MVQA
STA.2
INCD.1
; Left goes down by one, sixteen bits of it: a whole block is 256 bytes and a whole file
; is more than one block, so a byte counter would not reach.
SETD.2 Left
INCD.2
LDA.2
BNA checksumLow
DPDN.2 0d01
LDA.2
DECA
STA.2 ; Borrow out of the high byte.
DPUP.2 0d01
INIA 0xFF
STA.2
BRI checksumTest
checksumLow:
DECA
STA.2
checksumTest:
SETD.2 Left
LDA.2
INCD.2
LDB.2
OR ; Zero only when both halves are.
BNQ checksumLoop
RET
clearChecksum:
RSTA
SETD.2 Fletch1
STA.2
SETD.2 Fletch2
STA.2
RET
clearIndex:
RSTA
SETD.2 Index
STA.2
INCD.2
STA.2
RET
stepIndex:
SETD.2 Index
INCD.2
LDA.2
INCA
STA.2
BNC stepIndexDone
DPDN.2 0d01
LDA.2
INCA
STA.2
stepIndexDone:
RET
; Puts the checksum aside so that a second one can be compared with it.
keepChecksum:
SETD.2 Fletch1
LDA.2
SETD.2 Kept1
STA.2
SETD.2 Fletch2
LDA.2
SETD.2 Kept2
STA.2
RET
; Q is zero if the running checksum is the one that was put aside.
sameAsKept:
SETD.2 Fletch1
LDA.2
SETD.2 Kept1
LDB.2
XOR
BNQ sameAsKeptDone
SETD.2 Fletch2
LDA.2
SETD.2 Kept2
LDB.2
XOR
sameAsKeptDone:
RET
printChecksum:
SETD.2 Fletch1
LDA.2
SETD.2 Fletch2
LDB.2
SWI osPrintNumber
RET
printKept:
SETD.2 Kept1
LDA.2
SETD.2 Kept2
LDB.2
SWI osPrintNumber
RET
; Why the last service said no. Q survives a CALL, which is the only reason this can be a
; routine at all, but it does not survive the next SWI - so it is written down here and
; printed later, with whatever has to happen in between happening in between.
keepWhy:
MVQA
SETD.2 Why
STA.2
RET
printWhy:
RSTA
SETD.2 Why
LDB.2
SWI osPrintNumber
SETD.0 NewLine
SWI osPrintString
RET
#Data
#Base 0x1000
BigName:
"big.txt"
SmallName:
"small.txt"
OtherName:
"moved.txt"
MissingName:
"nothing.txt"
BigIs:
"big.txt is "
BlocksText:
" blocks
"
ReadText:
"read "
BlocksSumText:
" blocks, checksum "
StoppedText:
", stopped with "
BothText:
"small.txt streamed is "
AgainstText:
", read whole is "
SameText:
"the same
"
DifferText:
"DIFFERENT
"
InterleaveOk:
"the same block twice with another file between: the same
"
InterleaveBad:
"the same block twice with another file between: DIFFERENT
"
MovedText:
"renamed small.txt
"
OldNameText:
"the old name now answers "
NewNameText:
"the new name answers "
MissingText:
"a name that was never there answers "
PastText:
"a block past the end answers "
NoBigText:
"big.txt would not open, answer "
NoSmallText:
"small.txt would not read
"
NoRenameText:
"it would not rename
"
NewLine:
"
"
Index:
0x00 0x00
Left:
0x00 0x00
Fletch1:
0x00
Fletch2:
0x00
Kept1:
0x00
Kept2:
0x00
Why:
0x00
; One block, which is the whole point: the big file is four hundred times this.
Block:
#Reserve 0d256
; And room for the small one all at once, so that the two ways of reading it can be
; compared against each other.
Whole:
#Reserve 0d1024
+11
View File
@@ -180,6 +180,17 @@ the services CosmOS provides. The currently installed services are:
| `osFileDelete` | DP0 names a file to remove; Q reports success. |
| `osFileRename` | DP0 names an existing file and DP1 its new name; Q reports success. |
| `osPrintNumber` | A with B give a number to print in decimal without leading zeroes. |
| `osBreak` | Stops the application, shows every register as it had them, waits for a key, and carries on. |
| `osFileInfo` | DP0 names a file; Q reports whether it is there and DP3 returns how many blocks it occupies. |
| `osFileBlock` | DP0 names a file, DP1 a destination, and A with B give which block; Q reports success and DP3 returns how many of the block's bytes belong to the file. |
`osFileInfo` and `osFileBlock` are how an application reads a file too big to hold. A whole
file arrives through `osFileRead`, which cannot help with anything above 64K, and CosmOS's
own source is above it. Neither call keeps anything open: each one names the file and says
which block it wants, so an application that stops halfway leaves nothing behind. Both
report why they failed rather than only that they did - 1 for no disk, 2 for no such file,
3 for a block past the end, and 4 for a disk that would not read - because running off the
end is how a reader learns it has finished.
The filesystem services exist so that an application need not contain a second copy of the
filesystem in order to keep a file. There is deliberately no service to mount a disk: the
+262
View File
@@ -581,6 +581,8 @@ doDelete:
LDA.0
BRA deleteWhat
; What a name means on the disk is about to change, so the remembered file goes.
CALL fileForget
CALL sbfsDelete
BNQ deleteFailed
SETD.0 Deleted
@@ -620,6 +622,8 @@ doRename:
SETD.2 RenameFrom
LDD.0.2
; What a name means on the disk is about to change, so the remembered file goes.
CALL fileForget
CALL sbfsRename
BNQ renameFailed
SETD.0 Renamed
@@ -944,6 +948,8 @@ handleFileSave:
SETD.2 SbfsFileTail
STB.2
; What a name means on the disk is about to change, so the remembered file goes.
CALL fileForget
CALL sbfsSaveFile
MVQA
MVSD.2
@@ -964,6 +970,8 @@ handleFileDelete:
SETD.2 DiskReady
LDA.2
BRA serviceNoDisk
; What a name means on the disk is about to change, so the remembered file goes.
CALL fileForget
CALL sbfsDelete
MVQA
MVSD.2
@@ -976,6 +984,8 @@ handleFileRename:
SETD.2 DiskReady
LDA.2
BRA serviceNoDisk
; What a name means on the disk is about to change, so the remembered file goes.
CALL fileForget
CALL sbfsRename
MVQA
MVSD.2
@@ -990,6 +1000,225 @@ serviceNoDisk:
STA.2
RETI
; ---- Reading a file that will not fit ----
;
; A file bigger than Data Memory cannot be handed over whole, and CosmOS's own source is
; now that file, so these two are how anything reads one: ask how many blocks, then ask for
; each block in turn. There is no open and no close. A program that stops halfway through
; leaves nothing behind, because there was never anything to leave.
; Finds a file, or remembers that it already did. DP0 names it. Q is zero if it is there,
; and then SbfsFileStart, SbfsFileBlocks and SbfsFileTail describe it exactly the way
; sbfsFind leaves them - whether the search happened or not, which is the whole point.
fileLookup:
SETD.2 FileCacheValid
LDA.2
BRA fileLookupSearch
SETD.1 FileCacheName
CALL textSame
BNQ fileLookupSearch
; The same file as last time. The description still has to be put back, because anything
; that went to the disk in between - a directory listing, a program being loaded - left
; its own answer in those three.
SETD.0 SbfsFileStart
SETD.2 FileCacheStart
CALL sbfsSetWord
SETD.0 SbfsFileBlocks
SETD.2 FileCacheBlocks
CALL sbfsSetWord
SETD.0 FileCacheTail
LDA.0
SETD.0 SbfsFileTail
STA.0
RSTA
RSTB
CCF
ADD ; Q is zero: found.
RET
fileLookupSearch:
CALL sbfsFind
BNQ fileLookupMissing
; Remember it. DP0 still names the file: a CALL puts the pointers back, which is the one
; place that convention is a convenience rather than an obstacle.
SETD.1 FileCacheName
CALL sbfsKeepName
SETD.0 FileCacheStart
SETD.2 SbfsFileStart
CALL sbfsSetWord
SETD.0 FileCacheBlocks
SETD.2 SbfsFileBlocks
CALL sbfsSetWord
SETD.0 SbfsFileTail
LDA.0
SETD.0 FileCacheTail
STA.0
; Marked good last, so that a cache half filled is never a cache believed.
INIA 0d1
SETD.0 FileCacheValid
STA.0
RSTA
RSTB
CCF
ADD ; Q is zero: found.
RET
fileLookupMissing:
RET ; Q is not zero, and sbfsFind is what made it so.
; Throws the remembered file away. Everything that can change what a name means on the disk
; calls this before it does: a file saved over may have moved, a deleted one is gone, and a
; renamed one answers to something else. A remembered start block that survived any of
; those is a pointer at whatever took its place.
;
; Q is deliberately untouched, so this can be dropped into a handler without disturbing the
; answer that handler is in the middle of working out.
fileForget:
RSTA
SETD.0 FileCacheValid
STA.0
RET
; DP0 names it. Q is zero if it is there, and DP3 comes back holding how many blocks it
; occupies, counting a part one on the end.
;
; Blocks, not bytes, and that is forced rather than chosen: a file on a sixteen megabyte
; disk is up to twenty four bits long, which does not fit in a pointer. Blocks do, and the
; bytes in the last one come back from osFileBlock when the reader gets there.
handleFileInfo:
SETD.2 DiskReady
LDA.2
BRA fileInfoNoDisk
CALL fileLookup
BNQ fileInfoMissing
CALL sbfsFileExtent
SETD.2 SbfsWantBlocks
LDA.2
INCD.2
LDB.2
MVSD.2
DPUP.2 0d05 ; The saved DP3, high byte first.
STA.2
INCD.2
STB.2
MVSD.2
DPUP.2 0d02
RSTA
STA.2 ; And the saved Q: it is there.
RETI
fileInfoNoDisk:
MVSD.2
DPUP.2 0d02
INIA 0d1
STA.2
RETI
fileInfoMissing:
MVSD.2
DPUP.2 0d02
INIA 0d2
STA.2
RETI
; DP0 names it, DP1 says where to put it, and A and B together are which block, counting
; from zero. Q is zero if it read, and DP3 comes back holding how many of the block's bytes
; belong to the file.
handleFileBlock:
; Which block, before anything else, because finding out whether there is a disk needs A
; and there is nowhere else the number is written down.
SETD.2 SbfsIndex
STA.2
INCD.2
STB.2
SETD.2 DiskReady
LDA.2
BRA fileBlockNoDisk
CALL fileLookup
BNQ fileBlockMissing
; Running off the end is how a reader finds out it has finished, so it gets an answer of
; its own rather than being told the disk failed.
CALL sbfsFileExtent
SETD.0 SbfsIndex
SETD.2 SbfsWantBlocks
CALL sbfsCompareWord
BNC fileBlockPastEnd ; The Carry is set only when the index is the smaller.
CALL sbfsReadOne ; DP1 still says where. A CALL puts the pointers back.
BNQ fileBlockFailed
; How much of it is the file's. Every block but a short last one is a whole 256, and 256
; is why this answers in a pointer instead of a register.
SETD.2 SbfsFileTail
LDA.2
BRA fileBlockWhole ; Nothing partial on the end, so they are all whole.
SETD.0 SbfsIndex
SETD.2 SbfsFileBlocks
CALL sbfsCompareWord
BNQ fileBlockWhole ; Not the last one.
SETD.2 SbfsFileTail
LDB.2
RSTA
BRI fileBlockAnswer
fileBlockWhole:
INIA 0x01
RSTB
fileBlockAnswer:
MVSD.2
DPUP.2 0d05
STA.2
INCD.2
STB.2
MVSD.2
DPUP.2 0d02
RSTA
STA.2
RETI
fileBlockNoDisk:
MVSD.2
DPUP.2 0d02
INIA 0d1
STA.2
RETI
fileBlockMissing:
MVSD.2
DPUP.2 0d02
INIA 0d2
STA.2
RETI
fileBlockPastEnd:
MVSD.2
DPUP.2 0d02
INIA 0d3
STA.2
RETI
fileBlockFailed:
MVSD.2
DPUP.2 0d02
INIA 0d4
STA.2
RETI
; A breakpoint. Shows every register as the interrupted program had them, waits for a key,
; and returns as though nothing happened.
;
@@ -2465,6 +2694,37 @@ RunArgument:
PrintNumber:
0x00 0x00
; ---- Where the last file anybody asked about lives ----
;
; osFileBlock is handed a name every time it is called, because a stateless service has no
; handle to leak and nothing left open by a program that stops in the middle. Taken at its
; word that means searching the directory once per block, so reading a four hundred block
; file walks the directory four hundred times over to be told the same thing.
;
; So the last answer is kept. A call naming the same file as the one before it skips the
; search and puts these back where sbfs keeps them. MEASURED, on the 329 block file the
; streaming test reads: 7% of the whole run saved when the file is the first entry in the
; directory, 11% when it is the sixteenth. Modest, and worth having for the shape rather
; than the size - what it really removes is a cost that grows with how full the disk is,
; on the one operation that is repeated once per block. NOTHING IS EVER TRUSTED THAT WAS NOT
; PUT HERE BY A SEARCH: this is a copy of an answer, not a second place where the truth
; about a file is written, and every path that could make it wrong calls fileForget. That
; is what makes it a speed rather than a promise - a cache that has been thrown away is
; indistinguishable from one that was never filled.
;
; The name is twenty three bytes for a name of twenty two, so that a name filling the
; field still has a zero after it and can be compared as a string.
FileCacheValid:
0x00
FileCacheName:
#Reserve 0d23
FileCacheStart:
0x00 0x00
FileCacheBlocks:
0x00 0x00
FileCacheTail:
0x00
; ---- The vectors a loaded program brought with it ----
;
; Six bytes each: where it goes, what goes there, and what was there before. The last two
@@ -2633,6 +2893,8 @@ CommandLine:
osFileSave handleFileSave
osFileDelete handleFileDelete
osFileRename handleFileRename
osFileInfo handleFileInfo
osFileBlock handleFileBlock
osPrintNumber handlePrintNumber
osBreak handleBreak
Device 0x20 diskDone
+44
View File
@@ -473,6 +473,48 @@ sbfsReadDone:
ADD ; Q is zero: read.
RET
; Reads one block of the file sbfsFind found, the one SbfsIndex names counting from zero,
; into Data Memory at DP1. Q is zero if it worked.
;
; This is the whole of streaming: a file too big to hold is read a block at a time by
; asking for each in turn, and nothing has to be kept between the calls but the number.
; sbfsRead is what this would be if it were called in a loop, which is why the two look
; alike; it stays as it is because reading a whole small file is what most callers want
; and doing it in one call is both shorter and faster.
;
; The whole block comes across, including a last one that the file only partly fills, so
; the bytes past the end of it are whatever else was on the disk there. SbfsFileTail says
; where the file stops and it is the caller's business to respect it, the same bargain
; sbfsRead offers.
;
; An index past the end of the file is not caught here. It reads whatever block that
; works out to, which is somebody else's file or free space; the caller knows how many
; blocks there are, because sbfsFileExtent tells it.
sbfsReadOne:
PSHD.1
POPD.3 ; Where it goes. DP3 is the one pointer a CALL does not put back.
SETD.0 SbfsBlock
SETD.2 SbfsFileStart
CALL sbfsSetWord
SETD.0 SbfsBlock
SETD.2 SbfsIndex
CALL sbfsAddWord
CALL sbfsReadBlock
BNQ sbfsReadOneDone ; The read failed, and Q says so.
PSHD.3
POPD.1
CALL sbfsBufferOut
RSTA
RSTB
CCF
ADD ; Q is zero: read.
sbfsReadOneDone:
RET
; ---- Writing ----
; Where the first block that can hold a file is: past the superblock and the directory.
@@ -1239,6 +1281,8 @@ SbfsMatchLeft:
0x00
SbfsLeft:
0x00
SbfsIndex:
0x00 0x00
; ---- What a walk through the directory keeps between calls ----
+29
View File
@@ -45,6 +45,35 @@
osFileDelete 0d22 ; DP0 names it. Q is zero if it went.
osFileRename 0d23 ; DP0 is the name it has, DP1 the name it should have. Q is zero if it moved.
; ---- Reading a file that will not fit ----
;
; osFileRead answers with a whole file in Data Memory, which settles it for anything under
; 64K and settles nothing above. These two are the other way of asking: how big is it, and
; then give me one block of it at a time. Nothing is kept between the calls but the number
; of the block wanted, so there is no handle to open, none to close, and nothing left
; behind by a program that stops in the middle. The system remembers where the last file it
; was asked about lives, so asking for four hundred blocks of one file costs one search of
; the directory rather than four hundred; that is a speed, not a promise, and a caller
; never has to know about it.
;
; THESE TWO SAY WHY WHEN THE ANSWER IS NO, which the others do not. Everywhere else the
; only useful thing to do about a failure is to give up, so one value is enough. These
; exist to be asked questions with - is it there, is there any more of it - and the
; difference between "no disk", "no such file" and "that was the last block" is the answer
; rather than an excuse.
;
; 1 there is no disk
; 2 there is no file of that name
; 3 that block is past the end of the file (osFileBlock only)
; 4 the disk would not read it (osFileBlock only)
osFileInfo 0d26 ; DP0 names it. Q is zero if it is there, DP3 is how many blocks.
osFileBlock 0d27 ; DP0 names it, DP1 says where, A and B are which block from zero.
; Q is zero if it read, DP3 is how many of its bytes are the file's:
; a whole 0d256 except in a last block that is short. That count is
; why DP3 answers and not a register - 0d256 does not fit in a byte,
; and a count that lied about a full block would make every reader
; treat the end of a file as a special case.
; ---- And with the console ----
;
; printString is already up there. This is the other half of what a program prints: a
+1
View File
@@ -16,6 +16,7 @@ SplitBit is a custom 8 bit system designed for hobbyist projects and experimenta
- Loadable Programs: A program that was not booted from carries a header saying where it belongs, and Programs/loader.asm reads one off a disk, puts it there, and runs it.
- An Operating System: CosmOS boots the machine, mounts a disk, lists what is on it, loads a program and runs it, and takes the machine back when it finishes. It comes with a library of programs to run, including a game and a line editor that writes files a person typed.
- System Services: A loaded program reaches the console and the disk through numbered software interrupts rather than carrying a copy of the code that drives them. The numbers are written down in one file that both sides include, so neither ever types one. It took the editor from 4941 bytes to 1983 without changing a line of what it does.
- Streaming Reads: A file bigger than the machine's memory is read a block at a time, through services that keep nothing open between calls. CosmOS's own source is 104K against 64K of Data Memory, so this is what a self-hosted assembler will stand on.
- Storage: A block device with 256 byte blocks and 16 megabytes of them, backed by an image file on the host. It knows blocks and not files, because a filesystem is meant to be software SplitBit runs.
- Memory Controller: Reads and writes Program Memory, moves blocks between memory banks, reaches memory that devices bring with them, and guards a range against being written by accident. It is how a SplitBit machine loads a program.
- Assembler: Assemble human readable assembly language files directly into SplitBit compatible binary files. Supports including external files, handling labels, alignment and reservation, and defining Program, Data and Vector segments.
+46
View File
@@ -559,6 +559,8 @@ Those numbers are written down once, in `Programs/CosmOS/Source/services.asm`, w
| osFileSave | DP0 names a file, DP1 is the bytes, A and B together are how many. Q is zero if it saved, whether or not it was there before. |
| osFileDelete | DP0 names a file. Q is zero if it went. |
| osFileRename | DP0 is the name a file has, DP1 the name it should have. Q is zero if it moved. |
| osFileInfo | DP0 names a file. Q is zero if it is there, and DP3 comes back holding how many blocks it occupies. |
| osFileBlock | DP0 names a file, DP1 says where to put a block of it, A and B together are which block counting from zero. Q is zero if it read, and DP3 comes back holding how many of the block's bytes belong to the file. |
| osPrintNumber | A and B together are a number. Prints it in decimal, without leading zeroes. |
| osBreak | Stops the program, shows every register as it had them, waits for a key, and carries on. |
@@ -600,6 +602,49 @@ Sizes fit the registers exactly, in both directions. A file that can be read int
A file of 256 blocks or more is refused by `osFileRead` rather than partly read, because 64K will not fit in Data Memory and its length will not fit in the pointer that reports it. A length that lies would be worse than a file that will not open.
### Reading A File That Will Not Fit:
`osFileRead` hands over a whole file, which settles the question for anything under 64K and settles nothing above it. CosmOS's own source is above it: the sources together are a hundred kilobytes and Data Memory is sixty four. A machine that is one day going to assemble itself has to be able to read a file bigger than its memory.
So there is a second way to ask. `osFileInfo` says how big something is and `osFileBlock` hands over one block of it, and between them a program reads a file of any size through a buffer of 256 bytes.
```
SETD.0 Name
SWI osFileInfo ; DP3 is how many blocks, Q is zero if it is there.
BNQ noSuchFile
readLoop:
SETD.0 Name
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2 ; Which block, most significant first.
SWI osFileBlock
BNQ readDone ; Three when there are no more.
... ; DP3 is how many of its bytes are the file's.
```
**There is no open and no close.** Every call names the file and says which block it wants, so nothing is held between them: a program that stops halfway leaves nothing behind, and there is no handle to run out of. The system does remember where the last file it was asked about lives, so reading four hundred blocks searches the directory once rather than four hundred times — but that is a speed and not a promise, and a caller never has to know about it.
`osFileInfo` answers in **blocks rather than bytes**, and that is forced rather than chosen. A file on a sixteen megabyte disk can be twenty four bits long and a Data Pointer holds sixteen. Blocks fit; the bytes in the last one come back from `osFileBlock` when the reader gets there.
`osFileBlock` answers a count in DP3 rather than in a register for the same kind of reason: every block but a short last one holds a whole **256** bytes, and 256 does not fit in a byte. A count that reported a full block as zero would make every reader treat the end of a file as a special case.
**These two say why when the answer is no**, which the other services do not. Everywhere else the only useful thing to do about a failure is to give up, so one value is enough. These exist to be asked questions with, and the difference between the answers is the answer:
| Q | Means |
| --- | --- |
| 0 | it worked |
| 1 | there is no disk |
| 2 | there is no file of that name |
| 3 | that block is past the end of the file |
| 4 | the disk would not read it |
Running off the end is how a reader finds out it has finished, so it gets an answer of its own rather than being reported as a disk that failed.
`Programs/CosmOS/Apps/Stream.asm` reads an 84,000 byte file through a 256 byte buffer, then reads a small file both ways — whole with `osFileRead` and streamed — and checks that the two agree.
`Programs/CosmOS/Apps/Files.asm` does the whole round trip — write, read, report, rename, delete — in 645 bytes, and includes nothing but the service names.
`osArgument` is how a program is told what it is for. Everything written before it did the same thing however it was started, which is fine for a program that greets you and no use to one that edits a named document. What arrives is the whole rest of the line, spaces and all, rather than a list of words: what counts as an argument is the program's business, and handing over what was typed is the system's.
@@ -692,6 +737,7 @@ Whoever does the loading keeps its own code and data below the addresses the loa
| Files | Writes a file, reads it back, renames it and deletes it, in 645 bytes, including nothing but the service names. It is what says a program does not need a filesystem inside it. |
| Break | Stops itself twice with SWI osBreak, so that the registers can be seen changing between one stop and the next. |
| Edit | A line editor. |
| Stream | Reads an 84,000 byte file through a buffer of 256, which is what says a file bigger than Data Memory can be read at all. |
### The Monitor:
+2 -2
View File
@@ -3,11 +3,11 @@ CosmOS
> two stops, and what the registers were at each
break at 200E
A 11 B 22 Q 00 status 00
DP0 1030 DP1 0661 DP2 039A DP3 2000 SP FFFF
DP0 1030 DP1 0663 DP2 039C DP3 2000 SP FFFF
press a key
break at 2023
A 44 B 55 Q 00 status 00
DP0 1000 DP1 0661 DP2 039A DP3 2000 SP FFF5
DP0 1000 DP1 0663 DP2 039C DP3 2000 SP FFF5
press a key
carried on to the end
finished
+16
View File
@@ -0,0 +1,16 @@
CosmOS
> loaded, starting at 2000
> big.txt is 329 blocks
read 329 blocks, checksum 57368, stopped with 3
small.txt streamed is 15216, read whole is 15216
the same
the same block twice with another file between: the same
renamed small.txt
the old name now answers 2
the new name answers 0
a name that was never there answers 2
a block past the end answers 3
finished
> halted
Execution halted.
[exit 0]
+3
View File
@@ -0,0 +1,3 @@
load Stream.sbx
run
exit
+18
View File
@@ -138,3 +138,21 @@ printf 'the second one' > two.txt
"$ROOT/Assembler" -I "$ROOT/Programs/CosmOS/Source" \
"$ROOT/Programs/CosmOS/Apps/Files.asm" -o "$WORK/Files.sbx" >/dev/null
"$TOOL" put "$DISKS/services.img" "$WORK/Files.sbx" >/dev/null
# A disk for streaming, and the only one here that has to be big: the file on it is bigger
# than the machine's Data Memory, which is the entire point of the services it checks.
#
# THE FILE IS GENERATED RATHER THAN TAKEN FROM THE REPOSITORY. Concatenating the CosmOS
# sources would be a truer picture of what streaming is for, and would change the recorded
# checksum every time a line of CosmOS was edited - so a real difference would arrive in a
# crowd of meaningless ones, which is the same trap the cycle counts used to set. This is
# 84000 bytes, which is 328 whole blocks and 32 bytes over, so the short block at the end
# is exercised rather than assumed.
"$TOOL" format "$DISKS/stream.img" 1024 2 >/dev/null
awk 'BEGIN { for (i = 0; i < 4000; i++) printf "streaming line %05d\n", i }' > big.txt
awk 'BEGIN { for (i = 0; i < 50; i++) printf "small %05d\n", i }' > small.txt
"$TOOL" put "$DISKS/stream.img" big.txt >/dev/null
"$TOOL" put "$DISKS/stream.img" small.txt >/dev/null
"$ROOT/Assembler" -I "$ROOT/Programs/CosmOS/Source" \
"$ROOT/Programs/CosmOS/Apps/Stream.asm" -o "$WORK/Stream.sbx" >/dev/null
"$TOOL" put "$DISKS/stream.img" "$WORK/Stream.sbx" >/dev/null
+17
View File
@@ -300,6 +300,22 @@ cosmosEdit | CosmOS/Source/cosmos.asm | run | cosmosEdi
# with files and carries the filesystem inside it. That difference is the whole case for the
# service layer, and this is where it is checked rather than argued.
cosmosServices | CosmOS/Source/cosmos.asm | run | cosmosFiles2.in | - | disks/services.img
# Streaming, which is what a file bigger than memory needs. The disk here holds 84000
# bytes in one file and the machine has 64K of Data Memory, so there is no arrangement of
# osFileRead that could get at it: the program reads it through a buffer of one block.
#
# THE CHECK THAT MATTERS IS THE SECOND ONE. A small file is read both ways - whole with
# osFileRead, and streamed - and the two checksums have to agree, so streaming is measured
# against the path that already worked rather than against a number somebody wrote down.
# The checksum is Fletcher's rather than a sum, because a sum is the same whatever order
# the bytes arrived in and the order is exactly what streaming has to get right.
#
# The rest of it is the lookup the system keeps so that reading four hundred blocks of one
# file does not search the directory four hundred times. Two files read alternately catch a
# memory that does not notice the name changed, and a rename catches one that does not
# notice the file moved - and that one would otherwise pass, since the blocks are still
# there and still hold the same bytes.
cosmosStream | CosmOS/Source/cosmos.asm | run | cosmosStream.in | - | disks/stream.img
# The programs CosmOS loads, checked on their own so that a failure here reads as "the app
# does not assemble" rather than as a broken disk image.
app-greet | CosmOS/Apps/greet.asm | assemble | - | -
@@ -311,6 +327,7 @@ app-Say | CosmOS/Apps/Say.asm | assemble | -
app-Break | CosmOS/Apps/Break.asm | assemble | - | -
app-Edit | CosmOS/Apps/Edit.asm | assemble | - | -
app-Files | CosmOS/Apps/Files.asm | assemble | - | -
app-Stream | CosmOS/Apps/Stream.asm | assemble | - | -
# ---- Programs driven by console input ----
inputTest | inputTest.asm | run | inputTest.in | -