Streaming: read a file bigger than the machine's memory
osFileRead hands over a whole file, which settles anything under 64K and settles nothing above it. CosmOS's own source is above it - the sources together are 104K against 64K of Data Memory - so a machine that is going to assemble itself needs another way to ask. osFileInfo (0d26) says how many blocks a file occupies. osFileBlock (0d27) hands over one of them and says how many of its bytes belong to the file. Between them a program reads a file of any size through a buffer of 256. Blocks rather than bytes from osFileInfo is forced, not chosen: a file on a sixteen megabyte disk is up to twenty four bits long and a pointer holds sixteen. osFileBlock's count answers in DP3 for the same kind of reason - a whole block is 256 bytes, which does not fit in a register, and a count that reported it as zero would make every reader special-case the end. Nothing is kept open. Every call names the file, so there is no handle to leak and nothing left behind by a program that stops halfway. Taken at its word that means searching the directory once per block, so the system remembers where the last file it was asked about lives; every path that can change what a name means calls fileForget, including the shell's own delete and rename, which do not go through the services. Correctness never depends on the cache - a cache thrown away is indistinguishable from one never filled. Measured on a 329 block file: 7% of the run saved when the file is the first directory entry, 11% when it is the sixteenth. These two say WHY when the answer is no, which the others do not. Elsewhere the only useful response to a failure is to give up, so one value suffices. These are asked questions, and running off the end is how a reader learns it has finished, so it gets an answer of its own: 1 no disk, 2 no such file, 3 past the end, 4 the disk refused. Apps/Stream.asm reads an 84,000 byte file through 256 bytes. The check that matters is the second one: a small file read BOTH ways - whole with osFileRead and streamed - with the two checksums compared, so streaming is measured against the path already known to work rather than against a number someone wrote down. The checksum is Fletcher's rather than a sum, because a sum is the same whatever order the bytes arrived in and the order is exactly what streaming has to get right. Both checksums were also confirmed against the same arithmetic run on the host. The rest of the test is the cache: two files read alternately catch a memory that missed the name changing, and a rename catches one that missed the file moving - and that one would otherwise pass, since the blocks are still there holding the same bytes. The test file is generated rather than taken from the repository. The CosmOS sources would be a truer picture and would move the recorded checksum every time a line of CosmOS was edited, putting a real difference in a crowd of meaningless ones - the same trap the cycle counts used to set. cosmosBreak's recorded output moves by two bytes in two pointers: SbfsIndex added two bytes to the filesystem's data and Break prints the system addresses the registers happened to hold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
co-authored by
Claude Opus 5
parent
3b800a69e8
commit
3d2ab34229
@@ -300,6 +300,22 @@ cosmosEdit | CosmOS/Source/cosmos.asm | run | cosmosEdi
|
||||
# with files and carries the filesystem inside it. That difference is the whole case for the
|
||||
# service layer, and this is where it is checked rather than argued.
|
||||
cosmosServices | CosmOS/Source/cosmos.asm | run | cosmosFiles2.in | - | disks/services.img
|
||||
# Streaming, which is what a file bigger than memory needs. The disk here holds 84000
|
||||
# bytes in one file and the machine has 64K of Data Memory, so there is no arrangement of
|
||||
# osFileRead that could get at it: the program reads it through a buffer of one block.
|
||||
#
|
||||
# THE CHECK THAT MATTERS IS THE SECOND ONE. A small file is read both ways - whole with
|
||||
# osFileRead, and streamed - and the two checksums have to agree, so streaming is measured
|
||||
# against the path that already worked rather than against a number somebody wrote down.
|
||||
# The checksum is Fletcher's rather than a sum, because a sum is the same whatever order
|
||||
# the bytes arrived in and the order is exactly what streaming has to get right.
|
||||
#
|
||||
# The rest of it is the lookup the system keeps so that reading four hundred blocks of one
|
||||
# file does not search the directory four hundred times. Two files read alternately catch a
|
||||
# memory that does not notice the name changed, and a rename catches one that does not
|
||||
# notice the file moved - and that one would otherwise pass, since the blocks are still
|
||||
# there and still hold the same bytes.
|
||||
cosmosStream | CosmOS/Source/cosmos.asm | run | cosmosStream.in | - | disks/stream.img
|
||||
# The programs CosmOS loads, checked on their own so that a failure here reads as "the app
|
||||
# does not assemble" rather than as a broken disk image.
|
||||
app-greet | CosmOS/Apps/greet.asm | assemble | - | -
|
||||
@@ -311,6 +327,7 @@ app-Say | CosmOS/Apps/Say.asm | assemble | -
|
||||
app-Break | CosmOS/Apps/Break.asm | assemble | - | -
|
||||
app-Edit | CosmOS/Apps/Edit.asm | assemble | - | -
|
||||
app-Files | CosmOS/Apps/Files.asm | assemble | - | -
|
||||
app-Stream | CosmOS/Apps/Stream.asm | assemble | - | -
|
||||
|
||||
# ---- Programs driven by console input ----
|
||||
inputTest | inputTest.asm | run | inputTest.in | -
|
||||
|
||||
Reference in New Issue
Block a user