diff --git a/Programs/CosmOS/Apps/Stream.asm b/Programs/CosmOS/Apps/Stream.asm new file mode 100644 index 0000000..baf6ae6 --- /dev/null +++ b/Programs/CosmOS/Apps/Stream.asm @@ -0,0 +1,491 @@ +; Reading a file the machine cannot hold. +; +; Every other program here asks for a file and is handed the whole of it, which settles the +; question for anything under 64K and settles nothing above. CosmOS's own source is above: +; the sources together are a hundred kilobytes, and Data Memory is sixty four. A machine +; that is one day going to assemble itself has to be able to read a file bigger than its +; memory, and this is the program that proves it can. +; +; It uses osFileInfo and osFileBlock, and nothing else knows how a filesystem works. There +; is no open and no close - every call names the file and says which block it wants, so a +; program that stops halfway leaves nothing behind for anybody to clean up. +; +; ---- What it checks, and why each one is here ---- +; +; 1. A file of four hundred odd blocks is read from end to end, a block at a time, into a +; buffer of one block. That is the feature. +; 2. A small file is read BOTH WAYS - whole with osFileRead, and streamed - and the two +; have to agree. This is the real proof: it compares streaming against the path that +; was already known to work, so a fault in the block count or the order of the blocks +; shows up as a difference rather than as a plausible wrong answer. +; 3. Two files are read alternately. The system remembers where the last file it was +; asked about lives, and this is the case that catches a memory that does not notice +; the name has changed. +; 4. A rename in the middle. Same reason, from the other side: the file the system +; remembers has moved out from under the name it remembered it by. +; 5. The three ways of being told no, each with its own number. +; +; THE CHECKSUM IS FLETCHER'S, not a sum. A plain total is the same whatever order the bytes +; arrived in, and the order is exactly what streaming has to get right; carrying a second +; accumulator that adds the first one in each time makes a block delivered out of turn +; change the answer. +; +; Written by Anachronaut + +#Include services.asm + +#Program + + #Base 0x2000 + +start: + ; ---- 1. How big is something that will not fit ---- + ; + ; In blocks, not bytes, and that is forced rather than chosen: a file on a sixteen + ; megabyte disk can be twenty four bits long and a pointer holds sixteen. + SETD.0 BigName + SWI osFileInfo + BNQ noBig + SETD.0 BigIs + SWI osPrintString + PSHD.3 + POPB + POPA + SWI osPrintNumber + SETD.0 BlocksText + SWI osPrintString + + ; ---- 2. Read the whole of it through a hole one block wide ---- + CALL clearChecksum + CALL clearIndex + +bigLoop: + SETD.0 BigName + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 ; Which block, most significant first. + SWI osFileBlock + BNQ bigDone + + CALL takeCount + SETD.1 Block + CALL checksum + CALL stepIndex + BRI bigLoop + +bigDone: + ; The loop ends because a block past the end was asked for, which is answer three. Any + ; other answer stopped it early and would otherwise look exactly like success, so what + ; ended it is printed rather than assumed. + CALL keepWhy + SETD.0 ReadText + SWI osPrintString + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osPrintNumber + SETD.0 BlocksSumText + SWI osPrintString + CALL printChecksum + SETD.0 StoppedText + SWI osPrintString + CALL printWhy + + ; ---- 3. The same file both ways ---- + ; + ; osFileRead is the path that already worked, so it is what streaming is measured + ; against. If the two checksums agree, every byte arrived and they arrived in order. + SETD.0 SmallName + SETD.1 Whole + SWI osFileRead + BNQ noSmall + CALL takeCount + CALL clearChecksum + SETD.1 Whole + CALL checksum + CALL keepChecksum + + CALL clearChecksum + CALL clearIndex +smallLoop: + SETD.0 SmallName + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osFileBlock + BNQ smallDone + CALL takeCount + SETD.1 Block + CALL checksum + CALL stepIndex + BRI smallLoop + +smallDone: + SETD.0 BothText + SWI osPrintString + CALL printChecksum + SETD.0 AgainstText + SWI osPrintString + CALL printKept + SETD.0 NewLine + SWI osPrintString + CALL sameAsKept + BNQ differ + SETD.0 SameText + SWI osPrintString + BRI interleave +differ: + SETD.0 DifferText + SWI osPrintString + + ; ---- 4. Two files, alternately ---- + ; + ; Block zero of the big file, then a block of the small one, then block zero of the big + ; file again. The two readings of the same block have to match. A system that remembered + ; the first file and did not notice the name had changed would hand back a block of the + ; wrong file in the middle, and then the right one again, so only the middle call would + ; be wrong - which is why this asks for the same block twice rather than once. +interleave: + CALL clearChecksum + CALL readFirstBig + CALL keepChecksum + + SETD.0 SmallName + SETD.1 Block + RSTA + RSTB + SWI osFileBlock + + CALL clearChecksum + CALL readFirstBig + CALL sameAsKept + BNQ mixedUp + SETD.0 InterleaveOk + SWI osPrintString + BRI moved +mixedUp: + SETD.0 InterleaveBad + SWI osPrintString + + ; ---- 5. A file that moves out from under the name ---- + ; + ; The system has just been asked about the small file, so it is the one being remembered. + ; Renaming it has to throw that away: the blocks are still there and still hold the same + ; bytes, so a stale answer would work perfectly and be wrong. +moved: + SETD.0 SmallName + SETD.1 OtherName + SWI osFileRename + BNQ noRename + SETD.0 MovedText + SWI osPrintString + SETD.0 SmallName + SWI osFileInfo + CALL keepWhy + SETD.0 OldNameText + SWI osPrintString + CALL printWhy + SETD.0 NewNameText + SWI osPrintString + SETD.0 OtherName + SWI osFileInfo + CALL keepWhy + CALL printWhy + + ; ---- 6. The three ways of being told no ---- +missing: + SETD.0 MissingName + SWI osFileInfo + CALL keepWhy + SETD.0 MissingText + SWI osPrintString + CALL printWhy + + SETD.0 OtherName + SETD.1 Block + INIA 0xFF + INIB 0xFF + SWI osFileBlock + CALL keepWhy + SETD.0 PastText + SWI osPrintString + CALL printWhy + + SWI osExit + +noBig: + CALL keepWhy + SETD.0 NoBigText + SWI osPrintString + CALL printWhy + SWI osExit +noSmall: + SETD.0 NoSmallText + SWI osPrintString + SWI osExit +noRename: + SETD.0 NoRenameText + SWI osPrintString + SWI osExit + +; ---- Routines ---- + +; Block zero of the big file, into the running checksum. +readFirstBig: + SETD.0 BigName + SETD.1 Block + RSTA + RSTB + SWI osFileBlock + BNQ readFirstDone + CALL takeCount + SETD.1 Block + CALL checksum +readFirstDone: + RET + +; What the service just answered in DP3 becomes Left, which is what the checksum counts +; down. Kept in memory rather than in a pointer because a CALL does not preserve one. +takeCount: + PSHD.3 + POPB + POPA + SETD.2 Left + STA.2 + INCD.2 + STB.2 + RET + +; Adds the bytes at DP1 into the running checksum, as many of them as Left says. +; +; Two accumulators, each a byte wide, each throwing away what carries off the top. The +; first is the sum of the bytes and the second is the sum of the first, so a byte that +; arrives late counts for less than one that arrived early - which is what makes this +; notice a block delivered out of turn. +checksum: +checksumLoop: + LDA.1 + SETD.2 Fletch1 + LDB.2 + CCF + ADD + MVQA + STA.2 + SETD.2 Fletch2 + LDB.2 + CCF + ADD + MVQA + STA.2 + INCD.1 + + ; Left goes down by one, sixteen bits of it: a whole block is 256 bytes and a whole file + ; is more than one block, so a byte counter would not reach. + SETD.2 Left + INCD.2 + LDA.2 + BNA checksumLow + DPDN.2 0d01 + LDA.2 + DECA + STA.2 ; Borrow out of the high byte. + DPUP.2 0d01 + INIA 0xFF + STA.2 + BRI checksumTest +checksumLow: + DECA + STA.2 +checksumTest: + SETD.2 Left + LDA.2 + INCD.2 + LDB.2 + OR ; Zero only when both halves are. + BNQ checksumLoop + RET + +clearChecksum: + RSTA + SETD.2 Fletch1 + STA.2 + SETD.2 Fletch2 + STA.2 + RET + +clearIndex: + RSTA + SETD.2 Index + STA.2 + INCD.2 + STA.2 + RET + +stepIndex: + SETD.2 Index + INCD.2 + LDA.2 + INCA + STA.2 + BNC stepIndexDone + DPDN.2 0d01 + LDA.2 + INCA + STA.2 +stepIndexDone: + RET + +; Puts the checksum aside so that a second one can be compared with it. +keepChecksum: + SETD.2 Fletch1 + LDA.2 + SETD.2 Kept1 + STA.2 + SETD.2 Fletch2 + LDA.2 + SETD.2 Kept2 + STA.2 + RET + +; Q is zero if the running checksum is the one that was put aside. +sameAsKept: + SETD.2 Fletch1 + LDA.2 + SETD.2 Kept1 + LDB.2 + XOR + BNQ sameAsKeptDone + SETD.2 Fletch2 + LDA.2 + SETD.2 Kept2 + LDB.2 + XOR +sameAsKeptDone: + RET + +printChecksum: + SETD.2 Fletch1 + LDA.2 + SETD.2 Fletch2 + LDB.2 + SWI osPrintNumber + RET + +printKept: + SETD.2 Kept1 + LDA.2 + SETD.2 Kept2 + LDB.2 + SWI osPrintNumber + RET + +; Why the last service said no. Q survives a CALL, which is the only reason this can be a +; routine at all, but it does not survive the next SWI - so it is written down here and +; printed later, with whatever has to happen in between happening in between. +keepWhy: + MVQA + SETD.2 Why + STA.2 + RET + +printWhy: + RSTA + SETD.2 Why + LDB.2 + SWI osPrintNumber + SETD.0 NewLine + SWI osPrintString + RET + +#Data + + #Base 0x1000 + +BigName: +"big.txt" +SmallName: +"small.txt" +OtherName: +"moved.txt" +MissingName: +"nothing.txt" + +BigIs: +"big.txt is " +BlocksText: +" blocks +" +ReadText: +"read " +BlocksSumText: +" blocks, checksum " +StoppedText: +", stopped with " +BothText: +"small.txt streamed is " +AgainstText: +", read whole is " +SameText: +"the same +" +DifferText: +"DIFFERENT +" +InterleaveOk: +"the same block twice with another file between: the same +" +InterleaveBad: +"the same block twice with another file between: DIFFERENT +" +MovedText: +"renamed small.txt +" +OldNameText: +"the old name now answers " +NewNameText: +"the new name answers " +MissingText: +"a name that was never there answers " +PastText: +"a block past the end answers " + +NoBigText: +"big.txt would not open, answer " +NoSmallText: +"small.txt would not read +" +NoRenameText: +"it would not rename +" +NewLine: +" +" + +Index: + 0x00 0x00 +Left: + 0x00 0x00 +Fletch1: + 0x00 +Fletch2: + 0x00 +Kept1: + 0x00 +Kept2: + 0x00 +Why: + 0x00 + +; One block, which is the whole point: the big file is four hundred times this. +Block: + #Reserve 0d256 + +; And room for the small one all at once, so that the two ways of reading it can be +; compared against each other. +Whole: + #Reserve 0d1024 diff --git a/Programs/CosmOS/README.md b/Programs/CosmOS/README.md index 30dae23..10b30e7 100644 --- a/Programs/CosmOS/README.md +++ b/Programs/CosmOS/README.md @@ -180,6 +180,17 @@ the services CosmOS provides. The currently installed services are: | `osFileDelete` | DP0 names a file to remove; Q reports success. | | `osFileRename` | DP0 names an existing file and DP1 its new name; Q reports success. | | `osPrintNumber` | A with B give a number to print in decimal without leading zeroes. | +| `osBreak` | Stops the application, shows every register as it had them, waits for a key, and carries on. | +| `osFileInfo` | DP0 names a file; Q reports whether it is there and DP3 returns how many blocks it occupies. | +| `osFileBlock` | DP0 names a file, DP1 a destination, and A with B give which block; Q reports success and DP3 returns how many of the block's bytes belong to the file. | + +`osFileInfo` and `osFileBlock` are how an application reads a file too big to hold. A whole +file arrives through `osFileRead`, which cannot help with anything above 64K, and CosmOS's +own source is above it. Neither call keeps anything open: each one names the file and says +which block it wants, so an application that stops halfway leaves nothing behind. Both +report why they failed rather than only that they did - 1 for no disk, 2 for no such file, +3 for a block past the end, and 4 for a disk that would not read - because running off the +end is how a reader learns it has finished. The filesystem services exist so that an application need not contain a second copy of the filesystem in order to keep a file. There is deliberately no service to mount a disk: the diff --git a/Programs/CosmOS/Source/cosmos.asm b/Programs/CosmOS/Source/cosmos.asm index ae5ce57..700e780 100644 --- a/Programs/CosmOS/Source/cosmos.asm +++ b/Programs/CosmOS/Source/cosmos.asm @@ -581,6 +581,8 @@ doDelete: LDA.0 BRA deleteWhat + ; What a name means on the disk is about to change, so the remembered file goes. + CALL fileForget CALL sbfsDelete BNQ deleteFailed SETD.0 Deleted @@ -620,6 +622,8 @@ doRename: SETD.2 RenameFrom LDD.0.2 + ; What a name means on the disk is about to change, so the remembered file goes. + CALL fileForget CALL sbfsRename BNQ renameFailed SETD.0 Renamed @@ -944,6 +948,8 @@ handleFileSave: SETD.2 SbfsFileTail STB.2 + ; What a name means on the disk is about to change, so the remembered file goes. + CALL fileForget CALL sbfsSaveFile MVQA MVSD.2 @@ -964,6 +970,8 @@ handleFileDelete: SETD.2 DiskReady LDA.2 BRA serviceNoDisk + ; What a name means on the disk is about to change, so the remembered file goes. + CALL fileForget CALL sbfsDelete MVQA MVSD.2 @@ -976,6 +984,8 @@ handleFileRename: SETD.2 DiskReady LDA.2 BRA serviceNoDisk + ; What a name means on the disk is about to change, so the remembered file goes. + CALL fileForget CALL sbfsRename MVQA MVSD.2 @@ -990,6 +1000,225 @@ serviceNoDisk: STA.2 RETI +; ---- Reading a file that will not fit ---- +; +; A file bigger than Data Memory cannot be handed over whole, and CosmOS's own source is +; now that file, so these two are how anything reads one: ask how many blocks, then ask for +; each block in turn. There is no open and no close. A program that stops halfway through +; leaves nothing behind, because there was never anything to leave. + +; Finds a file, or remembers that it already did. DP0 names it. Q is zero if it is there, +; and then SbfsFileStart, SbfsFileBlocks and SbfsFileTail describe it exactly the way +; sbfsFind leaves them - whether the search happened or not, which is the whole point. +fileLookup: + SETD.2 FileCacheValid + LDA.2 + BRA fileLookupSearch + + SETD.1 FileCacheName + CALL textSame + BNQ fileLookupSearch + + ; The same file as last time. The description still has to be put back, because anything + ; that went to the disk in between - a directory listing, a program being loaded - left + ; its own answer in those three. + SETD.0 SbfsFileStart + SETD.2 FileCacheStart + CALL sbfsSetWord + SETD.0 SbfsFileBlocks + SETD.2 FileCacheBlocks + CALL sbfsSetWord + SETD.0 FileCacheTail + LDA.0 + SETD.0 SbfsFileTail + STA.0 + RSTA + RSTB + CCF + ADD ; Q is zero: found. + RET + +fileLookupSearch: + CALL sbfsFind + BNQ fileLookupMissing + + ; Remember it. DP0 still names the file: a CALL puts the pointers back, which is the one + ; place that convention is a convenience rather than an obstacle. + SETD.1 FileCacheName + CALL sbfsKeepName + SETD.0 FileCacheStart + SETD.2 SbfsFileStart + CALL sbfsSetWord + SETD.0 FileCacheBlocks + SETD.2 SbfsFileBlocks + CALL sbfsSetWord + SETD.0 SbfsFileTail + LDA.0 + SETD.0 FileCacheTail + STA.0 + + ; Marked good last, so that a cache half filled is never a cache believed. + INIA 0d1 + SETD.0 FileCacheValid + STA.0 + + RSTA + RSTB + CCF + ADD ; Q is zero: found. + RET + +fileLookupMissing: + RET ; Q is not zero, and sbfsFind is what made it so. + +; Throws the remembered file away. Everything that can change what a name means on the disk +; calls this before it does: a file saved over may have moved, a deleted one is gone, and a +; renamed one answers to something else. A remembered start block that survived any of +; those is a pointer at whatever took its place. +; +; Q is deliberately untouched, so this can be dropped into a handler without disturbing the +; answer that handler is in the middle of working out. +fileForget: + RSTA + SETD.0 FileCacheValid + STA.0 + RET + +; DP0 names it. Q is zero if it is there, and DP3 comes back holding how many blocks it +; occupies, counting a part one on the end. +; +; Blocks, not bytes, and that is forced rather than chosen: a file on a sixteen megabyte +; disk is up to twenty four bits long, which does not fit in a pointer. Blocks do, and the +; bytes in the last one come back from osFileBlock when the reader gets there. +handleFileInfo: + SETD.2 DiskReady + LDA.2 + BRA fileInfoNoDisk + + CALL fileLookup + BNQ fileInfoMissing + + CALL sbfsFileExtent + SETD.2 SbfsWantBlocks + LDA.2 + INCD.2 + LDB.2 + + MVSD.2 + DPUP.2 0d05 ; The saved DP3, high byte first. + STA.2 + INCD.2 + STB.2 + + MVSD.2 + DPUP.2 0d02 + RSTA + STA.2 ; And the saved Q: it is there. + RETI + +fileInfoNoDisk: + MVSD.2 + DPUP.2 0d02 + INIA 0d1 + STA.2 + RETI + +fileInfoMissing: + MVSD.2 + DPUP.2 0d02 + INIA 0d2 + STA.2 + RETI + +; DP0 names it, DP1 says where to put it, and A and B together are which block, counting +; from zero. Q is zero if it read, and DP3 comes back holding how many of the block's bytes +; belong to the file. +handleFileBlock: + ; Which block, before anything else, because finding out whether there is a disk needs A + ; and there is nowhere else the number is written down. + SETD.2 SbfsIndex + STA.2 + INCD.2 + STB.2 + + SETD.2 DiskReady + LDA.2 + BRA fileBlockNoDisk + + CALL fileLookup + BNQ fileBlockMissing + + ; Running off the end is how a reader finds out it has finished, so it gets an answer of + ; its own rather than being told the disk failed. + CALL sbfsFileExtent + SETD.0 SbfsIndex + SETD.2 SbfsWantBlocks + CALL sbfsCompareWord + BNC fileBlockPastEnd ; The Carry is set only when the index is the smaller. + + CALL sbfsReadOne ; DP1 still says where. A CALL puts the pointers back. + BNQ fileBlockFailed + + ; How much of it is the file's. Every block but a short last one is a whole 256, and 256 + ; is why this answers in a pointer instead of a register. + SETD.2 SbfsFileTail + LDA.2 + BRA fileBlockWhole ; Nothing partial on the end, so they are all whole. + SETD.0 SbfsIndex + SETD.2 SbfsFileBlocks + CALL sbfsCompareWord + BNQ fileBlockWhole ; Not the last one. + + SETD.2 SbfsFileTail + LDB.2 + RSTA + BRI fileBlockAnswer + +fileBlockWhole: + INIA 0x01 + RSTB + +fileBlockAnswer: + MVSD.2 + DPUP.2 0d05 + STA.2 + INCD.2 + STB.2 + + MVSD.2 + DPUP.2 0d02 + RSTA + STA.2 + RETI + +fileBlockNoDisk: + MVSD.2 + DPUP.2 0d02 + INIA 0d1 + STA.2 + RETI + +fileBlockMissing: + MVSD.2 + DPUP.2 0d02 + INIA 0d2 + STA.2 + RETI + +fileBlockPastEnd: + MVSD.2 + DPUP.2 0d02 + INIA 0d3 + STA.2 + RETI + +fileBlockFailed: + MVSD.2 + DPUP.2 0d02 + INIA 0d4 + STA.2 + RETI + ; A breakpoint. Shows every register as the interrupted program had them, waits for a key, ; and returns as though nothing happened. ; @@ -2465,6 +2694,37 @@ RunArgument: PrintNumber: 0x00 0x00 +; ---- Where the last file anybody asked about lives ---- +; +; osFileBlock is handed a name every time it is called, because a stateless service has no +; handle to leak and nothing left open by a program that stops in the middle. Taken at its +; word that means searching the directory once per block, so reading a four hundred block +; file walks the directory four hundred times over to be told the same thing. +; +; So the last answer is kept. A call naming the same file as the one before it skips the +; search and puts these back where sbfs keeps them. MEASURED, on the 329 block file the +; streaming test reads: 7% of the whole run saved when the file is the first entry in the +; directory, 11% when it is the sixteenth. Modest, and worth having for the shape rather +; than the size - what it really removes is a cost that grows with how full the disk is, +; on the one operation that is repeated once per block. NOTHING IS EVER TRUSTED THAT WAS NOT +; PUT HERE BY A SEARCH: this is a copy of an answer, not a second place where the truth +; about a file is written, and every path that could make it wrong calls fileForget. That +; is what makes it a speed rather than a promise - a cache that has been thrown away is +; indistinguishable from one that was never filled. +; +; The name is twenty three bytes for a name of twenty two, so that a name filling the +; field still has a zero after it and can be compared as a string. +FileCacheValid: + 0x00 +FileCacheName: + #Reserve 0d23 +FileCacheStart: + 0x00 0x00 +FileCacheBlocks: + 0x00 0x00 +FileCacheTail: + 0x00 + ; ---- The vectors a loaded program brought with it ---- ; ; Six bytes each: where it goes, what goes there, and what was there before. The last two @@ -2633,6 +2893,8 @@ CommandLine: osFileSave handleFileSave osFileDelete handleFileDelete osFileRename handleFileRename + osFileInfo handleFileInfo + osFileBlock handleFileBlock osPrintNumber handlePrintNumber osBreak handleBreak Device 0x20 diskDone diff --git a/Programs/CosmOS/Source/sbfs.asm b/Programs/CosmOS/Source/sbfs.asm index 05a49a2..ddf23aa 100644 --- a/Programs/CosmOS/Source/sbfs.asm +++ b/Programs/CosmOS/Source/sbfs.asm @@ -473,6 +473,48 @@ sbfsReadDone: ADD ; Q is zero: read. RET +; Reads one block of the file sbfsFind found, the one SbfsIndex names counting from zero, +; into Data Memory at DP1. Q is zero if it worked. +; +; This is the whole of streaming: a file too big to hold is read a block at a time by +; asking for each in turn, and nothing has to be kept between the calls but the number. +; sbfsRead is what this would be if it were called in a loop, which is why the two look +; alike; it stays as it is because reading a whole small file is what most callers want +; and doing it in one call is both shorter and faster. +; +; The whole block comes across, including a last one that the file only partly fills, so +; the bytes past the end of it are whatever else was on the disk there. SbfsFileTail says +; where the file stops and it is the caller's business to respect it, the same bargain +; sbfsRead offers. +; +; An index past the end of the file is not caught here. It reads whatever block that +; works out to, which is somebody else's file or free space; the caller knows how many +; blocks there are, because sbfsFileExtent tells it. +sbfsReadOne: + PSHD.1 + POPD.3 ; Where it goes. DP3 is the one pointer a CALL does not put back. + + SETD.0 SbfsBlock + SETD.2 SbfsFileStart + CALL sbfsSetWord + SETD.0 SbfsBlock + SETD.2 SbfsIndex + CALL sbfsAddWord + + CALL sbfsReadBlock + BNQ sbfsReadOneDone ; The read failed, and Q says so. + + PSHD.3 + POPD.1 + CALL sbfsBufferOut + + RSTA + RSTB + CCF + ADD ; Q is zero: read. +sbfsReadOneDone: + RET + ; ---- Writing ---- ; Where the first block that can hold a file is: past the superblock and the directory. @@ -1239,6 +1281,8 @@ SbfsMatchLeft: 0x00 SbfsLeft: 0x00 +SbfsIndex: + 0x00 0x00 ; ---- What a walk through the directory keeps between calls ---- diff --git a/Programs/CosmOS/Source/services.asm b/Programs/CosmOS/Source/services.asm index 1239f6b..42e059f 100644 --- a/Programs/CosmOS/Source/services.asm +++ b/Programs/CosmOS/Source/services.asm @@ -45,6 +45,35 @@ osFileDelete 0d22 ; DP0 names it. Q is zero if it went. osFileRename 0d23 ; DP0 is the name it has, DP1 the name it should have. Q is zero if it moved. +; ---- Reading a file that will not fit ---- +; +; osFileRead answers with a whole file in Data Memory, which settles it for anything under +; 64K and settles nothing above. These two are the other way of asking: how big is it, and +; then give me one block of it at a time. Nothing is kept between the calls but the number +; of the block wanted, so there is no handle to open, none to close, and nothing left +; behind by a program that stops in the middle. The system remembers where the last file it +; was asked about lives, so asking for four hundred blocks of one file costs one search of +; the directory rather than four hundred; that is a speed, not a promise, and a caller +; never has to know about it. +; +; THESE TWO SAY WHY WHEN THE ANSWER IS NO, which the others do not. Everywhere else the +; only useful thing to do about a failure is to give up, so one value is enough. These +; exist to be asked questions with - is it there, is there any more of it - and the +; difference between "no disk", "no such file" and "that was the last block" is the answer +; rather than an excuse. +; +; 1 there is no disk +; 2 there is no file of that name +; 3 that block is past the end of the file (osFileBlock only) +; 4 the disk would not read it (osFileBlock only) + osFileInfo 0d26 ; DP0 names it. Q is zero if it is there, DP3 is how many blocks. + osFileBlock 0d27 ; DP0 names it, DP1 says where, A and B are which block from zero. + ; Q is zero if it read, DP3 is how many of its bytes are the file's: + ; a whole 0d256 except in a last block that is short. That count is + ; why DP3 answers and not a register - 0d256 does not fit in a byte, + ; and a count that lied about a full block would make every reader + ; treat the end of a file as a special case. + ; ---- And with the console ---- ; ; printString is already up there. This is the other half of what a program prints: a diff --git a/README.md b/README.md index e69a800..8991662 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ SplitBit is a custom 8 bit system designed for hobbyist projects and experimenta - Loadable Programs: A program that was not booted from carries a header saying where it belongs, and Programs/loader.asm reads one off a disk, puts it there, and runs it. - An Operating System: CosmOS boots the machine, mounts a disk, lists what is on it, loads a program and runs it, and takes the machine back when it finishes. It comes with a library of programs to run, including a game and a line editor that writes files a person typed. - System Services: A loaded program reaches the console and the disk through numbered software interrupts rather than carrying a copy of the code that drives them. The numbers are written down in one file that both sides include, so neither ever types one. It took the editor from 4941 bytes to 1983 without changing a line of what it does. +- Streaming Reads: A file bigger than the machine's memory is read a block at a time, through services that keep nothing open between calls. CosmOS's own source is 104K against 64K of Data Memory, so this is what a self-hosted assembler will stand on. - Storage: A block device with 256 byte blocks and 16 megabytes of them, backed by an image file on the host. It knows blocks and not files, because a filesystem is meant to be software SplitBit runs. - Memory Controller: Reads and writes Program Memory, moves blocks between memory banks, reaches memory that devices bring with them, and guards a range against being written by accident. It is how a SplitBit machine loads a program. - Assembler: Assemble human readable assembly language files directly into SplitBit compatible binary files. Supports including external files, handling labels, alignment and reservation, and defining Program, Data and Vector segments. diff --git a/SplitBit Programming Manual.md b/SplitBit Programming Manual.md index f61e03e..a7c96cd 100644 --- a/SplitBit Programming Manual.md +++ b/SplitBit Programming Manual.md @@ -559,6 +559,8 @@ Those numbers are written down once, in `Programs/CosmOS/Source/services.asm`, w | osFileSave | DP0 names a file, DP1 is the bytes, A and B together are how many. Q is zero if it saved, whether or not it was there before. | | osFileDelete | DP0 names a file. Q is zero if it went. | | osFileRename | DP0 is the name a file has, DP1 the name it should have. Q is zero if it moved. | +| osFileInfo | DP0 names a file. Q is zero if it is there, and DP3 comes back holding how many blocks it occupies. | +| osFileBlock | DP0 names a file, DP1 says where to put a block of it, A and B together are which block counting from zero. Q is zero if it read, and DP3 comes back holding how many of the block's bytes belong to the file. | | osPrintNumber | A and B together are a number. Prints it in decimal, without leading zeroes. | | osBreak | Stops the program, shows every register as it had them, waits for a key, and carries on. | @@ -600,6 +602,49 @@ Sizes fit the registers exactly, in both directions. A file that can be read int A file of 256 blocks or more is refused by `osFileRead` rather than partly read, because 64K will not fit in Data Memory and its length will not fit in the pointer that reports it. A length that lies would be worse than a file that will not open. +### Reading A File That Will Not Fit: + +`osFileRead` hands over a whole file, which settles the question for anything under 64K and settles nothing above it. CosmOS's own source is above it: the sources together are a hundred kilobytes and Data Memory is sixty four. A machine that is one day going to assemble itself has to be able to read a file bigger than its memory. + +So there is a second way to ask. `osFileInfo` says how big something is and `osFileBlock` hands over one block of it, and between them a program reads a file of any size through a buffer of 256 bytes. + +``` + SETD.0 Name + SWI osFileInfo ; DP3 is how many blocks, Q is zero if it is there. + BNQ noSuchFile + +readLoop: + SETD.0 Name + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 ; Which block, most significant first. + SWI osFileBlock + BNQ readDone ; Three when there are no more. + ... ; DP3 is how many of its bytes are the file's. +``` + +**There is no open and no close.** Every call names the file and says which block it wants, so nothing is held between them: a program that stops halfway leaves nothing behind, and there is no handle to run out of. The system does remember where the last file it was asked about lives, so reading four hundred blocks searches the directory once rather than four hundred times — but that is a speed and not a promise, and a caller never has to know about it. + +`osFileInfo` answers in **blocks rather than bytes**, and that is forced rather than chosen. A file on a sixteen megabyte disk can be twenty four bits long and a Data Pointer holds sixteen. Blocks fit; the bytes in the last one come back from `osFileBlock` when the reader gets there. + +`osFileBlock` answers a count in DP3 rather than in a register for the same kind of reason: every block but a short last one holds a whole **256** bytes, and 256 does not fit in a byte. A count that reported a full block as zero would make every reader treat the end of a file as a special case. + +**These two say why when the answer is no**, which the other services do not. Everywhere else the only useful thing to do about a failure is to give up, so one value is enough. These exist to be asked questions with, and the difference between the answers is the answer: + +| Q | Means | +| --- | --- | +| 0 | it worked | +| 1 | there is no disk | +| 2 | there is no file of that name | +| 3 | that block is past the end of the file | +| 4 | the disk would not read it | + +Running off the end is how a reader finds out it has finished, so it gets an answer of its own rather than being reported as a disk that failed. + +`Programs/CosmOS/Apps/Stream.asm` reads an 84,000 byte file through a 256 byte buffer, then reads a small file both ways — whole with `osFileRead` and streamed — and checks that the two agree. + `Programs/CosmOS/Apps/Files.asm` does the whole round trip — write, read, report, rename, delete — in 645 bytes, and includes nothing but the service names. `osArgument` is how a program is told what it is for. Everything written before it did the same thing however it was started, which is fine for a program that greets you and no use to one that edits a named document. What arrives is the whole rest of the line, spaces and all, rather than a list of words: what counts as an argument is the program's business, and handing over what was typed is the system's. @@ -692,6 +737,7 @@ Whoever does the loading keeps its own code and data below the addresses the loa | Files | Writes a file, reads it back, renames it and deletes it, in 645 bytes, including nothing but the service names. It is what says a program does not need a filesystem inside it. | | Break | Stops itself twice with SWI osBreak, so that the registers can be seen changing between one stop and the next. | | Edit | A line editor. | +| Stream | Reads an 84,000 byte file through a buffer of 256, which is what says a file bigger than Data Memory can be read at all. | ### The Monitor: diff --git a/Tests/expected/cosmosBreak.out b/Tests/expected/cosmosBreak.out index b1987ee..1d0bf2e 100644 --- a/Tests/expected/cosmosBreak.out +++ b/Tests/expected/cosmosBreak.out @@ -3,11 +3,11 @@ CosmOS > two stops, and what the registers were at each break at 200E A 11 B 22 Q 00 status 00 -DP0 1030 DP1 0661 DP2 039A DP3 2000 SP FFFF +DP0 1030 DP1 0663 DP2 039C DP3 2000 SP FFFF press a key break at 2023 A 44 B 55 Q 00 status 00 -DP0 1000 DP1 0661 DP2 039A DP3 2000 SP FFF5 +DP0 1000 DP1 0663 DP2 039C DP3 2000 SP FFF5 press a key carried on to the end finished diff --git a/Tests/expected/cosmosStream.out b/Tests/expected/cosmosStream.out new file mode 100644 index 0000000..7a14b9e --- /dev/null +++ b/Tests/expected/cosmosStream.out @@ -0,0 +1,16 @@ +CosmOS +> loaded, starting at 2000 +> big.txt is 329 blocks +read 329 blocks, checksum 57368, stopped with 3 +small.txt streamed is 15216, read whole is 15216 +the same +the same block twice with another file between: the same +renamed small.txt +the old name now answers 2 +the new name answers 0 +a name that was never there answers 2 +a block past the end answers 3 +finished +> halted +Execution halted. +[exit 0] diff --git a/Tests/input/cosmosStream.in b/Tests/input/cosmosStream.in new file mode 100644 index 0000000..9cc2f5b --- /dev/null +++ b/Tests/input/cosmosStream.in @@ -0,0 +1,3 @@ +load Stream.sbx +run +exit diff --git a/Tests/makedisks.sh b/Tests/makedisks.sh index ed5baa8..5ee3542 100755 --- a/Tests/makedisks.sh +++ b/Tests/makedisks.sh @@ -138,3 +138,21 @@ printf 'the second one' > two.txt "$ROOT/Assembler" -I "$ROOT/Programs/CosmOS/Source" \ "$ROOT/Programs/CosmOS/Apps/Files.asm" -o "$WORK/Files.sbx" >/dev/null "$TOOL" put "$DISKS/services.img" "$WORK/Files.sbx" >/dev/null + +# A disk for streaming, and the only one here that has to be big: the file on it is bigger +# than the machine's Data Memory, which is the entire point of the services it checks. +# +# THE FILE IS GENERATED RATHER THAN TAKEN FROM THE REPOSITORY. Concatenating the CosmOS +# sources would be a truer picture of what streaming is for, and would change the recorded +# checksum every time a line of CosmOS was edited - so a real difference would arrive in a +# crowd of meaningless ones, which is the same trap the cycle counts used to set. This is +# 84000 bytes, which is 328 whole blocks and 32 bytes over, so the short block at the end +# is exercised rather than assumed. +"$TOOL" format "$DISKS/stream.img" 1024 2 >/dev/null +awk 'BEGIN { for (i = 0; i < 4000; i++) printf "streaming line %05d\n", i }' > big.txt +awk 'BEGIN { for (i = 0; i < 50; i++) printf "small %05d\n", i }' > small.txt +"$TOOL" put "$DISKS/stream.img" big.txt >/dev/null +"$TOOL" put "$DISKS/stream.img" small.txt >/dev/null +"$ROOT/Assembler" -I "$ROOT/Programs/CosmOS/Source" \ + "$ROOT/Programs/CosmOS/Apps/Stream.asm" -o "$WORK/Stream.sbx" >/dev/null +"$TOOL" put "$DISKS/stream.img" "$WORK/Stream.sbx" >/dev/null diff --git a/Tests/manifest b/Tests/manifest index 8a1e673..0d0e968 100644 --- a/Tests/manifest +++ b/Tests/manifest @@ -300,6 +300,22 @@ cosmosEdit | CosmOS/Source/cosmos.asm | run | cosmosEdi # with files and carries the filesystem inside it. That difference is the whole case for the # service layer, and this is where it is checked rather than argued. cosmosServices | CosmOS/Source/cosmos.asm | run | cosmosFiles2.in | - | disks/services.img +# Streaming, which is what a file bigger than memory needs. The disk here holds 84000 +# bytes in one file and the machine has 64K of Data Memory, so there is no arrangement of +# osFileRead that could get at it: the program reads it through a buffer of one block. +# +# THE CHECK THAT MATTERS IS THE SECOND ONE. A small file is read both ways - whole with +# osFileRead, and streamed - and the two checksums have to agree, so streaming is measured +# against the path that already worked rather than against a number somebody wrote down. +# The checksum is Fletcher's rather than a sum, because a sum is the same whatever order +# the bytes arrived in and the order is exactly what streaming has to get right. +# +# The rest of it is the lookup the system keeps so that reading four hundred blocks of one +# file does not search the directory four hundred times. Two files read alternately catch a +# memory that does not notice the name changed, and a rename catches one that does not +# notice the file moved - and that one would otherwise pass, since the blocks are still +# there and still hold the same bytes. +cosmosStream | CosmOS/Source/cosmos.asm | run | cosmosStream.in | - | disks/stream.img # The programs CosmOS loads, checked on their own so that a failure here reads as "the app # does not assemble" rather than as a broken disk image. app-greet | CosmOS/Apps/greet.asm | assemble | - | - @@ -311,6 +327,7 @@ app-Say | CosmOS/Apps/Say.asm | assemble | - app-Break | CosmOS/Apps/Break.asm | assemble | - | - app-Edit | CosmOS/Apps/Edit.asm | assemble | - | - app-Files | CosmOS/Apps/Files.asm | assemble | - | - +app-Stream | CosmOS/Apps/Stream.asm | assemble | - | - # ---- Programs driven by console input ---- inputTest | inputTest.asm | run | inputTest.in | -