diff --git a/NOTES.md b/NOTES.md new file mode 100644 index 0000000..ec79fd5 --- /dev/null +++ b/NOTES.md @@ -0,0 +1,106 @@ +# Delete me after these things are fixed + +Temporary notes from the SBFS v2 and CosmOS review, in priority order. + +## Streaming writes: reject a final size larger than the reservation + +`osFileStart` reserves an extent and `osFileWrite` correctly refuses a block index outside +that extent. `sbfsStreamDone`, however, appears to accept a final block/tail count larger +than the extent reserved at the start. It writes the larger size into the descriptor and +then subtracts that size from the reservation when updating the free-block count. + +A caller could reserve one block, finish while claiming two, and make the file descriptor +claim a neighboring block that was never allocated and may belong to another file. + +Before deleting the old destination or modifying the temporary entry, compare the final +extent (`whole blocks + a possible tail block`) with the reserved extent. Refuse the commit +if the final extent is larger. Add tests for a final size that is smaller than, equal to, +and larger than the reservation, including tail/no-tail boundaries. + +## Protect the temporary-file namespace + +Whole-file saves and streamed writes use the fixed names `sbfs.part` and `sbfs.out`. +Starting another operation deletes an existing entry with the corresponding name as stale +temporary output, but these are otherwise legal user filenames. A real user file with one +of those names can therefore be deleted. + +Possible resolutions: + +- Reserve these names and refuse ordinary creation under them. +- Use a spare descriptor flag to identify temporary entries, and clean up by identity + rather than by name. +- Move temporary artifacts into a defined `/tmp` policy where appropriate. A stream's + temporary currently needs to be in the destination directory because rename changes a + name but not a parent, so `/tmp` alone may require adding a move operation. + +Whatever rule is chosen should be enforced in both SplitDisk and the native SBFS code and +documented as part of the format or CosmOS policy. + +## Correct stale CosmOS memory-map documentation + +`Programs/CosmOS/README.md` currently gives overlapping Data Memory ranges: + +```text +CosmOS: 0x0000 through 0x3FFF +Application: 0x2000 and above +``` + +The intended 8 KiB system allocation appears to be `0x0000` through `0x1FFF`, with +applications at `0x2000` and above. The minimal application in the same section still uses +the old bases (`0x2000` Program and `0x1000` Data), while current applications use `0x4000` +and `0x2000`. The opening comments in `cosmos.asm` and several monitor examples also retain +old addresses. + +The README says `make test` checks these boundaries, but the overlapping table passed. +Strengthen `Tests/docs.sh` so the documented system endpoints and application bases are +parsed and checked for agreement/non-overlap. + +## Establish one portable path-length limit + +SplitDisk carries paths up to 511 characters, while the native path machinery appears to +use a 255-byte limit. Paths are not stored on disk, so this does not change the format, but +the host can construct a tree addressable by a path that CosmOS cannot express in one +operation. + +Declare a portable CosmOS/SBFS path limit and have SplitDisk enforce it when modifying an +image. The native 255-byte limit is reasonable; the important property is agreement and +documentation. Component names remain limited to 22 bytes independently. + +## Bound the number of directory entries + +The parent field is 16 bits and stores `descriptor index + 1`, with zero reserved for the +root. Descriptor index 65535 therefore cannot be represented as a parent because adding +one wraps to zero. SplitDisk currently accepts directory sizes large enough to exceed the +representable parent domain. + +Define and validate a maximum directory-block/entry count such that every directory entry +can be named as a parent. Apply the check while formatting and mounting/reading malformed +images in both implementations. + +## Document the save/stream crash guarantee precisely + +The temporary-first ordering protects the old file from expected failures such as no +contiguous run or a refused block write. It is not power-loss atomic: a reset between +deleting the old entry and renaming the temporary can lose the destination and leave the +temporary behind. + +That is an appropriate tradeoff for SBFS; it does not call for a journal. Document the +guarantee as protection against normal operation failures rather than full crash +atomicity. A future boot cleanup or small consistency checker could reclaim unmistakable +temporary entries, especially if they receive a descriptor flag. + +## Design strengths worth preserving + +- Parent-as-index-plus-one makes every version one entry a valid root child without + conversion. +- Directories consume one descriptor and no data blocks, leaving the descriptor array as + the complete allocation map. +- Files remain contiguous and the block allocator remains ignorant of hierarchy. +- Path resolution below the service boundary gave existing applications directories + without changing their interfaces. +- The current directory is a two-byte identity rather than a stored string, and the shell + restores it after applications run. +- The independent host/native implementations and byte-identical disk agreement tests are + unusually strong validation of the written format. +- `osFileStart`/`osFileWrite`/`osFileFetch`/`osFileDone` provide the bounded-memory output + abstraction needed by assemblers, compilers, linkers, and future sequential pipelines. diff --git a/Programs/CosmOS/Apps/Compare.asm b/Programs/CosmOS/Apps/Compare.asm new file mode 100644 index 0000000..d4eb647 --- /dev/null +++ b/Programs/CosmOS/Apps/Compare.asm @@ -0,0 +1,277 @@ +; Compare two files without asking either one to fit in Data Memory. +; +; Each occupied block is read into its own 256-byte buffer. The byte count returned for +; the block is compared before its contents, and only those bytes are examined: bytes +; after the end of a short final block belong to neither file and are allowed to differ. +; +; Written by ChatGPT for Anachronaut's SplitBit + +#Include services.asm + +#Program + #Base 0x4000 + +start: + SETD.0 Arguments + INIB 0xFF + SWI osArgument + SETD.0 Arguments + LDA.0 + BRA usage + CALL textSplit + SETD.0 TextRest + LDD.0.0 + LDA.0 + BRA usage + + SETD.0 Arguments + SWI osFileInfo + BNQ firstFailed + SETD.0 FirstBlocks + STD.3.0 + + SETD.0 TextRest + LDD.0.0 + SWI osFileInfo + BNQ secondFailed + SETD.0 SecondBlocks + STD.3.0 + + ; A different number of occupied blocks is immediately a different file. + SETD.0 FirstBlocks + LDA.0 + SETD.1 SecondBlocks + LDB.1 + CCF + SUB + BNQ different + INCD.0 + INCD.1 + LDA.0 + LDB.1 + CCF + SUB + BNQ different + + SETD.0 FirstBlocks + SETD.1 RemainingBlocks + CALL copyWord + RSTA + SETD.0 Index + STA.0 + INCD.0 + STA.0 + +nextBlock: + SETD.0 RemainingBlocks + LDA.0 + INCD.0 + LDB.0 + OR + BRQ alike + + SETD.0 Arguments + SETD.1 FirstBlock + CALL readAtIndex + BNQ firstReadFailed + SETD.0 FirstCount + STD.3.0 + + SETD.0 TextRest + LDD.0.0 + SETD.1 SecondBlock + CALL readAtIndex + BNQ secondReadFailed + SETD.0 SecondCount + STD.3.0 + + ; Equal occupied-block counts do not imply equal tails, so compare the valid byte count + ; returned for this block as well. + SETD.0 FirstCount + LDA.0 + SETD.1 SecondCount + LDB.1 + CCF + SUB + BNQ different + INCD.0 + INCD.1 + LDA.0 + LDB.1 + CCF + SUB + BNQ different + + SETD.0 FirstCount + SETD.1 BytesLeft + CALL copyWord + SETD.0 FirstBlock + SETD.1 SecondBlock + +compareByte: + SETD.2 BytesLeft + LDA.2 + INCD.2 + LDB.2 + OR + BRQ blockSame + LDA.0 + LDB.1 + CCF + SUB + BNQ different + INCD.0 + INCD.1 + SETD.2 BytesLeft + CALL takeByte + BRI compareByte + +blockSame: + CALL stepIndex + CALL takeBlock + BRI nextBlock + +; DP0 is the path and DP1 the destination buffer. Q and DP3 are the service answers. +readAtIndex: + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osFileBlock + RET + +; DP0 names the source word and DP1 the destination word. +copyWord: + LDA.0 + STA.1 + INCD.0 + INCD.1 + LDA.0 + STA.1 + RET + +stepIndex: + SETD.0 Index + INCD.0 + LDA.0 + INCA + STA.0 + BNC stepDone + DECD.0 + LDA.0 + INCA + STA.0 +stepDone: + RET + +takeBlock: + SETD.0 RemainingBlocks + INCD.0 + LDA.0 + BRA blockBorrow + DECA + STA.0 + RET +blockBorrow: + INIA 0xFF + STA.0 + DECD.0 + LDA.0 + DECA + STA.0 + RET + +; Decrement the big-endian sixteen-bit BytesLeft. A full block arrives as 0x0100, so an +; eight-bit counter would compare none of it and call many different files equal. +takeByte: + INCD.2 + LDA.2 + BRA byteBorrow + DECA + STA.2 + RET +byteBorrow: + INIA 0xFF + STA.2 + DECD.2 + LDA.2 + DECA + STA.2 + RET + +alike: + SETD.0 SameText + SWI osPrintString + SWI osExit +different: + SETD.0 DifferentText + SWI osPrintString + SWI osExit +usage: + SETD.0 Usage + SWI osPrintString + SWI osExit +firstFailed: + SETD.0 FirstError + SWI osPrintString + SWI osExit +secondFailed: + SETD.0 SecondError + SWI osPrintString + SWI osExit +firstReadFailed: + SETD.0 FirstReadError + SWI osPrintString + SWI osExit +secondReadFailed: + SETD.0 SecondReadError + SWI osPrintString + SWI osExit + +#Data + #Base 0x2000 + +Arguments: + #Reserve 0d256 +FirstBlocks: + 0x00 0x00 +SecondBlocks: + 0x00 0x00 +RemainingBlocks: + 0x00 0x00 +Index: + 0x00 0x00 +FirstCount: + 0x00 0x00 +SecondCount: + 0x00 0x00 +BytesLeft: + 0x00 0x00 +FirstBlock: + #Reserve 0d256 +SecondBlock: + #Reserve 0d256 + +Usage: +"compare: give me two files +" +FirstError: +"compare: cannot find the first file +" +SecondError: +"compare: cannot find the second file +" +FirstReadError: +"compare: cannot read the first file +" +SecondReadError: +"compare: cannot read the second file +" +SameText: +"the same +" +DifferentText: +"different +" + +#Include text.asm diff --git a/Programs/CosmOS/Apps/Copy.asm b/Programs/CosmOS/Apps/Copy.asm new file mode 100644 index 0000000..fe82fad --- /dev/null +++ b/Programs/CosmOS/Apps/Copy.asm @@ -0,0 +1,269 @@ +; Copy one file to another without asking either file to fit in Data Memory. +; +; The two paths are the two words in the argument. Filesystem names can contain spaces, +; but the CosmOS command line has no quoting yet, so this deliberately has the same rule +; as the shell: a space separates things. +; +; osFileInfo reports how many disk blocks the source occupies, while osFileStart wants the +; shape stored in an SBFS entry: whole blocks and a possible tail. Reading the last block +; first recovers that distinction. The ordinary copy then walks forward one block at a +; time, through one 256-byte buffer. +; +; Written by ChatGPT for Anachronaut's SplitBit + +#Include services.asm + +#Program + #Base 0x4000 + +start: + SETD.0 Arguments + INIB 0xFF + SWI osArgument + SETD.0 Arguments + LDA.0 + BRA usage + CALL textSplit + SETD.0 TextRest + LDD.0.0 + LDA.0 + BRA usage + + ; How many occupied blocks the source has. Keep it as both the loop bound and the + ; distinction between an empty file and one whose last block must be inspected. + SETD.0 Arguments + SWI osFileInfo + BNQ sourceFailed + SETD.0 SourceBlocks + STD.3.0 + PSHD.3 + POPB + POPA + OR + BRQ emptySource + + ; Read the final occupied block. DP3 says how many bytes of it belong to the file: + ; 0x0100 for a whole block, or 0x0001 through 0x00FF for a tail. + SETD.0 SourceBlocks + LDD.3.0 + DECD.3 + SETD.0 Index + STD.3.0 + SETD.0 Arguments + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osFileBlock + BNQ readFailed + SETD.0 LastCount + STD.3.0 + + ; A full final block means every occupied block is whole. A short final block means + ; there is one fewer whole block and its low-byte count is the tail. + SETD.0 LastCount + LDA.0 + BNA wholeEnding + SETD.0 Index + SETD.1 OutputWhole + CALL copyWord + SETD.0 LastCount + INCD.0 + LDA.0 + SETD.1 OutputTail + STA.1 + BRI sizeKnown + +wholeEnding: + SETD.0 SourceBlocks + SETD.1 OutputWhole + CALL copyWord + RSTA + SETD.0 OutputTail + STA.0 + +sizeKnown: + BRI startOutput + +emptySource: + RSTA + SETD.0 OutputWhole + STA.0 + INCD.0 + STA.0 + SETD.0 OutputTail + STA.0 + +startOutput: + SETD.0 TextRest + LDD.0.0 + SETD.2 OutputWhole + LDD.3.2 + SETD.2 OutputTail + LDA.2 + SWI osFileStart + BNQ startFailed + + ; Empty files have no blocks to transfer, but still need committing so that an existing + ; destination becomes an empty file. + SETD.0 SourceBlocks + SETD.1 RemainingBlocks + CALL copyWord + RSTA + SETD.0 Index + STA.0 + INCD.0 + STA.0 + +copyNext: + SETD.0 RemainingBlocks + LDA.0 + INCD.0 + LDB.0 + OR + BRQ copyDone + + SETD.0 Arguments + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osFileBlock + BNQ readFailed + + SETD.1 Block + SETD.2 Index + LDA.2 + INCD.2 + LDB.2 + SWI osFileWrite + BNQ writeFailed + + CALL stepIndex + CALL takeBlock + BRI copyNext + +copyDone: + SETD.2 OutputWhole + LDD.3.2 + SETD.2 OutputTail + LDA.2 + SWI osFileDone + BNQ doneFailed + SETD.0 Copied + SWI osPrintString + SWI osExit + +; DP0 names the source word and DP1 the destination word. +copyWord: + LDA.0 + STA.1 + INCD.0 + INCD.1 + LDA.0 + STA.1 + RET + +; Increment the big-endian sixteen-bit Index. +stepIndex: + SETD.0 Index + INCD.0 + LDA.0 + INCA + STA.0 + BNC stepDone + DECD.0 + LDA.0 + INCA + STA.0 +stepDone: + RET + +; Decrement the big-endian sixteen-bit RemainingBlocks. +takeBlock: + SETD.0 RemainingBlocks + INCD.0 + LDA.0 + BRA takeBorrow + DECA + STA.0 + RET +takeBorrow: + INIA 0xFF + STA.0 + DECD.0 + LDA.0 + DECA + STA.0 + RET + +usage: + SETD.0 Usage + SWI osPrintString + SWI osExit +sourceFailed: + SETD.0 SourceError + SWI osPrintString + SWI osExit +readFailed: + SETD.0 ReadError + SWI osPrintString + SWI osExit +startFailed: + SETD.0 StartError + SWI osPrintString + SWI osExit +writeFailed: + SETD.0 WriteError + SWI osPrintString + SWI osExit +doneFailed: + SETD.0 DoneError + SWI osPrintString + SWI osExit + +#Data + #Base 0x2000 + +Arguments: + #Reserve 0d256 +SourceBlocks: + 0x00 0x00 +RemainingBlocks: + 0x00 0x00 +Index: + 0x00 0x00 +LastCount: + 0x00 0x00 +OutputWhole: + 0x00 0x00 +OutputTail: + 0x00 +Block: + #Reserve 0d256 + +Usage: +"copy: give me a source and destination +" +SourceError: +"copy: cannot find the source +" +ReadError: +"copy: cannot read the source +" +StartError: +"copy: cannot create the destination +" +WriteError: +"copy: cannot write the destination +" +DoneError: +"copy: cannot finish the destination +" +Copied: +"copied +" + +#Include text.asm diff --git a/Programs/CosmOS/README.md b/Programs/CosmOS/README.md index 8724f5c..fac694e 100644 --- a/Programs/CosmOS/README.md +++ b/Programs/CosmOS/README.md @@ -327,6 +327,8 @@ from every assembly file in it. Several are old programs written for the bare ma | Stream | Reads an 84,000 byte file through a buffer of 256, which is what says a file bigger than Data Memory can be read at all. | | Type | Prints a named text file a block at a time, including one too large to fit in Data Memory. | | Pour | Writes a file a block at a time, never holding more than one block of it. Each block is filled with a byte naming itself, so a block written to the wrong place shows up as content rather than as a length. | +| Copy | Copies one path to another a block at a time, including an empty file or one larger than Data Memory. | +| Compare | Compares two files a block at a time, stopping at their real tails rather than comparing unused bytes in the final disk blocks. | | Wander | Goes to the directory it is given and reads a file there by a bare name. The only thing that moves the machine from inside a program, and so the only thing that can check the shell puts the working directory back afterwards. | | More | A forward-only pager. Space advances a screen, Return one line, and q stops. | diff --git a/Tests/agree.sh b/Tests/agree.sh index 18e58d5..eab5126 100755 --- a/Tests/agree.sh +++ b/Tests/agree.sh @@ -142,6 +142,39 @@ else report FAIL "and every block landed" "a block is not where it was written" fi +# ---- A large file copied and compared on the machine ---- +# +# Copy joins the read and write streaming services: the source and destination are both +# larger than Data Memory, while the application owns one block. Compare then reads both +# through separate blocks and must ignore bytes past the tail. The host extracts the copy +# afterwards, so two native programs agreeing with each other is not the only oracle. +"$ASM" -I "$ROOT/Programs/CosmOS/Source" \ + "$ROOT/Programs/CosmOS/Apps/Copy.asm" -o Copy.sbx >/dev/null 2>&1 +"$ASM" -I "$ROOT/Programs/CosmOS/Source" \ + "$ROOT/Programs/CosmOS/Apps/Compare.asm" -o Compare.sbx >/dev/null 2>&1 +"$TOOL" format copied.img 1024 4 >/dev/null +"$TOOL" mkdir copied.img /Apps >/dev/null +"$TOOL" mkdir copied.img /Input >/dev/null +"$TOOL" mkdir copied.img /Output >/dev/null +"$TOOL" put copied.img Copy.sbx /Apps/Copy.sbx >/dev/null +"$TOOL" put copied.img Compare.sbx /Apps/Compare.sbx >/dev/null +awk 'BEGIN { for (i = 0; i < 84000; i++) printf "%c", 65 + (i % 26) }' > toCopy.dat +"$TOOL" put copied.img toCopy.dat /Input/source.dat >/dev/null +printf 'Copy /Input/source.dat /Output/copy.dat\nCompare /Input/source.dat /Output/copy.dat\nexit\n' \ + | "$EMU" cosmos.bin --fast --cycles 1000000000 --disk copied.img > copied.txt 2>&1 +"$TOOL" get copied.img /Output/copy.dat copiedBack.dat >/dev/null 2>&1 + +if cmp -s toCopy.dat copiedBack.dat; then + report ok "large native copy" "84000 bytes, host-identical" +else + report FAIL "large native copy" "the host read back different bytes" +fi +if grep -q '^> the same$' copied.txt; then + report ok "native compare" "the streamed files agree" +else + report FAIL "native compare" "Compare did not call the copied files equal" +fi + # ---- And each can read what the other wrote ---- # # Matching bytes and being readable are not the same claim. A field both of them write diff --git a/Tests/expected/cosmosCopyCompare.out b/Tests/expected/cosmosCopyCompare.out new file mode 100644 index 0000000..b952297 --- /dev/null +++ b/Tests/expected/cosmosCopyCompare.out @@ -0,0 +1,30 @@ +CosmOS +> copied +finished +> the same +finished +> copied +finished +> the same +finished +> copied +finished +> the same +finished +> copied +finished +> the same +finished +> different +finished +> copy: cannot find the source +finished +> compare: cannot find the second file +finished +> copy: give me a source and destination +finished +> compare: give me two files +finished +> halted +Execution halted. +[exit 0] diff --git a/Tests/input/cosmosCopyCompare.in b/Tests/input/cosmosCopyCompare.in new file mode 100644 index 0000000..c12bcac --- /dev/null +++ b/Tests/input/cosmosCopyCompare.in @@ -0,0 +1,14 @@ +Copy /Input/empty.dat /Output/empty.dat +Compare /Input/empty.dat /Output/empty.dat +Copy /Input/exact.dat /Output/exact.dat +Compare /Input/exact.dat /Output/exact.dat +Copy /Input/tail.dat /Output/tail.dat +Compare /Input/tail.dat /Output/tail.dat +Copy /Input/large.dat /Output/large.dat +Compare /Input/large.dat /Output/large.dat +Compare /Input/large.dat /Input/different.dat +Copy /Input/missing.dat /Output/missing.dat +Compare /Input/large.dat /Input/missing.dat +Copy +Compare /Input/large.dat +exit