Add Copy and Compare, which stream in both directions at once

Copy joins the read and write streaming services: source and destination are
both larger than Data Memory while the program owns one block. Compare reads
two files through separate blocks and ignores the bytes past a short final
block, which belong to neither file. Between them they exercise empty,
exact-block, part-block and 84,000 byte files, and the host extracts the copy
afterwards so that two native programs agreeing with each other is not the only
oracle.

Written by ChatGPT, as their headers record, along with the agree.sh section
and the manifest entry that drive them.

THIS SHOULD HAVE COME FIRST. The commit before it staged whole files rather
than the hunks it meant, so its manifest already names these two programs while
their source was still untracked - that commit will not build on its own. Left
in place rather than rewritten, since the pair is right and only their order is
wrong.

NOTES.md is their review of the streaming work. The first item in it is fixed
by the commit before this one; the rest are still open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E2JrLzFvuFX9fgi1LDRjrW
This commit is contained in:
Anachronaut
2026-08-25 20:21:43 -04:00
co-authored by Claude Opus 5
parent a7d3e09d94
commit 54f5cfe8a4
7 changed files with 731 additions and 0 deletions
+106
View File
@@ -0,0 +1,106 @@
# Delete me after these things are fixed
Temporary notes from the SBFS v2 and CosmOS review, in priority order.
## Streaming writes: reject a final size larger than the reservation
`osFileStart` reserves an extent and `osFileWrite` correctly refuses a block index outside
that extent. `sbfsStreamDone`, however, appears to accept a final block/tail count larger
than the extent reserved at the start. It writes the larger size into the descriptor and
then subtracts that size from the reservation when updating the free-block count.
A caller could reserve one block, finish while claiming two, and make the file descriptor
claim a neighboring block that was never allocated and may belong to another file.
Before deleting the old destination or modifying the temporary entry, compare the final
extent (`whole blocks + a possible tail block`) with the reserved extent. Refuse the commit
if the final extent is larger. Add tests for a final size that is smaller than, equal to,
and larger than the reservation, including tail/no-tail boundaries.
## Protect the temporary-file namespace
Whole-file saves and streamed writes use the fixed names `sbfs.part` and `sbfs.out`.
Starting another operation deletes an existing entry with the corresponding name as stale
temporary output, but these are otherwise legal user filenames. A real user file with one
of those names can therefore be deleted.
Possible resolutions:
- Reserve these names and refuse ordinary creation under them.
- Use a spare descriptor flag to identify temporary entries, and clean up by identity
rather than by name.
- Move temporary artifacts into a defined `/tmp` policy where appropriate. A stream's
temporary currently needs to be in the destination directory because rename changes a
name but not a parent, so `/tmp` alone may require adding a move operation.
Whatever rule is chosen should be enforced in both SplitDisk and the native SBFS code and
documented as part of the format or CosmOS policy.
## Correct stale CosmOS memory-map documentation
`Programs/CosmOS/README.md` currently gives overlapping Data Memory ranges:
```text
CosmOS: 0x0000 through 0x3FFF
Application: 0x2000 and above
```
The intended 8 KiB system allocation appears to be `0x0000` through `0x1FFF`, with
applications at `0x2000` and above. The minimal application in the same section still uses
the old bases (`0x2000` Program and `0x1000` Data), while current applications use `0x4000`
and `0x2000`. The opening comments in `cosmos.asm` and several monitor examples also retain
old addresses.
The README says `make test` checks these boundaries, but the overlapping table passed.
Strengthen `Tests/docs.sh` so the documented system endpoints and application bases are
parsed and checked for agreement/non-overlap.
## Establish one portable path-length limit
SplitDisk carries paths up to 511 characters, while the native path machinery appears to
use a 255-byte limit. Paths are not stored on disk, so this does not change the format, but
the host can construct a tree addressable by a path that CosmOS cannot express in one
operation.
Declare a portable CosmOS/SBFS path limit and have SplitDisk enforce it when modifying an
image. The native 255-byte limit is reasonable; the important property is agreement and
documentation. Component names remain limited to 22 bytes independently.
## Bound the number of directory entries
The parent field is 16 bits and stores `descriptor index + 1`, with zero reserved for the
root. Descriptor index 65535 therefore cannot be represented as a parent because adding
one wraps to zero. SplitDisk currently accepts directory sizes large enough to exceed the
representable parent domain.
Define and validate a maximum directory-block/entry count such that every directory entry
can be named as a parent. Apply the check while formatting and mounting/reading malformed
images in both implementations.
## Document the save/stream crash guarantee precisely
The temporary-first ordering protects the old file from expected failures such as no
contiguous run or a refused block write. It is not power-loss atomic: a reset between
deleting the old entry and renaming the temporary can lose the destination and leave the
temporary behind.
That is an appropriate tradeoff for SBFS; it does not call for a journal. Document the
guarantee as protection against normal operation failures rather than full crash
atomicity. A future boot cleanup or small consistency checker could reclaim unmistakable
temporary entries, especially if they receive a descriptor flag.
## Design strengths worth preserving
- Parent-as-index-plus-one makes every version one entry a valid root child without
conversion.
- Directories consume one descriptor and no data blocks, leaving the descriptor array as
the complete allocation map.
- Files remain contiguous and the block allocator remains ignorant of hierarchy.
- Path resolution below the service boundary gave existing applications directories
without changing their interfaces.
- The current directory is a two-byte identity rather than a stored string, and the shell
restores it after applications run.
- The independent host/native implementations and byte-identical disk agreement tests are
unusually strong validation of the written format.
- `osFileStart`/`osFileWrite`/`osFileFetch`/`osFileDone` provide the bounded-memory output
abstraction needed by assemblers, compilers, linkers, and future sequential pipelines.
+277
View File
@@ -0,0 +1,277 @@
; Compare two files without asking either one to fit in Data Memory.
;
; Each occupied block is read into its own 256-byte buffer. The byte count returned for
; the block is compared before its contents, and only those bytes are examined: bytes
; after the end of a short final block belong to neither file and are allowed to differ.
;
; Written by ChatGPT for Anachronaut's SplitBit
#Include services.asm
#Program
#Base 0x4000
start:
SETD.0 Arguments
INIB 0xFF
SWI osArgument
SETD.0 Arguments
LDA.0
BRA usage
CALL textSplit
SETD.0 TextRest
LDD.0.0
LDA.0
BRA usage
SETD.0 Arguments
SWI osFileInfo
BNQ firstFailed
SETD.0 FirstBlocks
STD.3.0
SETD.0 TextRest
LDD.0.0
SWI osFileInfo
BNQ secondFailed
SETD.0 SecondBlocks
STD.3.0
; A different number of occupied blocks is immediately a different file.
SETD.0 FirstBlocks
LDA.0
SETD.1 SecondBlocks
LDB.1
CCF
SUB
BNQ different
INCD.0
INCD.1
LDA.0
LDB.1
CCF
SUB
BNQ different
SETD.0 FirstBlocks
SETD.1 RemainingBlocks
CALL copyWord
RSTA
SETD.0 Index
STA.0
INCD.0
STA.0
nextBlock:
SETD.0 RemainingBlocks
LDA.0
INCD.0
LDB.0
OR
BRQ alike
SETD.0 Arguments
SETD.1 FirstBlock
CALL readAtIndex
BNQ firstReadFailed
SETD.0 FirstCount
STD.3.0
SETD.0 TextRest
LDD.0.0
SETD.1 SecondBlock
CALL readAtIndex
BNQ secondReadFailed
SETD.0 SecondCount
STD.3.0
; Equal occupied-block counts do not imply equal tails, so compare the valid byte count
; returned for this block as well.
SETD.0 FirstCount
LDA.0
SETD.1 SecondCount
LDB.1
CCF
SUB
BNQ different
INCD.0
INCD.1
LDA.0
LDB.1
CCF
SUB
BNQ different
SETD.0 FirstCount
SETD.1 BytesLeft
CALL copyWord
SETD.0 FirstBlock
SETD.1 SecondBlock
compareByte:
SETD.2 BytesLeft
LDA.2
INCD.2
LDB.2
OR
BRQ blockSame
LDA.0
LDB.1
CCF
SUB
BNQ different
INCD.0
INCD.1
SETD.2 BytesLeft
CALL takeByte
BRI compareByte
blockSame:
CALL stepIndex
CALL takeBlock
BRI nextBlock
; DP0 is the path and DP1 the destination buffer. Q and DP3 are the service answers.
readAtIndex:
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osFileBlock
RET
; DP0 names the source word and DP1 the destination word.
copyWord:
LDA.0
STA.1
INCD.0
INCD.1
LDA.0
STA.1
RET
stepIndex:
SETD.0 Index
INCD.0
LDA.0
INCA
STA.0
BNC stepDone
DECD.0
LDA.0
INCA
STA.0
stepDone:
RET
takeBlock:
SETD.0 RemainingBlocks
INCD.0
LDA.0
BRA blockBorrow
DECA
STA.0
RET
blockBorrow:
INIA 0xFF
STA.0
DECD.0
LDA.0
DECA
STA.0
RET
; Decrement the big-endian sixteen-bit BytesLeft. A full block arrives as 0x0100, so an
; eight-bit counter would compare none of it and call many different files equal.
takeByte:
INCD.2
LDA.2
BRA byteBorrow
DECA
STA.2
RET
byteBorrow:
INIA 0xFF
STA.2
DECD.2
LDA.2
DECA
STA.2
RET
alike:
SETD.0 SameText
SWI osPrintString
SWI osExit
different:
SETD.0 DifferentText
SWI osPrintString
SWI osExit
usage:
SETD.0 Usage
SWI osPrintString
SWI osExit
firstFailed:
SETD.0 FirstError
SWI osPrintString
SWI osExit
secondFailed:
SETD.0 SecondError
SWI osPrintString
SWI osExit
firstReadFailed:
SETD.0 FirstReadError
SWI osPrintString
SWI osExit
secondReadFailed:
SETD.0 SecondReadError
SWI osPrintString
SWI osExit
#Data
#Base 0x2000
Arguments:
#Reserve 0d256
FirstBlocks:
0x00 0x00
SecondBlocks:
0x00 0x00
RemainingBlocks:
0x00 0x00
Index:
0x00 0x00
FirstCount:
0x00 0x00
SecondCount:
0x00 0x00
BytesLeft:
0x00 0x00
FirstBlock:
#Reserve 0d256
SecondBlock:
#Reserve 0d256
Usage:
"compare: give me two files
"
FirstError:
"compare: cannot find the first file
"
SecondError:
"compare: cannot find the second file
"
FirstReadError:
"compare: cannot read the first file
"
SecondReadError:
"compare: cannot read the second file
"
SameText:
"the same
"
DifferentText:
"different
"
#Include text.asm
+269
View File
@@ -0,0 +1,269 @@
; Copy one file to another without asking either file to fit in Data Memory.
;
; The two paths are the two words in the argument. Filesystem names can contain spaces,
; but the CosmOS command line has no quoting yet, so this deliberately has the same rule
; as the shell: a space separates things.
;
; osFileInfo reports how many disk blocks the source occupies, while osFileStart wants the
; shape stored in an SBFS entry: whole blocks and a possible tail. Reading the last block
; first recovers that distinction. The ordinary copy then walks forward one block at a
; time, through one 256-byte buffer.
;
; Written by ChatGPT for Anachronaut's SplitBit
#Include services.asm
#Program
#Base 0x4000
start:
SETD.0 Arguments
INIB 0xFF
SWI osArgument
SETD.0 Arguments
LDA.0
BRA usage
CALL textSplit
SETD.0 TextRest
LDD.0.0
LDA.0
BRA usage
; How many occupied blocks the source has. Keep it as both the loop bound and the
; distinction between an empty file and one whose last block must be inspected.
SETD.0 Arguments
SWI osFileInfo
BNQ sourceFailed
SETD.0 SourceBlocks
STD.3.0
PSHD.3
POPB
POPA
OR
BRQ emptySource
; Read the final occupied block. DP3 says how many bytes of it belong to the file:
; 0x0100 for a whole block, or 0x0001 through 0x00FF for a tail.
SETD.0 SourceBlocks
LDD.3.0
DECD.3
SETD.0 Index
STD.3.0
SETD.0 Arguments
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osFileBlock
BNQ readFailed
SETD.0 LastCount
STD.3.0
; A full final block means every occupied block is whole. A short final block means
; there is one fewer whole block and its low-byte count is the tail.
SETD.0 LastCount
LDA.0
BNA wholeEnding
SETD.0 Index
SETD.1 OutputWhole
CALL copyWord
SETD.0 LastCount
INCD.0
LDA.0
SETD.1 OutputTail
STA.1
BRI sizeKnown
wholeEnding:
SETD.0 SourceBlocks
SETD.1 OutputWhole
CALL copyWord
RSTA
SETD.0 OutputTail
STA.0
sizeKnown:
BRI startOutput
emptySource:
RSTA
SETD.0 OutputWhole
STA.0
INCD.0
STA.0
SETD.0 OutputTail
STA.0
startOutput:
SETD.0 TextRest
LDD.0.0
SETD.2 OutputWhole
LDD.3.2
SETD.2 OutputTail
LDA.2
SWI osFileStart
BNQ startFailed
; Empty files have no blocks to transfer, but still need committing so that an existing
; destination becomes an empty file.
SETD.0 SourceBlocks
SETD.1 RemainingBlocks
CALL copyWord
RSTA
SETD.0 Index
STA.0
INCD.0
STA.0
copyNext:
SETD.0 RemainingBlocks
LDA.0
INCD.0
LDB.0
OR
BRQ copyDone
SETD.0 Arguments
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osFileBlock
BNQ readFailed
SETD.1 Block
SETD.2 Index
LDA.2
INCD.2
LDB.2
SWI osFileWrite
BNQ writeFailed
CALL stepIndex
CALL takeBlock
BRI copyNext
copyDone:
SETD.2 OutputWhole
LDD.3.2
SETD.2 OutputTail
LDA.2
SWI osFileDone
BNQ doneFailed
SETD.0 Copied
SWI osPrintString
SWI osExit
; DP0 names the source word and DP1 the destination word.
copyWord:
LDA.0
STA.1
INCD.0
INCD.1
LDA.0
STA.1
RET
; Increment the big-endian sixteen-bit Index.
stepIndex:
SETD.0 Index
INCD.0
LDA.0
INCA
STA.0
BNC stepDone
DECD.0
LDA.0
INCA
STA.0
stepDone:
RET
; Decrement the big-endian sixteen-bit RemainingBlocks.
takeBlock:
SETD.0 RemainingBlocks
INCD.0
LDA.0
BRA takeBorrow
DECA
STA.0
RET
takeBorrow:
INIA 0xFF
STA.0
DECD.0
LDA.0
DECA
STA.0
RET
usage:
SETD.0 Usage
SWI osPrintString
SWI osExit
sourceFailed:
SETD.0 SourceError
SWI osPrintString
SWI osExit
readFailed:
SETD.0 ReadError
SWI osPrintString
SWI osExit
startFailed:
SETD.0 StartError
SWI osPrintString
SWI osExit
writeFailed:
SETD.0 WriteError
SWI osPrintString
SWI osExit
doneFailed:
SETD.0 DoneError
SWI osPrintString
SWI osExit
#Data
#Base 0x2000
Arguments:
#Reserve 0d256
SourceBlocks:
0x00 0x00
RemainingBlocks:
0x00 0x00
Index:
0x00 0x00
LastCount:
0x00 0x00
OutputWhole:
0x00 0x00
OutputTail:
0x00
Block:
#Reserve 0d256
Usage:
"copy: give me a source and destination
"
SourceError:
"copy: cannot find the source
"
ReadError:
"copy: cannot read the source
"
StartError:
"copy: cannot create the destination
"
WriteError:
"copy: cannot write the destination
"
DoneError:
"copy: cannot finish the destination
"
Copied:
"copied
"
#Include text.asm
+2
View File
@@ -327,6 +327,8 @@ from every assembly file in it. Several are old programs written for the bare ma
| Stream | Reads an 84,000 byte file through a buffer of 256, which is what says a file bigger than Data Memory can be read at all. |
| Type | Prints a named text file a block at a time, including one too large to fit in Data Memory. |
| Pour | Writes a file a block at a time, never holding more than one block of it. Each block is filled with a byte naming itself, so a block written to the wrong place shows up as content rather than as a length. |
| Copy | Copies one path to another a block at a time, including an empty file or one larger than Data Memory. |
| Compare | Compares two files a block at a time, stopping at their real tails rather than comparing unused bytes in the final disk blocks. |
| Wander | Goes to the directory it is given and reads a file there by a bare name. The only thing that moves the machine from inside a program, and so the only thing that can check the shell puts the working directory back afterwards. |
| More | A forward-only pager. Space advances a screen, Return one line, and q stops. |
+33
View File
@@ -142,6 +142,39 @@ else
report FAIL "and every block landed" "a block is not where it was written"
fi
# ---- A large file copied and compared on the machine ----
#
# Copy joins the read and write streaming services: the source and destination are both
# larger than Data Memory, while the application owns one block. Compare then reads both
# through separate blocks and must ignore bytes past the tail. The host extracts the copy
# afterwards, so two native programs agreeing with each other is not the only oracle.
"$ASM" -I "$ROOT/Programs/CosmOS/Source" \
"$ROOT/Programs/CosmOS/Apps/Copy.asm" -o Copy.sbx >/dev/null 2>&1
"$ASM" -I "$ROOT/Programs/CosmOS/Source" \
"$ROOT/Programs/CosmOS/Apps/Compare.asm" -o Compare.sbx >/dev/null 2>&1
"$TOOL" format copied.img 1024 4 >/dev/null
"$TOOL" mkdir copied.img /Apps >/dev/null
"$TOOL" mkdir copied.img /Input >/dev/null
"$TOOL" mkdir copied.img /Output >/dev/null
"$TOOL" put copied.img Copy.sbx /Apps/Copy.sbx >/dev/null
"$TOOL" put copied.img Compare.sbx /Apps/Compare.sbx >/dev/null
awk 'BEGIN { for (i = 0; i < 84000; i++) printf "%c", 65 + (i % 26) }' > toCopy.dat
"$TOOL" put copied.img toCopy.dat /Input/source.dat >/dev/null
printf 'Copy /Input/source.dat /Output/copy.dat\nCompare /Input/source.dat /Output/copy.dat\nexit\n' \
| "$EMU" cosmos.bin --fast --cycles 1000000000 --disk copied.img > copied.txt 2>&1
"$TOOL" get copied.img /Output/copy.dat copiedBack.dat >/dev/null 2>&1
if cmp -s toCopy.dat copiedBack.dat; then
report ok "large native copy" "84000 bytes, host-identical"
else
report FAIL "large native copy" "the host read back different bytes"
fi
if grep -q '^> the same$' copied.txt; then
report ok "native compare" "the streamed files agree"
else
report FAIL "native compare" "Compare did not call the copied files equal"
fi
# ---- And each can read what the other wrote ----
#
# Matching bytes and being readable are not the same claim. A field both of them write
+30
View File
@@ -0,0 +1,30 @@
CosmOS
> copied
finished
> the same
finished
> copied
finished
> the same
finished
> copied
finished
> the same
finished
> copied
finished
> the same
finished
> different
finished
> copy: cannot find the source
finished
> compare: cannot find the second file
finished
> copy: give me a source and destination
finished
> compare: give me two files
finished
> halted
Execution halted.
[exit 0]
+14
View File
@@ -0,0 +1,14 @@
Copy /Input/empty.dat /Output/empty.dat
Compare /Input/empty.dat /Output/empty.dat
Copy /Input/exact.dat /Output/exact.dat
Compare /Input/exact.dat /Output/exact.dat
Copy /Input/tail.dat /Output/tail.dat
Compare /Input/tail.dat /Output/tail.dat
Copy /Input/large.dat /Output/large.dat
Compare /Input/large.dat /Output/large.dat
Compare /Input/large.dat /Input/different.dat
Copy /Input/missing.dat /Output/missing.dat
Compare /Input/large.dat /Input/missing.dat
Copy
Compare /Input/large.dat
exit