diff --git a/Programs/CosmOS/README.md b/Programs/CosmOS/README.md index b947ba9..66e6818 100644 --- a/Programs/CosmOS/README.md +++ b/Programs/CosmOS/README.md @@ -693,6 +693,55 @@ compares the images byte for byte. Every field one writes and the other only rea checked there and nowhere else: which entry a thing lands in, which block, what a directory's unused fields hold, the version, the free count. +### What A Listing Says About The Disk: + +`dir` ends with what is there and what is left: + +```text +43 files, 3 directories +46 of 64 entries, 1893 blocks free +``` + +**Entries come first because they are the ceiling nobody notices until they hit it.** A disk +of small files runs out of directory slots long before it runs out of blocks, and saying both +means never having to work out which one is about to bite. + +**The free figure is counted, not asked for.** The superblock keeps a free count and +`sbfs.asm` calls it *"a note rather than the truth"* in three separate places; `sbfsSpace` +reads the whole directory table instead, which costs a read per directory block and is the +answer rather than a guess. SplitDisk reads the note as well and says so when the two +disagree, which is the right place for that check: the host tool is what you audit a disk +with, and this is what you work on one with. + +**And it is a fact about the disk, not about where you are standing.** The first version of +this added the blocks up as the listing walked past them, which cost nothing extra and was +wrong - that walk stops only on entries in the working directory, so the same disk came out as +1,996 blocks free from the root and 2,025 from `/Apps`. The two implementations of this format +disagreeing is what said so. + +A third line appears when it needs to: + +```text +4 of 16 entries, 37 blocks free +the longest run is 25 +``` + +**Files are laid down contiguously**, so the free total is not what decides whether a file +will fit - the longest unbroken run is. A disk with a thousand blocks free in ten pieces +refuses a file of two hundred, and nothing else in the listing would hint at it. + +**It is said only when it differs from the free total**, which on a healthy disk it does not. +Deleting is what fragments a contiguous store, and a disk that has only been appended to has +one gap, at the end. A line printed under every listing saying the same number twice is noise; +one that appears only when something is wrong is a line somebody reads. + +Finding it costs a pass of the directory table per gap. There is no sort on this machine and +the entries are in no order, so a candidate walks the disk instead: each pass looks for the +used extent nearest at or after it, and anything the candidate is standing inside pushes it to +the far end and starts the pass again - the same trick allocating uses, for the same reason. +So the measurement costs almost nothing on a disk with one gap and more the more fragmented +the disk is, which is the right way round: it is slow exactly when it has something to say. + ### Starting An Application By Name: A word the shell has no command for is not immediately an error. Before saying so, the diff --git a/Programs/CosmOS/Source/cosmos.asm b/Programs/CosmOS/Source/cosmos.asm index e509ffd..edcd497 100644 --- a/Programs/CosmOS/Source/cosmos.asm +++ b/Programs/CosmOS/Source/cosmos.asm @@ -3938,6 +3938,102 @@ dirFolderCount: CALL printString dirNoFolders: CALL newLine + + ; ---- And what is left of the disk ---- + ; + ; A listing that says what is there and not what is left says half of what anybody wants + ; to know. SplitDisk has printed this since it was written; the machine's own listing had + ; nothing to say about the disk it was listing. + ; + ; COUNTED RATHER THAN ASKED. The superblock keeps a free count, and sbfs.asm calls it "a + ; note rather than the truth" in three separate places. The truth is what the entries add + ; up to, and the walk above has just added them up - so this costs no disk read at all, + ; where believing the note would cost one and be a guess. SplitDisk reads the note as well + ; and says so when the two disagree, which is the right place for that check: the host tool + ; is what you audit a disk with, and this is what you work on one with. + ; + ; ---- AND IT IS THE DISK, NOT THIS DIRECTORY ---- + ; + ; The first version of this added the blocks up as the listing walked past them, which cost + ; no extra read and was wrong: that walk stops only on entries in the working directory, so + ; the same disk was called 1,996 blocks free from the root and 2,025 from /Apps. Free space + ; is a fact about the disk, so sbfsSpace reads the whole directory table for it - and the + ; two implementations of this format disagreeing is what said so. + CALL sbfsSpace + BNQ dirNoSpace + + ; Entries first, because they are the ceiling nobody notices until they hit it - a disk of + ; small files runs out of directory slots long before it runs out of blocks, and saying + ; both means never having to work out which one is about to bite. + SETD.0 DirEntries + SETD.2 SbfsDirBlocks + CALL sbfsSetWord + INIA 0d3 + SETD.0 DirDoubles + STA.0 +dirEntriesTimes: + ; Eight entries to a directory block, which is three doublings on a machine with no + ; multiply. Adding a number to itself is what a doubling is. + SETD.0 DirEntries + SETD.2 DirEntries + CALL sbfsAddWord + SETD.0 DirDoubles + LDA.0 + DECA + STA.0 + BNA dirEntriesTimes + + SETD.0 SbfsUsedEntries + CALL printWordDecimal + SETD.0 OfText + CALL printString + SETD.0 DirEntries + CALL printWordDecimal + SETD.0 EntriesText + CALL printString + + ; What is left is the disk, less where the files begin, less what they hold. + SETD.0 DirFree + SETD.2 SbfsDiskBlocks + CALL sbfsSetWord + CALL sbfsFirstData + SETD.0 DirFree + SETD.2 SbfsCandidate + CALL sbfsSubWord + SETD.0 DirFree + SETD.2 SbfsUsedBlocks + CALL sbfsSubWord + + SETD.0 DirFree + CALL printWordDecimal + SETD.0 FreeText + CALL printString + CALL newLine + + ; ---- And how much of that is in one piece ---- + ; + ; Said only when it is not all of it, which is the common case and the quiet one. Files are + ; laid down contiguously, so the free total is not what decides whether a file will fit - + ; the longest run is. A disk with a thousand blocks free in ten pieces refuses a file of two + ; hundred, and nothing in the listing would have hinted at it. + ; + ; A LISTING THAT SAID "longest run 1893" UNDER "1893 blocks free" WOULD BE NOISE on every + ; healthy disk, and a line that only appears when something is wrong is a line somebody + ; reads. It is also what makes the measurement worth its cost: a disk with one gap at the + ; end, which is what an append-only disk is, walks the table twice and says nothing. + CALL sbfsLargestRun + BNQ dirNoSpace + SETD.0 DirFree + SETD.2 SbfsBiggest + CALL sbfsCompareWord + BRQ dirNoSpace + + SETD.0 RunText + CALL printString + SETD.0 SbfsBiggest + CALL printWordDecimal + CALL newLine +dirNoSpace: BRI prompt dirNoDisk: @@ -7927,6 +8023,15 @@ IsDirectory: "that is a directory" AndText: ", " +OfText: +" of " +EntriesText: +" entries, " +FreeText: +" blocks free" +RunText: +"the longest run is " + FoldersText: " directories" FolderText: @@ -8219,6 +8324,14 @@ DirFolders: 0x00 DirTaken: 0x00 +; What the entries add up to, what is left, and how many slots there are to fill. Counted on +; the way past rather than asked of the superblock, whose count sbfs.asm calls a note. +DirFree: + 0x00 0x00 +DirEntries: + 0x00 0x00 +DirDoubles: + 0x00 DiskReady: 0x00 LoadedOk: diff --git a/Programs/CosmOS/Source/sbfs.asm b/Programs/CosmOS/Source/sbfs.asm index f1b83b2..cbb46d9 100644 --- a/Programs/CosmOS/Source/sbfs.asm +++ b/Programs/CosmOS/Source/sbfs.asm @@ -1916,6 +1916,278 @@ sbfsEntryBounds: sbfsBoundsDone: RET +; ---- What the whole disk is holding ---- +; +; SbfsUsedBlocks becomes what every entry on the disk holds between them, and SbfsUsedEntries +; how many slots are filled. Q is zero if the disk could be read. +; +; EVERY ENTRY, NOT EVERY ENTRY IN A DIRECTORY. Free space is a fact about the disk and not +; about where you are standing, so this walks the directory table itself the way allocating +; does, rather than using sbfsFirst and sbfsNext - which stop only on entries whose parent is +; the working directory, and would have called the same disk emptier from one directory than +; from another. +; +; A DIRECTORY HOLDS NO BLOCKS and adds nought, which is true by its fields rather than by a +; test here. An unfinished save holds all of its, and is counted, because the blocks are +; spoken for whatever the entry is called - and a listing that left them out would promise +; room that is not there. +; +; The count in the superblock is not consulted. sbfs.asm calls it "a note rather than the +; truth" in three places, and this is the truth: it costs a read of every directory block, +; which is what the honest answer costs. +sbfsSpace: + RSTA + SETD.0 SbfsUsedBlocks + STA.0 + INCD.0 + STA.0 + SETD.0 SbfsUsedEntries + STA.0 + INCD.0 + STA.0 + + SETD.0 SbfsDirStart + SETD.1 SbfsBlock + CALL sbfsCopyWord + SETD.0 SbfsDirBlocks + INCD.0 + LDA.0 + SETD.1 SbfsLeft + STA.1 + +sbfsSpaceBlock: + CALL sbfsReadBlock + BNQ sbfsSpaceFailed + SETD.1 SbfsBuffer + CALL sbfsBufferOut + SETD.2 SbfsBuffer + INIA 0d8 + SETD.1 SbfsCount + STA.1 + +sbfsSpaceEntry: + LDA.2 + INIB 0x01 + AND + BRQ sbfsSpaceNext ; A free slot holds nothing and fills nothing. + + SETD.0 SbfsUsedEntries + CALL sbfsStepWord + + ; The blocks this entry holds, which are at the same offset in an entry as they are + ; everywhere else. sbfsEntryBounds wants DP2 on the entry and gives back both ends, and the + ; length is the difference - worked out that way rather than read directly, so that one + ; routine owns where in an entry those fields are. + PSHD.2 + CALL sbfsEntryBounds + SETD.0 SbfsEntryEnd + SETD.2 SbfsEntryStart + CALL sbfsSubWord + SETD.0 SbfsUsedBlocks + SETD.2 SbfsEntryEnd + CALL sbfsAddWord + POPD.2 + +sbfsSpaceNext: + DPUP.2 0d32 + SETD.1 SbfsCount + LDA.1 + DECA + STA.1 + BNA sbfsSpaceEntry + + SETD.0 SbfsBlock + CALL sbfsStepWord + SETD.1 SbfsLeft + LDA.1 + DECA + STA.1 + BNA sbfsSpaceBlock + + RSTA + RSTB + CCF + ADD + RET + +sbfsSpaceFailed: + INIA 0x01 + RSTB + CCF + ADD + RET + +; ---- The longest run of free blocks there is ---- +; +; SbfsBiggest becomes the largest number of blocks that are free AND NEXT TO EACH OTHER. +; Q is zero if the disk could be read. +; +; This is the number that says whether a file will fit. Files are laid down contiguously - +; first fit, with the directory itself as the map and no allocation table anywhere - so a disk +; with a thousand blocks free in ten scattered pieces will refuse a file of two hundred, and +; the free total gives no hint of it. +; +; ---- The same trick allocating uses, for the same reason ---- +; +; There is no sort on this machine and the entries are in no order, so the gaps cannot simply +; be listed. Instead a candidate walks the disk: each pass looks for the used extent nearest +; at or after it, and anything the candidate is standing inside pushes the candidate to the +; far end of it and starts the pass again. +; +; So the cost is a pass of the directory table per gap, rather than per file - and it is a +; measurement that costs nothing on a disk with one gap at the end, which is what a disk that +; has only been appended to looks like, and costs more the more fragmented the disk is. Which +; is the right way round: it is slow exactly when it has something to say. +sbfsLargestRun: + RSTA + SETD.0 SbfsBiggest + STA.0 + INCD.0 + STA.0 + CALL sbfsFirstData + +sbfsRunPass: + ; Nothing is in the way yet this time round, and the nearest thing ahead is the end of the + ; disk - which is the answer when there is nothing ahead at all. + RSTA + SETD.0 SbfsRunMoved + STA.0 + SETD.0 SbfsRunNext + SETD.2 SbfsDiskBlocks + CALL sbfsSetWord + + SETD.0 SbfsDirStart + SETD.1 SbfsBlock + CALL sbfsCopyWord + SETD.0 SbfsDirBlocks + INCD.0 + LDA.0 + SETD.1 SbfsLeft + STA.1 + +sbfsRunBlock: + CALL sbfsReadBlock + BNQ sbfsRunFailed + SETD.1 SbfsBuffer + CALL sbfsBufferOut + SETD.2 SbfsBuffer + INIA 0d8 + SETD.1 SbfsCount + STA.1 + +sbfsRunEntry: + LDA.2 + INIB 0x01 + AND + BRQ sbfsRunNextEntry ; A free slot is in nobody's way. + LDA.2 + INIB 0x02 + AND + BNQ sbfsRunNextEntry ; Nor is a directory, which holds no blocks - see allocating. + + CALL sbfsEntryBounds + PSHD.2 ; The comparisons want DP2, and the walk wants it back. + + ; Behind the candidate entirely, so it says nothing about what is ahead. + SETD.0 SbfsEntryEnd + SETD.2 SbfsCandidate + CALL sbfsCompareWord + BRC sbfsRunClear + BRQ sbfsRunClear + + ; Beginning at or before the candidate, so the candidate is standing inside it. It moves to + ; the far end and the pass begins again, exactly as allocating does. + SETD.0 SbfsEntryStart + SETD.2 SbfsCandidate + CALL sbfsCompareWord + BRC sbfsRunStraddles + BRQ sbfsRunStraddles + + ; Ahead of the candidate. Is it the nearest thing ahead so far? + SETD.0 SbfsEntryStart + SETD.2 SbfsRunNext + CALL sbfsCompareWord + BNC sbfsRunClear + SETD.0 SbfsRunNext + SETD.2 SbfsEntryStart + CALL sbfsSetWord + BRI sbfsRunClear + +sbfsRunStraddles: + SETD.0 SbfsCandidate + SETD.2 SbfsEntryEnd + CALL sbfsSetWord + INIA 0x01 + SETD.0 SbfsRunMoved + STA.0 + +sbfsRunClear: + POPD.2 + +sbfsRunNextEntry: + DPUP.2 0d32 + SETD.1 SbfsCount + LDA.1 + DECA + STA.1 + BNA sbfsRunEntry + + SETD.0 SbfsBlock + CALL sbfsStepWord + SETD.1 SbfsLeft + LDA.1 + DECA + STA.1 + BNA sbfsRunBlock + + ; The pass is over. If the candidate moved, everything measured against where it used to be + ; was measured against the wrong place, so the pass is worth nothing and is done again. + SETD.0 SbfsRunMoved + LDA.0 + BNA sbfsRunPass + + ; Nothing moved, so the gap from the candidate to the nearest thing ahead is a real one. + SETD.0 SbfsRunGap + SETD.2 SbfsRunNext + CALL sbfsSetWord + SETD.0 SbfsRunGap + SETD.2 SbfsCandidate + CALL sbfsSubWord + + SETD.0 SbfsBiggest + SETD.2 SbfsRunGap + CALL sbfsCompareWord + BNC sbfsRunKept ; What is already remembered is the same or bigger. + SETD.0 SbfsBiggest + SETD.2 SbfsRunGap + CALL sbfsSetWord + +sbfsRunKept: + ; Past the end of the disk is the end of the walk. Otherwise the candidate goes to the + ; thing that stopped it, which the next pass finds itself standing inside and steps over. + SETD.0 SbfsRunNext + SETD.2 SbfsDiskBlocks + CALL sbfsCompareWord + BNC sbfsRunDone + SETD.0 SbfsCandidate + SETD.2 SbfsRunNext + CALL sbfsSetWord + BRI sbfsRunPass + +sbfsRunDone: + RSTA + RSTB + CCF + ADD + RET + +sbfsRunFailed: + INIA 0x01 + RSTB + CCF + ADD + RET + ; Finds a run of SbfsWantBlocks free blocks and puts where it begins in SbfsFileStart. ; Q is zero if there was room. ; @@ -3691,6 +3963,24 @@ SbfsScratch1: SbfsPathWanted: 0x00 +; What every entry on the disk holds between them, and how many slots are filled. Worked out +; by sbfsSpace, which reads the whole directory table to do it. +; The longest run of free blocks, and the sweep that finds it: where it has got to ahead of +; the candidate, whether anything pushed the candidate this pass, and the gap being measured. +SbfsBiggest: + 0x00 0x00 +SbfsRunNext: + 0x00 0x00 +SbfsRunGap: + 0x00 0x00 +SbfsRunMoved: + 0x00 + +SbfsUsedBlocks: + 0x00 0x00 +SbfsUsedEntries: + 0x00 0x00 + SbfsFileStart: 0x00 0x00 SbfsFileBlocks: diff --git a/Source/DiskTool/SplitDisk.c b/Source/DiskTool/SplitDisk.c index c05efe4..a0ca3a9 100644 --- a/Source/DiskTool/SplitDisk.c +++ b/Source/DiskTool/SplitDisk.c @@ -433,6 +433,56 @@ static uint16_t countFree(const Directory *directory, const Superblock *super) { return (uint16_t)(super->diskBlocks - overhead - used); } +// ---- The longest run of free blocks there is ---- +// +// The number that says whether a file will fit. Files are laid down contiguously - first fit, +// with the directory itself as the map - so a disk with a thousand blocks free in ten +// scattered pieces refuses a file of two hundred, and the free total gives no hint of it. +// +// A sweep rather than a sort: a candidate walks the disk, each pass looking for the used +// extent nearest at or after it, and anything the candidate is standing inside pushes it to +// the far end and starts the pass again. That is what sbfs.asm does, because that machine has +// no sort - and the two implementations of this format are worth more agreeing than they are +// each being clever separately. +static uint16_t largestRun(const Directory *directory, const Superblock *super) { + const uint32_t firstData = (uint32_t)super->directoryStart + super->directoryBlocks; + uint32_t candidate = firstData; + uint32_t biggest = 0; + for (;;) { + uint32_t next = super->diskBlocks; + int moved = 0; + for (int i = 0; i < directory->entries; i++) { + const uint8_t *entry = entryAt(directory, i); + if (!entryInUse(entry) || entryIsDirectory(entry)) { + continue; + } + const uint32_t start = readWord(entry + SBFS_ENTRY_START); + const uint32_t end = start + entryBlocksUsed(entry); + if (end <= candidate) { + continue; + } + if (start <= candidate) { + candidate = end; + moved = 1; + continue; + } + if (start < next) { + next = start; + } + } + if (moved) { + continue; + } + if (next - candidate > biggest) { + biggest = next - candidate; + } + if (next >= super->diskBlocks) { + return (uint16_t)biggest; + } + candidate = next; + } +} + // ---- Commands ---- static int commandFormat(const char *path, uint16_t blocks, uint16_t directoryBlocks, @@ -647,6 +697,15 @@ static int commandList(const char *path, const char *within) { } printf(".\n"); + // Said only when it is not all of it. On a disk that has only been appended to the two + // are the same number, and printing it twice would be noise on every healthy disk - a + // line that appears only when something is wrong is a line somebody reads. + uint16_t run = largestRun(&directory, &super); + if (run != counted) { + printf("The longest run of free blocks is %u, so nothing bigger than that will" + " fit.\n", run); + } + // A save that stopped between deleting the old entry and naming the new one. The // bytes are all there under the temporary's name and one rename brings them back, // which is the whole of the recovery this format offers - so the thing that matters diff --git a/SplitBit Test Manual.md b/SplitBit Test Manual.md index f7a97da..b6651e3 100644 --- a/SplitBit Test Manual.md +++ b/SplitBit Test Manual.md @@ -125,7 +125,7 @@ from `make`, not from here. ### 1. Recorded output `Tests/run.sh` assembles each program named in `Tests/manifest`, runs it, and compares -everything it printed against a file in `Tests/expected`. 217 tests, of which 155 run, 35 +everything it printed against a file in `Tests/expected`. 218 tests, of which 156 run, 35 only assemble, 16 are expected to fail to assemble, and 11 boot from ROM with no image given at all. @@ -447,7 +447,7 @@ the console's line editing was in when it broke twice in two days. ## Fixture Disks: -`Tests/makedisks.sh` builds 28 images with SplitDisk before anything runs, into +`Tests/makedisks.sh` builds 29 images with SplitDisk before anything runs, into `Tests/build/disks`. **That is the point of them.** A SplitBit program reading one of these is being checked against a filesystem written by different code from the same written specification, rather than against itself. diff --git a/Tests/expected/cfgFallback.out b/Tests/expected/cfgFallback.out index c7866da..6a93ae1 100644 --- a/Tests/expected/cfgFallback.out +++ b/Tests/expected/cfgFallback.out @@ -15,6 +15,7 @@ made > cd Notes /Notes> dir 0 files +8 of 32 entries, 308 blocks free /Notes> exit halted Execution halted. diff --git a/Tests/expected/cosmos.out b/Tests/expected/cosmos.out index a755708..34f9082 100644 --- a/Tests/expected/cosmos.out +++ b/Tests/expected/cosmos.out @@ -29,6 +29,7 @@ across.txt 700 empty.txt 0 aName22CharactersLong! 22 12 files +12 of 16 entries, 48 blocks free > > frobnicate I do not know: frobnicate diff --git a/Tests/expected/cosmosBuild.out b/Tests/expected/cosmosBuild.out index 85ff880..5fdf1f4 100644 --- a/Tests/expected/cosmosBuild.out +++ b/Tests/expected/cosmosBuild.out @@ -1,6 +1,7 @@ CosmOS > dir 0 files +0 of 16 entries, 125 blocks free > mkdir Apps made > mkdir Apps/Deep @@ -11,10 +12,12 @@ made Apps