web: a scan hands its frames to the strip as it reads them, and coming to the library no longer walks the roll behind the reader

LIBRARY drew the strip from the catalogue and the catalogue from a `getAll` of
the `photos` store: every row and every thumbnail in it, read back on a clock to
learn what a scan of its own had just stored. On a catalogue of 150,000 frames
one read measured 2036ms and sixty megabytes of rows; under a scan in the same
window, with four lanes of RAW bytes and a LibRaw of a quarter of a gigabyte
beside it, that read is the read that gives way — and a read that gave way
answered `[]`, so the screen threw away the frames it was showing. That is the
strip that loses its count and its thumbnails under a scan, and the frames the
reader cannot pick while the roll is being read.

The reading now hands its frames over as it stores them. `scanFolder` keeps the
rows it met for the first time in the batch they landed in, and `flush` gives
them to `watchRows` the moment the transaction closes; the screen appends them,
so the strip is the roll arriving and not the roll found again. A batch is fifty
frames, so this is fifty rows over a callback where it used to be a hundred and
fifty thousand rows over IndexedDB. A frame the catalogue already held is not
handed over — it is on the strip already, and it takes its place again when the
reading is through.

Which is why the read on the clock is now done for none of them. It stands for
the reading another window holds, whose frames never come through this one: a
reading of this window's hands over every frame it stores, and the counter that
says so is what decides. The scan's own tail read went with it — the screen
watching a reading reads the catalogue back once the reading ends, and that is
the whole catalogue read once per scan instead of twice at the end of every one.

`readPhotos` answers null where `listPhotos` answered an empty list. A read that
came back with nothing is a read that failed, not a catalogue that emptied, and
the frames it would have cleared are the frames the reader is working their way
through. The strip keeps them.

And the strip starts on what the last screen read. STUDIO and LIBRARY are two
screens of one page, not two pages: the reader who goes to develop a frame and
comes back was reading a hundred thousand rows again to see the strip they had
just left.

The folder is walked on the way in only when the last reading of it never
finished. A reading that reaches its end clears its position, so "is there a
position" is "was this roll cut off", answered by one small file opened and
shut. Before this, every visit to the library paid a walk of the folder the
reader was opening — a hundred thousand names off the disk, on the folder they
had just asked to browse — and the reader who came to choose a frame paid for a
scan they never asked for. A folder the reader wants looked at again says so
itself, from the menu on its row.

Checked on the running bundle: a 3000-frame scan reads the catalogue back twice,
one of the two the screen coming up on a catalogue that is still empty — against
three for the commit before this one and thirteen for the one before that, and
nothing read back for the whole of the scan. The peak heap is 116-138MB with no
long tasks, and 3001 rows went in. A reading stopped at 59 of 4000 still leaves
`walk/ROLL.json` at 113,626 characters with no local storage key beside it, and
the visit after a reload carries on with the strip filling under it — 162 rows,
312, 463, 612, 762, 913, 1062, 1212 as the scan went on. LIBRARY on 150,000
frames shows no "No folder yet" at any point and settles on "150000 photos" in
4.9s against the 8.1s it took. The wall, the strip, the grid, the deep link and
the phone's recipes all pass their checks.

docker/frontend/src/engine/library.ts

@@ -550,6 +550,11 @@ export async function scanFolder(
   // hands the list on with the rest of its position.
   const dirs: LibraryDir[] = progress.dirs;
   let batch: LibraryPhoto[] = [];
+  // The frames this reading found for the first time, waiting to be handed on
+  // with the batch they landed in. A frame the catalogue already had is not one
+  // of these, however it came back — it is on the strip already, and a second
+  // copy of it is a second row under one key.
+  let added: LibraryPhoto[] = [];
   // The frames in hand: the ones a lane has been handed, and the ones whose row is
   // in `batch` and not yet in the catalogue. They are on neither side of the line a
   // position is written on — the catalogue does not hold them and the walk will not
@@ -596,6 +601,16 @@ export async function scanFolder(
       tx.onerror = () => reject(tx.error);
     });
     progress.written += rows.length;
+    // Out of the batch and onto the strip in one move, before the position is
+    // written: the frames are in the catalogue now, and the screen that draws
+    // them needs no read to find them. Ordered by nothing — the order they were
+    // read in is the order the roll was shot in, near enough, and the sort that
+    // settles it comes when the reading is through.
+    if (added.length) {
+      const fresh = added;
+      added = [];
+      rowSink?.(fresh);
+    }
     await write();
   };
   // One frame, end to end: its bytes, its tile, its shutter time, and the row the
@@ -638,7 +653,7 @@ export async function scanFolder(
     const thumb = src ? await tile(src, size) : null;
     const taken = (await readCapturedAt(bytes)) ?? file.lastModified;
     const cut = rel.lastIndexOf('/');
-    batch.push({
+    const row: LibraryPhoto = {
       id,
       folder: folder.name,
       dir: cut < 0 ? '' : rel.slice(0, cut),
@@ -652,7 +667,13 @@ export async function scanFolder(
       // keeps the stars it was given.
       star: seen?.star,
       addedAt: seen?.addedAt ?? Date.now(),
-    });
+    };
+    batch.push(row);
+    // A frame the catalogue already held is on the strip already: its row takes
+    // the place of the one there, at the end of the reading. Only the frame the
+    // reading met for the first time is handed over as it lands — that is the
+    // one the reader is waiting to see.
+    if (!seen) added.push(row);
     progress.added++;
     inHand = inHand.filter((r) => r !== rel);
     if (batch.length >= BATCH) await flush();
@@ -890,6 +911,30 @@ export function watchScan(fn: () => void): () => void {
   };
 }
 
+// The rows a reading has just stored, handed to the screen that is up as they
+// land. A scan of a hundred thousand frames stores a batch a second, and the
+// strip that waited to be told by a read of the whole catalogue spent the roll
+// doing nothing else — a second and a half of the page held still, sixty
+// megabytes of rows thrown away, every few seconds, while beside it a RAW
+// decoder held a quarter of a gigabyte. The window that holds the reading is the
+// one that hears this; a window watching from the side reads the catalogue back
+// on its clock, as it always did.
+let rowSink: ((rows: LibraryPhoto[]) => void) | null = null;
+
+export function watchRows(fn: ((rows: LibraryPhoto[]) => void) | null): void {
+  rowSink = fn;
+}
+
+// Whether a reading of this folder was left half-done. A position on the disk is
+// a roll that was cut off before its end and the only thing that finishes it is
+// reading it again; a peer holding the same folder is the same answer, from the
+// window that is already doing it. A question and not a walk: one small file is
+// opened and shut, and not a name on the disk is looked at.
+export async function unfinished(folder: string): Promise<boolean> {
+  if (peer?.folder === folder) return true;
+  return (await loadWalk(folder)) !== null;
+}
+
 export function stopScan(): void {
   // A reading another window holds is stopped by telling it so: the button this
   // window drew is the same button, on the window that can act on it.
@@ -949,18 +994,40 @@ export function startScan(folder: LibraryFolder): Promise<ScanProgress> {
 
 // --- reading ---------------------------------------------------------------
 
-export async function listPhotos(folder?: string): Promise<LibraryPhoto[]> {
+// The catalogue, read once and kept. STUDIO and LIBRARY are two screens of one
+// page, and the reader who goes to the studio to develop a frame and comes back
+// is not a reader who wants a hundred thousand rows read and sorted again to see
+// the strip they were just looking at. The read is the one `listPhotos` always
+// did; what it found is left here, and the screen that opens next draws from it
+// before the read it just started has answered.
+let cached: LibraryPhoto[] | null = null;
+
+export function cachedPhotos(): LibraryPhoto[] {
+  return cached ?? [];
+}
+
+// Null, and not an empty list: a read that failed is not a catalogue with
+// nothing in it, and the screen that would draw an empty strip on one is the
+// screen that loses the frames it was showing — under a scan of its own, where
+// the read is the thing that gives way first, that is every time.
+export async function readPhotos(folder?: string): Promise<LibraryPhoto[] | null> {
   try {
     const rows = await ask<LibraryPhoto[]>(PHOTOS, 'readonly', (s) =>
       folder ? s.index('folder').getAll(folder) : s.getAll()
     );
     // Newest shutter first — a folder of stills reads in the order it was shot.
-    return rows.sort((a, b) => b.taken - a.taken);
+    rows.sort((a, b) => b.taken - a.taken);
+    if (!folder) cached = rows;
+    return rows;
   } catch {
-    return [];
+    return null;
   }
 }
 
+export async function listPhotos(folder?: string): Promise<LibraryPhoto[]> {
+  return (await readPhotos(folder)) ?? [];
+}
+
 export async function getPhoto(id: string): Promise<LibraryPhoto | null> {
   try {
     return (await ask<LibraryPhoto | undefined>(PHOTOS, 'readonly', (s) => s.get(id))) ?? null;