3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(library): land a reading's rows as they are read, and keep the copy the reader reads offline

Browse Source 
- scan: rows go down on a clock as well as on a full batch. The reader's folder
  of 26 RAWs is under one batch of 50, so nothing of it reached the screen until
  the reading was over — a wall that stood empty for the whole scan and then
  filled in one go. Frames now reach the screen every 400ms while the roll is
  still being read, so the count climbs and the first tiles are drawn as the
  frames land.
- tiles: a HEIC or a RAW has its tile written the first time it is drawn, so
  every later visit reads the tile back instead of the file.
- resume: a reading a reload cut off is offered by the next visit with a chip
  instead of being run behind the reader's back.
- offline listing: the desk's listing is kept whole in IndexedDB under
  `recipescam-offline`/`reads`, past the 200 000 characters localStorage held,
  so an offline visit opens on the listing it kept instead of on nothing.
- offline writes: a vote or a view cast with no server is held and goes out when
  one answers again, with a bar that says how many are waiting.
- sw: the shell precaches its own bundle, stylesheet and icon, read out of the
  document — a worker that precached only the document answered an offline
  launch with a page whose JavaScript the machine did not have.
- checks in docker/frontend/scripts: tree-scan-check (counts while reading,
  resume after a reload, tiles of a long roll), offline-read-check,
  offline-queue-check.
This commit is contained in:
3dtours
2026-10-09 12:02:05 +07:00
parent f834d10762
commit 45fc852d04
17 changed files with 2054 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/public/sw.js
+25 -7
View File
@@ -10,11 +10,14 @@
// hashed bundle with it, which the STATIC rule below then served cache-first
// forever. Every navigation here now goes past the browser's own cache, so the
// build a visitor gets is the one the server has.
const VERSION = 'recipescam-v11';
const VERSION = 'recipescam-v12';
// nginx answers all three with the same index.html (SPA fallback), so they are one
// document under three keys: an offline navigation finds it whichever key it asks.
const SHELL = ['/', '/app', '/library'];
// nginx answers all three routes with the same index.html (SPA fallback), so they
// are one document under three keys: an offline navigation finds it whichever key
// it asks. The manifest is not that document and is precached with them for the
// same reason — it is what makes the installed app an app, and offline it is
// asked for on every launch.
const SHELL = ['/', '/app', '/library', '/manifest.json'];
// Cache-first paths: vite hashes every filename in its own bundle, and the wasm,
// the models and the icons never change under a given build.
@@ -47,12 +50,27 @@ const offlineShell = async (request) =>
(await caches.match('/')) ??
Response.error();
// The files the shell is built out of, read out of the shell itself: the hashed
// bundle, the stylesheet, the icon. A worker that precached only the document
// would answer an offline launch with a page whose JavaScript this machine does
// not have — and it never would, because the visit that first loads the app is
// the visit this worker is not yet controlling, so its own STATIC rule below
// never sees those requests. The shell is the one place their names are written.
const shellFiles = (html) => [...html.matchAll(/(?:src|href)="(\/assets\/[^"]+)"/g)].map((m) => m[1]);
self.addEventListener('install', (event) => {
// Best effort, one route at a time: a deploy caught mid-flight must not leave the
// worker uninstalled, so each route fails on its own.
// Best effort, one file at a time: a deploy caught mid-flight must not leave the
// worker uninstalled, so each one fails on its own.
event.waitUntil(
caches.open(VERSION)
.then((cache) => Promise.all(SHELL.map((url) => cache.add(shellRequest(url)).catch(() => {}))))
.then(async (cache) => {
await Promise.all(SHELL.map((url) => cache.add(shellRequest(url)).catch(() => {})));
const shell = await cache.match('/app') ?? await cache.match('/');
if (!shell) return;
for (const url of shellFiles(await shell.text())) {
await cache.add(shellRequest(url)).catch(() => {});
}
})
.then(() => self.skipWaiting()),
);
});
docker/frontend/scripts/offline-queue-check.mjs
+152
View File
@@ -0,0 +1,152 @@
// The offline queue, checked in a browser: a write made with no server is held
// rather than lost, a second write to the same route replaces the first, the
// reload keeps it, and a server answering sends it on its own.
//
// The API is stubbed on the page's own origin the way the container's nginx
// serves it, and "no server" is a route that fails the request — which is
// exactly what a dead server looks like from `fetch`.
//
// npm run build && node scripts/offline-queue-check.mjs
// APP_PORT=4885 PLAYWRIGHT_CORE=<path to playwright-core/index.mjs> \
// node scripts/offline-queue-check.mjs
import { spawn } from 'node:child_process';
import { fileURLToPath } from 'node:url';
const FRONTEND = fileURLToPath(new URL('..', import.meta.url));
const APP_PORT = Number(process.env.APP_PORT ?? 4186);
// The key pwa/offline.ts holds the queue under.
const KEY = 'rc.pending';
let failures = 0;
function step(name, ok, detail = '') {
if (!ok) failures++;
console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ` — ${detail}` : ''}`);
return ok;
}
const playwright = await import(process.env.PLAYWRIGHT_CORE ?? 'playwright-core').catch(() => null);
if (!playwright) {
console.log('SKIP playwright-core is not installed');
process.exit(0);
}
const app = spawn('npx', ['vite', 'preview', '--port', String(APP_PORT), '--strictPort', '--host', '127.0.0.1'], {
cwd: FRONTEND,
stdio: 'ignore',
});
const base = `http://127.0.0.1:${APP_PORT}`;
let up = false;
for (let i = 0; i < 60 && !up; i++) {
up = await fetch(`${base}/`).then((r) => r.ok).catch(() => false);
if (!up) await new Promise((r) => setTimeout(r, 500));
}
if (!up) {
app.kill('SIGTERM');
console.log(`FAIL no preview on ${base} — run \`npm run build\` first`);
process.exit(1);
}
const browser = await playwright.chromium.launch({
executablePath: process.env.CHROME || undefined,
args: ['--no-sandbox', '--enable-unsafe-swiftshader'],
});
const context = await browser.newContext();
let gone = false; // whether the page's server is there
const sent = []; // every write the stub let through
const PHOTO = {
id: 1,
createdAt: '2026-01-01T00:00:00Z',
slots: ['strip'],
tag: '#CHECK',
title: 'Check frame',
meta: '#1',
consent: true,
};
await context.route('**/api/**', (route) => {
if (gone) return route.abort('failed');
const req = route.request();
const url = req.url();
const json = (body) => route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(body) });
if (req.method() !== 'GET') {
sent.push({ method: req.method(), path: new URL(url).pathname, body: req.postData() });
return json({ key: 'photo:1', rating: { avg: 5, n: 1, mine: 5 } });
}
if (url.includes('/auth/me')) return json({ user: { id: 1, email: 'check@example.com' } });
if (url.includes('/photos')) return json({ photos: [PHOTO] });
if (url.includes('/ratings')) return json({ ratings: {} });
if (url.includes('/highlights')) return json({ highlights: { day: [], week: [], ever: [] } });
if (url.includes('/recipes')) return json({ recipes: [] });
return json({});
});
// The bar's own words are asserted, so the page starts in Vietnamese.
await context.addInitScript(() => localStorage.setItem('rc.lang', 'vi'));
const page = await context.newPage();
const star = (n) => `[data-key="lp-star-photo:1-${n}"]`;
const bar = '[data-key="offline-bar"]';
const noise = [];
page.on('console', (m) => {
// The outage is the check's own doing, and a request that fails is a request
// the browser logs. Everything else is a real complaint.
if (m.type() === 'error' && !m.text().includes('net::ERR_FAILED')) noise.push(m.text());
});
const queued = () =>
page.evaluate((k) => JSON.parse(localStorage.getItem(k) ?? '[]'), KEY);
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector(star(5), { timeout: 60_000 });
// 1 — a write with no server is held, not lost.
gone = true;
await page.$eval(star(5), (el) => el.click());
await page
.waitForFunction((k) => JSON.parse(localStorage.getItem(k) ?? '[]').length === 1, KEY, { timeout: 15_000 })
.catch(() => {});
let held = await queued();
step(
'a vote cast with no server is held',
held.length === 1 && held[0].method === 'POST' && held[0].path === '/ratings',
JSON.stringify(held)
);
const notice = await page.textContent(bar).catch(() => null);
step('and the bar says how many are waiting', notice !== null && notice.includes('Chưa gửi được: 1'), notice ?? 'no bar');
// 2 — the same vote again replaces the held one instead of stacking.
await page.$eval(star(3), (el) => el.click());
await page.waitForTimeout(500);
held = await queued();
step(
'a second vote on the same frame is one write, the later one',
held.length === 1 && String(held[0].body).includes('"stars":3'),
JSON.stringify(held)
);
// 3 — the reload keeps it.
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector(star(5), { timeout: 60_000 });
held = await queued();
step('and it is still held across a reload with no server', held.length === 1, JSON.stringify(held));
// 4 — a server answering sends it, with no one asking.
gone = false;
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page
.waitForFunction((k) => JSON.parse(localStorage.getItem(k) ?? '[]').length === 0, KEY, { timeout: 30_000 })
.catch(() => {});
held = await queued();
step(
'the held write goes out when a server answers',
held.length === 0 &&
sent.some((s) => s.method === 'POST' && s.path === '/api/ratings' && s.body === '{"key":"photo:1","stars":3}'),
JSON.stringify(sent)
);
step('and the bar goes with it', (await page.$(bar)) === null);
step('nothing thrown along the way', noise.length === 0, noise[0] ?? '');
await browser.close();
app.kill('SIGTERM');
console.log(failures === 0 ? 'all pass' : `${failures} failed`);
process.exit(failures === 0 ? 0 : 1);
docker/frontend/scripts/offline-read-check.mjs
+155
View File
@@ -0,0 +1,155 @@
// The offline answer to a big read, checked in a browser: the strip's listing is
// a few hundred kilobytes and used to have no offline answer at all, because the
// last one was kept in localStorage under a cap of two hundred thousand
// characters. It is kept whole in IndexedDB now, and an offline visit opens on it.
//
// The API is stubbed on the page's own origin the way the container's nginx
// serves it, and "no server" is a route that fails the request — which is exactly
// what a dead server looks like from `fetch`.
//
// npm run build && node scripts/offline-read-check.mjs
// APP_PORT=4886 PLAYWRIGHT_CORE=<path to playwright-core/index.mjs> \
// node scripts/offline-read-check.mjs
import { spawn } from 'node:child_process';
import { fileURLToPath } from 'node:url';
const FRONTEND = fileURLToPath(new URL('..', import.meta.url));
const APP_PORT = Number(process.env.APP_PORT ?? 4187);
// The cap the read used to be kept under, and the store it is kept in now.
const OLD_CAP = 200_000;
const DB = 'recipescam-offline';
const READS = 'reads';
let failures = 0;
function step(name, ok, detail = '') {
if (!ok) failures++;
console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ` — ${detail}` : ''}`);
return ok;
}
const playwright = await import(process.env.PLAYWRIGHT_CORE ?? 'playwright-core').catch(() => null);
if (!playwright) {
console.log('SKIP playwright-core is not installed');
process.exit(0);
}
// The listing as a studio with a real catalogue has it: enough frames that the
// JSON is past the old cap on its own, each carrying the look it was saved with —
// a curve is the bulk of one, and a hundred points of it is a small one.
const PHOTOS = Array.from({ length: 60 }, (_, i) => ({
id: i + 1,
createdAt: '2026-01-01T00:00:00Z',
slots: ['strip'],
tag: `#P${i + 1}`,
title: `Frame ${i + 1}`,
meta: `ISO 400 · 35mm`,
consent: true,
hasPreset: false,
recipe: { name: 'PROVIPES', curve: Array.from({ length: 512 }, (_, n) => n / 512) },
}));
const LISTING = JSON.stringify({ photos: PHOTOS });
const app = spawn('npx', ['vite', 'preview', '--port', String(APP_PORT), '--strictPort', '--host', '127.0.0.1'], {
cwd: FRONTEND,
stdio: 'ignore',
});
const base = `http://127.0.0.1:${APP_PORT}`;
let up = false;
for (let i = 0; i < 60 && !up; i++) {
up = await fetch(`${base}/`).then((r) => r.ok).catch(() => false);
if (!up) await new Promise((r) => setTimeout(r, 500));
}
if (!up) {
app.kill('SIGTERM');
console.log(`FAIL no preview on ${base} — run \`npm run build\` first`);
process.exit(1);
}
const browser = await playwright.chromium.launch({
executablePath: process.env.CHROME || undefined,
args: ['--no-sandbox', '--enable-unsafe-swiftshader'],
});
const context = await browser.newContext();
let gone = false; // whether the page's server is there
await context.route('**/api/**', (route) => {
if (gone) return route.abort('failed');
const req = route.request();
const url = req.url();
const json = (body) => route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(body) });
if (url.includes('/auth/me')) return json({ user: null });
if (url.includes('/photos')) return route.fulfill({ status: 200, contentType: 'application/json', body: LISTING });
if (url.includes('/ratings')) return json({ ratings: {} });
if (url.includes('/highlights')) return json({ highlights: { day: [], week: [], ever: [] } });
if (url.includes('/recipes')) return json({ recipes: [] });
return json({});
});
const page = await context.newPage();
const noise = [];
page.on('console', (m) => {
// The outage is the check's own doing, and a request that fails is a request
// the browser logs. Everything else is a real complaint.
if (m.type() === 'error' && !m.text().includes('net::ERR_FAILED')) noise.push(m.text());
});
const frames = () => page.$$eval('.lp-frame', (els) => els.length);
const readStale = (key) =>
page
.evaluate(
([db, store, key]) =>
new Promise((resolve, reject) => {
const req = indexedDB.open(db, 1);
req.onupgradeneeded = () => {
if (!req.result.objectStoreNames.contains(store)) req.result.createObjectStore(store);
};
req.onsuccess = () => {
const d = req.result;
if (!d.objectStoreNames.contains(store)) return resolve(null);
const get = d.transaction(store, 'readonly').objectStore(store).get(key);
get.onsuccess = () => resolve(get.result ?? null);
get.onerror = () => reject(get.error);
};
req.onerror = () => reject(req.error);
}),
[DB, READS, key]
)
.catch(() => null);
step('the listing this check serves is past the cap the old code kept', LISTING.length > OLD_CAP, `${LISTING.length} chars`);
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.lp-frame', { timeout: 60_000 });
await page.waitForFunction(() => document.querySelectorAll('.lp-frame').length >= 60, null, { timeout: 30_000 }).catch(() => {});
const online = await frames();
const stored = await readStale('/photos');
step(
'a listing past the old cap is kept for offline',
typeof stored === 'string' && stored.length > OLD_CAP,
typeof stored === 'string' ? `${stored.length} chars in the store` : 'no record'
);
const leftBehind = await page.evaluate(() => Object.keys(localStorage).filter((k) => k.startsWith('rc.stale.')).length);
step('and nothing is left in localStorage under the old key', leftBehind === 0, `${leftBehind} keys`);
const ratings = await readStale('/ratings');
step('a small read is kept the same way', typeof ratings === 'string', ratings === null ? 'no record' : 'kept');
// The reload with no server: the listing is the last one it was shown, so the
// reel is the same reel rather than an empty band.
gone = true;
await page.goto(base, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.lp-frame', { timeout: 30_000 }).catch(() => {});
const offline = await frames();
step(
'and the offline visit opens on the listing it kept',
online >= 60 && offline === online,
`${online} frames online, ${offline} offline`
);
step('nothing thrown along the way', noise.length === 0, noise[0] ?? '');
await browser.close();
app.kill('SIGTERM');
console.log(failures === 0 ? 'all pass' : `${failures} failed`);
process.exit(failures === 0 ? 0 : 1);
docker/frontend/scripts/tree-scan-check.mjs
+746
View File
@@ -0,0 +1,746 @@
// The three things the reader reported, checked in a browser: the counts beside
// the folder names while a reading is running, what the column holds after the
// tab comes back — every folder the walk went into, the empty ones included, with
// an interrupted reading picking itself up — and a long roll's tiles, every frame
// the wall draws read out of the reading that brought them.
//
// The app is behind an account and answers `/api` on its own origin, so the check
// stands in for the API the way the container's nginx does; without it the page
// is a guest and there is no folder to pick. The catalogue is read back out of
// IndexedDB rather than out of the strip, which draws one node's own frames.
//
// npm run build && node scripts/tree-scan-check.mjs
// APP_PORT=4395 SAMPLE_PORT=4396 SAMPLE_DIR=<folder of sample frames> \
// PLAYWRIGHT_CORE=<path to playwright-core/index.mjs> CHROME=<path to chrome> \
// node scripts/tree-scan-check.mjs
import { spawn } from 'node:child_process';
import { readFile } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const FRONTEND = fileURLToPath(new URL('..', import.meta.url));
const SAMPLES = process.env.SAMPLE_DIR ?? path.resolve(FRONTEND, '..', 'sample');
const APP_PORT = Number(process.env.APP_PORT ?? 4185);
const SAMPLE_PORT = Number(process.env.SAMPLE_PORT ?? 4319);
const JPG = 'P1010256.JPG';
const RAW = 'P1010256.RW2';
// What the frames of the long roll are served: a small JPEG, because a hundred
// copies of an eight-megabyte one is eight hundred megabytes to prove a point
// about tiles.
const SMALL = 'Main.jpg';
const LONG = 60;
const LONG_NAMES = Array.from({ length: LONG }, (_, i) => `B${String(i).padStart(3, '0')}.JPG`);
let failures = 0;
function step(name, ok, detail = '') {
if (!ok) failures++;
console.log(`${ok ? 'ok ' : 'FAIL'} ${name}${detail ? ` — ${detail}` : ''}`);
return ok;
}
const playwright = await import(process.env.PLAYWRIGHT_CORE ?? 'playwright-core').catch(() => null);
if (!playwright) {
console.log('SKIP playwright-core is not installed');
process.exit(0);
}
// Every frame the page asks for, by extension: one real JPEG and one real RAW — see
// the route that hands them over, below.
const app = spawn('npx', ['vite', 'preview', '--port', String(APP_PORT), '--strictPort', '--host', '127.0.0.1'], {
cwd: FRONTEND,
stdio: 'ignore',
});
const base = `http://127.0.0.1:${APP_PORT}`;
let up = false;
for (let i = 0; i < 60 && !up; i++) {
up = await fetch(`${base}/library`).then((r) => r.ok).catch(() => false);
if (!up) await new Promise((r) => setTimeout(r, 500));
}
if (!up) {
app.kill('SIGTERM');
console.log(`FAIL no preview on ${base} — run \`npm run build\` first`);
process.exit(1);
}
const browser = await playwright.chromium.launch({
executablePath: process.env.CHROME || undefined,
args: ['--no-sandbox', '--enable-unsafe-swiftshader'],
});
const context = await browser.newContext();
// The sample frames are handed over inside the browser rather than off a socket of
// its own: a page is given six sockets to a host, and the walk's sixty reads fill
// all six, so the shelf's own reads would queue behind the reading they are meant
// to keep up with — a queue no real disk puts in the way.
await context.route(`http://127.0.0.1:${SAMPLE_PORT}/**`, async (route) => {
const name = path.basename(decodeURIComponent(new URL(route.request().url()).pathname));
const file = /^B\d+\.JPG$/.test(name) ? SMALL : /\.jpe?g$/i.test(name) ? JPG : /\.rw2$/i.test(name) ? RAW : null;
if (!file) return route.fulfill({ status: 404, body: '' });
return route.fulfill({
status: 200,
contentType: 'application/octet-stream',
headers: { 'Access-Control-Allow-Origin': '*' },
body: await readFile(path.join(SAMPLES, file)),
});
});
// The catalogue is behind an account, and the header reads the account's own
// folder: the API answers for both, the way the container beside the page does.
await context.route('**/api/**', (route) => {
const url = route.request().url();
const json = (body) => route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(body) });
if (url.includes('/auth/me')) return json({ user: { id: 1, email: 'check@example.com', name: 'check' } });
if (url.includes('/photos')) return json({ photos: [] });
if (url.includes('/recipes')) return json({ recipes: [] });
return route.fulfill({ status: 404, contentType: 'application/json', body: JSON.stringify({ error: 'no' }) });
});
await context.addInitScript(
({ origin, folder, rolls, small }) => {
const bytes = new Map();
// One fetch per real file whatever the handle is called: the frames of the
// long roll are one small JPEG served under sixty names.
const load = async (name) => {
const file = /^B\d+\.JPG$/.test(name) ? small : name;
if (!bytes.has(file)) bytes.set(file, fetch(`${origin}/${file}`).then((r) => r.arrayBuffer()));
return bytes.get(file);
};
const fileHandle = (name) => {
const handle = { kind: 'file', name, __fake: true };
Object.defineProperty(handle, 'getFile', {
value: async () => {
const gets = JSON.parse(sessionStorage.getItem('check.gets') ?? '{}');
gets[name] = (gets[name] ?? 0) + 1;
sessionStorage.setItem('check.gets', JSON.stringify(gets));
while (localStorage.getItem('check.hold') === '1') await new Promise((r) => setTimeout(r, 25));
// A read of a frame is nearly all waiting on a real disk, and a reading
// that is over in a blink is one the wall's own work cannot be watched
// against. `check.slow` is that wait, in milliseconds.
const slow = Number(localStorage.getItem('check.slow') ?? 0);
if (slow) await new Promise((r) => setTimeout(r, slow));
return new File([await load(name)], name, { type: 'application/octet-stream', lastModified: Date.UTC(2026, 0, 1) });
},
});
return handle;
};
const dirHandle = (label, node) => {
const handle = { kind: 'directory', name: label, __fake: true };
Object.defineProperties(handle, {
values: {
value: () => {
const lists = JSON.parse(sessionStorage.getItem('check.lists') ?? '{}');
lists[label] = (lists[label] ?? 0) + 1;
sessionStorage.setItem('check.lists', JSON.stringify(lists));
if (node.fail && localStorage.getItem('check.broken') !== 'open') throw new Error(`${label} will not open`);
return [
...(node.files ?? []).map(fileHandle),
...Object.entries(node.dirs ?? {}).map(([name, kid]) => dirHandle(name, kid)),
][Symbol.iterator]();
},
},
queryPermission: { value: async () => 'granted' },
requestPermission: { value: async () => 'granted' },
getDirectoryHandle: {
value: async (name) => {
const kid = node.dirs?.[name];
if (!kid) throw new Error(`no folder named ${name}`);
return dirHandle(name, kid);
},
},
getFileHandle: {
value: async (name) => {
if (!(node.files ?? []).includes(name)) throw new Error(`no frame named ${name}`);
return fileHandle(name);
},
},
});
return handle;
};
const pick = (name) => dirHandle(name, { files: rolls[name].files, dirs: rolls[name].dirs });
// The roll in play, not the one this script was handed: a handle that comes
// back out of the catalogue belongs to the folder being read, and the reading
// a reload takes up is of that one.
const root = () => pick(localStorage.getItem('check.roll') ?? folder);
window.showDirectoryPicker = async () => root();
// What the disk holds, for the check to change behind the app's back: a frame
// taken off the disk is a folder the walk meets one frame fewer in.
window.__rolls = rolls;
const revive = (row) => {
if (row?.handle?.__fake && !Object.getOwnPropertyDescriptor(row.handle, 'getFile')) {
row.handle = row.handle.kind === 'directory' ? root() : fileHandle(row.handle.name);
}
return row;
};
for (const method of ['get', 'getAll']) {
const original = IDBObjectStore.prototype[method];
IDBObjectStore.prototype[method] = function (...args) {
const request = original.apply(this, args);
const descriptor = Object.getOwnPropertyDescriptor(IDBRequest.prototype, 'result');
Object.defineProperty(request, 'result', {
configurable: true,
get() {
const value = descriptor.get.call(this);
return Array.isArray(value) ? value.map(revive) : revive(value);
},
});
return request;
};
}
},
{
origin: `http://127.0.0.1:${SAMPLE_PORT}`,
folder: 'CheckRoll',
small: SMALL,
rolls: {
// One frame at the top, one a folder down, one two folders down, one in a
// second branch — and a folder holding no frame at all.
CheckRoll: {
files: [JPG],
dirs: {
2026: { files: [RAW], dirs: { '04': { files: [JPG], dirs: {} } } },
2025: { files: [JPG], dirs: {} },
Empty: { files: ['notes.txt'], dirs: {} },
},
},
// A roll with a folder that will not open: a permission taken back, a
// volume pulled out — a folder the walk meets and cannot list.
BrokenRoll: {
files: [JPG],
dirs: { Broken: { fail: true, files: [JPG], dirs: {} } },
},
// A roll long enough that its reading outlasts the wall's own work: sixty
// frames read at a walking pace, so the wall has time to tile them while
// the reading is still going.
BigRoll: {
files: LONG_NAMES,
dirs: {},
},
// The same roll read for the first time, on a tab that goes away.
CutRoll: {
files: [JPG],
dirs: { Broken: { fail: true, files: [JPG], dirs: {} } },
},
// A roll picked at a drive root, which Chromium hands over named `\` — the
// one name a folder under it is spelled from and that is itself no folder
// name at all. See the check at the foot of this file.
'\\': {
files: [JPG],
dirs: { Roll: { files: [RAW], dirs: {} } },
},
},
}
);
const page = await context.newPage();
const rowStates = () =>
page.$$eval('[data-key^="lib-node-"]', (els) =>
els.map((e) => `${e.dataset.key.replace('lib-node-', '')}:${(e.querySelector('.adm-album-count')?.textContent ?? '').trim()}`)
);
const progress = () => page.$eval('[data-key="lib-progress"]', (e) => e.textContent ?? '').catch(() => '');
const counts = async () => {
const out = {};
for (const row of await rowStates()) {
const [key, n] = row.split(':');
out[key] = n;
}
return out;
};
// Row order is the tree's path order, not the order of this file's expectations.
const same = (a, b) => JSON.stringify(Object.entries(a).sort()) === JSON.stringify(Object.entries(b).sort());
// What the reading has stored, asked of the catalogue rather than of the strip:
// the strip draws the open node's own frames, and three of these four are one
// folder down.
const stored = () =>
page
.evaluate(async () => {
const db = await new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
return new Promise((resolve, reject) => {
const req = db.transaction('photos', 'readonly').objectStore('photos').count();
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
})
.catch(() => -1);
// What the walk found, which is the whole of what the column is asked to say.
const WANT = {
CheckRoll: '4',
'CheckRoll/2025': '1',
'CheckRoll/2026': '2',
'CheckRoll/2026/04': '1',
'CheckRoll/Empty': '0',
};
// --- a reading, watched while it runs ---------------------------------------
await page.goto(`${base}/library`);
// Held on the first frame it reads, so the walk is through its first pass — the
// names and their numbers are the reading's, and the reading is not done.
await page.evaluate(() => localStorage.setItem('check.hold', '1'));
await page.click('[data-key="lib-add"]');
await page.waitForSelector('[data-key="lib-node-CheckRoll"]', { timeout: 30_000 });
// Every row of the pass is a row now, and a count beside each name is what the
// walk met — the reading itself has read nothing.
await page
.waitForFunction(() => {
const rows = [...document.querySelectorAll('[data-key^="lib-node-"]')];
return rows.length === 4 && /\d+\/\d+/.test(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '');
}, null, { timeout: 60_000, polling: 50 })
.catch(() => {});
const midFloat = await counts();
const midLine = (await progress()).replace(/\s+/g, ' ').trim();
const midStored = await stored();
// The whole roll is one pass of four folders, so the numbers are the walk's the
// moment the reading has listed the tree — before a single frame is read.
step(
'the counts beside the folder names are up while the reading runs',
same(midFloat, WANT),
`${JSON.stringify(midFloat)} at "${midLine}"`
);
step(
'and they are the walk’s own, not the catalogue’s: nothing has been read yet',
midStored === 0,
`${midStored} rows stored`
);
step('and the reading is in flight while the column already says so', Boolean(await page.$('[data-key="lib-progress"]')), `"${midLine}"`);
// --- the reading runs out ----------------------------------------------------
await page.evaluate(() => localStorage.removeItem('check.hold'));
// The progress span is in the page whether or not a reading is, and its text is
// empty until the first pass has said what the roll is: silence is the reading
// being through.
await page
.waitForFunction(() => !(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim(), null, {
timeout: 120_000,
polling: 50,
})
.catch(() => {});
const done = await counts();
step(
'a reading that ran out leaves the roll counted, empty folders and all',
same(done, WANT),
`${JSON.stringify(done)}, ${await stored()} rows stored`
);
// --- the tab comes back ------------------------------------------------------
await page.reload();
await page.waitForSelector('[data-key="lib-node-CheckRoll"]', { timeout: 60_000 });
await page.waitForTimeout(1000);
const back = await counts();
step(
'and the next visit draws the same column with nothing right-clicked',
same(back, done),
JSON.stringify(back)
);
step(
'a folder the walk went into and met no frame in keeps its row',
typeof back['CheckRoll/Empty'] === 'string',
`Empty: ${back['CheckRoll/Empty']}`
);
// Nothing is left to read, so this visit offers nothing and starts nothing: the
// catalogue comes back off the disk and the screen waits to be asked.
step(
'and a visit with nothing left to read reads nothing at all',
!(await page.$('[data-key="lib-resume"]')) && (await progress()).trim() === '',
`resume ${Boolean(await page.$('[data-key="lib-resume"]'))}, line "${(await progress()).trim()}"`
);
// --- a reading cut off -------------------------------------------------------
// The catalogue is emptied first, so every frame is one this reading has to read
// itself: a reading that does not go on by itself leaves the strip at nothing.
await page.evaluate(async () => {
sessionStorage.clear();
const db = await new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
await new Promise((resolve, reject) => {
const tx = db.transaction('photos', 'readwrite');
tx.objectStore('photos').clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
});
localStorage.setItem('check.hold', '1');
});
const counting = () =>
page
.waitForFunction(() => /\d+\s*\/\s*\d+/.test(document.querySelector('[data-key="lib-progress"]')?.textContent ?? ''), null, {
timeout: 60_000,
polling: 50,
})
.then(() => progress())
.then((line) => line.replace(/\s+/g, ' ').trim())
.catch(() => '');
// A reading of the whole roll, held on the first frame it reads: what a tab that
// is closed mid-reading leaves behind is exactly this.
await page.click('[data-key="lib-node-CheckRoll"]', { button: 'right' });
await page.waitForSelector('[data-key="lib-rescan-CheckRoll"]', { timeout: 10_000 });
await page.click('[data-key="lib-rescan-CheckRoll"]');
const cut = await counting();
// The walk writes its own position out at most once a second, and the reload is
// what has to be cut off mid-reading rather than before it started.
await page.waitForTimeout(2000);
// Nothing here asks for another reading, and neither does the visit that comes
// back: the roll the reload cut off is put on offer beside the progress line, and
// the press of that offer is the only thing that starts the second reading. The
// first frame is still held, so a visit that went on by itself would hang on it
// rather than pass.
await page.reload();
await page.waitForSelector('[data-key="lib-resume"]', { timeout: 60_000 });
const offered = (await progress()).trim();
step(
'a reading a reload cut off is offered by the next visit, not run by it',
/\d+\/\d+/.test(cut) && offered === '',
`"${cut}" then idle "${offered}"`
);
await page.click('[data-key="lib-resume"]');
const cutAgain = await counting();
step('and the offer is that reading taken up where it stopped', /\d+\/\d+/.test(cutAgain), `"${cutAgain}"`);
await page.evaluate(() => localStorage.removeItem('check.hold'));
// The progress span is in the page whether or not a reading is, and its text is
// empty until the first pass has said what the roll is: silence is the reading
// being through.
await page
.waitForFunction(() => !(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim(), null, {
timeout: 120_000,
polling: 50,
})
.catch(() => {});
const resumed = await stored();
const lists = await page.evaluate(() => JSON.parse(sessionStorage.getItem('check.lists') ?? '{}'));
step(
'and the visit after it finishes the roll on its own, without listing a folder twice',
resumed === 4 && lists.CheckRoll === 1 && lists['2026'] === 1 && lists.Empty === 1 && lists['2025'] === 1,
`${resumed} rows stored from ${JSON.stringify(lists)}`
);
const keys = (store) =>
page
.evaluate(
(store) =>
new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => {
const db = req.result;
const get = db.transaction(store, 'readonly').objectStore(store).getAllKeys();
get.onsuccess = () => resolve(get.result);
get.onerror = () => reject(get.error);
};
req.onerror = () => reject(req.error);
}),
store
)
.catch(() => []);
const quiet = () =>
page
.waitForFunction(() => !(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim(), null, {
timeout: 120_000,
polling: 50,
})
.catch(() => {});
// A reading, watched out: the line is up once it has told the column what the
// roll is, and silence is the reading being through. Waiting for silence alone
// would read the line before the reading has written it.
const readOut = async () => {
await page
.waitForFunction(() => /\d+\s*\/\s*\d+/.test(document.querySelector('[data-key="lib-progress"]')?.textContent ?? ''), null, {
timeout: 20_000,
polling: 50,
})
.catch(() => {});
await quiet();
};
// A row for a frame the disk has not got, with the tile and the marks that go
// with it — the state a catalogue is left in when a frame is taken off the disk.
const seed = (ids) =>
page.evaluate(
(ids) =>
new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => {
const db = req.result;
const tx = db.transaction(['photos', 'thumbs', 'edits'], 'readwrite');
for (const id of ids) {
const parts = id.split('/');
tx.objectStore('photos').put({
id,
folder: parts[0],
dir: parts.slice(1, -1).join('/'),
name: parts[parts.length - 1],
handle: { kind: 'file', name: parts[parts.length - 1], __fake: true },
taken: 1,
size: 1,
mtime: 1,
addedAt: 1,
});
tx.objectStore('thumbs').put({ id, thumb: new Blob(['x']) });
tx.objectStore('edits').put({ photoId: id, recipe: {}, updatedAt: 1 });
}
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
};
req.onerror = () => reject(req.error);
}),
ids
);
// --- the catalogue brought to the disk --------------------------------------
// The reader's report: a folder's frames and the number kept for it disagree, and
// a rescan of the parent does not put them right. Both halves are here — a frame
// taken off the disk while its row stayed in the catalogue, and a row for a frame
// that was never on the disk at all — and one rescan of the roll has to settle
// both, the numbers under the subfolders included.
await seed(['CheckRoll/2025/Gone.JPG', 'BrokenRoll/Broken/x.JPG']);
await page.evaluate(() => {
window.__rolls.CheckRoll.dirs['2025'].files.length = 0;
});
step(
'the catalogue is holding two rows the disk has nothing for',
(await keys('photos')).includes('CheckRoll/2025/Gone.JPG') &&
(await keys('photos')).includes('CheckRoll/2025/P1010256.JPG'),
JSON.stringify(await keys('photos'))
);
await page.click('[data-key="lib-node-CheckRoll"]', { button: 'right' });
await page.waitForSelector('[data-key="lib-rescan-CheckRoll"]', { timeout: 10_000 });
await page.click('[data-key="lib-rescan-CheckRoll"]');
await readOut();
const repaired = await counts();
const rows = (await keys('photos')).filter((k) => k.startsWith('CheckRoll/'));
step(
'a rescan of the parent brings every number under it to the disk',
same(repaired, {
...WANT,
CheckRoll: '3',
'CheckRoll/2025': '0',
}),
JSON.stringify(repaired)
);
step(
'and the frames the reading did not meet are gone from the catalogue',
rows.length === 3 && !rows.includes('CheckRoll/2025/P1010256.JPG') && !rows.includes('CheckRoll/2025/Gone.JPG'),
JSON.stringify(rows)
);
step(
'with their tiles and their marks',
!(await keys('thumbs')).includes('CheckRoll/2025/Gone.JPG') &&
!(await keys('edits')).includes('CheckRoll/2025/Gone.JPG'),
JSON.stringify(await keys('edits'))
);
// --- a folder the walk could not open ---------------------------------------
// The other side of the same rule: the walk met nothing under `Broken`, and a
// reading that read silence for absence would empty a folder it never listed.
// The number beside its name is the catalogue's, which is the last reading that
// could open it: a reading that could not open a folder writes no nought there —
// a nought takes the place of the frames the catalogue holds and reads as a
// folder gone empty until the reader rescans the roll.
await page.evaluate(() => localStorage.setItem('check.roll', 'BrokenRoll'));
await page.click('[data-key="lib-add"]');
await page.waitForSelector('[data-key="lib-node-BrokenRoll"]', { timeout: 60_000 });
await readOut();
const broken = await counts();
step('a folder the walk could not open keeps its row', broken['BrokenRoll/Broken'] === '1', JSON.stringify(broken));
step(
'and nothing under it is dropped: silence is not absence',
(await keys('photos')).includes('BrokenRoll/Broken/x.JPG'),
JSON.stringify(await keys('photos'))
);
step(
'while the frame the walk did meet is stored',
(await keys('photos')).includes('BrokenRoll/P1010256.JPG'),
JSON.stringify(await keys('photos'))
);
// --- a folder the reading could not open, and a tab that goes away -----------
// Why the number beside a folder stays at nothing through update after update,
// which is the state the reader photographed. A roll is read for the first time
// and the tab goes while the reading is in it: the position it leaves behind must
// not say it walked a folder it could not open, because an update takes that
// position up — a folder filed as read is a folder no update opens again, and its
// number stays at nothing however many times the reader asks. The folder opens on
// the visit that takes the reading up, and the reading is there to open it.
const opens = () => page.evaluate(() => JSON.parse(sessionStorage.getItem('check.lists') ?? '{}'));
const opened = (m, name) => m[name] ?? 0;
await page.evaluate(async () => {
sessionStorage.removeItem('check.lists');
localStorage.setItem('check.roll', 'CutRoll');
localStorage.setItem('check.hold', '1');
// Nothing in hand: the frame is one this reading has to read itself, which is
// what the hold is holding — and a reading held on the frame it is reading is
// the reading a closed tab leaves behind.
const db = await new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
await new Promise((resolve, reject) => {
const tx = db.transaction('photos', 'readwrite');
tx.objectStore('photos').clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
});
});
await page.click('[data-key="lib-add"]');
await page.waitForSelector('[data-key="lib-node-CutRoll"]', { timeout: 60_000 });
await counting();
await new Promise((r) => setTimeout(r, 1500));
const askedBroken = opened(await opens(), 'Broken');
const held = await page.evaluate(async () => {
const root = await navigator.storage.getDirectory();
try {
const dir = await root.getDirectoryHandle('walk', { create: false });
const fh = await dir.getFileHandle(`${encodeURIComponent('CutRoll')}.json`, { create: false });
return JSON.parse(await (await fh.getFile()).text());
} catch {
return null;
}
});
step(
'the position a closed tab leaves files no folder it could not open as walked',
!!held && !held.walked.some((r) => r.startsWith('Broken')) && held.pending.includes('Broken'),
JSON.stringify(held && { walked: held.walked, pending: held.pending })
);
// The tab goes, and the roll it cut off is offered on the visit that comes back:
// the offer is pressed, and the reading it takes up is the reading that opens the
// folder the last one could not.
await page.evaluate(() => localStorage.setItem('check.broken', 'open'));
await page.reload();
await page.waitForSelector('[data-key="lib-resume"]', { timeout: 60_000 });
await page.click('[data-key="lib-resume"]');
const again = await counting();
await page.evaluate(() => localStorage.removeItem('check.hold'));
await readOut();
const restarted = await counts();
step(
'and the visit that takes the reading up is the visit that opens it',
opened(await opens(), 'Broken') > askedBroken,
`Broken asked ${askedBroken} times, then ${opened(await opens(), 'Broken')}`
);
step(
'so the folder is counted on the disk, not left at nothing',
restarted['CutRoll'] === '2' && restarted['CutRoll/Broken'] === '1',
JSON.stringify(restarted)
);
// --- a roll picked at a drive root ------------------------------------------
// The reader picks a whole drive and every number beside a folder name stands at
// nought. The picker hands that folder over named `\`, and a row's number is
// looked up by the path that name spells: swapped for a bare backslash the roll
// is `/` and its folders `//Roll`, while the walk files its frames — and writes
// its counts — under the empty root a `normPath` fold leaves behind. Two
// spellings of one folder, and a column of noughts over a roll full of frames.
await page.evaluate(async () => {
sessionStorage.removeItem('check.lists');
localStorage.removeItem('check.hold');
localStorage.setItem('check.roll', '\\');
const db = await new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
await new Promise((resolve, reject) => {
const tx = db.transaction('photos', 'readwrite');
tx.objectStore('photos').clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
});
});
await page.click('[data-key="lib-add"]');
await page.waitForSelector('[data-key="lib-node-/Roll"]', { timeout: 60_000 }).catch(() => {});
// The reading runs out on its own, and silence is the reading being through.
await page
.waitForFunction(() => !(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim(), null, {
timeout: 120_000,
polling: 50,
})
.catch(() => {});
const drive = await counts();
step(
'a roll picked at a drive root counts its frames beside its own name',
drive[''] === '2' && drive['/Roll'] === '1',
JSON.stringify(drive)
);
// --- the shelf, out of a reading ---------------------------------------------
// The reading hands the screen the frames it found a batch at a time, and a batch
// used to be handed on only when it was full: the reader's folder of 26 RAWs is
// under one batch, so the screen stood empty for the whole of the reading and then
// filled at once, and the tiles had nothing to be drawn from until the reading was
// over. Frames go down on the clock as well now, so the count climbs and the first
// tiles appear while the roll is still being read. Sixty frames read at a walking
// pace is a long roll read the way the reader read theirs.
await page.evaluate(async () => {
sessionStorage.removeItem('check.lists');
localStorage.removeItem('check.hold');
localStorage.setItem('check.slow', '150');
localStorage.setItem('check.roll', 'BigRoll');
const db = await new Promise((resolve, reject) => {
const req = indexedDB.open('recipescam-library');
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
await new Promise((resolve, reject) => {
const tx = db.transaction(['photos', 'thumbs'], 'readwrite');
tx.objectStore('photos').clear();
tx.objectStore('thumbs').clear();
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
});
});
await page.click('[data-key="lib-add"]');
await page.waitForSelector('[data-key="lib-node-BigRoll"]', { timeout: 60_000 });
await page
.waitForFunction(() => (document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim().length > 0, null, {
timeout: 60_000,
polling: 25,
})
.catch(() => {});
// The reading is in flight and the frames are being read: what is asked of it here
// is whether the screen is told of them as they land. A roll under one `BATCH`
// reached the screen in a single lump at the end of the reading, however long the
// reading was — the reader's folder of 26 RAWs was one batch, and so one empty
// screen for the whole of it. Every count between none and all is a reading that
// said something on the way past.
const landed = [];
for (let i = 0; i < 120; i++) {
const reading = await page
.$eval('[data-key="lib-progress"]', (e) => (e.textContent ?? '').trim())
.catch(() => '');
const listed = Number(
(await page.$eval('[data-key="lib-listed"]', (e) => e.textContent ?? '').catch(() => '0')).replace(/\D+/g, '')
);
if (reading && listed > 0) landed.push(listed);
if (!reading) break;
await new Promise((r) => setTimeout(r, 150));
}
const seen = [...new Set(landed)];
step(
'the frames the reading lands are on screen as it lands them',
seen.length >= 2,
`${seen.length} counts while the reading ran: ${seen.join(', ') || 'none'}`
);
await page
.waitForFunction(() => !(document.querySelector('[data-key="lib-progress"]')?.textContent ?? '').trim(), null, {
timeout: 120_000,
polling: 25,
})
.catch(() => {});
// The frames the wall draws are read one behind another and the tiles land in
// starts and stops, so the count is taken once it has stood still for a while.
let atEnd = 0;
for (let i = 0, same = 0; i < 160 && same < 10; i++) {
await new Promise((r) => setTimeout(r, 250));
const n = (await keys('thumbs')).length;
same = n === atEnd ? same + 1 : 0;
atEnd = n;
}
step('the wall tiles the frames a reading brought it', atEnd >= 10, `${atEnd} tiles of ${LONG} drawn`);
await page.evaluate(() => localStorage.removeItem('check.slow'));
await browser.close();
app.kill('SIGTERM');
console.log(failures ? `\n${failures} failed` : '\nall checks passed');
process.exit(failures ? 1 : 0);
docker/frontend/src/Library.tsx
+178 -43
View File
@@ -19,6 +19,7 @@ import {
jumpScan,
listEditedIds,
listFolders,
listWalkedDirs,
makeTile,
normPath,
onLibraryBlocked,
@@ -260,6 +261,11 @@ export function Library() {
const [state, setState] = useState<'loading' | 'guest' | 'ready'>(
() => (recall(ACCOUNT_KEY) as 'guest' | 'ready' | null) ?? 'loading'
);
// Whether the account on this browser may repair the catalogue: the API is the
// one that decides, and this only drives the button. Read from `me` below, and
// false until it answers — a control that is not there for a moment is not a
// control that was offered and taken back.
const [admin, setAdmin] = useState(false);
useEffect(() => {
let alive = true;
@@ -270,6 +276,7 @@ export function Library() {
const next = me?.user ? 'ready' : 'guest';
keep(ACCOUNT_KEY, next);
setState(next);
setAdmin(Boolean(me?.user?.admin));
})
// No answer is treated like no account: the API being away is not a
// reason to open the shelves to whoever is on the page. It is also not an
@@ -293,10 +300,10 @@ export function Library() {
</a>
</p>
);
return <Catalogue />;
return <Catalogue admin={admin} />;
}
function Catalogue() {
function Catalogue({ admin }: { admin: boolean }) {
const { t } = useI18n();
const [folders, setFolders] = useState<LibraryFolder[]>(() => cachedFolders());
const [ready, setReady] = useState<Set<string>>(new Set());
@@ -304,11 +311,13 @@ function Catalogue() {
// studio is a screen opening, not a page loading, and the frames the reader was
// looking at are already in hand. The read below replaces them a moment later.
const [photos, setPhotos] = useState<LibraryPhoto[]>(() => cachedPhotos());
// The folders the scan in flight has walked into: a roll's own shape is on
// screen from the first pass, before the frames under its folders are read.
// What a scan found is not filed away — the next visit draws the folders that
// hold frames, and the next scan names the rest again.
// The folders a reading has walked into, kept with the catalogue: a roll's own
// shape is on screen the moment the catalogue is, and a reading that was cut
// off leaves the names of what it found behind it. `walked` is what those names
// counted — the frames under each, off the disk as the walk last saw it, which
// is the one number that can go down.
const [dirs, setDirs] = useState<LibraryDir[]>([]);
const [walked, setWalked] = useState<Record<string, number>>({});
const [edited, setEdited] = useState<Set<string>>(new Set());
// Which tree node is open, which frame is up, and which of the two middle
// views (the frame, or every thumbnail of the open node) is on. The open node
@@ -420,12 +429,37 @@ function Catalogue() {
const [backBusy, setBackBusy] = useState(false);
const reload = useCallback(async () => {
const [rows, edits] = await Promise.all([readPhotos(), listEditedIds()]);
const [rows, edits, walkedRows] = await Promise.all([readPhotos(), listEditedIds(), listWalkedDirs()]);
// A read that came back with nothing is a read that failed, not a catalogue
// that emptied — and the frames it would have cleared are frames the reader
// was working their way through. The strip keeps them.
if (rows) setPhotos(rows);
setEdited(edits);
// What the readings have walked into, for the column. Lower-cased on the way
// in because that is how a row looks a number up, and the record holds both
// spellings of every key.
const counts: Record<string, number> = {};
for (const row of walkedRows) {
for (const [key, n] of Object.entries(row.counts)) counts[key.toLowerCase()] = n;
}
setWalked(counts);
// Folded into whatever the screen already has rather than put in its place: a
// folder the reader picked inside a roll is on the column from the click, and
// a reading in flight has named folders the record has not been written with
// yet. Both are rows, and neither is one to take away.
setDirs((prev) => {
const map = new Map(prev.map((d) => [d.id, d]));
let changed = false;
for (const row of walkedRows) {
for (const d of row.dirs) {
if (!map.has(d.id)) {
map.set(d.id, d);
changed = true;
}
}
}
return changed ? [...map.values()] : prev;
});
}, []);
// --- the catalogue's own copy ---------------------------------------------
@@ -786,7 +820,10 @@ function Catalogue() {
// for, and a request called off before it ran says nothing at all.
const scan = useCallback(
async (folder: LibraryFolder, quiet = false, from = '', mode: ScanMode = 'update') => {
setNote(null);
// A quiet reading is one nobody asked to hear about — the repair's own,
// whose note says the whole of what it is doing — so it leaves the note
// where it found it. A loud one clears it: its word is the one that lands.
if (!quiet) setNote(null);
try {
const result = await scanAsked(folder, from, mode);
if (!result) return;
@@ -804,18 +841,47 @@ function Catalogue() {
[reload, t]
);
// The folder this screen is on is read back to the disk on the way in, and again
// every time the tab is raised — but only when the last reading of it never
// finished. A reading that ran to its end has nothing left to find, and the
// reader who opens the library to choose a frame is not a reader who asked for
// their disk walked behind their back: on the folder they are opening, a walk
// is a hundred thousand names read off the disk and the band and the memory
// that goes with them, which is the whole of what this screen must not cost.
// A folder the reader wants looked at again says so itself, from the menu on
// its row; what is picked up here is the roll whose reading ended when the app
// did, and nothing else. No note either way: the ring on the row and the
// progress line are the report.
// The repair: every roll in the catalogue read back to the disk, one after the
// other, from the top. A reading is the only thing that knows what a folder
// holds now — it drops the rows the disk has nothing for, with their tiles and
// their marks, and draws every number off its own walk — so the repair of a
// catalogue that has drifted from the disk is exactly a rescan of everything in
// it. Frames that have not moved are skipped on their size and their time, so
// the cost is the disk's own listing and not a develop a frame.
const repair = useCallback(async () => {
if (!folders.length) return;
if (!window.confirm(t('lib.repairConfirm', { n: folders.length }))) return;
setNote(t('lib.repairRun', { n: folders.length }));
let done = 0;
for (const folder of folders) {
// One roll's failure is one roll's: a handle the browser will not hand
// back, a drive that is not there — the rest of the repair still runs.
try {
await scan(folder, true, '', 'rescan');
done++;
} catch {
// The roll keeps what the last reading that could open it left.
}
}
await reload();
setNote(t('lib.repairDone', { n: done }));
}, [folders, reload, scan, t]);
// A reading that ran to its end has nothing left to find, and the reader who
// opens the library to choose a frame is not a reader who asked for their disk
// walked behind their back: a walk of the folder they are opening is a hundred
// thousand names read off the disk and the band and the memory that goes with
// them, which is the whole of what this screen must not cost. So nothing walks
// on the way in. What this screen does instead is go looking — on the way in,
// and again every time the tab is raised — for the one roll whose reading ended
// when the app did, and offer it: the chip by the progress line names the folder
// and one press reads it on from where it stopped. A reading the reader wants of
// a folder whose last one ran to its end is still the menu on that folder's row;
// what the chip picks up is only the roll that was cut off mid-walk.
const rootKey = node ? node.split('/')[0] : null;
// The roll the screen found unfinished, named on the chip beside the progress
// line. It is what stands where the walk it used to run by itself used to be.
const [resume, setResume] = useState<string | null>(null);
// When the folder was last walked for being raised. It stands outside the
// effect because the effect is rebuilt for the folder the reader moves to, and
// the walk it guards is of the folder on the disk, where nothing has moved
@@ -831,11 +897,13 @@ function Catalogue() {
if (Date.now() - synced.current < CATCHUP_MS) return;
synced.current = Date.now();
void scanBusy().then(async (busy) => {
if (busy || scanSession()) return;
// A roll already offered is a roll already looked for: asking the disk
// again every time the tab is raised is the walk this screen just gave up.
if (busy || scanSession() || resume) return;
for (const folder of folders) {
if (!ready.has(folder.name)) continue;
if (await unfinished(folder.name)) {
void scan(folder, false);
setResume(folder.name);
break;
}
}
@@ -847,7 +915,7 @@ function Catalogue() {
};
document.addEventListener('visibilitychange', raised);
return () => document.removeEventListener('visibilitychange', raised);
}, [loaded, folders, ready, scan]);
}, [loaded, folders, ready, scan, resume]);
const add = useCallback(async () => {
setNote(null);
@@ -963,10 +1031,10 @@ function Catalogue() {
const rootFolder = useMemo(() => {
if (!node) return folders[0] ?? null;
const normNode = (node || '').replace(/\\/g, '/');
const normNode = normPath(node);
return (
folders.find((f) => {
const normF = (f.name || '').replace(/\\/g, '/');
const normF = normPath(f.name);
return normNode === normF || normNode.startsWith(`${normF}/`);
}) ?? folders[0] ?? null
);
@@ -1007,8 +1075,13 @@ function Catalogue() {
const hits = new Map<string, number>();
const foldersUnder = new Map<string, Set<string>>();
const add = (folder: string, rel: string, counted: boolean) => {
const normF = (folder || '').replace(/\\/g, '/');
const normRel = (rel || '').replace(/\\/g, '/');
// `normPath`, not a bare backslash swap: a picked drive root is named `\`,
// and `normPath` folds that to the empty root the walk files its frames and
// writes its counts under. Swapped only, the root would spell `/` here and
// `//sub` on the row keys below — two spellings of a folder the walk has
// never heard of, and a number every row would read as nought.
const normF = normPath(folder);
const normRel = normPath(rel);
const lowerF = normF.toLowerCase();
const set = foldersUnder.get(lowerF) ?? new Set<string>();
foldersUnder.set(lowerF, set);
@@ -1038,16 +1111,24 @@ function Catalogue() {
const nodes = useMemo<Node[]>(() => {
const { hits, foldersUnder } = baseTreeInfo;
// What a row says: the reading in flight, then what the reading that walked
// this roll found, then the catalogue. The reading first because it is the
// newest thing there is; the walk second because the catalogue is not a
// record of the disk — a frame taken off it is a row nothing takes out — and a
// number read off it is one that can only ever climb; the catalogue last,
// which is all a roll that has never been walked has.
const counted = (key: string) => {
const normKey = (key || '').replace(/\\/g, '/');
const normKey = normPath(key);
const lowerKey = normKey.toLowerCase();
const hitVal = hits.get(lowerKey) ?? 0;
const readVal = read?.[normKey] ?? read?.[lowerKey] ?? read?.[key] ?? 0;
return Math.max(hitVal, readVal);
const readVal = read?.[normKey] ?? read?.[lowerKey] ?? read?.[key];
if (readVal !== undefined) return readVal;
const walkVal = walked[lowerKey];
if (walkVal !== undefined) return walkVal;
return hits.get(lowerKey) ?? 0;
};
const out: Node[] = [];
for (const folder of folders) {
const normF = (folder.name || '').replace(/\\/g, '/');
const normF = normPath(folder.name);
const lowerF = normF.toLowerCase();
out.push({ key: normF, label: folder.label || folder.name, depth: 0, count: counted(normF) });
// Path order puts a folder before its own children, which is the order the
@@ -1064,7 +1145,7 @@ function Catalogue() {
}
}
return out;
}, [baseTreeInfo, folders, read]);
}, [baseTreeInfo, folders, read, walked]);
// A row with anything under it is the one that folds: its key is the prefix of
// its children's, in the path order the rows come in.
@@ -1159,11 +1240,20 @@ function Catalogue() {
const below = dir === ''
? (p: LibraryPhoto) => normPath(p.dir ?? '') !== ''
: (p: LibraryPhoto) => normPath(p.dir ?? '').startsWith(`${dir}/`);
return photos.filter((p) => {
const pFolder = normPath(p.folder ?? '');
const own: LibraryPhoto[] = [];
const both: LibraryPhoto[] = [];
for (const p of photos) {
if (normPath(p.folder ?? '') !== normRoot) continue;
const pDir = normPath(p.dir ?? '');
return pFolder === normRoot && (pDir === dir || (deep && below(p)));
});
const isOwn = pDir === dir;
if (isOwn) own.push(p);
if (isOwn || below(p)) both.push(p);
}
// A row counts what sits at or under its folder, so a folder whose frames all
// lie one level down would open on an empty screen under a number that says
// otherwise. The branch is drawn then, and only then: a folder holding frames
// of its own opens on those, and the toggle keeps the last word.
return deep || !own.length ? both : own;
}, [deep, node, photos, rootName]);
// What the thumbnail view draws, out of what the open node holds. A frame's
@@ -1670,6 +1760,23 @@ function Catalogue() {
>
{t('lib.restore')}
</button>
{/* The repair, for the account the API calls an admin: the catalogue
this screen draws is the browser's own, and a catalogue that has
drifted — counts under folders the disk emptied, rows for frames
that are gone — is put back by reading every roll to the disk
again. It is the one control here that is a long job by design,
so it asks first and it says how far it got. */}
{admin ? (
<button
type="button"
className="btn"
data-key="lib-repair"
disabled={backBusy || Boolean(live) || scanQueue().length > 0}
onClick={() => void repair()}
>
{t('lib.repair')}
</button>
) : null}
{/* The folder is a control, not a caption: it is the whole of what
tells one backup from another, and pressing it is how a folder
is picked, a name given, a permission handed back. */}
@@ -1698,11 +1805,28 @@ function Catalogue() {
roll of nothing over nothing is the line talking before the
first pass has come back, which is a scan of a hundred
thousand frames announced as `0/0`. */}
<span className="hint" data-key="lib-progress">
{live && live.progress.total
? t('lib.scanning', { done: live.progress.done, total: live.progress.total, added: live.progress.added })
: ''}
</span>
{live && live.progress.total ? (
<span className="hint" data-key="lib-progress">
{t('lib.scanning', { done: live.progress.done, total: live.progress.total, added: live.progress.added })}
</span>
) : resume && !live ? (
// The reading that was cut off when the app closed, kept in the
// reader's own time: one press picks it up where it stopped. It
// says nothing while a reading runs — the line above is the report.
<button
type="button"
className="chip"
data-key="lib-resume"
title={t('lib.resumeHint')}
onClick={() => {
const folder = folders.find((f) => f.name === resume);
setResume(null);
if (folder) void scan(folder, false);
}}
>
<span className="chip-label">{t('lib.resume', { folder: resume })}</span>
</button>
) : null}
<span className="adm-spacer" />
{folders.length > 0 ? (
<>
@@ -2625,6 +2749,17 @@ function useThumbUrls(photos: LibraryPhoto[], also?: LibraryPhoto | null): Recor
if (idsToFetch.length === 0) return;
// The strip draws one square whatever the frame behind it, so the preview a RAW
// or a HEIC hands back — a whole develop, seconds of it — is shrunk to the
// wall's size the one time it is paid for and then kept: every later visit
// reads the tile back instead of the file.
const tileOf = async (id: string, bytes: Uint8Array) => {
const preview = new Blob([bytes as BlobPart], { type: 'image/jpeg' });
const made = (await makeTile(preview)) ?? preview;
void putPhotoThumb(id, made);
return made;
};
void getPhotoThumbs(idsToFetch).then(async (thumbMap) => {
if (!alive) return;
const updates: Record<string, string> = {};
@@ -2639,12 +2774,12 @@ function useThumbUrls(photos: LibraryPhoto[], also?: LibraryPhoto | null): Recor
const fullBytes = new Uint8Array(await file.arrayBuffer());
let prevBytes = await heicThumbnail(fullBytes);
if (!prevBytes) prevBytes = await heicToJpeg(fullBytes, 320);
if (prevBytes) blob = new Blob([prevBytes as BlobPart], { type: 'image/jpeg' });
if (prevBytes) blob = await tileOf(photo.id, prevBytes);
} else if (isRawName(name) || /\.(tiff?)$/i.test(name)) {
const fullBytes = new Uint8Array(await file.arrayBuffer());
let prevBytes = await rawThumbnail(fullBytes, name);
if (!prevBytes && /\.(tiff?)$/i.test(name)) prevBytes = tiffThumbnail(fullBytes);
if (prevBytes) blob = new Blob([prevBytes as BlobPart], { type: 'image/jpeg' });
if (prevBytes) blob = await tileOf(photo.id, prevBytes);
} else {
// A frame the wall can decode itself: the tile is made here, at the
// size the wall paints, because this is the first time anyone has
docker/frontend/src/api.ts
+38 -8
View File
@@ -2,6 +2,8 @@
// /api), the Vite dev server proxies it too — so no base URL, no CORS.
import type { Recipe } from '../shared/types';
import { shrinkForUpload, resizedJpeg } from './engine/imageOps';
import { forgetStale, hold, keepStale, serverAnswered, serverGone, stale } from './pwa/offline';
import { t } from './i18n/I18nProvider';
// Upload caps. The API + nginx both refuse oversized bodies, so shrink in the
// browser first; the server then sniffs the bytes and requires the declared
@@ -157,14 +159,38 @@ export interface Stats {
}
async function call<T>(path: string, init?: RequestInit): Promise<T> {
const res = await fetch(`/api${path}`, {
credentials: 'same-origin',
headers: init?.body ? { 'content-type': 'application/json' } : undefined,
...init,
});
// A read is answered from what the server last said. A write is not answered
// at all when no server is there — pretending it landed is a lie the screen
// would act on — but it is not lost either: it is held, in order, and sent
// when one answers. See `hold`/`flush` in pwa/offline.ts.
const read = !init?.method || init.method === 'GET';
let res: Response;
try {
res = await fetch(`/api${path}`, {
credentials: 'same-origin',
headers: init?.body ? { 'content-type': 'application/json' } : undefined,
...init,
});
} catch (err) {
// No answer at all: the server is gone. Serve the last one it gave for this
// read, or fail as before — the caller has its own empty state.
serverGone();
// A write goes on the queue instead. The one exception is the sole call that
// is not about the server's own data: `open-explorer` asks it to pop a
// folder on this machine, and popping it minutes later is not popping it.
if (!read && path !== '/open-explorer') {
hold(path, init?.method ?? 'POST', typeof init?.body === 'string' ? init.body : null);
throw new Error(t('offline.queued'));
}
const staleOne = read ? await stale<T>(path) : null;
if (staleOne !== null) return staleOne;
throw err;
}
serverAnswered();
if (res.status === 204) return undefined as T;
const body = await readJson(res);
if (!res.ok) throw new Error(body.error ?? `HTTP ${res.status}`);
if (read) keepStale(path, body);
return body as T;
}
@@ -222,10 +248,14 @@ export const api = {
// null user = signed out; the API answers 200 either way.
me: () => call<{ user: User | null }>('/auth/me'),
signup: (email: string, password: string) =>
call<{ user: User }>('/auth/signup', { method: 'POST', body: JSON.stringify({ email, password }) }),
call<{ user: User }>('/auth/signup', { method: 'POST', body: JSON.stringify({ email, password }) })
// The answers held offline are the previous account's and the next one must
// not be shown them; this browser is not an account until it has one.
.finally(forgetStale),
login: (email: string, password: string) =>
call<{ user: User }>('/auth/login', { method: 'POST', body: JSON.stringify({ email, password }) }),
logout: () => call<void>('/auth/logout', { method: 'POST' }),
call<{ user: User }>('/auth/login', { method: 'POST', body: JSON.stringify({ email, password }) })
.finally(forgetStale),
logout: () => call<void>('/auth/logout', { method: 'POST' }).finally(forgetStale),
// Mail the verification link to the signed-in address again. Works while
// unverified (that is the whole point); 429 once the hourly cap is spent.
resendVerification: () => call<{ ok: boolean; verified?: boolean }>('/auth/resend-verification', { method: 'POST' }),
docker/frontend/src/engine/library.ts
+356 -46
View File
@@ -23,7 +23,8 @@ const FOLDERS = 'folders';
const PHOTOS = 'photos';
const THUMBS = 'thumbs';
const EDITS = 'edits';
const DB_VERSION = 2;
const DIRS = 'dirs';
const DB_VERSION = 3;
// The tile the catalogue keeps is the same tile the grid cell and the strip both
// paint, and there is one of them per frame of a roll that reaches six figures.
@@ -38,6 +39,15 @@ const THUMB_QUALITY = 0.8;
// Frames go down in batches, so a 3000-file folder is 60 transactions rather
// than 3000 of them.
const BATCH = 50;
// A batch that only goes down when it is full is a wall that stands empty for the
// whole of a reading and then fills in one go: the reader's folder of 26 RAWs is
// under one batch, so nothing of it reached the screen until the reading was over.
// The clock is the other end of the batch — whatever is in hand goes down once this
// long has passed since the last time, so the count climbs and the first tiles are
// drawn while the roll is still being read.
// ponytail: 400ms of the reading's own clock, no timer to start or stop; a roll of
// twenty thousand frames is fifty extra transactions on a reading of minutes.
const FLUSH_MS = 400;
// Frames read at once. Reading a frame is nearly all waiting — the bytes come off
// the disk, the decode runs on a thread of its own — and a wait that is not spent
// on the next frame is time the roll does not get back. Past a few, the disk and
@@ -236,6 +246,7 @@ function openDb(): Promise<IDBDatabase> {
db.createObjectStore(THUMBS, { keyPath: 'id' });
}
if (!db.objectStoreNames.contains(EDITS)) db.createObjectStore(EDITS, { keyPath: 'photoId' });
if (!db.objectStoreNames.contains(DIRS)) db.createObjectStore(DIRS, { keyPath: 'folder' });
};
req.onsuccess = () => {
blockedSink?.(false);
@@ -413,9 +424,11 @@ export async function removeFolder(name: string): Promise<void> {
const db = await openDb();
await new Promise<void>((resolve, reject) => {
const hasThumbs = db.objectStoreNames.contains(THUMBS);
const stores = hasThumbs ? [FOLDERS, PHOTOS, THUMBS, EDITS] : [FOLDERS, PHOTOS, EDITS];
const hasDirs = db.objectStoreNames.contains(DIRS);
const stores = [FOLDERS, PHOTOS, EDITS, ...(hasThumbs ? [THUMBS] : []), ...(hasDirs ? [DIRS] : [])];
const tx = db.transaction(stores, 'readwrite');
tx.objectStore(FOLDERS).delete(name);
if (hasDirs) tx.objectStore(DIRS).delete(name);
const photos = tx.objectStore(PHOTOS);
const thumbs = hasThumbs ? tx.objectStore(THUMBS) : null;
const edits = tx.objectStore(EDITS);
@@ -433,6 +446,192 @@ export async function removeFolder(name: string): Promise<void> {
await clearWalk(name);
}
// The frames a reading met are the frames the disk holds. The catalogue keeps a
// row for every frame it has ever seen and nothing ever took one out: a frame
// taken off the disk kept its row, and a folder with no reading of its own falls
// back on the catalogue for a number — a number that could only climb. And the
// row is not only a number: it is a tile the wall draws and a frame the strip
// offers, one whose file is not there. A reading that listed the whole of what
// it was pointed at is the one thing that knows the disk, so what it did not
// meet is dropped here — the row, its tile, and the marks on it, because a star
// or a rotation belongs to a frame that is there.
//
// `from` is the folder the reading was pointed at, '' for the whole roll; `met`
// is every frame it found, spelled relative to the roll.
async function prunePhotos(folder: string, from: string, met: Set<string>): Promise<void> {
try {
const root = normPath(folder);
const under = (from ? `${root}/${normPath(from)}` : root).toLowerCase();
const rows = (await listPhotos(folder)).filter((p) => {
const rel = (p.dir ? `${normPath(p.dir)}/${p.name}` : p.name).toLowerCase();
const key = `${root.toLowerCase()}/${rel}`;
if (from && key !== under && !key.startsWith(`${under}/`)) return false;
return !met.has(rel);
});
if (!rows.length) return;
const gone = new Set(rows.map((p) => p.id));
if (cached) cached = cached.filter((p) => !gone.has(p.id));
const db = await openDb();
const hasThumbs = db.objectStoreNames.contains(THUMBS);
await new Promise<void>((resolve, reject) => {
const tx = db.transaction([PHOTOS, EDITS, ...(hasThumbs ? [THUMBS] : [])], 'readwrite');
const photos = tx.objectStore(PHOTOS);
const edits = tx.objectStore(EDITS);
const thumbs = hasThumbs ? tx.objectStore(THUMBS) : null;
for (const p of rows) {
photos.delete(p.id);
edits.delete(p.id);
thumbs?.delete(p.id);
thumbCache.delete(p.id);
}
tx.oncomplete = () => resolve();
tx.onerror = () => reject(tx.error);
});
} catch {
// A repair that failed leaves the reading's own word standing: the counts
// are written either way, and the rows it did not meet are the next
// reading's to drop.
}
}
// --- what the walk found ----------------------------------------------------
// The folders a reading walked into, and how many frames it met under each, kept
// where the catalogue is kept. The column is drawn from this list, and a list
// that lived only for the length of the reading took the rows with it: the reader
// came back to a roll missing every folder that held no frame the catalogue had —
// a folder of frames the reading never reached, and an empty folder alike — and
// had to walk their parent a second time to get the names back. The numbers come
// from the walk for the same reason: a folder the reading never reached is a
// folder whose size the catalogue cannot say, and the walk is the only thing that
// can.
//
// `counts` is the reading's own map, folded to one spelling per path, which is
// how the screen looks a number up.
export interface WalkedDirs {
// The picked folder, by name.
folder: string;
// Every folder the walk has been into, `rel` spelled the way the walk spells
// it: `''` for the roll itself is not a row, so it is not one here.
dirs: LibraryDir[];
// Folder path in the reading's own spelling, lower-cased on the way in, to the
// number of frames the walk met under it.
counts: Record<string, number>;
}
export async function listWalkedDirs(): Promise<WalkedDirs[]> {
try {
const rows = await ask<WalkedDirs[]>(DIRS, 'readonly', (s) => s.getAll());
return rows.map((r) => ({ folder: r.folder, dirs: r.dirs ?? [], counts: r.counts ?? {} }));
} catch {
return [];
}
}
// What the walk found, written down where the screen can pick it up. A reading
// kept to one folder knows the folders under that one and nothing of the rest of
// the roll, so it is folded into what is already there; a reading of the whole
// roll is the roll, and what it did not find is not there any more.
export async function saveWalkedDirs(
folder: string,
dirs: LibraryDir[],
counts: Record<string, number> | null,
replace: boolean,
from = '',
// The folders the reading asked to open and could not, spelled the way the walk
// spells them. A reading has no word on these, and a nought beside the name of
// one of them is a word it does not have.
unlisted?: Iterable<string>
): Promise<void> {
try {
const root = normPath(folder).toLowerCase();
// One spelling to a path: the reading's own map holds every key twice, as the
// walk spelled it and in lower case, and a branch counted off both is twice
// its own size. Folded here once, and the record keeps the one spelling the
// screen looks a number up by.
const fold = (m: Record<string, number>): Record<string, number> => {
const out: Record<string, number> = {};
for (const [k, n] of Object.entries(m)) out[k.replace(/\\/g, '/').toLowerCase()] = n;
return out;
};
// The folders this reading was in, before another visit's are folded in.
const walked = dirs.slice();
if (!replace) {
const prev = (await ask<WalkedDirs | undefined>(DIRS, 'readonly', (s) => s.get(folder))) ?? null;
const byId = new Map((prev?.dirs ?? []).map((d) => [d.id, d]));
for (const d of dirs) byId.set(d.id, d);
dirs = [...byId.values()];
if (!counts) {
counts = fold(prev?.counts ?? {});
} else {
const was = fold(prev?.counts ?? {});
const now = fold(counts);
// A reading kept to one folder counts the frames under it and says
// nothing of the rest of the roll — but the rows above it hold those
// frames too. Written in as they came, the branch would move and its
// parents would not, a folder with more under it than the folder it sits
// in and a roll under its own tree. So the difference the branch came by
// is the difference each row above it comes by, and the reading's numbers
// take the place of the ones the last reading left under it — a branch
// the reader emptied is a branch this reading met nothing in.
const sub = from ? `${root}/${normPath(from).toLowerCase()}` : root;
const under = (k: string) => k === sub || k.startsWith(`${sub}/`);
const sum = (m: Record<string, number>) => {
let n = 0;
for (const [k, v] of Object.entries(m)) if (under(k)) n += v;
return n;
};
const delta = sum(now) - sum(was);
const merged: Record<string, number> = {};
for (const [k, v] of Object.entries(was)) if (!under(k)) merged[k] = v;
Object.assign(merged, now);
for (let anc = sub; delta && anc.length > root.length; ) {
anc = anc.slice(0, anc.lastIndexOf('/'));
merged[anc] = (merged[anc] ?? 0) + delta;
}
counts = merged;
}
}
if (counts) {
// A folder the reading was in and met no frame under is a folder with
// nothing under it: left out of the record, the row falls back on the
// catalogue, and a frame taken off the disk keeps its number for good.
const key = (rel: string) => (rel ? `${root}/${normPath(rel)}` : root).toLowerCase();
// A folder the reading could not open is a folder it has no word on, and
// nothing — not even nothing — is what a reading with no word on a folder
// may write beside its name. Written in as a nought it takes the place of
// the number the catalogue holds, and a column drawn off a drive that
// hiccupped once reads noughts over folders full of frames until the reader
// rescans the roll. Left out, the row falls back on the catalogue, which is
// what the last reading that could open it found.
const unsaid = new Set([...(unlisted ?? [])].map((r) => normPath(r).toLowerCase()));
for (const d of walked) {
if (unsaid.has(normPath(d.rel).toLowerCase())) continue;
counts[key(d.rel)] ??= 0;
}
if (replace) counts[root] ??= 0;
counts = fold(counts);
}
await ask(DIRS, 'readwrite', (s) => s.put({ folder, dirs, counts: counts ?? {} } satisfies WalkedDirs));
} catch {
// A catalogue that will not keep them is a column drawn from the catalogue, as
// it was before.
}
}
// Every record of what a walk found, dropped: the catalogue that was just put
// back is the one the reader asked for, and a number read off a disk the backup
// does not describe is not a number to draw under it. Everything is forgotten
// because a restore replaces the whole catalogue; what is left is a roll whose
// next reading names it again.
export async function forgetWalkedDirs(): Promise<void> {
try {
await ask(DIRS, 'readwrite', (s) => s.clear());
} catch {
// Nothing to forget.
}
}
// A handle kept in IndexedDB comes back without its permission: the browser
// dropped it when the tab closed, and only the visitor can hand it back. Asking
// costs one quiet prompt, which is why this runs at start-up rather than the
@@ -628,6 +827,11 @@ interface WalkSaved {
// The frames the walk has found and the reading has not got to. The front of
// this is the frame a reloaded reading picks up at.
frames: string[];
// Written by the builds that kept only the front of the queue: the count of
// frames in hand when the position went down. `frames` was a slice of them, so
// a position with more in hand than it wrote cannot say where the rest were —
// see `loadWalk`. No reading of this build writes it.
unread?: number;
// How many of the front of `frames` were already counted as read by the reading
// that wrote this down: the rows are in hand and not in the catalogue yet, so
// the next reading reads them again — and counts none of them twice.
@@ -646,13 +850,22 @@ async function loadWalk(folder: string): Promise<WalkSaved | null> {
const fh = await walkFile(folder, false);
if (!fh) return null;
const saved = JSON.parse(await (await fh.getFile()).text()) as WalkSaved;
return Array.isArray(saved?.walked) &&
const ok = Array.isArray(saved?.walked) &&
Array.isArray(saved?.pending) &&
Array.isArray(saved?.frames) &&
Number.isInteger(saved?.restored) &&
saved.progress
? saved
: null;
saved.progress;
if (!ok) return null;
// The queue was cut off when this was written down by an older build: it knew
// where the first few hundred unread frames were and nothing of the rest, and
// a walk that takes it up walks the folders it lists as already walked — an
// update over a roll with a few thousand frames left in it reads the queue out
// and calls the roll read, never reaching the frames past the cut. A position
// that cannot say where the reading had got to is no position: the reading
// starts from the top and skips the frames it knows on their size and their
// time. A position this build wrote holds the whole queue and is always taken.
if (saved.unread !== undefined && saved.unread > saved.frames.length) return null;
return saved;
} catch {
return null;
}
@@ -744,6 +957,12 @@ export async function scanFolder(
}
}
const tree: Walk = { root: folder.handle, pending: [], walked: treeWalked };
// Whether this reading took up a position, read here and not after the walk:
// `tree.walked` is this same set and the walk fills it as it goes, so a reading
// that started from the top and one that started from a position look alike
// once either has been through the roll — and the difference is exactly what a
// reading may say about folders it never listed. See `prunePhotos`.
const resumed = treeWalked.size > 0;
// A queue comes back as paths, so the folders are asked for again; a reading
// with nothing written down starts at the folder it was pointed at — the
// picked one, or the one the reader right-clicked inside it.
@@ -791,47 +1010,61 @@ export async function scanFolder(
progress.at ??= '';
const found = new Set(progress.dirs.map((d) => d.id));
const normRoot = normPath(folder.name);
// The rows are counted from what the catalogue already holds, and the reading
// only adds to them. A number that started at nothing would sit under the
// catalogue's own for as long as the reading took to pass it, and the screen
// shows the larger of the two — so an update, where the reading is mostly
// frames the catalogue has, is a number that stands still and then jumps. What
// the reading adds is the frames it found that the catalogue does not hold yet:
// the number climbs for news alone, and ends on the true total.
const counts: Record<string, number> = {};
// The rows are counted from what the walk meets, and from nothing else. The
// catalogue used to be counted in first, so an update over a roll it already
// held wore the catalogue's own numbers for the whole of the reading — and a
// catalogue only ever grows: a frame taken off the disk is a row nothing takes
// out, so a folder the reader emptied kept the number it had, and every scan
// after that drew a count the disk no longer had. The walk meets every frame of
// the roll and is the only thing that knows what is there now; what it counted
// is what the column says. A reading that takes up a position starts from the
// numbers that position was written with: the folders it walked before this
// visit are folders it does not walk again, and a number that started at
// nothing would leave their frames out of the count for good.
const counts: Record<string, number> = { ...(saved?.progress.counts ?? {}) };
// A reading kept to one folder says nothing about the rest of the roll: its
// numbers drawn over the column would count a roll down to a branch of itself,
// and the branch below the folder it was pointed at is the whole of what it has
// walked. The row of the roll itself falls back on what the roll's own reading
// found, which is a number this one has no business standing under.
const fromPath = bounded ? normPath(from) : '';
const under = (path: string) => !bounded || path === fromPath || path.startsWith(`${fromPath}/`);
const count = (rel: string, n = 1) => {
const relNorm = rel.replace(/\\/g, '/');
const cut = relNorm.lastIndexOf('/');
counts[normRoot] = (counts[normRoot] ?? 0) + n;
counts[normRoot.toLowerCase()] = counts[normRoot];
if (!bounded) {
counts[normRoot] = (counts[normRoot] ?? 0) + n;
counts[normRoot.toLowerCase()] = counts[normRoot];
}
let path = '';
for (const part of (cut < 0 ? '' : relNorm.slice(0, cut)).split('/')) {
if (!part) continue;
path = path ? `${path}/${part}` : part;
if (!under(path)) continue;
const key = `${normRoot}/${path}`;
counts[key] = (counts[key] ?? 0) + n;
counts[key.toLowerCase()] = counts[key];
}
};
const relOfId = (id: string) => {
const normId = normPath(id);
const prefix = `${normRoot}/`.toLowerCase();
if (normId.toLowerCase().startsWith(prefix)) {
return normId.slice(normRoot.length + 1);
}
return normId;
};
for (const id of known.keys()) count(relOfId(id));
const knownPaths = new Set(
[...known.keys()].map((id) => relOfId(id))
);
// A frame the walk meets once is one frame, whether it is one the catalogue
// holds or one it has never seen: the walk lists every folder it walks into, so
// a name met twice is two entries for one frame — and both are counted once.
const countedDiscovered = new Set<string>();
// The folders this reading could not open, and so has no word on: what they
// hold was never listed, and a reading that would repair the catalogue to the
// disk must not read silence for absence.
const failed = new Set<string>();
const countDiscovered = (rel: string) => {
const relClean = rel.replace(/\\/g, '/');
if (knownPaths.has(relClean) || countedDiscovered.has(relClean)) return;
countedDiscovered.add(relClean);
const key = relClean.toLowerCase();
if (countedDiscovered.has(key)) return;
countedDiscovered.add(key);
count(relClean);
};
// The frames a reading that was cut off had already counted come back with its
// position: the walk it takes up is the rest of the roll, and a frame it does
// not walk again is a frame it must not count again either.
for (const rel of saved?.frames ?? []) countedDiscovered.add(rel.replace(/\\/g, '/').toLowerCase());
for (const e of entries) countDiscovered(e.rel);
// The frames the catalogue already holds of the folder being read, which is
// what the reading is measured against: a roll of a hundred thousand that has
@@ -840,8 +1073,8 @@ export async function scanFolder(
// measured against the frames under that one alone.
const heldPrefix = `${folder.name}/${from}`.toLowerCase();
const held = bounded
? [...known.keys()].filter((id) => id.toLowerCase().startsWith(heldPrefix)).length
: known.size;
? rawKnown.filter((p) => p.id.toLowerCase().startsWith(heldPrefix)).length
: rawKnown.length;
// What the screen is told, and how often. The column is redrawn out of the
// count and the count moves with every frame, so telling the screen a frame at
// a time is a whole tree rebuilt a frame at a time — on a roll of a hundred
@@ -866,6 +1099,9 @@ export async function scanFolder(
// hands the list on with the rest of its position.
const dirs: LibraryDir[] = progress.dirs;
let batch: LibraryPhoto[] = [];
// When the batch last went down, for the clock above: a reading of a long roll is
// told its rows have landed at a beat the reader can watch, not in one lump.
let flushedAt = Date.now();
// The frames this reading found for the first time, waiting to be handed on
// with the batch they landed in. A frame the catalogue already had is not one
// of these, however it came back — it is on the strip already, and a second
@@ -884,18 +1120,55 @@ export async function scanFolder(
// string per batch, which is the page held still for the whole roll. A position
// a step behind is a frame or two read again on the next visit, and a frame read
// again is skipped on its size and its time.
//
// The whole queue and not the front of it: a position that kept a few hundred
// paths and dropped the rest could not say where the reading had got to, and a
// reload in the middle of a long roll either took that position up and called
// the roll read with thousands of frames still unread, or — as the guard in
// `loadWalk` had it — threw it away and left the roll unread with nothing to
// pick it up. A megabyte a second of string on the main thread is what the
// position costs; the alternative is the reading.
let wroteAt = 0;
const write = async () => {
const write = async (force = false) => {
// A reading kept to one folder has no position to write: the file it would
// go in is the whole roll's, and a fraction of the tree written there is a
// roll that comes back with the rest of itself missing — the next update
// walks that one branch and calls the roll read.
if (bounded) return;
const at = Date.now();
if (wroteAt !== 0 && at - wroteAt < WALK_MS) return;
// The clock is for the batches, which land faster than a position is worth
// writing. The two writes that bracket a pass are the reading's own shape and
// are not a matter of the clock: the one before it says the roll has not been
// listed yet, and a pass that is through in under `WALK_MS` — every roll of a
// few hundred folders — would leave that first one standing in for a walk that
// is over, so the next visit listed the whole tree again to learn what was
// already written down.
if (!force && wroteAt !== 0 && at - wroteAt < WALK_MS) return;
wroteAt = at;
// The numbers go down with the position. `publish` is what the screen reads
// and it writes on a clock of its own; the count of a folder is only in
// `counts` and reaches `progress` through it, so a position written between
// two tells would say the reading had walked a folder and hold the number of
// it from before that folder — and a reading that takes the position up skips
// the folders it lists as walked, counting none of them again. The number a
// reloaded reading has for a folder it never walks again is this one.
progress.counts = { ...counts };
const held = [...inHand, ...batch.map(pathOf)];
const savedEntries = entries.slice(0, 500).map((e) => e.rel);
// A folder this reading could not open is a folder it has no word on. Left
// among the walked, the position would say it had been read and the next
// reading — an update, which takes this position up — would skip it and never
// look inside it again: a number that went to nothing stays at nothing
// however many updates follow, and only a rescan, which throws the position
// away, brings it back. It goes back in the queue instead, where the next
// reading opens it and reads it as the folder it never managed to be.
const unsaid = failed.size ? new Set([...failed].map(normPath)) : null;
const walkedSaved = unsaid ? [...tree.walked].filter((r) => !unsaid.has(normPath(r))) : [...tree.walked];
const rest = tree.pending.map((p) => p.rel);
await saveWalk(
folder.name,
[...tree.walked],
tree.pending.map((p) => p.rel),
[...held, ...savedEntries],
walkedSaved,
unsaid ? [...unsaid, ...rest] : rest,
[...held, ...entries.map((e) => e.rel)],
held.length,
progress
);
@@ -909,6 +1182,7 @@ export async function scanFolder(
if (!batch.length) return;
const rows = batch;
batch = [];
flushedAt = Date.now();
const db = await openDb();
await new Promise<void>((resolve, reject) => {
const hasThumbs = db.objectStoreNames.contains(THUMBS);
@@ -935,6 +1209,11 @@ export async function scanFolder(
}
await write();
};
// Whether what is in hand goes down now: the batch is full, or the clock has come
// round. The frames of a roll smaller than `BATCH` never fill one, and waiting for
// the end of the reading to hand them over is the reading's length of an empty
// screen.
const due = () => batch.length > 0 && (batch.length >= BATCH || Date.now() - flushedAt >= FLUSH_MS);
const readOne = async (handle: FileSystemFileHandle, rel: string): Promise<void> => {
const file = await handle.getFile();
@@ -943,7 +1222,7 @@ export async function scanFolder(
if (seen && seen.size === file.size && seen.mtime === file.lastModified) {
if (!seen.handle) {
batch.push({ ...seen, handle });
if (batch.length >= BATCH) await flush();
if (due()) await flush();
}
return;
}
@@ -981,13 +1260,14 @@ export async function scanFolder(
batch.push(row);
if (!seen) {
added.push(row);
// A frame the catalogue did not hold is news the rows have to show: it is
// the one thing an update adds to a number the catalogue already set.
count(rel);
// The row is not counted here. The walk counted every frame it found,
// this one included, the moment it found it — a second count on the way
// to the catalogue is the same frame twice, and a number that climbs past
// the roll and falls back to it when the reading ends.
}
progress.added++;
inHand = inHand.filter((r) => r !== rel);
if (batch.length >= BATCH) await flush();
if (due()) await flush();
};
// The frames in hand, read a few at a time: reading a frame is almost all
@@ -1042,6 +1322,12 @@ export async function scanFolder(
let stop = false;
let walked = false;
// The position before a single folder has been listed: a roll whose first pass
// takes a minute to get through the folders at its top is a roll with nothing
// written down for the whole of that minute, and a reload in it found nothing
// to pick up and left the reading where it was — started again by hand. What
// this writes down is the folder the reading was pointed at and no frame of it.
await write();
// One bounded pass at a time, and the pass yields between frames, so a folder
// clicked while the scan runs is read on the next jump rather than after the
// whole tree. Nothing clicked: the picked folder and its frames, then each
@@ -1069,6 +1355,9 @@ export async function scanFolder(
},
(file) => {
countDiscovered(file.rel);
},
(rel) => {
failed.add(rel);
}
);
// The names come in before the frames they hold: the column grows one pass
@@ -1087,8 +1376,9 @@ export async function scanFolder(
// The position before a frame of this pass is looked at: a reload here
// reads the pass's frames again, and a frame already in the catalogue is
// skipped anyway — where the other order would leave the last pass's
// frames out of the catalogue until the next visit.
await write();
// frames out of the catalogue until the next visit. Written whatever the
// clock says: it is the pass that is over, not a batch.
await write(true);
// A pass is worth telling the screen about whether the clock says so or
// not: the names of a whole layer have just arrived, and the column is
// built from them.
@@ -1103,7 +1393,27 @@ export async function scanFolder(
// walks the roll from the top and skips what has not moved. A reading kept to
// one folder never wrote one down, and the file it would clear is the whole
// roll's — a position another reading is in the middle of.
if (!stop) await clearWalk(folder.name);
// The roll's own position, and only the reading of the whole roll has one to
// clear: a reading kept to a subfolder never wrote it, and clearing it here
// throws away the rest of a roll an earlier reading stopped in the middle of.
if (!stop && !bounded) await clearWalk(folder.name);
// The catalogue brought to the disk. What the reading met is what is there, but
// only for a reading that met the whole of what it was pointed at: one that was
// stopped, one that took up a position — the folders that position had already
// walked are folders this reading never listed — and one that met a folder it
// could not open are all readings with no word on the frames they did not
// reach, and what they did not reach is not theirs to drop. See `prunePhotos`.
if (!stop && !failed.size && (bounded || !resumed)) {
await prunePhotos(folder.name, bounded ? from : '', countedDiscovered);
}
// What the walk found, kept where the catalogue is kept, so the column has it
// on the next visit whether the reading ran to its end or was cut off. A
// reading of the whole roll that reached its end is the roll and replaces what
// was there — a folder it did not meet is a folder that is gone, and its number
// with it. A reading kept to one folder knows one branch of the roll and is
// folded in; a reading that was stopped keeps its numbers for the folders it
// walked and none for the rest, which is what it counted. See `saveWalkedDirs`.
await saveWalkedDirs(folder.name, dirs, stop ? null : counts, !stop && !bounded, bounded ? from : '', failed);
// The last word, whatever the clock says: the frames the reading got to are
// the frames the column is about to be told about one last time.
publish(true);
docker/frontend/src/engine/libraryBackup.ts
+5
View File
@@ -23,6 +23,7 @@
import type { LibraryEdit, LibraryFolder } from './library';
import {
ensurePermission,
forgetWalkedDirs,
getPhotoThumb,
listEdits,
listFolders,
@@ -396,6 +397,10 @@ export async function restoreNow(onProgress?: (p: BackupProgress) => void): Prom
// Instant catalog metadata & edits restoration (< 100ms)
const photos = await restoreCatalogue(photoList, editList);
// And the walks are forgotten with it: the numbers a walk wrote down are off a
// disk this backup does not describe, and a column drawn under them would say a
// folder holds what it held the last time the reader looked at it.
await forgetWalkedDirs();
// Activate backup tile fetcher for lazy rendering on screen
setBackupTileFetcher(readBackupTile);
docker/frontend/src/engine/rawDevelop.ts
+49 -10
View File
@@ -567,21 +567,57 @@ function getCamToSrgbMatrix(cd: any, brand?: CameraBrand): number[][] {
}
}
export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<Uint8Array> {
// 1. Fast extraction of the embedded JPEG preview.
// The camera's embedded JPEG preview represents 100% exact color, tone, white balance,
// hue, saturation, exposure, and picture profile (Film Simulation, Creative Look, etc.)
// generated directly by the camera hardware (Fujifilm, Sony, Olympus, Nikon, Canon,
// Ricoh, Lumix, Leica, iPhone ProRAW, DNG).
const preview = extractEmbeddedJpeg(bytes);
if (preview && preview.length > 10000) {
return preview;
// The size a JPEG declares for itself, off its SOF segment — a header read, no
// decoder. Null when the bytes carry no readable frame header.
export function jpegFrameSize(jpeg: Uint8Array): { width: number; height: number } | null {
for (let i = 2; i + 9 < jpeg.length; ) {
if (jpeg[i] !== 0xff) {
i++;
continue;
}
const marker = jpeg[i + 1];
if (marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc) {
return { height: (jpeg[i + 5] << 8) | jpeg[i + 6], width: (jpeg[i + 7] << 8) | jpeg[i + 8] };
}
i += 2 + ((jpeg[i + 2] << 8) | jpeg[i + 3]);
}
return null;
}
// Is the embedded preview the frame itself? Bodies that write one at the frame
// size (Nikon NEF 6016x4016 out of 6032x4032, Sony ARW, an iPhone ProRAW DNG)
// differ from the sensor by a margin row or two; a thumbnail out of the same
// sensor is 1920x1280, a third of the long side, so 90% divides them cleanly.
// Unreadable sizes keep the old behaviour and trust the preview.
function previewIsFrame(preview: Uint8Array, meta: any): boolean {
const size = jpegFrameSize(preview);
const frame = Math.max(
Number(meta?.iwidth) || 0,
Number(meta?.iheight) || 0,
Number(meta?.width) || 0,
Number(meta?.height) || 0
);
if (!size || !frame) return true;
return Math.max(size.width, size.height) >= frame * 0.9;
}
export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<Uint8Array> {
// The embedded JPEG is the camera's own rendering — exact colour, tone, white
// balance and picture profile, generated by the body itself. It is the frame
// only when it is the size of the frame; on most bodies it is a thumbnail
// (1920x1280 out of a 24MP sensor, measured on a Fuji RAF and a Lumix RW2), so
// handing it back is what opened a 24MP file as a 1920 wide one. Whether it is
// the frame is known from the sensor size, which costs one LibRaw open (91ms
// measured, metadata only — no demosaic) against seconds for the develop.
const preview = extractEmbeddedJpeg(bytes);
const raw = new LibRaw();
try {
await raw.open(bytes.slice() as unknown as BufferSource, SETTINGS);
const meta = await raw.metadata(true);
if (preview && preview.length > 10000 && previewIsFrame(preview, meta)) {
return preview;
}
const img = await raw.imageData();
const cd = meta?.color_data;
if (!img || (!cd?.cam_mul && !cd?.pre_mul) || (!cd.rgb_cam && !cd.cam_xyz)) {
@@ -657,7 +693,7 @@ export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<
if (!first) throw new Error('no surface for the develop');
let finalShot = first;
let refJpeg = extractEmbeddedJpeg(bytes);
let refJpeg = preview;
if (!refJpeg) {
refJpeg = await cameraPreview(raw).catch(() => null);
}
@@ -691,6 +727,9 @@ export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<
if (!jpeg?.length) throw new Error('develop produced no bytes');
return jpeg;
} catch (err) {
// The preview is also the fallback, as it always was: a file whose sensor
// will not decode still opens as the camera rendered it.
if (preview && preview.length > 10000) return preview;
const fallback = await rawThumbnail(bytes).catch(() => null);
if (fallback) return fallback;
throw err;
docker/frontend/src/engine/rollWalk.ts
+6 -1
View File
@@ -89,7 +89,8 @@ export async function walkPass(
keep: (name: string) => boolean,
jump: () => string | null,
onDir?: (rel: string) => void,
onFile?: (file: WalkFile) => void
onFile?: (file: WalkFile) => void,
onFail?: (rel: string) => void
): Promise<boolean> {
for (let i = 0; i < DIRS_PER_PASS && walk.pending.length; i++) {
await jumpQueue(walk, jump());
@@ -115,6 +116,10 @@ export async function walkPass(
}
} catch {
// A subfolder that will not open is skipped; the rest of the roll stands.
// It is also a folder this walk has no word on — what it holds was never
// listed, so a caller about to take the walk's word for what the disk
// holds is told here that this folder is not part of it.
onFail?.(here.rel);
}
}
return walk.pending.length === 0;
docker/frontend/src/i18n/I18nProvider.tsx
+16 -7
View File
@@ -25,6 +25,21 @@ interface I18nValue {
const Ctx = createContext<I18nValue | null>(null);
function translate(lang: Lang, key: MsgKey, params?: Record<string, string | number>): string {
let text: string = DICTS[lang][key] ?? vi[key] ?? key;
if (params) {
for (const [name, v] of Object.entries(params)) text = text.replace(`{${name}}`, String(v));
}
return text;
}
// The same lookup, for the code outside React that has to write a line the
// screen will show — the held write in api.ts, which throws its sentence out of
// a catch block. It reads the language the way the provider does.
export function t(key: MsgKey, params?: Record<string, string | number>): string {
return translate(readLang(), key, params);
}
export function I18nProvider({ children }: { children: ReactNode }) {
const [lang, setLangState] = useState<Lang>(readLang);
@@ -40,13 +55,7 @@ export function I18nProvider({ children }: { children: ReactNode }) {
const value = useMemo<I18nValue>(() => ({
lang,
setLang,
t: (key, params) => {
let text: string = DICTS[lang][key] ?? vi[key] ?? key;
if (params) {
for (const [name, v] of Object.entries(params)) text = text.replace(`{${name}}`, String(v));
}
return text;
},
t: (key, params) => translate(lang, key, params),
}), [lang, setLang]);
return <Ctx.Provider value={value}>{children}</Ctx.Provider>;
docker/frontend/src/i18n/en.ts
+17
View File
@@ -249,6 +249,8 @@ export const en: Dict = {
'lib.renameAsk': 'Name shown for this folder (the folder on the disk is not renamed):',
'lib.update': 'UPDATE...',
'lib.rescan': 'RESCAN FROM THE TOP',
'lib.resume': 'KEEP READING {folder}',
'lib.resumeHint': 'The last reading of this folder was cut off. Press to read it on from where it stopped.',
'lib.remove': 'REMOVE',
'lib.reconnect': 'GRANT ACCESS AGAIN',
'lib.nodeHint': 'Right-click to rename, update, rescan, collapse or remove the folder.',
@@ -322,6 +324,13 @@ export const en: Dict = {
'lib.restoreNothing': 'There is no catalogue of its own in {folder} — pick the folder a backup was written into.',
'lib.restoreFailed': 'The backup folder could not be read.',
// The repair (Library.tsx, admin only): every roll read back to the disk, so
// the catalogue and the disk say the same thing again.
'lib.repair': 'REPAIR DATA',
'lib.repairConfirm': 'Read all {n} folders from the top so the library matches the disk? Frames that are gone will be dropped from the catalogue.',
'lib.repairRun': 'Repairing the library ({n} folders)…',
'lib.repairDone': 'Library repaired: {n} folders read back.',
'nav.admin': 'Admin',
'nav.photos': 'My photos',
'nav.profile': 'Profile',
@@ -435,6 +444,14 @@ export const en: Dict = {
'err.generic': 'Something went wrong: {msg}',
'err.offline': 'Could not reach the server.',
// The offline bar (src/ui/OfflineBar.tsx).
'offline.notice': 'No server — showing the copy saved on this machine.',
// A write the server never got. It is held, not lost; the sentence rides
// `err.generic`'s `{msg}`, so it reads as the second half of that one, and the
// count is the bar's own line for the queue (`offline.pending`).
'offline.queued': 'the server is out of reach, so this change is saved here and sent when it is back.',
'offline.pending': 'Not sent yet: {n}',
'meta.exported': 'Downloaded {name} ({kb}KB, 300 ppi)',
};
docker/frontend/src/i18n/vi.ts
+15
View File
@@ -261,6 +261,8 @@ export const vi = {
'lib.renameAsk': 'Tên hiển thị cho thư mục này (thư mục trên máy không bị đổi tên):',
'lib.update': 'CẬP NHẬT...',
'lib.rescan': 'QUÉT LẠI TỪ ĐẦU',
'lib.resume': 'ĐỌC TIẾP {folder}',
'lib.resumeHint': 'Lượt đọc thư mục này bị cắt ngang. Bấm để đọc tiếp từ chỗ đã dừng.',
'lib.remove': 'BỎ',
'lib.reconnect': 'CẤP LẠI QUYỀN',
'lib.nodeHint': 'Chuột phải để đổi tên, cập nhật, quét lại, thu gọn hay bỏ thư mục.',
@@ -335,6 +337,13 @@ export const vi = {
'lib.restoreNothing': 'Thư mục {folder} không có danh mục sao lưu nào — hãy chọn đúng thư mục đã ghi sao lưu.',
'lib.restoreFailed': 'Không đọc được thư mục sao lưu.',
// Sửa dữ liệu (Library.tsx, chỉ quản trị): đọc lại từng cuộn ảnh về đúng đĩa,
// để danh mục và đĩa nói cùng một điều.
'lib.repair': 'SỬA DỮ LIỆU',
'lib.repairConfirm': 'Đọc lại cả {n} thư mục từ đầu để thư viện khớp với đĩa? Ảnh đã mất sẽ bị gỡ khỏi danh mục.',
'lib.repairRun': 'Đang sửa dữ liệu thư viện ({n} thư mục)…',
'lib.repairDone': 'Đã sửa dữ liệu thư viện: đọc lại {n} thư mục.',
'adm.title': 'Quản trị dải phim',
'adm.subtitle': 'Ảnh do người dùng đóng góp. Xoá một ảnh để gỡ nó khỏi trang chủ.',
'adm.upload': 'THÊM ẢNH',
@@ -445,6 +454,12 @@ export const vi = {
'err.generic': 'Có lỗi xảy ra: {msg}',
'err.offline': 'Không gọi được máy chủ.',
// The offline bar (src/ui/OfflineBar.tsx). Says both halves: the server is
// gone, and what is on the screen is the copy this machine kept.
'offline.notice': 'Mất kết nối máy chủ — đang xem bản đã lưu trên máy.',
'offline.queued': 'máy chủ không tới được, nên thay đổi này đã lưu trên máy và sẽ gửi khi kết nối lại.',
'offline.pending': 'Chưa gửi được: {n}',
'meta.exported': 'Đã tải {name} ({kb}KB, 300 ppi)',
} as const;
docker/frontend/src/main.tsx
+2
View File
@@ -15,6 +15,7 @@ import { Library } from './Library';
import { scanSession } from './engine/library';
import { go } from './nav';
import { InstallPrompt } from './ui/InstallPrompt';
import { OfflineBar } from './ui/OfflineBar';
import { installTracking } from './track';
// Six routes, no router: the landing page, the workspace, the strip moderation
@@ -105,6 +106,7 @@ createRoot(root).render(
<I18nProvider>
<Shell />
<InstallPrompt />
<OfflineBar />
</I18nProvider>
</ThemeProvider>
);
docker/frontend/src/pwa/offline.ts
+234
View File
@@ -0,0 +1,234 @@
// Whether the server can be reached, and what it last said.
//
// Two things the offline story needs and no more: a flag the screen can watch
// (the notice in ui/OfflineBar.tsx), and the last good answer to each read, so a
// reload with no server comes back on the data it was already showing instead of
// an empty page.
//
// The flag is set by the API layer, the one place that finds out: a fetch that
// never got an answer is a server that is gone, and the next one that answers is
// the server back. `navigator.onLine` is the browser's guess and is wrong exactly
// where it matters — a server down behind a working connection — so it only sets
// the flag one way, and never clears it.
//
// The last good answers live in a store of their own in IndexedDB, keyed by the
// route that gave them: the strip's listing is a few hundred kilobytes and
// localStorage's few megabytes hold the theme, the language and the library's
// view besides. They are this browser's, not this account's, so they are dropped
// whenever the account changes (see `forgetStale`, called from the auth calls in
// api.ts).
//
// The writes that found no server are held here too, in order, and sent the
// moment one answers — same storage, same account, same clearing. See `hold`.
import { keep, recall } from '../remember';
// The writes waiting for a server. A write the API layer carries is always a
// JSON string (the two that carry bytes — a photo, an avatar — never reach here,
// see `hold`), so this is a handful of small bodies in the same small
// localStorage the theme and the language already live in. A queue long enough
// to matter is a queue the visitor has walked away from, and a reload keeps it
// anyway.
const PENDING = 'rc.pending';
// A read is kept whole, whatever its size: the strip's listing carries thousands
// of rows and it is exactly the read an offline visit wants an answer to. The
// ceiling is only there so a body that is not a listing at all — a few megabytes
// of image someone sent through a JSON route — is not written into the store.
const CAP = 2_000_000;
let reachable = true;
const listeners = new Set<() => void>();
const notify = () => {
for (const listener of listeners) listener();
};
function set(next: boolean): void {
if (reachable === next) return;
reachable = next;
notify();
}
// The shape `useEffect` already has: an unsubscribe.
export function subscribe(listener: () => void): () => void {
listeners.add(listener);
return () => {
listeners.delete(listener);
};
}
export function isReachable(): boolean {
return reachable;
}
// A request that never got an answer.
export function serverGone(): void {
set(false);
}
// Any answer at all — a 401 included: a server that refuses is a server that is
// there, and what the screen shows for it is its own business.
export function serverAnswered(): void {
set(true);
flush();
}
// One held write: the route, the verb, and the JSON body `call()` was handed.
// `body: null` is a write that carries none (`DELETE`).
export interface Held {
method: string;
path: string;
body: string | null;
}
function queue(): Held[] {
const raw = recall(PENDING);
if (raw === null) return [];
try {
const list = JSON.parse(raw) as Held[];
return Array.isArray(list) ? list : [];
} catch {
// A half-written queue is no queue.
return [];
}
}
function setQueue(list: Held[]): void {
keep(PENDING, JSON.stringify(list));
notify();
}
// How many writes the server has not seen yet — the count the bar shows.
export function heldCount(): number {
return queue().length;
}
// Hold a write the server did not answer. The same route twice is the later
// write in the earlier one's place: two edits of one recipe offline are one
// recipe to save, not two.
//
// Only what the API layer carries is held, and a file is only ever carried
// round it (a photo upload, an avatar) — those are not held at all: there is no
// trustworthy place for a few megabytes in this browser, and the uploader's own
// retry is the honest answer for a file.
export function hold(path: string, method: string, body: string | null): void {
const list = queue();
const at = list.findIndex((item) => item.path === path && item.method === method);
const one: Held = { method, path, body };
if (at === -1) list.push(one);
else list[at] = one;
setQueue(list);
}
let sending = false;
// A server answered: the writes it never got go now, oldest first. A refusal
// (4xx) is that server's own answer, so the entry goes — asking the same
// question forever is not a queue, it is a loop. No answer at all, or a server
// that broke (5xx, 429), stops the run and leaves the rest for the next answer.
export function flush(): void {
if (sending || !reachable) return;
const list = queue();
if (list.length === 0) return;
sending = true;
void (async () => {
for (const item of list) {
let res: Response;
try {
res = await fetch(`/api${item.path}`, {
method: item.method,
credentials: 'same-origin',
headers: item.body === null ? undefined : { 'content-type': 'application/json' },
body: item.body ?? undefined,
});
} catch {
// Gone again mid-run: what is left stays right where it is, in order.
serverGone();
break;
}
if (res.status >= 500 || res.status === 429) break;
setQueue(queue().filter((it) => !(it.path === item.path && it.method === item.method)));
}
sending = false;
})();
}
// The reads are a database of their own, not a store added to the library's: a
// tab left open holds the library at the version it opened it with, and a new
// store there would have every visit wait on that tab. Their own key is the
// route, their value the JSON string it answered with.
const DB_NAME = 'recipescam-offline';
const READS = 'reads';
let dbPromise: Promise<IDBDatabase> | null = null;
function openDb(): Promise<IDBDatabase> {
dbPromise ??= new Promise<IDBDatabase>((resolve, reject) => {
const req = indexedDB.open(DB_NAME, 1);
req.onupgradeneeded = () => req.result.createObjectStore(READS);
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
});
return dbPromise;
}
// One request per call, the shape the library's own `ask` has: a keyed get, put
// or clear is one transaction and nothing more.
function ask(mode: IDBTransactionMode, run: (s: IDBObjectStore) => IDBRequest): Promise<unknown> {
return openDb().then(
(db) =>
new Promise((resolve, reject) => {
const req = run(db.transaction(READS, mode).objectStore(READS));
req.onsuccess = () => resolve(req.result);
req.onerror = () => reject(req.error);
})
);
}
// What this route answered the last time a server was there, or null. Answered
// out of the store, so it is a promise where the old localStorage read was not;
// the one caller already awaits.
export async function stale<T>(path: string): Promise<T | null> {
let raw: unknown;
try {
raw = await ask('readonly', (s) => s.get(path));
} catch {
// No store to read, no answer to give — the caller's own empty state.
return null;
}
if (typeof raw !== 'string') return null;
try {
return JSON.parse(raw) as T;
} catch {
// A half-written value is no value.
return null;
}
}
// Written in the background: the answer on screen is not waiting on the disk.
export function keepStale(path: string, body: unknown): void {
let raw: string;
try {
raw = JSON.stringify(body);
} catch {
return;
}
if (raw.length > CAP) return;
void ask('readwrite', (s) => s.put(raw, path)).catch(() => {});
}
// The account changed: the answers held are the last account's, and the next one
// must not be shown them. Neither must it send its writes under the new name —
// they were made by whoever was signed in when they landed.
export function forgetStale(): void {
void ask('readwrite', (s) => s.clear()).catch(() => {});
try {
localStorage.removeItem(PENDING);
} catch {
// Private mode: there was nothing to drop.
}
notify();
}
// The browser going offline is the flag arriving early, and no more: it is never
// used to clear it, because a connection says nothing about the server behind it.
window.addEventListener('offline', serverGone);
docker/frontend/src/styles/app.css
+26
View File
@@ -2128,3 +2128,29 @@ input[type="range"].track-temp::-webkit-slider-thumb, input[type="range"].track-
border-radius: var(--radius-sm, 4px);
transition: background 0.3s ease, outline 0.3s ease;
}
/* --- the offline bar ------------------------------------------------------
ui/OfflineBar.tsx, over every route. A pill at the foot of the screen, out of
the way of the header the studio is read across and of nothing else: the
workspace's own bottom bar on a phone is the one thing there, and the pill
clears it (44px pill + the bar's 16px of padding, and a hair). */
.offline-bar {
position: fixed;
left: 50%;
bottom: calc(12px + env(safe-area-inset-bottom));
transform: translateX(-50%);
z-index: 55; /* under a dialog (60), over the page */
max-width: calc(100vw - 24px);
padding: 7px 14px;
border: 1px solid var(--accent-line, var(--border));
border-radius: 999px;
background: var(--bg-elev);
color: var(--accent);
box-shadow: var(--shadow);
font-family: var(--mono);
font-size: 12px;
text-align: center;
}
@media (max-width: 860px) {
.offline-bar { bottom: calc(72px + env(safe-area-inset-bottom)); }
}
docker/frontend/src/ui/OfflineBar.tsx
+34
View File
@@ -0,0 +1,34 @@
import { useEffect, useState } from 'react';
import { useI18n } from '../i18n/I18nProvider';
import { heldCount, isReachable, subscribe } from '../pwa/offline';
// The server cannot be reached. A bar, not a dialog: nothing is being asked of
// the visitor, and the answer — the saved copy of what the screen was holding —
// is already on the screen. It goes away by itself the moment a call answers.
//
// It also stays up while writes are still held: the server being back is not the
// same thing as the server having them, and a queue that drained quietly would
// leave the visitor wondering whether the last change landed. See `hold`/`flush`
// in pwa/offline.ts.
export function OfflineBar() {
const { t } = useI18n();
const [gone, setGone] = useState(() => !isReachable());
const [held, setHeld] = useState(heldCount);
useEffect(
() =>
subscribe(() => {
setGone(!isReachable());
setHeld(heldCount());
}),
[]
);
if (!gone && held === 0) return null;
const lines = [gone ? t('offline.notice') : '', held > 0 ? t('offline.pending', { n: held }) : ''];
return (
<div className="offline-bar" role="status" data-key="offline-bar">
{lines.filter(Boolean).join(' ')}
</div>
);
}