3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(studio): the heal fill is a cascade of blurs, the brush is put down with its tab, and Immich is re-read on the way back

Browse Source 
HEAL painted a grey blob over the photo it was meant to repair, and the
cause was one draw Skia gets wrong on this build: a snapshot used as a
SCALED source comes back with the alpha multiplied into its rgb a second
time, so a partly transparent rim — which is what the mask's feather is —
was read as a dark wash. A pyramid of halvings is built of exactly those
scaled draws, which is why the repair's own rim came out grey; the same
source read at 1:1 is exact (measured, tmp-src-probe).

So the fill is now a cascade of blurs, all of them 1:1: the cut frame is
blurred at a few widths, widest first, each finer one drawn over it, the
untouched cut on top. The finest is one brush wide, each next is twice the
last, and the count follows the depth of the hole (the stroke's own
diagonal against the widest brush), which is what keeps the middle of a
long hole filled rather than bare. Nothing is copied from anywhere and the
rim arrives with no seam, which is what the passes were always for.

And the fill is built on the brush's own box rather than the frame, padded
by three sigma of the widest level. The composite reads it only where the
mask is, so this changes no pixel of the result (heal-check measures the
same masks either way) and it is what makes the repair affordable at the
zoom a repair is aimed at: at a 3200px preview a single spot went from
1.7s of frozen main thread to 1.2s, and the heal's own share of that from
0.8s to 0.3s. What is left is the pipeline itself, which is a separate
matter.

The FIX brush outlived its tab. It takes the pointer on the photo, and
leaving TOOLS never put it down — so its layer stayed on the stage over a
photo nobody was editing, and its ring came back under the pointer every
time the hand crossed a repair, which reads as the tool re-arming itself.
It now goes down with the tab, with another chip or another strip, and on
a press outside its own two chips, the same way the mask tool does.

And the Immich button was missing for a visitor who had just connected a
server: the studio and the library are two screens of one page, both
mounted, so coming back is a route change and not a mount — the studio's
own effect, which reads the server list once a session, never ran again.
It is read again on a popstate that lands on /app.

Checks: node scripts/heal-check.mjs (the cascade and the box, to the
letter), npx tsc --noEmit, and the studio driven in a browser: the button
appears on the way back from the library, the brush goes down with TOOLS
and re-arms with HEAL, and a stroke over a photo lays one band with no grey
in it.
This commit is contained in:
3dtours
2026-10-12 18:12:45 +07:00
parent b4bb600cca
commit 00063a7f7c
5 changed files with 284 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/scripts/heal-check.mjs
+102 -52
View File
@@ -25,7 +25,7 @@ const transpile = (path) =>
const dir = mkdtempSync(join(tmpdir(), 'heal-check-'));
writeFileSync(join(dir, 'heal.mjs'), transpile('../shared/utils/heal.ts'));
const { readHeal, HEAL_MASK_STOPS, HEAL_MAX_LEVELS, HEAL_CUT_SKSL, HEAL_FILL_SKSL } = await import(
const { readHeal, HEAL_MASK_STOPS, HEAL_FILL_BLUR, HEAL_FILL_MAX_LEVELS, HEAL_CUT_SKSL, HEAL_FILL_SKSL } = await import(
pathToFileURL(join(dir, 'heal.mjs')).href
);
@@ -37,6 +37,11 @@ assert.match(engine, /healCutEffect = Skia\.RuntimeEffect\.Make\(HEAL_CUT_SKSL\)
assert.match(engine, /healFillEffect = Skia\.RuntimeEffect\.Make\(HEAL_FILL_SKSL\)/, 'exportEngine does not compile HEAL_FILL_SKSL');
assert.doesNotMatch(engine, /healSkSL\(/, 'the per-spot heal shader is still in exportEngine');
assert.doesNotMatch(engine, /healUniforms\(/, 'the per-spot uniform block is still in exportEngine');
// ...and the fill's level count is the one this check builds, not a constant.
assert.match(engine, /HEAL_FILL_MAX_LEVELS/, 'exportEngine does not cap the fill levels at HEAL_FILL_MAX_LEVELS');
assert.match(engine, /1 \+ Math\.ceil\(Math\.log2\(2 \+ span \/ 2 \/ widest\)\)/, 'exportEngine no longer scales the fill levels with the depth of the hole');
// ...and the fill is built on the brush's box, read back at the frame's origin.
assert.match(engine, /imageShaderChildAt\(fillImage, boxLeft, boxTop\)/, 'the fill child is read at its own origin instead of the frame\'s');
// --- the stored spots --------------------------------------------------------
// A recipe written by the older brush carries sx/sy. It still loads: those two
@@ -119,6 +124,16 @@ const pixelsOf = (image) => {
};
const asChild = (image) =>
image.makeShaderOptions(CanvasKit.TileMode.Clamp, CanvasKit.TileMode.Clamp, CanvasKit.FilterMode.Linear, CanvasKit.MipmapMode.None);
// The fill's own child, read in the frame's coordinates: the fill is built on
// the brush's box (below), and a whole-pixel shift is what the engine does.
const asChildAt = (image, x, y) =>
image.makeShaderOptions(
CanvasKit.TileMode.Clamp,
CanvasKit.TileMode.Clamp,
CanvasKit.FilterMode.Linear,
CanvasKit.MipmapMode.None,
[1, 0, x, 0, 1, y, 0, 0, 1]
);
// The brush's circles as ONE mask: a radial gradient per spot, solid out to
// HEAL_FEATHER of its radius and clear at its edge, each laid over the last.
@@ -142,8 +157,8 @@ function maskOf(spotsIn) {
return surf.makeImageSnapshot();
}
// exportEngine's 6e, pass for pass: cut, the pyramid down and up again, and the
// fill laid back under the brush.
// exportEngine's 6e, pass for pass: cut, the blurred fill, and the fill laid
// back under the brush.
function healPass(rgba, spotsIn) {
const src = imageOf(rgba);
const mask = maskOf(spotsIn);
@@ -154,53 +169,53 @@ function healPass(rgba, spotsIn) {
cutPaint.setShader(cutEffect.makeShaderWithChildren([], [asChild(src), asChild(mask)]));
cut.getCanvas().drawRect(CanvasKit.XYWHRect(0, 0, W, H), cutPaint);
const levels = [];
let carried = cut.makeImageSnapshot();
for (let k = 1; k <= HEAL_MAX_LEVELS; k++) {
const lw = Math.max(1, W >> k);
const lh = Math.max(1, H >> k);
const level = surface(lw, lh);
level
.getCanvas()
.drawImageRectOptions(
carried,
CanvasKit.XYWHRect(0, 0, carried.width(), carried.height()),
CanvasKit.XYWHRect(0, 0, lw, lh),
CanvasKit.FilterMode.Linear,
CanvasKit.MipmapMode.None
);
levels.push({ surf: level, w: lw, h: lh });
carried = level.makeImageSnapshot();
if (lw === 1 && lh === 1) break;
}
let up = null;
let fillImage = null;
for (let k = levels.length - 1; k >= 0; k--) {
const { surf, w: lw, h: lh } = levels[k];
if (up) {
surf
.getCanvas()
.drawImageRectOptions(
up,
CanvasKit.XYWHRect(0, 0, up.width(), up.height()),
CanvasKit.XYWHRect(0, 0, lw, lh),
CanvasKit.FilterMode.Linear,
CanvasKit.MipmapMode.None
);
}
const pulled = surf.makeImageSnapshot();
if (k === 0) {
fillImage = pulled;
break;
}
up = pulled;
// The fill: the cut frame blurred at a few widths, widest first, each finer
// one drawn over it, the untouched cut on top. A blur is a weighted mean of
// the premultiplied pixels around each pixel, and the hole weighs nothing, so
// the hole comes back as the light of its rim. Widest first means the widest
// blur only has a say where every finer one has nothing — the middle of a hole
// deeper than a brush — and the untouched cut wins wherever the photo is still
// there, so the rim is the rim exactly.
//
// Blurs and not the pyramid of halvings the engine used to build: this Skia
// gets the premultiply wrong on a snapshot drawn with a SCALE — its rgb comes
// back with the alpha multiplied in a second time — and a pyramid is made of
// exactly those halvings, so a partly transparent rim came back a wash of
// grey. Every draw here is 1:1, which filters correctly.
const cutImage = cut.makeImageSnapshot();
// The count and the widths are exportEngine's own, to the letter: a check
// that picks its own levels is checking a renderer nobody ships. So is the
// surface the fill is built on — the brush's own box, padded by three sigma of
// the widest level, rather than the frame.
const widest = Math.max(...spotsIn.map((s) => s.r * W));
const xs = spotsIn.map((s) => s.x * W);
const ys = spotsIn.map((s) => s.y * H);
const span = Math.hypot(Math.max(...xs) - Math.min(...xs), Math.max(...ys) - Math.min(...ys));
const levelCount = Math.min(
HEAL_FILL_MAX_LEVELS,
Math.max(2, 1 + Math.ceil(Math.log2(2 + span / 2 / widest)))
);
const pad = 3 * HEAL_FILL_BLUR * widest * 2 ** (levelCount - 1);
const boxLeft = Math.max(0, Math.floor(Math.min(...xs) - widest - pad));
const boxTop = Math.max(0, Math.floor(Math.min(...ys) - widest - pad));
const boxRight = Math.min(W, Math.ceil(Math.max(...xs) + widest + pad));
const boxBottom = Math.min(H, Math.ceil(Math.max(...ys) + widest + pad));
const box = CanvasKit.XYWHRect(boxLeft, boxTop, boxRight - boxLeft, boxBottom - boxTop);
const fillBox = surface(boxRight - boxLeft, boxBottom - boxTop);
for (let k = levelCount; k >= 1; k--) {
const sigma = HEAL_FILL_BLUR * widest * 2 ** (k - 1);
const blurPaint = new CanvasKit.Paint();
blurPaint.setImageFilter(CanvasKit.ImageFilter.MakeBlur(sigma, sigma, CanvasKit.TileMode.Clamp, null));
fillBox.getCanvas().drawImageRect(cutImage, box, box, blurPaint);
}
fillBox.getCanvas().drawImageRect(cutImage, box, box, new CanvasKit.Paint());
const fillImage = fillBox.makeImageSnapshot();
const fillEffect = CanvasKit.RuntimeEffect.Make(HEAL_FILL_SKSL);
assert.ok(fillEffect, 'HEAL_FILL_SKSL does not compile');
const out = surface(W, H);
const paint = new CanvasKit.Paint();
paint.setShader(fillEffect.makeShaderWithChildren([], [asChild(src), asChild(fillImage), asChild(mask)]));
paint.setShader(fillEffect.makeShaderWithChildren([], [asChild(src), asChildAt(fillImage, boxLeft, boxTop), asChild(mask)]));
out.getCanvas().drawRect(CanvasKit.XYWHRect(0, 0, W, H), paint);
return { pixels: pixelsOf(out.makeImageSnapshot()), mask: pixelsOf(mask) };
}
@@ -272,10 +287,10 @@ assert.ok(worstStep <= 4, `the fill steps by ${worstStep} levels between neighbo
// 4. The repair carries the LIGHT of where it landed. On a photo running from
// dark to light across the frame, the painted pixels come back on that ramp
// rather than at one flat value borrowed from somewhere else. The bound is
// the ramp's own steepness, not zero: the pyramid reads the rim of the hole,
// and where the hole takes weight off one side of a pixel its average slides
// that way by about a pixel's worth of the ramp — 0.75 levels a pixel across
// this frame. On the flat ground of 2 and 3 it is nothing at all.
// the ramp's own steepness, not zero: the fill reads the rim of the hole, and
// where the hole takes weight off one side of a pixel its average slides that
// way by about a pixel's worth of the ramp — 0.75 levels a pixel across this
// frame. On the flat ground of 2 and 3 it is nothing at all.
const ramped = healPass(bytesOf(shadeRamp), spots);
let worstRamp = 0;
for (let y = 0; y < H; y++) {
@@ -287,13 +302,24 @@ for (let y = 0; y < H; y++) {
assert.ok(worstRamp <= 6, `the fill is off the ramp it sits in by ${worstRamp} levels`);
// 5. Outside the mask, nothing moved. The pass is asked for the painted pixels
// and no others.
// and no others — and "painted" is the whole brush, feather included: the last
// 15% of a spot is where the repair is mixed back in, so it is part of the
// repair. Past the brush's own edge the photo is the photo.
let outWorst = 0;
let outAt = 0;
for (let p = 0; p < W * H; p++) {
if (inMask(p % W, (p - (p % W)) / W)) continue;
for (let c = 0; c < 3; c++) outWorst = Math.max(outWorst, Math.abs(made.pixels[p * 4 + c] - flat[p * 4 + c]));
const x = p % W;
const y = (p - x) / W;
if (alphaAt(x, y) > 0) continue;
for (let c = 0; c < 3; c++) {
const off = Math.abs(made.pixels[p * 4 + c] - flat[p * 4 + c]);
if (off > outWorst) {
outWorst = off;
outAt = x;
}
}
}
assert.equal(outWorst, 0, `a pixel outside the mask changed by ${outWorst} levels`);
assert.equal(outWorst, 0, `a pixel outside the mask (x=${outAt}) changed by ${outWorst} levels`);
// 6. The same photo and the same spots give the same photo, byte for byte.
assert.deepEqual(
@@ -312,4 +338,28 @@ for (let p = 0; p < W * H; p++) {
}
assert.equal(worstAll, 0, `a frame-wide mask changed the photo by ${worstAll} levels`);
// 8. The shape the brush is actually used in: one long thin run right across the
// frame, on a photo that runs dark to light along it. Its rim is the pixels
// just above and below it wherever it passes, so the repair comes back on the
// ramp there — not at the frame's average colour. A hole this long needs the
// level count to grow with it: too few levels and the middle of the run is
// farther from any rim than the widest blur reaches, so the fill comes back
// the average of a blur too wide to have read the rim — one flat mark.
const band = [];
for (let i = 0; i <= 60; i++) band.push({ x: 0.12 + (0.76 * i) / 60, y: 0.5, r: 0.02 });
const across = healPass(bytesOf(shadeRamp), band);
let worstBand = 0;
let bandAt = 0;
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
if (across.mask[(y * W + x) * 4 + 3] < 250) continue;
const off = Math.abs(across.pixels[(y * W + x) * 4] - shadeRamp(x)[0]);
if (off > worstBand) {
worstBand = off;
bandAt = x;
}
}
}
assert.ok(worstBand <= 8, `a painted pixel at x=${bandAt} is ${worstBand} levels off the rim it sits in`);
console.log('ok the stroke is one mask, the fill carries the light, and nothing outside it moved');