3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(studio): the heal fill is a cascade of blurs, the brush is put down with its tab, and Immich is re-read on the way back

Browse Source 
HEAL painted a grey blob over the photo it was meant to repair, and the
cause was one draw Skia gets wrong on this build: a snapshot used as a
SCALED source comes back with the alpha multiplied into its rgb a second
time, so a partly transparent rim — which is what the mask's feather is —
was read as a dark wash. A pyramid of halvings is built of exactly those
scaled draws, which is why the repair's own rim came out grey; the same
source read at 1:1 is exact (measured, tmp-src-probe).

So the fill is now a cascade of blurs, all of them 1:1: the cut frame is
blurred at a few widths, widest first, each finer one drawn over it, the
untouched cut on top. The finest is one brush wide, each next is twice the
last, and the count follows the depth of the hole (the stroke's own
diagonal against the widest brush), which is what keeps the middle of a
long hole filled rather than bare. Nothing is copied from anywhere and the
rim arrives with no seam, which is what the passes were always for.

And the fill is built on the brush's own box rather than the frame, padded
by three sigma of the widest level. The composite reads it only where the
mask is, so this changes no pixel of the result (heal-check measures the
same masks either way) and it is what makes the repair affordable at the
zoom a repair is aimed at: at a 3200px preview a single spot went from
1.7s of frozen main thread to 1.2s, and the heal's own share of that from
0.8s to 0.3s. What is left is the pipeline itself, which is a separate
matter.

The FIX brush outlived its tab. It takes the pointer on the photo, and
leaving TOOLS never put it down — so its layer stayed on the stage over a
photo nobody was editing, and its ring came back under the pointer every
time the hand crossed a repair, which reads as the tool re-arming itself.
It now goes down with the tab, with another chip or another strip, and on
a press outside its own two chips, the same way the mask tool does.

And the Immich button was missing for a visitor who had just connected a
server: the studio and the library are two screens of one page, both
mounted, so coming back is a route change and not a mount — the studio's
own effect, which reads the server list once a session, never ran again.
It is read again on a popstate that lands on /app.

Checks: node scripts/heal-check.mjs (the cascade and the box, to the
letter), npx tsc --noEmit, and the studio driven in a browser: the button
appears on the way back from the library, the brush goes down with TOOLS
and re-arms with HEAL, and a stroke over a photo lays one band with no grey
in it.
This commit is contained in:
3dtours
2026-10-12 18:12:45 +07:00
parent b4bb600cca
commit 00063a7f7c
5 changed files with 284 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/src/engine/exportEngine.ts
+82 -59
View File
@@ -49,8 +49,9 @@ import {
HEAL_CUT_SKSL,
HEAL_FILL_SKSL,
HEAL_FEATHER,
HEAL_FILL_BLUR,
HEAL_FILL_MAX_LEVELS,
HEAL_MASK_STOPS,
HEAL_MAX_LEVELS,
readHeal,
} from '../../shared/utils/heal';
import { mosaicSkSL, mosaicUniforms, readMosaic } from '../../shared/utils/mosaic';
@@ -670,6 +671,22 @@ function imageShaderChild(image: any): any {
);
}
// The same child, read in the frame's coordinates rather than its own: a pass
// that builds its image on a surface smaller than the frame (HEAL's fill, which
// is only ever asked for the pixels around the brush) has to say where that
// surface's corner sits. The shift is a whole number of pixels and the draw
// that filled it was 1:1, so the sampling stays exact — the reason HEAL uses a
// box at all rather than a scaled draw (see the fill in step 6e).
function imageShaderChildAt(image: any, x: number, y: number): any {
return image.makeShaderOptions(
Skia.TileMode.Clamp,
Skia.TileMode.Clamp,
Skia.FilterMode.Linear,
Skia.MipmapMode.None,
[1, 0, x, 0, 1, y, 0, 0, 1]
);
}
// Snapshot → clear → redraw through the shader the callback builds FROM that
// snapshot: the shape every pass that reads the frame it is about to overwrite
// uses (the negative blurs, the gradient masks, HEAL), in one place so none of
@@ -1293,8 +1310,8 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
// Three moves, in this order. The brush's circles become ONE mask — a union,
// which is what makes a stroke a single shape instead of a row of circles.
// The mask is cut out of the frame, leaving a hole with the photo's own
// pixels all around its rim. And the hole is filled from that rim, by the
// pyramid below, so nothing is copied from anywhere: what was under the
// pixels all around its rim. And the hole is filled from that rim, by a
// blur of the frame, so nothing is copied from anywhere: what was under the
// brush is the light of what borders it.
const heal = readHeal(adjustments.heal);
if (heal.length) {
@@ -1346,63 +1363,69 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
flush(cutSurface);
const cutImage = own(cutSurface.makeImageSnapshot());
// Down: each level is the one below it halved, drawn with a linear filter
// over the premultiplied pixels the cut left behind, so a halving is a
// weighted mean of the pixels that are still there — the hole weighs
// nothing at all, and a level holds the light of the largest
// neighbourhoods around it.
const levels: { surface: any; w: number; h: number }[] = [];
let carried: any = cutImage;
if (carried) {
for (let k = 1; k <= HEAL_MAX_LEVELS; k++) {
const lw = Math.max(1, width >> k);
const lh = Math.max(1, height >> k);
const level = own(createSurface(lw, lh));
const lc = level.getCanvas();
lc.clear(Skia.Color('transparent'));
lc.drawImageRectOptions(
carried,
Skia.XYWHRect(0, 0, carried.width(), carried.height()),
Skia.XYWHRect(0, 0, lw, lh),
Skia.FilterMode.Linear,
Skia.MipmapMode.None
);
flush(level);
levels.push({ surface: level, w: lw, h: lh });
const smaller = own(level.makeImageSnapshot());
if (!smaller || (lw === 1 && lh === 1)) break;
carried = smaller;
// The fill: the cut frame blurred at a few widths, widest first, each
// finer one drawn over it, the untouched cut on top. A blur weighs every
// pixel by how much of the frame's light reaches it, and the hole has none
// of its own left to weigh — so the hole comes back as the light of its
// rim, carried in with no seam and nothing copied from anywhere. Widest
// first means the widest blur only has a say where every finer one has
// nothing (the middle of a hole deeper than a brush), and the untouched
// cut wins wherever the photo is still there, so the rim is the rim.
//
// Blurs and not a pyramid of halvings, on purpose. This Skia gets the
// premultiply wrong on a snapshot used as a SCALED draw source — its rgb
// comes back with the alpha multiplied in a second time — and a pyramid is
// built of exactly those halvings; a partly transparent rim then came out
// a wash of grey, which is what the heal was painting over the photo.
// Every draw here is 1:1, which filters correctly (measured, see
// heal-check.mjs).
//
// How many levels: the finest is HEAL_FILL_BLUR brushes wide and each one
// is twice the last, so n of them reach HEAL_FILL_BLUR * 2^(n-1) radii —
// enough to span half the stroke's own diagonal from either end of it.
// Fewer and the middle of a long hole stays bare; more just costs time,
// and past HEAL_FILL_MAX_LEVELS the blur is wider than the frame anyway.
//
// All of it is drawn on a surface the size of the brush's own box rather
// than the frame. The fill is read only where the mask is (the composite
// below mixes it at the mask's alpha, and nowhere else), so the pixels a
// wide blur would carry in from the frame's far corners are pixels nobody
// ever asks for — and at the zoom a repair is aimed at, lifting a wide
// blur over 3200x2400 instead of over the 500px around the repair is the
// difference between a second and a thirtieth of one (measured). The box
// is the mask's own, padded by three sigma of the widest level, so the
// blur at its edge still reads the rim it is made of rather than a clamp.
const widest = Math.max(...heal.map((s) => s.r * width));
const xs = heal.map((s) => s.x * width);
const ys = heal.map((s) => s.y * height);
const span = Math.hypot(Math.max(...xs) - Math.min(...xs), Math.max(...ys) - Math.min(...ys));
const levelCount = Math.min(
HEAL_FILL_MAX_LEVELS,
Math.max(2, 1 + Math.ceil(Math.log2(2 + span / 2 / widest)))
);
const pad = 3 * HEAL_FILL_BLUR * widest * 2 ** (levelCount - 1);
const boxLeft = Math.max(0, Math.floor(Math.min(...xs) - widest - pad));
const boxTop = Math.max(0, Math.floor(Math.min(...ys) - widest - pad));
const boxRight = Math.min(width, Math.ceil(Math.max(...xs) + widest + pad));
const boxBottom = Math.min(height, Math.ceil(Math.max(...ys) + widest + pad));
const fillSurface = own(createSurface(boxRight - boxLeft, boxBottom - boxTop));
const fillCanvas = fillSurface.getCanvas();
// The box's own corner is the surface's, and both rects are the same one:
// a 1:1 copy at an offset, which is the draw this Skia filters correctly.
const box = Skia.XYWHRect(boxLeft, boxTop, boxRight - boxLeft, boxBottom - boxTop);
if (cutImage) {
for (let k = levelCount; k >= 1; k--) {
const sigma = HEAL_FILL_BLUR * widest * 2 ** (k - 1);
const blurPaint = own(Skia.Paint());
blurPaint.setImageFilter(own(Skia.ImageFilter.MakeBlur(sigma, sigma, Skia.TileMode.Clamp, null)));
fillCanvas.drawImageRect(cutImage, box, box, blurPaint);
}
// The cut itself, unblurred, on top: the pixels the brush did not touch
// are the photo exactly, not a blur of it.
fillCanvas.drawImageRect(cutImage, box, box, own(Skia.Paint()));
}
// Up: the coarsest level is already an answer for the whole frame, so it
// is drawn over the level below it — whose hole only that level could
// reach — that result over the one below it, and so on down to the
// half-size level. What comes back is the photo where the photo was, and
// the rim's own light carried inward where it was not.
let up: any = null;
let fillImage: any = null;
for (let k = levels.length - 1; k >= 0; k--) {
const { surface: level, w: lw, h: lh } = levels[k];
if (up) {
const lc = level.getCanvas();
lc.drawImageRectOptions(
up,
Skia.XYWHRect(0, 0, up.width(), up.height()),
Skia.XYWHRect(0, 0, lw, lh),
Skia.FilterMode.Linear,
Skia.MipmapMode.None
);
}
flush(level);
const pulled = own(level.makeImageSnapshot());
if (!pulled) break;
if (k === 0) {
fillImage = pulled;
break;
}
up = pulled;
}
flush(fillSurface);
const fillImage = own(fillSurface.makeImageSnapshot());
// And the fill back under the brush, in one draw: the frame, the fill and
// the mask as its three children, with the mask the weight the frame and
@@ -1412,7 +1435,7 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
const shader = own(
fillEffect.makeShaderWithChildren([], [
own(imageShaderChild(snap)),
own(imageShaderChild(fillImage)),
own(imageShaderChildAt(fillImage, boxLeft, boxTop)),
own(imageShaderChild(maskImage)),
])
);