3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(studio): a heal stroke is one mask, and the fill comes from its rim

Browse Source 
The dust brush had two faults and the second was hidden behind the first.

A stroke was a row of independent repairs. Every circle the brush laid down
found a patch of its own somewhere else in the frame and pasted it, so a stroke
of twenty spots was twenty pastes with twenty rims to match, and what showed on
the photo was the run of circles itself. The mask was their union on paper but
not on the picture.

And the mask was not a stroke either. The brush only ever dropped a spot where
the pointer's own sample fell, and a fast drag reports a sample every few
centimetres of screen — several brush radii apart. Circles that far apart do not
touch, so their union was the row of dots the tool was complained about. The
segment the pointer travelled is now walked at the brush's own spacing and a spot
is laid at every step, whatever speed the stroke was drawn at.

With the mask one shape, the patch had nothing left to justify it. The fill is
now a push-pull pyramid over the whole mask: the painted pixels are cut out of
the frame as a hole, the remaining pixels are halved and halved again until a
level is small enough that no hole is left in it, and each level is drawn back
over the one below it. What comes up out of that is the photo where the photo
was, and the light of the mask's own rim carried inward where it was not —
nothing is copied from anywhere, and a repair matches the grade, the blur and
the grain around it because those are the pixels it was made from.

Non-destructive as before: the recipe still holds the circles, and the preview
and the full-size export run the two passes in engine/exportEngine.ts, the
shaders and the mask stops in shared/utils/heal.ts. A recipe written by the old
brush still loads; the patch coordinates it stored are simply not read any more.
This commit is contained in:
3dtours
2026-10-12 16:15:18 +07:00
parent 3ac58826d5
commit 5ac2ffb916
6 changed files with 678 additions and 519 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/scripts/heal-check.mjs
+315
View File
@@ -0,0 +1,315 @@
// HEAL is a mask and a fill now, not a patch pasted under each spot. Four
// things can go wrong and none of them throws: the circles of one stroke can
// stay separate shapes (which is what a row of round grey blobs on the photo
// is), the fill can fail to carry the light of the photo around it (a repair
// that reads as a mark of its own), the search can still borrow a patch from
// somewhere (the card three radii away, which is the brush the user complained
// about), or the composite can touch pixels it was never given (a smear across
// the whole frame). So the mask, the fill and the untouched ground are all
// measured here, on synthetic photos whose answer is known, with CanvasKit
// running the same two passes exportEngine runs.
//
// node scripts/heal-check.mjs
import assert from 'node:assert/strict';
import { mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import ts from 'typescript';
import CanvasKitInit from 'canvaskit-wasm/bin/full/canvaskit.js';
const transpile = (path) =>
ts.transpileModule(readFileSync(new URL(path, import.meta.url), 'utf8'), {
compilerOptions: { module: ts.ModuleKind.ESNext, target: ts.ScriptTarget.ES2022 },
}).outputText;
const dir = mkdtempSync(join(tmpdir(), 'heal-check-'));
writeFileSync(join(dir, 'heal.mjs'), transpile('../shared/utils/heal.ts'));
const { readHeal, HEAL_MASK_STOPS, HEAL_MAX_LEVELS, HEAL_CUT_SKSL, HEAL_FILL_SKSL } = await import(
pathToFileURL(join(dir, 'heal.mjs')).href
);
// The other half of the check: the passes below are these two shaders, so the
// engine has to be compiling them — and it must not still be building the
// per-spot shader this replaced.
const engine = readFileSync(new URL('../src/engine/exportEngine.ts', import.meta.url), 'utf8');
assert.match(engine, /healCutEffect = Skia\.RuntimeEffect\.Make\(HEAL_CUT_SKSL\)/, 'exportEngine does not compile HEAL_CUT_SKSL');
assert.match(engine, /healFillEffect = Skia\.RuntimeEffect\.Make\(HEAL_FILL_SKSL\)/, 'exportEngine does not compile HEAL_FILL_SKSL');
assert.doesNotMatch(engine, /healSkSL\(/, 'the per-spot heal shader is still in exportEngine');
assert.doesNotMatch(engine, /healUniforms\(/, 'the per-spot uniform block is still in exportEngine');
// --- the stored spots --------------------------------------------------------
// A recipe written by the older brush carries sx/sy. It still loads: those two
// numbers are no longer read, and nothing else about the spot moves.
assert.deepEqual(readHeal([{ x: 0.5, y: 0.5, r: 0.02, sx: 0.9, sy: 0.1 }]), [{ x: 0.5, y: 0.5, r: 0.02 }]);
// Out of frame, non-numeric, or a radius of nothing: a spot the brush and the
// renderer would disagree about is dropped either way.
assert.deepEqual(readHeal([{ x: 2, y: -1, r: 0.05 }, { x: 'a', y: 0, r: 0.05 }, { x: 0.5, y: 0.5, r: 0 }]), [
{ x: 1, y: 0, r: 0.05 },
{ x: 0, y: 0, r: 0.05 },
]);
const CanvasKit = await CanvasKitInit({
locateFile: () => fileURLToPath(new URL('../node_modules/canvaskit-wasm/bin/full/canvaskit.wasm', import.meta.url)),
});
const W = 160;
const H = 160;
const GREY = 118;
const DUST = 38;
const R = 0.05; // the brush: an 8px radius on this frame
// A stroke of nine spots, one every 0.6 of a radius, running down the frame.
const spots = Array.from({ length: 9 }, (_, i) => ({ x: 0.5, y: (48 + i * R * 0.6 * W) / H, r: R }));
const cx = 0.5 * W;
const cy = 48 + 4 * R * 0.6 * W;
// Three radii to the right of the stroke: where the old search looked.
const CARD_X = Math.round(cx + 3.2 * R * W);
const shadeFlat = (x, y) => {
if (Math.abs(x - CARD_X) < 6 && Math.abs(y - cy) < 6) return [220, 30, 30];
if (Math.hypot(x - cx, y - cy) < 6) return [DUST, DUST, DUST];
return [GREY, GREY, GREY];
};
const shadeRamp = (x) => {
const v = 40 + (120 * x) / (W - 1);
return [v, v, v];
};
function bytesOf(shade) {
const rgba = new Uint8Array(W * H * 4);
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
const [r, g, b] = shade(x, y);
const i = (y * W + x) * 4;
rgba[i] = r;
rgba[i + 1] = g;
rgba[i + 2] = b;
rgba[i + 3] = 255;
}
}
return rgba;
}
const imageOf = (rgba) =>
CanvasKit.MakeImage(
{
width: W,
height: H,
colorType: CanvasKit.ColorType.RGBA_8888,
alphaType: CanvasKit.AlphaType.Unpremul,
colorSpace: CanvasKit.ColorSpace.SRGB,
},
rgba,
W * 4
);
const surface = (w, h) => {
const s = CanvasKit.MakeSurface(w, h);
s.getCanvas().clear(CanvasKit.TRANSPARENT);
return s;
};
const pixelsOf = (image) => {
const s = surface(image.width(), image.height());
s.getCanvas().drawImage(image, 0, 0);
return s.getCanvas().readPixels(0, 0, {
width: image.width(),
height: image.height(),
colorType: CanvasKit.ColorType.RGBA_8888,
alphaType: CanvasKit.AlphaType.Unpremul,
colorSpace: CanvasKit.ColorSpace.SRGB,
});
};
const asChild = (image) =>
image.makeShaderOptions(CanvasKit.TileMode.Clamp, CanvasKit.TileMode.Clamp, CanvasKit.FilterMode.Linear, CanvasKit.MipmapMode.None);
// The brush's circles as ONE mask: a radial gradient per spot, solid out to
// HEAL_FEATHER of its radius and clear at its edge, each laid over the last.
function maskOf(spotsIn) {
const surf = surface(W, H);
const canvas = surf.getCanvas();
for (const s of spotsIn) {
const rad = s.r * W;
const shader = CanvasKit.Shader.MakeRadialGradient(
[s.x * W, s.y * H],
rad,
new Float32Array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0]),
HEAL_MASK_STOPS,
CanvasKit.TileMode.Clamp
);
const paint = new CanvasKit.Paint();
paint.setAntiAlias(true);
paint.setShader(shader);
canvas.drawCircle(s.x * W, s.y * H, rad, paint);
}
return surf.makeImageSnapshot();
}
// exportEngine's 6e, pass for pass: cut, the pyramid down and up again, and the
// fill laid back under the brush.
function healPass(rgba, spotsIn) {
const src = imageOf(rgba);
const mask = maskOf(spotsIn);
const cutEffect = CanvasKit.RuntimeEffect.Make(HEAL_CUT_SKSL);
assert.ok(cutEffect, 'HEAL_CUT_SKSL does not compile');
const cut = surface(W, H);
const cutPaint = new CanvasKit.Paint();
cutPaint.setShader(cutEffect.makeShaderWithChildren([], [asChild(src), asChild(mask)]));
cut.getCanvas().drawRect(CanvasKit.XYWHRect(0, 0, W, H), cutPaint);
const levels = [];
let carried = cut.makeImageSnapshot();
for (let k = 1; k <= HEAL_MAX_LEVELS; k++) {
const lw = Math.max(1, W >> k);
const lh = Math.max(1, H >> k);
const level = surface(lw, lh);
level
.getCanvas()
.drawImageRectOptions(
carried,
CanvasKit.XYWHRect(0, 0, carried.width(), carried.height()),
CanvasKit.XYWHRect(0, 0, lw, lh),
CanvasKit.FilterMode.Linear,
CanvasKit.MipmapMode.None
);
levels.push({ surf: level, w: lw, h: lh });
carried = level.makeImageSnapshot();
if (lw === 1 && lh === 1) break;
}
let up = null;
let fillImage = null;
for (let k = levels.length - 1; k >= 0; k--) {
const { surf, w: lw, h: lh } = levels[k];
if (up) {
surf
.getCanvas()
.drawImageRectOptions(
up,
CanvasKit.XYWHRect(0, 0, up.width(), up.height()),
CanvasKit.XYWHRect(0, 0, lw, lh),
CanvasKit.FilterMode.Linear,
CanvasKit.MipmapMode.None
);
}
const pulled = surf.makeImageSnapshot();
if (k === 0) {
fillImage = pulled;
break;
}
up = pulled;
}
const fillEffect = CanvasKit.RuntimeEffect.Make(HEAL_FILL_SKSL);
assert.ok(fillEffect, 'HEAL_FILL_SKSL does not compile');
const out = surface(W, H);
const paint = new CanvasKit.Paint();
paint.setShader(fillEffect.makeShaderWithChildren([], [asChild(src), asChild(fillImage), asChild(mask)]));
out.getCanvas().drawRect(CanvasKit.XYWHRect(0, 0, W, H), paint);
return { pixels: pixelsOf(out.makeImageSnapshot()), mask: pixelsOf(mask) };
}
const flat = bytesOf(shadeFlat);
const made = healPass(flat, spots);
const alphaAt = (x, y) => made.mask[(y * W + x) * 4 + 3];
const inMask = (x, y) => alphaAt(x, y) > 200;
const rgbAt = (x, y) => {
const i = (y * W + x) * 4;
return [made.pixels[i], made.pixels[i + 1], made.pixels[i + 2]];
};
// 1. The mask is one shape. Painting a stroke as nine overlapping spots has to
// leave one painted region, not nine: every painted pixel is reachable from
// the first spot's centre without ever stepping off the mask. This is what
// "mask liền mạch" is, and it is the thing the row of circles failed.
const painted = [];
for (let y = 0; y < H; y++) for (let x = 0; x < W; x++) if (inMask(x, y)) painted.push(y * W + x);
const seen = new Set();
const queue = [painted[0]];
seen.add(painted[0]);
while (queue.length) {
const p = queue.pop();
const x = p % W;
const y = (p - x) / W;
for (const [dx, dy] of [
[1, 0],
[-1, 0],
[0, 1],
[0, -1],
]) {
const nx = x + dx;
const ny = y + dy;
if (nx < 0 || ny < 0 || nx >= W || ny >= H) continue;
const q = ny * W + nx;
if (seen.has(q) || !inMask(nx, ny)) continue;
seen.add(q);
queue.push(q);
}
}
assert.equal(seen.size, painted.length, `${painted.length - seen.size} painted pixels are not reachable from the rest of the stroke`);
// ...and the mask is solid in the middle and clear well outside it.
assert.equal(alphaAt(Math.round(cx), Math.round(cy)), 255, 'the middle of the stroke is not fully painted');
assert.equal(alphaAt(2, 2), 0, 'the mask leaked into the corner of the frame');
// 2. The dust is gone and nothing was borrowed to do it: every painted pixel
// ends on the ground's own value, 3.2 radii from a red card that a patch
// search would have pasted in.
let worstIn = 0;
for (const p of painted) {
const i = p * 4;
for (let c = 0; c < 3; c++) worstIn = Math.max(worstIn, Math.abs(made.pixels[i + c] - GREY));
}
assert.ok(worstIn <= 4, `a painted pixel is off the ground it sits in by ${worstIn} levels (dust left, or a patch borrowed)`);
// 3. Nothing survives INSIDE the stroke either: neighbouring painted pixels
// differ by a level or two, the way the ground does, where a spot solved on
// its own would step at every seam.
let worstStep = 0;
for (let y = 0; y + 1 < H; y++) {
for (let x = 0; x + 1 < W; x++) {
if (!inMask(x, y) || !inMask(x + 1, y)) continue;
worstStep = Math.max(worstStep, Math.abs(rgbAt(x, y)[0] - rgbAt(x + 1, y)[0]));
}
}
assert.ok(worstStep <= 4, `the fill steps by ${worstStep} levels between neighbours inside the mask`);
// 4. The repair carries the LIGHT of where it landed. On a photo running from
// dark to light across the frame, the painted pixels come back on that ramp
// rather than at one flat value borrowed from somewhere else. The bound is
// the ramp's own steepness, not zero: the pyramid reads the rim of the hole,
// and where the hole takes weight off one side of a pixel its average slides
// that way by about a pixel's worth of the ramp — 0.75 levels a pixel across
// this frame. On the flat ground of 2 and 3 it is nothing at all.
const ramped = healPass(bytesOf(shadeRamp), spots);
let worstRamp = 0;
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
if (ramped.mask[(y * W + x) * 4 + 3] < 250) continue;
worstRamp = Math.max(worstRamp, Math.abs(ramped.pixels[(y * W + x) * 4] - shadeRamp(x)[0]));
}
}
assert.ok(worstRamp <= 6, `the fill is off the ramp it sits in by ${worstRamp} levels`);
// 5. Outside the mask, nothing moved. The pass is asked for the painted pixels
// and no others.
let outWorst = 0;
for (let p = 0; p < W * H; p++) {
if (inMask(p % W, (p - (p % W)) / W)) continue;
for (let c = 0; c < 3; c++) outWorst = Math.max(outWorst, Math.abs(made.pixels[p * 4 + c] - flat[p * 4 + c]));
}
assert.equal(outWorst, 0, `a pixel outside the mask changed by ${outWorst} levels`);
// 6. The same photo and the same spots give the same photo, byte for byte.
assert.deepEqual(
Buffer.from(healPass(flat, spots).pixels),
Buffer.from(made.pixels),
'the pass is not deterministic'
);
// 7. A mask with no rim to read from — one spot over the whole frame — leaves
// the photo alone. Nothing is painted black, because nothing was found to
// paint it with, and a frame is still a frame.
const all = healPass(flat, [{ x: 0.5, y: 0.5, r: 2 }]);
let worstAll = 0;
for (let p = 0; p < W * H; p++) {
for (let c = 0; c < 3; c++) worstAll = Math.max(worstAll, Math.abs(all.pixels[p * 4 + c] - flat[p * 4 + c]));
}
assert.equal(worstAll, 0, `a frame-wide mask changed the photo by ${worstAll} levels`);
console.log('ok the stroke is one mask, the fill carries the light, and nothing outside it moved');
docker/frontend/shared/types/index.ts
+11 -7
View File
@@ -100,17 +100,21 @@ export type ToneCurvePoint = [number, number];
export type ToneCurveChannel = 'rgb' | 'r' | 'g' | 'b';
export type ToneCurve = Partial<Record<ToneCurveChannel, ToneCurvePoint[]>>;
// FX tab > HEAL. One repaired spot: the circle the brush painted on the photo
// and the patch it borrowed, both as fractions of the rendered photo — x/y/sx/sy
// of its width and height, r of its width. Fractions rather than pixels because
// the same spot has to land on the preview and on the full-size export, and a
// fraction of the width keeps the circle round on any photo shape.
// FX tab > HEAL. One repaired spot: the circle the brush painted on the photo,
// as fractions of the rendered photo — x/y of its width and height, r of its
// width. Fractions rather than pixels because the same spot has to land on the
// preview and on the full-size export, and a fraction of the width keeps the
// circle round on any photo shape.
//
// There is no patch here, and there used to be (sx/sy): what the brush paints is
// where the photo is to be repaired, and the pixels that repair it are worked
// out by the renderer from the photo around it (shared/utils/heal.ts), not
// borrowed from a second circle the user never placed. Recipes written by the
// older brush still load — the extra two numbers are simply no longer read.
export interface HealSpot {
x: number;
y: number;
r: number;
sx: number;
sy: number;
}
// FX tab > MOSAIC. One hidden spot: the circle the brush painted on the photo,
docker/frontend/shared/utils/heal.ts
+80 -293
View File
@@ -1,324 +1,111 @@
import type { HealSpot } from '../types';
// FX tab > HEAL — the dust brush, and the patch search behind it.
// FX tab > HEAL — the dust brush, and the fill behind it.
//
// A spot is a circle on the rendered photo plus the patch it borrows: the
// renderer copies the pixels at (sx, sy) onto (x, y), feathers the edge, and
// shifts what it pasted onto the light of the place it landed in, so a repair
// is a draw of the same picture rather than a blur over the dust. All five
// numbers are fractions of the render — x/y/sx/sy of its width and height, r of
// its width — which is what makes one set of spots survive the preview and the
// export rendering the same photo at two sizes, and keeps the circle round
// whatever the photo's shape.
// What the brush paints is a mask: the circles it laid down, all of them at
// once, as ONE shape. What fills that shape is not borrowed from anywhere — the
// renderer takes the photo around the mask's rim and works inward, so the
// repair is made of the light and the colour that already border it. That is
// the whole of the change from a patch brush: a stroke of twenty overlapping
// spots used to be twenty independent repairs, each pasting its own patch and
// matching its own rim, which is exactly what a run of circles looks like on the
// photo; one mask and one solve have no circles in them to see.
//
// There is no ceiling on the list. The shader is built to carry exactly the
// spots the recipe holds (healSkSL), so a new repair can never push an old one
// out: the dust you healed first is the dust that stays healed.
// Non-destructive like everything else here: the recipe keeps the circles, and
// the preview and the export run the same two passes over them.
//
// The source is SEARCHED for rather than asked for. Lightroom picks the patch
// and lets you drag the second circle afterwards; the search below is the same
// idea without the second circle, and it is a pure function of a sampler so a
// synthetic picture can hold it to account.
//
// Feather, as a fraction of the radius: inside it the patch is copied, outward
// it fades to nothing, so the circle leaves no rim of its own. It is the outer
// 15% and no more, because that band is the only place the dust being repaired
// is mixed back into the patch — a wide fade keeps the speck's own edge alive
// as a faint ring inside the circle, which is a blur of the dust rather than a
// repair. sub-pixel at the default brush, still a soft edge at a big one.
export const HEAL_FEATHER = 0.85;
// x/y are fractions of the render's width and height, r of its width. The
// fractions are what make one set of spots land on the preview and on the
// full-size export, and keep a spot round whatever the photo's shape.
// The brush's radius, as a fraction of the photo's width — where this tool
// starts, at the sensor-dust end, where a speck is a few thousandths of the
// frame. The bounds and the wheel that moves it between them belong to the
// brush both FX tools share (brush.ts).
// frame. The bounds and the wheel that move it between them belong to the brush
// both FX tools share (brush.ts).
export const HEAL_DEFAULT_R = 0.012;
// How far the search looks, in radii, and how many directions it looks in.
const SEARCH_DISTANCES = [2.6, 4.2, 6.5];
const SEARCH_DIRS = 8;
// The taps that stand for "the light around a spot": a ring just OUTSIDE the
// brush, where the dust being repaired is not — the scale the eye reads a
// spot's surroundings at. Reading the inside instead compares a candidate with
// the very dark the repair is trying to erase, and the patch that matches a
// speck best is then the one carrying a speck of its own — which is how a
// repair ends up moving dust a few pixels instead of removing it.
//
// Twelve taps read as their MEDIAN, not their mean. The ring can only be a
// little way out — far enough out and it would be reading a light the spot does
// not sit in — so a few of its taps land on the speck's own softened edge, or
// on a neighbour's: readings in the minority, which the median drops and the
// mean would drag the whole light down by.
const RING_R = 1.15;
const RING_TAPS: [number, number][] = Array.from({ length: 12 }, (_, i) => {
const a = (i / 12) * Math.PI * 2;
return [Math.cos(a), Math.sin(a)] as [number, number];
});
// The taps that decide whether a candidate patch is clean: the inside of the
// patch, held against the patch's own mean. Dust is an outlier in its own
// neighbourhood; skin, however grainy, is not. The last ring is the patch's own
// edge — the circle is copied at full strength out to HEAL_FEATHER of its
// radius, so that is where a neighbour's dust leaking into the patch shows up,
// and the middle of the patch would never see it.
const INSIDE_TAPS: [number, number][] = [
[0, 0],
[-0.5, 0],
[0.5, 0],
[0, -0.5],
[0, 0.5],
[-0.35, -0.35],
[0.35, -0.35],
[-0.35, 0.35],
[0.35, 0.35],
...Array.from({ length: 8 }, (_, i) => {
const a = (i / 8) * Math.PI * 2;
return [Math.cos(a) * HEAL_FEATHER, Math.sin(a) * HEAL_FEATHER] as [number, number];
}),
];
// How far a candidate's light may sit from the spot's own before it is refused
// outright, in the 0-255 levels the sampler answers in. Sending a patch from a
// different light is exactly what makes a repair show up as a mark of its own —
// a light square where dark ground was, a dark one where light was — so past
// this the search gives up and the caller leaves the speck alone. The user sees
// an untouched speck, which is honest, and tries another size or another spot.
const LIGHT_GATE = 20;
// Light outweighs cleanliness in the score, so the winner is the same light
// first and the cleaner patch second; within the gate the cleanliness term only
// breaks ties between patches the eye would call the same.
const LIGHT_WEIGHT = 3;
const num = (v: unknown, fallback: number) => {
const n = Number(v);
return Number.isFinite(n) ? n : fallback;
};
const clamp01 = (v: number) => (v < 0 ? 0 : v > 1 ? 1 : v);
// How much of a spot is painted at full strength, as a fraction of its radius:
// out to here the mask is solid, and the last 15% is the feather that carries it
// to nothing. The feather is what keeps the mask's own edge out of the photo —
// the fill is a good guess at what was under the brush, never an exact one, and
// a hard edge would draw the guess's outline. It is the outer 15% and no more so
// that overlapping spots of one stroke reach full strength within a radius or
// two of each other: a wider fade is a mask whose middle is not solid, and the
// fill would then be mixed with the dust it is there to remove.
export const HEAL_FEATHER = 0.85;
// How many halvings of the frame the fill reads, at most. Each level is half of
// the one below it, so twelve covers any photo a camera makes and stops a
// pathological surface size from walking off the end of the pyramid.
export const HEAL_MAX_LEVELS = 12;
// The stored spots, made readable: numbers, inside the frame. Everything below
// reads a recipe through this, so a hand-written or older file cannot produce a
// spot the brush and the renderer disagree about.
// spot the brush and the renderer disagree about. The patch the older brush
// stored (sx/sy) is not read: a repair is no longer a copy of another place.
export function readHeal(heal: HealSpot[] | undefined): HealSpot[] {
if (!Array.isArray(heal)) return [];
return heal
.map((s) => ({
x: clamp01(num(s?.x, 0)),
y: clamp01(num(s?.y, 0)),
x: Math.min(1, Math.max(0, num(s?.x, 0))),
y: Math.min(1, Math.max(0, num(s?.y, 0))),
r: Math.max(0, num(s?.r, 0)),
sx: clamp01(num(s?.sx, s?.x ?? 0)),
sy: clamp01(num(s?.sy, s?.y ?? 0)),
}))
.filter((s) => s.r > 0);
}
// The uniform block the shader for `n` spots reads: the circles, the patches,
// then the frame the fractions are of. Declaration order, arrays expanded —
// that is how the runtime effect wants its uniforms, and one buffer is one
// upload per render. Its length is a function of the list, not a fixed
// capacity, because the shader carries exactly the spots the recipe holds.
export function healUniforms(spots: HealSpot[], width: number, height: number): Float32Array {
const list = readHeal(spots);
const n = list.length;
const u = new Float32Array((n * 2 + 1) * 4);
for (let i = 0; i < n; i++) {
const s = list[i];
u.set([s.x, s.y, s.r, 0], i * 4);
u.set([s.sx, s.sy, 0, 0], (n + i) * 4);
}
u.set([width, height, 0, 0], n * 2 * 4);
return u;
function num(v: unknown, fallback: number): number {
const n = Number(v);
return Number.isFinite(n) ? n : fallback;
}
// The ring the pasted circle is matched at, and how its taps are laid out.
// Measured just OUTSIDE the brush — RING_R of its radius, where the search
// reads a spot's light too — because that is the only ring with nothing of the
// repair in it: inside the circle is the borrowed patch at one radius and the
// speck being covered at another, and a light read off either of those is the
// dust talking rather than the photo. Outside, both sides are photographs — the
// ground the repair has to sit in, and the ground the patch was borrowed from.
// Sixteen taps are spread around it, at the SAME sixteen places on both sides,
// so a tap's pair of readings differ by the light between the two places and
// not by where in the photo they were taken. Sixteen rather than eight because
// a tap that lands on an edge bends its own direction only as far as the next
// ring tap: the finer the ring, the less of the rim one hard pixel of the photo
// can reach around.
const BLEND_TAPS = 16;
// ...and one pixel further out again than RING_R. The ring is read by a pixel
// that lies under the brush, so the reading has to clear the dust's own edge,
// and that edge is soft: a speck r pixels across spreads about a pixel past
// where it is drawn, in the pixels the shader samples. RING_R alone leaves the
// nearest tap or two inside that spread, and a tap that lands on the dust is
// the dust's light — which is exactly the blotch this reading exists to avoid.
// One pixel is the width of the spread, in the same pixels the sampler works
// in, so it is a pixel here rather than a fraction of the radius: a fraction
// would be nothing at all at the sensor-dust end of the brush.
const RING_PAD = 1;
// ...but one light for the whole circle is a light no ring has when the ring is
// not all one thing. A twig against the sky, a hairline, the edge of a table
// under the brush: a few taps of the ring land on the thing and the rest on the
// ground, and the MEAN of them all is a colour that is neither — a level the
// place does not have, pasted over the whole circle. The donor can be of exactly
// the right light and still come out as a blotch, which is the mark the repair
// was supposed to stop leaving. So the light is read once PER DIRECTION, and a
// pixel takes the correction of the two readings it lies between, interpolated
// by its own angle around the spot: the rim then meets the place all the way
// round instead of on average, and the taps that landed on the twig correct the
// part of the rim that is near the twig rather than the whole of it. It is the
// same taps and the same one draw — each pixel reads them at its own angle
// instead of sharing everyone's single number.
// One unrolled block per spot. SkSL indexes a uniform array by constant only
// (see TONE_SKSL's mixer), so the spots are written out rather than looped, and
// the shader is built for the count it is handed rather than for a capacity —
// that is what lets the list be uncapped. A count costs one RuntimeEffect to
// compile, so the renderer caches them by count (exportEngine's healEffectFor).
//
// What is pasted is the patch's own pixels SHIFTED onto the light of the place
// it lands in — the gradient of the borrowed texture, carried at the level of
// the destination: the border is made to match, and everything inside keeps the
// texture it was borrowed for. That is the whole of "seamless" the brush needs:
// at the rim the patch sits within a level or two of the photo around it,
// instead of up to LIGHT_GATE levels off it, so the repair stops reading as a
// soft blotch of its own and the feather has almost nothing left to hide. The
// correction is read per direction around the ring, so the border is matched
// where it is rather than on average, and the pixels of a twig that run through
// the ring bend the part of the rim they are near instead of the whole circle.
//
// ponytail: the correction lives on the rim — every pixel takes the two ring
// readings it lies between and interpolates — so it is the boundary of a Poisson
// edit and not its interior: a repair laid over something the patch cannot
// reproduce still carries the patch's own texture inside, bent to fit the rim.
// The other half of the solve (a correction that relaxes inside the circle, a
// Jacobi ping-pong per spot) lands when a real photo shows an interior the eye
// can find from the rim alone. A direction's level is also one tap pair on each
// side, so the grain the mean used to average away now rides the rim as a wedge
// of a level or two across the circle — a tangential average of three taps per
// direction is the rung for that, when a photo shows the wedge.
const spotBlock = (i: number) => `
{
float4 s = spots[${i}];
if (s.z > 0.0) {
float rad = s.z * size.x;
float d = distance(pos, s.xy * size.xy);
if (d < rad) {
float4 t = srcs[${i}];
float2 pd = s.xy * size.xy;
float2 ps = t.xy * size.xy;
float r = rad * ${HEAL_FEATHER};
float rr = rad * ${RING_R} + ${RING_PAD.toFixed(1)};
// Which way round the spot this pixel sits, one millionth of a pixel off
// the exact centre so the angle is defined there too.
float ang = atan(pos.y - pd.y + 1e-6, pos.x - pd.x + 1e-6);
half3 corr = half3(0.0);
float wsum = 0.0;
for (int k = 0; k < ${BLEND_TAPS}; k++) {
float a = float(k) * 6.283185307 / float(${BLEND_TAPS});
float2 o = float2(cos(a), sin(a)) * rr;
// How far this pixel's angle is from this tap's, the short way round.
float da = ang - a;
da = abs(da - 6.283185307 * floor(da * 0.1591549431 + 0.5));
float w = max(0.0, 1.0 - da * float(${BLEND_TAPS}) * 0.1591549431);
corr += half(w) * (img.eval(pd + o).rgb - img.eval(ps + o).rgb);
wsum += w;
}
corr *= half(1.0 / wsum);
half m = half(1.0 - smoothstep(r, rad, d));
c = mix(c, img.eval(pos + ps - pd) + half4(corr, 0.0), m);
}
}
}
`;
// The pass. It reads the pixels the pipeline has already built (the child is a
// snapshot of the surface) and writes the borrowed patches back over them, so a
// repair is one draw: no blur, no smoothing, and the grain and the frame land
// on top of it afterwards exactly as they land on the rest of the photo.
export function healSkSL(count: number): string {
return `
// The first of the fill's two passes: the photo with the painted pixels taken
// out of it. The colour of every pixel is kept and its alpha drops to what the
// brush did not touch, which is what turns the mask into a hole the second pass
// can see the rim of. Written as premultiplied — rgb times that alpha — because
// that is what every surface here holds, and the pyramid below is built on it:
// a linear halving of premultiplied pixels is a weighted average of the pixels
// that are still there, with the hole weighing nothing.
export const HEAL_CUT_SKSL = `
uniform shader img;
uniform float4 spots[${count}];
uniform float4 srcs[${count}];
uniform float4 size;
uniform shader mask;
half4 main(float2 pos) {
half4 c = img.eval(pos);${Array.from({ length: count }, (_, i) => spotBlock(i)).join('')}
return c;
half4 c = img.eval(pos);
half a = 1.0 - mask.eval(pos).a;
half3 straight = c.a > 0.0001 ? c.rgb / c.a : c.rgb;
return half4(straight * a, a);
}
`;
}
// The patch to borrow for a spot at (x, y) of radius r, from a sampler that
// answers fractions of the same photo. The candidates are a ring of offsets in
// eight directions at three distances — the patch has to be far enough that the
// dust is not in it, near enough that the light is the same — plus each one
// mirrored through the spot, which is the pair Lightroom's own auto-source
// leans on.
// The second pass: the fill laid back under the brush. `fill` is the pyramid's
// answer for every pixel of the frame — at the mask it is what the rim implies,
// away from it it is the photo itself — and the mask is the weight it is mixed
// at, so the rim is the only place the two meet and they already agree there.
//
// Each candidate is asked two questions. "Is this the same light?" holds the
// mean of what would be pasted against the mean of the spot's own ring, the
// pixels just outside the dust — the light the repair has to sit in, read where
// the dust is not, because comparing against the spot's own inside compares a
// candidate with the very dark the repair is trying to erase, and the patch
// that matches a speck best is then the one carrying a speck of its own. "Is
// there dust on it?" holds the candidate's inside against its own inside mean:
// dust is an outlier in its own neighbourhood, grain is not. A candidate whose
// light is off by more than LIGHT_GATE is not scored at all — a patch from the
// wrong light is a mark of its own, and moving it a few pixels for a wrong
// clone is not a repair.
//
// Returns null when the frame is too small to hold any candidate, or when every
// candidate is refused: the caller then leaves the speck where it is rather
// than pasting a patch it cannot stand behind.
export function findHealSource(
sample: (fx: number, fy: number) => { r: number; g: number; b: number },
x: number,
y: number,
r: number
): { sx: number; sy: number } | null {
if (!(r > 0)) return null;
const inside = (cx: number, cy: number) => cx - r >= 0 && cx + r <= 1 && cy - r >= 0 && cy + r <= 1;
const rgb = (v: { r: number; g: number; b: number }) => (v.r + v.g + v.b) / 3;
// The light the repair has to sit in: the median of the spot's ring, where
// the dust is not. Measured once, because it is the same number for every
// candidate.
const ring = RING_TAPS.map(([dx, dy]) => rgb(sample(clamp01(x + dx * RING_R * r), clamp01(y + dy * RING_R * r)))).sort(
(a, b) => a - b
);
const target = (ring[RING_TAPS.length / 2 - 1] + ring[RING_TAPS.length / 2]) / 2;
// The score of one candidate: null when it is refused for its light, else how
// far off that light is times its weight, plus how much the patch varies
// inside itself.
const score = (cx: number, cy: number): number | null => {
let content = 0;
const inside: number[] = [];
for (const [dx, dy] of INSIDE_TAPS) {
const v = rgb(sample(clamp01(cx + dx * r), clamp01(cy + dy * r)));
inside.push(v);
content += v;
}
content /= INSIDE_TAPS.length;
const light = Math.abs(target - content);
if (light > LIGHT_GATE) return null;
let dirty = 0;
for (const v of inside) dirty += Math.abs(v - content);
return light * LIGHT_WEIGHT + dirty / INSIDE_TAPS.length;
};
let best: { sx: number; sy: number; score: number } | null = null;
for (let d = 0; d < SEARCH_DIRS; d++) {
const a = (d / SEARCH_DIRS) * Math.PI * 2;
for (const dist of SEARCH_DISTANCES) {
const cx = x + Math.cos(a) * dist * r;
const cy = y + Math.sin(a) * dist * r;
for (const [px, py] of [
[cx, cy],
[2 * x - cx, 2 * y - cy],
]) {
if (!inside(px, py)) continue;
const s = score(px, py);
if (s === null) continue;
// A tie keeps the earlier candidate: the ring is walked from the right,
// so the patch nearest the spot wins — the one most likely to share its
// light.
if (!best || s < best.score) best = { sx: px, sy: py, score: s };
}
}
}
return best ? { sx: best.sx, sy: best.sy } : null;
// An alpha of nothing means the fill found no photo anywhere to work from (a
// mask over the whole frame): the pixel is left as it was rather than painted
// black, which is the one thing the user can still recognise as a mistake.
export const HEAL_FILL_SKSL = `
uniform shader img;
uniform shader fill;
uniform shader mask;
half4 main(float2 pos) {
half4 c = img.eval(pos);
half m = mask.eval(pos).a;
if (m <= 0.0) return c;
half4 f = fill.eval(pos);
half3 here = c.a > 0.0001 ? c.rgb / c.a : c.rgb;
half3 made = f.a > 0.0001 ? f.rgb / f.a : here;
return half4(mix(here, made, m) * c.a, c.a);
}
`;
// How the mask is painted onto a surface, as the stops of one radial gradient
// per spot: solid white out to HEAL_FEATHER of the radius, nothing at its edge,
// a straight ramp between. The renderer builds the gradient per spot (Skia has
// no one paint for a list of circles) and draws each at its own radius, so a
// brush turned down to three pixels and one at a quarter of the frame feather by
// the same fraction of themselves.
export const HEAL_MASK_STOPS: number[] = [0, HEAL_FEATHER, 1];
docker/frontend/src/engine/exportEngine.ts
+154 -43
View File
@@ -45,7 +45,14 @@ import {
} from '../../shared/utils/toneShader';
import { CINEMA_SKSL, getCinemaUniforms, cinemaIsActive } from '../../shared/utils/cinemaShader';
import { CURVE_SKSL, CURVE_LUT_SIZE, curveIsActive, curveLut } from '../../shared/utils/toneCurve';
import { healSkSL, healUniforms, readHeal } from '../../shared/utils/heal';
import {
HEAL_CUT_SKSL,
HEAL_FILL_SKSL,
HEAL_FEATHER,
HEAL_MASK_STOPS,
HEAL_MAX_LEVELS,
readHeal,
} from '../../shared/utils/heal';
import { mosaicSkSL, mosaicUniforms, readMosaic } from '../../shared/utils/mosaic';
import { gradientMaskSkSL, maskUniforms, readMasks, masksHaveSpatial } from '../../shared/utils/gradientMask';
import {
@@ -180,6 +187,8 @@ let clarityBlurEffect: any = null;
let clarityBlendEffect: any = null;
let nrEffect: any = null;
let dehazeEffect: any = null;
let healCutEffect: any = null;
let healFillEffect: any = null;
function effects() {
if (!sharpenEffect) sharpenEffect = Skia.RuntimeEffect.Make(SHARPEN_SKSL);
@@ -193,6 +202,8 @@ function effects() {
if (!clarityBlendEffect) clarityBlendEffect = Skia.RuntimeEffect.Make(CLARITY_BLEND_SKSL);
if (!nrEffect) nrEffect = Skia.RuntimeEffect.Make(NR_SKSL);
if (!dehazeEffect) dehazeEffect = Skia.RuntimeEffect.Make(DEHAZE_SKSL);
if (!healCutEffect) healCutEffect = Skia.RuntimeEffect.Make(HEAL_CUT_SKSL);
if (!healFillEffect) healFillEffect = Skia.RuntimeEffect.Make(HEAL_FILL_SKSL);
return {
sharpenEffect,
toneEffect,
@@ -205,23 +216,16 @@ function effects() {
clarityBlendEffect,
nrEffect,
dehazeEffect,
healCutEffect,
healFillEffect,
};
}
// FX's HEAL is the one effect whose shader depends on the recipe rather than on
// constants: it carries exactly the spots the photo has (shared/utils/heal.ts),
// which is what lets the list be uncapped. Every distinct count is therefore a
// shader of its own, kept here so a stroke's worth of spots is paid for once
// and every later render at that count is free.
const healEffects = new Map<number, any>();
function healEffectFor(count: number): any {
let effect = healEffects.get(count);
if (effect === undefined) {
effect = Skia.RuntimeEffect.Make(healSkSL(count)) ?? null;
healEffects.set(count, effect);
}
return effect;
}
// FX's HEAL no longer depends on the recipe: what the brush paints is a mask,
// and the fill behind it is one solve over that mask rather than one block per
// spot, so the two passes are the same for a speck and for a long stroke and are
// built once like the rest. The list is uncapped for the same reason — a hundred
// more spots are a hundred more circles in the mask and not a longer shader.
// FX's MOSAIC is the same kind of shader — one unrolled block per spot the
// recipe holds (shared/utils/mosaic.ts) — so it is cached by count the same way.
@@ -1259,7 +1263,7 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
// that shape's own alpha, masks in the order they were drawn, so the second
// reads what the first left. It sits here, after the frame-wide grade and
// before HEAL, because a local adjustment is part of the look and not a
// repair: the pixels a repair borrows are meant to carry it already.
// repair: the pixels a repair works from are meant to carry it already.
// Same shape as the passes below — the shader reads the frame it is about to
// overwrite — and, when a mask carries CLARITY or DEHAZE, the frame's own
// blurred reference and the atmospheric light come in exactly as they did for
@@ -1279,38 +1283,145 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
}
// 6e. HEAL — the FX tab's dust brush (see heal.ts). It is the last thing
// that touches the photo's own pixels, and deliberately so: the borrowed
// patch is copied out of the pixels the pipeline has already built, so a
// repair matches the grade, the blur and the grain around it by
// construction instead of being a second render of the same look that has
// to agree. The card, the wall and the watermarks are drawn after it, so a
// healed spot still carries the frame's own edge.
// that touches the photo's own pixels, and deliberately so: the fill is made
// of the pixels the pipeline has already built, so a repair matches the
// grade, the blur and the grain around it by construction instead of being a
// second render of the same look that has to agree. The card, the wall and
// the watermarks are drawn after it, so a healed spot still carries the
// frame's own edge.
//
// Three moves, in this order. The brush's circles become ONE mask — a union,
// which is what makes a stroke a single shape instead of a row of circles.
// The mask is cut out of the frame, leaving a hole with the photo's own
// pixels all around its rim. And the hole is filled from that rim, by the
// pyramid below, so nothing is copied from anywhere: what was under the
// brush is the light of what borders it.
const heal = readHeal(adjustments.heal);
if (heal.length) {
// The shader for this many spots, cached by count — every spot the recipe
// holds is drawn, so no repair is ever dropped to make room for a later
// one.
const effect = healEffectFor(heal.length);
const { healCutEffect: cutEffect, healFillEffect: fillEffect } = effects();
flush(surface);
const snap = own(surface.makeImageSnapshot());
const child = snap
? own(
snap.makeShaderOptions(
Skia.TileMode.Clamp, Skia.TileMode.Clamp, Skia.FilterMode.Linear, Skia.MipmapMode.None
)
// The mask, at the frame's own resolution: one radial gradient per spot,
// solid out to HEAL_FEATHER of its radius and clear at its edge, each laid
// over the last — so two spots of one stroke are one shape with no seam
// between them, and the mask's own border is a fade rather than an outline
// the fill would have to match exactly.
const maskSurface = own(createSurface(width, height));
const maskCanvas = maskSurface.getCanvas();
maskCanvas.clear(Skia.Color('transparent'));
for (const s of heal) {
const cx = s.x * width;
const cy = s.y * height;
const rad = s.r * width;
const shader = own(
Skia.Shader.MakeRadialGradient(
[cx, cy],
rad,
new Float32Array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0]),
HEAL_MASK_STOPS,
Skia.TileMode.Clamp
)
: null;
const shader =
effect && child
? own(effect.makeShaderWithChildren(healUniforms(heal, width, height), [child]))
: null;
if (shader) {
// Snapshot, clear, redraw — the pass reads the frame it is about to
// overwrite (see replaceThrough).
canvas.clear(Skia.Color('transparent'));
const healPaint = own(Skia.Paint());
healPaint.setShader(shader);
canvas.drawRect(Skia.XYWHRect(0, 0, width, height), healPaint);
);
const paint = own(Skia.Paint());
paint.setAntiAlias(true);
paint.setShader(shader);
maskCanvas.drawCircle(cx, cy, rad, paint);
}
flush(maskSurface);
const maskImage = own(maskSurface.makeImageSnapshot());
// The frame with the mask taken out of it: the hole the fill works on.
const cutSurface = own(createSurface(width, height));
if (snap && maskImage && cutEffect) {
const cutShader = own(
cutEffect.makeShaderWithChildren([], [own(imageShaderChild(snap)), own(imageShaderChild(maskImage))])
);
if (cutShader) {
const cutCanvas = cutSurface.getCanvas();
cutCanvas.clear(Skia.Color('transparent'));
const cutPaint = own(Skia.Paint());
cutPaint.setShader(cutShader);
cutCanvas.drawRect(Skia.XYWHRect(0, 0, width, height), cutPaint);
}
}
flush(cutSurface);
const cutImage = own(cutSurface.makeImageSnapshot());
// Down: each level is the one below it halved, drawn with a linear filter
// over the premultiplied pixels the cut left behind, so a halving is a
// weighted mean of the pixels that are still there — the hole weighs
// nothing at all, and a level holds the light of the largest
// neighbourhoods around it.
const levels: { surface: any; w: number; h: number }[] = [];
let carried: any = cutImage;
if (carried) {
for (let k = 1; k <= HEAL_MAX_LEVELS; k++) {
const lw = Math.max(1, width >> k);
const lh = Math.max(1, height >> k);
const level = own(createSurface(lw, lh));
const lc = level.getCanvas();
lc.clear(Skia.Color('transparent'));
lc.drawImageRectOptions(
carried,
Skia.XYWHRect(0, 0, carried.width(), carried.height()),
Skia.XYWHRect(0, 0, lw, lh),
Skia.FilterMode.Linear,
Skia.MipmapMode.None
);
flush(level);
levels.push({ surface: level, w: lw, h: lh });
const smaller = own(level.makeImageSnapshot());
if (!smaller || (lw === 1 && lh === 1)) break;
carried = smaller;
}
}
// Up: the coarsest level is already an answer for the whole frame, so it
// is drawn over the level below it — whose hole only that level could
// reach — that result over the one below it, and so on down to the
// half-size level. What comes back is the photo where the photo was, and
// the rim's own light carried inward where it was not.
let up: any = null;
let fillImage: any = null;
for (let k = levels.length - 1; k >= 0; k--) {
const { surface: level, w: lw, h: lh } = levels[k];
if (up) {
const lc = level.getCanvas();
lc.drawImageRectOptions(
up,
Skia.XYWHRect(0, 0, up.width(), up.height()),
Skia.XYWHRect(0, 0, lw, lh),
Skia.FilterMode.Linear,
Skia.MipmapMode.None
);
}
flush(level);
const pulled = own(level.makeImageSnapshot());
if (!pulled) break;
if (k === 0) {
fillImage = pulled;
break;
}
up = pulled;
}
// And the fill back under the brush, in one draw: the frame, the fill and
// the mask as its three children, with the mask the weight the frame and
// the fill are mixed at — so the rim is the only place the two meet, and
// there they already agree.
if (fillImage && snap && maskImage && fillEffect) {
const shader = own(
fillEffect.makeShaderWithChildren([], [
own(imageShaderChild(snap)),
own(imageShaderChild(fillImage)),
own(imageShaderChild(maskImage)),
])
);
if (shader) {
canvas.clear(Skia.Color('transparent'));
const healPaint = own(Skia.Paint());
healPaint.setShader(shader);
canvas.drawRect(Skia.XYWHRect(0, 0, width, height), healPaint);
}
}
}
docker/frontend/src/styles/app.css
+16 -21
View File
@@ -698,10 +698,9 @@ button.hint:disabled { cursor: default; }
/* FX's brush — HEAL and MOSAIC, one layer. It takes the pointer the way the
eyedropper's does, so a click paints and a drag paints a run instead of
panning; the circle under the cursor is the size the wheel has set, the trail
is the stroke being drawn, and every spot on the photo keeps its own circle.
A HEAL spot shows the pair the renderer works with — the hole it fills, and
the patch it borrowed; a MOSAIC spot has nothing beside it, since what it
covers is gone. */
is the stroke being drawn, and every spot on the photo keeps a print of its
own. A repair and a hidden patch look the same here — what a spot means is
the renderer's business, and neither of them borrows a second circle. */
.brush-layer {
position: absolute;
border-radius: var(--radius-sm);
@@ -710,8 +709,7 @@ button.hint:disabled { cursor: default; }
}
.brush-set { position: absolute; inset: 0; pointer-events: none; }
.brush-cursor,
.brush-spot,
.brush-source {
.brush-spot {
position: absolute;
border-radius: 50%;
transform: translate(-50%, -50%);
@@ -739,14 +737,14 @@ button.hint:disabled { cursor: default; }
stroke-opacity: 0.45;
}
.brush-spot { border: 1px solid rgba(255, 255, 255, 0.85); box-shadow: 0 0 0 1px rgba(0, 0, 0, 0.45); }
.brush-source { border: 1px dashed rgba(255, 255, 255, 0.7); }
/* The spots a stroke left are drawn as the one mark it was painted as: a band
of the brush's own width through them, in the same translucent white with the
same dark hairline the circles wear, so the band reads on a white photo too.
The dashed one is the patch HEAL borrowed, which is a band of its own and is
meant to look borrowed. A stroke is one path, so where it crosses itself its
translucency is laid once rather than twice — that doubling-up is exactly
what a row of overlapping circles looks like. */
of the brush's own width through them, in the same white with the same dark
hairline the circles wear, so the band reads on a white photo too. The wash
stays faint — the hairline is what carries it on a white photo, and a heavier
wash turns into an opaque stripe over a warm one.
A stroke is one path, so where it crosses itself its translucency is laid once
rather than twice — that doubling-up is exactly what a row of overlapping
circles looks like. */
.brush-band {
position: absolute;
inset: 0;
@@ -758,21 +756,18 @@ button.hint:disabled { cursor: default; }
}
.brush-band path {
fill: none;
stroke: rgba(255, 255, 255, 0.18);
stroke: rgba(255, 255, 255, 0.08);
stroke-linecap: round;
stroke-linejoin: round;
}
.brush-band.band-src path { stroke: rgba(255, 255, 255, 0.12); stroke-dasharray: 5 5; }
/* A spot the band stands for: its box on the photo is still the spot's, its own
edge is the band's now. */
.brush-spot[data-band],
.brush-source[data-band] { border-color: transparent; box-shadow: none; }
.brush-spot[data-band] { border-color: transparent; box-shadow: none; }
/* A repair the pointer is not on and no press has chosen: the ring would be one
more circle on a photo that already carries as many repairs as the user laid,
so an idle spot is the patch it borrowed and a faint print of the place it
mended — soft, because it is a record of an edit rather than something to
aim at, and it is the ring under the pointer that says the spot can be taken
hold of again. */
so an idle spot is a faint print of the mark it made — soft, because it is a
record of an edit rather than something to aim at, and it is the ring under
the pointer that says the spot can be taken hold of again. */
.brush-spot[data-idle] {
border-color: transparent;
background: rgba(127, 127, 127, 0.14);
docker/frontend/src/ui/ImageStage.tsx
+102 -155
View File
@@ -1,12 +1,11 @@
import { useCallback, useEffect, useMemo, useRef, useState, type ReactNode } from 'react';
import { useI18n } from '../i18n/I18nProvider';
import { MIN_CROP_FRAC, type CropRect, type FrameId, type GradientMask, type HealSpot, type MosaicSpot } from '../../shared/types';
import { bitmapReader, samplePixel, specsLine, type ShotSpecs } from '../engine/imageOps';
import { samplePixel, specsLine, type ShotSpecs } from '../engine/imageOps';
import { RAW_ACCEPT } from '../engine/rawDevelop';
// A phone's HEIC is offered beside the camera RAWs: it is the same offer to a
// file the browser will not open by itself (see engine/heicDevelop.ts).
import { HEIC_ACCEPT } from '../engine/heicDevelop';
import { findHealSource } from '../../shared/utils/heal';
import { MASK_DEFAULT_FEATHER, MASK_MIN } from '../../shared/utils/gradientMask';
import { BRUSH_SPACING, wheelBrushR } from '../../shared/utils/brush';
import { measureMark } from '../engine/exportEngine';
@@ -110,8 +109,8 @@ const strokeRuns = (spots: { x: number; y: number; r: number }[], box: { width:
// there too — the band then reads the same on a portrait photo as on a
// landscape one. Different brushes are different marks and are not joined.
const gap = Math.hypot((a.x - b.x) * box.width, (a.y - b.y) * box.height);
// A spot with no radius is one this run is not about — the patch band's
// patchless spots — and two of those are not a band either.
// A spot with no radius is one no run is about, and two of those are not a
// band either.
if (a.r !== b.r || a.r <= 0 || gap > (a.r + b.r) * box.width) {
flush();
continue;
@@ -270,10 +269,9 @@ export function ImageStage({
straightenRuler = null,
curvePanel = null,
// FX's brush: the tool that takes the pointer on the photo, and the size a
// spot is laid down at. One spot is a circle and, for HEAL, the patch it
// borrowed; MOSAIC's spots carry no patch. Either way they are fractions of
// the render, which is why one set of spots lands on the preview and on the
// export alike.
// spot is laid down at. One spot is a circle — HEAL's the area it fills,
// MOSAIC's the cell it hides. Either way they are fractions of the render,
// which is why one set of spots lands on the preview and on the export alike.
brushTool = null,
brushSpots = [],
brushR = 0.02,
@@ -354,9 +352,8 @@ export function ImageStage({
// FX's brush, HEAL or MOSAIC: the circle under the cursor is the size of the
// spot a gesture lays down (the wheel sets it), and one gesture reports its
// spots together — one gesture is one undo step however many spots it laid.
// HEAL's spots carry the patch each one borrows, searched for at the end of the
// gesture off the render on the stage (shared/utils/heal.ts), which is the
// picture the repair is being judged against.
// HEAL's spots carry the area each one fills (shared/utils/heal.ts), which is
// the picture the repair is being judged against.
brushTool?: 'heal' | 'mosaic' | null;
brushSpots?: (HealSpot | MosaicSpot)[];
brushR?: number;
@@ -602,40 +599,26 @@ export function ImageStage({
// is drawn from the pointer track as it moves.
//
// A spot already on the photo is the other thing the pointer can take: HEAL's
// spots are aimed repairs, so pressing one and dragging moves it — the hole,
// or the patch it borrowed — and a press that never travels only chooses it,
// which is what puts its × on the photo. MOSAIC's spots are not held: they are
// cells of a covered region rather than a repair to place, and a mosaic that
// could be grabbed would also be one the user cannot paint over again.
// spots are aimed repairs, so pressing one and dragging moves it — and a press
// that never travels only chooses it, which is what puts its × on the photo.
// MOSAIC's spots are not held: they are cells of a covered region rather than a
// repair to place, and a mosaic that could be grabbed would also be one the
// user cannot paint over again.
type BrushPoint = { fx: number; fy: number; px: number; py: number };
const strokeRef = useRef<{ pts: BrushPoint[]; w: number } | null>(null);
const [brushTrail, setBrushTrail] = useState<BrushPoint[]>([]);
// The circle the pointer has hold of, then the spot a press chose, which
// wears the × that deletes it. undoPushedRef is the gesture's own undo
// boundary: the step is taken on the first change, so a drag is one step and
// a press that changed nothing costs none.
const grabRef = useRef<{ i: number; part: 'dst' | 'src' } | null>(null);
// The spot a press chose, which wears the × that deletes it. undoPushedRef is
// the gesture's own undo boundary: the step is taken on the first change, so a
// drag is one step and a press that changed nothing costs none.
const grabRef = useRef<number | null>(null);
const [selSpot, setSelSpot] = useState<number | null>(null);
// The spot in hand: the one the pointer is over, or the repair just laid,
// which is the one the user is looking at. A HEAL spot wears its ring only
// while it is in hand or chosen; the rest of them fall back to the patch they
// borrowed and a faint print of what they mended, so a photo with a dozen
// repairs is not a photo with a dozen rings.
// which is the one the user is looking at. A spot wears its ring only while it
// is in hand or chosen; the rest keep a faint print of the mark they made, so
// a photo with a dozen repairs is not a photo with a dozen rings.
const [handSpot, setHandSpot] = useState<number | null>(null);
const undoPushedRef = useRef(false);
// The same decode the eyedropper keeps, under the same rule: one bitmap per
// URL, and HEAL's search reads it a few hundred times rather than once.
const previewReader = useCallback(async () => {
if (!previewUrl) return null;
if (!pickCache.current || pickCache.current.url !== previewUrl) {
const blob = await (await fetch(previewUrl)).blob();
pickCache.current?.bmp.close();
pickCache.current = { url: previewUrl, bmp: await createImageBitmap(blob) };
}
return bitmapReader(pickCache.current.bmp);
}, [previewUrl]);
// A pointer position as both a fraction of the render (what a spot stores)
// and a pixel offset in the photo's box (what the spacing is measured in).
// Off the photo there is no point: a stroke may run out over the frame and
@@ -651,17 +634,15 @@ export function ImageStage({
return { fx, fy, px: clientX - rect.left, py: clientY - rect.top };
}, []);
// Which spot the pointer is on, if any: HEAL's own circles, holes and the
// patches they borrowed alike. The nearest one wins — the patch a repair
// borrowed sits a few pixels from the hole it fills on a small brush, so
// taking the first circle found would hand the patch's own centre to the
// hole — and the hole wins a tie with its own patch. The reach is grown past
// the circle so a brush turned down to a few pixels is still a repair the
// Which spot the pointer is on, if any: the circle the brush painted, and
// nothing else — a repair borrows no patch any more, so the mark the user can
// take hold of is the one they made. The nearest wins, and the reach is grown
// past the circle so a brush turned down to a few pixels is still a repair the
// user can take hold of.
const spotUnder = useCallback(
(p: BrushPoint) => {
if (!box || brushTool !== 'heal' || !onBrushEdit) return null;
let best: { i: number; part: 'dst' | 'src' } | null = null;
let best: number | null = null;
let bestGap = Infinity;
for (let i = brushSpots.length - 1; i >= 0; i--) {
const s = brushSpots[i];
@@ -669,16 +650,10 @@ export function ImageStage({
// a couple of pixels is still a repair the user can take hold of, and a
// press a little way off one is still a press that means to paint.
const reach = Math.max(4, s.r * box.width);
const at = [
{ part: 'dst' as const, x: s.x, y: s.y },
...('sx' in s ? [{ part: 'src' as const, x: s.sx, y: s.sy }] : []),
];
for (const c of at) {
const gap = Math.hypot((p.fx - c.x) * box.width, (p.fy - c.y) * box.height);
if (gap > reach || gap >= bestGap) continue;
best = { i, part: c.part };
bestGap = gap;
}
const gap = Math.hypot((p.fx - s.x) * box.width, (p.fy - s.y) * box.height);
if (gap > reach || gap >= bestGap) continue;
best = i;
bestGap = gap;
}
return best;
},
@@ -699,17 +674,12 @@ export function ImageStage({
setHandSpot(null);
};
// The band each stroke left, read off the spots the recipe holds — the holes
// one way and, for HEAL, the patches they borrowed the other. Nothing about a
// stroke is stored, so this is where "that long drag was one mark" comes from,
// and it is the same band after a reload.
// The band each stroke left, read off the spots the recipe holds. Nothing
// about a stroke is stored, so this is where "that long drag was one mark"
// comes from, and it is the same band after a reload.
const bands = useMemo(() => {
if (!box) return null;
// Only HEAL's spots borrowed anything, so a spot with no patch has no
// circle in the patch band: it is mapped to its own centre with no radius,
// which joins nothing and draws nothing.
const src = brushSpots.map((s) => ('sx' in s ? { x: s.sx, y: s.sy, r: s.r } : { x: s.x, y: s.y, r: 0 }));
return { dst: strokeRuns(brushSpots, box), src: strokeRuns(src, box) };
return { dst: strokeRuns(brushSpots, box) };
}, [brushSpots, box]);
// Where the × of the spot the pointer chose goes: on its top-right diagonal,
@@ -755,10 +725,10 @@ export function ImageStage({
// press moves it rather than painting over it; anywhere else paints the
// stroke it always did.
const hit = spotUnder(p);
if (hit) {
if (hit !== null) {
grabRef.current = hit;
undoPushedRef.current = false;
setSelSpot(hit.i);
setSelSpot(hit);
return;
}
setSelSpot(null);
@@ -777,75 +747,80 @@ export function ImageStage({
const p = brushPoint(e.clientX, e.clientY);
setBrushPos(p ? { fx: p.fx, fy: p.fy } : null);
const grab = grabRef.current;
if (grab && onBrushEdit) {
if (grab !== null && onBrushEdit) {
// Off the photo the spot stays where it was: a repair is a place on the
// picture, and a pointer past the frame has left it.
if (!p) return;
const was = brushSpots[grab.i];
const next = brushSpots.slice();
next[grab.i] =
grab.part === 'src' && 'sx' in was ? { ...was, sx: p.fx, sy: p.fy } : { ...was, x: p.fx, y: p.fy };
next[grab] = { ...next[grab], x: p.fx, y: p.fy };
onBrushEdit(next, !undoPushedRef.current);
undoPushedRef.current = true;
return;
}
const stroke = strokeRef.current;
// Which repair is the one in hand, for the circle it wears: the spot under
// Which repair is the one in hand, for the ring it wears: the spot under
// the pointer. Only while nothing is being painted or dragged, so a stroke
// does not flicker the rings it runs over.
if (!stroke) setHandSpot(p ? (spotUnder(p)?.i ?? null) : null);
if (!stroke) setHandSpot(p ? (spotUnder(p) ?? null) : null);
if (!stroke || !p) return;
const last = stroke.pts[stroke.pts.length - 1];
// The spacing is a fraction of the brush, measured in the photo's own
// pixels: any closer and the run is the same spot written twice.
if (Math.hypot(p.px - last.px, p.py - last.py) < Math.max(1, brushR * stroke.w * BRUSH_SPACING)) return;
stroke.pts.push(p);
const step = Math.max(1, brushR * stroke.w * BRUSH_SPACING);
const dist = Math.hypot(p.px - last.px, p.py - last.py);
if (dist < step) return;
// The pointer's own samples are wherever the browser happened to report a
// move, which on a fast drag is several radii apart — spots that far apart
// do not touch, and a row of circles that do not touch is a row of circles
// and not a stroke (the renderer's mask is their union). So the segment the
// pointer travelled is walked at the brush's own spacing and a spot is laid
// at every step: a run is joined whatever speed it was drawn at.
const n = Math.floor(dist / step);
for (let i = 1; i <= n; i++) {
const t = (i * step) / dist;
stroke.pts.push({
fx: last.fx + (p.fx - last.fx) * t,
fy: last.fy + (p.fy - last.fy) * t,
px: last.px + (p.px - last.px) * t,
py: last.py + (p.py - last.py) * t,
});
}
setBrushTrail(stroke.pts.slice());
};
const brushUp = useCallback(async (e?: React.PointerEvent<HTMLDivElement>) => {
if (panRef.current) {
panRef.current = null;
if (e?.currentTarget.hasPointerCapture?.(e.pointerId)) e.currentTarget.releasePointerCapture(e.pointerId);
return;
}
if (grabRef.current) {
// A press on a spot that never travelled is a choice, not an edit: nothing
// was written, so the caller is not asked for a step.
grabRef.current = null;
return;
}
const stroke = strokeRef.current;
strokeRef.current = null;
setBrushTrail([]);
if (!stroke || !onBrushSpots) return;
// MOSAIC has nothing to read off the photo: a cell takes the colour the
// shader finds at the cell's own middle, and every spot is the same circle
// the brush drew. HEAL does — its patch is searched for in the pixels under
// the stroke.
if (brushTool === 'mosaic') {
onBrushSpots(stroke.pts.map((p) => ({ x: p.fx, y: p.fy, r: brushR })));
return;
}
const reader = await previewReader();
if (!reader) return;
// A spot whose patch the search refuses is not laid down at all: borrowing
// the spot's own pixels would be a repair that changes nothing, and pasting
// a patch from another light would show as a mark of its own, which is worse
// than the speck. The speck stays; the user tries another size or another
// spot. A stroke that is refused end to end reports nothing, and the caller
// records no step for it.
const spots = stroke.pts.flatMap((p) => {
const src = findHealSource(reader, p.fx, p.fy, brushR);
return src ? [{ x: p.fx, y: p.fy, r: brushR, sx: src.sx, sy: src.sy }] : [];
});
onBrushSpots(spots);
// The repair just laid is the one in hand: the pointer has not moved since,
// so no move has reported a hover, and without this the ring of the spot the
// user is looking at is the one thing that would not be there. It gives the
// ring up again the moment the pointer walks anywhere.
if (spots.length) setHandSpot(brushSpots.length + spots.length - 1);
}, [brushR, brushSpots.length, brushTool, previewReader, onBrushSpots]);
const brushUp = useCallback(
(e?: React.PointerEvent<HTMLDivElement>) => {
if (panRef.current) {
panRef.current = null;
if (e?.currentTarget.hasPointerCapture?.(e.pointerId)) e.currentTarget.releasePointerCapture(e.pointerId);
return;
}
if (grabRef.current !== null) {
// A press on a spot that never travelled is a choice, not an edit:
// nothing was written, so the caller is not asked for a step.
grabRef.current = null;
return;
}
const stroke = strokeRef.current;
strokeRef.current = null;
setBrushTrail([]);
if (!stroke || !onBrushSpots) return;
// Every spot the stroke laid down, reported together: the spots land with
// the gesture rather than under it, which is one undo step however long
// the stroke was. Nothing is read off the photo while they land — where a
// repair is taken from is the renderer's business, not the brush's
// (shared/utils/heal.ts), so a stroke over dust and a stroke over a face
// are the same gesture and neither can be refused.
const spots = stroke.pts.map((p) => ({ x: p.fx, y: p.fy, r: brushR }));
onBrushSpots(spots);
// The repair just laid is the one in hand: the pointer has not moved
// since, so no move has reported a hover, and without this the ring of
// the spot the user is looking at is the one thing that would not be
// there. It gives the ring up again the moment the pointer walks anywhere.
setHandSpot(brushSpots.length + spots.length - 1);
},
[brushR, brushSpots.length, onBrushSpots]
);
// The wheel over the brush is the stage's own, as it is everywhere else in
// the app: a repair is aimed at a zoom, and the wheel is the gesture a user
@@ -1605,13 +1580,10 @@ export function ImageStage({
{/* FX's brush — HEAL or MOSAIC. The layer swallows the pointer the way
the eyedropper's does, so a click paints instead of panning the
photo and a drag paints a run; the circle rides the cursor at the
size a spot will be, and the wheel changes that size. A HEAL spot
shows the pair the renderer works with — the circle it fills and
the patch it borrowed — and wears that circle only while it is the
spot in hand: the repairs already on the photo read as the marks
they left rather than as a field of rings, and the ring comes back
under the pointer. A MOSAIC spot has nothing to show beside
itself, since what is under it is gone. */}
size a spot will be, and the wheel changes that size. A spot
wears its circle only while it is the spot in hand: the repairs
already on the photo read as the marks they left rather than as a
field of rings, and the ring comes back under the pointer. */}
{brushTool && !cropEditing && box ? (
<div
className="brush-layer"
@@ -1662,13 +1634,10 @@ export function ImageStage({
</svg>
) : null}
{/* The band each stroke left, under the circles: the spots the
run is made of, joined into the one mark the drag painted —
the holes here, and HEAL's borrowed patches in the dashed band
beside them. What is drawn is the recipe's own spots, so the
band is what survives a reload, an undo and a copy of the
recipe to another photo, where a stroke of its own would not.
Laid in the same pass, the two bands cannot double up on
themselves and print a darker patch at every joint. */}
run is made of, joined into the one mark the drag painted.
What is drawn is the recipe's own spots, so the band is what
survives a reload, an undo and a copy of the recipe to another
photo, where a stroke of its own would not. */}
{bands?.dst.paths.length ? (
<svg
className="brush-band"
@@ -1682,19 +1651,6 @@ export function ImageStage({
))}
</svg>
) : null}
{bands?.src.paths.length ? (
<svg
className="brush-band band-src"
data-key="brush-band-src"
data-bands={bands.src.paths.length}
viewBox={`0 0 ${box.width} ${box.height}`}
aria-hidden="true"
>
{bands.src.paths.map((b, i) => (
<path key={i} d={b.d} style={{ strokeWidth: b.w }} />
))}
</svg>
) : null}
{brushSpots.map((s, i) => {
// r is a fraction of the photo's WIDTH, so the circle is round
// whatever the picture's shape is — the same measurement the
@@ -1712,21 +1668,12 @@ export function ImageStage({
// circles was showing.
//
// A HEAL spot that is neither in hand nor chosen is idle: it
// keeps the patch it borrowed and a faint print of the place it
// mended, and gives up the ring, which comes back under the
// pointer — that is where the spot is taken hold of again, so
// nothing about it is lost.
// keeps a faint print of the repair it made, and gives up the
// ring, which comes back under the pointer — that is where the
// spot is taken hold of again, so nothing about it is lost.
const idle = brushTool === 'heal' && i !== selSpot && i !== handSpot;
return (
<span className="brush-set" data-key="brush-set" data-i={i} key={i}>
{'sx' in s ? (
<span
className="brush-source"
data-key="brush-source"
data-band={bands?.src.joined[i] ? '1' : undefined}
style={at(s.sx, s.sy)}
/>
) : null}
<span
className="brush-spot"
data-key="brush-spot"