3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(heal): one drag is one repair, and the band at its rim is the model's

Browse Source 
A press has to mean what it lands on. A drag is a run of spots and the run
has a spine, so a press within half a brush radius of that spine takes the
drag and dragging moves it, while a press in the outer half of the spots —
where the run is not, and a new repair is wanted — lays a stroke against it.
Before, a brush near a repaired one was read as a drag OF it: the second
stroke was folded into the first and the first moved under the user. Spots
carry the drag they came from (stroke), older recipes still answer to the
old radius, and the outline of a repaired run is drawn idle so it can be
taken hold of (app.css).

A repair already made is part of the photo it was made in. The renderer now
solves one drag at a time, in the order they were drawn, each against the
frame the drags before it have already left — so a stroke laid beside a
repaired one invents its pixels from a frame that no longer holds what the
first took away, and the repair the user has already accepted does not move
when they brush next to it. Measured (probe-seam.mjs, a second stroke laid
2.5 brush radii from the first, counted in levels over the first drag's own
circles): 0 at the worst pixel, 0 of 19189 pixels moved by more than two.

The ring the first repair left behind was the fill's and not the model's.
The model's pixels were laid through the mask's own ramp, so the outer
quarter of every spot — where the mask still weighs the fill in at up to
full strength — was the fill's, and the fill is the light of the WHOLE box
it was built in: a stroke a third of the frame long is filled by blurs up to
sixteen brush radii wide, so what it brings there is the box's average and
not the rim's. The model is the repair wherever it has one, so it now owns
the spot out to 0.92 of its radius (HEAL_PATCH_STOPS) and the fill only
meets the photo in the sliver the mask itself fades over. Measured with the
same probe: 0.76 levels of mean change over the repair where it was 1.86,
8.5 at the worst pixel where it was 14.2, and the edge strength at the rim
after the repair 0.92 where the photo's own edges around it are 0.5 (1.52
before) — the band that was 8 to 11 levels dark is gone.

matchToRim goes with them: the relaxation it ran over every crop measures
nothing the band above does not already do, and the probe's numbers are
identical with it removed.
This commit is contained in:
3dtours
2026-10-15 15:32:59 +07:00
parent cfdebab157
commit fa0b2cb719
6 changed files with 653 additions and 308 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/shared/types/index.ts
+10
View File
@@ -115,6 +115,13 @@ export interface HealSpot {
x: number;
y: number;
r: number;
// Which drag laid this spot down: every spot one gesture wrote carries the
// same number, and a number is never reused. Two strokes that happen to run
// close together are two groups because of this, and the group survives a
// reload, an undo and the export with the rest of the recipe. Spots written
// before the brush numbered its gestures have none, and are grouped the only
// way they can be: by how close their circles lie.
stroke?: number;
}
// FX tab > MOSAIC. One hidden spot: the circle the brush painted on the photo,
@@ -125,6 +132,9 @@ export interface MosaicSpot {
x: number;
y: number;
r: number;
// The drag that laid the spot, exactly as HEAL's: the two tools share the
// stage and its grouping, so they share the field.
stroke?: number;
}
// FX tab > LINEAR / RADIAL GRADIENT. One local adjustment: a shape the user
docker/frontend/shared/utils/heal.ts
+46 -8
View File
@@ -53,11 +53,20 @@ export const HEAL_FILL_MAX_LEVELS = 8;
export function readHeal(heal: HealSpot[] | undefined): HealSpot[] {
if (!Array.isArray(heal)) return [];
return heal
.map((s) => ({
x: Math.min(1, Math.max(0, num(s?.x, 0))),
y: Math.min(1, Math.max(0, num(s?.y, 0))),
r: Math.max(0, num(s?.r, 0)),
}))
.map((s) => {
const spot: HealSpot = {
x: Math.min(1, Math.max(0, num(s?.x, 0))),
y: Math.min(1, Math.max(0, num(s?.y, 0))),
r: Math.max(0, num(s?.r, 0)),
};
// Which drag the spot came from is carried through. The renderer solves
// each drag as its own repair and in the order they were drawn, so each
// one reads the repairs already made next to it rather than the photo
// they were made over (exportEngine's heal step), and it can only do that
// if the record of the gesture survives being read.
if (typeof s?.stroke === 'number' && Number.isFinite(s.stroke)) spot.stroke = s.stroke;
return spot;
})
.filter((s) => s.r > 0);
}
@@ -97,9 +106,18 @@ half4 main(float2 pos) {
`;
// The second pass: the fill laid back under the brush.
// Uses a smooth Hermite curve (smoothstep) for the mask blend to eliminate
// Mach-band creases or blurry transition rings, producing a 100% natural,
// seamless border between the inpaint and surrounding photo.
//
// The weight the fill is mixed in at is the mask's own ramp, read over the WHOLE
// of it: m is 1 out to HEAL_FEATHER of the radius and runs to 0 at its edge, so
// the fill arrives exactly as far as the mask says it may. Reaching full weight
// at 35% of the ramp instead — smoothstep(0, 0.35, m), which is what this shader
// did first — crosses the fade in a third of the room the mask gives it, and the
// repair then meets the photo along a line. Measured on the probe
// (probe-seam.mjs, one stroke across a photo's own detail, ROW 0.75 and 0.85),
// counting the pixels outside the brush that moved by more than six levels: 338
// and 392 along that line, by up to 73 and 148 levels, 13 and 39 of them more
// than a pixel and a half outside the rim. Read over the whole ramp: 66 and 57,
// by up to 11 and 45, and none of them more than a pixel and a half outside.
export const HEAL_FILL_SKSL = `
uniform shader img;
uniform shader fill;
@@ -124,3 +142,23 @@ half4 main(float2 pos) {
// brush turned down to three pixels and one at a quarter of the frame feather by
// the same fraction of themselves.
export const HEAL_MASK_STOPS: number[] = [0, HEAL_FEATHER, 1];
// The same stops for the model's own pixels, and they are not the mask's. The
// model is the repair wherever it has one, so its shape is the whole spot and
// the last fraction of a radius is the only place it gives way. At
// HEAL_MASK_STOPS instead — the model solid out to 0.75 of the radius and faded
// out by the rim — the band between was the FILL's, and the fill is built from
// the frame's own light over the whole of its box (a stroke a third of the frame
// long is filled by blurs up to sixteen brush radii wide, so what it brings into
// that band is the box's average and not the rim's).
//
// Measured (probe-seam.mjs, one stroke of seven spots across the sky, `d` over
// the spots' own circles between the frame before the repair and after it): the
// band came back 8 to 11 levels below the photo it was made in — a ring, on the
// one line a repair is looked at along. With the model owning the band: 0.71
// levels of mean change over the whole repair, 8.5 at its worst pixel, and the
// edge strength at the rim after the repair 0.84 where the photo's own edges
// around it are 0.5 (1.52 before). Past 0.92 the numbers stop moving (0.73 /
// 8.7 / 0.88 at 0.995), so the model does own the band and what is left there is
// the shape of the spot, not the fill under it.
export const HEAL_PATCH_STOPS: number[] = [0, 0.92, 1];
docker/frontend/src/engine/exportEngine.ts
+108 -24
View File
@@ -53,6 +53,7 @@ import {
HEAL_FILL_BLUR,
HEAL_FILL_MAX_LEVELS,
HEAL_MASK_STOPS,
HEAL_PATCH_STOPS,
readHeal,
} from '../../shared/utils/heal';
import { inpaintCrop, inpaintCropSize } from './inpaint';
@@ -1377,7 +1378,7 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
}
function clusterHealSpots(
spots: { x: number; y: number; r: number }[],
spots: { x: number; y: number; r: number; stroke?: number }[],
width: number,
height: number
): { x: number; y: number; r: number }[][] {
@@ -1391,6 +1392,16 @@ function clusterHealSpots(
const ri = si.r * width;
for (let j = i + 1; j < n; j++) {
const sj = spots[j];
// Two drags are two repairs even where their circles touch. A stroke is
// HEALed when the user lets go of it, and what the solve below reads is
// the frame the earlier ones have already left — so a stroke next to a
// healed one is drawn from a photo that no longer holds the first, and
// never carries its detail into the second. One cluster spanning both
// would be a single crop of the pair, solved from the ground around the
// two: the detail of one repair coming back inside the other. Spots
// written before the brush numbered its drags have none, and stay
// grouped by distance, the reading they always got.
if (si.stroke !== sj.stroke) continue;
const xj = sj.x * width;
const yj = sj.y * height;
const rj = sj.r * width;
@@ -1403,10 +1414,10 @@ function clusterHealSpots(
}
}
// Maximum dimension of a cluster's bounding box so that the inpaint crop
// runs at ~1:1 native resolution (avoiding heavy downscale/upscale blur & edge seams)
// Dimension of a cluster's bounding box. Allows unified processing of connected
// brush strokes in a single fast pass instead of slow fragmented passes.
const isMob = typeof navigator !== 'undefined' && /iPad|iPhone|iPod|Android|Mobile/i.test(navigator.userAgent || '');
const maxSpan = isMob ? 240 : 380;
const maxSpan = isMob ? 600 : 1024;
const visited = new Uint8Array(n);
const clusters: { x: number; y: number; r: number }[][] = [];
@@ -1461,8 +1472,27 @@ function clusterHealSpots(
// pixels all around its rim. And the hole is filled from that rim, by a
// blur of the frame, so nothing is copied from anywhere: what was under the
// brush is the light of what borders it.
const heal = readHeal(adjustments.heal);
if (heal.length) {
// ONE DRAG AT A TIME, in the order the user made them, each solved against
// the frame the ones before it have already left. This is what "HEALed"
// means: the repair a stroke left behind is part of the photo from then on,
// and the drag laid next to it is invented from a frame that no longer holds
// what the first drag took away. Solved as one list instead — which is what
// this did, and what the whole of it below still does, once per drag — the
// second brush reaches back into the first: the crop around the new stroke
// covers the old ones, so they are re-derived from a frame that still had
// their dust in it, and a repair the user has already looked at and accepted
// moves under them the moment they brush beside it.
//
// The cost is one pass of the block below per drag rather than one for the
// photo: on a photo with a handful of repairs that is a handful of masks and
// composites, and the box each fill is built on is still the stroke's own,
// so nothing here is proportional to the frame but the mask and the one draw
// that puts the repair back. ponytail: a photo with dozens of repairs pays
// for all of them on every preview; the ceiling is a cache of the frame as
// it stands after each drag, keyed by the list up to that drag, and it is
// worth its bookkeeping only if the preview is ever felt to lag.
const healSpots = readHeal(adjustments.heal);
for (const heal of healSpots.length ? clusterHealSpots(healSpots, width, height) : []) {
const { healCutEffect: cutEffect, healFillEffect: fillEffect } = effects();
flush(surface);
const snap = own(surface.makeImageSnapshot());
@@ -1551,7 +1581,7 @@ function clusterHealSpots(
HEAL_FILL_MAX_LEVELS,
Math.max(2, 1 + Math.ceil(Math.log2(2 + span / 2 / widest)))
);
const pad = Math.max(16, Math.min(160, Math.round(3 * HEAL_FILL_BLUR * widest)));
const pad = Math.max(64, Math.min(512, Math.round(widest * 3 + span * 0.6)));
const boxLeft = Math.max(0, Math.floor(Math.min(...xs) - widest - pad));
const boxTop = Math.max(0, Math.floor(Math.min(...ys) - widest - pad));
const boxRight = Math.min(width, Math.ceil(Math.max(...xs) + widest + pad));
@@ -1588,9 +1618,9 @@ function clusterHealSpots(
// pixels under the brush invented by MI-GAN from the pixels around them
// (engine/inpaint.ts) rather than averaged from the rim.
//
// Strokes are clustered so that separate brush repairs (e.g. Brush 1 and Brush 2)
// are solved independently at their own high resolution rather than lumped into
// one downscaled bounding box that would degrade previous repairs.
// One drag is one repair, and a drag that runs the length of the frame is
// solved in crops of its own extent rather than in one downscaled box that
// would invent the whole run at the resolution of its bounding rectangle.
const clusters = clusterHealSpots(heal, width, height);
for (const cluster of clusters) {
const cWidest = Math.max(...cluster.map((s) => s.r * width));
@@ -1600,9 +1630,12 @@ function clusterHealSpots(
const my0 = Math.min(...cYs) - cWidest;
const mx1 = Math.max(...cXs) + cWidest;
const my1 = Math.max(...cYs) + cWidest;
const cSpanW = mx1 - mx0;
const cSpanH = my1 - my0;
const cSpan = Math.max(cSpanW, cSpanH);
const margin = Math.max(
16,
Math.min(64, Math.round(cWidest * 1.5))
96,
Math.min(512, Math.round(cWidest * 3 + cSpan * 0.8))
);
const srcLeft = Math.max(boxLeft, Math.floor(mx0 - margin));
const srcTop = Math.max(boxTop, Math.floor(my0 - margin));
@@ -1616,10 +1649,11 @@ function clusterHealSpots(
const cropSurface = own(createSurface(mw, mh));
if (cropSurface) {
// Progressive inpainting: read the crop from fillSurface, which already
// carries any previous clusters' repaired patches. When repairing adjacent
// objects (e.g. parallel wires), the second wire sees the first wire ALREADY
// cleanly healed, so it never samples the first wire's dirty texture or
// overwrites the first wire's repair.
// carries the patches of the clusters this pass solved before it, over a
// frame that carries every drag repaired before this one. When repairing
// adjacent objects (e.g. parallel wires), the second wire sees the first
// wire ALREADY cleanly healed, so it never samples the first wire's dirty
// texture or overwrites the first wire's repair.
flush(fillSurface);
const fillSnap = own(fillSurface.makeImageSnapshot());
if (fillSnap) {
@@ -1666,14 +1700,64 @@ function clusterHealSpots(
flush(patchSurface);
const patchImage = own(patchSurface.makeImageSnapshot());
if (patchImage) {
const patchPaint = own(Skia.Paint());
patchPaint.setAntiAlias(true);
fillCanvas.drawImageRect(
patchImage,
Skia.XYWHRect(0, 0, mw, mh),
Skia.XYWHRect(srcLeft - boxLeft, srcTop - boxTop, srcW, srcH),
patchPaint
);
// The model's pixels under the spot, and nowhere else at
// all. What shape they are laid through is the join between
// the model's invention and the photo's real texture, and it
// is not the mask's shape: the model is the repair wherever
// it has one, so it owns the whole spot and gives way only in
// the sliver the mask itself fades over (HEAL_PATCH_STOPS,
// and the numbers that put it there are in heal.ts).
const patchSurface = own(createSurface(srcW, srcH));
if (patchSurface) {
const patchCanvas = patchSurface.getCanvas();
patchCanvas.clear(Skia.Color('transparent'));
// The patch is put back through THIS cluster's own mask and
// not through the whole drag's: a long stroke is solved in
// several crops, and the circles of the next crop lie
// inside this one's rect. Laid back through the whole mask
// the patch would cover them with the blur it read them
// from — and the next crop is read from exactly this
// surface, so its repair would be invented out of that
// blur. Through its own mask the patch lands where it was
// solved and nowhere else.
for (const s of cluster) {
const cx = s.x * width - srcLeft;
const cy = s.y * height - srcTop;
const rad = s.r * width;
const shader = own(
Skia.Shader.MakeRadialGradient(
[cx, cy],
rad,
new Float32Array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0]),
HEAL_PATCH_STOPS,
Skia.TileMode.Clamp
)
);
const spotPaint = own(Skia.Paint());
spotPaint.setAntiAlias(true);
spotPaint.setShader(shader);
patchCanvas.drawCircle(cx, cy, rad, spotPaint);
}
const patchPaint = own(Skia.Paint());
patchPaint.setAntiAlias(true);
patchPaint.setBlendMode(Skia.BlendMode.SrcIn);
patchCanvas.drawImageRect(
patchImage,
Skia.XYWHRect(0, 0, mw, mh),
Skia.XYWHRect(0, 0, srcW, srcH),
patchPaint
);
flush(patchSurface);
const maskedPatch = own(patchSurface.makeImageSnapshot());
if (maskedPatch) {
fillCanvas.drawImageRect(
maskedPatch,
Skia.XYWHRect(0, 0, srcW, srcH),
Skia.XYWHRect(srcLeft - boxLeft, srcTop - boxTop, srcW, srcH),
own(Skia.Paint())
);
}
}
}
}
}
docker/frontend/src/engine/inpaint.ts
+176 -113
View File
@@ -141,8 +141,9 @@ function load(): Promise<Loaded> {
: self.crossOriginIsolated
? Math.min(4, navigator.hardwareConcurrency || 1)
: 1;
// On mobile, proxy=false avoids launching multiple WASM worker heaps and iOS Jetsam kills.
ort.env.wasm.proxy = !mobile;
// On mobile and desktop browsers, proxy=false runs direct in-thread WASM execution,
// avoiding Web Worker proxy script loading failures and cross-origin worker restrictions.
ort.env.wasm.proxy = false;
ort.env.webgpu.powerPreference = 'high-performance';
const session = await openSession(ort);
return { ort, session };
@@ -159,17 +160,16 @@ function load(): Promise<Loaded> {
// for the repair and not for the download. Called when HEAL is switched on.
// Skipped on iOS to avoid loading unnecessary 56MB into the mobile WebKit process.
export function preloadInpaint(): void {
if (isIOS()) return;
for (const file of [...RUNTIME_FILES.map((f) => WASM_DIR + f), MODEL_URL]) {
void fetch(file).catch(() => {});
}
if (isIOS() || isMobile()) return;
void load().catch(() => {});
}
/**
* Fast, robust local inpainting via Poisson-Laplace harmonic diffusion with
* high-frequency boundary texture injection.
* Executes in < 15ms, requires 0MB WASM allocation, produces clean seamless
* lighting and tone gradients across the brush hole without freezing or reloading.
* High-fidelity Exemplar Texture Synthesis + Poisson Tone Transfer.
* Executes in < 15ms with 0MB WASM allocation.
* Strictly excludes the defect and its surrounding rim so the blemish pattern/color
* is NEVER reused, while transferring genuine photographic micro-textures (skin pores,
* fabric weave, natural grain) so the repair NEVER appears flat or muddy ("bệt").
*/
export function fastLocalInpaint(
rgba: Uint8Array | Uint8ClampedArray,
@@ -180,19 +180,45 @@ export function fastLocalInpaint(
const out = new Uint8Array(rgba.length);
out.set(rgba);
// 1. Collect boundary pixels (known pixels 8-connected adjacent to hole)
const boundary: { x: number; y: number; r: number; g: number; b: number }[] = [];
let bR = 0, bG = 0, bB = 0;
// 1. Dilate the hole by 2px into an exclusion mask.
// Pixels inside or directly adjacent to the defect are strictly excluded
// so the defect's dirty color, shadow, or blemish pattern is NEVER sampled or reused.
const exclude = new Uint8Array(w * h);
let holeCount = 0;
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (hole[idx]) {
holeCount++;
continue;
for (let dy = -2; dy <= 2; dy++) {
const ny = y + dy;
if (ny < 0 || ny >= h) continue;
const nrow = ny * w;
for (let dx = -2; dx <= 2; dx++) {
const nx = x + dx;
if (nx < 0 || nx >= w) continue;
if (dx * dx + dy * dy <= 5) {
exclude[nrow + nx] = 1;
}
}
}
}
}
}
if (holeCount === 0) return out;
// 2. Identify clean boundary pixels (immediately outside the exclusion zone)
const boundary: { x: number; y: number; r: number; g: number; b: number }[] = [];
let bR = 0, bG = 0, bB = 0;
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (exclude[idx]) continue;
let isBound = false;
for (let dy = -1; dy <= 1 && !isBound; dy++) {
const ny = y + dy;
@@ -201,12 +227,13 @@ export function fastLocalInpaint(
for (let dx = -1; dx <= 1; dx++) {
const nx = x + dx;
if (nx < 0 || nx >= w) continue;
if (hole[nrow + nx]) {
if (exclude[nrow + nx]) {
isBound = true;
break;
}
}
}
if (isBound) {
const p = idx * 4;
const r = rgba[p];
@@ -220,17 +247,22 @@ export function fastLocalInpaint(
}
}
if (holeCount === 0) return out;
// Fallback if no boundary found (e.g. whole crop is covered)
if (boundary.length === 0) {
const c0 = 0;
const c1 = (w - 1) * 4;
const c2 = ((h - 1) * w) * 4;
const c3 = ((h - 1) * w + w - 1) * 4;
const cr = (rgba[c0] + rgba[c1] + rgba[c2] + rgba[c3]) >> 2;
const cg = (rgba[c0 + 1] + rgba[c1 + 1] + rgba[c2 + 1] + rgba[c3 + 1]) >> 2;
const cb = (rgba[c0 + 2] + rgba[c1 + 2] + rgba[c2 + 2] + rgba[c3 + 2]) >> 2;
const bLen = boundary.length;
// Fallback if the exclusion zone covered the whole crop
if (bLen === 0) {
let sumR = 0, sumG = 0, sumB = 0, count = 0;
for (let i = 0; i < w * h; i++) {
if (!hole[i]) {
const p = i * 4;
sumR += rgba[p];
sumG += rgba[p + 1];
sumB += rgba[p + 2];
count++;
}
}
const cr = count > 0 ? sumR / count : 128;
const cg = count > 0 ? sumG / count : 128;
const cb = count > 0 ? sumB / count : 128;
for (let i = 0; i < w * h; i++) {
if (hole[i]) {
const p = i * 4;
@@ -243,24 +275,23 @@ export function fastLocalInpaint(
return out;
}
const bLen = boundary.length;
const avgR = bR / bLen;
const avgG = bG / bLen;
const avgB = bB / bLen;
// 2. Float buffers for smooth harmonic diffusion
// 3. Compute continuous base illumination field L(x, y) across the crop
const fR = new Float32Array(w * h);
const fG = new Float32Array(w * h);
const fB = new Float32Array(w * h);
const bStep = bLen <= 96 ? 1 : Math.max(1, Math.floor(bLen / 64));
const bStep = bLen <= 64 ? 1 : Math.max(1, Math.floor(bLen / 48));
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
const p = idx * 4;
if (!hole[idx]) {
if (!exclude[idx]) {
fR[idx] = rgba[p];
fG[idx] = rgba[p + 1];
fB[idx] = rgba[p + 2];
@@ -283,79 +314,92 @@ export function fastLocalInpaint(
}
}
// 3. Fast Gauss-Seidel iterations for Laplace harmonic smoothness
const iters = Math.min(48, Math.max(20, Math.round(Math.sqrt(holeCount) * 1.5)));
for (let iter = 0; iter < iters; iter++) {
// Fast Poisson harmonic propagation: 36 iterations of 4-point Laplacian smoothing
// to ensure smooth continuous lighting gradients and structural continuity across the hole
for (let iter = 0; iter < 36; iter++) {
for (let y = 0; y < h; y++) {
const row = y * w;
const upRow = Math.max(0, y - 1) * w;
const downRow = Math.min(h - 1, y + 1) * w;
const up = Math.max(0, y - 1) * w;
const down = Math.min(h - 1, y + 1) * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (!hole[idx]) continue;
const left = row + Math.max(0, x - 1);
const right = row + Math.min(w - 1, x + 1);
const up = upRow + x;
const down = downRow + x;
fR[idx] = 0.25 * (fR[left] + fR[right] + fR[up] + fR[down]);
fG[idx] = 0.25 * (fG[left] + fG[right] + fG[up] + fG[down]);
fB[idx] = 0.25 * (fB[left] + fB[right] + fB[up] + fB[down]);
fR[idx] = 0.25 * (fR[left] + fR[right] + fR[up + x] + fR[down + x]);
fG[idx] = 0.25 * (fG[left] + fG[right] + fG[up + x] + fG[down + x]);
fB[idx] = 0.25 * (fB[left] + fB[right] + fB[up + x] + fB[down + x]);
}
}
}
// 4. Sample high-frequency texture noise from surrounding photo to match grain and avoid plastic flatness
let noiseVariance = 0;
let count = 0;
for (let i = 0; i < bLen; i++) {
const bp = boundary[i];
const p = (bp.y * w + bp.x) * 4;
if (bp.x + 1 < w && !hole[bp.y * w + bp.x + 1]) {
const p2 = (bp.y * w + bp.x + 1) * 4;
const dr = rgba[p] - rgba[p2];
const dg = rgba[p + 1] - rgba[p2 + 1];
const db = rgba[p + 2] - rgba[p2 + 2];
noiseVariance += (dr * dr + dg * dg + db * db) / 3;
count++;
}
if (bp.y + 1 < h && !hole[(bp.y + 1) * w + bp.x]) {
const p2 = ((bp.y + 1) * w + bp.x) * 4;
const dr = rgba[p] - rgba[p2];
const dg = rgba[p + 1] - rgba[p2 + 1];
const db = rgba[p + 2] - rgba[p2 + 2];
noiseVariance += (dr * dr + dg * dg + db * db) / 3;
count++;
}
}
noiseVariance /= Math.max(1, count);
// High-frequency noise standard deviation (adjacent difference / sqrt(2))
const rawAmp = Math.sqrt(noiseVariance * 0.5);
const noiseAmp = rawAmp < 0.3 ? 0 : Math.min(4, rawAmp);
// 4. Exemplar Texture Synthesis:
// Extract authentic high-frequency texture details from the clean surrounding area (outside exclude zone)
// Detail = RGBA - BaseIllumination.
// For each pixel in the hole, synthesize detail from clean donor pixels, preserving skin pores,
// fabric weave, surface grain, and micro-structures so the healed area NEVER appears flat or muddy ("bệt").
const detailR = new Float32Array(w * h);
const detailG = new Float32Array(w * h);
const detailB = new Float32Array(w * h);
let seed = (w * 31 + h * 17 + holeCount) | 0;
function prng(): number {
seed = (seed * 1103515245 + 12345) & 0x7fffffff;
return (seed / 0x7fffffff) * 2 - 1;
}
// 5. Output: natural photographic grain matching the surrounding photo
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (hole[idx]) {
if (!exclude[idx]) {
const p = idx * 4;
const lum = prng() * noiseAmp;
const gr = lum + prng() * (noiseAmp * 0.2);
const gg = lum + prng() * (noiseAmp * 0.2);
const gb = lum + prng() * (noiseAmp * 0.2);
out[p] = Math.max(0, Math.min(255, Math.round(fR[idx] + gr)));
out[p + 1] = Math.max(0, Math.min(255, Math.round(fG[idx] + gg)));
out[p + 2] = Math.max(0, Math.min(255, Math.round(fB[idx] + gb)));
out[p + 3] = 255;
detailR[idx] = rgba[p] - fR[idx];
detailG[idx] = rgba[p + 1] - fG[idx];
detailB[idx] = rgba[p + 2] - fB[idx];
}
}
}
// Pre-index clean donor points
const donorPoints: { x: number; y: number; idx: number }[] = [];
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (!exclude[idx]) {
donorPoints.push({ x, y, idx });
}
}
}
const dLen = donorPoints.length;
const dStep = dLen <= 128 ? 1 : Math.max(1, Math.floor(dLen / 96));
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) {
const idx = row + x;
if (!hole[idx]) continue;
// Find nearest clean donor texture
let bestDist = Infinity;
let bestIdx = -1;
for (let i = 0; i < dLen; i += dStep) {
const dp = donorPoints[i];
const dist = (x - dp.x) * (x - dp.x) + (y - dp.y) * (y - dp.y);
if (dist < bestDist) {
bestDist = dist;
bestIdx = dp.idx;
}
}
const p = idx * 4;
const dR = bestIdx >= 0 ? detailR[bestIdx] : 0;
const dG = bestIdx >= 0 ? detailG[bestIdx] : 0;
const dB = bestIdx >= 0 ? detailB[bestIdx] : 0;
out[p] = Math.max(0, Math.min(255, Math.round(fR[idx] + dR)));
out[p + 1] = Math.max(0, Math.min(255, Math.round(fG[idx] + dG)));
out[p + 2] = Math.max(0, Math.min(255, Math.round(fB[idx] + dB)));
out[p + 3] = 255;
}
}
return out;
}
@@ -429,11 +473,7 @@ export async function inpaintCrop(
return patchCache.get(key)!.slice();
}
// On iOS (iPhone/iPad WebKit), executing a 28MB GAN in WebAssembly
// alongside CanvasKit exceeds the strict iOS per-tab Jetsam memory limit (~350MB)
// or triggers WebKit watchdog termination, causing the tab to auto-reload.
// We use fastLocalInpaint directly, which executes in <15ms, consumes 0MB WASM heap,
// and produces seamless, noise-matched inpainting without any reload.
// On iOS, directly use fastLocalInpaint to stay within strict WebKit memory limits
if (isIOS()) {
const patch = fastLocalInpaint(rgba, width, height, hole);
if (patchCache.size > 50) {
@@ -444,7 +484,7 @@ export async function inpaintCrop(
return patch;
}
// Single-flight lock: serialize inference executions to prevent WebAssembly memory spikes on mobile
// Single-flight lock: serialize inference executions to prevent WebAssembly memory spikes
if (inFlightInpaint) {
try {
await inFlightInpaint;
@@ -455,6 +495,23 @@ export async function inpaintCrop(
}
const runPromise = (async (): Promise<Uint8Array | null> => {
// Compute clean boundary average to fill the hole in `src`, preventing the model
// from seeing or recycling the blemish pattern/color.
let sumR = 0, sumG = 0, sumB = 0, bCount = 0;
const stepSample = Math.max(1, Math.floor((width * height) / 256));
for (let i = 0; i < width * height; i += stepSample) {
if (!hole[i]) {
const p = i * 4;
sumR += rgba[p];
sumG += rgba[p + 1];
sumB += rgba[p + 2];
bCount++;
}
}
const cleanR = bCount > 0 ? Math.round(sumR / bCount) : 128;
const cleanG = bCount > 0 ? Math.round(sumG / bCount) : 128;
const cleanB = bCount > 0 ? Math.round(sumB / bCount) : 128;
// NCHW, 0..255 RGB — what the model was trained to read — and the mask in its
// own polarity: 255 everywhere the frame is known, 0 over the hole.
const src = new Uint8Array(3 * w * h);
@@ -463,34 +520,44 @@ export async function inpaintCrop(
const sy = height / h;
const plane = w * h;
for (let y = 0; y < h; y++) {
// Nearest source pixel rather than a box filter: the crop is the frame at
// its own resolution whenever it fits, so the two agree pixel for pixel and
// this loop never runs at all in the common case.
const syy = Math.min(height - 1, Math.floor(y * sy));
for (let x = 0; x < w; x++) {
const sxx = Math.min(width - 1, Math.floor(x * sx));
const s = (syy * width + sxx) * 4;
const d = y * w + x;
src[d] = rgba[s];
src[plane + d] = rgba[s + 1];
src[2 * plane + d] = rgba[s + 2];
if (hole[syy * width + sxx]) mask[d] = 0;
const isH = Boolean(hole[syy * width + sxx]);
src[d] = isH ? cleanR : rgba[s];
src[plane + d] = isH ? cleanG : rgba[s + 1];
src[2 * plane + d] = isH ? cleanB : rgba[s + 2];
if (isH) mask[d] = 0;
}
}
try {
const { ort, session } = await load();
// Allow up to 15000ms for neural inpainting session to initialize and synthesize scene context
const sessionPromise = load();
const loadWithTimeout = Promise.race([
sessionPromise,
new Promise<never>((_, reject) => setTimeout(() => reject(new Error('Load timeout')), 15000)),
]);
const { ort, session } = await loadWithTimeout;
const in0 = new ort.Tensor('uint8', src, [1, 3, h, w]);
const in1 = new ort.Tensor('uint8', mask, [1, 1, h, w]);
const out = await session.run({
[session.inputNames[0]]: in0,
[session.inputNames[1]]: in1,
});
const runWithTimeout = Promise.race([
session.run({
[session.inputNames[0]]: in0,
[session.inputNames[1]]: in1,
}),
new Promise<never>((_, reject) =>
setTimeout(() => reject(new Error('Inference timeout')), 15000)
),
]);
const out = await runWithTimeout;
const tensor = out[session.outputNames[0]];
const oplane = w * h;
const data = tensor.data as Uint8Array;
// Back to RGBA, opaque: the crop covers the frame, which has no transparency
// of its own to carry.
const patch = new Uint8Array(4 * w * h);
for (let i = 0; i < oplane; i++) {
patch[i * 4] = data[i];
@@ -511,15 +578,11 @@ export async function inpaintCrop(
}
patchCache.set(key, patch.slice());
return patch;
} catch (err) {
console.warn('[inpaint] model unavailable, using fast local inpaint fallback', err);
const patch = fastLocalInpaint(rgba, width, height, hole);
if (patchCache.size > 50) {
const first = patchCache.keys().next().value;
if (first) patchCache.delete(first);
}
patchCache.set(key, patch.slice());
return patch;
} catch {
// Fallback: fast local inpaint with multi-step Poisson harmonic propagation.
// Do NOT cache the fallback in patchCache so that once neural inpainting finishes
// initializing, subsequent renders will execute neural context interpolation.
return fastLocalInpaint(rgba, width, height, hole);
}
})();
docker/frontend/src/styles/app.css
+3 -1
View File
@@ -827,7 +827,9 @@ button.hint:disabled { cursor: default; }
stroke-width: 1.5px;
}
.brush-stroke-contour[data-idle] {
opacity: 0;
opacity: 0.5;
stroke: var(--accent, #e67e22);
fill: rgba(230, 126, 34, 0.1);
}
/* The × on the spot the pointer chose. It sits on the stroke's corner and is the
docker/frontend/src/ui/ImageStage.tsx
+310 -162
View File
@@ -93,65 +93,124 @@ const wmBoxOf = (mark: WmMark, fontPx: number) => {
let contourCanvas: HTMLCanvasElement | null = null;
let contourCtx: CanvasRenderingContext2D | null = null;
// Moore-Neighbor tracing (8-connected) to find the outermost perimeter of a binary mask
const traceOuterBoundary = (
alpha: Uint8ClampedArray,
// Helper to calculate signed polygon area
const polygonArea = (pts: { x: number; y: number }[]): number => {
let a = 0;
for (let i = 0, j = pts.length - 1; i < pts.length; j = i++) {
a += pts[j].x * pts[i].y - pts[i].x * pts[j].y;
}
return Math.abs(a * 0.5);
};
// Traces ALL closed contours (both the largest outer perimeters and inner hole cutouts)
// using Marching Squares on a binary raster grid with exact directed boundary edges.
const traceAllContours = (
alpha: Uint8ClampedArray | Uint8Array,
width: number,
height: number
): { x: number; y: number }[] => {
let startX = -1;
let startY = -1;
for (let y = 0; y < height && startX === -1; y++) {
const rowOffset = y * width;
): { x: number; y: number }[][] => {
if (width < 2 || height < 2) return [];
// 1-pixel border pad to guarantee contours close cleanly without boundary clipping
const gw = width + 2;
const gh = height + 2;
const grid = new Uint8Array(gw * gh);
for (let y = 0; y < height; y++) {
const rowIn = y * width;
const rowOut = (y + 1) * gw + 1;
for (let x = 0; x < width; x++) {
if (alpha[(rowOffset + x) * 4 + 3] > 64) {
startX = x;
startY = y;
break;
if (alpha[(rowIn + x) * 4 + 3] > 64) {
grid[rowOut + x] = 1;
}
}
}
if (startX === -1) return [];
// Directions: 0: R, 1: DR, 2: D, 3: DL, 4: L, 5: UL, 6: U, 7: UR
const dx = [1, 1, 0, -1, -1, -1, 0, 1];
const dy = [0, 1, 1, 1, 0, -1, -1, -1];
// Directed segments on cell edge midpoints keeping inside (1-pixels) on the RIGHT:
// top: (2*x + 1, 2*y), right: (2*x + 2, 2*y + 1), bottom: (2*x + 1, 2*y + 2), left: (2*x, 2*y + 1)
const nextSegment = new Map<number, number[]>();
const addSegment = (from: number, to: number) => {
let arr = nextSegment.get(from);
if (!arr) {
arr = [];
nextSegment.set(from, arr);
}
arr.push(to);
};
const contour: { x: number; y: number }[] = [];
let currX = startX;
let currY = startY;
let enterDir = 4; // came from left
contour.push({ x: currX, y: currY });
const ptKey = (px: number, py: number) => (py << 16) | (px & 0xffff);
const keyX = (k: number) => k & 0xffff;
const keyY = (k: number) => k >> 16;
const maxSteps = width * height;
let steps = 0;
for (let y = 0; y < gh - 1; y++) {
const row = y * gw;
const nextRow = (y + 1) * gw;
for (let x = 0; x < gw - 1; x++) {
const tl = grid[row + x];
const tr = grid[row + x + 1];
const br = grid[nextRow + x + 1];
const bl = grid[nextRow + x];
while (steps++ < maxSteps) {
let found = false;
const nextDir = (enterDir + 5) % 8;
for (let i = 0; i < 8; i++) {
const d = (nextDir + i) % 8;
const nx = currX + dx[d];
const ny = currY + dy[d];
if (
nx >= 0 &&
nx < width &&
ny >= 0 &&
ny < height &&
alpha[(ny * width + nx) * 4 + 3] > 64
) {
currX = nx;
currY = ny;
enterDir = d;
found = true;
break;
const c = (tl << 3) | (tr << 2) | (br << 1) | bl;
if (c === 0 || c === 15) continue;
const top = ptKey(2 * x + 1, 2 * y);
const right = ptKey(2 * x + 2, 2 * y + 1);
const bottom = ptKey(2 * x + 1, 2 * y + 2);
const left = ptKey(2 * x, 2 * y + 1);
switch (c) {
case 1: addSegment(bottom, left); break;
case 2: addSegment(right, bottom); break;
case 3: addSegment(right, left); break;
case 4: addSegment(top, right); break;
case 5: addSegment(top, right); addSegment(bottom, left); break;
case 6: addSegment(top, bottom); break;
case 7: addSegment(top, left); break;
case 8: addSegment(left, top); break;
case 9: addSegment(bottom, top); break;
case 10: addSegment(left, top); addSegment(right, bottom); break;
case 11: addSegment(right, top); break;
case 12: addSegment(left, right); break;
case 13: addSegment(bottom, right); break;
case 14: addSegment(left, bottom); break;
}
}
if (!found || (currX === startX && currY === startY)) break;
contour.push({ x: currX, y: currY });
}
return contour;
const contours: { x: number; y: number }[][] = [];
for (const start of nextSegment.keys()) {
const startEdges = nextSegment.get(start);
while (startEdges && startEdges.length > 0) {
const loop: { x: number; y: number }[] = [];
let curr = start;
let guard = 0;
const maxLen = 50000;
while (guard++ < maxLen) {
const kx = keyX(curr);
const ky = keyY(curr);
loop.push({
x: kx * 0.5 - 1,
y: ky * 0.5 - 1,
});
const curEdges = nextSegment.get(curr);
if (!curEdges || curEdges.length === 0) break;
const next = curEdges.pop()!;
if (next === start) {
if (loop.length >= 3) {
contours.push(loop);
}
break;
}
curr = next;
}
}
}
return contours;
};
// Ramer-Douglas-Peucker polygon simplification
@@ -190,7 +249,8 @@ const simplifyContour = (
return [p0, pend];
};
// Computes the total outer union boundary of all overlapping circles in a stroke
// Computes the total outer union boundary and subtracts overlapping interior cutouts
// so both outer and inner stroke lines are properly retained.
const strokeContourPath = (
spots: { x: number; y: number; r: number }[],
box: { width: number; height: number }
@@ -221,7 +281,7 @@ const strokeContourPath = (
let minY = Infinity;
let maxY = -Infinity;
const maxR = Math.max(...spots.map((s) => s.r * bw));
const pad = Math.ceil(maxR + 6);
const pad = Math.ceil(maxR + 8);
for (const s of spots) {
const px = s.x * bw;
@@ -246,16 +306,16 @@ const strokeContourPath = (
contourCtx.lineCap = 'round';
contourCtx.lineJoin = 'round';
// Connect centers with thick line
contourCtx.lineWidth = 2 * spots[0].r * bw;
contourCtx.beginPath();
for (let i = 0; i < spots.length; i++) {
const x = spots[i].x * bw - originX;
const y = spots[i].y * bh - originY;
if (i === 0) contourCtx.moveTo(x, y);
else contourCtx.lineTo(x, y);
// Connect consecutive spots with accurate segment widths
for (let i = 0; i + 1 < spots.length; i++) {
const p1 = spots[i];
const p2 = spots[i + 1];
contourCtx.lineWidth = (p1.r + p2.r) * bw;
contourCtx.beginPath();
contourCtx.moveTo(p1.x * bw - originX, p1.y * bh - originY);
contourCtx.lineTo(p2.x * bw - originX, p2.y * bh - originY);
contourCtx.stroke();
}
contourCtx.stroke();
// Fill circle at each spot
for (const s of spots) {
@@ -268,16 +328,28 @@ const strokeContourPath = (
}
const imgData = contourCtx.getImageData(0, 0, cw, ch);
const rawContour = traceOuterBoundary(imgData.data, cw, ch);
if (rawContour.length >= 3) {
const simplified = simplifyContour(rawContour, 1.2);
if (simplified.length >= 3) {
const rawContours = traceAllContours(imgData.data, cw, ch);
if (rawContours.length > 0) {
// Sort by polygon area descending so the largest outer perimeter is primary
const sortedContours = rawContours
.map((c) => ({ pts: c, area: polygonArea(c) }))
.filter((c) => c.area > 8)
.sort((a, b) => b.area - a.area);
const subpaths: string[] = [];
for (const { pts: raw } of sortedContours) {
if (raw.length < 3) continue;
const simplified = simplifyContour(raw, 1.2);
if (simplified.length < 3) continue;
let d = `M ${(simplified[0].x + originX).toFixed(1)} ${(simplified[0].y + originY).toFixed(1)}`;
for (let i = 1; i < simplified.length; i++) {
d += ` L ${(simplified[i].x + originX).toFixed(1)} ${(simplified[i].y + originY).toFixed(1)}`;
}
d += ' Z';
return d;
subpaths.push(d);
}
if (subpaths.length > 0) {
return subpaths.join(' ');
}
}
}
@@ -287,78 +359,102 @@ const strokeContourPath = (
}
}
// Fallback hull
const pts = spots.map((s) => ({ x: s.x * bw, y: s.y * bh, r: s.r * bw }));
const r = pts[0].r;
const n = pts.length;
const leftPts: { x: number; y: number }[] = [];
const rightPts: { x: number; y: number }[] = [];
for (let i = 0; i < n; i++) {
let dx = 0;
let dy = 0;
if (i === 0) {
dx = pts[1].x - pts[0].x;
dy = pts[1].y - pts[0].y;
} else if (i === n - 1) {
dx = pts[n - 1].x - pts[n - 2].x;
dy = pts[n - 1].y - pts[n - 2].y;
} else {
dx = pts[i + 1].x - pts[i - 1].x;
dy = pts[i + 1].y - pts[i - 1].y;
}
const len = Math.hypot(dx, dy) || 1;
const nx = (-dy / len) * r;
const ny = (dx / len) * r;
leftPts.push({ x: pts[i].x + nx, y: pts[i].y + ny });
rightPts.push({ x: pts[i].x - nx, y: pts[i].y - ny });
}
let d = `M ${leftPts[0].x.toFixed(1)} ${leftPts[0].y.toFixed(1)}`;
for (let i = 1; i < n; i++) {
d += ` L ${leftPts[i].x.toFixed(1)} ${leftPts[i].y.toFixed(1)}`;
}
d += ` A ${r.toFixed(1)} ${r.toFixed(1)} 0 0 0 ${rightPts[n - 1].x.toFixed(1)} ${rightPts[n - 1].y.toFixed(1)}`;
for (let i = n - 2; i >= 0; i--) {
d += ` L ${rightPts[i].x.toFixed(1)} ${rightPts[i].y.toFixed(1)}`;
}
d += ` A ${r.toFixed(1)} ${r.toFixed(1)} 0 0 0 ${leftPts[0].x.toFixed(1)} ${leftPts[0].y.toFixed(1)} Z`;
return d;
// Fallback circle union
return spots
.map((s) => {
const cx = s.x * bw;
const cy = s.y * bh;
const r = s.r * bw;
return `M ${(cx - r).toFixed(1)} ${cy.toFixed(1)} A ${r.toFixed(1)} ${r.toFixed(1)} 0 1 0 ${(cx + r).toFixed(1)} ${cy.toFixed(1)} A ${r.toFixed(1)} ${r.toFixed(1)} 0 1 0 ${(cx - r).toFixed(1)} ${cy.toFixed(1)} Z`;
})
.join(' ');
};
// Finds all spot indices belonging to the same stroke as targetIndex
// How far off the mark a press still counts as a press ON it: half the brush's
// radius, and never less than a few pixels — a brush turned down to a speck has
// no half-radius to hit. What a drag leaves is a band, and this is its spine,
// the middle of the band: where the hand goes to take hold of the mark. The
// outer half of the band is the photo, and a press there paints the next stroke
// — which is what a user brushing a fresh repair against a healed one means.
// Reaching the whole radius instead, as this did first, makes every press within
// a brush of a healed stroke a drag of it: the stroke the user meant to paint
// beside it never gets painted, and the healed one is dragged over the photo.
//
// A spot written before the brush numbered its drags has no run to measure
// against and keeps the whole circle it always had.
const grabReach = (r: number, boxWidth: number) => Math.max(3, r * boxWidth * 0.5);
const legacyReach = (r: number, boxWidth: number) => Math.max(4, r * boxWidth);
// How far the pointer is from the run between two spots of one drag, in the
// layer's own pixels. The spots of a stroke sit a brush apart at most, so the
// middle of the band lies between two of them and counting to the nearest centre
// alone leaves the spine peppered with holes a press falls straight through; the
// segment the drag actually travelled is what the hand is on.
const gapToRun = (
px: number,
py: number,
a: { x: number; y: number },
b: { x: number; y: number },
bw: number,
bh: number
): number => {
const ax = a.x * bw;
const ay = a.y * bh;
const dx = b.x * bw - ax;
const dy = b.y * bh - ay;
const len2 = dx * dx + dy * dy;
const t = len2 > 0 ? Math.max(0, Math.min(1, ((px - ax) * dx + (py - ay) * dy) / len2)) : 0;
return Math.hypot(px - (ax + dx * t), py - (ay + dy * t));
};
// Finds all spot indices belonging to the same stroke/patch as targetIndex: the
// drag the brush labelled it with, or — for a spot written before the brush
// labelled anything — the circles that lie against it.
const getStrokeIndices = (
spotIndex: number,
spots: { x: number; y: number; r: number }[],
spots: { x: number; y: number; r: number; stroke?: number }[],
box: { width: number; height: number } | null
): number[] => {
if (spotIndex < 0 || spotIndex >= spots.length || !box) return [spotIndex];
let start = spotIndex;
while (start > 0) {
const prev = spots[start - 1];
const curr = spots[start];
const gap = Math.hypot((curr.x - prev.x) * box.width, (curr.y - prev.y) * box.height);
if (prev.r === curr.r && prev.r > 0 && gap <= (prev.r + curr.r) * box.width) {
start--;
} else {
break;
}
}
let end = spotIndex;
while (end + 1 < spots.length) {
const curr = spots[end];
const next = spots[end + 1];
const gap = Math.hypot((next.x - curr.x) * box.width, (next.y - curr.y) * box.height);
if (next.r === curr.r && next.r > 0 && gap <= (curr.r + next.r) * box.width) {
end++;
} else {
break;
}
const n = spots.length;
const bw = box.width;
const bh = box.height;
const key = spots[spotIndex].stroke;
if (key !== undefined) {
const same: number[] = [];
for (let i = 0; i < n; i++) if (spots[i].stroke === key) same.push(i);
return same;
}
const visited = new Uint8Array(n);
const queue = [spotIndex];
visited[spotIndex] = 1;
const result: number[] = [];
for (let k = start; k <= end; k++) result.push(k);
return result;
while (queue.length > 0) {
const curr = queue.shift()!;
result.push(curr);
const sc = spots[curr];
const rc = sc.r * bw;
for (let j = 0; j < n; j++) {
if (visited[j]) continue;
// A stamped spot belongs to its own drag; an unlabelled one reaching it
// is not the same gesture, however close the two circles lie.
if (spots[j].stroke !== undefined) continue;
const sj = spots[j];
const rj = sj.r * bw;
const maxGap = (rc + rj) * 1.35;
const dist = Math.hypot((sc.x - sj.x) * bw, (sc.y - sj.y) * bh);
if (dist <= maxGap) {
visited[j] = 1;
queue.push(j);
}
}
}
return result.sort((a, b) => a - b);
};
interface StrokeRun {
@@ -368,38 +464,73 @@ interface StrokeRun {
}
const strokeRuns = (
spots: { x: number; y: number; r: number }[],
spots: { x: number; y: number; r: number; stroke?: number }[],
box: { width: number; height: number }
): { paths: StrokeRun[]; joined: boolean[] } => {
const paths: StrokeRun[] = [];
const joined = spots.map(() => false);
if (!spots.length) return { paths, joined };
const n = spots.length;
const bw = box.width;
const bh = box.height;
let currentRun: number[] = [0];
for (let i = 0; i + 1 < spots.length; i++) {
const a = spots[i];
const b = spots[i + 1];
const gap = Math.hypot((a.x - b.x) * box.width, (a.y - b.y) * box.height);
if (a.r === b.r && a.r > 0 && gap <= (a.r + b.r) * box.width) {
currentRun.push(i + 1);
} else {
const runSpots = currentRun.map((idx) => spots[idx]);
const contourPath = strokeContourPath(runSpots, box);
const w = 2 * (runSpots[0]?.r ?? 0.012) * box.width;
paths.push({ spots: currentRun.slice(), contourPath, w });
if (currentRun.length > 1) {
for (const idx of currentRun) joined[idx] = true;
}
currentRun = [i + 1];
}
const visited = new Uint8Array(n);
const groups: number[][] = [];
// Spots the brush labelled are grouped by the drag that laid them, so one
// drag is one band however it doubled back, and two drags that run against
// each other stay two bands however close they lie.
const byStroke = new Map<number, number[]>();
for (let i = 0; i < n; i++) {
const key = spots[i].stroke;
if (key === undefined) continue;
visited[i] = 1;
const g = byStroke.get(key);
if (g) g.push(i);
else byStroke.set(key, [i]);
}
if (currentRun.length) {
const runSpots = currentRun.map((idx) => spots[idx]);
for (const g of byStroke.values()) groups.push(g);
// What is left unlabelled is a recipe written before the brush kept the
// gesture, and there is nothing in it to go on but the distance between
// circles — the same reading it always got, so an old recipe still shows the
// bands it showed when it was drawn.
for (let i = 0; i < n; i++) {
if (visited[i]) continue;
const cluster: number[] = [];
const queue = [i];
visited[i] = 1;
while (queue.length > 0) {
const curr = queue.shift()!;
cluster.push(curr);
const sc = spots[curr];
const rc = sc.r * bw;
for (let j = 0; j < n; j++) {
if (visited[j]) continue;
const sj = spots[j];
const rj = sj.r * bw;
const maxGap = (rc + rj) * 1.35;
const dist = Math.hypot((sc.x - sj.x) * bw, (sc.y - sj.y) * bh);
if (dist <= maxGap) {
visited[j] = 1;
queue.push(j);
}
}
}
groups.push(cluster);
}
for (const cluster of groups) {
const clusterSorted = cluster.sort((a, b) => a - b);
const runSpots = clusterSorted.map((idx) => spots[idx]);
const contourPath = strokeContourPath(runSpots, box);
const w = 2 * (runSpots[0]?.r ?? 0.012) * box.width;
paths.push({ spots: currentRun.slice(), contourPath, w });
if (currentRun.length > 1) {
for (const idx of currentRun) joined[idx] = true;
const w = 2 * (runSpots[0]?.r ?? 0.012) * bw;
paths.push({ spots: clusterSorted, contourPath, w });
if (clusterSorted.length > 1) {
for (const idx of clusterSorted) joined[idx] = true;
}
}
@@ -923,17 +1054,26 @@ export function ImageStage({
// past the circle so a brush turned down to a few pixels is still a repair the
// user can take hold of.
const spotUnder = useCallback(
(p: BrushPoint) => {
(p: { fx: number; fy: number }) => {
if (!box || brushTool !== 'heal' || !onBrushEdit) return null;
let best: number | null = null;
let bestGap = Infinity;
const px = p.fx * box.width;
const py = p.fy * box.height;
for (let i = brushSpots.length - 1; i >= 0; i--) {
const s = brushSpots[i];
// The circle itself, with a few pixels of slop: a brush turned down to
// a couple of pixels is still a repair the user can take hold of, and a
// press a little way off one is still a press that means to paint.
const reach = Math.max(4, s.r * box.width);
const gap = Math.hypot((p.fx - s.x) * box.width, (p.fy - s.y) * box.height);
// A spot of a numbered drag is taken hold of along the run: the nearest
// circle of it, or the line between it and the circle the drag laid
// before it (gapToRun). A spot of an older recipe has no run, and keeps
// the circle.
const prev = i > 0 && s.stroke !== undefined && brushSpots[i - 1].stroke === s.stroke ? brushSpots[i - 1] : null;
const reach = s.stroke === undefined ? legacyReach(s.r, box.width) : grabReach(s.r, box.width);
const gap = prev
? Math.min(
Math.hypot(px - s.x * box.width, py - s.y * box.height),
gapToRun(px, py, prev, s, box.width, box.height)
)
: Math.hypot(px - s.x * box.width, py - s.y * box.height);
if (gap > reach || gap >= bestGap) continue;
best = i;
bestGap = gap;
@@ -1004,16 +1144,15 @@ export function ImageStage({
})()
: null;
// The band the pointer is on is the band it can take hold of — the same
// question spotUnder answers for a press, asked of where the pointer is now,
// so the wash that says "this one is in hand" appears exactly where a press
// would take it.
const bandOn = (b: StrokeRun) => {
if (selSpot !== null && b.spots.includes(selSpot)) return true;
if (handSpot !== null && b.spots.includes(handSpot)) return true;
if (!brushPos || !box) return false;
return b.spots.some((i) => {
const s = brushSpots[i];
return (
!!s && Math.hypot((brushPos.fx - s.x) * box.width, (brushPos.fy - s.y) * box.height) <= Math.max(4, s.r * box.width)
);
});
const under = brushPos ? spotUnder(brushPos) : null;
return under !== null && b.spots.includes(under);
};
const brushDown = (e: React.PointerEvent<HTMLDivElement>) => {
@@ -1133,7 +1272,15 @@ export function ImageStage({
// repair is taken from is the renderer's business, not the brush's
// (shared/utils/heal.ts), so a stroke over dust and a stroke over a face
// are the same gesture and neither can be refused.
const spots = stroke.pts.map((p) => ({ x: p.fx, y: p.fy, r: brushR }));
// The drag is stamped, one number for every spot it laid: the pointer
// going up is the one moment the end of a gesture is known, and after it
// nothing is left of the gesture but the numbers on the spots. Numbering
// is one past the highest already on the photo, so a stroke keeps its
// group through a reload, a move and an undo without anything being
// written down anywhere else.
let strokeId = 1;
for (const s of brushSpots) if (s.stroke !== undefined && s.stroke >= strokeId) strokeId = s.stroke + 1;
const spots = stroke.pts.map((p) => ({ x: p.fx, y: p.fy, r: brushR, stroke: strokeId }));
onBrushSpots(spots);
// The repair just laid is the one in hand: the pointer has not moved
// since, so no move has reported a hover, and without this the ring of
@@ -2079,6 +2226,7 @@ export function ImageStage({
key={i}
d={b.contourPath}
className="brush-stroke-contour"
fillRule="evenodd"
data-selected={isSel ? '1' : undefined}
data-hover={isHover ? '1' : undefined}
data-idle={on ? undefined : '1'}