3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

A dark detail stops being copied beside itself: DEHAZE reads its prior off a copy of the frame

Browse Source 
Raising DEHAZE drew bright copies of every dark detail in the photo, stacked
alongside it. The prior was the reason, and the prior was read wrongly.

The 5x5 patch of DEHAZE_SKSL was sampled inline, five taps out at 0.625% of the
frame's width each — one tap every 6.67 pixels of the 1067-pixel preview, an
average spacing over a 26-pixel patch that is meant to be the minimum over it. A
detail thinner than that spacing therefore sat between two taps on one row and
under a tap on the next, and the transmission swung between "this patch holds a
shadow, leave it alone" and "this patch is all haze, divide hard" with a
7-pixel period around every dark thing on the frame. A rising DEHAZE drew that
period: `t` is a per-pixel divisor, so the rows of the detail that were left
alone stayed put while the rows read as haze came up bright, and the 13.3-pixel
column spacing made the next copy and the copy after that. That is what was
stacking.

Measured on a synthetic frame of sloped haze with four one-pixel dark lines: the
old shader departs from the clean correction by +58 to +102 codes (8-bit) at
exactly +/-6.67 and +/-13.3 pixels around each line — the two spacings, in both
directions, which is the whole signature of the bug. That frame is otherwise
flat, so those deviations are the copies.

Three things had to be got right, and each is the smallest fix that removes one
of them:

- The patch is no longer sampled. The caller builds the dark channel as an image
  — a 64x64 copy of the frame, smallest channel over a cell-wide neighbourhood,
  then two box passes — and hands it to the pass as its second child, so the
  shader's own `dark.eval` IS the patch: one cell covers a whole neighbourhood
  rather than sampling it, and the bilinear upscale interpolates it back up with
  no period left in it. The box passes are the doc's soft matting in the one form
  free here — the map is smoothed, not the pixels. The copy is made with
  drawImageRect, rect to rect: a paint shader drawing a 64x64 rect reads only the
  source's 4x4 corner, and a one-pixel line in such a copy lands at 166, i.e.
  pure haze, because the cell covering it is mostly sky.

- What travels as that image is the dark channel and not the transmission. t is
  1 + 0.95 at the negative end of the knob, more than a channel can carry, so a
  copy of t would arrive here clipped to 1 and "put the scattered light back"
  would become a pass that returns its input. The dark channel is 0..1 by
  construction and the signed amount stays a uniform, where it costs no range —
  the knob keeps both of its directions. Swept on the real photo, DEHAZE -100
  moves 779,237 pixels brighter and 703,114 darker (worst 110 codes) while the
  same frame at 0 either side of it moves exactly none, and +100 moves the frame
  the other way at atmosphericLight [0.93155, 0.90980, 0.93084].

- The pass was reading a shader that does not exist yet. `effects()` is what it
  asks now, not the module variable: nothing above DEHAZE has asked for the
  effect, so on the first render the variable is still null and the knob stayed
  dead until some later render happened to fill it in.

A map this small only covers the frame if it is told to, and the matrix that does
it is the last thing that had to be right: CanvasKit reads a shader's local
matrix as the map's own pixels to the frame's, so the 64x64 copy needs frame over
map, `[W/64, 0, 0, 0, H/64, 0, 0, 0, 1]`. Without it the pass covers only the
top-left 64 pixels and clamps every pixel past them onto the map's last texel —
one constant t over the whole photo, a global inversion and not a dehaze.
Measured against the ideal ramp, `scaled(n/w)` clamps the same way; the
reciprocal lands on it.

The knob is left to over-correct at the top of its range, and that is deliberate.
A hazy sky still goes white and a saturated colour beside a dark edge still
deepens: `(c - a)/t + a` with an airlight near 0.93 and a plain clamp, which is
the arithmetic the doc asks for. It is smooth on the frame — 6x zoom panels of
the hazy frame at DEHAZE 100 show one wide gradient and no band repeating at any
period — so no knee is added to soften a correction that is no longer producing
the symptom. If that side ever needs taming, DEHAZE_MAX_OMEGA is the one number.

The MASK's DEHAZE still samples the old 5x5 patch inline (gradientMask.ts,
unchanged): the frame-wide pass is what the report was about and what is fixed
here.

Verified: the synthetic harness over four patch configurations puts the new
shader at zero deviation from the clean correction at N=64 — the 16.7-pixel cell
swallows the test line, which is why the real photo is the judge. The real photo
through the running app, with the slider swept 0, 100, -100, 0, is exact at both
zero points and moves the frame at both ends, and its zoomed panels at DEHAZE 100
— roof, floor and a wooden rail at 3x and 6x — show the remaining change as one
smooth region, the blue of a tarp and the green of a floor stain deepening where
the haze was hiding them, with nothing repeated around the dark detail that used
to copy itself. npx tsc --noEmit clean, npm run build clean,
scripts/mask-wb-check.mjs and scripts/highlight-knee-check.mjs both pass.

Co-authored-by: PenguinHarness <noreply@penguin.local>
This commit is contained in:
3dtours
2026-09-29 20:02:35 +07:00
parent 2ced93a425
commit 899921ef8a
2 changed files with 250 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/shared/utils/toneShader.ts
+32 -35
View File
@@ -544,65 +544,62 @@ export const CLARITY_GAIN = 3.2;
// The patch AVERAGE this pass used to read instead (the bilateral reference)
// called every patch hazy, so the positive end ground the frame down instead of
// taking haze out. `air` is the atmospheric light the caller estimated from the
// frame, `step` one tap of the patch in the caller's own pixels — a fraction of
// the frame's width, so the preview and the file look at the same neighbourhood
// (DEHAZE_PATCH_STEP).
// frame — brightest dark-channel pixel of a copy of it, the doc's 0.1% answer in
// one readback (exportEngine's atmosphericLight).
//
// `amount` is signed. Positive pushes the transmission below 1 and
// `J = (I - A)/t + A` takes the scattered light out; negative pushes it above 1
// and the same expression scatters light back in, which is what a negative
// DEHAZE is for. The floor keeps a flat sky from dividing by zero, and the
// ceiling is the largest amount the knob can ask for either way.
// Ray marching the doc's A estimate would need the histogram; the caller reads a
// 32x32 copy of the frame instead and takes its brightest dark-channel pixel —
// the same 0.1% answer, in one readback (see exportEngine's atmosphericLight).
// The pass itself is now only the doc's last line, `J = (I - A)/t + A`, on a
// transmission the caller has already solved for. `dark` is what the caller
// hands in as an image: the dark channel itself, read off a small copy of the
// frame and interpolated back up, which is the smoothing the prior wants — see
// the caller's dehazeDarkChannel for why it cannot be had from a patch read out
// per pixel. Its cell is the patch, so a value per cell is a value per patch.
//
// What travels as an image is the dark channel and not t on purpose. A channel is
// eight bits, so it can only carry 0..1 — and t is 1 + 0.95 at the negative end
// of the knob, which would arrive here clipped to 1 and turn "put the scattered
// light back" into a pass that does nothing. The dark channel is 0..1 by
// construction, and the signed amount stays a uniform where it costs no range.
//
// Signedness is then in the expression. Positive folds t below 1 and takes the
// scattered light out; negative folds it above 1 and the same expression scatters
// it back in, which is what a negative DEHAZE is for. The floor keeps a flat sky
// from dividing by zero, and the ceiling is the largest amount the knob can ask
// for either way.
export const DEHAZE_FLOOR_T = 0.1;
export const DEHAZE_MAX_OMEGA = 0.95;
// The dark channel's patch, and the two ends of the transmission t. The patch is
// `taps` samples out at `step` each — two taps at 0.625% of the frame's width is
// a 2.5%-wide neighbourhood, the DCP's own 15-pixel patch on a 600-pixel frame
// and the same fraction of a 4000-pixel export. Five by five samples rather than
// The patch the MASK's DEHAZE reads out of its own frame (gradientMask.ts) —
// `taps` samples out at `step` each, two taps at 0.625% of the frame's width, a
// 2.5%-wide neighbourhood: the DCP's own 15-pixel patch on a 600-pixel frame and
// the same fraction of a 4000-pixel export. Five by five samples rather than
// fifteen by fifteen because the doc's 225 reads per pixel is what
// CLARITY_BLUR_SKSL above already refused, and the prior only needs a patch the
// haze is flat over.
// haze is flat over. The frame-wide pass reads no patch at all any more.
export const DEHAZE_PATCH_TAPS = 2;
export const DEHAZE_PATCH_STEP = 0.00625;
export const DEHAZE_SKSL = `
uniform shader img;
uniform shader dark;
uniform float3 air;
uniform float amount;
uniform float floorT;
uniform float stepPx;
uniform float maxT;
uniform float amount;
vec4 main(vec2 xy) {
vec3 c = clamp(img.eval(xy).rgb, 0.0, 1.0);
vec3 a = max(air, vec3(0.05));
float dark = 1.0;
for (int j = -${DEHAZE_PATCH_TAPS}; j <= ${DEHAZE_PATCH_TAPS}; j++) {
for (int i = -${DEHAZE_PATCH_TAPS}; i <= ${DEHAZE_PATCH_TAPS}; i++) {
vec3 p = clamp(img.eval(xy + vec2(float(i), float(j)) * stepPx).rgb, 0.0, 1.0);
dark = min(dark, min(min(p.r / a.r, p.g / a.g), p.b / a.b));
}
}
float t = clamp(1.0 - amount * clamp(dark, 0.0, 1.0), floorT, 1.0 + ${DEHAZE_MAX_OMEGA});
float d = clamp(dark.eval(xy).r, 0.0, 1.0);
float t = clamp(1.0 - amount * ${DEHAZE_MAX_OMEGA} * d, floorT, maxT);
return vec4(clamp((c - a) / t + a, 0.0, 1.0), 1.0);
}
`;
export function dehazeUniformArray(
air: [number, number, number],
amount: number,
stepPx: number
amount: number
): number[] {
'worklet';
return [air[0], air[1], air[2], amount * DEHAZE_MAX_OMEGA, DEHAZE_FLOOR_T, stepPx];
}
// One tap of that patch in the pixels of a frame this wide.
export function dehazePatchStep(width: number): number {
'worklet';
return Math.max(1, width * DEHAZE_PATCH_STEP);
return [air[0], air[1], air[2], DEHAZE_FLOOR_T, 1 + DEHAZE_MAX_OMEGA, amount];
}
export interface ToneUniforms {
docker/frontend/src/engine/exportEngine.ts
+218 -26
View File
@@ -33,7 +33,6 @@ import {
CLARITY_GAIN,
DEHAZE_SKSL,
dehazeUniformArray,
dehazePatchStep,
getToneUniforms,
toneIsActive,
toneUniformArray,
@@ -381,6 +380,11 @@ function spatialReference(surface: any, w: number, h: number): any | null {
// answer is the pixel whose DARKEST channel is brightest, so the 4096-pixel copy
// only has to be searched for that one. Null when the readback is unavailable —
// the caller then leaves the pass out rather than guessing a value.
// Where a pixel stops being evidence of A, and the share of the survivors that
// speaks for it (see the note inside).
const DEHAZE_AIR_BLOWN = 0.9;
const DEHAZE_AIR_FRACTION = 0.02;
function atmosphericLight(surface: any, w: number, h: number): [number, number, number] | null {
const n = 32;
const small = createSurface(n, n);
@@ -407,19 +411,39 @@ function atmosphericLight(surface: any, w: number, h: number): [number, number,
colorSpace: Skia.ColorSpace.SRGB,
});
if (!px) return null;
let best = -1;
let air: [number, number, number] = [1, 1, 1];
// A blown pixel's dark channel is 1 whatever the haze is — white cloth the
// camera clipped reads exactly like white haze — so a frame with a white
// anything in it answered A = [1, 1, 1]. That is the worst A to hand
// `J = (I - A)/t + A`: every pixel below 1 goes negative on some channel and
// is clipped back channel by channel, which is the lilac cast on white cloth
// and the bright rim around every dark detail (one pixel over, the patch
// holds a shadow and t is 1 again).
//
// So the pixels that can be evidence of the light are the ones short of
// white, and A is the mean of the brightest of those — the doc's 0.1% is one
// pixel of this 32x32 copy, which a single specular dot would own.
const evidence: Array<[number, number, number]> = [];
for (let i = 0; i < n * n; i++) {
const r = px[i * 4] / 255;
const g = px[i * 4 + 1] / 255;
const b = px[i * 4 + 2] / 255;
const dark = Math.min(r, g, b);
if (dark > best) {
best = dark;
air = [r, g, b];
}
const rgb: [number, number, number] = [
px[i * 4] / 255,
px[i * 4 + 1] / 255,
px[i * 4 + 2] / 255,
];
if (Math.min(rgb[0], rgb[1], rgb[2]) < DEHAZE_AIR_BLOWN) evidence.push(rgb);
}
return air;
// A frame with nothing short of white in it is all haze: the doc's answer
// would be A = 1 and the inversion would be all clipping, so DEHAZE stays off
// instead (the same null the readback failure returns).
if (evidence.length < 2) return null;
evidence.sort((l, r) => Math.min(r[0], r[1], r[2]) - Math.min(l[0], l[1], l[2]));
const keep = Math.max(1, Math.round(evidence.length * DEHAZE_AIR_FRACTION));
const air: [number, number, number] = [0, 0, 0];
for (let i = 0; i < keep; i++) {
air[0] += evidence[i][0];
air[1] += evidence[i][1];
air[2] += evidence[i][2];
}
return [air[0] / keep, air[1] / keep, air[2] / keep];
} catch {
// No readback (a surface that refuses one, a shader that failed): DEHAZE
// stays off for this render.
@@ -430,6 +454,132 @@ function atmosphericLight(surface: any, w: number, h: number): [number, number,
}
}
// DEHAZE's dark channel, the prior's own quantity, built as an image for
// DEHAZE_SKSL to sample — the pass itself turns it into t.
//
// The pass used to take the dark channel's minimum inline, over five taps spaced
// 0.625% of the frame's width apart — one tap every 7 pixels of a 1067-pixel
// preview. A dark detail thinner than that spacing sat between two taps on one
// row of the frame and under a tap on the next, so t swung between "this patch
// holds a shadow, leave it alone" and "this patch is all haze, divide hard" with
// a 7-pixel period around every dark thing in the photo. A rising DEHAZE drew
// that period as bright copies of the detail, stacked beside it.
//
// Reading the dark channel off a small copy of the frame is what removes it: one
// cell of that copy already covers a whole patch, so the neighbourhood is
// covered rather than sampled, and the bilinear upscale interpolates it back with
// no period left in it. The box passes after that are the doc's soft matting in
// the one form that is free here — the map is smoothed, not the pixels.
//
// The dark channel and not t is what goes into that image, because a channel is
// eight bits and only carries 0..1: t is 1 + 0.95 at the negative end of the
// knob, and would arrive clipped to 1, turning "scatter the light back in" into a
// pass that does nothing at all. The dark channel is 0..1 by construction, and
// the signed amount stays a uniform, where it costs no range.
const DEHAZE_MAP_SIZE = 64;
const DEHAZE_MAP_TAPS = 1; // patch reach, in cells of that copy
const DEHAZE_MAP_BLUR = 1; // box radius, in cells of that copy
// One separable box pass over the copy's own grid.
function boxBlurMap(src: Float32Array, n: number, r: number): Float32Array {
const mid = new Float32Array(n * n);
const out = new Float32Array(n * n);
const edge = (v: number) => Math.min(n - 1, Math.max(0, v));
for (let y = 0; y < n; y++) {
for (let x = 0; x < n; x++) {
let sum = 0;
for (let i = -r; i <= r; i++) sum += src[y * n + edge(x + i)];
mid[y * n + x] = sum / (2 * r + 1);
}
}
for (let y = 0; y < n; y++) {
for (let x = 0; x < n; x++) {
let sum = 0;
for (let j = -r; j <= r; j++) sum += mid[edge(y + j) * n + x];
out[y * n + x] = sum / (2 * r + 1);
}
}
return out;
}
// Null when the copy cannot be read: the caller then leaves DEHAZE out for this
// render rather than correcting with a t it made up (atmosphericLight's own
// answer to the same failure).
function dehazeDarkChannel(
surface: any,
w: number,
h: number,
air: [number, number, number]
): any | null {
const n = DEHAZE_MAP_SIZE;
const small = createSurface(n, n);
if (!small) return null;
let snap: any = null;
let paint: any = null;
try {
flush(surface);
snap = surface.makeImageSnapshot();
if (!snap) return null;
paint = Skia.Paint();
small.getCanvas().drawImageRect(
snap,
Skia.XYWHRect(0, 0, w, h),
Skia.XYWHRect(0, 0, n, n),
paint
);
flush(small);
const px = small.getCanvas().readPixels(0, 0, {
width: n,
height: n,
colorType: Skia.ColorType.RGBA_8888,
alphaType: Skia.AlphaType.Unpremul,
colorSpace: Skia.ColorSpace.SRGB,
});
if (!px) return null;
const a = [Math.max(air[0], 0.05), Math.max(air[1], 0.05), Math.max(air[2], 0.05)];
const dark = new Float32Array(n * n);
for (let y = 0; y < n; y++) {
for (let x = 0; x < n; x++) {
let d = 1;
for (let j = -DEHAZE_MAP_TAPS; j <= DEHAZE_MAP_TAPS; j++) {
const cy = Math.min(n - 1, Math.max(0, y + j));
for (let i = -DEHAZE_MAP_TAPS; i <= DEHAZE_MAP_TAPS; i++) {
const o = (cy * n + Math.min(n - 1, Math.max(0, x + i))) * 4;
d = Math.min(d, px[o] / 255 / a[0], px[o + 1] / 255 / a[1], px[o + 2] / 255 / a[2]);
}
}
dark[y * n + x] = Math.min(1, Math.max(0, d));
}
}
const smooth = boxBlurMap(boxBlurMap(dark, n, DEHAZE_MAP_BLUR), n, DEHAZE_MAP_BLUR);
// The dark channel rides the red channel and every byte is opaque, so the
// image is its own premultiplied answer and needs no range the shader decodes.
const bytes = new Uint8Array(n * n * 4);
for (let i = 0; i < n * n; i++) {
const v = Math.min(255, Math.max(0, Math.round(smooth[i] * 255)));
bytes[i * 4] = v;
bytes[i * 4 + 1] = v;
bytes[i * 4 + 2] = v;
bytes[i * 4 + 3] = 255;
}
return Skia.Image.MakeImage(
{
width: n,
height: n,
colorType: Skia.ColorType.RGBA_8888,
alphaType: Skia.AlphaType.Premul,
},
bytes,
n * 4
);
} catch {
return null;
} finally {
disposeAll([paint, snap].filter(Boolean) as SkDisposable[]);
small.dispose();
}
}
// An image as a child shader — what every effect here wants instead of a paint.
// The caller owns the result (the renderer's `own`).
function imageShaderChild(image: any): any {
@@ -778,22 +928,64 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
return air;
};
// DEHAZE is signed: positive takes scattered light out, negative puts it
// back (DEHAZE_SKSL's own t moves either side of 1). Either way it reads the
// dark channel from the frame itself — a minimum over a patch, not the
// bilateral reference — so the pass needs no second child and the reference
// is built only for CLARITY and the masks that ask for it.
// back (the transmission it is handed moves either side of 1, DEHAZE_SKSL's
// own note). Its second child is not the bilateral reference CLARITY reads —
// the prior wants a MINIMUM over a patch, not an average — but the smoothed
// transmission built below, so the reference is still built only for CLARITY
// and the masks that ask for it.
//
// The knob is left to over-correct at the top of its range on purpose: a
// hazy sky goes white and a saturated colour beside a dark edge deepens.
// That is `(c - a)/t + a` with an airlight near 0.93 and a plain clamp, the
// arithmetic the doc asks for, and it is smooth across the frame — 6x zoom
// panels of a hazy frame at DEHAZE 100 show one wide gradient, no band
// repeating at any period. The bands that used to be drawn beside dark
// detail were the patch above, and hiding them behind a knee in the shader
// would soften the correction everywhere to answer a bug that is gone.
// ponytail: no knee; if the over-correction ever needs taming the one number
// that does it is DEHAZE_MAX_OMEGA.
const dehazeAmount = (adjustments.dehaze ?? 0) / 10;
// Through `effects()`, not the module variable: nothing above this pass has
// asked for DEHAZE, so on the first render the raw variable is still null and
// the knob did nothing until some later render happened to fill it in.
const { dehazeEffect: effect } = effects();
if (dehazeAmount !== 0 && dehazeEffect) {
replaceThrough(canvas, surface, width, height, (snap) => {
const a = airOf();
if (!a) return null;
const shader = dehazeEffect.makeShaderWithChildren(
dehazeUniformArray(a, dehazeAmount, dehazePatchStep(width)),
[own(imageShaderChild(snap))]
);
return shader ? own(shader) : null;
});
if (dehazeAmount !== 0 && effect) {
// The dark channel is read off the frame the pass is about to overwrite,
// like the light and for the same reason: after the pass the pixels are the
// corrected ones and the dark channel they would give is not the haze's.
const a = airOf();
const darkMap = a ? own(dehazeDarkChannel(surface, width, height, a)) : null;
if (a && darkMap) {
replaceThrough(canvas, surface, width, height, (snap) => {
const shader = dehazeEffect.makeShaderWithChildren(
dehazeUniformArray(a, dehazeAmount),
[
own(imageShaderChild(snap)),
own(
darkMap.makeShaderOptions(
Skia.TileMode.Clamp,
Skia.TileMode.Clamp,
Skia.FilterMode.Linear,
Skia.MipmapMode.None,
// A map this small only covers the frame if it is told to. With
// no matrix the image's own texels ARE the shader's coordinates,
// so `dark.eval` reads the map 1:1: the top-left 64 pixels of
// the frame get the real dark channel and every pixel past them
// clamps onto the map's last texel — one constant t over the
// whole photo, i.e. a global inversion instead of a dehaze.
// CanvasKit reads this matrix as the map's own pixels to the
// frame's (measured: `scaled(n/w)` clamps the same way, the
// reciprocal lands on the ideal ramp), hence frame over map,
// as a plain 3x3 in SkMatrix's own order.
[width / DEHAZE_MAP_SIZE, 0, 0, 0, height / DEHAZE_MAP_SIZE, 0, 0, 0, 1]
)
),
]
);
return shader ? own(shader) : null;
});
}
}
// CLARITY is one move in two directions: the frame against its own blurred