3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(tone): read the ramp's base from the pixel's own neighbourhood, so highlights and shadows move a region and keep the detail in it

Browse Source
This commit is contained in:
3dtours
2026-10-15 00:55:09 +07:00
parent 8809eb84b0
commit ba181da86b
5 changed files with 1017 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/shared/utils/toneShader.ts
+193 -31
View File
@@ -244,26 +244,95 @@ const KNOB_SKSL = (['bl', 'sh', 'hl', 'wh'] as const)
// deflection on the sample frame keeps 0.77 of the band's spread at 0.7%, 0.80 at
// 2.5%, 0.81 at 4.8%.
//
// The caller turns this into a BLUR, and it took a bug to make that a blur in
// fact and not only in name. The base used to be nine point samples of the child
// out at plus or minus this radius, and point samples are not an average: on a
// frame with texture at the sampling scale — a waterfall, a mountainside — the
// nine-tap luma aliases, the gain o(base)/base inherits the alias, and the
// reconstruction paints it as mottle. Measured on a 1160x774 frame at
// SHADOW +100, the high-frequency (9px high-pass) part of that gain field was
// 0.063 against 0.005 for a real blur of the same radius — 13x. So the base is
// now a real gaussian blur of the child, made by the caller (Skia's own
// MakeBlur, see blurredBase() in exportEngine.ts) and read here as ONE tap. That
// is also the cheaper pass: nine child evals walk the whole exposure/matrix
// chain nine times, one eval does not.
// The base used to be nine point samples of the child out at plus or minus this
// radius, and point samples are not an average: on a frame with texture at the
// sampling scale — a waterfall, a mountainside — the nine-tap luma aliases, the
// gain o(base)/base inherits the alias, and the reconstruction paints it as
// mottle. Measured on a 1160x774 frame at SHADOW +100, the high-frequency (9px
// high-pass) part of that gain field was 0.063 against 0.005 for a real blur of
// the same radius — 13x.
//
// So the base is made by the caller (TONE_BASE_SKSL, see toneBase() in
// exportEngine.ts): one bilateral pass per axis, the same 1xN/Nx1 frame
// CLARITY_BLUR_SKSL runs (`spatialReference`), read here as ONE tap. A GAUSSIAN —
// which is what the caller drew next — averages the neighbourhood whatever is in
// it, so at a hard edge it lands halfway across: the pixels beside a bright sky
// take the shadow side's move and the halo the doc reports
// (docker/halo_highlight_shadow_white_black.md) comes back with the knob. The
// range weight is what keeps a bright field's move off the dark side of it: a tap
// counts while its luma is within TONE_BASE_RANGE_EV of the base's own, and an
// edge of ~0.6 EV is three of those out, so the base stops there.
//
// A bilateral alone still is not enough, and the second half of the pass answers
// it: a range-weighted average PRESERVES a 1 px line (the line's own taps carry
// all the weight; its neighbours are cut), so a 1 px twig lands in the base and
// every pixel of it takes its own move — the ramp then compresses the twig with
// the sky exactly as a global curve would, which is the local contrast this pass
// exists to keep. The reference filter cuts the base of the MEDIAN of the
// neighbourhood for that reason (lightcraft #632). The centre reference here is
// that median — med3 along the pass's own axis, three taps a step apart — so a
// line one or two pixels wide is detail and not tone: its own base comes out at
// its surroundings' level, it takes the neighbourhood's move, and its own
// difference from the base is added back whole (Base' + Detail, TONE_MATH_SKSL).
export const TONE_BASE_RADIUS = 0.025;
// One gaussian sigma of the base blur, as a fraction of TONE_BASE_RADIUS. A box
// Taps per side of each axis pass. The step follows from the radius, so the
// neighbourhood is still the one the radius names at 2 * TAPS + 1 taps.
export const TONE_BASE_TAPS = 8;
// One gaussian sigma of the base pass, as a fraction of TONE_BASE_RADIUS. A box
// of the same radius and a gaussian of this sigma carry the same weight at the
// radius, so the neighbourhood is the one the radius has always named while the
// cuts are smooth instead of hard.
// radius, so the spatial falloff still reaches the neighbourhood the radius
// names while the cuts are smooth instead of hard — 2 * 0.35 * TONE_BASE_RADIUS,
// the gaussian the base used to be.
export const TONE_BASE_SIGMA = 0.35;
// The range sigma of the base's bilateral weight, in stops of the pixel's own
// luma: a tap counts while log2(tap / reference) is inside it. 0.25 EV puts an
// edge of 0.6 EV (the reference's "an edge of > ~0.6 EV is preserved") three
// sigmas out and cuts it; a texture of 0.05..0.1 EV is inside one sigma and is
// averaged away, which is what a base has to do with texture.
export const TONE_BASE_RANGE_EV = 0.25;
// The base's own pedestal, in luma: the luma a ratio is taken against, so a pixel
// with almost no light on it still divides by something. The same 0.005 floor
// lightMove reads, at the one place a base can be read on a black frame.
const TONE_BASE_PEDESTAL = 0.02;
// One sample step of the base pass, in px, for a surface this size: TONE_BASE_TAPS
// of them per side reach TONE_BASE_RADIUS of the LONG edge. A whole number of
// pixels, so a tap lands ON a pixel — the pass then reads the same pixels on the
// preview and on the file, and a harness's twin can add the same step.
export function toneBaseStep(width: number, height: number): number {
return Math.max(1, Math.round((Math.max(width, height) * TONE_BASE_RADIUS) / TONE_BASE_TAPS));
}
// The base pass runs on a surface this fraction of the frame per axis, because
// the base IS the level of a neighbourhood TONE_BASE_RADIUS of the long edge
// wide (toneBaseStep of the half above): half the samples are all of the detail
// that level can carry, and the pass costs a quarter of what it does on the
// frame. Measured on the 1067x1600 preview with the same drag: the two axis
// passes at 8 taps made a 715 ms render on the FRAME and a 410 ms one on this
// half — against 405 ms for the MakeBlur they replaced (scratchpad
// preview-render-measure.mjs, timed in one session), which is what the quarter
// buys. The base is unmoved to the eye at that scale
// (tone-local-contrast-check.mjs measures the base and the halo).
export const TONE_BASE_SCALE = 0.5;
// The surface the base pass draws on, and the tap step that goes with it: the
// step is taken from the RADIUS and the half's own long edge, so the
// neighbourhood is the one TONE_BASE_RADIUS names of the FRAME either way. The
// caller scales the pass's draw to reach it (see toneBase in exportEngine.ts),
// which is also what box-downscales the frame's child into it.
export function toneBaseSurface(
width: number,
height: number
): { width: number; height: number; step: number } {
const w = Math.max(1, Math.round(width * TONE_BASE_SCALE));
const h = Math.max(1, Math.round(height * TONE_BASE_SCALE));
return { width: w, height: h, step: toneBaseStep(w, h) };
}
// The tone and exposure maths, in ONE copy, because two passes ask it: the
// whole-frame passes here and a gradient mask, which moves the same knobs on the
// shape the user drew. What "HIGHLIGHT" or "EXPOSURE" means must not depend on
@@ -323,10 +392,21 @@ vec3 toneRamp(vec3 c, float t, float base, float bl, float sh, float hl, float w
float hlVal = clamp(hl - dr * 0.09, -1.0, 1.0);
float whVal = clamp(wh - dr * 0.18, -1.0, 1.0);
// Evaluate the tone curve at pixel luma t so every pixel gets a spatially uniform
// global move across the entire frame without any halo or aura ring!
float o = toneCurve(t, blVal, shVal, hlVal, whVal);
float target = clamp(o, 0.0, 1.0);
// BASE' + DETAIL: the ramp is read at the neighbourhood's luma and the pixel's
// own difference from it is ADDED to the result, never scaled by it — the move
// belongs to the neighbourhood (which is what stops a knob turning one region
// of the frame into a different exposure from the one next to it) and the
// detail belongs to the pixel (which is what keeps a twig a twig). Read at the
// pixel, the move is read off the pixel itself and a hole in the sky is only in the
// statistics, not in the picture; read at the base, the same pixels take the
// same move and the 1 px lines inside the region keep their own contrast.
//
// The base lands ON the pixel when the caller has no neighbourhood to speak of
// — a gradient mask hands in the pixel's own luma (see gradientMask.ts) — and there
// two readings are the same instruction, so the degenerate call is the global
// move this used to do and not a second opinion about it.
float o = toneCurve(base, blVal, shVal, hlVal, whVal);
float target = clamp(o + (t - base), 0.0, 1.0);
vec3 res = lightMove(c, t, target);
@@ -395,11 +475,93 @@ vec3 exposureMove(vec3 rgb, float ev) {
}
`;
// THE BASE LAYER of the tone ramp, ONE AXIS PER PASS. The caller runs it twice
// (horizontal, then vertical, both on surfaces the size of the frame) and hands
// the result to TONE_SKSL as its `base` child; it returns the neighbourhood's
// LUMA, so the second pass and the ramp read it without re-weighting anything.
//
// Two properties, and the pass is the two of them:
//
// EDGE-AWARE. The spatial falloff is a gaussian of TONE_BASE_SIGMA out to
// TONE_BASE_RADIUS (see those), and each tap is weighted again by how far its
// luma is from the centre's, in stops — exp(-ev²) with TONE_BASE_RANGE_EV the
// sigma — so a tap across an edge of more than ~0.6 EV is worth nothing and the
// base is the level of THIS side of the edge. That is what a gaussian base
// cannot do and the reason the ramp had to fall back to a global curve: at a
// hard edge a gaussian hands the dark pixels a share of the bright side's move
// and the halo (halo_highlight_shadow_white_black.md) follows the knob.
//
// THE CENTRE IS A MEDIAN. The range weight is read against med3 along the
// pass's own axis — the centre and its two neighbours a step out — and not
// against the centre pixel. A range-weighted average PRESERVES a 1 px line (at
// the line, every tap but the line's own is cut, so the line IS the base and
// the ramp then compresses it against the sky like any global curve: the local
// contrast this whole pass exists to keep). A median is blind to a line one or
// two pixels wide, so at a twig the reference is the sky, the twig takes the
// sky's move, and its own difference from the base rides through the ramp
// untouched (Base' + Detail, TONE_MATH_SKSL). The two passes together are the
// reference filter's median_hv + guided filter (lightcraft local.rs, #632).
//
// The luma weights are the ramp's own, and the difference is taken as
// (tap - reference) / (reference + TONE_BASE_PEDESTAL): a ratio, so the range
// sigma is the same size of edge on a dark frame as on a bright one.
export const TONE_BASE_SKSL = `
uniform shader src;
// One tap step along this pass's axis, in px OF THIS PASS'S SURFACE — the half
// of the frame toneBaseSurface names: whole, so every tap lands on a pixel (see
// toneBaseStep). Read out at the frame's own size it is TONE_BASE_RADIUS wide.
uniform float2 dir;
const float3 TONE_BASE_LUM = vec3(0.2126, 0.7152, 0.0722);
// The spatial falloff, in tap indices: a gaussian of TONE_BASE_SIGMA of the
// radius, TAPS taps out.
const float TONE_BASE_SPACE_SIGMA = ${(TONE_BASE_TAPS * TONE_BASE_SIGMA).toFixed(3)};
// ln(2) * TONE_BASE_RANGE_EV: (tap - reference) / (reference + pedestal) over
// this IS the difference in stops, which is what the range sigma counts.
const float TONE_BASE_RANGE = ${(Math.LN2 * TONE_BASE_RANGE_EV).toFixed(6)};
const float TONE_BASE_PED = ${TONE_BASE_PEDESTAL.toFixed(4)};
float toneBaseLum(vec4 c) {
return dot(clamp(c.rgb, 0.0, 1.0), TONE_BASE_LUM);
}
// The median of three: max(min(a, b), min(max(a, b), c)).
float toneBaseMed3(float a, float b, float c) {
return max(min(a, b), min(max(a, b), c));
}
vec4 main(vec2 xy) {
vec4 c = src.eval(xy);
float l0 = toneBaseLum(c);
float ref = toneBaseMed3(toneBaseLum(src.eval(xy - dir)), l0, toneBaseLum(src.eval(xy + dir)));
// The centre contributes its MEDIAN, so a line the median is blind to cannot
// pull the base with it.
float sum = ref;
float total = 1.0;
for (int i = 1; i <= ${TONE_BASE_TAPS}; i++) {
float fi = float(i);
float g = exp(-0.5 * (fi / TONE_BASE_SPACE_SIGMA) * (fi / TONE_BASE_SPACE_SIGMA));
float a1 = toneBaseLum(src.eval(xy + dir * fi));
float a2 = toneBaseLum(src.eval(xy - dir * fi));
float d1 = (a1 - ref) / (ref + TONE_BASE_PED) / TONE_BASE_RANGE;
float d2 = (a2 - ref) / (ref + TONE_BASE_PED) / TONE_BASE_RANGE;
float r1 = exp(-d1 * d1);
float r2 = exp(-d2 * d2);
sum += g * (r1 * a1 + r2 * a2);
total += g * (r1 + r2);
}
float base = sum / total;
return vec4(base, base, base, c.a);
}
`;
export const TONE_SKSL = `
uniform shader src;
// The BASE layer's child: a blur of the image above, of TONE_BASE_RADIUS, made
// by the caller. Read for its luma alone, and read ONCE per pixel — the
// neighbourhood is the blur's, not a sampling loop's (see TONE_BASE_RADIUS).
// The BASE layer's child: the image above run through TONE_BASE_SKSL, of
// TONE_BASE_RADIUS, made by the caller. Read for its luma alone, and read ONCE
// per pixel — the neighbourhood is the base pass's, not a sampling loop's (see
// TONE_BASE_RADIUS). The caller builds it at TONE_BASE_SCALE and hands it in
// with the local matrix that puts it back in the frame's coordinates, so the
// reads below are the frame's own — and with a NEAREST sampler: that matrix is a
// whole 2x for TONE_BASE_SCALE, and a linear read of a step edge at 2x leaves a
// quarter of the far side in the pixel beside it, which is the halo the base
// exists to not have (see baseUpscale in exportEngine.ts).
uniform shader base;
uniform float dr;
uniform float hl;
@@ -468,17 +630,17 @@ float bandW(float hue, float anchor, float gapL, float gapR) {
}
${TONE_MATH_SKSL}
// The BASE layer: the light the frame carries where this pixel sits, at the
// radius the caller handed in. One tap of a real blur of the same child the
// pixel comes from, so it is an AVERAGE of the neighbourhood and not a handful
// of point samples of it — that distinction is the whole bug (see
// TONE_BASE_RADIUS). Luma, because the luma is the one quantity the ramp moves
// and the colour rides the ratio afterwards; the blur being linear, blurring the
// child and taking its luma is the same as blurring the luma.
// radius the caller handed in — one tap of an edge-aware pass over the same
// child the pixel comes from (TONE_BASE_SKSL), so it is the LEVEL of this
// pixel's own side of an edge and not a handful of point samples of it. That
// distinction is the whole bug (see TONE_BASE_RADIUS). The pass already returns
// the luma in all three channels; the clamp is here for a shader that cannot be
// built, where the caller hands in the child itself.
//
// A caller with no neighbourhood to speak of hands in the child itself as the
// base (see blurredBase()): the tap then lands exactly on t and the pass falls
// back to the global move, which is what a shape with a mask's degenerate base
// wants and what it got from a bx of zero before.
// base (see toneBase()): the tap then lands exactly on t and the pass falls back
// to the global move, which is what a shape with a mask's degenerate base wants
// and what it got from a bx of zero before.
float baseLuma(vec2 xy) {
vec3 s = clamp(base.eval(xy).rgb, 0.0, 1.0);
return clamp(dot(s, vec3(0.2126, 0.7152, 0.0722)), 0.0, 1.0);