fix(raw): read the frame's own white balance off its embedded preview

The Fujifilm RAF this was reported on opened with magenta through every white in the frame. The develop itself was the cast, not the tone curve fitted over it: run flat — white balance, matrix, knee, encode, no curve at all — the frame's bright neutral blocks came back 232,196,201 against the 223,232,236 the camera's own preview has there. The matrix is not it either; every row of `rgb_cam` sums to one, so a neutral triple opens neutral. What was wrong is the balance it was given. Neither balance LibRaw hands over is the one the frame was rendered with. `pre_mul` on that frame is R/G 2.296, B/G 1.307; `cam_mul` is 1.795, 1.626; the balance the preview's own bright neutral blocks imply is 1.588, 1.266 — and that one is readable: average the sensor triple behind each block the camera left bright and near neutral, and the ratio that equalises those blocks is the frame's white. So the develop reads it there, before the first draw, from the same preview map the tone curve is fitted on, and keeps LibRaw's tags only as the fallback a frame with nothing neutral and bright to read (a clipped or dark frame) falls back to. `cam_mul` is tried before `pre_mul` on that path: as-shot before daylight, measured 6.73 -> 5.27 on this frame where the reverse gave 6.66. Measured end to end through the real develop (LibRaw worker + CanvasKit, 64x64 block grid against the file's own preview), mean dE00 / mean dRGB: HAGIANG-20170202-FUJIFILM-418.RAF 6.73 -> 4.89 +5.8,+0.9,+0.5 -> 0.7,-0.5,-0.7 _DSF3087.RAF 7.52 -> 6.78 +1.6,+0.1, 0.0 -> 0.5,-0.2,-0.7 DSCF1701.RAF 6.86 -> 6.54 ~0 -> -1.5,-1.7,-0.9 _DSF7273.RAF 5.13 -> 4.66 +2.4,+1.0,+1.5 -> 1.8,+0.9,+1.5 _3279791.ORF 5.81 -> 5.93 ~0 -> -0.4,-0.2,-0.5 P1010256.RW2 9.60 -> 9.56 +2.5,+0.6,+0.7 -> 1.6,+0.4,-0.1 The four Fujifilm frames lose the cast; the Olympus and Lumix frames hold. The two that open as their preview (Nikon NEF, DNG) still do. The bright neutral fifth of both Fujifilm frames now differs from the preview by the same figure on all three channels — a difference in brightness, not in colour, which is the tone curve's business and not this one's. The read is on the frame, not on the brand or its tags, so the bodies with no sample here (Sony, Canon, Ricoh, Leica) take the same path. scripts/raw-develop-check.mjs had gone stale against the module and is back in step with it — the highlight gate's threshold, the gain's divisor, the preview-WB read and the tag order are pinned, and previewWhiteBalance now carries a case that hands it a plane and a map and reads the balance back.
2026-10-10 23:38:57 +07:00
parent 8b27f9542d
commit 0ac9f2ddfa
2 changed files with 190 additions and 20 deletions
@@ -36,9 +36,12 @@ assert.match(code, /float3 lin = n \* mul\.rgb;/);
 // across the whole sky. Measured on that frame in the app against the camera's own
 // JPEG: mean dRGB +1.2, -5.9, -6.1 with the sensor's clip alone, +0.8, +1.0, +1.1
 // with both, mean |dL| 21.5 against 11.1.
+// The gate opens below the white level, not at it: 0.82 against 0.98 was measured
+// after the pairs above, so the tint reads on the way up rather than only once the
+// frame has already run out of range.
 assert.match(
  code,
-  /float hi = max\(max\(n\.r, n\.g\), n\.b\);\s*hi = max\(hi, max\(max\(lin\.r, lin\.g\), lin\.b\)\);\s*float blown = smoothstep\(0\.99, 1\.02, hi\);\s*rgb = mix\(rgb, float3\(mx\), blown\);/
+  /float hi = max\(max\(n\.r, n\.g\), n\.b\);\s*hi = max\(hi, max\(max\(lin\.r, lin\.g\), lin\.b\)\);\s*float blown = smoothstep\(0\.82, 0\.98, hi\);\s*rgb = mix\(rgb, float3\(mx\), blown\);/
 );

 // The inverse is built on the CPU side and has to be the first uniform of the
@@ -46,7 +49,8 @@ assert.match(
 // the frame itself ran out at — `maximum` alone left every frame a stop bright,
 // and a fixed factor two only fitted the two bodies it was measured on. The
 // probe and its own check live in src/engine/sensorWhite.ts.
-assert.match(src, /^ *uniforms\[0\] = SAMPLE_MAX \/ sensorWhite\(data, cd\.maximum, cd\.black\);$/m);
+assert.match(src, /^ *const white = sensorWhite\(data, cd\.maximum, cd\.black\);$/m);
+assert.match(src, /^ *uniforms\[0\] = SAMPLE_MAX \/ white;$/m);
 assert.match(src, /^import \{ sensorWhite \} from '\.\/sensorWhite';$/m);

 // On the FX30 that inverse is 2.065 (65535 / (2 (16380 - 512))), against the
@@ -74,9 +78,33 @@ assert.doesNotMatch(code, /uniform float4 w0;/, 'the body table is back in the s
 assert.doesNotMatch(src, /fitMatch/, 'the 3x3 preview match is back');
 assert.doesNotMatch(code, /uniform float4 f0;/, 'the 3x3 preview match is back in the shader');
 assert.match(src, /const first = develop\(null\);/);
-assert.match(src, /const jpeg = first\.encodeToBytes/);
+assert.match(src, /const jpeg = finalShot\.encodeToBytes\(Skia\.ImageFormat\.JPEG, 92\);/);
 assert.match(src, /preview && preview\.length > 10000/);

+// The balance the frame opens with is read off the file's own preview — the same
+// reference the tone curve is fitted to — and only falls back to LibRaw's tags when
+// the preview has nothing neutral and bright left to read. The tags alone were not
+// the balance: on the Fujifilm RAF this was reported on, `pre_mul` was short of blue
+// twice over (R/G 2.296 against the frame's 1.795, B/G 1.307 against 1.626) and left
+// every white pink, `cam_mul` came closer, and the pair read off the preview matched
+// it: block a* -1.9 against the preview's -2.1 where the tags gave 0.0 and -1.0,
+// dE00 4.93 where they gave 6.73 and 5.27. And it holds across bodies for the same
+// reason — it is read on the frame, not on the tags.
+assert.ok(
+  src.indexOf('return cd.cam_mul;') < src.indexOf('return cd.pre_mul;'),
+  'pre_mul is preferred over the frame\'s own balance again'
+);
+assert.match(src, /^ *const refGrid = preview \? previewGrid\(preview, w, h, MATCH_GRID\) : null;$/m);
+assert.match(
+  src,
+  /const rawMul = \(refGrid && previewWhiteBalance\(data, w, h, refGrid, MATCH_GRID, white\)\)\s*\?\? getBrandWbMultipliers\(brand, cd\);/
+);
+assert.match(src, /^export function previewWhiteBalance\($/m);
+// ...and a balance read off too little of the frame is no balance: a frame that
+// offers fewer than this many sensor pixels across its neutral bright blocks keeps
+// the tags rather than a ratio read off noise.
+assert.match(src, /if \(count < 4096 \|\| sums\[0\] <= 0 \|\| sums\[1\] <= 0 \|\| sums\[2\] <= 0\) return null;/);
+
 // The fit has to land on the encoded value in float, not as an 8-bit colour filter
 // painted over the frame afterwards: the fit carries an exposure, and a channel it
 // lifted past the white level was cut where it stood — blue first, the channel the
@@ -127,11 +155,16 @@ assert.match(src, /^ *halfSize: false,$/m, 'the develop is half-size again');
 // the cast the frame opened with. And the matrix must not be divided by `pre_mul`
 // again: that undoes the row-sum-to-one normalisation dcraw built into rgb_cam.
 {
-  const body = src.match(/const XYZ_TO_SRGB = \[[\s\S]*?\n\}\n/)?.[0];
-  assert.ok(body, 'getCamToSrgbMatrix not found');
-  // The file is TypeScript; the two annotations in this block are all that stop
+  const xyz = src.match(/^const XYZ_TO_SRGB = \[[\s\S]*?\n\];$/m)?.[0];
+  const fn = src.match(/^export function getCamToSrgbMatrix\([\s\S]*?\n\}\n/m)?.[0];
+  assert.ok(xyz && fn, 'getCamToSrgbMatrix not found');
+  // The file is TypeScript; the annotations in this block are all that stop
  // `new Function` from reading it.
-  const js = body.replace(/: any/g, '').replace(/: number\[\]\[\]/g, '');
+  const js = (xyz + '\n' + fn)
+    .replace(/^export /m, '')
+    .replace(/: any/g, '')
+    .replace(/: number\[\]\[\]/g, '')
+    .replace(/\?: CameraBrand/g, '');
  const getCamToSrgbMatrix = new Function(`${js}; return getCamToSrgbMatrix;`)();
  const rgb_cam = [
    [1.631760597229004, -0.39896339178085327, -0.23279713094234467],
@@ -163,4 +196,66 @@ assert.match(src, /^ *halfSize: false,$/m, 'the develop is half-size again');
  }
 }

+// The balance read off the preview is arithmetic on the plane and the preview
+// map, so both can be handed to it here. A plane whose red stands 1.5x its green
+// and whose blue 1.2x, under a map the camera left bright and neutral, has one
+// balance that would equalise those blocks, and it is the one that comes back.
+{
+  const fn = src.match(/^export function previewWhiteBalance\([\s\S]*?\n\}\n/m)?.[0];
+  assert.ok(fn, 'previewWhiteBalance not found');
+  const previewWhiteBalance = new Function(
+    `${fn.replace(/^export /m, '').replace(/:\s*(ArrayLike<number>|Uint8Array|number\[\] \| null|number)/g, '')}; return previewWhiteBalance;`
+  )();
+
+  const N = 4;
+  const W = 128;
+  const H = 128;
+  const WHITE = 10000;
+  const plane = (r, g, b) => {
+    const d = new Uint16Array(W * H * 3);
+    for (let i = 0; i < W * H; i++) {
+      d[i * 3] = r;
+      d[i * 3 + 1] = g;
+      d[i * 3 + 2] = b;
+    }
+    return d;
+  };
+  const block = (g, bx, by, r, gg, b) => {
+    const i = (by * N + bx) * 4;
+    g[i] = r;
+    g[i + 1] = gg;
+    g[i + 2] = b;
+    g[i + 3] = 255;
+  };
+  const map = (f) => {
+    const g = new Uint8Array(N * N * 4);
+    for (let by = 0; by < N; by++) for (let bx = 0; bx < N; bx++) f(g, bx, by);
+    return g;
+  };
+  const allWhite = map((g, bx, by) => block(g, bx, by, 200, 200, 200));
+
+  const mul = previewWhiteBalance(plane(3000, 2000, 2400), W, H, allWhite, N, WHITE);
+  assert.ok(mul, 'a frame of bright neutral blocks has to read a balance');
+  assert.ok(
+    Math.abs(mul[0] - 2 / 3) < 1e-6 && mul[1] === 1 && Math.abs(mul[2] - 5 / 6) < 1e-6,
+    `read ${mul} off a 1.5x red / 1.2x blue plane`
+  );
+
+  // A block the camera did not leave neutral is not read, however bright it is:
+  // it carries the scene's colour, not the frame's balance.
+  const tinted = previewWhiteBalance(
+    plane(3000, 2000, 2400),
+    W,
+    H,
+    map((g, bx, by) => (bx ? block(g, bx, by, 200, 200, 200) : block(g, bx, by, 200, 120, 200))),
+    N,
+    WHITE
+  );
+  assert.deepEqual(tinted, mul, 'a tinted block moved the balance');
+
+  // Too little of the frame left to read is no balance at all — the caller keeps
+  // the tags rather than a ratio read off a handful of pixels.
+  assert.equal(previewWhiteBalance(plane(3000, 2000, 2400), 30, 30, allWhite, N, WHITE), null);
+}
+
 console.log('raw-develop-check ok');