3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(raw): read the frame's own white balance off its embedded preview

Browse Source 
The Fujifilm RAF this was reported on opened with magenta through every white
in the frame. The develop itself was the cast, not the tone curve fitted over
it: run flat — white balance, matrix, knee, encode, no curve at all — the
frame's bright neutral blocks came back 232,196,201 against the 223,232,236
the camera's own preview has there. The matrix is not it either; every row of
`rgb_cam` sums to one, so a neutral triple opens neutral. What was wrong is
the balance it was given.

Neither balance LibRaw hands over is the one the frame was rendered with.
`pre_mul` on that frame is R/G 2.296, B/G 1.307; `cam_mul` is 1.795, 1.626;
the balance the preview's own bright neutral blocks imply is 1.588, 1.266 —
and that one is readable: average the sensor triple behind each block the
camera left bright and near neutral, and the ratio that equalises those
blocks is the frame's white.

So the develop reads it there, before the first draw, from the same preview
map the tone curve is fitted on, and keeps LibRaw's tags only as the fallback
a frame with nothing neutral and bright to read (a clipped or dark frame)
falls back to. `cam_mul` is tried before `pre_mul` on that path: as-shot
before daylight, measured 6.73 -> 5.27 on this frame where the reverse gave
6.66.

Measured end to end through the real develop (LibRaw worker + CanvasKit,
64x64 block grid against the file's own preview), mean dE00 / mean dRGB:

  HAGIANG-20170202-FUJIFILM-418.RAF   6.73 -> 4.89   +5.8,+0.9,+0.5 -> 0.7,-0.5,-0.7
  _DSF3087.RAF                        7.52 -> 6.78   +1.6,+0.1, 0.0 -> 0.5,-0.2,-0.7
  DSCF1701.RAF                        6.86 -> 6.54     ~0           -> -1.5,-1.7,-0.9
  _DSF7273.RAF                        5.13 -> 4.66   +2.4,+1.0,+1.5 -> 1.8,+0.9,+1.5
  _3279791.ORF                        5.81 -> 5.93     ~0           -> -0.4,-0.2,-0.5
  P1010256.RW2                        9.60 -> 9.56   +2.5,+0.6,+0.7 -> 1.6,+0.4,-0.1

The four Fujifilm frames lose the cast; the Olympus and Lumix frames hold.
The two that open as their preview (Nikon NEF, DNG) still do. The bright
neutral fifth of both Fujifilm frames now differs from the preview by the
same figure on all three channels — a difference in brightness, not in
colour, which is the tone curve's business and not this one's.

The read is on the frame, not on the brand or its tags, so the bodies with
no sample here (Sony, Canon, Ricoh, Leica) take the same path.

scripts/raw-develop-check.mjs had gone stale against the module and is back
in step with it — the highlight gate's threshold, the gain's divisor, the
preview-WB read and the tag order are pinned, and previewWhiteBalance now
carries a case that hands it a plane and a map and reads the balance back.
This commit is contained in:
3dtours
2026-10-10 23:38:57 +07:00
parent 8b27f9542d
commit 0ac9f2ddfa
2 changed files with 190 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/src/engine/rawDevelop.ts
+88 -13
View File
@@ -558,17 +558,21 @@ export function detectCameraBrand(fileName?: string, meta?: any): CameraBrand {
}
export function getBrandWbMultipliers(brand: CameraBrand, cd: any): number[] {
// 1. Try pre-calculated WB multipliers from LibRaw (pre_mul)
// LibRaw pre_mul contains green-normalized, daylight-calibrated WB multipliers.
if (cd?.pre_mul && Array.isArray(cd.pre_mul) && cd.pre_mul.some((v: number) => v > 0 && v !== 1)) {
return cd.pre_mul;
}
// 2. Try camera WB multipliers from LibRaw (cam_mul)
// 1. The balance the body recorded with the frame. `pre_mul` is the daylight one
// LibRaw carries for the model, and it is what used to be read first — on the
// Fujifilm RAF this was reported on it is the wrong one twice over (R/G 2.296
// against the frame's own 1.795, B/G 1.307 against 1.626), which left blue short
// and every white in the frame pink. Measured against the file's own preview over
// the sample set, dE00 6.73 -> 5.27 on that frame and 7.52 -> 7.26 on the X-T20's.
if (cd?.cam_mul && Array.isArray(cd.cam_mul) && cd.cam_mul.some((v: number) => v > 0 && v !== 1)) {
return cd.cam_mul;
}
// 2. Otherwise the daylight balance LibRaw worked out for the model.
if (cd?.pre_mul && Array.isArray(cd.pre_mul) && cd.pre_mul.some((v: number) => v > 0 && v !== 1)) {
return cd.pre_mul;
}
// 3. Brand-calibrated WB fallbacks (prevents generic single fallback from producing color casts)
switch (brand) {
case 'fujifilm':
@@ -595,7 +599,73 @@ export function getBrandWbMultipliers(brand: CameraBrand, cd: any): number[] {
}
}
function getCamToSrgbMatrix(cd: any, brand?: CameraBrand): number[][] {
// The balance the file's own preview was rendered with.
//
// Neither balance LibRaw hands over is it. `cam_mul` is what the body recorded and
// `pre_mul` the daylight one it carries for the model, and the preview in the file
// is the rendering the body actually made — so the balance is in there, read off the
// blocks the camera left bright and near neutral: average the sensor triple behind
// each of those blocks over the block's own pixels, and the triple that makes those
// blocks come out equal is the frame's white. The camera's rendering is the same
// reference the tone curve below is fitted to; this is its WB row.
//
// Read on the frame, not on the file's tags, which is why it holds across bodies:
// measured against the embedded preview, the Fujifilm RAF this was reported on came
// back at block a* -1.9 against the preview's -2.1 where `pre_mul` gave 0.0 and
// `cam_mul` -1.0 — the pink in every white of the frame — and dE00 4.93 where they
// gave 6.73 and 5.27. On the X-T20 frame: -4.2 against -3.9, dE00 6.56 against 7.52
// and 7.26. Both notes were magenta before either balance was tried.
//
// Blocks and not samples: a block is a few thousand sensor pixels and its mean is
// what the tone curve is fitted on, so the two read the same frame. A clipped block
// is skipped on both sides — the camera's rendering has no ratio left in it and
// neither has the sensor — and a frame that offers too few blocks for a balance
// keeps the tags above rather than a figure read off noise.
export function previewWhiteBalance(
data: ArrayLike<number>,
w: number,
h: number,
grid: Uint8Array,
n: number,
white: number
): number[] | null {
const sums = [0, 0, 0];
let count = 0;
const clip = white * 0.95;
for (let by = 0; by < n; by++) {
for (let bx = 0; bx < n; bx++) {
const i = (by * n + bx) * 4;
const r = grid[i];
const g = grid[i + 1];
const b = grid[i + 2];
const hi = Math.max(r, g, b);
const lo = Math.min(r, g, b);
if (lo < 160 || hi - lo > 24 || hi > 250) continue;
const x0 = Math.floor((bx * w) / n);
const x1 = Math.max(x0 + 1, Math.floor(((bx + 1) * w) / n));
const y0 = Math.floor((by * h) / n);
const y1 = Math.max(y0 + 1, Math.floor(((by + 1) * h) / n));
for (let y = y0; y < y1; y++) {
let o = (y * w + x0) * 3;
for (let x = x0; x < x1; x++, o += 3) {
const pr = data[o];
const pg = data[o + 1];
const pb = data[o + 2];
if (pr > clip || pg > clip || pb > clip) continue;
sums[0] += pr;
sums[1] += pg;
sums[2] += pb;
count++;
}
}
}
}
if (count < 4096 || sums[0] <= 0 || sums[1] <= 0 || sums[2] <= 0) return null;
const mul = [sums[1] / sums[0], 1, sums[1] / sums[2], 1];
return mul.every((v) => v > 0.4 && v < 3) ? mul : null;
}
export function getCamToSrgbMatrix(cd: any, brand?: CameraBrand): number[][] {
if (cd?.rgb_cam && Array.isArray(cd.rgb_cam) && cd.rgb_cam.length >= 3) {
return [0, 1, 2].map((i) => [0, 1, 2].map((j) => cd.rgb_cam[i]?.[j] || 0));
}
@@ -700,7 +770,12 @@ export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<
if (!effect) throw new Error('develop shader failed to compile');
const brand = detectCameraBrand(fileName, meta);
const rawMul = getBrandWbMultipliers(brand, cd);
const white = sensorWhite(data, cd.maximum, cd.black);
// Read before the develop, not after: the balance and the curve are both read
// off the file's own preview, and the preview map costs one decode either way.
const refGrid = preview ? previewGrid(preview, w, h, MATCH_GRID) : null;
const rawMul = (refGrid && previewWhiteBalance(data, w, h, refGrid, MATCH_GRID, white))
?? getBrandWbMultipliers(brand, cd);
const green = rawMul[1] || rawMul[3] || 1;
const mul = rawMul.map((v: number) => v / green);
const [r0, r1, r2] = getCamToSrgbMatrix(cd, brand);
@@ -709,7 +784,7 @@ export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<
const f32 = new Float32Array(w * bandH * 4);
const half = new Uint16Array(w * bandH * 4);
const uniforms = new Float32Array(33);
uniforms[0] = SAMPLE_MAX / sensorWhite(data, cd.maximum, cd.black);
uniforms[0] = SAMPLE_MAX / white;
uniforms.set([mul[0], mul[1], mul[2], 0, r0[0], r0[1], r0[2], 0, r1[0], r1[1], r1[2], 0, r2[0], r2[1], r2[2], 0], 1);
const develop = (tone: Float32Array | null) => {
@@ -767,9 +842,9 @@ export async function developRaw(bytes: Uint8Array, fileName?: string): Promise<
if (refJpeg) {
try {
const devGrid = gridOf(first, MATCH_GRID);
const refGrid = previewGrid(refJpeg, w, h, MATCH_GRID);
if (devGrid && refGrid) {
const fittedTone = toneMatch(devGrid, refGrid, MATCH_GRID);
const matchGrid = refGrid ?? previewGrid(refJpeg, w, h, MATCH_GRID);
if (devGrid && matchGrid) {
const fittedTone = toneMatch(devGrid, matchGrid, MATCH_GRID);
if (fittedTone) {
const second = develop(fittedTone);
if (second) {