3dtours/RecipesCam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(raw): keep the ORF off olive by dropping the 3x3 preview match

Browse Source 
The least-squares 3x3 that mapped the develop onto the camera's JPEG won
on luminance, whose variance is ~80x the chroma's, so it paid for its
match by crushing the R-G axis: 11.6 -> 4.6 standard deviations, against
8.3 in the camera's own preview and 12.3 in its JPEG. That is the olive
cast -- the sky and the rice both shifted yellow-green.

The colour chain was never at fault: camera_mul x rgb_cam already agrees
with LibRaw to the digit, and the app's fit is bit-identical to a numpy
reference of the same problem, which is how the 3x3 was cleared.

Fit only the per-channel tone curve now. R-G comes back to 8.6 (900px)
and 10.5 (full), and B/G lands on the preview's 0.841 to three places.
This commit is contained in:
3dtours
2026-10-06 08:29:07 +07:00
parent 7363bd9099
commit 9aa9bc0db0
2 changed files with 106 additions and 185 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/frontend/src/engine/rawDevelop.ts
+54 -71
View File
@@ -5,22 +5,22 @@
// balance and tone (below) and knows nothing of the body's picture style, so the
// develop is fitted to the one rendering the file does carry with it: the preview
// the camera wrote inside it, which is the frame the photographer saw and the one
// a desktop viewer shows. Develop the sensor, least-squares a 3x3 from a block
// grid of each onto the other on the encoded values, fit what that 3x3 leaves a
// cubic a channel, and develop the sensor through both (see previewMatch.ts).
// a desktop viewer shows. Develop the sensor, take a block grid out of each on
// the encoded values, fit the cubic a channel between them, and develop the
// sensor through that curve (see previewMatch.ts).
//
// Measured against the camera's own 24MP JPEG of the A5100 frame this was
// reported on, blocks of a 64x64 grid, dE00: 12.0 as the develop left it, 19.7
// with no white balance at all, 6.6 matched to the file's own preview by the 3x3
// alone and 4.4 through the 3x3 and the curve together. What neither reaches is
// the camera's own sharpening, which no per-pixel transform holds.
// The curve is the whole fit. A 3x3 was least squared ahead of it for a long
// time and is what put the olive cast on the Olympus frame this was reported on:
// a develop keeps its luminance and its colour in the same three numbers and the
// luminance is two orders of magnitude the larger, so a least squares dominated
// by it holds the channel means and buys them with the colour axis. The camera's
// picture style is a curve, not a matrix.
//
// The fit is the only profile: a body-used-to-be table of six matrices fitted on
// this develop was dropped, because a table fitted on one path stops matching the
// moment the path changes under it — its rows no longer summed to 1 once the
// highlight knee landed ahead of it, and every body that had one opened with a
// cast the body that had none did not. Offline-fitted matrices age; the file's
// own preview does not.
// Measured on that frame, standard deviation of R-G over it (the file's own
// preview 8.3, the camera's own JPEG 12.3, an olive frame being one where this
// has gone to nothing): 11.6 as the develop left it, 11.9 through the curve
// alone, 4.6 through the curve under the fitted 3x3. What no per-pixel transform
// reaches either way is the camera's own sharpening.
//
// Developing the sensor is also what keeps the highlights: 2.3% of the frame at
// pure white against 3.3% in the preview itself, since the 8-bit preview threw
@@ -53,7 +53,7 @@
import LibRaw from 'libraw-wasm';
import { sensorWhite } from './sensorWhite';
import { f32ToF16 } from './halfFloat';
import { fitMatch, FLAT_TONE, MATCH_GRID, type Match, type Mat3 } from './previewMatch';
import { toneMatch, FLAT_TONE, MATCH_GRID } from './previewMatch';
import { Skia } from './skiaShim';
// What `imageData()` returns for the settings below: 16-bit, 3 channels, black
@@ -85,8 +85,6 @@ const SAMPLE_MAX = 65535;
// Pixels per band, held under the point where the Float32 copy dominates the
// memory the page is allowed to use.
const BAND_PIXELS = 2_000_000;
// Leaves the develop alone when the body has no fitted profile.
const IDENTITY: Mat3 = [1, 0, 0, 0, 1, 0, 0, 0, 1];
const RAW_DEVELOP_SKSL = `
uniform shader raw;
@@ -96,15 +94,12 @@ uniform float4 m0; // camera -> sRGB, the first three columns of rgb_cam
uniform float4 m1;
uniform float4 m2;
uniform float4 crop; // (y offset of this band, 0, 0, 0)
uniform float4 f0; // the per-file fit, on the encoded value (identity when
uniform float4 f1; // the frame carries no preview to be fitted to)
uniform float4 f2;
uniform float4 t0; // what that fit then leaves a channel: one cubic a
uniform float4 t1; // channel, r a b c d per float4 (flat when there is no fit)
uniform float4 t2;
uniform float4 t0; // the file's own rendering, one cubic a channel on the
uniform float4 t1; // encoded value: r a b c d per float4 (flat when there is
uniform float4 t2; // no preview to fit to)
// A 3x3 can only scale a channel; the gap to the camera is mostly a shape, and
// this is that shape, read on the value the 3x3 left.
// The camera's rendering is a shape, and this is that shape, read on the value
// the develop left.
float tone(float4 w, float x) {
return clamp(w.x + x * (w.y + x * (w.z + x * w.w)), 0.0, 1.0);
}
@@ -142,7 +137,7 @@ half4 main(float2 pos) {
// (227,184,245 at the gate's own value against 245,245,245 read where the gains
// put the clip, the camera's preview white at that block). Both are read, so a
// body whose gains do not lift a channel keeps the sensor's own clip as its gate.
// The matrix was fitted luma-preserving, so mx is the value to hold.
// mx is the value to hold: it is the pixel's own lightness.
float hi = max(max(n.r, n.g), n.b);
hi = max(hi, max(max(lin.r, lin.g), lin.b));
rgb = mix(rgb, float3(mx), smoothstep(0.99, 1.02, hi));
@@ -159,7 +154,7 @@ half4 main(float2 pos) {
// spends 0.85..0.94. Below T the frame is untouched and the curve leaves T with
// the slope it arrived with (1), so there is no seam to mask; above it the frame
// darkens, which is the one move no later pass can undo — which is the point,
// and what the per-file fit below then measures the REST of the frame back from.
// and what the per-file curve below then measures the REST of the frame back from.
//
// ponytail: 4.7 stops of headroom now share ~5% of the ramp, and the develop
// still leaves as an 8-bit JPEG. Give it a float16 output when RAW highlights
@@ -169,24 +164,16 @@ half4 main(float2 pos) {
rgb *= (0.7 + over / (1.0 + over * 3.3333)) / mx;
}
float3 e = encode(rgb);
// The file's own colour: the fit, in float, on the encoded value it was fitted
// The file's own colour: the curve, in float, on the encoded value it was fitted
// on — and the last step the frame leaves through. Not the 8-bit colour filter
// this used to be painted through: the fit carries an exposure (the preview is
// the reference, so matching its brightness is part of matching its colour).
// No rolloff here either. The fit is fitted whole to the preview's own values,
// and white is one of them, so it already maps the frame's white to the preview's
// — while a divide by the row max is a white-preserving move the fit does not
// need: every highlight came back at ~0.74 / 1.0 / 0.86, cyan, and not one cell
// of the frame reached white on all three channels (0.0% against the preview's
// 3.3%). Past the white level only the clamp is left, exactly as the develop
// above lets its own overflow run.
float3 q = clamp(float3(dot(f0.xyz, e), dot(f1.xyz, e), dot(f2.xyz, e)), 0.0, 1.0);
// ...and then the curve, one cubic a channel, which is what carries the body's
// own tone: a 3x3 can only scale, so the frame without it came back bright and
// green in the shadows (dL +13.8 and green +0.128 at the bottom of the range,
// dE00 6.6) instead of matching (dL +4.5, green +0.017, 4.4). The curve is
// pinned at white, so a blown pixel still lands on white.
return half4(half3(tone(t0, q.r), tone(t1, q.g), tone(t2, q.b)), 1.0);
// this used to be painted through: the curve carries an exposure (the preview is
// the reference, so matching its brightness is part of matching its colour), and
// it is what carries the body's own tone — a 3x3 can only scale, and the one that
// used to sit here bought the channel means by collapsing the colour axis it was
// fitted against (see previewMatch.ts). No rolloff: white is one of the values
// the curve is fitted on, and it is pinned there, so a blown pixel still lands on
// white while the frame's own highlights stay where the knee above left them.
return half4(half3(tone(t0, e.r), tone(t1, e.g), tone(t2, e.b)), 1.0);
}
`;
@@ -204,11 +191,12 @@ export function isRawName(name: string): boolean {
return name.includes('.') && RAW_EXT.includes(ext);
}
// The camera's own preview, when the file carries one: the colour reference to
// fit against, and what the file opens as when the sensor does not decode.
// The camera's own preview, when the file carries one: the colour reference the
// curve below is fitted to, and what the file opens as when the sensor does not
// decode.
// ponytail: it is 1616x1080 on an A5100 and 1620x1080 on an FX30, so handing it
// back is a 1.7MP frame — a print past it has to come off the develop, which is
// what a fitted file already gives.
// what a developed file already gives.
async function cameraPreview(raw: LibRaw): Promise<Uint8Array | null> {
const thumb = await raw.thumbnailData().catch(() => undefined);
if (thumb?.format !== 'jpeg' || !thumb.data?.length) return null;
@@ -354,7 +342,7 @@ function gridOf(image: any, n = MATCH_GRID): Uint8Array | null {
if (!surface) return null;
const canvas = surface.getCanvas();
// Cubic, not a linear tap: this is a 45x reduction and linear reads a handful
// of source pixels per block — noise for the least squares to fit.
// of source pixels per block — noise for the curve to be fitted on.
canvas.drawImageRectCubic(
image,
Skia.XYWHRect(0, 0, image.width(), image.height()),
@@ -376,11 +364,11 @@ function gridOf(image: any, n = MATCH_GRID): Uint8Array | null {
// The same grid out of the preview, which the file carries as a JPEG. Decoded and
// reduced through the same Skia call as the develop's own grid, because the two
// grids are only comparable — and the fit only meaningful — when one resampler
// made both. A 2D canvas here instead left the fit following its own smoothing:
// on the A5100 frame the same develop scored dE00 4.7 against 4.4, and the dark
// end of the frame came out 6 L further from the preview than the fit it was
// handed asked for.
// grids are only comparable — and a curve between them only meaningful — when one
// resampler made both. A 2D canvas here instead left the curve following its own
// smoothing: on the A5100 frame the same develop scored dE00 4.7 against 4.4, and
// the dark end of the frame came out 6 L further from the preview than the curve
// it was handed asked for.
function previewGrid(jpeg: Uint8Array, w: number, h: number, n = MATCH_GRID): Uint8Array | null {
const bmp = Skia.Image.MakeImageFromEncoded(jpeg);
if (!bmp) return null;
@@ -503,26 +491,21 @@ export async function developRaw(bytes: Uint8Array): Promise<Uint8Array> {
// The band goes up as half, not float32: the GPU backend puts an F32 image
// on the 1/255 grid and the shadows quantise to black (see halfFloat.ts).
const half = new Uint16Array(w * bandH * 4);
// Every uniform but the crop and the fit, which are the two the passes change:
// the shader's own order is gain, mul, rgb_cam, crop, the fit, then the fit's
// tone curve.
const uniforms = new Float32Array(45);
// Every uniform but the crop and the curve, which are the two the passes change:
// the shader's own order is gain, mul, rgb_cam, crop, then the curve.
const uniforms = new Float32Array(33);
uniforms[0] = SAMPLE_MAX / sensorWhite(data, cd.maximum, cd.black);
uniforms.set([mul[0], mul[1], mul[2], 0, r0[0], r0[1], r0[2], 0, r1[0], r1[1], r1[2], 0, r2[0], r2[1], r2[2], 0], 1);
// One develop of the frame, band by band, through `fit` when there is one. A
// function because the frame is drawn twice: once on the sensor alone, to fit
// against the preview, and then again with the fit in the shader.
// One develop of the frame, band by band, through `tone` when the file carried a
// preview to fit a curve to. A function because the frame is drawn twice: once
// as the sensor left it, to fit that curve against, and then again through it.
// ponytail: two full band passes, on the main thread. Give develop an F16
// intermediate (one develop, one colour pass) if the second pass ever shows.
const develop = (fit: Match | null) => {
const develop = (tone: Float32Array | null) => {
const surface = Skia.Surface.MakeOffscreen(w, h) ?? Skia.Surface.Make(w, h);
if (!surface) return null;
const f = fit?.m ?? IDENTITY;
uniforms[21] = f[0]; uniforms[22] = f[1]; uniforms[23] = f[2]; uniforms[24] = 0;
uniforms[25] = f[3]; uniforms[26] = f[4]; uniforms[27] = f[5]; uniforms[28] = 0;
uniforms[29] = f[6]; uniforms[30] = f[7]; uniforms[31] = f[8]; uniforms[32] = 0;
uniforms.set(fit?.tone ?? FLAT_TONE, 33);
uniforms.set(tone ?? FLAT_TONE, 21);
for (let y0 = 0; y0 < h; y0 += bandH) {
const rows = Math.min(bandH, h - y0);
let o = 0;
@@ -563,15 +546,15 @@ export async function developRaw(bytes: Uint8Array): Promise<Uint8Array> {
};
// The file's own colour: a grid of the develop as it stands, the same grid out
// of the preview the camera wrote into the file, and the 3x3 and the curve
// between them — which the second develop then draws in the shader. No
// preview, no fit: the frame opens as the sensor left it.
// of the preview the camera wrote into the file, and the curve between them —
// which the second develop then draws in the shader. No preview, no curve: the
// frame opens as the sensor left it.
const first = develop(null);
if (!first) throw new Error('no surface for the develop');
const blocks = preview ? gridOf(first) : null;
const ref = blocks ? previewGrid(preview as Uint8Array, first.width(), first.height()) : null;
const match = blocks && ref ? fitMatch(blocks, ref) : null;
const matched = match ? develop(match) : null;
const tone = blocks && ref ? toneMatch(blocks, ref) : null;
const matched = tone ? develop(tone) : null;
const jpeg = (matched ?? first).encodeToBytes(Skia.ImageFormat.JPEG, 92);
(matched ?? first).dispose();
if (matched) first.dispose();