web: shoot through the live camera
The one path where the look is chosen before the picture exists: OPEN CAMERA grades the camera's own feed with the recipe in force, many times a second, and the shutter hands the studio the sensor's still under that same recipe. Preview and file differ in resolution only — the still is `takePhoto`'s own frame, not a copy of the small preview video, with `grabFrame` and a 2d copy of the element behind it for the browsers that ship no ImageCapture. The renderer gains two inputs for it: `sourceImage`, a picture the caller already decoded (re-encoding the camera's frame to JPEG only to decode it again would cost more than the whole render), and `drawTo`, which paints the finished picture instead of encoding it. One render is in flight at a time; a frame that arrives during one is dropped, so a slow device shows a lower frame rate rather than a queue of moments that have passed. The view flashed black on a phone. Setting width/height on a canvas resets its bitmap: measured on the preview, a resize leaves mean 0 until the next render lands, which on this box is 0.5s and on a phone more. The buffer was sized from every incoming frame, and a capture that renegotiates its resolution — which Chromium does when the page is too slow to consume its frames, and this pipeline runs ~2 fps at 720p under software GL — strobed black/picture at every switch. The buffer is now sized on the first frame and after that only when the frame's aspect changes: a same-aspect frame is scaled into it. Swapping a 1280x720 stream for a 640x360 one mid-view now leaves the buffer at 1280x720 with no black frame, and 640x360 renders at 6-13 fps instead of 2. The frames are read from a <video>, which is now IN the document (1px, behind the black backdrop) rather than detached: Safari draws blank frames from a detached video, which is the same black-between-pictures. It leaves the document with the view, and the tracks are stopped, so the camera light goes out. Probes: cam-smoke (feed painted, resolution, frame rate, a monochrome sim reaching the live frames, shutter into the studio, close, console clean), cam-renegotiate (no resize, no blank frame, status line on the frames), cam-close-flip (flip returns a picture; video gone on close).
This commit is contained in:
@@ -96,13 +96,22 @@ export interface RenderOptions {
|
||||
}
|
||||
|
||||
export interface RenderInput {
|
||||
source: Uint8Array;
|
||||
source?: Uint8Array;
|
||||
recipe: Recipe;
|
||||
frameId: FrameId;
|
||||
useGeotag?: boolean;
|
||||
gpsInfo?: GPSInfo | null;
|
||||
options?: RenderOptions;
|
||||
assets: RenderAssets;
|
||||
// A picture the caller already decoded, instead of `source`'s bytes. The live
|
||||
// camera has a new one every frame: re-encoding it to JPEG only for this to
|
||||
// decode it again would cost more than the whole render.
|
||||
sourceImage?: any;
|
||||
// Paint the finished picture instead of encoding it, and hand it over here.
|
||||
// The live view blits it and lets it go; the bytes of a JPEG would be thrown
|
||||
// away thirty times a second. Called synchronously, with an image that is
|
||||
// already dead once this returns.
|
||||
drawTo?: (image: any) => void;
|
||||
}
|
||||
|
||||
export interface RenderResult {
|
||||
@@ -370,7 +379,7 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
|
||||
return obj as T;
|
||||
};
|
||||
try {
|
||||
const skImage0 = Skia.Image.MakeImageFromEncoded(input.source);
|
||||
const skImage0 = input.sourceImage ?? (input.source ? Skia.Image.MakeImageFromEncoded(input.source) : null);
|
||||
if (!skImage0) {
|
||||
console.error('Failed to parse source image');
|
||||
return null;
|
||||
@@ -1056,6 +1065,13 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
|
||||
}
|
||||
}
|
||||
|
||||
// 10. The live view stops here: it wants the picture, not a file, and it
|
||||
// says so with `drawTo` — no encode, no DPI patch, no bytes.
|
||||
if (input.drawTo) {
|
||||
input.drawTo(resultImage);
|
||||
return { bytes: new Uint8Array(), width: resultImage.width(), height: resultImage.height() };
|
||||
}
|
||||
|
||||
let bytes = resultImage.encodeToBytes(Skia.ImageFormat.JPEG, 95);
|
||||
if (!bytes || bytes.length === 0) {
|
||||
console.error('Failed to encode image to JPEG');
|
||||
|
||||
Reference in New Issue
Block a user