fix(studio): the repair runs off the page's own thread, and a stroke is answered twice so the hand is never held
The graph is seconds of solid arithmetic however large the crop is, and it ran where the pointer lives: the brush lagged behind the hand for exactly as long as the model took. The runtime's wasm provider now works in a worker of its own, so the page keeps drawing while the repair is thought through. The wait itself stays — it is the model's, not the page's — so a preview paints the blurred fill first and lets the model's version of the same mask replace it. Filing and export still wait for the model: the faster answer is for the eye waiting on the brush and for nothing else. Measured on the same photo and stroke, worst main-thread gap 3703 ms before and 487 ms after; previews land at 1.0 s and 5.3 s, and the model's render is the one left on the stage.
This commit is contained in:
@@ -1032,6 +1032,30 @@ export function Workspace() {
|
||||
// render and the compare copy, so the two can never drift apart.
|
||||
const frame = cropEditing ? 'none' : frameId;
|
||||
const options = stageOptions;
|
||||
// A repair in hand is drawn twice, and the first one is the cheap half:
|
||||
// the blurred fill lands in a fraction of the model's fixed seconds and
|
||||
// is a repair in its own right (it is what a machine without the runtime
|
||||
// gets), so the stroke is answered at once and the model's version of
|
||||
// the same mask replaces it when it arrives. Everything the app files or
|
||||
// exports still waits for the model.
|
||||
if (shownRecipe.adjustments.heal?.length) {
|
||||
const quick = await renderPhoto({
|
||||
source: previewBytes,
|
||||
recipe: shownRecipe,
|
||||
frameId: frame,
|
||||
useGeotag,
|
||||
gpsInfo,
|
||||
assets: engine.assets,
|
||||
options: { ...options, healModel: false },
|
||||
});
|
||||
if (token !== renderToken.current) return;
|
||||
if (quick) {
|
||||
setPreviewUrl((prev) => {
|
||||
if (prev) URL.revokeObjectURL(prev);
|
||||
return URL.createObjectURL(new Blob([quick.bytes as BlobPart], { type: 'image/jpeg' }));
|
||||
});
|
||||
}
|
||||
}
|
||||
const out = await renderPhoto({
|
||||
source: previewBytes,
|
||||
recipe: shownRecipe,
|
||||
|
||||
@@ -120,6 +120,14 @@ export interface RenderOptions {
|
||||
// band appended below it, mirroring the landing page's `.lp-shot` +
|
||||
// `.lp-frame-meta`. Left off the live preview, which must not show it.
|
||||
caption?: { tag: string; title: string; meta: string } | null;
|
||||
// The preview's own rush: draw the frame with the blurred fill alone, and the
|
||||
// model's repair of the same mask arrives in the next render. The graph is a
|
||||
// fixed cost whatever it is handed — through the runtime the app loads, 64x64
|
||||
// and 512x512 both measure the same two seconds — so the only thing that makes
|
||||
// a stroke feel answered is showing the half of HEAL that is instant.
|
||||
// Everything filed or exported leaves this on: a faster answer is worth
|
||||
// something to an eye waiting on the brush, and to nothing else.
|
||||
healModel?: boolean;
|
||||
}
|
||||
|
||||
export interface RenderInput {
|
||||
@@ -1462,7 +1470,11 @@ export async function renderPhoto(input: RenderInput): Promise<RenderResult | nu
|
||||
const srcBottom = Math.min(boxBottom, Math.ceil(my1 + margin));
|
||||
const srcW = srcRight - srcLeft;
|
||||
const srcH = srcBottom - srcTop;
|
||||
if (srcW >= 8 && srcH >= 8) {
|
||||
// ...and the preview may pass on the model entirely, keeping the blurred
|
||||
// fill above (see RenderOptions.healModel): the stroke is answered while
|
||||
// the model is still thinking, and the answer it is owed arrives in the
|
||||
// render after.
|
||||
if (options?.healModel !== false && srcW >= 8 && srcH >= 8) {
|
||||
const { w: mw, h: mh } = inpaintCropSize(srcW, srcH);
|
||||
const cropSurface = own(createSurface(mw, mh));
|
||||
if (cropSurface) {
|
||||
|
||||
@@ -99,6 +99,13 @@ function load(): Promise<Loaded> {
|
||||
ort.env.wasm.numThreads = self.crossOriginIsolated
|
||||
? Math.min(4, navigator.hardwareConcurrency || 1)
|
||||
: 1;
|
||||
// ...and the one thread there is runs in a worker of its own rather than
|
||||
// on the page's own. The graph is seconds of solid arithmetic at this size,
|
||||
// and a second of it on the main thread is a second in which the pointer
|
||||
// does not move: the brush lagged behind the hand exactly as long as the
|
||||
// repair took. The worker loads the same file WASM_DIR serves. WebGPU is
|
||||
// off-thread on its own and ignores this.
|
||||
ort.env.wasm.proxy = true;
|
||||
ort.env.webgpu.powerPreference = 'high-performance';
|
||||
const session = await openSession(ort);
|
||||
return { ort, session };
|
||||
|
||||
Reference in New Issue
Block a user