P0: cat torch/torchvision khoi engine bundle (1842MB -> 147MB) + hop dong API + golden conformance + baseline

2026-10-12 21:03:17 +07:00
parent 7c236d472f
commit 322748898b
8 changed files with 7749 additions and 6 deletions
@@ -0,0 +1,72 @@
+# Hợp đồng realtime FX (WS + worklet) — SonicForgeStudio
+
+Trích từ `app/api/v1/plugins.py`, `app/core/fx_realtime.py`,
+`app/static/js/sf-fx-realtime.js`, `app/static/js/fx-rt-worker.js`, `app/static/js/app.jsx`
+tại HEAD `7c236d4`. Đây là đặc tả để port sang Rust (P1/P4).
+
+## 1. HTTP (REST)
+
+| Method | Path | Body | Trả về |
+|---|---|---|---|
+| POST | `/api/v1/plugins/fx-realtime/start` | `{fx_chain: [], sample_rate: 44100}` | `{success, session_id, ws_url, block_size, sample_rate}` |
+| POST | `/api/v1/plugins/fx-realtime/stop` | `{session_id}` | `{success, stopped}` |
+| POST | `/api/v1/plugins/fx-realtime/set-chain` | `{session_id, fx_chain}` | `{success, updated}` |
+| GET | `/api/v1/plugins/bridge/status` | — | trạng thái bridge/SHM |
+| POST | `/api/v1/plugins/render` | job render offline (VST) | file kết quả |
+| POST | `/api/v1/plugins/soundfont-render` | job render SF2 | WAV |
+
+- `block_size` = **256 mẫu** (`FXRT_BLOCK` trong `FxRealtimeIPC.h`).
+- Auth: Bearer token hoặc `?token=`; desktop loopback không token → auto admin.
+
+## 2. WebSocket audio
+
+`GET /api/v1/plugins/ws/fx-realtime/{session_id}?token=...`
+
+| Hướng | Kiểu frame | Nội dung |
+|---|---|---|
+| client → server | **binary** | 2048 B = 256 frame × 2 kênh `float32` interleaved |
+| server → client | **binary** | như trên, đã qua chain FX |
+| client → server | text JSON | `{"cmd":"set_param","slot":n,"key":"...","value":f}` · `{"cmd":"latency","slot":n,"samples":n}` · `{"cmd":"flush"}` |
+| server → client | text JSON | `{"cmd":"latency","slot":n,"samples":n}` · `{"cmd":"flush_done"}` |
+
+Server: `ws_fx_realtime()` (plugins.py ~1344) — vòng lặp receive block → SHM ring →
+`fx_vst_bridge --realtime-fx` (hoặc `--juce-fx`) → SHM → send.
+
+## 3. Worklet ↔ main thread (MessagePort)
+
+`sf-fx-realtime.js` (AudioWorkletProcessor `sf-fx-realtime`).
+
+Hằng số: `BLOCK=256`, `BURST=4` (gom 4 block = 1024 mẫu/frame), `FILL=40` (≈213 ms @48k),
+`RESUME_FILL=24`, `CAP=64` (≈341 ms), `FADE=512`, `HOLD_FADE=4`, `WRAP_FADE=256`.
+
+| type | Hướng | Payload |
+|---|---|---|
+| `__direct` | main → worklet | chuyển `MessagePort` (worker↔worklet trực tiếp, bỏ main thread) |
+| `in` | worklet → main/worker | `{data: ArrayBuffer}` 1024 mẫu stereo interleaved |
+| `out` | worker → worklet | `{l, r: Float32Array}` block đã xử lý |
+| `started` | worklet → main | phát block đầu tiên (đã đủ FILL) |
+| `clear_queue` | main → worklet | `{mode: 'play'\|'seek'\|'loop'}` — wipe queue, `hasStarted=false` (mode `play`) |
+| `flush` | worklet → main | yêu cầu engine xả ring stale |
+| `flush_done` | worker → worklet | engine đã flush |
+| `flush_for_stop` | main → worklet | dừng phát (giữ queue) |
+| `set_param` | worklet → main | forward nguyên văn tới WS |
+| `latency` | main → worklet | `{samples}` — latency chain thật |
+| `stats` | worklet → main | `{underruns, holds, fades, silences, capDrops, fadeIns, queueLen}` (≈2 s/lần) |
+| `dbg` | worklet → main | `{tag, t, ...}`: `fx_in_first`, `fx_out_first`, `fx_play_first`, `fx_seek_first`, `fx_loop_clear`, `fx_seek_clear`, `fx_wrap_xfade` |
+| `dbg_state` | main → worklet | yêu cầu snapshot trạng thái nội bộ |
+| `dbg_state_r` | worklet → main | `{k, t, q, playing, hasStarted, silenced, flushPending, stopped, wrapMode, wrapFadeBlocks, latencySamples, resumeFill, inCount, outPos, fadeLeft, holdFadeLeft}` |
+
+## 4. Đồng bộ playhead / PDC (phía UI, app.jsx)
+
+- `FX_RT_WET_LAG_SAMPLES = 10240` (= FILL 40 × 256) — độ trễ nội dung khi vào FX path.
+- `pdcDelay.delayTime` = 10240/sr khi FX active, 0 khi dry (`fxRtApplyPdc`).
+- Crossfade dry↔wet: `masterFxDryGain` / `masterFxGain` rampe `setTargetAtTime(tc=0.025)`.
+- `fxRt.crossfade(token)` bất biến: `dryGain + fxGain = 1`.
+- Warm bypass: giữ session/node/ws khi PWR OFF giữa lúc play (`fxRtWarmPlayHold`).
+
+## 5. Ghi chú cho bản Rust (P4)
+
+Hợp đồng tối thiểu cần giữ để frontend không phải sửa:
+1. `POST /fx-realtime/start` → `session_id` + `ws_url`, `block_size=256`.
+2. WS binary 2048 B/bản tin, text JSON 3 lệnh (`set_param`, `latency`, `flush`) + 2 phản hồi (`latency`, `flush_done`).
+3. Thay hop Python bằng Rust: `worklet → Tauri IPC/Rust → SHM`, giữ nguyên SHM layout với `fx_vst_bridge`.