73 lines
4.3 KiB
Markdown
73 lines
4.3 KiB
Markdown
# Hợp đồng realtime FX (WS + worklet) — SonicForgeStudio
|
||
|
||
Trích từ `app/api/v1/plugins.py`, `app/core/fx_realtime.py`,
|
||
`app/static/js/sf-fx-realtime.js`, `app/static/js/fx-rt-worker.js`, `app/static/js/app.jsx`
|
||
tại HEAD `7c236d4`. Đây là đặc tả để port sang Rust (P1/P4).
|
||
|
||
## 1. HTTP (REST)
|
||
|
||
| Method | Path | Body | Trả về |
|
||
|---|---|---|---|
|
||
| POST | `/api/v1/plugins/fx-realtime/start` | `{fx_chain: [], sample_rate: 44100}` | `{success, session_id, ws_url, block_size, sample_rate}` |
|
||
| POST | `/api/v1/plugins/fx-realtime/stop` | `{session_id}` | `{success, stopped}` |
|
||
| POST | `/api/v1/plugins/fx-realtime/set-chain` | `{session_id, fx_chain}` | `{success, updated}` |
|
||
| GET | `/api/v1/plugins/bridge/status` | — | trạng thái bridge/SHM |
|
||
| POST | `/api/v1/plugins/render` | job render offline (VST) | file kết quả |
|
||
| POST | `/api/v1/plugins/soundfont-render` | job render SF2 | WAV |
|
||
|
||
- `block_size` = **256 mẫu** (`FXRT_BLOCK` trong `FxRealtimeIPC.h`).
|
||
- Auth: Bearer token hoặc `?token=`; desktop loopback không token → auto admin.
|
||
|
||
## 2. WebSocket audio
|
||
|
||
`GET /api/v1/plugins/ws/fx-realtime/{session_id}?token=...`
|
||
|
||
| Hướng | Kiểu frame | Nội dung |
|
||
|---|---|---|
|
||
| client → server | **binary** | 2048 B = 256 frame × 2 kênh `float32` interleaved |
|
||
| server → client | **binary** | như trên, đã qua chain FX |
|
||
| client → server | text JSON | `{"cmd":"set_param","slot":n,"key":"...","value":f}` · `{"cmd":"latency","slot":n,"samples":n}` · `{"cmd":"flush"}` |
|
||
| server → client | text JSON | `{"cmd":"latency","slot":n,"samples":n}` · `{"cmd":"flush_done"}` |
|
||
|
||
Server: `ws_fx_realtime()` (plugins.py ~1344) — vòng lặp receive block → SHM ring →
|
||
`fx_vst_bridge --realtime-fx` (hoặc `--juce-fx`) → SHM → send.
|
||
|
||
## 3. Worklet ↔ main thread (MessagePort)
|
||
|
||
`sf-fx-realtime.js` (AudioWorkletProcessor `sf-fx-realtime`).
|
||
|
||
Hằng số: `BLOCK=256`, `BURST=4` (gom 4 block = 1024 mẫu/frame), `FILL=40` (≈213 ms @48k),
|
||
`RESUME_FILL=24`, `CAP=64` (≈341 ms), `FADE=512`, `HOLD_FADE=4`, `WRAP_FADE=256`.
|
||
|
||
| type | Hướng | Payload |
|
||
|---|---|---|
|
||
| `__direct` | main → worklet | chuyển `MessagePort` (worker↔worklet trực tiếp, bỏ main thread) |
|
||
| `in` | worklet → main/worker | `{data: ArrayBuffer}` 1024 mẫu stereo interleaved |
|
||
| `out` | worker → worklet | `{l, r: Float32Array}` block đã xử lý |
|
||
| `started` | worklet → main | phát block đầu tiên (đã đủ FILL) |
|
||
| `clear_queue` | main → worklet | `{mode: 'play'\|'seek'\|'loop'}` — wipe queue, `hasStarted=false` (mode `play`) |
|
||
| `flush` | worklet → main | yêu cầu engine xả ring stale |
|
||
| `flush_done` | worker → worklet | engine đã flush |
|
||
| `flush_for_stop` | main → worklet | dừng phát (giữ queue) |
|
||
| `set_param` | worklet → main | forward nguyên văn tới WS |
|
||
| `latency` | main → worklet | `{samples}` — latency chain thật |
|
||
| `stats` | worklet → main | `{underruns, holds, fades, silences, capDrops, fadeIns, queueLen}` (≈2 s/lần) |
|
||
| `dbg` | worklet → main | `{tag, t, ...}`: `fx_in_first`, `fx_out_first`, `fx_play_first`, `fx_seek_first`, `fx_loop_clear`, `fx_seek_clear`, `fx_wrap_xfade` |
|
||
| `dbg_state` | main → worklet | yêu cầu snapshot trạng thái nội bộ |
|
||
| `dbg_state_r` | worklet → main | `{k, t, q, playing, hasStarted, silenced, flushPending, stopped, wrapMode, wrapFadeBlocks, latencySamples, resumeFill, inCount, outPos, fadeLeft, holdFadeLeft}` |
|
||
|
||
## 4. Đồng bộ playhead / PDC (phía UI, app.jsx)
|
||
|
||
- `FX_RT_WET_LAG_SAMPLES = 10240` (= FILL 40 × 256) — độ trễ nội dung khi vào FX path.
|
||
- `pdcDelay.delayTime` = 10240/sr khi FX active, 0 khi dry (`fxRtApplyPdc`).
|
||
- Crossfade dry↔wet: `masterFxDryGain` / `masterFxGain` rampe `setTargetAtTime(tc=0.025)`.
|
||
- `fxRt.crossfade(token)` bất biến: `dryGain + fxGain = 1`.
|
||
- Warm bypass: giữ session/node/ws khi PWR OFF giữa lúc play (`fxRtWarmPlayHold`).
|
||
|
||
## 5. Ghi chú cho bản Rust (P4)
|
||
|
||
Hợp đồng tối thiểu cần giữ để frontend không phải sửa:
|
||
1. `POST /fx-realtime/start` → `session_id` + `ws_url`, `block_size=256`.
|
||
2. WS binary 2048 B/bản tin, text JSON 3 lệnh (`set_param`, `latency`, `flush`) + 2 phản hồi (`latency`, `flush_done`).
|
||
3. Thay hop Python bằng Rust: `worklet → Tauri IPC/Rust → SHM`, giữ nguyên SHM layout với `fx_vst_bridge`.
|