67 lines
4.1 KiB
Markdown
67 lines
4.1 KiB
Markdown
# PLAN — Standalone playback 100% client-side WASM (no Carla dependency)
|
|
|
|
Goal: fix "mastering fx chain không xử lí" + "vsti không qua main out" bằng cách
|
|
chuyển live playback/preview của VSTi sang WASM (autosampled SF2 qua FluidSynth
|
|
WASM → track sfEntry → FX chain → masterBus mastering → main out). Export WAV
|
|
giữ nguyên backend Python (pedalboard VST3 + FluidSynth C++).
|
|
|
|
## 1. Server — autosample endpoint (VSTi → SF2)
|
|
- `tools/autosample_vsti.py`: tách core thành `autosample_sf2(...)` callable
|
|
(giữ `main()` argparse cho CLI).
|
|
- `app/api/v1/plugins.py`: `AutosampleRequest` + `POST /autosample` — render
|
|
preset qua từng note bằng pedalboard (đúng plugin+preset như export) → SF2
|
|
16-bit mono vào `app/storage/soundfonts/{uuid}.sf2` + `.meta` → trả `sf_id`.
|
|
- Chỉ SF2 (không SF3): client WASM load SF2 trực tiếp, endpoint download ưu
|
|
tiên SF2 sẵn có. 30-60 note ≈ 10-20MB — chấp nhận, IndexedDB cache.
|
|
ponytail: SF3 (OGG nhỏ hơn) khi cần size nhỏ — thêm `--sf3` vào endpoint.
|
|
|
|
## 2. Server — mastering FX chain trong render_project
|
|
- `app/core/mastering_engine.py` (mới): `apply_mastering(buffer, settings, fs)`
|
|
tái hiện chain WebAudio client (initMasterBus + applyMasteringSettings):
|
|
- `eq` (lowshelf 100 / peaking 822 Q0.7 / peaking 3200 Q1.2 / highshelf 10k)
|
|
- `imager` (M/S width 4 band: 20-100 / 100-1k / 1k-6k / 6k-20k)
|
|
- `maximizer` (boost → softclip atan → +upward comp → ceiling hard clip)
|
|
- `compressor` (threshold/ratio/knee/makeup)
|
|
- `limiter` (tanh soft clip)
|
|
- `exciter` (hp 2k + saturator, dry/wet)
|
|
- `rebalance` (M/S gains qua L/R crossfeed)
|
|
- chain order từ `settings['chain']`, gate `masterConnected && !isBypassed`.
|
|
- `app/core/render_engine.py`: `render_project` đọc `project_json['mastering_settings']`
|
|
→ áp lên master_buffer trước `sf.write`; khi có mastering → clip [-1,1]
|
|
(giống WAV encoder client), không mastering → normalize như cũ.
|
|
|
|
## 3. Client — VSTi live playback qua WASM autosample
|
|
- `app/static/js/services/api.js`: thêm `autosampleVsti(payload)`.
|
|
- `app/static/js/services/vstiAutosample.js` (mới): `SonicVstiAutosample` —
|
|
`ensure(synthEngine)` POST autosample 1 lần, cache map plugin_id+preset →
|
|
sf_id (localStorage), `sfIdFor` sync lookup.
|
|
- `app/static/js/services/soundfontPlayer.js`: `_playNoteFluid` +
|
|
`applyAITrackInstrument` fallback `synthEngine.autosampled_sf_id` khi thiếu
|
|
`soundfont_id`; VSTi chưa sample → ensure rồi defer note (pendingNoteOns có sẵn).
|
|
- `app/static/js/services/runtime.js`: `SonicCarlaMidi.shouldRoute` /
|
|
`shouldRoutePlayback` trả false trừ khi `window.__enableCarlaLivePlayback`
|
|
(mặc định off) — playback không phụ thuộc Carla nữa.
|
|
- `app/static/js/app.jsx`:
|
|
- `shouldRouteCarla` → false (VSTi đi qua SonicSF → masterBus → main out).
|
|
- chọn instrument VSTi (applyTrackInstrument) → fire-and-forget
|
|
`SonicVstiAutosample.ensure(synth_engine)`.
|
|
- timeline/local-loop/ghost/preview: VSTi đã chạy nhánh `SonicSF.playNote`
|
|
sẵn (isSfTrackEngine false → else-if); autosampled_sf_id làm nó kêu đúng âm.
|
|
- `ensureMidiCapture` đã capture `node.sfEntry` → VSTi vào midiCache →
|
|
clientSideExport (offline mix qua track FX + mastering) hoạt động cho VSTi.
|
|
- `app/templates/index.html`: script tag `vstiAutosample.js`.
|
|
- Build: `npm run build` (app.jsx → app.precompiled.js).
|
|
|
|
## 4. Verify
|
|
- pytest `tests/` (kỳ vọng 108 pass / 1 skip / 1 env-specific fail như cũ).
|
|
- Test mastering: render project JSON có mastering_settings (chain eq+imager+
|
|
maximizer) → assert output peak ≤ ceiling & khác bản không mastering.
|
|
- Test autosample: gọi `autosample_sf2` với VSTi (Nexus.vst3) → sf2utils parse
|
|
sạch; endpoint return sf_id + file tồn tại.
|
|
- Manual (browser): track VSTi → Preview/Play → âm qua masterBus (VU master
|
|
nhảy), Export WAV server ra file có mastering.
|
|
|
|
## 5. Commit/push
|
|
- Commit trên branch `standalone`. Push vẫn chờ user cấu hình git auth
|
|
(credential.helper / token URL) — không retry khi chưa có.
|