fix(fx): realtime VST ngắt quãng khi mở MASTERING PANEL + PWR master không ảnh hưởng âm; pacing WS input/output, gate fxRt theo masterConnected, worklet cap 32
This commit is contained in:
@@ -0,0 +1,109 @@
|
||||
# PLAN: VST FX SHM realtime
|
||||
|
||||
Mục tiêu: VST3 FX (master FX chain + track chain) xử lý THẬT realtime qua
|
||||
fx_vst_bridge — không còn pass-through khi nghe live.
|
||||
|
||||
Kiến trúc (cloud-FX loop, latency ~2 block):
|
||||
WebAudio (masterBus.output)
|
||||
→ AudioWorklet (sf-fx-realtime.js) — input block
|
||||
→ WebSocket (engine /ws/fx-realtime/{id})
|
||||
→ Python SHM input ring (SonicForge_FXRT_*)
|
||||
→ fx_vst_bridge.exe --realtime-fx (RealtimeFxChain::process, SEH-guarded)
|
||||
→ SHM output ring
|
||||
→ Python → WebSocket → AudioWorklet output → graph
|
||||
|
||||
SHM layout FxRealtimeIPC (C++ struct = Python ctypes mirror, all u32/float 4B):
|
||||
header u32: magic(0x46585254) state sampleRate blockSize running heartbeat
|
||||
inWrite inRead inSlots outWrite outRead outSlots (48 bytes)
|
||||
inL[4][256] inR[4][256] outL[8][256] outR[8][256]
|
||||
|
||||
Files:
|
||||
1. native_bridge/include/FxRealtimeIPC.h — struct + constants (mới)
|
||||
2. native_bridge/src/RealtimeFxLoop.cpp — loop (mới)
|
||||
3. native_bridge/src/main_fx.cpp — thêm --realtime-fx
|
||||
4. native_bridge/CMakeLists.txt — thêm 2 file vào fx_vst_bridge
|
||||
5. app/core/fx_realtime.py — session SHM + spawn (mới)
|
||||
6. app/api/v1/plugins.py — /fx-realtime/start, /stop, WS
|
||||
7. app/static/js/sf-fx-realtime.js — AudioWorklet (mới)
|
||||
8. app/static/js/app.jsx — fxRealtimeManager wiring
|
||||
9. engine.spec — hiddenimport websockets
|
||||
|
||||
v1 scope: MASTER chain realtime (chỗ VST3 bị skip trong rebuildMasteringGraph).
|
||||
Track chain realtime: ponytail — per-track session sau (export offline đã có).
|
||||
|
||||
## Trạng thái: ✅ v1 DONE (2026-08-20)
|
||||
|
||||
- Bridge build OK (native_bridge/build/Release/fx_vst_bridge.exe) + deploy sidecar
|
||||
(src-tauri/binaries/fx_vst_bridge-x86_64-pc-windows-msvc.exe +
|
||||
src-tauri/target/release/fx_vst_bridge.exe).
|
||||
- Engine build OK (websockets bundled) + deploy target/release + resources.
|
||||
- Frontend build OK (app.precompiled.js + sf-fx-realtime.js static serve).
|
||||
- E2E đã test trên engine deploy (port 8000):
|
||||
- POST /fx-realtime/start (purecomp.vst3, 44100) → session_id + ws_url ✓
|
||||
- WS binary loop: chain rỗng = passthrough bit-exact (peak 0.5/rms 0.35) ✓
|
||||
- WS + purecomp: peak 0.5→0.462, rms 0.35→0.246 (compressor attenuate) ✓
|
||||
- WS disconnect → session tự stop, fx_vst_bridge.exe thoát ✓
|
||||
- POST /fx-realtime/stop → stopped:true ✓
|
||||
- WS token sai → reject (handshake 403) ✓
|
||||
- FIX phát hiện khi test: WS drain output chỉ trong vòng receive_bytes → output
|
||||
thừa kẹt SHM ring khi client ngừng gửi → tách _pump_output() task riêng.
|
||||
|
||||
## Trạng thái: ✅ TRACK REALTIME DONE (2026-08-20)
|
||||
|
||||
- Track chain realtime: mọi luồng âm thanh track qua bridge
|
||||
(fx_vst_bridge --realtime-fx + SHM SonicForge_FXRT_*) trước main out.
|
||||
- Module-level `fxRtTracks` map 'trackId:audio|sf' + fxRtTrackChain/Paths/
|
||||
Start/Splice/Stop/Reconcile/StopAll trong app.jsx; gọi từ rebuildTrackFxGraph,
|
||||
startTrackPlayback, stopAllPlayback, FX Rack panel.
|
||||
- `fxRtTrackPaths`: audio nếu clips.length>0 || buffer || sections.length>0;
|
||||
sf nếu midiItems.length>0 || type==='MIDI' || instrumentId.
|
||||
- E2E ổn định (pw_e2e2.py ×3): splice 3 audio + 3 sf, track audio out
|
||||
peak≈0.47 rms≈0.33, sf câm hợp lệ (MIDI taiko không route sfEntry), master
|
||||
Ozone peak≈0.85-0.99. Master câm khi track mastering_bypass=true — hợp lệ.
|
||||
- FIX bug bypass toggle: reconcile đọc state CŨ (React setState async) khi
|
||||
`onChange` gọi `__rebuildTrackFxGraph(track.id)` đồng bộ → bypass không stop
|
||||
session. Fix: `rebuildTrackFxGraph = (trackId, patch)` merge
|
||||
`track = { ...track, ...patch }` TRƯỚC khi rebuild/reconcile; 4 call site
|
||||
(bypass, fxActive, setVstChain, setChain) truyền patch giá trị MỚI.
|
||||
Verify: toggle bypass-on → reconcile want=[] → stop ngay; bypass-off →
|
||||
want=["sf"] → start mới + spliced.
|
||||
- Debug log tạm đã gỡ (giữ `[FX RT track] spliced` + console.warn lỗi);
|
||||
rebuild bundle 1370231 bytes, served LEN khớp local.
|
||||
|
||||
Ponytail còn lại:
|
||||
- Sub-tab (piano roll) play chưa start fxRt — chỉ main/section play.
|
||||
- Chain đổi giữa lúc play chưa restart session (start/stop theo play boundary).
|
||||
- Browser-side queue drop-oldest (8 block) — thay adaptive stretch nếu cần.
|
||||
- Chưa test UI thật trong WebView (graph insertion AudioWorkletNode) — plumbing
|
||||
đã verify qua WS; mở app + bật master VST chain + play để xác nhận âm thanh.
|
||||
|
||||
## Trạng thái: ✅ MASTER REALTIME UI VERIFIED (2026-08-20, engine sống)
|
||||
|
||||
- Verify UI-level: login → inject `window.currentMasteringSettings.vstFxChain`
|
||||
(path đúng `C:\Program Files\Common Files\VST3\Sonible\purecomp.vst3`) →
|
||||
insert clip → Play. Monkey-patch `SonicAPI.fxRealtimeStart` ghi payload→sid.
|
||||
Kết quả (pw_master3.py): call #1 = master (purecomp, engine trả session_id +
|
||||
ws_url) → WS `SonicForge_FXRT_d19c01fc33da` nhận audio 738 frame, out
|
||||
peak 0.4622 / rms 0.1442 vs input sine 0.5/0.35 = attenuate ✓.
|
||||
Track sessions (call #2/#3) chạy song song; stop → WS close, `--realtime-fx`
|
||||
bridge thoát, chỉ còn 2 `--fx-gui` của user (không đụng). Engine @8000 OK.
|
||||
- WS-level (rt_test2.py) trước đó: master realtime attenuate peak 0.4621 rms
|
||||
0.2462 với path đúng `Sonible\purecomp.vst3` (path sai `SoniBook` = passthrough
|
||||
thầm lặng).
|
||||
- Nguyên nhân "VST FX PLUGIN trong master chain peak -inf" (trả lời user):
|
||||
1) Engine bị kill lúc test → `SonicAPI.fxRealtimeStart` fail → `fxRtStart`
|
||||
catch + toast cảnh báo, KHÔNG session → masterBus.output nối thẳng
|
||||
analyser → âm nghe được nhưng KHÔNG qua VST → meter plugin -inf.
|
||||
2) Chain rỗng / vstBypass=true → `fxRtChain()` = [] → không session (âm qua
|
||||
thẳng, không lỗi) → meter -inf.
|
||||
3) Meter trong GUI embed (`--fx-gui`) là của instance GUI RIÊNG (config JSON
|
||||
chỉ path+name; VstGuiEmbed chỉ /frame /ping /input — KHÔNG có audio
|
||||
endpoint) → GUI embed KHÔNG nhận audio realtime SHM session → meter -inf
|
||||
VÔ ĐIỀU KIỆN kể cả khi realtime đang chạy. Audio thật đi qua instance
|
||||
`--realtime-fx` riêng (không hiển thị meter). → ponytail thật: user không
|
||||
thấy meter plugin thật; nơi thấy tín hiệu = master VU meter của app
|
||||
(post-chain) hoặc WS session.
|
||||
- Lưu ý: `masterFxList` (Plugin Picker mastering) load trong useEffect([]) của
|
||||
MasteringModal mount sẵn từ đầu → trước khi SonicAPI sẵn sàng → picker rỗng
|
||||
trong headless; track modal mount trễ hơn nên đầy. Workaround: chờ SonicAPI
|
||||
rồi mới mở modal, hoặc inject chain trực tiếp. (Không phải lỗi engine.)
|
||||
+14
-8
@@ -9,13 +9,19 @@ router = APIRouter()
|
||||
# WebView2 standalone chặn anchor-download → không thấy save dialog. Giải pháp:
|
||||
# frontend gọi /save-dialog → engine mở NATIVE SaveFileDialog (Windows Explorer)
|
||||
# → trả đường dẫn user chọn → frontend gửi bytes qua /write → ghi file thật.
|
||||
SAVE_DIALOG_TIMEOUT = 120 # user có thể để dialog mở lâu
|
||||
SAVE_DIALOG_TIMEOUT = 120 # fallback native (PowerShell) — user có thể để dialog mở lâu
|
||||
# Bridge timeout phải LỚN HƠN Rust wait (300s trong lib.rs) — nếu không, user
|
||||
# chọn path sau 300s thì engine đã bỏ cuộc → save_file.response bị bỏ lại, file
|
||||
# không được ghi dù user đã chọn nơi lưu (bug "hỏi nơi lưu nhưng không có file").
|
||||
SAVE_BRIDGE_TIMEOUT = 360
|
||||
|
||||
|
||||
def _save_file_via_tauri_bridge(default_name: str) -> Optional[str]:
|
||||
"""Tauri shell (Rust watcher trong lib.rs) mở NATIVE SaveFileDialog qua
|
||||
file IPC: engine ghi save_file.request (chứa tên mặc định) → Rust mở dialog
|
||||
→ ghi save_file.response. Trả None nếu bridge không tồn tại (standalone)."""
|
||||
→ ghi save_file.response. Trả None nếu bridge không tồn tại (standalone) →
|
||||
caller fallback native. Trả "" nếu bridge ĐÃ dùng nhưng user hủy / hết hạn →
|
||||
KHÔNG fallback (tránh 2 dialog cùng lúc + tránh ghi đè path user đã chọn)."""
|
||||
root = os.environ.get("APPDATA") or os.path.expanduser("~")
|
||||
ipc = os.path.join(root, "SonicForgeDAW", "ipc")
|
||||
if not os.path.isdir(ipc):
|
||||
@@ -35,7 +41,7 @@ def _save_file_via_tauri_bridge(default_name: str) -> Optional[str]:
|
||||
os.remove(f)
|
||||
with open(req, "w", encoding="utf-8") as fh:
|
||||
fh.write(default_name or "untitled")
|
||||
deadline = _time.time() + SAVE_DIALOG_TIMEOUT
|
||||
deadline = _time.time() + SAVE_BRIDGE_TIMEOUT
|
||||
while _time.time() < deadline:
|
||||
if os.path.exists(resp):
|
||||
try:
|
||||
@@ -43,11 +49,11 @@ def _save_file_via_tauri_bridge(default_name: str) -> Optional[str]:
|
||||
val = fh.read().strip()
|
||||
finally:
|
||||
os.remove(resp)
|
||||
return val or None
|
||||
return val # "" khi user hủy — KHÔNG fallback PowerShell (tránh 2 dialog)
|
||||
_time.sleep(0.1)
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
return ""
|
||||
return "" # bridge đã dùng nhưng hết hạn — KHÔNG fallback (dialog Rust vẫn có thể đang mở)
|
||||
|
||||
|
||||
def _save_file_native_engine(default_name: str, filter_name: str, filter_ext: str) -> Optional[str]:
|
||||
@@ -107,9 +113,9 @@ def save_file_dialog(req: SaveDialogRequest):
|
||||
Trả {"path": "<đường dẫn user chọn>"} hoặc {"path": None} (hủy/không có
|
||||
dialog). def (sync) → FastAPI chạy threadpool — không block loop."""
|
||||
path = _save_file_via_tauri_bridge(req.default_name)
|
||||
if path is None:
|
||||
if path is None: # không có bridge (standalone thuần) → fallback native
|
||||
path = _save_file_native_engine(req.default_name, req.filter_name, req.filter_ext)
|
||||
return {"path": path}
|
||||
return {"path": path or None}
|
||||
|
||||
|
||||
@router.post("/write")
|
||||
|
||||
+112
-3
@@ -1,7 +1,7 @@
|
||||
import os, sys, uuid, json, tempfile, subprocess, time as _time, threading
|
||||
import os, sys, uuid, json, tempfile, subprocess, time as _time, threading, asyncio
|
||||
import numpy as np
|
||||
import soundfile as sf
|
||||
from fastapi import APIRouter, HTTPException, Depends, UploadFile, File, BackgroundTasks, Header, Query
|
||||
from fastapi import APIRouter, HTTPException, Depends, UploadFile, File, BackgroundTasks, Header, Query, WebSocket, WebSocketDisconnect
|
||||
from fastapi.responses import FileResponse
|
||||
from pydantic import BaseModel
|
||||
from typing import Optional, Any
|
||||
@@ -12,6 +12,8 @@ from app.core.soundfont_inspector import SoundFontInspector
|
||||
from app.core.soundfont_converter import SoundFontConverter
|
||||
from app.core.soundfont_scanner import SoundFontAutoScanner
|
||||
from app.api.v1.auth import get_current_user, enforce_password_changed
|
||||
from app.core import fx_realtime
|
||||
from app.core.auth import decode_token
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -1179,7 +1181,12 @@ async def fx_render(req: FxRenderRequest, current_user: dict = Depends(get_curre
|
||||
if not fid:
|
||||
raise HTTPException(status_code=400,
|
||||
detail="Cần input_file_id hoặc input_path")
|
||||
src = os.path.join(settings.PROCESSED_DIR, os.path.basename(fid))
|
||||
# File từ client upload (audio/upload → UPLOADS_DIR) hoặc đã xử lý
|
||||
# (PROCESSED_DIR) — resolve cả hai như audio.download.
|
||||
fid = os.path.basename(fid)
|
||||
src = os.path.join(settings.PROCESSED_DIR, fid)
|
||||
if not os.path.isfile(src):
|
||||
src = os.path.join(settings.UPLOADS_DIR, fid)
|
||||
if not os.path.isfile(src):
|
||||
raise HTTPException(status_code=404,
|
||||
detail=f"File đầu vào không tồn tại: {src}")
|
||||
@@ -1222,6 +1229,108 @@ def _render_fx_locked(input_path: str, fx_chain: list, sample_rate: int, out_pat
|
||||
sample_rate=sample_rate, out_path=out_path)
|
||||
|
||||
|
||||
class FxRealtimeStartRequest(BaseModel):
|
||||
"""Start một session FX realtime (master chain qua fx_vst_bridge
|
||||
--realtime-fx + SHM ring). Trả session_id để client mở WebSocket
|
||||
/ws/fx-realtime/{session_id} đẩy/kéo block audio."""
|
||||
fx_chain: list = []
|
||||
sample_rate: int = 44100
|
||||
|
||||
class FxRealtimeStopRequest(BaseModel):
|
||||
session_id: str = ""
|
||||
|
||||
@router.post("/fx-realtime/start")
|
||||
async def fx_realtime_start(req: FxRealtimeStartRequest,
|
||||
current_user: dict = Depends(get_current_user)):
|
||||
"""Start bridge realtime FX loop + SHM. Client mở WS /ws/fx-realtime/{id}
|
||||
(token qua query param 'token' hoặc Authorization Bearer) rồi gửi block
|
||||
stereo float32 interleaved (block_size*2 float = 2048B), nhận block đã xử lý."""
|
||||
enforce_password_changed(current_user)
|
||||
try:
|
||||
sess = fx_realtime.start_session(req.fx_chain, req.sample_rate)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=500,
|
||||
detail=f"Không khởi động FX realtime: {e}")
|
||||
return {
|
||||
"success": True,
|
||||
"session_id": sess.name,
|
||||
"ws_url": f"/api/v1/plugins/ws/fx-realtime/{sess.name}",
|
||||
"block_size": sess.block_size,
|
||||
"sample_rate": sess.sample_rate,
|
||||
}
|
||||
|
||||
@router.post("/fx-realtime/stop")
|
||||
async def fx_realtime_stop(req: FxRealtimeStopRequest,
|
||||
current_user: dict = Depends(get_current_user)):
|
||||
"""Dừng session realtime (running=0 → bridge thoát, SHM unlink)."""
|
||||
enforce_password_changed(current_user)
|
||||
ok = fx_realtime.stop_session(req.session_id or "")
|
||||
return {"success": True, "stopped": ok}
|
||||
|
||||
@router.websocket("/ws/fx-realtime/{session_id}")
|
||||
async def ws_fx_realtime(websocket: WebSocket, session_id: str):
|
||||
"""Vòng lặp FX realtime: receive block input → SHM → bridge → SHM → send
|
||||
block output. Binary frames: 2048B stereo float32 interleaved (256*2)."""
|
||||
# Auth: token qua query param (JS WebSocket không set header dễ) hoặc
|
||||
# Authorization Bearer. Desktop app có thể chưa login → token rỗng được
|
||||
# phép (endpoint /start vẫn yêu cầu auth).
|
||||
token = websocket.query_params.get("token", "") or ""
|
||||
if not token:
|
||||
auth = websocket.headers.get("authorization", "")
|
||||
if auth.lower().startswith("bearer "):
|
||||
token = auth.split(" ", 1)[1].strip()
|
||||
if token:
|
||||
try:
|
||||
payload = decode_token(token)
|
||||
except Exception:
|
||||
payload = None
|
||||
if not payload:
|
||||
await websocket.close(code=4401)
|
||||
return
|
||||
sess = fx_realtime.get_session(session_id)
|
||||
if not sess:
|
||||
await websocket.close(code=4404)
|
||||
return
|
||||
await websocket.accept()
|
||||
# Output pump riêng: bridge sản xuất block KHÔNG đồng bộ với input client —
|
||||
# nếu chỉ drain trong vòng receive_bytes, output thừa kẹt lại trong SHM ring
|
||||
# (client ngừng gửi → server block ở receive → âm tắc).
|
||||
async def _pump_output():
|
||||
# Pace mỗi block theo tốc độ âm thật (256/sr ~ 5.8ms): bridge sản xuất
|
||||
# output theo burst khi nhận input dồn; bắn toàn bộ cùng lúc → worklet
|
||||
# overflow drop block cũ nhất → âm ngắt quãng.
|
||||
block_sec = (sess.block_size or 256) / max(int(sess.sample_rate or 44100), 1)
|
||||
try:
|
||||
while True:
|
||||
blks = sess.read_output()
|
||||
if not blks:
|
||||
await asyncio.sleep(0.002)
|
||||
continue
|
||||
for blk in blks:
|
||||
await websocket.send_bytes(blk)
|
||||
await asyncio.sleep(block_sec)
|
||||
except Exception:
|
||||
pass
|
||||
pump_task = asyncio.create_task(_pump_output())
|
||||
try:
|
||||
while True:
|
||||
data = await websocket.receive_bytes()
|
||||
if not sess.alive():
|
||||
break
|
||||
sess.write_input(data)
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
pump_task.cancel()
|
||||
except Exception:
|
||||
pass
|
||||
fx_realtime.stop_session(session_id)
|
||||
|
||||
class SoundfontRenderRequest(BaseModel):
|
||||
"""Render MIDI notes → WAV bằng soundfont THẬT qua native FluidSynth
|
||||
(pyfluidsynth) — luồng âm instrument cho môi trường STANDALONE (xử lí
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
# app/core/fx_realtime.py
|
||||
"""VST FX SHM realtime (cloud-FX loop): engine tạo mapping FxRealtimeIPC,
|
||||
spawn fx_vst_bridge --realtime-fx, đẩy/kéo block audio qua SHM ring.
|
||||
|
||||
Layout mirror chính xác native_bridge/include/FxRealtimeIPC.h (mọi field
|
||||
u32/float 4-byte, không padding):
|
||||
header 12 u32 (48B): magic state sampleRate blockSize running heartbeat
|
||||
inWrite inRead inSlots outWrite outRead outSlots
|
||||
inL[4][256] inR[4][256] outL[8][256] outR[8][256]
|
||||
|
||||
Luồng dữ liệu: browser → WS → write_input() → [bridge xử lý] → read_output()
|
||||
→ WS → browser. Block = FXRT_BLOCK=256 stereo float32 = 2048 bytes.
|
||||
"""
|
||||
import ctypes
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
|
||||
import numpy as np
|
||||
|
||||
from app.config import settings
|
||||
from app.core.native_render import find_fx_bridge_exe
|
||||
|
||||
FXRT_MAGIC = 0x46585254
|
||||
FXRT_BLOCK = 256
|
||||
FXRT_IN_SLOTS = 4
|
||||
FXRT_OUT_SLOTS = 8
|
||||
FXRT_STATE_STARTING = 0
|
||||
FXRT_STATE_READY = 1
|
||||
FXRT_STATE_ERROR = 2
|
||||
|
||||
|
||||
class FxRTHeader(ctypes.Structure):
|
||||
_fields_ = [
|
||||
("magic", ctypes.c_uint32),
|
||||
("state", ctypes.c_uint32),
|
||||
("sample_rate", ctypes.c_uint32),
|
||||
("block_size", ctypes.c_uint32),
|
||||
("running", ctypes.c_uint32),
|
||||
("heartbeat", ctypes.c_uint32),
|
||||
("in_write", ctypes.c_uint32),
|
||||
("in_read", ctypes.c_uint32),
|
||||
("in_slots", ctypes.c_uint32),
|
||||
("out_write", ctypes.c_uint32),
|
||||
("out_read", ctypes.c_uint32),
|
||||
("out_slots", ctypes.c_uint32),
|
||||
]
|
||||
|
||||
|
||||
HEADER_SIZE = ctypes.sizeof(FxRTHeader) # 48
|
||||
IN_L_OFF = HEADER_SIZE # 48
|
||||
IN_R_OFF = IN_L_OFF + FXRT_IN_SLOTS * FXRT_BLOCK * 4
|
||||
OUT_L_OFF = IN_R_OFF + FXRT_IN_SLOTS * FXRT_BLOCK * 4
|
||||
OUT_R_OFF = OUT_L_OFF + FXRT_OUT_SLOTS * FXRT_BLOCK * 4
|
||||
TOTAL_SIZE = OUT_R_OFF + FXRT_OUT_SLOTS * FXRT_BLOCK * 4
|
||||
|
||||
|
||||
class FxRealtimeSession:
|
||||
"""Một session realtime: SHM + subprocess fx_vst_bridge + I/O block."""
|
||||
|
||||
def __init__(self, fx_chain, sample_rate, block_size=FXRT_BLOCK):
|
||||
from multiprocessing import shared_memory
|
||||
self.fx_chain = fx_chain or []
|
||||
self.sample_rate = int(sample_rate)
|
||||
self.block_size = int(block_size or FXRT_BLOCK)
|
||||
if self.block_size != FXRT_BLOCK:
|
||||
raise ValueError("block_size phải = 256 (khớp C++ FxRealtimeIPC)")
|
||||
self.name = "SonicForge_FXRT_" + uuid.uuid4().hex[:12]
|
||||
self.shm = shared_memory.SharedMemory(name=self.name, create=True,
|
||||
size=TOTAL_SIZE)
|
||||
self.shm.buf[:TOTAL_SIZE] = b"\x00" * TOTAL_SIZE
|
||||
self.h = FxRTHeader.from_buffer(self.shm.buf)
|
||||
self.h.magic = FXRT_MAGIC
|
||||
self.h.state = FXRT_STATE_STARTING
|
||||
self.h.sample_rate = self.sample_rate
|
||||
self.h.block_size = self.block_size
|
||||
self.h.running = 1
|
||||
self.h.in_slots = FXRT_IN_SLOTS
|
||||
self.h.out_slots = FXRT_OUT_SLOTS
|
||||
self._np = np.frombuffer(self.shm.buf, dtype=np.float32)
|
||||
self._lock = threading.Lock()
|
||||
self.proc = None
|
||||
self._job_path = ""
|
||||
self._started_at = time.time()
|
||||
self._spawn()
|
||||
|
||||
# ── process ─────────────────────────────────────────────────────────────
|
||||
def _chain_json_for_bridge(self):
|
||||
"""JSON mảng {path,bypass,preset_b64} — RealtimeFxChain::buildChain đọc
|
||||
đúng các field này (type/name bỏ qua)."""
|
||||
out = []
|
||||
for s in self.fx_chain:
|
||||
if not s or not s.get("path"):
|
||||
continue
|
||||
out.append({
|
||||
"path": s["path"],
|
||||
"bypass": bool(s.get("bypass")),
|
||||
"preset_b64": s.get("preset_b64") or "",
|
||||
})
|
||||
return out
|
||||
|
||||
def _spawn(self):
|
||||
exe = find_fx_bridge_exe()
|
||||
if not exe:
|
||||
raise RuntimeError("Không tìm thấy fx_vst_bridge.exe")
|
||||
job = {
|
||||
"sample_rate": self.sample_rate,
|
||||
"block_size": self.block_size,
|
||||
"fx_chain": self._chain_json_for_bridge(),
|
||||
}
|
||||
fd, self._job_path = tempfile.mkstemp(suffix=".json", prefix="fxrt_")
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(job, f)
|
||||
cmd = [exe, "--realtime-fx", self._job_path, "--shm", self.name,
|
||||
"--parent", str(os.getpid())]
|
||||
self.proc = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
|
||||
)
|
||||
|
||||
def wait_ready(self, timeout=30.0):
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
if self.proc.poll() is not None:
|
||||
raise RuntimeError(
|
||||
f"fx_vst_bridge thoát sớm (rc={self.proc.returncode})")
|
||||
if self.h.state == FXRT_STATE_READY:
|
||||
return True
|
||||
time.sleep(0.02)
|
||||
raise RuntimeError("fx_vst_bridge không sẵn sàng trong %ss" % timeout)
|
||||
|
||||
def alive(self):
|
||||
return self.proc is not None and self.proc.poll() is None
|
||||
|
||||
def close(self):
|
||||
try:
|
||||
self.h.running = 0
|
||||
except Exception:
|
||||
pass
|
||||
if self.proc is not None:
|
||||
try:
|
||||
self.proc.wait(timeout=3)
|
||||
except Exception:
|
||||
try:
|
||||
self.proc.kill()
|
||||
except Exception:
|
||||
pass
|
||||
self.proc = None
|
||||
try:
|
||||
self.shm.close()
|
||||
self.shm.unlink()
|
||||
except Exception:
|
||||
pass
|
||||
if self._job_path and os.path.exists(self._job_path):
|
||||
try:
|
||||
os.remove(self._job_path)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ── audio I/O ───────────────────────────────────────────────────────────
|
||||
def write_input(self, interleaved):
|
||||
"""Ghi 1 block stereo float32 (2048 bytes) vào input ring. Ring đầy →
|
||||
drop block cũ nhất (giữ latency thấp)."""
|
||||
arr = np.frombuffer(interleaved, dtype=np.float32)
|
||||
if arr.size != self.block_size * 2:
|
||||
return
|
||||
arr = arr.reshape(self.block_size, 2)
|
||||
with self._lock:
|
||||
h = self.h
|
||||
while True:
|
||||
if h.in_write - h.in_read < h.in_slots:
|
||||
slot = h.in_write & (h.in_slots - 1)
|
||||
bl = IN_L_OFF // 4 + slot * self.block_size
|
||||
br = IN_R_OFF // 4 + slot * self.block_size
|
||||
self._np[bl:bl + self.block_size] = arr[:, 0]
|
||||
self._np[br:br + self.block_size] = arr[:, 1]
|
||||
h.in_write += 1
|
||||
return
|
||||
h.in_read += 1 # full → drop oldest input
|
||||
|
||||
def read_output(self):
|
||||
"""Drain output ring → list bytes (mỗi block 2048 bytes interleaved)."""
|
||||
out = []
|
||||
with self._lock:
|
||||
h = self.h
|
||||
while h.out_read < h.out_write:
|
||||
slot = h.out_read & (h.out_slots - 1)
|
||||
bl = OUT_L_OFF // 4 + slot * self.block_size
|
||||
br = OUT_R_OFF // 4 + slot * self.block_size
|
||||
inter = np.empty(self.block_size * 2, dtype=np.float32)
|
||||
inter[0::2] = self._np[bl:bl + self.block_size]
|
||||
inter[1::2] = self._np[br:br + self.block_size]
|
||||
out.append(inter.tobytes())
|
||||
h.out_read += 1
|
||||
return out
|
||||
|
||||
def heartbeat_age(self):
|
||||
return time.time() - self._started_at # placeholder: engine đọc h.heartbeat
|
||||
|
||||
|
||||
# Registry toàn tiến trình (desktop app 1 user).
|
||||
_SESSIONS = {}
|
||||
_SESSIONS_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def start_session(fx_chain, sample_rate, block_size=FXRT_BLOCK):
|
||||
sess = FxRealtimeSession(fx_chain, sample_rate, block_size)
|
||||
try:
|
||||
sess.wait_ready(timeout=30)
|
||||
except Exception:
|
||||
sess.close()
|
||||
raise
|
||||
with _SESSIONS_LOCK:
|
||||
_SESSIONS[sess.name] = sess
|
||||
return sess
|
||||
|
||||
|
||||
def get_session(session_id):
|
||||
with _SESSIONS_LOCK:
|
||||
return _SESSIONS.get(session_id)
|
||||
|
||||
|
||||
def stop_session(session_id):
|
||||
with _SESSIONS_LOCK:
|
||||
sess = _SESSIONS.pop(session_id, None)
|
||||
if sess:
|
||||
sess.close()
|
||||
return sess is not None
|
||||
|
||||
|
||||
def stop_all():
|
||||
with _SESSIONS_LOCK:
|
||||
ids = list(_SESSIONS.keys())
|
||||
for sid in ids:
|
||||
stop_session(sid)
|
||||
+276
-41
@@ -63,7 +63,235 @@ def _find_default_sf2() -> str:
|
||||
return p
|
||||
return ""
|
||||
|
||||
# ── Builtin custom FX (spec fx_chain_architecture §2 Step 2: Custom FX) ─────
|
||||
# Track FX chain slots: eq / eqpro / compressor / limiter / exciter / rebalance
|
||||
# (khớp tham số WebAudio track FX modules + EQ Pro bands) + legacy chorus/reverb
|
||||
# (fx_type cũ). Chạy numpy/scipy thuần — thay pedalboard GPL; lazy import scipy
|
||||
# chỉ khi thực sự có FX (không tăng boot time).
|
||||
|
||||
def _rbj_peaking(f0, gain_db, q, sr):
|
||||
A = 10 ** (gain_db / 40.0)
|
||||
w0 = 2 * math.pi * f0 / sr
|
||||
alpha = math.sin(w0) / (2 * q)
|
||||
cw = math.cos(w0)
|
||||
a0 = 1 + alpha / A
|
||||
b0 = 1 + alpha * A
|
||||
b1 = -2 * cw
|
||||
b2 = 1 - alpha * A
|
||||
a1 = -2 * cw
|
||||
a2 = 1 - alpha / A
|
||||
return [b0 / a0, b1 / a0, b2 / a0], [1.0, a1 / a0, a2 / a0]
|
||||
|
||||
def _rbj_shelf(f0, gain_db, q, sr, low):
|
||||
A = 10 ** (gain_db / 40.0)
|
||||
w0 = 2 * math.pi * f0 / sr
|
||||
alpha = math.sin(w0) / (2 * q)
|
||||
cw = math.cos(w0)
|
||||
sA = 2 * math.sqrt(A) * alpha
|
||||
if low:
|
||||
b0 = A * ((A + 1) - (A - 1) * cw + sA)
|
||||
b1 = 2 * A * ((A - 1) - (A + 1) * cw)
|
||||
b2 = A * ((A + 1) - (A - 1) * cw - sA)
|
||||
a0 = (A + 1) + (A - 1) * cw + sA
|
||||
a1 = -2 * ((A - 1) + (A + 1) * cw)
|
||||
a2 = (A + 1) + (A - 1) * cw - sA
|
||||
else:
|
||||
b0 = A * ((A + 1) + (A - 1) * cw + sA)
|
||||
b1 = -2 * A * ((A - 1) + (A + 1) * cw)
|
||||
b2 = A * ((A + 1) + (A - 1) * cw - sA)
|
||||
a0 = (A + 1) - (A - 1) * cw + sA
|
||||
a1 = 2 * ((A - 1) - (A + 1) * cw)
|
||||
a2 = (A + 1) - (A - 1) * cw - sA
|
||||
return [b0 / a0, b1 / a0, b2 / a0], [1.0, a1 / a0, a2 / a0]
|
||||
|
||||
def _rbj_highpass(f0, q, sr):
|
||||
w0 = 2 * math.pi * f0 / sr
|
||||
alpha = math.sin(w0) / (2 * q)
|
||||
cw = math.cos(w0)
|
||||
a0 = 1 + alpha
|
||||
b0 = (1 + cw) / 2
|
||||
b1 = -(1 + cw)
|
||||
b2 = (1 + cw) / 2
|
||||
a1 = -2 * cw
|
||||
a2 = 1 - alpha
|
||||
return [b0 / a0, b1 / a0, b2 / a0], [1.0, a1 / a0, a2 / a0]
|
||||
|
||||
def _apply_biquad(buf, b, a):
|
||||
from scipy.signal import lfilter
|
||||
out = np.empty_like(buf)
|
||||
for ch in range(buf.shape[0]):
|
||||
out[ch] = lfilter(b, a, buf[ch])
|
||||
return out
|
||||
|
||||
def _apply_eq4(buf, params, sr):
|
||||
params = params or {}
|
||||
gains = [float(params.get(k, 0) or 0) for k in ("g1", "g2", "g3", "g4")]
|
||||
if not any(gains):
|
||||
return buf
|
||||
# WebAudio track 'eq': lowshelf 100Hz, peaking 800Hz Q0.7, peaking 3200Hz
|
||||
# Q1.2, highshelf 10kHz — cùng thứ tự/đáp ứng.
|
||||
for (f0, gain, q, low) in ((100, gains[0], 0.707, True),
|
||||
(800, gains[1], 0.7, False),
|
||||
(3200, gains[2], 1.2, False),
|
||||
(10000, gains[3], 0.707, False)):
|
||||
if gain == 0:
|
||||
continue
|
||||
b, a = (_rbj_shelf(f0, gain, q, sr, True) if low
|
||||
else _rbj_peaking(f0, gain, q, sr))
|
||||
buf = _apply_biquad(buf, b, a)
|
||||
return buf
|
||||
|
||||
def _apply_eqpro(buf, params, sr):
|
||||
params = params or {}
|
||||
bands = params.get("bands") or []
|
||||
if not bands:
|
||||
return buf
|
||||
amount = float(params.get("amount", 100) or 100) / 100.0
|
||||
for b in bands:
|
||||
if b.get("active") is False:
|
||||
continue
|
||||
gain = (float(b.get("gain", 0) or 0)) * amount
|
||||
if gain == 0:
|
||||
continue
|
||||
f0 = float(b.get("freq", 1000))
|
||||
q = float(b.get("q", 1.0) or 1.0)
|
||||
typ = b.get("type", "peaking")
|
||||
if typ in ("lowshelf", "highshelf"):
|
||||
bq, aq = _rbj_shelf(f0, gain, q, sr, typ == "lowshelf")
|
||||
elif typ == "highpass":
|
||||
bq, aq = _rbj_highpass(f0, q, sr)
|
||||
else: # peaking / lowpass / notch / bandpass → peaking (biquad gần đúng)
|
||||
bq, aq = _rbj_peaking(f0, gain, q, sr)
|
||||
buf = _apply_biquad(buf, bq, aq)
|
||||
return buf
|
||||
|
||||
def _apply_compressor(buf, params, sr):
|
||||
params = params or {}
|
||||
threshold_db = float(params.get("threshold", -16))
|
||||
ratio = max(1.0, float(params.get("ratio", 3) or 3))
|
||||
makeup_db = float(params.get("makeup", 0) or 0)
|
||||
makeup = 10 ** (makeup_db / 20.0)
|
||||
block = 256
|
||||
out = np.empty_like(buf)
|
||||
rel = math.exp(-1.0 / (sr * 0.25)) # release 250ms, khớp WebAudio default
|
||||
for ch in range(buf.shape[0]):
|
||||
x = buf[ch]
|
||||
n = x.shape[0]
|
||||
nblocks = (n + block - 1) // block
|
||||
gr = np.ones(n, dtype=np.float32)
|
||||
env = 0.0
|
||||
for bi in range(nblocks):
|
||||
seg = x[bi * block:(bi + 1) * block]
|
||||
peak = float(np.max(np.abs(seg))) if seg.size else 0.0
|
||||
env = max(peak, env * rel) # release smoothing giữa block
|
||||
if env > 1e-9:
|
||||
db = 20 * math.log10(env)
|
||||
over = db - threshold_db
|
||||
if over > 0:
|
||||
gain_db = -over * (1.0 - 1.0 / ratio)
|
||||
gr[bi * block:(bi + 1) * block] = (10 ** (gain_db / 20.0)) * makeup
|
||||
else:
|
||||
gr[bi * block:(bi + 1) * block] = makeup
|
||||
out[ch] = x * gr
|
||||
return out
|
||||
|
||||
def _apply_limiter(buf, params, sr):
|
||||
params = params or {}
|
||||
ceiling_db = min(0.0, float(params.get("ceiling", -1.0)))
|
||||
th = 10 ** (ceiling_db / 20.0)
|
||||
k = 1.0 / max(0.02, th)
|
||||
tanh_k = math.tanh(k)
|
||||
# Brickwall tanh soft-clip — mirror WebAudio limiter (tránh NaN của
|
||||
# DynamicsCompressor Chromium; offline dùng cùng curve). WaveShaper
|
||||
# curve chỉ định nghĩa trên [-1,1] → clamp đầu vào như WebAudio.
|
||||
x = np.clip(buf, -1.0, 1.0)
|
||||
return np.tanh(x * k) / tanh_k
|
||||
|
||||
def _apply_exciter(buf, params, sr):
|
||||
params = params or {}
|
||||
drive = float(params.get("drive", 40) or 40)
|
||||
b, a = _rbj_highpass(2000.0, 0.7, sr)
|
||||
hp = _apply_biquad(buf, b, a)
|
||||
wet = (drive / 100.0) * 0.6
|
||||
return buf + np.tanh(hp * 3.0) * wet
|
||||
|
||||
def _apply_rebalance(buf, params, sr):
|
||||
params = params or {}
|
||||
mid = 10 ** (float(params.get("mid", 0) or 0) / 20.0)
|
||||
side = 10 ** (float(params.get("side", 0) or 0) / 20.0)
|
||||
a = (mid + side) / 2.0
|
||||
b = (mid - side) / 2.0
|
||||
L = buf[0]
|
||||
R = buf[1]
|
||||
return np.stack([a * L + b * R, b * L + a * R]).astype(np.float32)
|
||||
|
||||
def _apply_builtin_fx_chain(buf, chain, sr):
|
||||
"""Serial pipeline qua các builtin custom FX slots (spec §2 Step 2: Slot 1..n
|
||||
in-place). Slot inactive/bypass → bỏ qua. vst3/carla slots = xử lý ngoài
|
||||
(bridge) → không thêm DSP ở đây."""
|
||||
for slot in chain or []:
|
||||
if isinstance(slot, str):
|
||||
slot = {"type": slot}
|
||||
if not isinstance(slot, dict):
|
||||
continue
|
||||
if slot.get("active") is False or slot.get("bypass"):
|
||||
continue
|
||||
typ = slot.get("type") or ""
|
||||
params = slot.get("params") or {}
|
||||
try:
|
||||
if typ == "eq":
|
||||
buf = _apply_eq4(buf, params, sr)
|
||||
elif typ == "eqpro":
|
||||
buf = _apply_eqpro(buf, params, sr)
|
||||
elif typ == "compressor":
|
||||
buf = _apply_compressor(buf, params, sr)
|
||||
elif typ == "limiter":
|
||||
buf = _apply_limiter(buf, params, sr)
|
||||
elif typ == "exciter":
|
||||
buf = _apply_exciter(buf, params, sr)
|
||||
elif typ == "rebalance":
|
||||
buf = _apply_rebalance(buf, params, sr)
|
||||
except Exception as e:
|
||||
logger.warning("[RenderEngine] Builtin FX slot %s failed: %s", typ, e)
|
||||
return buf
|
||||
|
||||
def _apply_legacy_chorus(buf, sr):
|
||||
total_samples = buf.shape[1]
|
||||
lfo = 0.020 + 0.005 * np.sin(2 * np.pi * 1.5 * np.arange(total_samples) / sr)
|
||||
dry = buf * 0.6
|
||||
wet = np.zeros_like(buf)
|
||||
for ch in range(2):
|
||||
indices = np.arange(total_samples) - (lfo * sr)
|
||||
indices = np.clip(indices, 0, total_samples - 1).astype(np.int32)
|
||||
wet[ch, :] = buf[ch, indices]
|
||||
return dry + wet * 0.5
|
||||
|
||||
def _apply_legacy_reverb(buf, sr):
|
||||
total_samples = buf.shape[1]
|
||||
len_ir = int(sr * 2.0)
|
||||
t_ir = np.arange(len_ir) / sr
|
||||
decay = np.exp(-t_ir / 0.5)
|
||||
ir_l = (np.random.rand(len_ir) * 2 - 1) * decay
|
||||
ir_r = (np.random.rand(len_ir) * 2 - 1) * decay
|
||||
dry = buf * 0.6
|
||||
wet = np.zeros_like(buf)
|
||||
for ch in range(2):
|
||||
ir = ir_l if ch == 0 else ir_r
|
||||
from scipy.signal import convolve
|
||||
conv = convolve(buf[ch, :], ir, mode='full')[:total_samples]
|
||||
wet[ch, :] = conv
|
||||
return dry + wet * 0.4
|
||||
|
||||
def _track_latency_samples(track):
|
||||
"""PDC (spec §3): tổng latency (samples) của track từ chain slots. Plugin
|
||||
khai qua field `latency_samples` (bridge hiện chưa báo → 0); khi có
|
||||
getLatencySamples() native, slot điền field này và engine tự align."""
|
||||
total = 0
|
||||
for chain_key in ("fx_chain", "vst_fx_chain"):
|
||||
for slot in (track.get(chain_key) or []):
|
||||
if isinstance(slot, dict) and slot.get("latency_samples"):
|
||||
total += int(slot["latency_samples"])
|
||||
return total
|
||||
|
||||
class PythonRenderEngine:
|
||||
def __init__(self, sample_rate=44100):
|
||||
@@ -102,6 +330,17 @@ class PythonRenderEngine:
|
||||
tracks = session.get("tracks", [])
|
||||
solo_ids = {t.get("id") for t in tracks if t.get("solo")}
|
||||
|
||||
# PDC pre-scan (spec §3): latency mỗi track từ chain slots; track ngắn
|
||||
# hơn bị delay khi mix để mọi track sum sample-accurate phase-aligned.
|
||||
_track_lat = {}
|
||||
_max_lat = 0
|
||||
for _t in tracks:
|
||||
if solo_ids and _t.get("id") not in solo_ids:
|
||||
continue
|
||||
_lat = _track_latency_samples(_t)
|
||||
_track_lat[_t.get("id")] = _lat
|
||||
_max_lat = max(_max_lat, _lat)
|
||||
|
||||
_channel_counter = 0
|
||||
|
||||
for track in tracks:
|
||||
@@ -347,7 +586,7 @@ class PythonRenderEngine:
|
||||
if actual_len > 0:
|
||||
track_buffer[:, start_sample:write_end] += sliced_sec[:, :actual_len]
|
||||
|
||||
# Apply Track Gain (via Pedalboard or fallback)
|
||||
# Track volume / pan / mute (spec §2 Step 3: summing gains)
|
||||
vol_db = track.get("volume_db", 0.0)
|
||||
pan = track.get("pan", 0.0)
|
||||
mute = track.get("mute", False)
|
||||
@@ -355,45 +594,29 @@ class PythonRenderEngine:
|
||||
if mute:
|
||||
continue
|
||||
|
||||
# Apply Track FX (Chorus or Reverb)
|
||||
fx_type = track.get("fx_type")
|
||||
if fx_type == "chorus":
|
||||
# Chorus: LFO delay modulation (scipy/numpy fallback — thay
|
||||
# pedalboard GPL-3.0; chất lượng thấp hơn, upgrade native DSP sau).
|
||||
# ── Track FX chain (spec §2 Step 2): serial custom FX → VST FX ──
|
||||
# Builtin DSP slots (fx_chain) → legacy fx_type (chorus/reverb) →
|
||||
# VST FX (vst_fx_chain) qua native_bridge. Mỗi slot sửa buffer
|
||||
# in-place trước khi chuyển tới slot kế — đúng Buffer_out =
|
||||
# Plugin_n(...Plugin_1(Buffer_in)).
|
||||
if track.get("fx_active", True):
|
||||
try:
|
||||
lfo = 0.020 + 0.005 * np.sin(2 * np.pi * 1.5 * np.arange(total_samples) / self.sample_rate)
|
||||
dry = track_buffer * 0.6
|
||||
wet = np.zeros_like(track_buffer)
|
||||
for ch in range(2):
|
||||
indices = np.arange(total_samples) - (lfo * self.sample_rate)
|
||||
indices = np.clip(indices, 0, total_samples - 1).astype(np.int32)
|
||||
wet[ch, :] = track_buffer[ch, indices]
|
||||
track_buffer = dry + wet * 0.5
|
||||
track_buffer = _apply_builtin_fx_chain(
|
||||
track_buffer, track.get("fx_chain"), self.sample_rate)
|
||||
except Exception as e:
|
||||
logger.warning("[RenderEngine] Fallback Chorus failed: %s", e)
|
||||
elif fx_type == "reverb":
|
||||
# Reverb: decaying-noise IR convolution (scipy — thay pedalboard GPL).
|
||||
try:
|
||||
len_ir = int(self.sample_rate * 2.0)
|
||||
t_ir = np.arange(len_ir) / self.sample_rate
|
||||
decay = np.exp(-t_ir / 0.5)
|
||||
ir_l = (np.random.rand(len_ir) * 2 - 1) * decay
|
||||
ir_r = (np.random.rand(len_ir) * 2 - 1) * decay
|
||||
logger.warning("[RenderEngine] Builtin FX chain failed: %s", e)
|
||||
fx_type = track.get("fx_type")
|
||||
if fx_type == "chorus":
|
||||
try:
|
||||
track_buffer = _apply_legacy_chorus(track_buffer, self.sample_rate)
|
||||
except Exception as e:
|
||||
logger.warning("[RenderEngine] Fallback Chorus failed: %s", e)
|
||||
elif fx_type == "reverb":
|
||||
try:
|
||||
track_buffer = _apply_legacy_reverb(track_buffer, self.sample_rate)
|
||||
except Exception as e:
|
||||
logger.warning("[RenderEngine] Fallback Reverb failed: %s", e)
|
||||
|
||||
dry = track_buffer * 0.6
|
||||
wet = np.zeros_like(track_buffer)
|
||||
for ch in range(2):
|
||||
ir = ir_l if ch == 0 else ir_r
|
||||
from scipy.signal import convolve
|
||||
conv = convolve(track_buffer[ch, :], ir, mode='full')[:total_samples]
|
||||
wet[ch, :] = conv
|
||||
track_buffer = dry + wet * 0.4
|
||||
except Exception as e:
|
||||
logger.warning("[RenderEngine] Fallback Reverb failed: %s", e)
|
||||
|
||||
# Track FX chain (VST3/builtin) — FX Rack per-track: render track
|
||||
# buffer qua native_bridge --render-fx. Ưu tiên fx_chain; fx_type
|
||||
# cũ (chorus/reverb scipy) giữ làm fallback khi không có fx_chain.
|
||||
track_fx_chain = track.get("vst_fx_chain") or []
|
||||
if track_fx_chain and not track.get("vst_fx_bypass"):
|
||||
if settings.RENDER_ENGINE == "bridge":
|
||||
@@ -424,18 +647,25 @@ class PythonRenderEngine:
|
||||
else:
|
||||
logger.warning("[RenderEngine] Track FX chain cần RENDER_ENGINE=bridge — bỏ qua")
|
||||
|
||||
# Process track volume
|
||||
# Process track volume (spec §2 Step 3: Gain_i scaling)
|
||||
gain_linear = 10 ** (vol_db / 20.0)
|
||||
processed_track = track_buffer * gain_linear
|
||||
|
||||
# Apply Track Pan
|
||||
# Apply Track Pan (spec §2 Step 3: constant-power pan law)
|
||||
if pan != 0.0:
|
||||
# Constant power panning
|
||||
theta = ((np.clip(pan, -1.0, 1.0) + 1.0) / 2.0) * (np.pi / 2.0)
|
||||
processed_track[0, :] *= np.cos(theta)
|
||||
processed_track[1, :] *= np.sin(theta)
|
||||
|
||||
# Mix track to session
|
||||
# PDC (spec §3): delay track ngắn hơn (max_lat - own_lat) để mọi
|
||||
# track sum sample-accurate. latency 0 hiện tại → no-op.
|
||||
delay = _max_lat - _track_lat.get(track.get("id"), 0)
|
||||
if delay > 0:
|
||||
shifted = np.zeros_like(processed_track)
|
||||
shifted[:, delay:] = processed_track[:, :total_samples - delay]
|
||||
processed_track = shifted
|
||||
|
||||
# Mix track to session (spec §2 Step 3: linear summing)
|
||||
session_buffer += processed_track
|
||||
|
||||
return session_buffer
|
||||
@@ -494,6 +724,11 @@ class PythonRenderEngine:
|
||||
else:
|
||||
logger.warning("[RenderEngine] Masterbus FX cần RENDER_ENGINE=bridge — bỏ qua")
|
||||
|
||||
# Master bus volume (spec §2 Step 4: master gain stage after chain)
|
||||
master_vol_db = master.get("volume_db", 0.0)
|
||||
if master_vol_db:
|
||||
master_buffer *= 10 ** (master_vol_db / 20.0)
|
||||
|
||||
# Normalization to prevent clipping
|
||||
max_peak = np.max(np.abs(master_buffer))
|
||||
if max_peak > 1.0:
|
||||
|
||||
+536
-61
@@ -1541,9 +1541,10 @@ function createTrackFxModule(type, ctx, params) {
|
||||
nodes = { gLL, gRL, gLR, gRR };
|
||||
} else if (type === 'eqpro') {
|
||||
return createEqProModule(ctx, params);
|
||||
} else if (type === 'carla') {
|
||||
// Carla Bridge = VST FX chạy NGOÀI (ứng dụng ngoài, user tự cài) — trong
|
||||
// WebAudio graph chỉ là pass-through (không thêm DSP): âm track đi thẳng.
|
||||
} else if (type === 'carla' || type === 'vst3') {
|
||||
// Carla Bridge / VST FX = xử lý NGOÀI (Carla standalone / fx_vst_bridge
|
||||
// --render-fx khi export) — trong WebAudio graph chỉ là pass-through
|
||||
// (không thêm DSP): âm track đi thẳng, đúng vị trí chain.
|
||||
input.connect(output);
|
||||
nodes = { passthrough: input };
|
||||
} else {
|
||||
@@ -1567,8 +1568,12 @@ function createTrackFxModule(type, ctx, params) {
|
||||
// keyed in a LOCAL map so the live graph is never touched. Audio clips are then
|
||||
// scheduled through node.gainNode, and the offline master bus applies the
|
||||
// mastering chain — the exported file therefore matches what you hear.
|
||||
function buildOfflineTrackNode(track, ctx, nodeMap) {
|
||||
function buildOfflineTrackNode(track, ctx, nodeMap, opts) {
|
||||
if (!track || nodeMap[track.id]) return nodeMap[track.id] || null;
|
||||
// directOut: render track RIÊNG (track FX chain, chưa qua mastering bus) —
|
||||
// dùng cho export có VST FX: mỗi track render solo → áp VST chain qua
|
||||
// fx_vst_bridge → cộng mix → mastering 1 lần trên tổng.
|
||||
const directOut = !!(opts && opts.directOut);
|
||||
const gainNode = ctx.createGain();
|
||||
const volDb = track.volumeDb ?? 0;
|
||||
gainNode.gain.setValueAtTime(volDb <= -50 ? 0 : Math.pow(10, volDb / 20), 0);
|
||||
@@ -1577,10 +1582,16 @@ function buildOfflineTrackNode(track, ctx, nodeMap) {
|
||||
const analyserNode = ctx.createAnalyser();
|
||||
analyserNode.fftSize = 2048;
|
||||
pannerNode.connect(analyserNode);
|
||||
if (!masterBus) initMasterBus(ctx);
|
||||
const route = createMasteringRoute(ctx, track, masterBus);
|
||||
analyserNode.connect(route.routeGain);
|
||||
analyserNode.connect(route.dryGain);
|
||||
const node = { gainNode, pannerNode, analyserNode, route: null, fxStopFn: null, sfEntry: null, sfOut: null, sfRouteGain: null, sfDryGain: null };
|
||||
if (directOut) {
|
||||
analyserNode.connect(ctx.destination);
|
||||
} else {
|
||||
if (!masterBus) initMasterBus(ctx);
|
||||
const route = createMasteringRoute(ctx, track, masterBus);
|
||||
analyserNode.connect(route.routeGain);
|
||||
analyserNode.connect(route.dryGain);
|
||||
node.route = route;
|
||||
}
|
||||
let fxStopFn;
|
||||
const fxEntry = ctx.createGain();
|
||||
const fxLegacyIn = ctx.createGain();
|
||||
@@ -1605,7 +1616,7 @@ function buildOfflineTrackNode(track, ctx, nodeMap) {
|
||||
} else {
|
||||
fxLegacyIn.connect(pannerNode);
|
||||
}
|
||||
const node = { gainNode, pannerNode, analyserNode, route, fxStopFn, sfEntry: null, sfOut: null, sfRouteGain: null, sfDryGain: null };
|
||||
node.fxStopFn = fxStopFn || null;
|
||||
// Soundfont (MIDI cache) chain — mirrors the live node so cached MIDI buffers
|
||||
// flow through the track FX modules + mastering exactly like playback.
|
||||
if ((track.midiItems && track.midiItems.length > 0)) {
|
||||
@@ -1622,8 +1633,12 @@ function buildOfflineTrackNode(track, ctx, nodeMap) {
|
||||
sfDryGain.gain.value = sfBypass ? 1 : 0;
|
||||
sfPan.connect(sfRouteGain);
|
||||
sfPan.connect(sfDryGain);
|
||||
sfRouteGain.connect(masterBus.input);
|
||||
sfDryGain.connect(masterBus.dryInput);
|
||||
if (directOut) {
|
||||
sfPan.connect(ctx.destination);
|
||||
} else {
|
||||
sfRouteGain.connect(masterBus.input);
|
||||
sfDryGain.connect(masterBus.dryInput);
|
||||
}
|
||||
const sfMods = fxEnabled ? fxChain.filter(m => m && m.active !== false).map(m => {
|
||||
try { return createTrackFxModule(m.type || m, ctx, m.params); } catch (e) { return null; }
|
||||
}).filter(Boolean) : [];
|
||||
@@ -1639,6 +1654,187 @@ function buildOfflineTrackNode(track, ctx, nodeMap) {
|
||||
return node;
|
||||
}
|
||||
|
||||
// ── VST FX export helpers (fx_vst_bridge --render-fx qua engine /fx-render) ──
|
||||
// Track + master VST chains là pass-through trong WebAudio (live + offline
|
||||
// graph); xử lý THẬT chỉ chạy được ở export OFFLINE qua fx_vst_bridge (one-shot
|
||||
// render, không SHM realtime — xem native_bridge/src/main_fx.cpp).
|
||||
function audioBufferToInterleaved(buf) {
|
||||
const ch = buf.numberOfChannels;
|
||||
const len = buf.length;
|
||||
const mono = ch < 2;
|
||||
const out = new Float32Array(len * (mono ? 1 : 2));
|
||||
const L = buf.getChannelData(0);
|
||||
if (mono) {
|
||||
out.set(L);
|
||||
} else {
|
||||
const R = buf.getChannelData(1);
|
||||
for (let i = 0; i < len; i++) { out[i * 2] = L[i]; out[i * 2 + 1] = R[i]; }
|
||||
}
|
||||
return { pcm: out, channels: mono ? 1 : 2, sampleRate: buf.sampleRate };
|
||||
}
|
||||
function interleavedToAudioBuffer(pcm, sampleRate, channels, ctx) {
|
||||
const n = Math.floor(pcm.length / channels);
|
||||
const buf = ctx.createBuffer(channels, n, sampleRate);
|
||||
for (let ch = 0; ch < channels; ch++) {
|
||||
const d = buf.getChannelData(ch);
|
||||
for (let i = 0; i < n; i++) d[i] = pcm[i * channels + ch];
|
||||
}
|
||||
return buf;
|
||||
}
|
||||
function mixInterleaved(a, b) {
|
||||
const n = Math.max(a.length, b.length);
|
||||
const out = new Float32Array(n);
|
||||
for (let i = 0; i < n; i++) {
|
||||
out[i] = (i < a.length ? a[i] : 0) + (i < b.length ? b[i] : 0);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
function activeVstChain(chain) {
|
||||
return (chain || []).filter(s => s && s.path && !s.bypass);
|
||||
}
|
||||
function encodeFloatWavBlob(pcm, sampleRate, channels) {
|
||||
// IEEE float32 WAV — bit-exact input cho fx_vst_bridge (không dither/quantize
|
||||
// ở đây; final export encode giữ nguyên chính sách hiện có).
|
||||
const len = Math.floor(pcm.length / channels);
|
||||
const bps = 4;
|
||||
const dataBytes = len * bps * channels;
|
||||
const buf = new ArrayBuffer(44 + dataBytes);
|
||||
const v = new DataView(buf);
|
||||
const ws = (o, s) => { for (let i = 0; i < s.length; i++) v.setUint8(o + i, s.charCodeAt(i)); };
|
||||
ws(0, 'RIFF'); v.setUint32(4, 36 + dataBytes, true); ws(8, 'WAVE');
|
||||
ws(12, 'fmt '); v.setUint32(16, 16, true); v.setUint16(20, 3, true);
|
||||
v.setUint16(22, channels, true); v.setUint32(24, sampleRate, true);
|
||||
v.setUint32(28, sampleRate * bps * channels, true);
|
||||
v.setUint16(32, bps, true); v.setUint16(34, 32, true);
|
||||
ws(36, 'data'); v.setUint32(40, dataBytes, true);
|
||||
let o = 44;
|
||||
for (let i = 0; i < pcm.length; i++) v.setFloat32(o + i * 4, pcm[i], true);
|
||||
return new Blob([buf], { type: 'audio/wav' });
|
||||
}
|
||||
|
||||
// ── Realtime VST FX (SHM bridge) — master chain LIVE qua fx_vst_bridge ─────
|
||||
// --realtime-fx: engine giữ SHM ring FxRealtimeIPC, spawn bridge; browser gửi
|
||||
// block 256 stereo float32 qua WS → engine SHM → bridge xử lý → SHM → engine →
|
||||
// WS → AudioWorklet (sf-fx-realtime.js) phát. Master: masterBus.output →
|
||||
// worklet → analyser (sau master fader, trước metering/destination — khớp thứ
|
||||
// tự offline export). Track: mỗi track 1 session/worklet splice vào CẢ 2 đường
|
||||
// chain (audio-clips + MIDI/SF) — mọi luồng track qua bridge trước main out.
|
||||
const fxRt = { active: false, sessionId: null, ws: null, node: null, inQueue: [], pumpTimer: null };
|
||||
function fxRtChain() {
|
||||
const ms = window.currentMasteringSettings;
|
||||
if (!ms || !ms.masterConnected || ms.isBypassed) return [];
|
||||
const chain = activeVstChain((ms && ms.vstFxChain) || []);
|
||||
return (chain.length > 0 && !(ms && ms.vstBypass)) ? chain : [];
|
||||
}
|
||||
async function fxRtStart() {
|
||||
if (fxRt.active) {
|
||||
// masterBus bị rebuild (recovery/export) → node thuộc graph cũ → restart.
|
||||
if (!fxRt.node || fxRt.node.context !== getAudioContext()) fxRtStop(true);
|
||||
else return;
|
||||
}
|
||||
const chain = fxRtChain();
|
||||
if (!chain.length) return;
|
||||
const ctx = getAudioContext();
|
||||
if (!ctx || !ctx.audioWorklet || !masterBus || !masterBus.output || !masterBus.analyser) return;
|
||||
const chainPayload = chain.map(s => ({ type: 'vst3', path: s.path, name: s.name || s.path, preset_b64: s.preset_b64 || '', bypass: false }));
|
||||
let st;
|
||||
try {
|
||||
st = await window.SonicAPI.fxRealtimeStart({ fx_chain: chainPayload, sample_rate: ctx.sampleRate || 44100 });
|
||||
} catch (e) {
|
||||
console.warn('[FX RT] start fail:', e);
|
||||
showToast('Không khởi động VST FX realtime: ' + (e.message || e), 'warning');
|
||||
return;
|
||||
}
|
||||
fxRt.sessionId = st.session_id;
|
||||
try {
|
||||
await ctx.audioWorklet.addModule(`${API_BASE_URL}/static/js/sf-fx-realtime.js?v=202608212200`);
|
||||
fxRt.node = new AudioWorkletNode(ctx, 'sf-fx-realtime', { numberOfInputs: 1, numberOfOutputs: 1, outputChannelCount: [2] });
|
||||
const u = new URL(API_BASE_URL);
|
||||
const wsProto = u.protocol === 'https:' ? 'wss://' : 'ws://';
|
||||
const tok = localStorage.getItem('sonic_token') || '';
|
||||
const wsUrl = `${wsProto}${u.host}${st.ws_url}${tok ? '?token=' + encodeURIComponent(tok) : ''}`;
|
||||
const ws = new WebSocket(wsUrl);
|
||||
fxRt.ws = ws;
|
||||
ws.binaryType = 'arraybuffer';
|
||||
ws.onopen = () => { fxRt.active = true; };
|
||||
ws.onmessage = (ev) => {
|
||||
if (!(ev.data instanceof ArrayBuffer) || !fxRt.node) return;
|
||||
const f = new Float32Array(ev.data);
|
||||
const half = f.length / 2;
|
||||
const l = new Float32Array(half);
|
||||
const r = new Float32Array(half);
|
||||
for (let i = 0; i < half; i++) { l[i] = f[i * 2]; r[i] = f[i * 2 + 1]; }
|
||||
fxRt.node.port.postMessage({ type: 'out', l, r });
|
||||
};
|
||||
ws.onclose = () => fxRtStop(true);
|
||||
ws.onerror = () => { try { ws.close(); } catch (e) {} };
|
||||
fxRt.node.port.onmessage = (ev) => {
|
||||
const d = ev.data;
|
||||
if (!d || d.type !== 'in' || !fxRt.ws) return;
|
||||
if (fxRt.inQueue.length > 64) fxRt.inQueue.shift();
|
||||
fxRt.inQueue.push(d.data);
|
||||
};
|
||||
// Pace input: tối đa 1 block/kỳ (~5.8ms). Gửi dồn theo burst (main thread bị
|
||||
// rAF renderFrame block khi mở MASTERING PANEL) → SHM ring 4-slot tràn → mất
|
||||
// block → ngắt quãng.
|
||||
const blockMs = Math.max(4, Math.round((256 / (ctx.sampleRate || 44100)) * 1000));
|
||||
if (fxRt.pumpTimer) clearInterval(fxRt.pumpTimer);
|
||||
fxRt.pumpTimer = setInterval(() => {
|
||||
const ws = fxRt.ws;
|
||||
if (!ws || ws.readyState !== 1 || !fxRt.inQueue.length) return;
|
||||
const d = fxRt.inQueue.shift();
|
||||
try { ws.send(d); } catch (e) { fxRt.inQueue.unshift(d); }
|
||||
}, blockMs);
|
||||
try { masterBus.output.disconnect(masterBus.analyser); } catch (e) {}
|
||||
masterBus.output.connect(fxRt.node);
|
||||
fxRt.node.connect(masterBus.analyser);
|
||||
} catch (e) {
|
||||
console.warn('[FX RT] setup fail:', e);
|
||||
fxRtStop(true);
|
||||
}
|
||||
}
|
||||
function fxRtStop(silent) {
|
||||
if (fxRt.pumpTimer) { clearInterval(fxRt.pumpTimer); fxRt.pumpTimer = null; }
|
||||
fxRt.inQueue = [];
|
||||
if (!fxRt.active && !fxRt.ws && !fxRt.node && !fxRt.sessionId) return;
|
||||
try {
|
||||
if (fxRt.node) {
|
||||
fxRt.node.disconnect();
|
||||
if (masterBus && masterBus.output && masterBus.analyser) {
|
||||
try { masterBus.output.connect(masterBus.analyser); } catch (e) {}
|
||||
}
|
||||
fxRt.node.port.onmessage = null;
|
||||
fxRt.node = null;
|
||||
}
|
||||
} catch (e) {}
|
||||
try { if (fxRt.ws) { fxRt.ws.onclose = null; fxRt.ws.onmessage = null; try { fxRt.ws.close(); } catch (e) {} fxRt.ws = null; } } catch (e) {}
|
||||
const sid = fxRt.sessionId;
|
||||
fxRt.sessionId = null;
|
||||
fxRt.active = false;
|
||||
if (sid) {
|
||||
try { window.SonicAPI.fxRealtimeStop({ session_id: sid }).catch(() => {}); } catch (e) {}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Realtime VST FX per-track (SHM bridge) — mọi luồng track qua bridge ─────
|
||||
// Mỗi track: session riêng (1 WS/SHM) cho TỪNG đường chain có âm — audio-clips
|
||||
// (path 'audio', splice fxEntry→mods→worklet→fxLegacyIn) và MIDI/SF (path 'sf',
|
||||
// splice sfEntry→mods→worklet→sfOut). Track vừa clips vừa MIDI → 2 session
|
||||
// cùng chain (khớp app: builtin FX cũng chạy instance riêng mỗi path; offline
|
||||
// export áp VST lên track mix tổng). Thứ tự: builtin FX → VST (khớp offline).
|
||||
const fxRtTracks = {}; // 'trackId:audio|sf' → { sessionId, ws, node, ready, path, spliced, chainKey }
|
||||
function fxRtTrackChain(track) {
|
||||
if (!track) return [];
|
||||
const chain = activeVstChain(track.vstFxChain || []);
|
||||
return (chain.length > 0 && !track.vstFxBypass) ? chain : [];
|
||||
}
|
||||
function fxRtTrackPaths(track) {
|
||||
const paths = [];
|
||||
if (track && ((track.clips && track.clips.length > 0) || !!track.buffer || (track.sections && track.sections.length > 0))) paths.push('audio');
|
||||
if (track && ((track.midiItems && track.midiItems.length > 0) || track.type === 'MIDI' || !!track.instrumentId)) paths.push('sf');
|
||||
return paths;
|
||||
}
|
||||
|
||||
// Track-EQ preset library (unified_fx_rack_panel.md §III.1) — band gains only.
|
||||
const TRACK_EQ_PRESETS = {
|
||||
flat: { name: 'Flat / Reset', g: [0, 0, 0, 0] },
|
||||
@@ -2268,7 +2464,6 @@ const MixerStrip = ({ track, index, onUpdateTrack, trackVuRefs }) => {
|
||||
|
||||
// ── Master Strip Console Component (from md/47_MASTER_STRIP_CONSOLE.md) ──
|
||||
const MasterStripConsole = ({ masterVolume, setMasterVolume, showMasteringModal, setShowMasteringModal, masteringSettings, setMasteringSettings, isPlaying }) => {
|
||||
const [isFxActive, setIsFxActive] = React.useState(true);
|
||||
const [isTestPlaying, setIsTestPlaying] = React.useState(false);
|
||||
const [isMuted, setIsMuted] = React.useState(false);
|
||||
const [isMono, setIsMono] = React.useState(false);
|
||||
@@ -2535,7 +2730,7 @@ const MasterStripConsole = ({ masterVolume, setMasterVolume, showMasteringModal,
|
||||
}
|
||||
},
|
||||
title: "Bật/Tắt MASTER FX CHAIN Bypass"
|
||||
}, [React.createElement("i", { className: "fa-solid fa-power-off" + (isFxActive ? " text-emerald-400" : " text-slate-500"), key: "ico" }), React.createElement("span", { key: "lbl", className: "text-[7px] font-bold" + (isFxActive ? " text-emerald-300" : " text-slate-400") }, "PWR")]),
|
||||
}, [React.createElement("i", { className: "fa-solid fa-power-off" + (isMasterActive ? " text-emerald-400" : " text-slate-500"), key: "ico" }), React.createElement("span", { key: "lbl", className: "text-[7px] font-bold" + (isMasterActive ? " text-emerald-300" : " text-slate-400") }, "PWR")]),
|
||||
React.createElement("button", { className: "btn-daw h-[18px] rounded text-slate-400 text-[8px]", title: "Trim Envelope" }, "TRIM"),
|
||||
React.createElement("button", { className: "btn-daw h-[18px] rounded text-slate-400 text-[9px]", title: "Session Info" },
|
||||
React.createElement("i", { className: "fa-solid fa-info" })
|
||||
@@ -2742,7 +2937,7 @@ const TrackStripConsole = ({ track, index, onUpdateTrack, trackVuRefs, style })
|
||||
if (onUpdateTrack) onUpdateTrack(track.id, { fxActive: next });
|
||||
// Live re-route: rebuild both FX chains (audio + SF) so EVERY item
|
||||
// (midi/section/audioclip) immediately follows the FX power state.
|
||||
if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id);
|
||||
if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id, { fxActive: next });
|
||||
},
|
||||
className: "btn-daw h-[24px] rounded flex items-center justify-center text-[8px]" + (track.fxActive !== false ? " text-emerald-400" : " text-slate-500"),
|
||||
title: "Toggle FX Power: ON (sáng) = MỌI items (MIDI/section/audioclip) qua FX Rack Panel (nút A/♪ chỉ ảnh hưởng Mastering FX Chain); OFF (tối) = bỏ toàn bộ track FX"
|
||||
@@ -11296,7 +11491,7 @@ const serializeProjectToSchema = (projectId, name, bpmVal, tracksList, subTabsLi
|
||||
fx_chain: masteringSettings.vstFxChain.map(s => ({ type: s.type || 'vst3', path: s.path || '', name: s.name || '', preset_b64: _capturedPresetByPath[s.path] || s.preset_b64 || '', bypass: !!s.bypass })),
|
||||
volume_db: (typeof masterVolume === 'number' && isFinite(masterVolume)) ? masterVolume : 0,
|
||||
mute: false,
|
||||
bypass: !!masteringSettings.vstBypass
|
||||
bypass: !masteringSettings.masterConnected || !!masteringSettings.vstBypass
|
||||
}
|
||||
} : {})
|
||||
},
|
||||
@@ -12192,7 +12387,7 @@ const FXRackModal = ({ track, onUpdateTrack, onClose }) => {
|
||||
...chain.map((m, i) => ({ kind: 'builtin', m, i })),
|
||||
...vstChain.map((s, i) => ({ kind: 'vst', s, i })),
|
||||
];
|
||||
const setVstChain = (next) => { if (onUpdateTrack) onUpdateTrack(track.id, { vstFxChain: next, vstFxBypass: vstBypass }); };
|
||||
const setVstChain = (next) => { if (onUpdateTrack) onUpdateTrack(track.id, { vstFxChain: next, vstFxBypass: vstBypass }); if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id, { vstFxChain: next, vstFxBypass: vstBypass }); };
|
||||
// Embedded VST GUI state (PLAN_MASTERBUS_VST_GUI_EMBED.md Phase 3)
|
||||
const [fxGuiOpen, setFxGuiOpenState] = React.useState(_persistFxGuiOpen); // { path, url } | null
|
||||
const setFxGuiOpen = (v) => { _persistFxGuiOpen = v; _persistFxGuiTrackId = v ? (track && track.id) : null; setFxGuiOpenState(v); };
|
||||
@@ -12262,7 +12457,7 @@ const addVst = (path) => { const fx = (fxList || []).find(f => f.path === path);
|
||||
|
||||
const setChain = (next) => {
|
||||
if (onUpdateTrack) onUpdateTrack(track.id, { fxChain: next });
|
||||
if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id);
|
||||
if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id, { fxChain: next });
|
||||
};
|
||||
const paramsOf = (m) => ({ ...(TRACK_FX_DEFAULTS[m.type] || {}), ...(m.params || {}) });
|
||||
const setParams = (idx, patch) => {
|
||||
@@ -12564,13 +12759,13 @@ const addVst = (path) => { const fx = (fxList || []).find(f => f.path === path);
|
||||
|
||||
{/* VST FX toolbar — VST3 (Ozone...) áp dụng lúc Export/Offline render (preview = âm export) */}
|
||||
<div className="flex items-center justify-between flex-wrap gap-2 shrink-0">
|
||||
<span className="text-[9px] text-slate-500 font-mono truncate">VST3 (Ozone...) — realtime browser vẫn WebAudio builtin; VST áp dụng lúc Export/Offline render</span>
|
||||
<span className="text-[9px] text-slate-500 font-mono truncate">VST3 — realtime qua bridge (SHM) khi play + áp dụng lúc Export/Offline render</span>
|
||||
<div className="flex items-center gap-2 shrink-0">
|
||||
<button onClick={previewFx} className="px-2 py-1 bg-teal-800 hover:bg-teal-700 text-teal-100 text-[10px] font-semibold rounded transition flex items-center gap-1">
|
||||
<i data-lucide="play" className="w-3 h-3"></i> Nghe thử (render offline)
|
||||
</button>
|
||||
<label className="flex items-center gap-1 text-[10px] text-slate-400 font-mono cursor-pointer">
|
||||
<input type="checkbox" checked={vstBypass} onChange={e => { const val = e.target.checked; setVstBypass(val); if (onUpdateTrack) onUpdateTrack(track.id, { vstFxBypass: val }); }} className="w-3 h-3" /> Bypass
|
||||
<input type="checkbox" checked={vstBypass} onChange={e => { const val = e.target.checked; setVstBypass(val); if (onUpdateTrack) onUpdateTrack(track.id, { vstFxBypass: val }); if (window.__rebuildTrackFxGraph) window.__rebuildTrackFxGraph(track.id, { vstFxBypass: val }); }} className="w-3 h-3" /> Bypass
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
@@ -18552,6 +18747,12 @@ const App = () => {
|
||||
if (audioCtx && masterBus) {
|
||||
toggleMasteringOnMaster(masteringSettings.masterConnected, masteringSettings.isBypassed);
|
||||
applyMasteringSettings(masteringSettings);
|
||||
// Đồng bộ realtime VST master với PWR/Bypass: WebAudio graph đã đổi theo
|
||||
// masterConnected nhưng fxRt không tự dừng → tắt PWR mà âm không đổi.
|
||||
try {
|
||||
if (fxRtChain().length) { if (!fxRt.active) fxRtStart(); }
|
||||
else if (fxRt.active) { fxRtStop(); }
|
||||
} catch (e) { console.warn('[FX RT] resync fail:', e); }
|
||||
// Re-sync live track routes: mastering ON → mọi track qua chain (♪ bị
|
||||
// override bởi effMidiBypass/effAudioBypass) — nút PWR bật/tắt phải áp
|
||||
// ngay lên node đang phát (không chờ tracks effect).
|
||||
@@ -23094,13 +23295,14 @@ const App = () => {
|
||||
// re-routes ONLY this track's fx chain (fxEntry → active modules → fxLegacyIn),
|
||||
// leaving other tracks and the master bus untouched. Called by the FX Rack
|
||||
// panel on add/remove/reorder/toggle/param change — live while playing.
|
||||
const rebuildTrackFxGraph = (trackId) => {
|
||||
const rebuildTrackFxGraph = (trackId, patch) => {
|
||||
const node = activeTrackNodesRef.current[trackId];
|
||||
if (!node || !node.fxEntry || !node.fxLegacyIn) return;
|
||||
try {
|
||||
const list = (activeTracksRef.current && activeTracksRef.current.length) ? activeTracksRef.current : tracks;
|
||||
const track = list.find(t => t.id === trackId);
|
||||
let track = list.find(t => t.id === trackId);
|
||||
if (!track) return;
|
||||
if (patch) track = { ...track, ...patch };
|
||||
node.fxEntry.disconnect();
|
||||
const fxEnabled = track.fxActive !== false;
|
||||
const chainMods = fxEnabled ? (track.fxChain || []).filter(m => m && m.active !== false).map(m => {
|
||||
@@ -23130,6 +23332,10 @@ const App = () => {
|
||||
if (node.sfPan && getAudioContext()) {
|
||||
node.sfPan.pan.setTargetAtTime((track.pan ?? 0) / 100, getAudioContext().currentTime, 0.02);
|
||||
}
|
||||
// Realtime VST FX per-track: chain mới → reconcile session/worklet
|
||||
// (start nếu chain active khi play / stop nếu chain bị bypass / re-splice
|
||||
// nếu mods đổi). VST áp SAU builtin FX — khớp offline export.
|
||||
fxRtTrackReconcile(trackId, node, track);
|
||||
} catch (e) {
|
||||
console.warn('rebuildTrackFxGraph error:', e);
|
||||
}
|
||||
@@ -23138,6 +23344,143 @@ const App = () => {
|
||||
rebuildTrackFxGraphRef.current = rebuildTrackFxGraph;
|
||||
window.__rebuildTrackFxGraph = rebuildTrackFxGraph;
|
||||
|
||||
// ── Realtime VST FX per-track (SHM bridge) — component scope ──────────────
|
||||
// Cần activeTrackNodesRef/isPlayingRef. Session giống master (fxRtStart):
|
||||
// fetch start → AudioWorkletNode 'sf-fx-realtime' → WS; WS open → splice
|
||||
// worklet vào chain hiện tại (đọc mods MỚI NHẤT — an toàn với rebuild).
|
||||
const fxRtTrackStart = async (trackId, track, path) => {
|
||||
const key = trackId + ':' + path;
|
||||
const chain = fxRtTrackChain(track);
|
||||
if (!chain.length) return;
|
||||
if (fxRtTracks[key]) {
|
||||
if (!fxRtTracks[key].node || fxRtTracks[key].node.context !== getAudioContext()) fxRtTrackStop(trackId, path, true);
|
||||
else return;
|
||||
}
|
||||
const ctx = getAudioContext();
|
||||
if (!ctx || !ctx.audioWorklet) return;
|
||||
const chainPayload = chain.map(s => ({ type: 'vst3', path: s.path, name: s.name || s.path, preset_b64: s.preset_b64 || '', bypass: false }));
|
||||
let st;
|
||||
try {
|
||||
st = await window.SonicAPI.fxRealtimeStart({ fx_chain: chainPayload, sample_rate: ctx.sampleRate || 44100 });
|
||||
} catch (e) {
|
||||
console.warn('[FX RT track] start fail', trackId, path, e);
|
||||
return;
|
||||
}
|
||||
const entry = { sessionId: st.session_id, ws: null, node: null, ready: false, path, spliced: null, chainKey: JSON.stringify(chainPayload), inQueue: [], pumpTimer: null };
|
||||
fxRtTracks[key] = entry;
|
||||
try {
|
||||
await ctx.audioWorklet.addModule(`${API_BASE_URL}/static/js/sf-fx-realtime.js?v=202608212200`);
|
||||
const node = new AudioWorkletNode(ctx, 'sf-fx-realtime', { numberOfInputs: 1, numberOfOutputs: 1, outputChannelCount: [2] });
|
||||
entry.node = node;
|
||||
const u = new URL(API_BASE_URL);
|
||||
const wsProto = u.protocol === 'https:' ? 'wss://' : 'ws://';
|
||||
const tok = localStorage.getItem('sonic_token') || '';
|
||||
const ws = new WebSocket(`${wsProto}${u.host}${st.ws_url}${tok ? '?token=' + encodeURIComponent(tok) : ''}`);
|
||||
entry.ws = ws;
|
||||
ws.binaryType = 'arraybuffer';
|
||||
ws.onopen = () => { entry.ready = true; fxRtTrackSplice(trackId, path); };
|
||||
ws.onmessage = (ev) => {
|
||||
if (!(ev.data instanceof ArrayBuffer) || !entry.node) return;
|
||||
const f = new Float32Array(ev.data);
|
||||
const half = f.length / 2;
|
||||
const l = new Float32Array(half);
|
||||
const r = new Float32Array(half);
|
||||
for (let i = 0; i < half; i++) { l[i] = f[i * 2]; r[i] = f[i * 2 + 1]; }
|
||||
entry.node.port.postMessage({ type: 'out', l, r });
|
||||
};
|
||||
ws.onclose = () => fxRtTrackStop(trackId, path, true);
|
||||
ws.onerror = () => { try { ws.close(); } catch (e) {} };
|
||||
entry.node.port.onmessage = (ev) => {
|
||||
const d = ev.data;
|
||||
if (!d || d.type !== 'in' || !entry.ws) return;
|
||||
if (entry.inQueue.length > 64) entry.inQueue.shift();
|
||||
entry.inQueue.push(d.data);
|
||||
};
|
||||
const blockMs = Math.max(4, Math.round((256 / (ctx.sampleRate || 44100)) * 1000));
|
||||
entry.pumpTimer = setInterval(() => {
|
||||
const ws = entry.ws;
|
||||
if (!ws || ws.readyState !== 1 || !entry.inQueue.length) return;
|
||||
const d = entry.inQueue.shift();
|
||||
try { ws.send(d); } catch (e) { entry.inQueue.unshift(d); }
|
||||
}, blockMs);
|
||||
} catch (e) {
|
||||
console.warn('[FX RT track] setup fail', trackId, path, e);
|
||||
fxRtTrackStop(trackId, path, true);
|
||||
}
|
||||
};
|
||||
const fxRtTrackUnsplice = (trackId, path) => {
|
||||
const entry = fxRtTracks[trackId + ':' + path];
|
||||
if (!entry || !entry.spliced || !entry.node) return;
|
||||
try {
|
||||
entry.node.disconnect();
|
||||
try { entry.spliced.tail.connect(entry.spliced.dest); } catch (e) {}
|
||||
entry.spliced = null;
|
||||
} catch (e) { console.warn('[FX RT track] unsplice fail', trackId, path, e); }
|
||||
};
|
||||
const fxRtTrackSplice = (trackId, path) => {
|
||||
const key = trackId + ':' + path;
|
||||
const entry = fxRtTracks[key];
|
||||
const node = activeTrackNodesRef.current[trackId];
|
||||
if (!entry || !entry.ready || !entry.node || !node || entry.spliced) return;
|
||||
const tail = path === 'audio'
|
||||
? (node.fxMods && node.fxMods.length ? node.fxMods[node.fxMods.length - 1].output : node.fxEntry)
|
||||
: (node.sfMods && node.sfMods.length ? node.sfMods[node.sfMods.length - 1].output : node.sfEntry);
|
||||
const dest = path === 'audio' ? node.fxLegacyIn : node.sfOut;
|
||||
if (!tail || !dest) return;
|
||||
try {
|
||||
tail.disconnect(dest);
|
||||
entry.node.connect(dest);
|
||||
tail.connect(entry.node);
|
||||
entry.spliced = { tail, dest };
|
||||
console.log('[FX RT track] spliced', trackId, path);
|
||||
} catch (e) { console.warn('[FX RT track] splice fail', trackId, path, e); }
|
||||
};
|
||||
const fxRtTrackStop = (trackId, path, silent) => {
|
||||
const key = trackId + ':' + path;
|
||||
const entry = fxRtTracks[key];
|
||||
if (!entry) return;
|
||||
if (entry.pumpTimer) { clearInterval(entry.pumpTimer); entry.pumpTimer = null; }
|
||||
entry.inQueue = [];
|
||||
try {
|
||||
fxRtTrackUnsplice(trackId, path);
|
||||
if (entry.node) { entry.node.port.onmessage = null; try { entry.node.disconnect(); } catch (e) {} entry.node = null; }
|
||||
} catch (e) {}
|
||||
try { if (entry.ws) { entry.ws.onclose = null; entry.ws.onmessage = null; try { entry.ws.close(); } catch (e) {} entry.ws = null; } } catch (e) {}
|
||||
const sid = entry.sessionId;
|
||||
delete fxRtTracks[key];
|
||||
if (sid) {
|
||||
try { window.SonicAPI.fxRealtimeStop({ session_id: sid }).catch(() => {}); } catch (e) {}
|
||||
}
|
||||
};
|
||||
const fxRtTrackReconcile = (trackId, node, track) => {
|
||||
try {
|
||||
const chain = fxRtTrackChain(track);
|
||||
const wantPaths = chain.length ? fxRtTrackPaths(track) : [];
|
||||
const prefix = trackId + ':';
|
||||
Object.keys(fxRtTracks).forEach(k => {
|
||||
if (!k.startsWith(prefix)) return;
|
||||
const p = k.slice(prefix.length);
|
||||
if (wantPaths.indexOf(p) < 0) fxRtTrackStop(trackId, p, true);
|
||||
});
|
||||
if (!isPlayingRef.current) return;
|
||||
wantPaths.forEach(p => {
|
||||
const key = trackId + ':' + p;
|
||||
const entry = fxRtTracks[key];
|
||||
if (!entry) fxRtTrackStart(trackId, track, p);
|
||||
else if (entry.ready && node) {
|
||||
fxRtTrackUnsplice(trackId, p); // chain mới (sau rebuild) — splice lại đúng mods
|
||||
fxRtTrackSplice(trackId, p);
|
||||
}
|
||||
});
|
||||
} catch (e) { console.warn('fxRtTrackReconcile error:', e); }
|
||||
};
|
||||
const fxRtTrackStopAll = () => {
|
||||
Object.keys(fxRtTracks).forEach(k => {
|
||||
const i = k.lastIndexOf(':');
|
||||
fxRtTrackStop(k.slice(0, i), k.slice(i + 1), true);
|
||||
});
|
||||
};
|
||||
|
||||
// Route the SHARED FluidSynth output through the gainNode of the single
|
||||
// audible MIDI track so its FX Rack chain / fader / pan affect the soundfont
|
||||
// instrument (unified_fx_rack_panel.md + user request). With more than one
|
||||
@@ -23546,6 +23889,8 @@ const App = () => {
|
||||
const context = getAudioContext();
|
||||
_nativePlayEpoch++; // new play — vô hiệu doHardStop đang fade (play trong cửa sổ 80ms)
|
||||
restorePendingFadeSession(); // V17: huy fade dang treo — replay khong bi dip gain
|
||||
// Realtime VST FX (master chain qua SHM bridge): bật khi play + chain active.
|
||||
try { fxRtStart(); } catch (e) { console.warn('[FX RT] start error:', e); }
|
||||
// D2: bridge active -> báo transport PLAY (bridge flush note-off cũ + đồng
|
||||
// bộ timeline; A13 C++ xử lý arg1=playheadSamples sau này).
|
||||
if (window.NativeBridgeService && window.NativeBridgeService.isBridgeConnected) {
|
||||
@@ -23578,6 +23923,14 @@ const App = () => {
|
||||
if (!isPlayable) return;
|
||||
|
||||
const gainNode = getOrCreateTrackNode(track, context);
|
||||
// Realtime VST FX per-track (SHM bridge): mọi luồng track qua bridge —
|
||||
// session/worklet cho TỪNG path có âm (audio-clips + MIDI/SF); splice
|
||||
// vào graph khi WS ready.
|
||||
try {
|
||||
const rtPaths = fxRtTrackPaths(track);
|
||||
if (rtPaths.indexOf('audio') >= 0) fxRtTrackStart(track.id, track, 'audio');
|
||||
if (rtPaths.indexOf('sf') >= 0) fxRtTrackStart(track.id, track, 'sf');
|
||||
} catch (e) { console.warn('[FX RT track] start error:', e); }
|
||||
const pannerNode = activeTrackNodesRef.current[track.id].pannerNode;
|
||||
// Re-apply route gains theo content-type NGAY tại thời điểm play — node
|
||||
// có thể STALE (tạo từ MAIN play với track cùng id — route theo data cũ)
|
||||
@@ -24203,6 +24556,10 @@ const App = () => {
|
||||
const stopAllPlayback = (fade) => {
|
||||
try {
|
||||
isPlayingRef.current = false;
|
||||
// Realtime VST FX: dừng session + khôi phục output→analyser.
|
||||
try { fxRtStop(); } catch (e) { console.warn('[FX RT] stop error:', e); }
|
||||
// Realtime VST FX per-track: dừng mọi session track + restore chain trực tiếp.
|
||||
try { fxRtTrackStopAll(); } catch (e) { console.warn('[FX RT track] stop error:', e); }
|
||||
_nativePlayEpoch++; // hủy mọi render/decode pending của lần play cũ
|
||||
// V11: hủy mọi note timer của lần play cũ (stop/seek/loop restart) — timer
|
||||
// cũ bắn vào vòng mới = NOTE_OFF cắt note đang kêu → crack (xem ref).
|
||||
@@ -26936,9 +27293,24 @@ const App = () => {
|
||||
else if (m.startTime) needDur = Math.max(needDur, m.startTime + (m.duration || 8));
|
||||
});
|
||||
});
|
||||
// Cache có thể "đủ" (chunks + duration) nhưng RỖNG — khi bridge VSTi active,
|
||||
// âm đi qua BridgeAudioNode, KHÔNG qua track.sfEntry nơi ensureMidiCapture
|
||||
// tap → chunks toàn zero → export offline ra file câm tuyệt đối. Kiểm tra
|
||||
// năng lượng cache: nếu không có tín hiệu → bounce realtime (bắt đúng âm bridge).
|
||||
const cacheHasEnergy = (c) => {
|
||||
if (!c) return false;
|
||||
const chunks = c.chunks || [];
|
||||
for (let i = 0; i < chunks.length; i++) {
|
||||
const ch = chunks[i];
|
||||
for (let j = 0; j < ch.length; j++) {
|
||||
if (Math.abs(ch[j]) > 0.0005) return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
};
|
||||
const cacheOK = midiTracks.length > 0 && midiTracks.every(t => {
|
||||
const c = midiCacheRef.current[t.id];
|
||||
return c && c.chunks.length > 0 && c.duration >= needDur - 0.3;
|
||||
return c && c.chunks.length > 0 && c.duration >= needDur - 0.3 && cacheHasEnergy(c);
|
||||
});
|
||||
if (cacheOK) {
|
||||
showToast("Dùng MIDI cache từ preview — export OFFLINE NHANH (vẫn đủ FX Rack + Mastering Chain).", "info");
|
||||
@@ -27417,6 +27789,75 @@ const App = () => {
|
||||
if (c && !loopActive && c.duration > durationLimit) durationLimit = c.duration;
|
||||
});
|
||||
const outChannels = exportSettings.channels === 'mono' ? 1 : 2;
|
||||
const scheduleTrackInto = (t, ctx, nodes, directOut) => {
|
||||
const node = buildOfflineTrackNode(t, ctx, nodes, { directOut });
|
||||
if (!node) return null;
|
||||
const clips = t.clips && t.clips.length > 0 ? t.clips : t.buffer ? [{
|
||||
id: 'default',
|
||||
buffer: t.buffer,
|
||||
startTime: t.startTime || 0,
|
||||
name: t.name,
|
||||
speed: t.speed || 1.0
|
||||
}] : [];
|
||||
clips.forEach(clip => {
|
||||
if (!clip.buffer) return;
|
||||
const source = ctx.createBufferSource();
|
||||
source.buffer = clip.buffer;
|
||||
source.playbackRate.value = clip.speed || 1.0;
|
||||
source.connect(node.gainNode);
|
||||
const clipStart = clip.startTime || 0;
|
||||
source.start(clipStart);
|
||||
});
|
||||
// MIDI preview cache → schedule through the SF chain (track FX + mastering)
|
||||
if (node.sfEntry) {
|
||||
const midiBuf = midiCacheToBuffer(t.id);
|
||||
if (midiBuf) {
|
||||
let mSrcBuf = midiBuf;
|
||||
if (loopInfo && loopInfo.loopDur) {
|
||||
const ln = Math.min(midiBuf.length, Math.max(1, Math.ceil(loopInfo.loopDur * midiBuf.sampleRate)));
|
||||
mSrcBuf = ctx.createBuffer(midiBuf.numberOfChannels, ln, midiBuf.sampleRate);
|
||||
for (let mc = 0; mc < midiBuf.numberOfChannels; mc++) mSrcBuf.copyToChannel(midiBuf.getChannelData(mc).subarray(0, ln), mc);
|
||||
}
|
||||
const mSrc = ctx.createBufferSource();
|
||||
mSrc.buffer = mSrcBuf;
|
||||
mSrc.connect(node.sfEntry);
|
||||
mSrc.start(0);
|
||||
}
|
||||
}
|
||||
return node;
|
||||
};
|
||||
// VST FX THẬT qua fx_vst_bridge (engine /fx-render): encode WAV float32 →
|
||||
// upload → bridge render → download → decode. Lỗi → giữ nguyên PCM (export
|
||||
// vẫn chạy, toast cảnh báo) — không chặn export.
|
||||
const applyVstChainToPcm = async (pcm, rate, chain, label) => {
|
||||
const active = activeVstChain(chain);
|
||||
if (!active.length) return pcm;
|
||||
try {
|
||||
const blob = encodeFloatWavBlob(pcm, rate, 2);
|
||||
const fd = new FormData();
|
||||
fd.append('file', blob, 'vst_fx_input.wav');
|
||||
const up = await fetch(`${API_AUDIO}/upload`, { method: 'POST', body: fd });
|
||||
if (!up.ok) throw new Error('upload VST input thất bại (HTTP ' + up.status + ')');
|
||||
const upData = await up.json();
|
||||
const fxChain = active.map(s => ({ type: 'vst3', path: s.path, name: s.name || s.path, preset_b64: s.preset_b64 || '', bypass: false }));
|
||||
const r = await window.SonicAPI.fxRender({ input_file_id: upData.file_id, fx_chain: fxChain, sample_rate: rate });
|
||||
if (!r || !r.url) throw new Error('fx-render không trả url');
|
||||
const resp = await fetch(`${API_BASE_URL}${r.url}`);
|
||||
if (!resp.ok) throw new Error('download VST output thất bại (HTTP ' + resp.status + ')');
|
||||
const ab = await resp.arrayBuffer();
|
||||
const dctx = new OfflineAudioContext(2, 1, rate);
|
||||
const audioBuf = await dctx.decodeAudioData(ab);
|
||||
try { if (window.SonicAPI.deleteMyFile) window.SonicAPI.deleteMyFile(upData.file_id); } catch (e) { }
|
||||
return audioBufferToInterleaved(audioBuf).pcm;
|
||||
} catch (e) {
|
||||
console.warn('[VST FX]', label, 'không áp dụng:', e);
|
||||
showToast(`VST FX "${label}" không áp dụng được: ${e.message || e} — export không có VST này.`, 'warning');
|
||||
return pcm;
|
||||
}
|
||||
};
|
||||
const trackVstChains = activeTracks.filter(t => activeVstChain(t.vstFxChain || []).length > 0);
|
||||
const masterVstChain = activeVstChain((masteringSettings && masteringSettings.vstFxChain) || []);
|
||||
const masterVstOn = masterVstChain.length > 0 && !(masteringSettings && masteringSettings.vstBypass);
|
||||
const offlineCtx = new OfflineAudioContext(outChannels, Math.ceil(targetRate * Math.max(0.1, durationLimit)), targetRate);
|
||||
// Build the FULL graph (per-track FX Rack chains + mastering chain — the
|
||||
// same DSP as playback), so the exported file matches what you hear.
|
||||
@@ -27439,43 +27880,60 @@ const App = () => {
|
||||
masterBus.output.gain.setValueAtTime(mv <= -50 ? 0 : Math.pow(10, mv / 20), 0);
|
||||
}
|
||||
} catch (e) { console.warn('offline main-out volume error:', e); }
|
||||
activeTracks.forEach(t => {
|
||||
const node = buildOfflineTrackNode(t, offlineCtx, offlineNodes);
|
||||
if (!node) return;
|
||||
const clips = t.clips && t.clips.length > 0 ? t.clips : t.buffer ? [{
|
||||
id: 'default',
|
||||
buffer: t.buffer,
|
||||
startTime: t.startTime || 0,
|
||||
name: t.name,
|
||||
speed: t.speed || 1.0
|
||||
}] : [];
|
||||
clips.forEach(clip => {
|
||||
if (!clip.buffer) return;
|
||||
const source = offlineCtx.createBufferSource();
|
||||
source.buffer = clip.buffer;
|
||||
source.playbackRate.value = clip.speed || 1.0;
|
||||
source.connect(node.gainNode);
|
||||
const clipStart = clip.startTime || 0;
|
||||
source.start(clipStart);
|
||||
});
|
||||
// MIDI preview cache → schedule through the SF chain (track FX + mastering)
|
||||
if (node.sfEntry) {
|
||||
const midiBuf = midiCacheToBuffer(t.id);
|
||||
if (midiBuf) {
|
||||
let mSrcBuf = midiBuf;
|
||||
if (loopInfo && loopInfo.loopDur) {
|
||||
const ln = Math.min(midiBuf.length, Math.max(1, Math.ceil(loopInfo.loopDur * midiBuf.sampleRate)));
|
||||
mSrcBuf = offlineCtx.createBuffer(midiBuf.numberOfChannels, ln, midiBuf.sampleRate);
|
||||
for (let mc = 0; mc < midiBuf.numberOfChannels; mc++) mSrcBuf.copyToChannel(midiBuf.getChannelData(mc).subarray(0, ln), mc);
|
||||
}
|
||||
const mSrc = offlineCtx.createBufferSource();
|
||||
mSrc.buffer = mSrcBuf;
|
||||
mSrc.connect(node.sfEntry);
|
||||
mSrc.start(0);
|
||||
}
|
||||
let renderedBuffer;
|
||||
if (trackVstChains.length > 0 || masterVstOn) {
|
||||
// Export có VST FX → render TRACK RIÊNG (không mastering) → áp track
|
||||
// VST chain qua fx_vst_bridge → cộng mix → mastering built-in 1 lần
|
||||
// trên tổng → master VST chain cuối chain. Đúng thứ tự live: track fx
|
||||
// → (sum) → master fx → out.
|
||||
let mix = null;
|
||||
for (const t of activeTracks) {
|
||||
const chain = activeVstChain(t.vstFxChain || []);
|
||||
const tctx = new OfflineAudioContext(2, Math.ceil(targetRate * Math.max(0.1, durationLimit)), targetRate);
|
||||
const tNodes = {};
|
||||
const node = scheduleTrackInto(t, tctx, tNodes, true);
|
||||
if (!node) continue;
|
||||
const tb = await tctx.startRendering();
|
||||
try { if (node.fxStopFn) node.fxStopFn(); } catch (e) { }
|
||||
let pcm = audioBufferToInterleaved(tb).pcm;
|
||||
if (chain.length) pcm = await applyVstChainToPcm(pcm, targetRate, chain, t.name || 'track');
|
||||
mix = mix ? mixInterleaved(mix, pcm) : pcm;
|
||||
}
|
||||
});
|
||||
const renderedBuffer = await offlineCtx.startRendering();
|
||||
if (!mix) mix = new Float32Array(Math.ceil(targetRate * Math.max(0.1, durationLimit)) * 2);
|
||||
const mctx = new OfflineAudioContext(2, Math.floor(mix.length / 2), targetRate);
|
||||
masterBus = null;
|
||||
initMasterBus(mctx);
|
||||
try {
|
||||
toggleMasteringOnMaster(!!(masteringSettings && masteringSettings.masterConnected), !!(masteringSettings && masteringSettings.isBypassed));
|
||||
} catch (e) { console.warn('offline mastering toggle error (vst):', e); }
|
||||
try {
|
||||
if (masterBus && masterBus.output) {
|
||||
const mv = (typeof masterVolume === 'number' && isFinite(masterVolume)) ? masterVolume : 0;
|
||||
masterBus.output.gain.setValueAtTime(mv <= -50 ? 0 : Math.pow(10, mv / 20), 0);
|
||||
}
|
||||
} catch (e) { console.warn('offline main-out volume error (vst):', e); }
|
||||
const src = mctx.createBufferSource();
|
||||
src.buffer = interleavedToAudioBuffer(mix, targetRate, 2, mctx);
|
||||
src.connect(masterBus.input);
|
||||
src.start(0);
|
||||
renderedBuffer = await mctx.startRendering();
|
||||
if (masterVstOn) {
|
||||
const conv = audioBufferToInterleaved(renderedBuffer);
|
||||
let outPcm = await applyVstChainToPcm(conv.pcm, targetRate, masterVstChain, 'master');
|
||||
if (outChannels === 1) {
|
||||
const n = Math.floor(outPcm.length / 2);
|
||||
const mono = new Float32Array(n);
|
||||
for (let i = 0; i < n; i++) mono[i] = (outPcm[i * 2] + outPcm[i * 2 + 1]) * 0.5;
|
||||
outPcm = mono;
|
||||
}
|
||||
renderedBuffer = interleavedToAudioBuffer(outPcm, targetRate, outChannels, mctx);
|
||||
}
|
||||
} else {
|
||||
activeTracks.forEach(t => {
|
||||
scheduleTrackInto(t, offlineCtx, offlineNodes, false);
|
||||
});
|
||||
renderedBuffer = await offlineCtx.startRendering();
|
||||
}
|
||||
const numExportCh = renderedBuffer.numberOfChannels;
|
||||
const exportLength = renderedBuffer.length;
|
||||
const bytesPerSample = bitDepth / 8;
|
||||
@@ -27590,9 +28048,26 @@ const App = () => {
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
// Restore the live master bus + live mastering state after offline render.
|
||||
masterBus = savedMasterBus;
|
||||
try { if (savedMasterBus) toggleMasteringOnMaster(!!(masteringSettings && masteringSettings.masterConnected), !!(masteringSettings && masteringSettings.isBypassed)); } catch (e) { }
|
||||
// Restore the LIVE master bus after offline render: initMasterBus(offlineCtx)
|
||||
// ghi đè cả `masterBus` lẫn `window.masterBus` bằng graph OfflineAudioContext
|
||||
// (state closed) → track route (window.masterBus.input) đổ vào context câm →
|
||||
// play/bounce sau export CÂM. Khôi phục bus live (giữ wiring cũ nếu còn sống,
|
||||
// ngược lại rebuild graph trên audioCtx thật) và luôn trả lại window.masterBus.
|
||||
try {
|
||||
const liveCtx = audioCtx;
|
||||
let restored = savedMasterBus;
|
||||
if (!(restored && liveCtx && restored.output && restored.output.context === liveCtx && liveCtx.state !== 'closed')) {
|
||||
masterBus = null;
|
||||
initMasterBus(liveCtx || getAudioContext());
|
||||
restored = masterBus;
|
||||
}
|
||||
masterBus = restored;
|
||||
window.masterBus = restored;
|
||||
} catch (e) {
|
||||
masterBus = savedMasterBus;
|
||||
window.masterBus = savedMasterBus;
|
||||
}
|
||||
try { toggleMasteringOnMaster(!!(masteringSettings && masteringSettings.masterConnected), !!(masteringSettings && masteringSettings.isBypassed)); } catch (e) { }
|
||||
Object.keys(offlineNodes).forEach(k => { try { if (offlineNodes[k].fxStopFn) offlineNodes[k].fxStopFn(); } catch (e) { } });
|
||||
}
|
||||
} catch (err) {
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -92,6 +92,10 @@ window.API_BASE_URL = (window.API_BASE_URL || window.location.origin).replace(/\
|
||||
midiRender: (payload) => apiRequest('/api/v1/plugins/midi-render', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
listFx: () => apiRequest('/api/v1/plugins/fx', { method: 'GET' }),
|
||||
fxRender: (payload) => apiRequest('/api/v1/plugins/fx-render', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
// Realtime VST FX qua SHM bridge (master chain): start session → WS
|
||||
// /ws/fx-realtime/{id} đẩy/kéo block audio; stop → tắt bridge.
|
||||
fxRealtimeStart: (payload) => apiRequest('/api/v1/plugins/fx-realtime/start', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
fxRealtimeStop: (payload) => apiRequest('/api/v1/plugins/fx-realtime/stop', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
// Mở native GUI của VST FX (mastering chain / FX rack) qua bridge
|
||||
openFxGui: (payload) => apiRequest('/api/v1/plugins/fx-gui', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
soundfontRender: (payload) => apiRequest('/api/v1/plugins/soundfont-render', { method: 'POST', body: JSON.stringify(payload) }),
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
// sf-fx-realtime.js — AudioWorkletProcessor nối WebAudio → SHM FX engine.
|
||||
// process() gom 128-sample quantum thành block 256 stereo float32 (khớp
|
||||
// FXRT_BLOCK trong FxRealtimeIPC.h) → postMessage cho main thread gửi WS;
|
||||
// nhận block đã xử lý từ main thread → queue → phát ra output. Queue rỗng →
|
||||
// zero fill (WS chưa kịp round-trip).
|
||||
//
|
||||
// Latency ~2 block; backlog > 32 block (~190ms) → drop block cũ nhất. Cap cao
|
||||
// hấp thụ jitter main thread (rAF renderFrame nặng khi mở MASTERING PANEL) mà
|
||||
// không drop. ponytail: thay drop bằng adaptive stretch khi cần chất lượng cao.
|
||||
class SfFxRealtimeProcessor extends AudioWorkletProcessor {
|
||||
constructor() {
|
||||
super();
|
||||
this.BLOCK = 256; // FXRT_BLOCK
|
||||
this.inL = new Float32Array(this.BLOCK);
|
||||
this.inR = new Float32Array(this.BLOCK);
|
||||
this.inCount = 0;
|
||||
this.outQueue = []; // [{l: Float32Array, r: Float32Array}]
|
||||
this.outPos = 0;
|
||||
this.port.onmessage = (e) => {
|
||||
const d = e.data;
|
||||
if (!d || d.type !== 'out' || !d.l || !d.r) return;
|
||||
this.outQueue.push({ l: d.l, r: d.r });
|
||||
while (this.outQueue.length > 32) this.outQueue.shift();
|
||||
};
|
||||
}
|
||||
process(inputs, outputs) {
|
||||
const input = inputs[0];
|
||||
const output = outputs[0];
|
||||
const n = output[0] ? output[0].length : 128;
|
||||
const outL = output[0];
|
||||
const outR = output[1];
|
||||
|
||||
// Góp input thành block 256 → gửi main thread (transferable buffer).
|
||||
if (input && input[0]) {
|
||||
const inL = input[0];
|
||||
const inR = input[1] || input[0];
|
||||
for (let i = 0; i < n; i++) {
|
||||
if (this.inCount < this.BLOCK) {
|
||||
this.inL[this.inCount] = inL[i];
|
||||
this.inR[this.inCount] = inR[i];
|
||||
}
|
||||
this.inCount++;
|
||||
if (this.inCount >= this.BLOCK) {
|
||||
const inter = new Float32Array(this.BLOCK * 2);
|
||||
for (let j = 0; j < this.BLOCK; j++) {
|
||||
inter[j * 2] = this.inL[j];
|
||||
inter[j * 2 + 1] = this.inR[j];
|
||||
}
|
||||
this.port.postMessage({ type: 'in', data: inter.buffer }, [inter.buffer]);
|
||||
this.inCount = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Phát output từ queue; rỗng → zero.
|
||||
for (let i = 0; i < n; i++) {
|
||||
if (this.outQueue.length === 0) {
|
||||
outL[i] = 0;
|
||||
if (outR) outR[i] = 0;
|
||||
continue;
|
||||
}
|
||||
const blk = this.outQueue[0];
|
||||
outL[i] = blk.l[this.outPos];
|
||||
if (outR) outR[i] = blk.r[this.outPos];
|
||||
this.outPos++;
|
||||
if (this.outPos >= this.BLOCK) {
|
||||
this.outQueue.shift();
|
||||
this.outPos = 0;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor('sf-fx-realtime', SfFxRealtimeProcessor);
|
||||
@@ -51,7 +51,7 @@
|
||||
<script src="/static/js/services/promptTemplateManager.js?v=202607281039"></script>
|
||||
<script src="/static/js/services/undoRedoEngine.js?v=202607290941"></script>
|
||||
<script src="/static/js/services/chordTheory.js?v=202608211300"></script>
|
||||
<script src="/static/js/app.precompiled.js?v=202608211300" defer></script>
|
||||
<script src="/static/js/app.precompiled.js?v=202608212200" defer></script>
|
||||
<link rel="stylesheet" href="/static/css/styles.css?v=202607271016">
|
||||
<style>
|
||||
:root {
|
||||
|
||||
@@ -98,6 +98,9 @@ _hidden = [
|
||||
'uvicorn.protocols.http.auto',
|
||||
'uvicorn.protocols.websockets',
|
||||
'uvicorn.protocols.websockets.auto',
|
||||
'websockets', # uvicorn WS protocol runtime dep — /ws/fx-realtime/{id}
|
||||
'websockets.legacy',
|
||||
'websockets.legacy.client',
|
||||
# pedalboard (GPL-3.0) DA XOA — native_bridge thay the (xem PLAN_REPLACE_PEDALBOARD_NATIVE_BRIDGE.md)
|
||||
'soundfile',
|
||||
'sf2utils',
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
# TECHNICAL SPECIFICATION: AUDIO FX PROCESSING PIPELINE
|
||||
|
||||
This document describes the Digital Signal Processing (DSP) architecture within the Audio Engine, tracing the complete path from raw sound sources (MIDI, Audio Clips) through insert FX chains to hardware outputs and offline rendering.
|
||||
|
||||
---
|
||||
|
||||
## 1. SIGNAL FLOW TOPOLOGY
|
||||
|
||||
Audio processing executes sequentially (or via a Directed Acyclic Graph - DAG) in fixed-size sample buffers (e.g., 512 samples/block).
|
||||
|
||||
```text
|
||||
[ SOUND SOURCES ON TRACK ]
|
||||
├── MIDI Item --> [ VSTi / Soundfont ] --(Rendered Audio)--> Audio Buffer
|
||||
└── Audio Clip -------------------------------------------> Audio Buffer
|
||||
│
|
||||
▼
|
||||
[ TRACK FX CHAIN ] <---------------- (Managed via Modal UI) │
|
||||
├── Slot 1: Custom FX (In-house DSP) │
|
||||
├── Slot 2: VST FX (e.g., FabFilter, Waves) │
|
||||
└── Slot n: VST FX (e.g., iZotope Ozone) │
|
||||
│
|
||||
[ AUDIO MIXER / SUMMING BUS ] <-----------------------------------+ (Summed from all tracks)
|
||||
│ (Linear Signal Summing)
|
||||
▼
|
||||
[ MASTER FX CHAIN ]
|
||||
├── Slot 1: VST EQ / Compressor (Mastering)
|
||||
└── Slot n: VST Limiter (Peak Limiting / Output Optimization)
|
||||
│
|
||||
▼
|
||||
[ MAIN OUT (FINAL DISTRIBUTION) ]
|
||||
├── 1. Realtime Preview (Direct Hardware Output via ASIO/WASAPI/WebAudio)
|
||||
└── 2. Offline Export (Rendered to .WAV, .MP3, .OGG)
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. COMPONENT DEEP DIVE
|
||||
|
||||
### Step 1: Sound Sources
|
||||
|
||||
Every track hosts source items (Clips/Regions). Prior to entering the FX Chain, source data is converted into a normalized 32-bit or 64-bit floating-point Audio Buffer ($[-1.0, +1.0]$).
|
||||
|
||||
* **Audio Clip:** Raw audio data (WAV, MP3) is read from disk or memory, decoded, and buffered directly to the track's audio buffer.
|
||||
* **MIDI Item:** MIDI events (`Note On`, `Note Off`, `Pitch Bend`) are read in real time and routed to a Virtual Instrument (VSTi or SoundFont player). The synthesizer processes these events, renders audio samples, and writes them directly to the track buffer.
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Track FX Chain
|
||||
|
||||
Performs isolated signal processing on individual tracks.
|
||||
|
||||
* **UI Layer:** Exposed via a dedicated FX Rack Modal on each track. Users can insert, remove, bypass, and reorder processing slots.
|
||||
* **Engine Architecture:** Supports two parallel plugin formats:
|
||||
* **Custom FX:** Built-in DSP algorithms compiled into the core engine (C++, WebAssembly, or DSP Scripting).
|
||||
* **3rd-Party VST/VST3 Plugins:** External commercial processors (e.g., FabFilter, Waves, iZotope). Communication occurs via the VST SDK or a dedicated process bridge using Shared Memory (SHM).
|
||||
|
||||
|
||||
* **Serial Processing Pipeline:** Track audio buffers pass sequentially from Slot 1 through Slot $n$. Each slot modifies the memory buffer in place before passing it to the subsequent plugin:
|
||||
|
||||
$$Buffer_{\text{out}} = Plugin_n(Plugin_{n-1}(...Plugin_1(Buffer_{\text{in}})...))$$
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Summing & Master FX Chain
|
||||
|
||||
Once tracks complete their individual Track FX processing, their outputs route to the main mixer:
|
||||
|
||||
* **Signal Summing:** Audio buffers from Track 1 through Track $N$ are summed linearly—applying track volume gain scaling and constant-power pan laws—into a unified stereo Master Audio Buffer:
|
||||
|
||||
$$Buffer_{\text{Master}}(t) = \sum_{i=1}^{N} Gain_i \cdot Pan_i(Buffer_i(t))$$
|
||||
|
||||
* **Master FX Chain:** The summed Master Buffer passes through global insert slots. Operates identical to the Track FX Chain, targeting master bus processing (e.g., Master Bus EQ, Multiband Compression, and Brickwall Limiting) to ensure target loudness compliance without digital clipping.
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Main Output & Final Routing
|
||||
|
||||
Exiting the Master FX Chain, the engine directs the stereo buffer based on execution mode:
|
||||
|
||||
* **Realtime Preview Mode:**
|
||||
* Audio buffers stream in fixed blocks (e.g., 256/512 samples) to low-latency driver APIs (ASIO, WASAPI, CoreAudio, or WebAudio).
|
||||
* Enables zero-latency parameter updates as users adjust UI controls.
|
||||
|
||||
|
||||
* **Offline Export Mode (Bouncing):**
|
||||
* Bypasses realtime driver clocking to process buffers at maximum CPU speed.
|
||||
* Writes output buffers sequentially into audio encoders using libraries such as `libsndfile` (WAV), `LAME` (MP3), or `libvorbis` (OGG).
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 3. STATE & LIFECYCLE MANAGEMENT
|
||||
|
||||
* **Thread Safety (Zero-Allocation Audio Thread):** Plugin parameters and structural chain updates originate on the UI Thread. The real-time Audio Thread communicates via lock-free SPSC queues or spinlocks to avoid audio dropouts (glitches) caused by thread contention or heap allocations (`malloc`).
|
||||
* **State Persistence:** Parameters across all Custom FX and VST FX are serialized into JSON or Base64 binary chunks and saved within the main DAW project file.
|
||||
* **Plugin Delay Compensation (PDC):** Lookahead processors and complex plugins introduce execution latency. The engine queries each plugin's latency (`getLatencySamples()`), calculates the cumulative delay along each track path, and delays shorter paths accordingly so all tracks sum in sample-accurate phase alignment.
|
||||
@@ -100,13 +100,12 @@ else()
|
||||
endif()
|
||||
|
||||
# ── fx_vst_bridge: VST FX host (split from daw_vst_bridge). Windows-only:
|
||||
# one-shot modes --render-fx / --scan / --open-fx-gui / --fx-gui. No SHM
|
||||
# realtime loop (the live master FX chain stays in daw_vst_bridge because
|
||||
# it processes the instrument mix in-process). ──
|
||||
# --render-fx (offline) + --realtime-fx (SHM realtime cloud-FX loop). ──
|
||||
if(WIN32)
|
||||
add_executable(fx_vst_bridge
|
||||
src/main_fx.cpp
|
||||
src/RenderFxJob.cpp
|
||||
src/RealtimeFxLoop.cpp
|
||||
src/FxGuiServer.cpp
|
||||
)
|
||||
if(VST3_SDK_TARGET)
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
// native_bridge/include/FxRealtimeIPC.h
|
||||
// SHM layout cho fx_vst_bridge --realtime-fx (VST FX xử lý realtime, cloud-FX
|
||||
// loop): engine (Python) tạo mapping, bridge mở và chạy vòng lặp audio.
|
||||
// Tất cả field u32/float (4-byte) — KHÔNG padding — mirror chính xác bằng
|
||||
// ctypes trong app/core/fx_realtime.py.
|
||||
#pragma once
|
||||
#include <cstdint>
|
||||
|
||||
#define FXRT_MAGIC 0x46585254u // "FXRT"
|
||||
#define FXRT_BLOCK 256
|
||||
#define FXRT_IN_SLOTS 4 // engine -> bridge
|
||||
#define FXRT_OUT_SLOTS 8 // bridge -> engine
|
||||
#define FXRT_STATE_STARTING 0
|
||||
#define FXRT_STATE_READY 1
|
||||
#define FXRT_STATE_ERROR 2
|
||||
|
||||
// Header 48 bytes (12 u32) — giữ thứ tự khớp fx_realtime.py.
|
||||
struct FxRealtimeHeader {
|
||||
volatile uint32_t magic;
|
||||
volatile uint32_t state;
|
||||
uint32_t sampleRate;
|
||||
uint32_t blockSize;
|
||||
volatile uint32_t running; // engine set 0 → bridge thoát
|
||||
volatile uint32_t heartbeat; // bridge tăng mỗi 100ms (watchdog)
|
||||
volatile uint32_t inWrite; // producer: engine
|
||||
volatile uint32_t inRead; // consumer: bridge
|
||||
uint32_t inSlots;
|
||||
volatile uint32_t outWrite; // producer: bridge
|
||||
volatile uint32_t outRead; // consumer: engine
|
||||
uint32_t outSlots;
|
||||
};
|
||||
|
||||
struct FxRealtimeIPC {
|
||||
FxRealtimeHeader h;
|
||||
float inL[FXRT_IN_SLOTS][FXRT_BLOCK];
|
||||
float inR[FXRT_IN_SLOTS][FXRT_BLOCK];
|
||||
float outL[FXRT_OUT_SLOTS][FXRT_BLOCK];
|
||||
float outR[FXRT_OUT_SLOTS][FXRT_BLOCK];
|
||||
};
|
||||
@@ -81,3 +81,8 @@ private:
|
||||
};
|
||||
|
||||
#endif // RENDER_FX_JOB_H
|
||||
// Realtime SHM FX loop (fx_vst_bridge --realtime-fx): engine (Python) tạo
|
||||
// mapping FxRealtimeIPC, bridge mở, đọc input ring, xử lý qua RealtimeFxChain,
|
||||
// ghi output ring. Thoát khi running==0 hoặc parent chết.
|
||||
int run_realtime_fx_loop(const std::string& jobPath, const std::string& shmName,
|
||||
uint32_t parentPid);
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
// native_bridge/src/RealtimeFxLoop.cpp
|
||||
// fx_vst_bridge --realtime-fx <job.json> --shm <name> [--parent <pid>]
|
||||
// Vòng lặp SHM realtime: engine (Python) tạo mapping FxRealtimeIPC, bridge mở,
|
||||
// đọc block input ring (engine->bridge), chạy qua RealtimeFxChain (VST3 FX,
|
||||
// SEH-guarded, in-place), ghi block output ring (bridge->engine). Heartbeat
|
||||
// 100ms riêng (watchdog). Thoát khi running==0 hoặc parent chết.
|
||||
//
|
||||
// job.json: { "sample_rate": 48000, "block_size": 256, "fx_chain": [ ... ] }
|
||||
// fx_chain: mảng {type,path,name,preset_b64,bypass} — RealtimeFxChain::setChain
|
||||
// nhận JSON mảng {path,bypass,preset_b64} (type/name bỏ qua).
|
||||
#include "RenderFxJob.h"
|
||||
#include "FxRealtimeIPC.h"
|
||||
|
||||
#ifdef _WIN32
|
||||
#ifndef NOMINMAX
|
||||
#define NOMINMAX
|
||||
#endif
|
||||
#include <windows.h>
|
||||
#endif
|
||||
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <chrono>
|
||||
#include <cstring>
|
||||
#include <fstream>
|
||||
#include <iostream>
|
||||
#include <iterator>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
|
||||
namespace {
|
||||
|
||||
struct ShmView {
|
||||
void* map = nullptr;
|
||||
void* view = nullptr;
|
||||
};
|
||||
|
||||
ShmView* openShm(const std::string& name, size_t size) {
|
||||
int wlen = MultiByteToWideChar(CP_UTF8, 0, name.c_str(), -1, nullptr, 0);
|
||||
std::wstring wname(wlen, L'\0');
|
||||
MultiByteToWideChar(CP_UTF8, 0, name.c_str(), -1, &wname[0], wlen);
|
||||
HANDLE map = OpenFileMappingW(FILE_MAP_ALL_ACCESS, FALSE, wname.c_str());
|
||||
if (!map) return nullptr;
|
||||
void* view = MapViewOfFile(map, FILE_MAP_ALL_ACCESS, 0, 0, size);
|
||||
if (!view) { CloseHandle(map); return nullptr; }
|
||||
ShmView* v = new ShmView();
|
||||
v->map = map;
|
||||
v->view = view;
|
||||
return v;
|
||||
}
|
||||
|
||||
void closeShm(ShmView* v) {
|
||||
if (!v) return;
|
||||
if (v->view) UnmapViewOfFile(v->view);
|
||||
if (v->map) CloseHandle((HANDLE)v->map);
|
||||
delete v;
|
||||
}
|
||||
|
||||
static bool parentAlive(uint32_t pid) {
|
||||
if (pid == 0) return true;
|
||||
HANDLE h = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, pid);
|
||||
if (!h) return false;
|
||||
CloseHandle(h);
|
||||
return true;
|
||||
}
|
||||
|
||||
static void sleepMs(uint32_t ms) { Sleep(ms); }
|
||||
|
||||
std::string readFile(const std::string& path) {
|
||||
std::ifstream f(path, std::ios::binary);
|
||||
return std::string((std::istreambuf_iterator<char>(f)),
|
||||
std::istreambuf_iterator<char>());
|
||||
}
|
||||
|
||||
// Tách giá trị số khỏi JSON flat: "key":<num> hoặc "key": <num>
|
||||
double jsonNumber(const std::string& s, const std::string& key, double def) {
|
||||
std::string k = "\"" + key + "\"";
|
||||
size_t p = s.find(k);
|
||||
if (p == std::string::npos) return def;
|
||||
p += k.size();
|
||||
while (p < s.size() && (s[p] == ':' || s[p] == ' ' || s[p] == '\t' ||
|
||||
s[p] == '\r' || s[p] == '\n'))
|
||||
++p;
|
||||
size_t e = p;
|
||||
while (e < s.size() && (std::isdigit((unsigned char)s[e]) || s[e] == '.' ||
|
||||
s[e] == '-' || s[e] == '+' || s[e] == 'e' ||
|
||||
s[e] == 'E'))
|
||||
++e;
|
||||
if (e == p) return def;
|
||||
try { return std::stod(s.substr(p, e - p)); } catch (...) { return def; }
|
||||
}
|
||||
|
||||
// Cắt substring mảng JSON "fx_chain":[...] (base64 không chứa [ ] nên depth
|
||||
// scan an toàn). Trả rỗng nếu không có.
|
||||
std::string extractChainJson(const std::string& s) {
|
||||
std::string k = "\"fx_chain\"";
|
||||
size_t p = s.find(k);
|
||||
if (p == std::string::npos) return "";
|
||||
p = s.find('[', p);
|
||||
if (p == std::string::npos) return "";
|
||||
size_t depth = 0, i = p;
|
||||
for (; i < s.size(); ++i) {
|
||||
if (s[i] == '[') ++depth;
|
||||
else if (s[i] == ']') { --depth; if (depth == 0) break; }
|
||||
}
|
||||
if (i >= s.size()) return "";
|
||||
return s.substr(p, i - p + 1);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int run_realtime_fx_loop(const std::string& jobPath, const std::string& shmName,
|
||||
uint32_t parentPid) {
|
||||
std::cerr << "[RealtimeFxLoop] start job=" << jobPath << " shm=" << shmName
|
||||
<< " parent=" << parentPid << std::endl;
|
||||
const std::string job = readFile(jobPath);
|
||||
if (job.empty()) {
|
||||
std::cerr << "[RealtimeFxLoop] cannot read job file" << std::endl;
|
||||
return 1;
|
||||
}
|
||||
const double srD = jsonNumber(job, "sample_rate", 44100.0);
|
||||
const uint32_t sampleRate = (uint32_t)srD;
|
||||
const double blkD = jsonNumber(job, "block_size", (double)FXRT_BLOCK);
|
||||
const uint32_t block = (uint32_t)std::max<double>(32.0, std::min<double>(blkD, (double)FXRT_BLOCK));
|
||||
const std::string chainJson = extractChainJson(job);
|
||||
if (chainJson.empty()) {
|
||||
std::cerr << "[RealtimeFxLoop] no fx_chain in job (empty chain = passthrough)" << std::endl;
|
||||
}
|
||||
|
||||
RealtimeFxChain chain;
|
||||
chain.setChain(chainJson, sampleRate, block);
|
||||
|
||||
ShmView* v = openShm(shmName, sizeof(FxRealtimeIPC));
|
||||
if (!v) {
|
||||
std::cerr << "[RealtimeFxLoop] cannot open SHM: " << shmName
|
||||
<< " (engine phải tạo trước)" << std::endl;
|
||||
return 2;
|
||||
}
|
||||
auto* ipc = static_cast<FxRealtimeIPC*>(v->view);
|
||||
|
||||
// Chờ engine khởi tạo header (magic) — tối đa 2s.
|
||||
for (int i = 0; i < 200 && ipc->h.magic != FXRT_MAGIC; ++i) sleepMs(10);
|
||||
if (ipc->h.magic != FXRT_MAGIC) {
|
||||
std::cerr << "[RealtimeFxLoop] SHM magic mismatch (engine chưa init?)" << std::endl;
|
||||
closeShm(v);
|
||||
return 2;
|
||||
}
|
||||
if (ipc->h.inSlots != FXRT_IN_SLOTS || ipc->h.outSlots != FXRT_OUT_SLOTS) {
|
||||
std::cerr << "[RealtimeFxLoop] slot count mismatch" << std::endl;
|
||||
closeShm(v);
|
||||
return 2;
|
||||
}
|
||||
ipc->h.state = FXRT_STATE_READY;
|
||||
std::cerr << "[RealtimeFxLoop] ready sr=" << sampleRate << " block=" << block
|
||||
<< " chain=" << (chainJson.empty() ? "(empty)" : "loaded") << std::endl;
|
||||
|
||||
// Heartbeat thread riêng — watchdog của engine (loop chậm không = chết).
|
||||
std::thread hb([&]() {
|
||||
while (ipc->h.running && ipc->h.state == FXRT_STATE_READY) {
|
||||
ipc->h.heartbeat++;
|
||||
sleepMs(100);
|
||||
}
|
||||
});
|
||||
|
||||
const uint32_t n = block;
|
||||
uint32_t inMask = ipc->h.inSlots - 1;
|
||||
uint32_t outMask = ipc->h.outSlots - 1;
|
||||
uint64_t processed = 0;
|
||||
while (ipc->h.running) {
|
||||
if (parentPid && !parentAlive(parentPid)) {
|
||||
std::cerr << "[RealtimeFxLoop] parent gone — exiting" << std::endl;
|
||||
break;
|
||||
}
|
||||
const uint32_t avail = ipc->h.inWrite - ipc->h.inRead;
|
||||
if (avail == 0) { sleepMs(1); continue; }
|
||||
const uint32_t slot = ipc->h.inRead & inMask;
|
||||
float L[FXRT_BLOCK], R[FXRT_BLOCK];
|
||||
std::memcpy(L, ipc->inL[slot], n * sizeof(float));
|
||||
std::memcpy(R, ipc->inR[slot], n * sizeof(float));
|
||||
ipc->h.inRead++; // consume
|
||||
chain.process(L, R, n); // SEH-guarded; chain rỗng = passthrough
|
||||
const uint32_t oslot = ipc->h.outWrite & outMask;
|
||||
std::memcpy(ipc->outL[oslot], L, n * sizeof(float));
|
||||
std::memcpy(ipc->outR[oslot], R, n * sizeof(float));
|
||||
MemoryBarrier(); // dữ liệu trước index (đúng thứ tự trên ARM64)
|
||||
ipc->h.outWrite++; // publish
|
||||
++processed;
|
||||
}
|
||||
ipc->h.state = FXRT_STATE_STARTING; // đã dừng
|
||||
hb.join();
|
||||
closeShm(v);
|
||||
std::cerr << "[RealtimeFxLoop] exit processed=" << processed << std::endl;
|
||||
return 0;
|
||||
}
|
||||
@@ -1,12 +1,15 @@
|
||||
// native_bridge/src/main_fx.cpp
|
||||
// Entry point of fx_vst_bridge.exe — VST FX host bridge (task: split FX out
|
||||
// of daw_vst_bridge). Windows only. One-shot modes only:
|
||||
// --render-fx <job.json> --in <input.wav> --out <output.wav>
|
||||
// of daw_vst_bridge). Windows only. Modes:
|
||||
// --render-fx <job.json> --in <input.wav> --out <output.wav> (offline)
|
||||
// --realtime-fx <job.json> --shm <name> [--parent <pid>] (SHM realtime)
|
||||
// --scan <dir>
|
||||
// --open-fx-gui <job.json>
|
||||
// --fx-gui <job.json>
|
||||
// No SHM realtime loop: the realtime master FX chain (control type 6) stays
|
||||
// in daw_vst_bridge because it processes the live instrument mix in-process.
|
||||
// SHM realtime loop (--realtime-fx): cloud-FX loop cho VST3 trong master FX
|
||||
// chain — browser gửi mix qua engine → SHM input ring → bridge xử lý (cùng
|
||||
// RealtimeFxChain như daw_vst_bridge master chain) → SHM output ring →
|
||||
// engine → browser. Xem FxRealtimeIPC.h + RealtimeFxLoop.cpp.
|
||||
#include "RenderFxJob.h"
|
||||
|
||||
#ifdef _WIN32
|
||||
@@ -113,6 +116,8 @@ int main(int argc, char* argv[]) {
|
||||
std::string scanDir;
|
||||
std::string openFxGuiJob;
|
||||
std::string fxGuiServerJob;
|
||||
std::string realtimeFxJob, realtimeShm;
|
||||
uint32_t realtimeParent = 0;
|
||||
for (int i = 1; i < argc; ++i) {
|
||||
if (std::strcmp(argv[i], "--render-fx") == 0 && i + 1 < argc)
|
||||
renderFxJob = argv[i + 1];
|
||||
@@ -126,6 +131,20 @@ int main(int argc, char* argv[]) {
|
||||
openFxGuiJob = argv[i + 1];
|
||||
else if (std::strcmp(argv[i], "--fx-gui") == 0 && i + 1 < argc)
|
||||
fxGuiServerJob = argv[i + 1];
|
||||
else if (std::strcmp(argv[i], "--realtime-fx") == 0 && i + 1 < argc)
|
||||
realtimeFxJob = argv[i + 1];
|
||||
else if (std::strcmp(argv[i], "--shm") == 0 && i + 1 < argc)
|
||||
realtimeShm = argv[i + 1];
|
||||
else if (std::strcmp(argv[i], "--parent") == 0 && i + 1 < argc)
|
||||
realtimeParent = (uint32_t)std::strtoul(argv[i + 1], nullptr, 10);
|
||||
}
|
||||
if (!realtimeFxJob.empty()) {
|
||||
if (realtimeShm.empty()) {
|
||||
std::cerr << "[RealtimeFxLoop] --realtime-fx requires --shm <name>"
|
||||
<< std::endl;
|
||||
return 1;
|
||||
}
|
||||
return run_realtime_fx_loop(realtimeFxJob, realtimeShm, realtimeParent);
|
||||
}
|
||||
if (!renderFxJob.empty()) {
|
||||
if (renderFxIn.empty() || renderFxOut.empty()) {
|
||||
@@ -141,6 +160,7 @@ int main(int argc, char* argv[]) {
|
||||
|
||||
std::cerr << "fx_vst_bridge: no mode. Usage:\n"
|
||||
" --render-fx <job.json> --in <input.wav> --out <output.wav>\n"
|
||||
" --realtime-fx <job.json> --shm <name> [--parent <pid>]\n"
|
||||
" --scan <dir>\n"
|
||||
" --open-fx-gui <job.json>\n"
|
||||
" --fx-gui <job.json>" << std::endl;
|
||||
|
||||
Reference in New Issue
Block a user