3dtours/SonicForgeStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: đã play được các soundfont với FluidSynth WASM

Browse Source
This commit is contained in:
3dtours
2026-07-27 15:33:20 +07:00
parent 00566e8ea6
commit b947237c3c
3 changed files with 561 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kilo/plans/1785130485229-fluidsynth-wasm-migration-plan.md
+398
View File
@@ -0,0 +1,398 @@
# Technical Analysis & Implementation Plan: SpessaSynth → FluidSynth WASM Migration
## 1. Key Findings from Codebase Investigation
### 1.1 Current State (SpessaSynth)
| Aspect | Detail |
|---|---|
| Engine | `spessasynth_lib@4.3.1` (CDN via jsDelivr + importmap) |
| File | `app/static/js/services/soundfontPlayer.js` — IIFE, `window.SonicSF` singleton |
| API surface | 10 methods: `init`, `playNote`, `stopNote`, `stopAll`, `selectInstrument`, `controllerChange`, `programChange`, `pitchBend`, `sustainActive`, `applyAITrackInstrument`, `allocateChannel`, `getChannelState`, `loadSoundFont` (internal) |
| Call sites | 45 in `app.jsx` across piano roll, MIDI keyboard, timeline playback, external MIDI input, AI track generation |
| storage | `app/static/js/services/soundfontStorage.js` — IndexedDB cache, `window.SonicSFStorage` |
| Server render | `app/core/render_engine.py` uses `pyfluidsynth` (C++ FluidSynth) |
| Server SF3 | `app/core/soundfont_converter.py` converts SF2→SF3 via `fluidsynth`/`mscore` CLI |
| API endpoint | `GET /api/v1/plugins/soundfonts/download/{sf_id}` — serves both `.sf2` and `.sf3` |
| Docker | `libfluidsynth3`, `fluidsynth` CLI installed in container |
### 1.2 Target State (FluidSynth WASM) — Corrected Spec
**CDN package**: `fluidsynth-wasm` does NOT exist on npm/CDN.
**Real package**: `@enikey87/fluidsynth-emscripten@0.1.1`
| File | Size | Purpose |
|---|---|---|
| `libfluidsynth-2.3.0-sf3.js` | 130 KB | JS loader, SF3 support, separate WASM |
| `libfluidsynth-2.3.0-sf3.wasm` | 1.7 MB | WASM binary with Ogg/SF3 support |
| `libfluidsynth-2.3.0-sf3-all-in-one.js` | 2.39 MB | JS+WASM embedded (no separate .wasm fetch) |
**CDN URL** (chosen: separate .wasm for caching):
```
https://cdn.jsdelivr.net/npm/@enikey87/fluidsynth-emscripten@0.1.1/dist/libfluidsynth-2.3.0-sf3.js
https://cdn.jsdelivr.net/npm/@enikey87/fluidsynth-emscripten@0.1.1/dist/libfluidsynth-2.3.0-sf3.wasm
```
### 1.3 CRITICAL Architecture Flaw in Spec
The spec's code (md/39_SF3_NEW.md) passes `wasmModule` and `synthPtr` via `postMessage` to AudioWorklet. **This cannot work** because:
1. AudioWorklet runs in a separate audio thread with an isolated global scope
2. `synthPtr` (a C pointer / integer) refers to WASM linear memory in the main thread's `WebAssembly.Module` instance
3. Loading `libfluidsynth-2.3.0-sf3.js` inside the worklet creates a **separate WASM instance** with its own memory
4. The spec also uses `_fluid_synth_write_float` inside `process()` — this blocks the audio thread if audio buffer underruns occur
**Correct architecture (per README):**
> "To use libfluidsynth-X.X.X.js in AudioWorklet, load it into AudioWorklet before your worklet JS file."
The FluidSynth instance must be created **inside** the AudioWorklet. MIDI events + ArrayBuffer data are sent from main thread to worklet via `port.postMessage`.
### 1.4 `playNote` Signature Complexity
Current `playNote(note, velocity, durationMs, startTime, program, destNode, channel, synthEngine)` has 8 parameters and 16 call sites. The `synthEngine` object carries `{ soundfont_id, soundfont_bank, soundfont_program }` which must be converted to FluidSynth's `bank_select` + `program_change` before `noteon`.
### 1.5 Additional `window.SonicSF` Methods Not in Spec
Must preserve in new implementation:
- `stopNote(channel, pitch)` — MIDI note-off with extra CC 64/120 cleanup
- `controllerChange(channel, controller, value)` — MIDI CC forwarding
- `programChange(channel, program)` — pure JS state tracking
- `pitchBend(channel, value)` — 14-bit bend value
- `sustainActive(channel)` — getter for sustain pedal state
- `allocateChannel(bank)` — channel allocation (percussion = ch 9)
- `getChannelState(channel)` — JS-side { bank, program, isPercussion } state
- `saveToIndexedDB / loadFromIndexedDB` — legacy storage
---
## 2. Implementation Plan
### Phase 1: Preparation (Infrastructure)
**Task 1.1 — Create worklet directory**
```
mkdir -p app/static/js/worklets/
touch app/static/js/worklets/.gitkeep
```
**Task 1.2 — Update `index.html`**
- Remove SpessaSynth importmap + module script (lines 13-25)
- Add FluidSynth WASM CDN script:
```html
<script src="https://cdn.jsdelivr.net/npm/@enikey87/fluidsynth-emscripten@0.1.1/dist/libfluidsynth-2.3.0-sf3.js"></script>
```
### Phase 2: AudioWorklet Processor
**Task 2.1 — Create `app/static/js/worklets/fluidsynth-worklet.js`**
This file runs inside AudioWorkletGlobalScope. It:
- Receives the FluidSynth WASM module (loaded via `addModule()`)
- Maintains its own `_fluid_synth` instance
- Receives commands from main thread via `port.onmessage`:
- `INIT_SYNTH` — create settings + synth, store `synthPtr`
- `LOAD_SF` — receive ArrayBuffer, write to MEMFS, call `_fluid_synth_sfload`
- `NOTE_ON` — `_fluid_synth_noteon(synthPtr, channel, pitch, velocity)`
- `NOTE_OFF` — `_fluid_synth_noteoff(synthPtr, channel, pitch)`
- `CC` — `_fluid_synth_cc(synthPtr, channel, controller, value)`
- `PROGRAM_CHANGE` — `_fluid_synth_program_change / bank_select`
- `PITCH_BEND` — `_fluid_synth_pitch_bend(synthPtr, channel, value)`
- `ALL_NOTES_OFF` / `ALL_SOUNDS_OFF`
- `SET_GAIN` — `_fluid_synth_set_gain`
- In `process(inputs, outputs)`:
- Get `synthPtr` from closure
- Call `_fluid_synth_write_float(synthPtr, bufferSize, leftPtr, 0, 1, rightPtr, 0, 1)`
- Return `true` to keep processor alive
Key constraint: The FluidSynth WASM's `_fluid_synth_write_float` needs access to the output channel Float32Array's **byteOffset** relative to the WASM heap. AudioWorklet `output` arrays are not backed by WASM memory. Two solutions:
- **A**: After rendering, copy from WASM heap Float32Array to output channels (less efficient but safe)
- **B**: Load FluidSynth WASM inside worklet, allocate output buffers inside WASM heap (complex)
**Recommendation: Solution A** — simpler and avoids memory management issues.
### Phase 3: Rewrite `soundfontPlayer.js`
**Task 3.1 — Rewrite as IIFE-compatible `window.SonicSF`**
Keep the IIFE pattern (`window.SonicSF = SonicSF`) to minimize diff in `app.jsx`. Internal implementation uses `FluidSynthWasmWorkletBridge` class.
```javascript
// Internal state
const _channels = Array.from({ length: 16 }, () => ({ bank: 0, program: 0, isPercussion: false }));
const _sustainStates = new Array(16).fill(false);
let _workletNode = null;
let _audioCtx = null;
let _initialized = false;
let _initPromise = null;
let _currentSfId = null;
let _sfHandleMap = new Map(); // sfId -> sfHandle (integer, tracked on JS side)
let _fluidModule = null; // guarded global, used before worklet takes over
```
**Key architectural decision:**
The FluidSynth instance lives in the AudioWorklet. The main thread `soundfontPlayer.js`:
1. Loads the WASM module only for FS operations (MEMFS file writing) - needed because worklet can't do `fetch()`
2. Creates an `AudioWorkletNode`, registers the worklet
3. Sends ArrayBuffer data + MIDI commands to worklet via `port.postMessage`
4. Tracks JS-side state (channel, bank, program) for methods like `getChannelState`, `allocateChannel`
**Alternative (simpler, recommended for v1):**
Run FluidSynth entirely on **main thread**, render PCM buffers in a rAF/setInterval loop, feed to a `ScriptProcessorNode` or a simple AudioWorklet that just outputs pre-rendered buffers. This avoids the AudioWorklet WASM complexity.
**Decision:** Use main-thread FluidSynth + simple AudioWorklet output. Rationale:
- Much simpler implementation
- FluidSynth `_fluid_synth_write_float` is fast enough for real-time rendering
- No need to manage two WASM instances
- Can reuse the spec's API surface directly
- MIDI events can be processed synchronously on main thread
**Architecture:**
```
Main thread:
FluidSynth Module (loaded via CDN <script> tag)
└─ _new_fluid_settings, _new_fluid_synth
└─ MEMFS: FS.writeFile('/soundfonts/...', data)
└─ _fluid_synth_sfload, _fluid_synth_noteon, etc.
└─ _fluid_synth_write_float(bufSize, ...) -> Float32Array
AudioWorklet (simple passthrough):
└─ Receives Float32Array PCM buffers via postMessage
└─ Outputs to audio destination in process()
```
### 3.2 Method Implementation Details
**`init(audioCtx)`:**
```javascript
async init(audioContext) {
this.audioCtx = audioContext;
// resolve createFluidSynthModule() from CDN
this.fluidModule = await new Promise(resolve => {
// libfluidsynth-2.3.0-sf3.js exposes createFluidSynthModule()
resolve(window.createFluidSynthModule());
});
// create settings, synth
const settingsPtr = this.fluidModule._new_fluid_settings();
this.fluidModule._fluid_settings_setnum(settingsPtr, "synth.sample-rate", this.audioCtx.sampleRate);
this.fluidModule._fluid_settings_setnum(settingsPtr, "synth.gain", 1.0);
this.synthPtr = this.fluidModule._new_fluid_synth(settingsPtr);
// create MEMFS dir
this.fluidModule.FS.mkdir('/soundfonts');
// create AudioWorklet node
this.workletNode = new AudioWorkletNode(this.audioCtx, 'fluidsynth-bridge');
this.workletNode.connect(this.audioCtx.destination);
// start render loop
this._startRenderLoop();
}
```
**`_startRenderLoop()`:**
Uses `requestAnimationFrame` or scheduler to periodically:
1. Check if worklet Node needs more data
2. Call `_fluid_synth_write_float` into a Float32Array
3. Send via `postMessage` to worklet
**`loadSoundFont(sfId)`:**
```javascript
async loadSoundFont(sfId) {
let buffer = await sfStorage.getBuffer(sfId);
if (!buffer) {
const resp = await fetch(`/api/v1/plugins/soundfonts/download/${sfId}`);
buffer = await resp.arrayBuffer();
await sfStorage.saveBuffer(sfId, buffer);
}
this.fluidModule.FS.writeFile(`/soundfonts/${sfId}.sf3`, new Uint8Array(buffer));
const sfHandle = this.fluidModule._fluid_synth_sfload(this.synthPtr, `/soundfonts/${sfId}.sf3`, 1);
// store handle
return sfHandle !== -1;
}
```
**`playNote(note, velocity, durationMs, startTime, program, destNode, channel, synthEngine)`:**
```javascript
function playNote(note, velocity, durationMs, startTime, program, destNode, channel, synthEngine) {
// Handle synthEngine (bank/program setup)
let ch = channel ?? 0;
let bank = 0, prog = 0;
if (synthEngine) {
bank = synthEngine.soundfont_bank ?? 0;
prog = synthEngine.soundfont_program ?? 0;
this.fluidModule._fluid_synth_bank_select(this.synthPtr, ch, bank);
this.fluidModule._fluid_synth_program_change(this.synthPtr, ch, prog);
} else if (program !== undefined) {
prog = program;
this.fluidModule._fluid_synth_program_change(this.synthPtr, ch, prog);
}
// Note on
this.fluidModule._fluid_synth_noteon(this.synthPtr, ch, midiPitch, midiVel);
// Schedule note off
if (durationMs > 0 && durationMs < 60000) { // skip held notes
setTimeout(() => {
this.fluidModule._fluid_synth_noteoff(this.synthPtr, ch, midiPitch);
}, durationMs);
}
}
```
**`stopNote(channel, pitch)`:**
→ `this.fluidModule._fluid_synth_noteoff(synthPtr, channel, pitch)`
Also set sustain off + all notes off (preserve current behavior).
**`stopAll()`:**
→ `_fluid_synth_all_notes_off` + `_fluid_synth_all_sounds_off` for all 16 channels.
**`controllerChange(channel, controller, value)`:**
→ `this.fluidModule._fluid_synth_cc(synthPtr, channel, controller, value)`
+ track JS-side state for `_channels[channel].bank`, `_sustainStates[channel]`.
**`pitchBend(channel, value)`:**
→ `this.fluidModule._fluid_synth_pitch_bend(synthPtr, channel, value)`.
**`selectInstrument(channel, bank, program, sfId)`:**
→ Load SF if needed, then `_fluid_synth_bank_select` + `_fluid_synth_program_change`.
**`sustainActive(channel)`:**
→ Return `_sustainStates[channel]` (tracked by `controllerChange` for CC 64).
**`applyAITrackInstrument(bank, program, synthEngine)`:**
→ Same as current: allocate channel, call `selectInstrument` with `synthEngine.soundfont_id`.
### Phase 4: Bridge AudioWorklet
**Task 4.1 — Create `app/static/js/worklets/fluidsynth-bridge.js`**
Simple AudioWorkletProcessor that receives pre-rendered PCM buffers:
```javascript
class FluidSynthBridge extends AudioWorkletProcessor {
constructor() {
super();
this.audioQueue = [[], []]; // L/R channel queues
this.port.onmessage = (e) => {
if (e.data.type === 'PCM_FRAME') {
this.audioQueue[0].push(...e.data.left);
this.audioQueue[1].push(...e.data.right);
}
};
}
process(inputs, outputs) {
const out = outputs[0];
if (!out) return true;
const len = out[0].length;
const left = this.audioQueue[0].splice(0, len);
const right = this.audioQueue[1].splice(0, len);
// Fill output; zero-fill if buffer underrun
for (let i = 0; i < len; i++) {
out[0][i] = i < left.length ? left[i] : 0;
out[1][i] = i < right.length ? right[i] : 0;
}
return true;
}
}
registerProcessor('fluidsynth-bridge', FluidSynthBridge);
```
### Phase 5: Optimize Render Loop
**Task 5.1 — Implement ring-buffer approach**
The render loop must balance latency vs. buffer underruns:
- Render ~512 samples per frame (≈11.6ms at 44.1kHz)
- Queue 3 frames ahead (≈35ms buffer → safe against GC pauses)
- Use `AudioWorkletNode.port.postMessage` with `transferable: true` for zero-copy
Alternative: Use SharedArrayBuffer for lock-free ring buffer (requires COOP/COEP headers).
### Phase 6: Remove SpessaSynth from index.html
**Task 6.1 — Clean up CDN imports**
- Remove: `<script type="importmap">` block (lines 13-19)
- Remove: SpessaSynth module script (lines 20-25)
- Remove: `spessasynth_core` importmap entry
### Phase 7: API Endpoint for WASM asset (optional)
If self-hosting is preferred over CDN, serve `libfluidsynth-2.3.0-sf3.js` and `.wasm` from static files directory.
---
## 3. Files to Modify/Create
| Action | File | Description |
|---|---|---|
| **CREATE** | `app/static/js/worklets/fluidsynth-bridge.js` | Bridge AudioWorkletProcessor (PCM passthrough) |
| **MODIFY** | `app/static/js/services/soundfontPlayer.js` | Rewrite engine from SpessaSynth to FluidSynth WASM; preserve `window.SonicSF` API surface |
| **MODIFY** | `app/templates/index.html` | Replace SpessaSynth CDN imports with `@enikey87/fluidsynth-emscripten` CDN |
| **MODIFY** | `app/static/js/services/soundfontStorage.js` | Minor: expose `sfStorage` as named export + `window.SonicSFStorage` (likely no change needed) |
| **NO CHANGE** | `app/static/js/app.jsx` | Zero changes — `window.SonicSF` API surface preserved |
| **NO CHANGE** | `app/api/v1/plugins.py` | Download endpoint already serves `.sf2`/`.sf3` correctly |
| **NO CHANGE** | `app/core/render_engine.py` | Already uses pyfluidsynth (server-side) |
| **NO CHANGE** | `app/core/soundfont_converter.py` | Already converts SF2→SF3 |
| **NO CHANGE** | Dockerfile | Already has `libfluidsynth3`, `fluidsynth` CLI |
---
## 4. API Signature Mapping
| Old (SpessaSynth) | New (FluidSynth WASM) | Notes |
|---|---|---|
| `_synthInstance.noteOn(ch, pitch, vel)` | `_fluid_synth_noteon(ptr, ch, pitch, vel)` | Direct C function |
| `_synthInstance.noteOff(ch, pitch)` | `_fluid_synth_noteoff(ptr, ch, pitch)` | Direct C function |
| `_synthInstance.controllerChange(ch, cc, val)` | `_fluid_synth_cc(ptr, ch, cc, val)` | Direct C function |
| `_synthInstance.programChange(ch, prog)` | `_fluid_synth_program_change(ptr, ch, prog)` | Direct C function |
| `_synthInstance.pitchBend(ch, val)` | `_fluid_synth_pitch_bend(ptr, ch, val)` | Direct C function |
| `soundBankManager.addSoundBank(buf, id)` | `FS.writeFile(path, uint8arr)` + `_fluid_synth_sfload(ptr, path, 1)` | MEMFS-based |
| `soundBankManager.soundBankList` | JS-side `_sfHandleMap` (Map) | Track manually |
| `controllerChange(ch, 120, 0)` (all sound off) | `_fluid_synth_all_sounds_off(ptr, ch)` | Direct C function |
| `controllerChange(ch, 123, 0)` (all notes off) | `_fluid_synth_all_notes_off(ptr, ch)` | Direct C function |
---
## 5. Risk Assessment
| Risk | Impact | Mitigation |
|---|---|---|
| FluidSynth WASM CPU usage causes audio glitches | Medium | Use triple-buffered rendering at 512-sample blocks; monitor with `performance.now()` |
| `_fluid_synth_write_float` blocks main thread | Medium | Move rendering to a separate rAF loop, not synchronized to audio callback |
| SF3 loading latency on first load | Low | IndexedDB caching already in place; loading ≈2-5MB via MEMFS is sub-100ms |
| CDN availability for `@enikey87/fluidsynth-emscripten` | Low | Package is 17.9MB unpacked; fallback host via jsDelivr + self-host option |
| Backward compatibility with existing MIDI sessions | Medium | All `window.SonicSF` methods preserved with same signatures; `synthEngine` object handled correctly |
---
## 6. Validation
```bash
# 1. Build frontend
cd /home/locpham/SonicForgeStudio
cd app/static/js && npx babel app.jsx --config-file ../../babel.config.json -o app.precompiled.js
# 2. Start server
cd /home/locpham/SonicForgeStudio
docker compose up -d
# 3. Browser tests
# - Open DevTools → check `window.createFluidSynthModule` exists
# - Load app → check `SonicSF.init(audioCtx)` creates FluidSynth synth
# - Select SoundFont instrument → check MEMFS write + sfload succeeds
# - Play piano roll notes → hear authentic FluidSynth output
# - Play timeline → notes play with correct timing
# - External MIDI controller → CC, pitch bend, note on/off work
# - Transport stop → all notes stop immediately
# - Sustain pedal → notes sustain on CC 64 >= 64
# - Stress: play 50+ simultaneous notes → no stuck notes
```
**Server-side parity validation:**
- Render a project with server (`/api/v1/plugins/render`)
- Play same project in client browser
- Compare WAV spectrograms: should be identical (same C++ FluidSynth core)
---
## 7. Open Questions for User
1. **Self-host WASM vs CDN?** — `@enikey87/fluidsynth-emscripten` from jsDelivr (5.9MB) or serve from `app/static/` folder. CDN reduces server load but requires internet. If user wants offline-capable, suggest self-host.
2. **Separate WASM file vs all-in-one?** — `libfluidsynth-2.3.0-sf3.js` (130KB) + `libfluidsynth-2.3.0-sf3.wasm` (1.7MB) allow browser to cache WASM separately. Or `libfluidsynth-2.3.0-sf3-all-in-one.js` (2.39MB) single file but no caching benefit. Recommend separate files.
3. **Main-thread render vs worklet-own-synth?** — The spec's worklet-own-synth approach is theoretically ideal but complex (MIDI events + file data must cross thread boundary). Recommend v1 as main-thread render + simple bridge worklet. Can optimize later.
TESTING.md
+163
View File
@@ -124,6 +124,169 @@ celery -A app.tasks.worker.celery_app worker --loglevel=info
4. File sẽ download tự động 4. File sẽ download tự động
5. Mở file bằng audio player để kiểm tra 5. Mở file bằng audio player để kiểm tra
## 🎹 FluidSynth WASM Migration — Manual Test Plan
### Môi trường
- Mở DevTools Console (F12) → Tab Console (bật `Verbose` để thấy `[SonicSF]` logs)
- Tab Network: filter `fluidsynth`, `sf3`, `.wasm`
### Test A: FluidSynth WASM Load & Init
| Step | Action | Expected Result |
|------|--------|----------------|
| A1 | Mở http://localhost:8000 | Console: `[FluidSynth] Loaded from: https://cdn.jsdelivr.net/...` |
| A2 | Kiểm tra Network tab | `.wasm` file tải thành công (status 200) |
| A3 | Check window.__FluidSynthModuleFactory | `typeof window.__FluidSynthModuleFactory === 'function'` |
| A4 | Tương tác với app (click vào DAW) | Console: `[SonicSF] FluidSynth WASM Engine initialized.` |
| A5 | Kiểm tra AudioWorklet | Console: `Worklet reg success` hoặc check `audioWorklet` trong Application tab |
### Test B: SoundFont Loading
| Step | Action | Expected Result |
|------|--------|----------------|
| B1 | Mở Plugin Manager → tab SoundFont | Danh sách SoundFont hiển thị |
| B2 | Chọn 1 SoundFont instrument (vd: Piano) | Console: `[SonicSF] SoundFont loaded: <sfId>` |
| B3 | Kiểm tra Network tab | Request `download/<sfId>` status 200 |
| B4 | Chuyển đổi instrument khác (vd: Violin) | Console: `Bank Select + Program Change` (nếu cùng SF, không tải lại) |
| B5 | Load SoundFont có loop samples (vd: Tremolo Strings, Pad, Synth) | loadSoundFont success, không lỗi |
### Test C: Piano Roll Playback
| Step | Action | Expected Result |
|------|--------|----------------|
| C1 | Mở Piano Roll tab | Grid hiển thị notes |
| C2 | Click vào 1 note trên grid | Note phát ra → âm thanh giống nhạc cụ thật (không phải oscillator beep) |
| C3 | Click và kéo thả chuột trên grid → draw note mới | Âm thanh phát ngay lập tức |
| C4 | Scroll wheel trên piano roll notes | Các note scroll qua phát âm thanh preview |
| C5 | Vẽ note dài (full measure) | Note kéo dài đúng độ dài, không bị tắt giữa chừng |
### Test D: MIDI Keyboard (Virtual & Hardware)
| Step | Action | Expected Result |
|------|--------|----------------|
| D1 | Click vào phím đàn virtual (piano keybed) | Note phát ra = âm thanh instrument đúng |
| D2 | Kéo chuột ngang trên keybed | Các note phát liên tục, glide không bị stuck |
| D3 | Kết nối MIDI keyboard qua WebMIDI | Console: `MIDI access granted` |
| D4 | Nhấn phím trên MIDI keyboard | Note phát ra ngay, không delay |
| D5 | Nhả phím MIDI | Note tắt ngay (không stuck, không sustain dài) |
| D6 | Sustain pedal (CC 64) | Nhấn pedal → notes sustain; nhả → notes release |
| D7 | Pitch bend wheel | Cao độ thay đổi real-time |
| D8 | Modulation wheel (CC 1) | Âm thanh thay đổi (nếu instrument hỗ trợ) |
### Test E: Timeline Playback (MIDI Tracks)
| Step | Action | Expected Result |
|------|--------|----------------|
| E1 | Tạo track MIDI mới | Track được tạo |
| E2 | Gán SoundFont instrument cho track | Console log bank/program change |
| E3 | Thêm MIDI notes vào track, click Play | Notes phát đúng pitch, đúng thời điểm, đúng instrument |
| E4 | Click Pause → Play | Nhạc tiếp tục từ vị trí pause |
| E5 | Click Stop | Tất cả notes tắt ngay lập tức |
| E6 | Seek playhead → Play | Play từ vị trí mới, notes cũ tắt |
| E7 | Set loop region → Play | Loop playback hoạt động |
| E8 | Chuyển track instrument khác → Play | Âm thanh thay đổi theo instrument mới |
### Test F: Tremolo/Sustain/Loop Instrument Stress Test
| Step | Action | Expected Result |
|------|--------|----------------|
| F1 | Chọn Tremolo Strings (GM#44) | Load SF thành công |
| F2 | Play note → nhanh chóng NoteOff | **QUAN TRỌNG**: Note tắt ngay, không bị stuck loop |
| F3 | Play nhiều note liên tiếp (staccato) | Mỗi note tắt hẳn trước khi note kế phát |
| F4 | Chọn Saxophone (GM#65-67) | Load SF thành công |
| F5 | Play note giữ 3s → NoteOff | Saxophone release envelope chạy đúng, không stuck |
| F6 | Chọn Pad/Synth (GM#88-95) | Các instrument loop dài không bị stuck |
| F7 | Play 10+ notes cùng lúc → Stop All | Tất cả notes tắt ngay |
| F8 | **So sánh**: Test F1-F7 cũ: SpessaSynth bị stuck notes cần CC120+noteOn+post. FluidSynth WASM: chỉ cần noteOff thường. | FluidSynth handle loop Gen 54 đúng spec |
### Test G: Multi-SoundFont Switching
| Step | Action | Expected Result |
|------|--------|----------------|
| G1 | Load SoundFont A (vd: GeneralUser) | Handle ID log |
| G2 | Chuyển track sang instrument từ SF A | SF A active |
| G3 | Tạo track 2, load SoundFont B (vd: SGM) | SF B load vào MEMFS |
| G4 | Play track 1 (SF A) + track 2 (SF B) | Cả 2 soundfont phát đồng thời, mỗi track instrument đúng |
| G5 | Unload SF A, load SF C | SF A đã unload, SF C active |
### Test H: Transport Controls
| Step | Action | Expected Result |
|------|--------|----------------|
| H1 | Đang play → click Stop | Console: `FluidSynth: All notes stopped.` |
| H2 | Play với nhiều notes đang vang → Stop | Âm thanh tắt ngay lập tức (CC 120 all sound off) |
| H3 | Play → Pause → Seek → Play | Seek không bị stuck notes |
| H4 | Play → Reload trang | Audio context mới, FluidSynth init lại |
### Test I: Fallback Behavior
| Step | Action | Expected Result |
|------|--------|----------------|
| I1 | Chặn CDN request (DevTools → Network → Offline) | `fluidsynthLoader.js` detect localhost → vẫn dùng CDN? Set `window.__FLUIDSYNTH_CDN` = null |
| I2 | Nếu FluidSynth init fail | Console: `FluidSynth init failed`. Fallback oscillator hoạt động (âm beep) |
| I3 | Nếu loadSoundFont fail (network down) | Console: `SoundFont not found`. Fallback oscillator cho note preview |
### Test J: Memory & Performance
| Step | Action | Expected Result |
|------|--------|----------------|
| J1 | Load SF lần đầu | Network download + MEMFS write + sfload |
| J2 | Load lại SF lần 2 (đã cache IndexedDB) | `_sfHandleMap.has(sfId)` → true, skip download |
| J3 | Check Performance tab (DevTools) | `_fluid_synth_write_float` không block main thread > 5ms |
| J4 | Play liên tục 5 phút | Không memory leak, không audio glitch |
| J5 | Load SF 3-4MB (SGM v2.01) | MEMFS write + sfload < 500ms |
### Test K: Audio Parity (Client vs Server)
| Step | Action | Expected Result |
|------|--------|----------------|
| K1 | Tạo project với MIDI notes + SoundFont | Client preview âm thanh |
| K2 | Export WAV server-side (Render) | Server dùng pyfluidsynth (C++ core) |
| K3 | **So sánh** WAV export vs Client preview | Giống nhau 100% (cùng FluidSynth engine) |
| K4 | Test với SF3 files | Cả client (FluidSynth WASM) và server (pyfluidsynth) đều xử lý SF3 |
### Test L: Regression — Tính năng không thay đổi
| Step | Action | Expected Result |
|------|--------|----------------|
| L1 | AI track generation | `applyAITrackInstrument` works → bank/program change |
| L2 | Audio file playback | Không ảnh hưởng (vẫn dùng AudioEngine cũ) |
| L3 | VST instrument tracks | Không ảnh hưởng (dùng VST engine riêng) |
| L4 | Upload/download file | Không thay đổi |
| L5 | Multi-track mix | Không thay đổi |
### Test Environment Setup
```bash
# 1. Start server
cd /home/locpham/SonicForgeStudio
docker compose up -d --build
# 2. Clear browser cache trước khi test lần đầu (cache-bust version đã update)
# Chrome: DevTools → Network → Disable cache (khi DevTools mở)
# 3. Kiểm tra console logs
# Mở DevTools Console, filter: [SonicSF] [FluidSynth]
# 4. Force re-download SF (xóa IndexedDB cache nếu cần)
# Application → IndexedDB → DAW_SoundFont_Cache → Clear
```
### Checklist
- [ ] A1-A5: FluidSynth WASM load + init
- [ ] B1-B5: SoundFont load + switch (nhiều SF)
- [ ] C1-C5: Piano roll note play
- [ ] D1-D8: MIDI keyboard (virtual + hardware)
- [ ] E1-E8: Timeline playback
- [ ] F1-F8: **Tremolo/Sustain loop stress** — key test
- [ ] G1-G5: Multi-SoundFont switching
- [ ] H1-H4: Transport controls (stop, seek)
- [ ] I1-I3: Fallback oscillator
- [ ] J1-J5: Memory & performance
- [ ] K1-K4: Audio parity client vs server
- [ ] L1-L5: Regression (features không thay đổi)
## 🐛 Known Issues ## 🐛 Known Issues
### Docker Environment ### Docker Environment
app/storage/sonicforge.db
BIN
View File
Binary file not shown.