69 Commits

Author SHA1 Message Date
3dtours 2fed3eee0b fix: sửa lỗi snap to scale trong piano roll tab 2026-07-25 12:44:08 +07:00
3dtours eaedfebdf0 fix: snap to scale now works on initial click + use refs
- Initial click note pitch now snaps when snapToScale is on
- Added snapToScaleRef and selectedScaleRef for stale-closure safety
- All brush-draw snap paths (mousedown + mousemove) use refs
2026-07-25 12:42:50 +07:00
3dtours f5bc0a8810 feat: Snap to Scale toggle + remove tool buttons + no menu on erase sweep
- Added snapToScale state + toggle switch in piano roll toolbar
- snapPitchToScale only applies when snapToScale is true
- Removed Select/Pen/Eraser tool buttons
- Right-click drag erase sweep sets swallowContextMenuRef=true
  so context menu won't appear during sweep
2026-07-25 12:39:32 +07:00
3dtours 221561d58a fix: right-click delete no menu + scale only for brush drag
- Separate swallowContextMenuRef (not tied to rightClickDragRef) so
  menu is skipped on right-click delete
- Initial click draw uses raw pitch, not snapPitchToScale (scale only
  applies during brush-drag in mousemove)
2026-07-25 12:34:12 +07:00
3dtours 0a17cc91ea fix: right-click delete note no longer shows context menu
Set rightClickDragRef.erasedNote=true on delete. handleContextMenu
checks and skips menu when true.
2026-07-25 12:30:08 +07:00
3dtours 7174773cb3 fix: separate right-click menu vs drag-erase behavior
Right-click on note = delete.
Right-click empty space (no drag) = context menu only (no erase sweep).
Right-click empty space + drag >5px = erase sweep (starts on mousemove).
Uses rightClickDragRef to track pending right-click state.
2026-07-25 12:28:44 +07:00
3dtours 0beb957185 fix: restore erase sweep on right-click-drag
Right-click on note = delete. Right-click + drag on empty space =
erase sweep. Resets on mouseup so brush works after menu closes.
2026-07-25 12:26:11 +07:00
3dtours 963a5519af fix: click empty space always draws note regardless of tool
Removed tool check for drawing. Default tool back to 'select'.
Click + drag on empty piano roll = draw notes, no tool switch needed.
2026-07-25 12:24:47 +07:00
3dtours bf22679506 fix: set default piano roll tool to 'pen' so brush works
ActiveRollTool default changed from 'select' to 'pen' so clicking
empty space draws notes immediately. Users who switch tool must
switch back to 'pen' to draw.
2026-07-25 12:23:23 +07:00
3dtours 66cb05f26c fix: scale sub-menu as separate floating panel, not shifted main menu
Sub-menu items now render as a standalone fixed-position panel
at origin.x+150, not mixed into the main menu. Main menu stays
at its initial position. Hover to open, click to select scale.
2026-07-25 12:22:18 +07:00
3dtours 5497bd1256 fix: scale menu on grid canvas, not keybed; erase sweep removed
- Context menu (Scales) appears at right-click position on grid canvas
- Removed erase sweep on right-click (blocked brush after menu close)
- Removed keybed onContextMenu (was wrong position)
- Store menu origin ref so sub-menus don't drift on hover
2026-07-25 12:19:32 +07:00
3dtours 225c44d98a feat: brush-pen scale snap + scale sub-menu hover + keybed swipe
- Brush (pen tool only) draws notes snapped to selected scale
- Scale sub-menus open on hover instead of click
- Keybed: swipe/drag across keys plays notes (mouseenter+mousedown)
- Global mouseup resets keybed drag state
2026-07-25 12:01:42 +07:00
3dtours 51787e055e feat: piano roll velocity scroll, alt+scroll play, scale menu
- Shift+scroll on note: change velocity; on empty space: horizontal scroll
- Alt+scroll: playhead scrub + play notes (was shift+scroll)
- Fix CC lane height drag direction (up=increase, down=decrease)
- Restructure SCALES hierarchy: Diatonic, Pentatonic, Church, Jazz, Asian, Middle Eastern
- Right-click on keybed pitch row opens scale context menu
2026-07-25 11:56:50 +07:00
3dtours ac78d0aa7a fix: sửa lỗi vẽ section item trên main session 2026-07-25 11:41:57 +07:00
3dtours e90c929885 fix: clip section sub-track content to section bounds
When section duration is resized shorter, sub-track clips and MIDI
notes were still drawn outside the section's visual rectangle.
Added ctx.save/clip/restore around sub-track rendering.
2026-07-25 11:39:43 +07:00
3dtours e0531be65b fix: redraw canvas when section/MIDI item duration changes on resize
Add setCanvasRedrawCount after updateActiveTracks in resize handler
so WaveformLane re-renders with updated notes/waveform.
2026-07-25 11:33:35 +07:00
3dtours a956ae407c fix: sửa lỗi drag nhóm midi note trong piano roll tab 2026-07-25 11:31:01 +07:00
3dtours 569ae9ef3e fix: allow drag-move of already-selected notes in piano roll
Previously clicking an already-selected note played preview and returned
early, preventing drag. Now reuses existing selectedNoteIds for drag.
2026-07-25 11:29:09 +07:00
3dtours 0039f85913 fix: sửa lỗi hiển thị hiển thị bar và timeline hoàn thành cho main session 2026-07-25 11:23:46 +07:00
3dtours 7530738444 fix: remove overflow-hidden wrapper causing canvas width mismatch
Div wrapper with overflow-hidden clipped the sticky canvas right edge.
TempoTrackLane now renders like TimelineRuler — Fragment + spacer + canvas.
2026-07-25 11:16:56 +07:00
3dtours a6a12b36a7 fix: use DOM scrollLeftVal for clip/section/MIDI rendering in WaveformLane
6 render positions (clip, marker, section, MIDI, note, selection) were
using React prop scrollLeft instead of DOM scrollLeftVal, causing
grid-vs-item desync during fast auto-scroll.
2026-07-25 11:12:39 +07:00
3dtours c397545383 fix: read scrollLeft from DOM wrapper for canvas sync
Canvas effects now find nearest scrollable ancestor and read
scrollLeft directly, bypassing React prop latency during auto-scroll.
Applies to TempoTrackLane, WaveformLane, TimelineRuler.
2026-07-25 11:09:20 +07:00
3dtours 5bd61516aa fix: PADDING_LEFT=0, tEnd uses CLIP_BUFFER for right buffer
- PADDING_LEFT=0 in all 3 components (grid aligns with clip origin)
- TempoTrackLane/WaveformLane tEnd uses CLIP_BUFFER/zoom so bars render past right edge
- TimelineRuler tEnd uses CLIP_BUFFER/zoom instead of PADDING_LEFT
2026-07-25 11:06:12 +07:00
3dtours 01cd3d1867 fix: allow item drag/resize to snap at column 0
- Removed Math.max(beatSec, ...) clamp from clip drag, section drag, section resize handlers
- Replaced with Math.max(0, ...) so items can snap to time 0
2026-07-25 10:59:17 +07:00
3dtours cb3e8117b2 fix: sửa lỗi hiển thị hiển thị bar và timeline 2026-07-25 10:58:28 +07:00
3dtours 8be4ce99e1 fix: tempo track height 40px fixed + clamp selection >= 0
- TempoTrackLane canvas height fixed to 40px, wrapper h-10 shrink-0 overflow-hidden
- selLeft/selRight clamped to Math.max(0, ...)
- rulerAnchorRef/setSelectionStart/setSelectionEnd clamped >= 0
2026-07-25 10:55:46 +07:00
3dtours e14006ffcc fix: sửa lỗi hiển thị hiển thị bar và timeline grid 3 2026-07-25 10:51:08 +07:00
3dtours b410de8855 refactor: zero lead-in, split time ruler + tempo track
- leadInMargin=0, time 0 at left edge, only 8px PADDING_LEFT
- TimelineRuler simplified: time-only labels, 40px height
- TempoTrackLane re-added to layout below time ruler (40px)
- WaveformLane PADDING_LEFT=8/zoom, leadIn=0
2026-07-25 10:48:45 +07:00
3dtours f299b93372 fix: sync TempoTrackLane leadIn with ruler, time labels to top
- Added leadInMargin prop to TempoTrackLane so time-0 aligns with ruler
- Added CLIP_BUFFER to TempoTrackLane grid rendering
- Moved TimelineRuler time labels from bottom (y=40) to top (y=11)
2026-07-25 10:41:13 +07:00
3dtours 70d1abaa7a fix: restore time duration labels on TimelineRuler + TempoTrackLane
Time labels (e.g. '0.00s', '1.00s') were lost in ff44fd4 merge.
Re-added as amber/orange text at ruler bottom, dynamic interval.
2026-07-25 10:37:12 +07:00
3dtours db759fc4fc fix: sửa lỗi hiển thị hiển thị bar và timeline grid 2026-07-25 10:33:01 +07:00
3dtours 8372ee1b98 feat: lead-in margin, 0-indexed bars, fix bar number clipping
- Add 1-bar lead-in margin at timeline start (bar 0, time 0s at margin edge)
- Playhead position includes lead-in offset
- Bar numbers now 0-indexed (0, 1, 2...) in ruler and track grid
- Fix bar number clipping: CLIP_BUFFER increased from 200px to
  max(400px, barWidth+200px) so bars past viewport edge render
- All mouse/time handlers account for lead-in margin (ruler, clip
  drag, section drag/resize, context menu, double-click)
- formatBeat returns 0-indexed bar
- Sub-tab ruler has leadInMargin=0 (no lead-in in clip editor)
2026-07-25 10:25:30 +07:00
3dtours ff44fd4b8d feat: merge tempo bar + timebar into beat-based TimelineRuler
Replace old seconds-based ruler (0.00s, 1.00s...) and TempoTrackLane
with unified 48px canvas ruler showing bar:beat grid lines, snap
sub-ticks, and BPM label. WaveformLane grid aligns with ruler
positions — items snap to same beat grid visually.
2026-07-25 10:12:59 +07:00
3dtours 6822f5ed33 fix: WaveformLane grid now matches TempoTrackLane bar:beat structure, bar:beat always shown on tempo lane 2026-07-25 10:05:40 +07:00
3dtours 52ecc38e13 fix: auto-follow 80px both sides for tempo lane bar visibility + 200px buffer 2026-07-25 10:01:56 +07:00
3dtours fbbe2c933c docs: update wiki.md with latest changes 2026-07-25 09:58:23 +07:00
3dtours e56ee5cb0a fix: remove aggressive auto-follow, smooth autoScrollTimeline only, PADDING_RIGHT to show bar numbers past edge, 12-bar buffer 2026-07-25 09:57:00 +07:00
3dtours b746f9659b fix: add snapTime to section/MIDI item drag, followMargin 0.75 for 75% viewport ahead 2026-07-25 09:54:14 +07:00
3dtours 373c018025 fix: sửa lỗi hiển thị bar 2026-07-25 09:52:02 +07:00
3dtours 5a152e4951 fix: sync PADDING_LEFT = barDuration in both lanes, add snap '4' (1-bar) 2026-07-25 09:51:07 +07:00
3dtours 7c99fca168 fix: sync PADDING_LEFT = barDuration in both WaveformLane and TempoTrackLane 2026-07-25 09:49:15 +07:00
3dtours 8dc57d4501 fix: sửa lỗi di chuyển items 2026-07-25 09:46:06 +07:00
3dtours 0554934ebe fix: increase followMargin from 120px to 0.4*viewport for bar visibility ahead of dragged item 2026-07-25 09:40:53 +07:00
3dtours dc8b1f59fa fix: revert TempoTrackLane PADDING_LEFT to 2s, remove DOM scrollLeft hack 2026-07-25 09:37:57 +07:00
3dtours 186c5353d4 fix: tempo lane DOM scrollLeft traversal finds overflow-x:auto parent 2026-07-25 09:33:11 +07:00
3dtours e5a563d343 fix: TempoTrackLane reads scrollLeft from DOM instead of React prop 2026-07-25 09:30:42 +07:00
3dtours 6aed5e2ec8 fix: sync bar ruler redraw during drag via canvasRedrawCount 2026-07-25 09:24:31 +07:00
3dtours f1057ab7f2 feat: auto-follow dragged item in viewport for clip/section/MIDI drag and resize 2026-07-25 09:21:19 +07:00
3dtours aa63a310fc fix: brush velocity isolation from CC lane + AGENTs.md compliance 2026-07-25 09:18:41 +07:00
3dtours c5c3ec19cc chore: update wiki.md for velocity/redraw fix 2026-07-25 08:08:51 +07:00
3dtours 893f42c36c fix: velocity start position + shift-scroll play on note start + canvas redraw deps 2026-07-25 08:07:49 +07:00
3dtours 8dee5c8cdb fix: sync ruler/grid/CC lane scroll via ccWrapperRef 2026-07-25 08:01:55 +07:00
3dtours eb805c04a4 fix: piano roll bars extend to fill viewport on zoom out 2026-07-25 07:50:53 +07:00
3dtours a90a3cfad1 chore: update AGENTs.md with clearer commit instructions 2026-07-25 07:45:25 +07:00
3dtours 46530c8896 refactor: simplify piano roll zoom - use CSS clip instead of manual grid extension 2026-07-25 07:45:06 +07:00
3dtours b04a76ebfb fix: piano roll zoom/scroll + AI copilot + SF cache + auto-scroll drag 2026-07-25 07:37:27 +07:00
3dtours 1c0d35f69b fix: chỉnh sửa section item dù được lưu lại vẫn là item 2026-07-24 22:15:28 +07:00
3dtours d262f2e778 fix: Chỉnh sửa các hành động của vẽ các note midi trong piano roll tab 2026-07-24 21:25:04 +07:00
3dtours 9a61a1f616 feat: vẽ section item canvas 2026-07-24 20:58:15 +07:00
3dtours b9eb840b11 feat: vẽ nhanh MIDI note 2026-07-24 20:33:01 +07:00
3dtours 17a284ad17 feat: bổ sung phần section-item và section tab 2026-07-24 17:00:22 +07:00
3dtours 6a17e7036c fix: nhãn của soundfont hiển thị sai và không load được instrument 2026-07-23 22:32:19 +07:00
3dtours 1a28ebabee fix: piano roll và main session play 2026-07-23 21:05:26 +07:00
3dtours f0210dd019 fix: main session có thể play midi item 2026-07-23 20:52:50 +07:00
3dtours dac661e6c6 fix: piano roll tab có thể play với soundfont 2026-07-23 20:34:52 +07:00
3dtours a21266736c fix: synth để gắn soundfont cho midi 2026-07-23 19:26:26 +07:00
3dtours 055f351a17 fix: sửa lỗi cho AI thực hiện tạo MIDI 2026-07-23 18:19:12 +07:00
3dtours b46a2c45f2 feat: cài đặt tools cho AI thực hiện tạo MIDI 2026-07-23 17:58:44 +07:00
3dtours 225f23516f feat: thêm soundfont và VSTi cho MIDI 2026-07-23 17:47:46 +07:00
31 changed files with 5602 additions and 819 deletions
+1 -1
View File
@@ -22,4 +22,4 @@ app/storage/processed/*
.vscode/ .vscode/
*.log *.log
celerybeat-schedule celerybeat-schedule
node_modules/ node_modules
+12
View File
@@ -0,0 +1,12 @@
# Kilo Post-Task Automation Rule
Mỗi khi bạn (Kilo) hoàn thành một tác vụ viết code, sửa lỗi, hoặc refactor code theo yêu cầu của tôi, bạn BẮT BUỘC phải thực hiện bước cuối cùng sau:
1. Kiểm tra sự tồn tại của file `wiki.md` ở thư mục gốc (nếu chưa có, hãy tạo mới).
2. Tự động chèn (append) một đoạn báo cáo ngắn về task vừa hoàn thành vào CUỐI file `wiki.md` theo cấu trúc sau:
### [YYYY-MM-DD HH:mm] Task: <Tên ngắn gọn của công việc>
- **Tóm tắt thay đổi:** <Mô tả 1-2 câu về nội dung đã thực hiện>
- **Các file ảnh hưởng:** `<file_1>`, `<file_2>`
- **Ghi chú/Test (nếu có):** <Các lưu ý hoặc lệnh chạy test/build nếu có>
---
+17
View File
@@ -0,0 +1,17 @@
# Kilo Post-Task Automation Rule
Mỗi khi bạn (Kilo) hoàn thành một tác vụ viết code, sửa lỗi, hoặc refactor code theo yêu cầu của tôi, bạn BẮT BUỘC phải thực hiện bước cuối cùng sau:
**BẮT BUỘC**
1. Kiểm tra sự tồn tại của file `wiki.md` ở thư mục gốc (nếu chưa có, hãy tạo mới).
2. Tự động chèn (append) một đoạn báo cáo ngắn về task vừa hoàn thành vào CUỐI file `wiki.md` theo cấu trúc sau:
### [YYYY-MM-DD HH:mm] Task: <Tên ngắn gọn của công việc>
- **Tóm tắt thay đổi:** <Mô tả 1-2 câu về nội dung đã thực hiện>
- **Các file ảnh hưởng:** `<file_1>`, `<file_2>`
- **Ghi chú/Test (nếu có):** <Các lưu ý hoặc lệnh chạy test/build nếu có>
---
3. Kiểm tra `git status`, add các file code đã sửa (chỉ add file code, KHÔNG add file database/soundfonts), và commit với message ngắn gọn.
4. Nếu có file `.gitignore` bị conflict, resolve bằng `git checkout --theirs .gitignore && git add .gitignore` trước khi commit.
+205
View File
@@ -0,0 +1,205 @@
# Plan: TCP Resizable Width + Instrument Search Dropdown + Section Save Fix
## Task 1: User-Resizable TCP Width
**Files:** `app/static/js/app.jsx`
### Root Cause
TCP containers are hardcoded `w-[320px]` (lines 14253, 14809). Components like Synth button, FX button, volume/pan sliders, input select overflow when content is wide.
### Implementation Steps
**1a — Add TCP width state**
Add near line 6394 (near existing `rightSidebarWidth` state):
```javascript
const [tcpWidth, setTcpWidth] = useState(320);
```
**1b — Add TCP resize handler**
Add near line 6365 (near `startColResize`):
```javascript
const startTcpResize = e => {
e.preventDefault();
const startX = e.clientX;
const startW = tcpWidth;
const onMove = ev => {
const deltaX = ev.clientX - startX;
const newWidth = Math.max(280, Math.min(600, startW + deltaX));
setTcpWidth(newWidth);
};
const onUp = () => {
document.removeEventListener('mousemove', onMove);
document.removeEventListener('mouseup', onUp);
};
document.addEventListener('mousemove', onMove);
document.addEventListener('mouseup', onUp);
};
```
**1c — Replace `w-[320px]` with dynamic width in main TCP container (line 14253)**
Change `className: "w-[320px] shrink-0 ..."` to `style: { width: tcpWidth + 'px', ... }`.
**1d — Replace `w-[320px]` with dynamic width in sub-tab TCP container (line 14809)**
Same pattern as 1c.
**1e — Add resize handle (right edge of TCP)**
Add a vertical resize handle bar on the right edge of both TCP containers. Pattern:
```jsx
React.createElement("div", {
onMouseDown: startTcpResize,
className: "absolute right-0 top-0 bottom-0 w-1 cursor-col-resize z-40 hover:bg-cyan-500/50 transition-colors",
style: { right: 0 }
})
```
**1f — Ensure the main layout accommodates variable TCP width**
The main timeline area should use `flex-1` so it fills remaining space. Verify existing layout handles this.
### Verification
- Drag TCP right edge → width changes between 280px and 600px
- Components fit properly at various widths
- Timeline area fills remaining space
- Works in section-tab view too
---
## Task 2: Instrument Search Dropdown in TCP
**Files:** `app/static/js/app.jsx`
### Root Cause
Current instrument selector is a modal overlay (lines 15661-15726) with no search/filter. Requires clicking Synth button → modal → scroll to find instrument.
### Implementation Steps
**2a — Add per-track dropdown open/close state**
Add state:
```javascript
const [instrumentDropdownTrackId, setInstrumentDropdownTrackId] = useState(null);
```
This tracks which track's dropdown is open (null = all closed).
**2b — Add search query state**
```javascript
const [instrumentSearchQuery, setInstrumentSearchQuery] = useState('');
```
**2c — Replace Synth button (top toolbar, line 14444-14448) with dropdown toggle**
Convert the icon-only `<button>` into a container that:
1. Shows current instrument name (truncated) + chevron-down icon when assigned
2. Shows "Synth" + chevron-down icon when no instrument
3. Click toggles `instrumentDropdownTrackId` for this track
**2d — Render the dropdown panel (conditional, below the button)**
When `instrumentDropdownTrackId === track.id`, render a dropdown panel:
```jsx
React.createElement("div", {
className: "absolute left-0 top-full mt-0.5 z-50 bg-[#1e1e1e] border border-zinc-700 rounded shadow-xl w-[220px] max-h-[300px] flex flex-col",
onClick: e => e.stopPropagation()
},
// Search input
React.createElement("input", {
type: "text",
placeholder: "Tìm nhạc cụ...",
value: instrumentSearchQuery,
onChange: e => setInstrumentSearchQuery(e.target.value),
className: "w-full bg-black border-b border-zinc-700 px-2 py-1.5 text-xs text-zinc-200 outline-none"
}),
// Filtered list
React.createElement("div", {
className: "flex-1 overflow-y-auto"
},
// Filtered items from instrumentSelectorData
// "None (Default Synth)" always shown first
// Then filtered soundfonts
// Then filtered VST instruments
)
)
```
**2e — Filter logic**
```javascript
const filteredInstruments = useMemo(() => {
if (!instrumentSelectorData) return { soundfonts: [], vst: [] };
const q = instrumentSearchQuery.toLowerCase();
return {
soundfonts: (instrumentSelectorData.soundfonts || []).filter(sf =>
(sf.display || sf.name || sf.id).toLowerCase().includes(q)
),
vst: (instrumentSelectorData.vst_instruments || []).filter(v =>
(v.name || v.id).toLowerCase().includes(q)
)
};
}, [instrumentSearchQuery, instrumentSelectorData]);
```
**2f — Click outside to close**
Add a global click handler that closes the dropdown when clicking outside.
**2g — Preload `instrumentSelectorData` on first TCP mount**
Instead of only loading on `openInstrumentSelector`, preload `listPlugins()` when the first track renders (or on app mount).
**2h — Apply selection**
On click of a dropdown item, call existing `setTrackInstrumentWithProgram` or `setTrackInstrument`. Close dropdown.
### Verification
- Click Synth button → dropdown opens with search input focused
- Type instrument name → list filters in real-time
- Click instrument → dropdown closes, track assigned, Synth button shows name
- Click outside → dropdown closes
---
## Task 3: Section-Tab Save Fix — Replace Instead of Draw On Top
**Files:** `app/static/js/app.jsx`
### Root Cause
`handleEditSectionInTab` (line 7178) clones ALL main-session tracks (empty) into the session-tab when `section.tracks` doesn't exist. `handleSaveSectionTab` (line 7156) saves ALL those empty tracks + edited ones into `s.tracks`. The rendering code (lines 696-791) draws ALL stored tracks inside the section box, creating a cluttered preview with empty/minimal tracks.
### Implementation Steps
**3a — Fix `handleEditSectionInTab` (line 7169) to only initialize relevant track**
Change the fallback cloning (line 7178) from cloning ALL main tracks to creating a minimal set of tracks based on the section's parent track:
```javascript
const clonedTracks = section.tracks ? section.tracks : [{
...tracks.find(tr => tr.id === trackId),
clips: [],
sections: [],
midiItems: [],
markers: [],
isArmed: false,
monitoringEnabled: true,
instrumentId: null,
instrumentProgram: undefined,
instrumentName: null
}];
```
This only clones the track that owns the section, not ALL main tracks.
**3b — Fix `handleSaveSectionTab` (line 7138) to filter non-empty tracks**
After building the updated section, filter `tab.tracks` to only include tracks that have actual content:
```javascript
const contentTracks = tab.tracks.filter(t =>
(t.clips && t.clips.length > 0) ||
(t.midiItems && t.midiItems.length > 0)
);
```
Store `tracks: contentTracks` instead of `tracks: tab.tracks`.
**3c — Improve section-item rendering (lines 694-791)**
The rendering already draws waveform from clips and MIDI notes from midiItems. Ensure:
- Waveform rendering for clips with `clip.buffer` is correct (already done at lines 720-739)
- MIDI note colors are per-track-index (already done at line 783: `noteColors[trackIdx % noteColors.length]`)
- Add a subtle track label inside each sub-track row so users can identify which track is which
**3d — Ensure waveform preview is properly sized**
The section preview currently allocates `subTrackHeight = (height - 24) / maxSubTracks` for each sub-track (line 698). Verify this is sufficient for waveform + MIDI note rendering when there are 1-2 tracks (typical case).
### Verification
- Open a section for editing → session-tab shows only the relevant track(s), not all main tracks
- Add MIDI items, sound clips, soundfonts, FX to tracks
- Save section → section-item shows waveform preview + MIDI note preview (replacing previous content, not appending)
- Open section again → previous edits are loaded correctly
- Multiple save cycles → no doubling of content
- Waveform rendered as background, MIDI notes in distinct colors per track
+1
View File
@@ -18,6 +18,7 @@ RUN apt-get update && apt-get install -y \
xvfb \ xvfb \
ffmpeg \ ffmpeg \
libsndfile1 \ libsndfile1 \
libfluidsynth3 \
build-essential \ build-essential \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
+1 -1
View File
@@ -15,7 +15,7 @@ import json
@router.post("/proxy") @router.post("/proxy")
async def proxy_llm(req: ProxyRequest): async def proxy_llm(req: ProxyRequest):
try: try:
async with httpx.AsyncClient(timeout=60.0) as client: async with httpx.AsyncClient(timeout=180.0) as client:
resp = await client.post( resp = await client.post(
req.url, req.url,
headers={k: v for k, v in req.headers.items() if k.lower() not in ('host', 'origin', 'referer')}, headers={k: v for k, v in req.headers.items() if k.lower() not in ('host', 'origin', 'referer')},
+115
View File
@@ -0,0 +1,115 @@
import os, uuid, json, tempfile
from fastapi import APIRouter, HTTPException, Depends, UploadFile, File
from pydantic import BaseModel
from typing import Optional, Any
from app.config import settings
from app.core.vst_engine import PluginManager, HAS_PEDALBOARD, HAS_PYFLUIDSYNTH
from app.core.render_engine import PythonRenderEngine
from app.api.v1.auth import get_current_user
router = APIRouter()
UPLOAD_SF_DIR = os.path.join(settings.STORAGE_DIR, "soundfonts")
os.makedirs(UPLOAD_SF_DIR, exist_ok=True)
@router.get("/available")
async def list_plugins(current_user: dict = Depends(get_current_user)):
pm = PluginManager(upload_sf_dir=UPLOAD_SF_DIR)
return pm.list_available()
@router.get("/default-soundfonts")
async def list_default_soundfonts():
static_sf_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "static", "soundfonts")
result = []
if os.path.isdir(static_sf_dir):
for f in os.listdir(static_sf_dir):
if f.endswith(".sf2") or f.endswith(".sf3"):
result.append({
"id": os.path.splitext(f)[0],
"name": f,
"file": f,
"url": f"/soundfonts/{f}"
})
return result
@router.get("/soundfont-instruments/{sf_id}")
async def list_soundfont_instruments(sf_id: str, current_user: dict = Depends(get_current_user)):
pm = PluginManager(upload_sf_dir=UPLOAD_SF_DIR)
presets = pm.list_soundfont_instruments(sf_id)
return {"presets": presets, "count": len(presets)}
@router.post("/upload-soundfont")
async def upload_soundfont(
file: UploadFile = File(...),
current_user: dict = Depends(get_current_user)
):
if not (file.filename and (file.filename.endswith(".sf2") or file.filename.endswith(".sf3"))):
raise HTTPException(status_code=400, detail="Only .sf2 / .sf3 files are allowed")
contents = await file.read()
if not PluginManager.validate_sf2_header(contents):
raise HTTPException(status_code=400, detail="Invalid SoundFont file: missing RIFF/sfbk header")
file_ext = os.path.splitext(file.filename)[1]
file_uuid = str(uuid.uuid4())
# Store original name in a sidecar file
base_name = os.path.splitext(file.filename)[0].replace('/', '_').replace('\\', '_')
file_id = file_uuid + file_ext
dest_path = os.path.join(UPLOAD_SF_DIR, file_id)
with open(dest_path, "wb") as f:
f.write(contents)
# Save metadata with original name
meta_path = os.path.join(UPLOAD_SF_DIR, file_uuid + ".meta")
with open(meta_path, "w", encoding="utf-8") as f:
import json
json.dump({"original_name": file.filename, "uuid": file_uuid, "file": file_id}, f)
return {"id": file_id, "name": file.filename, "path": dest_path, "size_bytes": len(contents)}
@router.delete("/soundfont/{sf_id}")
async def delete_soundfont(sf_id: str, current_user: dict = Depends(get_current_user)):
base_id = sf_id.replace("sf_", "")
deleted = False
for d in [UPLOAD_SF_DIR, os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), "static", "soundfonts")]:
if not os.path.isdir(d):
continue
for f in os.listdir(d):
if os.path.splitext(f)[0] == base_id:
path = os.path.join(d, f)
os.remove(path)
# Remove associated .meta file
meta_path = os.path.join(d, os.path.splitext(f)[0] + ".meta")
if os.path.isfile(meta_path):
os.remove(meta_path)
deleted = True
break
if deleted:
break
if not deleted:
raise HTTPException(status_code=404, detail="SoundFont not found")
return {"deleted": True, "sf_id": sf_id}
class RenderRequest(BaseModel):
project_json: dict
output_filename: Optional[str] = "render_output.wav"
@router.post("/render")
async def render_project(
req: RenderRequest,
current_user: dict = Depends(get_current_user)
):
engine = PythonRenderEngine()
output_path = os.path.join(settings.PROCESSED_DIR, req.output_filename or "render_output.wav")
try:
result_path = engine.render_project(req.project_json, output_path)
return {"url": f"/static/audio/processed/{os.path.basename(result_path)}", "path": result_path}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Render failed: {str(e)}")
+122 -24
View File
@@ -1,28 +1,18 @@
import os import os
import numpy as np import numpy as np
import soundfile as sf import soundfile as sf
import scipy.signal as signal
from app.config import settings from app.config import settings
from app.core.vst_engine import render_midi_events_to_audio from app.core.vst_engine import (
render_midi_events_to_audio,
import subprocess PluginManager,
import sys HAS_PEDALBOARD,
HAS_PYFLUIDSYNTH,
def check_pedalboard_safe():
try:
res = subprocess.run(
[sys.executable, "-c", "import pedalboard"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=2.0
) )
return res.returncode == 0
except Exception:
return False
HAS_PEDALBOARD = check_pedalboard_safe()
if HAS_PEDALBOARD: if HAS_PEDALBOARD:
try: try:
from pedalboard import Pedalboard, Gain from pedalboard import Pedalboard, Gain, Chorus, Reverb
except Exception: except Exception:
HAS_PEDALBOARD = False HAS_PEDALBOARD = False
@@ -130,14 +120,71 @@ class PythonRenderEngine:
if midi_events: if midi_events:
try: try:
# Synthesize MIDI track notes instrument_id = track.get("instrument", "")
synth_buffer = render_midi_events_to_audio( plugin_mgr = PluginManager()
midi_events=midi_events, vst = plugin_mgr.load_vst(instrument_id) if instrument_id else None
sr=self.sample_rate,
bpm=bpm, if vst and HAS_PEDALBOARD:
instrument='synth' from pedalboard import Pedalboard
# Convert MIDI events with precise sample offset
midi_messages = PluginManager.midi_events_to_messages(
midi_events, bpm, self.sample_rate
)
total_needed = 0
for ev in midi_events:
end_sec = (ev.get("start_beat", 0) + ev.get("duration_beats", 1)) * (60.0 / bpm)
dur_samples = int(end_sec * self.sample_rate)
if dur_samples > total_needed:
total_needed = dur_samples
total_needed = max(total_needed, 1024)
silent = np.zeros((2, total_needed), dtype=np.float32)
board = Pedalboard([vst])
synth_buffer = board(silent, sample_rate=self.sample_rate, midi_messages=midi_messages)
elif instrument_id and instrument_id.startswith("sf_"):
sf_path = os.path.join(
os.path.dirname(os.path.dirname(__file__)), "static", "soundfonts",
instrument_id.replace("sf_", "") + ".sf2"
)
if os.path.exists(sf_path) and HAS_PYFLUIDSYNTH:
import fluidsynth
fl = fluidsynth.FluidSynth(sample_rate=self.sample_rate, gain=0.5)
fid = fl.sfload(sf_path)
fl.program_select(0, fid, 0, 0)
beat_sec = 60.0 / bpm
total_sec = 0
for ev in midi_events:
end_sec = (ev.get("start_beat", 0) + ev.get("duration_beats", 1)) * beat_sec
if end_sec > total_sec:
total_sec = end_sec
total_samples = int((total_sec + 1.0) * self.sample_rate)
midi_data = np.zeros((2, total_samples), dtype=np.float32)
for ev in midi_events:
note = ev.get("note", 60)
velocity = ev.get("velocity", 100)
start_beat = ev.get("start_beat", 0.0)
dur_beats = ev.get("duration_beats", 1.0)
start_sec = start_beat * beat_sec
dur_sec = dur_beats * beat_sec
fl.noteon(0, note, velocity)
start_s = int(start_sec * self.sample_rate)
dur_s = int(dur_sec * self.sample_rate)
block = fl.get_samples(int(dur_s)) if hasattr(fl, 'get_samples') else np.zeros((2, dur_s), dtype=np.float32)
fl.noteoff(0, note)
if block.shape[1] > 0:
end_s = min(start_s + block.shape[1], total_samples)
actual = end_s - start_s
if actual > 0:
midi_data[:, start_s:end_s] += block[:, :actual]
synth_buffer = midi_data
fl.delete()
else:
synth_buffer = render_midi_events_to_audio(
midi_events=midi_events, sr=self.sample_rate, bpm=bpm, instrument='synth'
)
else:
synth_buffer = render_midi_events_to_audio(
midi_events=midi_events, sr=self.sample_rate, bpm=bpm, instrument='synth'
) )
# Add to track buffer
actual_len = min(synth_buffer.shape[1], total_samples) actual_len = min(synth_buffer.shape[1], total_samples)
track_buffer[:, :actual_len] += synth_buffer[:, :actual_len] track_buffer[:, :actual_len] += synth_buffer[:, :actual_len]
except Exception as e: except Exception as e:
@@ -176,6 +223,57 @@ class PythonRenderEngine:
if mute: if mute:
continue continue
# Apply Track FX (Chorus or Reverb)
fx_type = track.get("fx_type")
if fx_type == "chorus":
if HAS_PEDALBOARD:
try:
board = Pedalboard([Chorus(rate_hz=1.5, depth=0.25)])
track_buffer = board(track_buffer, sample_rate=self.sample_rate)
except Exception as e:
print(f"[RenderEngine] Pedalboard Chorus failed: {e}")
else:
# Fallback chorus using simple LFO delay modulation in scipy/numpy
try:
# 1.5 Hz sine LFO, modulating delay time between 15ms and 25ms (average 20ms)
lfo = 0.020 + 0.005 * np.sin(2 * np.pi * 1.5 * np.arange(total_samples) / self.sample_rate)
dry = track_buffer * 0.6
wet = np.zeros_like(track_buffer)
for ch in range(2):
indices = np.arange(total_samples) - (lfo * self.sample_rate)
indices = np.clip(indices, 0, total_samples - 1).astype(np.int32)
wet[ch, :] = track_buffer[ch, indices]
track_buffer = dry + wet * 0.5
except Exception as e:
print(f"[RenderEngine] Fallback Chorus failed: {e}")
elif fx_type == "reverb":
if HAS_PEDALBOARD:
try:
board = Pedalboard([Reverb(room_size=0.5, wet_level=0.4, dry_level=0.6)])
track_buffer = board(track_buffer, sample_rate=self.sample_rate)
except Exception as e:
print(f"[RenderEngine] Pedalboard Reverb failed: {e}")
else:
# Fallback reverb using exponentially decaying noise room impulse response
try:
# Generate impulse response (decaying noise)
len_ir = int(self.sample_rate * 2.0)
t_ir = np.arange(len_ir) / self.sample_rate
decay = np.exp(-t_ir / 0.5)
ir_l = (np.random.rand(len_ir) * 2 - 1) * decay
ir_r = (np.random.rand(len_ir) * 2 - 1) * decay
dry = track_buffer * 0.6
wet = np.zeros_like(track_buffer)
for ch in range(2):
ir = ir_l if ch == 0 else ir_r
# Convolve
conv = signal.convolve(track_buffer[ch, :], ir, mode='full')[:total_samples]
wet[ch, :] = conv
track_buffer = dry + wet * 0.4
except Exception as e:
print(f"[RenderEngine] Fallback Reverb failed: {e}")
# Process track volume # Process track volume
if HAS_PEDALBOARD: if HAS_PEDALBOARD:
try: try:
+269 -31
View File
@@ -1,77 +1,315 @@
# SonicForge Studio VST / VSTi Engine Service (22_CLIENT_DESK.md §3) # SonicForge Studio VST / VSTi Engine Service
import os
import numpy as np import numpy as np
import functools
from ctypes import c_int, c_char_p, c_void_p
def midi_note_to_freq(note_number: int) -> float: def midi_note_to_freq(note_number: int) -> float:
"""Quy đổi số nốt MIDI (0 - 127) sang tần số Hertz (Hz)."""
return 440.0 * (2.0 ** ((note_number - 69) / 12.0)) return 440.0 * (2.0 ** ((note_number - 69) / 12.0))
def render_midi_events_to_audio(midi_events: list, sr: int = 44100, bpm: float = 120.0, instrument: str = 'synth') -> np.ndarray: def render_midi_events_to_audio(midi_events: list, sr: int = 44100, bpm: float = 120.0, instrument: str = 'synth') -> np.ndarray:
"""
Tổng hợp mảng âm thanh NumPy Stereo từ sự kiện MIDI Piano Roll (22_CLIENT_DESK.md §3.1 & §3.2).
Args:
midi_events: Danh sách nốt MIDI [{"note": 60, "start_beat": 0, "duration_beats": 1, "velocity": 100}, ...]
sr: Tần số lấy mẫu (Sample Rate)
bpm: Nhịp BPM của dự án
instrument: Loại nhạc cụ tổng hợp
Returns:
np.ndarray: Mảng 2D Stereo Float32 [2, num_samples]
"""
beat_duration_sec = 60.0 / max(30.0, bpm) beat_duration_sec = 60.0 / max(30.0, bpm)
max_duration_sec = 2.0 max_duration_sec = 2.0
for event in midi_events: for event in midi_events:
start_beat = event.get('start_beat', 0.0) start_beat = event.get('start_beat', 0.0)
dur_beats = event.get('duration_beats', 1.0) dur_beats = event.get('duration_beats', 1.0)
end_sec = (start_beat + dur_beats) * beat_duration_sec end_sec = (start_beat + dur_beats) * beat_duration_sec
if end_sec > max_duration_sec: if end_sec > max_duration_sec:
max_duration_sec = end_sec max_duration_sec = end_sec
total_samples = int((max_duration_sec + 0.5) * sr) total_samples = int((max_duration_sec + 0.5) * sr)
out_l = np.zeros(total_samples, dtype=np.float32) out_l = np.zeros(total_samples, dtype=np.float32)
out_r = np.zeros(total_samples, dtype=np.float32) out_r = np.zeros(total_samples, dtype=np.float32)
for event in midi_events: for event in midi_events:
note = event.get('note', 60) note = event.get('note', 60)
velocity = event.get('velocity', 100) / 127.0 velocity = event.get('velocity', 100) / 127.0
start_beat = event.get('start_beat', 0.0) start_beat = event.get('start_beat', 0.0)
dur_beats = event.get('duration_beats', 1.0) dur_beats = event.get('duration_beats', 1.0)
start_sample = int(start_beat * beat_duration_sec * sr) start_sample = int(start_beat * beat_duration_sec * sr)
dur_samples = int(dur_beats * beat_duration_sec * sr) dur_samples = int(dur_beats * beat_duration_sec * sr)
end_sample = min(total_samples, start_sample + dur_samples) end_sample = min(total_samples, start_sample + dur_samples)
actual_len = end_sample - start_sample actual_len = end_sample - start_sample
if actual_len <= 0 or start_sample >= total_samples: if actual_len <= 0 or start_sample >= total_samples:
continue continue
freq = midi_note_to_freq(note) freq = midi_note_to_freq(note)
t = np.arange(actual_len) / float(sr) t = np.arange(actual_len) / float(sr)
# Synth tone + fundamental harmonics
tone = 0.6 * np.sin(2 * np.pi * freq * t) + 0.3 * np.sin(2 * np.pi * freq * 2 * t) + 0.1 * np.sin(2 * np.pi * freq * 3 * t) tone = 0.6 * np.sin(2 * np.pi * freq * t) + 0.3 * np.sin(2 * np.pi * freq * 2 * t) + 0.1 * np.sin(2 * np.pi * freq * 3 * t)
# ADSR Envelope
attack = min(int(0.01 * sr), actual_len // 4) attack = min(int(0.01 * sr), actual_len // 4)
release = min(int(0.05 * sr), actual_len // 4) release = min(int(0.05 * sr), actual_len // 4)
sustain_len = actual_len - attack - release
env = np.ones(actual_len, dtype=np.float32) env = np.ones(actual_len, dtype=np.float32)
if attack > 0: if attack > 0:
env[:attack] = np.linspace(0.0, 1.0, attack) env[:attack] = np.linspace(0.0, 1.0, attack)
if release > 0: if release > 0:
env[-release:] = np.linspace(1.0, 0.0, release) env[-release:] = np.linspace(1.0, 0.0, release)
signal = tone * env * velocity signal = tone * env * velocity
out_l[start_sample:end_sample] += signal out_l[start_sample:end_sample] += signal
out_r[start_sample:end_sample] += signal out_r[start_sample:end_sample] += signal
# Clamping normalization to prevent clipping
max_peak = max(np.max(np.abs(out_l)), np.max(np.abs(out_r))) max_peak = max(np.max(np.abs(out_l)), np.max(np.abs(out_r)))
if max_peak > 1.0: if max_peak > 1.0:
out_l /= max_peak out_l /= max_peak
out_r /= max_peak out_r /= max_peak
return np.vstack([out_l, out_r]) return np.vstack([out_l, out_r])
def check_pedalboard_safe():
import subprocess, sys
try:
res = subprocess.run(
[sys.executable, "-c", "import pedalboard"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=2.0
)
return res.returncode == 0
except Exception:
return False
def check_pyfluidsynth_safe():
import subprocess, sys
try:
res = subprocess.run(
[sys.executable, "-c", "import fluidsynth"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=2.0
)
return res.returncode == 0
except Exception:
return False
HAS_PEDALBOARD = check_pedalboard_safe()
HAS_PYFLUIDSYNTH = check_pyfluidsynth_safe()
def ensure_pyfluidsynth():
global HAS_PYFLUIDSYNTH
if not HAS_PYFLUIDSYNTH:
HAS_PYFLUIDSYNTH = check_pyfluidsynth_safe()
return HAS_PYFLUIDSYNTH
if HAS_PEDALBOARD:
try:
from pedalboard import VST3Plugin, Pedalboard, Gain, MidiMessage
except Exception:
HAS_PEDALBOARD = False
if HAS_PYFLUIDSYNTH:
try:
import fluidsynth
except Exception:
HAS_PYFLUIDSYNTH = False
# ── Module-level caches ──
_FLUID_CACHE = {} # path → (fluidsynth.FluidSynth, refcount)
_PLUGIN_MANAGER_INSTANCE = None
_PLUGIN_MANAGER_ARGS = None
_SF_INSTRUMENTS_CACHE = {} # sf_id → list[presets]
def get_plugin_manager(vst_dir="/opt/daw_engine/vst3", sf_dir="/opt/daw_engine/soundfonts", upload_sf_dir=None) -> "PluginManager":
"""Singleton: reuse PluginManager when args match, else create new."""
global _PLUGIN_MANAGER_INSTANCE, _PLUGIN_MANAGER_ARGS
args = (vst_dir, sf_dir, upload_sf_dir)
if _PLUGIN_MANAGER_INSTANCE is not None and _PLUGIN_MANAGER_ARGS == args:
return _PLUGIN_MANAGER_INSTANCE
_PLUGIN_MANAGER_ARGS = args
_PLUGIN_MANAGER_INSTANCE = PluginManager(vst_dir, sf_dir, upload_sf_dir)
return _PLUGIN_MANAGER_INSTANCE
def load_soundfont_cached(path: str):
"""Return a cached FluidSynth instance for path, incrementing refcount."""
global _FLUID_CACHE
if not HAS_PYFLUIDSYNTH:
return None
if path in _FLUID_CACHE:
fl, ref = _FLUID_CACHE[path]
_FLUID_CACHE[path] = (fl, ref + 1)
return fl
try:
import fluidsynth
fl = fluidsynth.FluidSynth(sample_rate=44100, gain=0.5)
font_id = fl.sfload(path)
fl.program_select(0, font_id, 0, 0)
_FLUID_CACHE[path] = (fl, 1)
return fl
except Exception:
return None
def release_soundfont(path: str):
"""Decrement refcount; delete FluidSynth when count reaches 0."""
global _FLUID_CACHE
if path not in _FLUID_CACHE:
return
fl, ref = _FLUID_CACHE[path]
if ref <= 1:
try:
fl.delete()
except Exception:
pass
del _FLUID_CACHE[path]
else:
_FLUID_CACHE[path] = (fl, ref - 1)
class PluginManager:
def __init__(self, vst_dir="/opt/daw_engine/vst3", sf_dir="/opt/daw_engine/soundfonts", upload_sf_dir=None):
self.vst_dir = vst_dir
self.sf_dir = sf_dir
self.upload_sf_dir = upload_sf_dir
self._sf_scan_cache = None # cache for _scan_soundfonts()
def _scan_plugins(self) -> dict:
plugins = {}
if not os.path.isdir(self.vst_dir):
return plugins
for root, dirs, files in os.walk(self.vst_dir):
for file in files:
if file.endswith(".vst3") or file.endswith(".so"):
plugin_path = os.path.join(root, file)
plugin_name = os.path.splitext(file)[0]
plugins[plugin_name] = plugin_path
return plugins
def _scan_soundfonts(self) -> list:
sfonts = []
dirs = [self.sf_dir]
if self.upload_sf_dir and self.upload_sf_dir != self.sf_dir:
dirs.append(self.upload_sf_dir)
# Load metadata cache for upload soundfonts
meta_cache = {}
if self.upload_sf_dir and os.path.isdir(self.upload_sf_dir):
for f in os.listdir(self.upload_sf_dir):
if f.endswith(".meta"):
try:
import json
with open(os.path.join(self.upload_sf_dir, f), "r") as mf:
meta_cache[os.path.splitext(f)[0]] = json.load(mf)
except Exception:
pass
for d in dirs:
if not os.path.isdir(d):
continue
for f in os.listdir(d):
if f.endswith(".sf2") or f.endswith(".sf3"):
base_id = os.path.splitext(f)[0]
meta = meta_cache.get(base_id, None)
if meta:
display_name = meta.get("original_name", f)
else:
# Generate a friendly name from UUID: truncate to first 8 chars
short_id = base_id[:8] if len(base_id) > 8 else base_id
display_name = f"SoundFont_{short_id}"
sfonts.append({"id": base_id, "name": display_name, "file": f, "display": os.path.splitext(display_name)[0][:40]})
return sfonts
def load_vst(self, plugin_name: str, preset_data: dict = None):
if not HAS_PEDALBOARD:
return None
plugins = self._scan_plugins()
if plugin_name not in plugins:
return None
path = plugins[plugin_name]
vst = VST3Plugin(path)
if preset_data:
for k, v in preset_data.items():
try:
setattr(vst, k, v)
except Exception:
pass
return vst
def _scan_soundfonts_cached(self):
if self._sf_scan_cache is not None:
return self._sf_scan_cache
self._sf_scan_cache = self._scan_soundfonts()
return self._sf_scan_cache
def load_soundfont(self, path: str):
return load_soundfont_cached(path)
def list_soundfont_instruments(self, sf_id: str):
if not ensure_pyfluidsynth():
return []
if sf_id in _SF_INSTRUMENTS_CACHE:
return _SF_INSTRUMENTS_CACHE[sf_id]
search_dirs = []
if os.path.isdir(self.sf_dir):
search_dirs.append(self.sf_dir)
if self.upload_sf_dir and os.path.isdir(self.upload_sf_dir) and self.upload_sf_dir != self.sf_dir:
search_dirs.append(self.upload_sf_dir)
for d in search_dirs:
for f in os.listdir(d):
if not (f.endswith(".sf2") or f.endswith(".sf3")):
continue
base = os.path.splitext(f)[0]
if base == sf_id or base == sf_id.replace("sf_", ""):
path = os.path.join(d, f)
try:
import fluidsynth
fl = fluidsynth.Synth()
fid = fl.sfload(path)
if fid < 0:
fl.delete()
continue
presets = []
_fl = fluidsynth._fl
_fl.fluid_synth_get_sfont_by_id.restype = c_void_p
_fl.fluid_preset_get_name.restype = c_char_p
_fl.fluid_sfont_get_preset.restype = c_void_p
sfont_ptr = _fl.fluid_synth_get_sfont_by_id(c_void_p(fl.synth), c_int(fid))
if sfont_ptr:
for bank in range(0, 2):
for prog_num in range(0, 128):
try:
preset = fluidsynth.fluid_sfont_get_preset(sfont_ptr, c_int(bank), c_int(prog_num))
except Exception:
break
if preset:
name_ptr = fluidsynth.fluid_preset_get_name(preset)
if name_ptr:
name_val = c_char_p(name_ptr).value
if name_val:
presets.append({
"bank": bank,
"program": prog_num,
"name": name_val.decode("utf-8", errors="replace")
})
fl.delete()
_SF_INSTRUMENTS_CACHE[sf_id] = presets[:256]
return presets[:256]
except Exception:
import traceback; traceback.print_exc()
_SF_INSTRUMENTS_CACHE[sf_id] = []
return []
def list_available(self) -> dict:
return {
"vst_instruments": [
{"id": k, "name": k, "type": "VST3", "has_native_support": HAS_PEDALBOARD}
for k in self._scan_plugins().keys()
],
"soundfonts": self._scan_soundfonts()
}
@staticmethod
def midi_events_to_messages(midi_events: list, bpm: float, sr: int) -> list:
if not HAS_PEDALBOARD:
return []
beat_duration_sec = 60.0 / max(30.0, bpm)
messages = []
for ev in midi_events:
note = ev.get("note", 60)
velocity = ev.get("velocity", 100)
start_beat = ev.get("start_beat", 0.0)
dur_beats = ev.get("duration_beats", 1.0)
start_sec = start_beat * beat_duration_sec
dur_sec = dur_beats * beat_duration_sec
sample_offset = int(start_sec * sr)
end_sample_offset = int((start_sec + dur_sec) * sr)
messages.append(MidiMessage(note_on=note, velocity=velocity, sample_offset=sample_offset))
messages.append(MidiMessage(note_off=note, velocity=0, sample_offset=end_sample_offset))
return messages
@staticmethod
def validate_sf2_header(data: bytes) -> bool:
if len(data) < 12:
return False
if data[0:4] != b'RIFF':
return False
if data[8:12] != b'sfbk':
return False
return True
+2
View File
@@ -12,6 +12,7 @@ from app.api.v1.admin import router as admin_router
from app.api.v1.projects import router as projects_router from app.api.v1.projects import router as projects_router
from app.api.v1.user_config import router as user_config_router from app.api.v1.user_config import router as user_config_router
from app.api.v1.ai_proxy import router as ai_proxy_router from app.api.v1.ai_proxy import router as ai_proxy_router
from app.api.v1.plugins import router as plugins_router
from app.core.auth import seed_admin from app.core.auth import seed_admin
# Ensure storage directories exist # Ensure storage directories exist
@@ -47,6 +48,7 @@ app.include_router(admin_router, prefix="/api/v1/admin", tags=["admin"])
app.include_router(projects_router, prefix="/api/v1/projects", tags=["projects"]) app.include_router(projects_router, prefix="/api/v1/projects", tags=["projects"])
app.include_router(user_config_router, prefix="/api/v1/user", tags=["user_config"]) app.include_router(user_config_router, prefix="/api/v1/user", tags=["user_config"])
app.include_router(ai_proxy_router, prefix="/api/v1/ai", tags=["ai"]) app.include_router(ai_proxy_router, prefix="/api/v1/ai", tags=["ai"])
app.include_router(plugins_router, prefix="/api/v1/plugins", tags=["plugins"])
# Seed admin user on startup # Seed admin user on startup
@app.on_event("startup") @app.on_event("startup")
+6
View File
@@ -47,3 +47,9 @@ body {
font-size: 11px; font-size: 11px;
backdrop-filter: blur(4px); backdrop-filter: blur(4px);
} }
/* Disable mouse pointer events on SVG icons to prevent SVGAnimatedString className type errors in event listeners */
svg, i[data-lucide] {
pointer-events: none;
}
+3038 -427
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+41 -3
View File
@@ -46,6 +46,43 @@ const AIGateway = (function() {
name: 'export_audio', description: 'Xuất file WAV/MP3/OGG và tải về', parameters: { type: 'object', properties: { track_id: { type: 'string' }, format: { type: 'string', enum: ['wav', 'mp3', 'ogg'] }, sample_rate: { type: 'string', enum: ['22500', '44100'] }, bit_depth: { type: 'string', enum: ['8', '16', '24'] }, quality: { type: 'string', enum: ['44khz', 'lossless'] }, channels: { type: 'string', enum: ['mono', 'stereo'] }, start_time: { type: 'number' }, end_time: { type: 'number' }, start_bar: { type: 'number' }, length_bars: { type: 'number' } }, required: ['format'] } name: 'export_audio', description: 'Xuất file WAV/MP3/OGG và tải về', parameters: { type: 'object', properties: { track_id: { type: 'string' }, format: { type: 'string', enum: ['wav', 'mp3', 'ogg'] }, sample_rate: { type: 'string', enum: ['22500', '44100'] }, bit_depth: { type: 'string', enum: ['8', '16', '24'] }, quality: { type: 'string', enum: ['44khz', 'lossless'] }, channels: { type: 'string', enum: ['mono', 'stereo'] }, start_time: { type: 'number' }, end_time: { type: 'number' }, start_bar: { type: 'number' }, length_bars: { type: 'number' } }, required: ['format'] }
}, { }, {
name: 'fade_out', description: 'Fade-out clip (0.5s đến max)', parameters: { type: 'object', properties: { track_id: { type: 'string' }, duration_seconds: { type: 'number' }, clip_index: { type: 'number', description: 'Chỉ số của clip trên track (1-based, ví dụ: 1 cho clip 1, 2 cho clip 2)' }, clip_id: { type: 'string', description: 'ID của clip cụ thể' } } } name: 'fade_out', description: 'Fade-out clip (0.5s đến max)', parameters: { type: 'object', properties: { track_id: { type: 'string' }, duration_seconds: { type: 'number' }, clip_index: { type: 'number', description: 'Chỉ số của clip trên track (1-based, ví dụ: 1 cho clip 1, 2 cho clip 2)' }, clip_id: { type: 'string', description: 'ID của clip cụ thể' } } }
}, {
name: 'generate_multitrack_midi',
description: 'Generates multi-track MIDI data based on genre, bar duration, and requested instruments list.',
parameters: {
type: 'object',
properties: {
composition_title: { type: 'string', description: 'Title of the musical piece (e.g., Epic Orchestra Intro 8-Bars)' },
bpm: { type: 'integer' },
total_bars: { type: 'integer' },
tracks: {
type: 'array',
description: 'Array of instrument tracks along with their corresponding MIDI notes',
items: {
type: 'object',
properties: {
track_name: { type: 'string', description: 'Track name (e.g., String Ensemble, Epic Brass, Taiko Drums)' },
instrument_type: { type: 'string', enum: ['STRINGS', 'BRASS', 'SYNTH', 'PERCUSSION', 'DRUMS'] },
notes: {
type: 'array',
items: {
type: 'object',
properties: {
pitch: { type: 'integer', description: 'MIDI note pitch from 0 to 127 (e.g., C4 = 60, C3 = 48)' },
start_beat: { type: 'number', description: 'Note start position in beats (from 0.0 to total_bars * 4.0)' },
duration_beats: { type: 'number', description: 'Note length in beats (e.g., Quarter note = 1.0, Eighth note = 0.5)' },
velocity: { type: 'number', description: 'Keypress velocity intensity from 0.0 to 1.0' }
},
required: ['pitch', 'start_beat', 'duration_beats', 'velocity']
}
}
},
required: ['track_name', 'instrument_type', 'notes']
}
}
},
required: ['composition_title', 'bpm', 'total_bars', 'tracks']
}
}]; }];
function parseOrigin(urlStr) { function parseOrigin(urlStr) {
@@ -139,12 +176,13 @@ const AIGateway = (function() {
return calls; return calls;
} }
function buildUserMessage(prompt, context) { function buildUserMessage(prompt, context, systemInstruction = '') {
const contextStr = JSON.stringify(context, null, 2); const contextStr = JSON.stringify(context, null, 2);
const toolNames = DEFAULT_TOOLS.map(t => ` - ${t.name}: ${t.description}`).join('\n'); const toolNames = DEFAULT_TOOLS.map(t => ` - ${t.name}: ${t.description}`).join('\n');
return [ return [
{ role: 'system', content: `Bạn là trợ lý điều khiển DAW chuyên nghiệp. { role: 'system', content: `Bạn là trợ lý điều khiển DAW chuyên nghiệp.
Nhiệm vụ của bạn là phân tích yêu cầu của người dùng và chuyển đổi thành danh sách các function calls tương ứng. Nhiệm vụ của bạn là phân tích yêu cầu của người dùng và chuyển đổi thành danh sách các function calls tương ứng.
${systemInstruction ? `\nHướng dẫn tạo nhạc đặc biệt từ Preset:\n${systemInstruction}\n` : ''}
QUAN TRỌNG: QUAN TRỌNG:
1. Bạn đang hoạt động ở chế độ một lượt (one-shot). Hãy trả về TẤT CẢ các function calls cần thiết để thực hiện toàn bộ các bước trong yêu cầu của người dùng trong một phản hồi duy nhất. Đừng thực hiện từng bước qua nhiều lượt chat. 1. Bạn đang hoạt động ở chế độ một lượt (one-shot). Hãy trả về TẤT CẢ các function calls cần thiết để thực hiện toàn bộ các bước trong yêu cầu của người dùng trong một phản hồi duy nhất. Đừng thực hiện từng bước qua nhiều lượt chat.
2. Có thể gọi nhiều function cùng một lúc (gọi song song/nối tiếp). Chúng sẽ được thực thi theo thứ tự bạn trả về. 2. Có thể gọi nhiều function cùng một lúc (gọi song song/nối tiếp). Chúng sẽ được thực thi theo thứ tự bạn trả về.
@@ -187,8 +225,8 @@ Ví dụ: "Hãy chọn và copy từ bar 4 đến bar 12 của track 1 sau đó
}; };
} }
async function executeAIPrompt({ prompt, provider, model, apiKey, baseUrl, dawContext, tools }) { async function executeAIPrompt({ prompt, provider, model, apiKey, baseUrl, dawContext, tools, systemInstruction }) {
const messages = buildUserMessage(prompt, dawContext); const messages = buildUserMessage(prompt, dawContext, systemInstruction);
const toolList = tools || DEFAULT_TOOLS; const toolList = tools || DEFAULT_TOOLS;
const completion = await callLLM({ const completion = await callLLM({
+22 -1
View File
@@ -60,6 +60,27 @@ window.API_BASE_URL = window.API_BASE_URL || window.location.origin;
saveAIConfigs: (providers) => apiRequest('/api/v1/user/config/ai', { method: 'POST', body: JSON.stringify({ providers }) }), saveAIConfigs: (providers) => apiRequest('/api/v1/user/config/ai', { method: 'POST', body: JSON.stringify({ providers }) }),
getPreferences: () => apiRequest('/api/v1/user/preferences', { method: 'GET' }), getPreferences: () => apiRequest('/api/v1/user/preferences', { method: 'GET' }),
savePreferences: (prefs) => apiRequest('/api/v1/user/preferences', { method: 'POST', body: JSON.stringify({ preferences: prefs }) }) savePreferences: (prefs) => apiRequest('/api/v1/user/preferences', { method: 'POST', body: JSON.stringify({ preferences: prefs }) }),
listPlugins: () => apiRequest('/api/v1/plugins/available', { method: 'GET' }),
listDefaultSoundfonts: () => apiRequest('/api/v1/plugins/default-soundfonts', { method: 'GET' }),
listSoundfontInstruments: (sfId) => apiRequest(`/api/v1/plugins/soundfont-instruments/${sfId}`, { method: 'GET' }),
renderProject: (projectJson, outputFilename) => apiRequest('/api/v1/plugins/render', { method: 'POST', body: JSON.stringify({ project_json: projectJson, output_filename: outputFilename }) }),
deleteSoundFont: (sfId) => apiRequest(`/api/v1/plugins/soundfont/${sfId}`, { method: 'DELETE' }),
uploadSoundFont: async (file) => {
const formData = new FormData();
formData.append('file', file);
const token = localStorage.getItem('sonic_token') || '';
const resp = await fetch(`${window.API_BASE_URL}/api/v1/plugins/upload-soundfont`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${token}` },
body: formData
});
if (!resp.ok) {
const err = await resp.json().catch(() => ({}));
throw new Error(err.detail || 'Upload failed');
}
return resp.json();
}
}; };
})(); })();
@@ -68,9 +68,10 @@ const DAWCommandDispatcher = (function() {
register('SET_PLAYHEAD', (args) => api.setPlayhead(args)); register('SET_PLAYHEAD', (args) => api.setPlayhead(args));
register('SELECT_ITEM', (args) => api.selectItem(args)); register('SELECT_ITEM', (args) => api.selectItem(args));
register('ADD_MARKER', (args) => api.addMarker(args)); register('ADD_MARKER', (args) => api.addMarker(args));
register('CREATE_MIDI_ITEM', (args) => AIGateway.createMidiItem(args)); register('CREATE_MIDI_ITEM', (args) => api.createMidiItem(args));
register('MODIFY_MIDI_NOTES', (args) => AIGateway.modifyMidiNotes(args)); register('MODIFY_MIDI_NOTES', (args) => api.modifyMidiNotes(args));
register('PROCESS_AI_DSP', (args) => AIGateway.processAIDSP(args)); register('PROCESS_AI_DSP', (args) => api.processAudioDsp(args));
register('GENERATE_MULTITRACK_MIDI', (args) => api.generateMultitrackMidi(args));
} }
return { return {
+190
View File
@@ -0,0 +1,190 @@
// SonicForge Studio SoundFont Player Service
(function () {
'use strict';
const activeOscillators = {};
// Use the shared AudioContext from the main app (lazy init)
let __gainNode = null;
const getCtx = () => {
if (typeof getAudioContext === 'function') {
const ctx = getAudioContext();
if (!__gainNode) {
__gainNode = ctx.createGain();
__gainNode.gain.value = 0.3;
__gainNode.connect(ctx.destination);
}
return ctx;
}
if (!window.__sharedAudioCtx) {
window.__sharedAudioCtx = new (window.AudioContext || window.webkitAudioContext)();
}
if (window.__sharedAudioCtx.state === 'suspended') {
window.__sharedAudioCtx.resume();
}
if (!__gainNode) {
__gainNode = window.__sharedAudioCtx.createGain();
__gainNode.gain.value = 0.3;
__gainNode.connect(window.__sharedAudioCtx.destination);
}
return window.__sharedAudioCtx;
};
const SonicSF = {
loadedFonts: {},
// Load SoundFont from URL → ArrayBuffer → store in memory
loadSoundFont: async function (url) {
if (this.loadedFonts[url]) return this.loadedFonts[url];
const resp = await fetch(url);
if (!resp.ok) throw new Error('Failed to load SoundFont: ' + url);
const buffer = await resp.arrayBuffer();
this.loadedFonts[url] = buffer;
return buffer;
},
playNote: function (note, velocity, durationMs, startTime, program, destinationNode) {
const ctx = getCtx();
const freq = 440 * Math.pow(2, (note - 69) / 12);
if (freq <= 0 || isNaN(freq)) return null;
const osc = ctx.createOscillator();
const noteGain = ctx.createGain();
// Default settings
let oscType = 'triangle';
let attackTime = 0.01;
let decayTime = 0.1;
let sustainLevel = 0.5;
let releaseTime = 0.2;
let volFactor = 0.3;
const prog = program !== undefined ? parseInt(program) : 0;
if (prog >= 0 && prog <= 7) { // Pianos
oscType = 'sine';
decayTime = 0.3;
sustainLevel = 0.1;
releaseTime = 0.2;
} else if (prog >= 8 && prog <= 15) { // Chromatic Perc
oscType = 'sine';
decayTime = 0.1;
sustainLevel = 0.0;
releaseTime = 0.1;
} else if (prog >= 16 && prog <= 23) { // Organs
oscType = 'sine';
attackTime = 0.05;
sustainLevel = 0.8;
releaseTime = 0.1;
} else if (prog >= 24 && prog <= 31) { // Guitars
oscType = 'triangle';
decayTime = 0.4;
sustainLevel = 0.2;
releaseTime = 0.3;
} else if (prog >= 32 && prog <= 39) { // Basses
oscType = 'triangle';
attackTime = 0.02;
decayTime = 0.2;
sustainLevel = 0.6;
releaseTime = 0.2;
} else if (prog >= 40 && prog <= 47) { // Strings
oscType = 'sawtooth';
attackTime = 0.15;
sustainLevel = 0.8;
releaseTime = 0.5;
volFactor = 0.15;
} else if (prog >= 48 && prog <= 55) { // Ensemble / Choir
oscType = 'sawtooth';
attackTime = 0.2;
sustainLevel = 0.8;
releaseTime = 0.6;
volFactor = 0.12;
} else if (prog >= 56 && prog <= 63) { // Brass
oscType = 'sawtooth';
attackTime = 0.08;
sustainLevel = 0.7;
releaseTime = 0.3;
volFactor = 0.15;
} else if (prog >= 64 && prog <= 71) { // Reed
oscType = 'square';
attackTime = 0.05;
sustainLevel = 0.6;
releaseTime = 0.2;
volFactor = 0.15;
} else if (prog >= 72 && prog <= 79) { // Pipe
oscType = 'sine';
attackTime = 0.1;
sustainLevel = 0.7;
releaseTime = 0.3;
volFactor = 0.2;
} else if (prog >= 80 && prog <= 119) { // Synth Lead/Pad/FX
oscType = 'sawtooth';
attackTime = 0.05;
sustainLevel = 0.6;
releaseTime = 0.4;
volFactor = 0.15;
}
osc.type = oscType;
osc.frequency.value = freq;
const startAt = startTime !== undefined ? startTime : ctx.currentTime;
const durSec = durationMs / 1000;
const vel = typeof velocity === 'number' ? (velocity > 1 ? velocity / 127 : velocity) : 0.8;
const targetGain = vel * volFactor;
// ADSR Envelope
noteGain.gain.setValueAtTime(0, startAt);
noteGain.gain.linearRampToValueAtTime(targetGain, startAt + attackTime);
noteGain.gain.linearRampToValueAtTime(targetGain * sustainLevel, startAt + attackTime + decayTime);
const releaseStart = startAt + Math.max(attackTime + decayTime, durSec);
noteGain.gain.setValueAtTime(targetGain * sustainLevel, releaseStart);
noteGain.gain.exponentialRampToValueAtTime(0.001, releaseStart + releaseTime);
osc.connect(noteGain);
const dest = destinationNode || __gainNode || ctx.destination;
noteGain.connect(dest);
osc.start(startAt);
const stopAt = releaseStart + releaseTime + 0.05;
osc.stop(stopAt);
const oscId = `${note}_${Date.now()}_${Math.random()}`;
activeOscillators[oscId] = osc;
// Clean up active oscillator reference after it stops
setTimeout(() => {
delete activeOscillators[oscId];
}, (stopAt - ctx.currentTime) * 1000 + 100);
return osc;
},
stopAll: function () {
Object.values(activeOscillators).forEach(osc => {
try { osc.stop(); } catch (e) { }
});
Object.keys(activeOscillators).forEach(k => delete activeOscillators[k]);
},
// Save user SoundFont to IndexedDB via window.SonicStorage
saveToIndexedDB: async function (name, arrayBuffer) {
if (window.SonicStorage && window.SonicStorage.saveToIndexedDB) {
await window.SonicStorage.saveToIndexedDB('soundfont_' + name, arrayBuffer);
}
},
// Load user SoundFont from IndexedDB
loadFromIndexedDB: async function (name) {
if (window.SonicStorage && window.SonicStorage.loadFromIndexedDB) {
return await window.SonicStorage.loadFromIndexedDB('soundfont_' + name);
}
return null;
}
};
window.SonicSF = SonicSF;
})();
Binary file not shown.
@@ -0,0 +1 @@
{"original_name": "weedsgm3.sf2", "uuid": "518e850f-a5d3-4790-b1f9-0c90c203c524", "file": "518e850f-a5d3-4790-b1f9-0c90c203c524.sf2"}
+13 -6
View File
@@ -10,12 +10,14 @@
<script src="https://unpkg.com/lucide@latest"></script> <script src="https://unpkg.com/lucide@latest"></script>
<script src="https://unpkg.com/react@18.3.1/umd/react.production.min.js"></script> <script src="https://unpkg.com/react@18.3.1/umd/react.production.min.js"></script>
<script src="https://unpkg.com/react-dom@18.3.1/umd/react-dom.production.min.js"></script> <script src="https://unpkg.com/react-dom@18.3.1/umd/react-dom.production.min.js"></script>
<script src="/static/js/services/api.js"></script> <script src="/static/js/services/api.js?v=202607232105"></script>
<script src="/static/js/services/audioEngine.js"></script> <script src="/static/js/services/audioEngine.js?v=202607232105"></script>
<script src="/static/js/services/storage.js"></script> <script src="/static/js/services/storage.js?v=202607232105"></script>
<script src="/static/js/services/aiGateway.js"></script> <script src="/static/js/services/soundfontPlayer.js?v=202607232105"></script>
<script src="/static/js/services/dawCommandDispatcher.js"></script> <script src="/static/js/services/aiGateway.js?v=202607232105"></script>
<script src="/static/js/app.precompiled.js?v=202607231122" defer></script> <script src="/static/js/services/dawCommandDispatcher.js?v=202607232105"></script>
<script src="/static/js/app.precompiled.js?v=202607232105" defer></script>
<link rel="stylesheet" href="/static/css/styles.css?v=202607232105">
<style> <style>
:root { :root {
--right-sidebar-width: 320px; --right-sidebar-width: 320px;
@@ -228,6 +230,11 @@
.resizer-row-handle:active { .resizer-row-handle:active {
background: #00ffcc; background: #00ffcc;
} }
/* Disable mouse pointer events on SVG icons to prevent SVGAnimatedString className type errors in event listeners */
svg, i[data-lucide] {
pointer-events: none;
}
</style> </style>
</head> </head>
+105
View File
@@ -0,0 +1,105 @@
# Kế hoạch Thực hiện: Tính năng DAW mới (Track FX, Section Preview & Piano Roll Edit Shortcuts)
Bản kế hoạch này mô tả thiết kế kỹ thuật và các bước thực hiện các tính năng nâng cao liên quan đến xử lý hiệu ứng âm thanh (FX Chain) song song giữa Web Audio Client và Python Backend, kết xuất trực quan Section item trên Timeline, và bổ sung các phím tắt biên tập thông minh trong Piano Roll.
## Điểm lưu ý từ Người dùng
1. **Kiến trúc hiệu ứng Song song (Dual FX Architecture)**:
- **Phía Client (Trình duyệt)**: Vẫn phải sử dụng Web Audio API để xử lý hiệu ứng thời gian thực (Real-time Preview) khi nhấn Play nghe thử.
- **Phía Server (Backend)**: Sử dụng các thư viện Python (`Pedalboard`, `pydub`, `pysndfx`, `Librosa`...) để xử lý hiệu ứng khi xuất bản kết xuất (Offline Export / Render).
2. **Timeline Playback**: Khi Playhead trên Main Session quét qua Section item, trình phát nhạc sẽ phát nội dung của Section item đó giống như đang nghe thử trên Section-tab.
3. **Sửa lỗi Release Note**: Khi click chọn nốt mới để kéo thả, các nốt cũ đã chọn trước đó phải được giải phóng hoàn toàn và không di chuyển theo nốt mới.
---
## Phân tích Ưu/Nhược điểm & Giải pháp Tối ưu hóa (Dual FX)
### 1. Ưu điểm (Pros)
* **Trải nghiệm Tức thì (Zero Latency)**: Người dùng nghe thấy hiệu ứng ngay lập tức khi kéo nút hoặc đổi chế độ mà không cần đợi gửi file lên server xử lý.
* **Giảm tải cực lớn cho Server**: Trình duyệt tự gánh vác phần giải mã và xử lý DSP thời gian thực trên thiết bị người dùng, máy chủ chỉ cần chạy xử lý khi người dùng xuất bản (Export).
* **Chất lượng Master tuyệt đối**: Bản xuất ra (Render) trên Backend sử dụng thuật toán C++ tối ưu cao của `pedalboard` hoặc các bộ lọc khoa học của `scipy` giúp chất lượng âm thanh đạt chuẩn phòng thu, không bị méo tiếng do giới hạn tài nguyên của trình duyệt.
### 2. Nhược điểm (Cons)
* **Độ lệch âm thanh (Sound Discrepancy)**: Bản nghe thử trên trình duyệt (Web Audio API) và bản xuất ra (Python Backend) có thể có khác biệt nhỏ về màu sắc âm thanh nếu thuật toán tạo Chorus/Reverb khác nhau.
* **Trùng lặp mã nguồn (Code Duplication)**: Phải viết logic xử lý hiệu ứng ở cả 2 ngôn ngữ (JavaScript trên Client và Python trên Server).
### 3. Giải pháp Tối ưu hóa tốt nhất (Optimal Strategies)
Để tối ưu hóa hiệu suất và chất lượng, chúng ta áp dụng các giải pháp sau:
* **Đồng bộ hóa tham số (Unified Parameters)**: Cả Client và Server đều đọc chung các tham số hiệu ứng từ tệp JSON cấu hình dự án (ví dụ: `chorus_rate = 1.5 Hz`, `reverb_room_size = 0.5`).
* **Đồng hóa thuật toán (DSP Matching)**:
* *Chorus*: Cài đặt LFO sine điều tần thời gian trễ đồng bộ ở cả 2 bên.
* *Reverb*:
* **Client**: Sử dụng `ConvolverNode` nạp bộ đệm xung tự sinh (synthetic impulse response) dài 2 giây từ thuật toán nhiễu trắng phân rã lũy thừa.
* **Server**: Sử dụng `pedalboard.Reverb` hoặc thuật toán chập tín hiệu phân rã tương đương trong Python.
* **Tối ưu hóa tài nguyên phía Server**:
* Không nạp lại SoundFont hoặc thư viện nhiều lần; sử dụng cơ chế singleton hoặc caching cho các module xử lý hiệu ứng của Python.
---
## Đề xuất Thay đổi cụ thể
### 1. Kiến trúc hiệu ứng Track FX (Chorus & Reverb)
#### Phía Client (Web Audio API)
Chúng ta sẽ bổ sung chuỗi liên kết hiệu ứng âm thanh trực tiếp vào nút nguồn của từng track trong [app.jsx](file:///home/locpham/SonicForgeStudio/app/static/js/app.jsx). Bất cứ nguồn âm thanh nào đi qua track (cả clip tiếng động lẫn âm thanh MIDI qua SoundFont) đều chịu tác dụng của hiệu ứng:
- **Chorus**: Sử dụng một dry gain và một wet gain kết nối tới LFO-modulated delay node. LFO dao động hình sin tần số `1.5 Hz` với độ lệch delay `2 ms` tạo tiếng đồng ca.
- **Reverb**: Sử dụng dry gain, wet gain và `ConvolverNode` chứa bộ đệm phản hồi âm thanh nhân tạo tự sinh dài 2 giây từ thuật toán nhiễu trắng phân rã mũ.
- **Giao diện Chọn hiệu ứng**: Khi người dùng nhấn nút **FX: None** trên Track Header, một popup selector sẽ hiển thị để người dùng chọn: **None / Chorus / Reverb**, cập nhật thuộc tính `track.fxType`.
#### Phía Server (Python Rendering)
Khi kết xuất dự án trong [render_engine.py](file:///home/locpham/SonicForgeStudio/app/core/render_engine.py), chúng ta đọc thuộc tính `fx_type` từ track:
- Nếu `track.fx_type == 'chorus'`: Sử dụng lớp `Chorus` của `pedalboard` (nếu có thư viện), hoặc dùng `scipy.signal` để tạo dịch pha điều tần chậm.
- Nếu `track.fx_type == 'reverb'`: Sử dụng lớp `Reverb` của `pedalboard`, hoặc sử dụng chập phản hồi âm để tạo vang.
---
### 2. Vẽ lại trực quan Section item (Waveform nền & Note màu)
Cập nhật thuật toán vẽ Section item trên Canvas Timeline trong [app.jsx](file:///home/locpham/SonicForgeStudio/app/static/js/app.jsx):
- **Phông nền Waveform**: Duyệt qua tối đa 4 track con trong `sec.tracks`. Nếu track con chứa audio clip có `buffer` dữ liệu, lấy mẫu thu gọn 100 peaks và vẽ thành đồ thị cột phân rã đối xứng màu xanh cyan làm nền bên trong phân vùng của Section.
- **Vẽ note MIDI theo màu sắc**: Quét các MIDI item trong các track con. Vẽ các note nhạc đè lên waveform với màu sắc tương trưng cho thứ tự track con (ví dụ: track 1 dùng màu vàng hổ phách `#fbbf24`, track 2 dùng màu tím `#a78bfa`, v.v.), giúp người dùng nhận diện nhanh cấu trúc hòa âm.
- **Phát Section Item trên Main Session**: Đảm bảo thuật toán lập lịch phát âm thanh trong `startTrackPlayback` quét qua toàn bộ clip và nốt nhạc con của Section item để lên lịch phát nhạc đồng bộ khi playhead quét qua.
---
### 3. Phím tắt Nâng cao trong Piano Roll
Chúng ta sẽ chỉnh sửa các hàm xử lý sự kiện chuột của canvas Piano Roll trong [app.jsx](file:///home/locpham/SonicForgeStudio/app/static/js/app.jsx):
#### Giải phóng nốt cũ khi chọn nốt mới (Sửa lỗi Drag & Release)
- Trong `handleGridMouseDown`, khi người dùng click vào một nốt mới nằm ngoài vùng chọn hiện tại, chúng ta sẽ cập nhật danh sách chọn lựa bằng một biến cục bộ đồng bộ `nextSelectedIds = [clickedNote.id]` thay vì sử dụng state bất đồng bộ `selectedNoteIds`. Điều này đảm bảo danh sách nốt bị dịch chuyển (`selectedNotesOffset`) chỉ chứa duy nhất nốt mới click, giải phóng hoàn toàn nốt cũ.
#### Sao chép nốt nhanh (Ctrl + Click + Drag)
- Trong `handleGridMouseDown`, nếu nhấn chuột trái đồng thời đè phím `Ctrl` trên một nốt nhạc:
- Tạo các bản sao nhân bản (cloned copies) của tất cả các nốt đang được chọn với ID ngẫu nhiên mới.
- Thêm các nốt nhân bản này vào danh sách `notes` của tab.
- Chuyển trạng thái `draggedNote` sang chế độ di chuyển (`move`) áp dụng trực tiếp lên các bản sao mới này, giữ nguyên các nốt gốc ở vị trí cũ.
#### Giãn/Thu tỷ lệ thời gian các nốt (Alt + Drag Resize)
- Khi người dùng nhấn giữ phím `Alt` và kéo cạnh phải (resize edge) của một nốt trong nhóm đang được chọn:
- Xác định thời điểm bắt đầu của nốt đầu tiên trong nhóm tuyển chọn (`firstStartBeat`) và điểm kết thúc ban đầu của nốt bị kéo (`originalDraggedEndBeat`).
- Tính toán tỷ lệ co giãn thời gian:
$$\text{scaleFactor} = \frac{\text{newDraggedEndBeat} - \text{firstStartBeat}}{\text{originalDraggedEndBeat} - \text{firstStartBeat}}$$
- Cập nhật thời điểm bắt đầu (`start_beat`) và thời lượng (`duration_beats`) của tất cả các nốt nhạc được chọn có thời điểm bắt đầu nhỏ hơn hoặc bằng điểm kết thúc ban đầu của nốt bị kéo bằng cách nhân với `scaleFactor`.
- Các nốt nhạc nằm sau vị trí kéo (ví dụ nốt thứ 4) sẽ được giữ nguyên không đổi.
#### Vẽ nhiều nốt bằng cách di chuột (Brush/Drag to Draw)
- Khi đang kéo vẽ nốt mới, lưu vết mảng các cao độ (pitch/row) đã đi qua trong `draggedNote.visitedPitches`.
- Khi di chuyển chuột qua cao độ mới, thêm cao độ đó vào danh sách và phân bổ đều tổng khoảng cách kéo (`beat - startOffsetBeat`) thành các nốt nhạc nối tiếp nhau, mỗi nốt có thời lượng bằng `totalSpan / visitedPitches.length`.
---
## Kế hoạch Kiểm thử
### 1. Kiểm thử hiệu ứng âm thanh FX
- **Thời gian thực (Client)**: Kích hoạt Chorus/Reverb trên track, bấm Play nghe thử để xác nhận tiếng vang/tiếng đồng ca chạy mượt mà.
- **Kết xuất (Backend)**: Mixdown dự án có track bật Chorus hoặc Reverb, kiểm tra file wav đầu ra xem hiệu ứng có được áp dụng chuẩn.
### 2. Kiểm thử vẽ trực quan & Phát nhạc Section
- Thêm nốt nhạc và audio clip vào Section, nhấn lưu. Kiểm tra xem Section item hiển thị đúng dạng sóng và nốt màu.
- Phát nhạc trên Main Session, kiểm tra xem khi playhead đi qua Section item thì âm thanh của Section có phát ra đúng nhịp.
### 3. Kiểm thử phím tắt Piano Roll
- Kiểm tra click chọn nốt mới để di chuyển xem nốt cũ có được giải phóng hoàn toàn và không di chuyển theo.
- Giữ `Ctrl` kéo nốt để sao chép.
- Giữ `Alt` kéo giãn nhóm nốt.
- Di chuột chéo để vẽ chuỗi nốt bậc thang (Brush tool).
-174
View File
@@ -1,174 +0,0 @@
# DIAGNOSTIC REPORT: WHY MIDI SIGNAL IS RECEIVED BUT NOT RECORDED / RENDERED
---
## 1. Executive Summary & Root Cause Analysis
Based on the DAW UI screenshot provided, the system is successfully receiving MIDI hardware signals (as indicated by the active VU meter on Track 01 set to `MIDIIN2 (SE49)`), but no MIDI data is being written or displayed on the timeline.
This issue occurs due to four architectural and state-management gaps in the current implementation.
---
## 2. Detailed Root Causes
### Root Cause 1: Global Transport Record vs. Track Arm Disconnect
* **Observed State:** Track 01 has its individual Arm `[R]` button active (red indicator ON). However, the Global Transport Record button (red circle on the top toolbar) is inactive/stopped at time position `0:01.951`.
* **Technical Issue:** Arming a track only enables Live Monitoring (routing MIDI input to the virtual synth engine for real-time audio playback). Recording MIDI into timeline buffers requires both **Track Arm = `true**` AND **Transport Engine State = `RECORDING**`.
```text
[ Track Armed ] + [ Transport STOPPED ] --> Live Monitoring ONLY (VU meter lights up, no recording)
[ Track Armed ] + [ Transport RECORDING ] --> Live Monitoring + Event Buffer Write + Canvas Redraw
```
### Root Cause 2: Gate Condition in `handleMIDIMessage`
In the client recording engine (`ClientMIDIRecorder`), incoming MIDI events trigger live synth audio, but note recording is gated behind a transport flag:
```javascript
handleMIDIMessage(event) {
// BUG: If global transport is not in RECORD mode, execution stops here.
// Synth gets triggered elsewhere, but recordedNotes array remains empty.
if (!this.isRecording) return;
const [status, pitch, velocity] = event.data;
// ... logic to write to activeNotes and recordedNotes
}
```
### Root Cause 3: Absence of Real-Time Canvas Redraw Loop
For notes to render dynamically inside the MIDI Item clip as keys are pressed:
* The UI Canvas must run a `requestAnimationFrame` render loop while `isRecording === true`.
* The renderer must query the `activeNotes` Map (currently held keys) in addition to finalized `recordedNotes`.
* If the UI only renders on static session updates (e.g., when clicking or stopping transport), live notes will not appear on screen during playback.
### Root Cause 4: Track Target ID Unbound to Input Stream
If multiple tracks exist, `ClientMIDIRecorder` must know which `track_id` is currently armed and matched to device `MIDIIN2 (SE49)`. If events arrive without a target track context, they cannot be routed into the target `MIDIItem.source_data.notes` array.
---
## 3. Technical Solutions & Code Adjustments
### Step 1: Ensure Dual-Stage Recording State Verification
Update the transport control logic so pressing **Record + Play** on the top toolbar initializes active record buffers on all armed tracks:
```javascript
// Transport Controller
function startTransportRecording() {
const armedTracks = session.tracks.filter(t => t.is_armed);
if (armedTracks.length === 0) {
console.warn("No tracks armed for recording.");
startPlaybackOnly();
return;
}
// Activate global transport record state
transport.isRecording = true;
transport.isPlaying = true;
// Initialize temporary recording items on each armed track
armedTracks.forEach(track => {
const newRecordingItem = {
id: `rec_item_${Date.now()}`,
type: "MIDI_ITEM",
start_bar: transport.currentBar,
duration_bars: 0.1, // Expands dynamically during recording
clip_start_offset_bars: 0.0,
source_data: { total_buffer_bars: 8.0, notes: [] }
};
track.activeRecordingItem = newRecordingItem;
midiRecorder.start(track.id, transport.currentBar);
});
// Start UI animation loop for live waveform/note preview
requestAnimationFrame(renderLiveRecordingUI);
}
```
### Step 2: Live MIDI Note Binding & Duration Expansion
Update `ClientMIDIRecorder` to feed both the active buffer and the active recording clip:
```javascript
handleMIDIMessage(event) {
const [status, pitch, velocity] = event.data;
const command = status >> 4;
// 1. Always trigger Live Audio Preview (VU Meter + Synth Node)
this.triggerSynthPreview(pitch, velocity);
// 2. Gate recording buffer write behind global transport record state
if (!transport.isRecording || !this.targetTrack) return;
const currentBeat = this.calculateLatencyCompensatedBeat();
// Command 0x9: Note On
if (command === 0x9 && velocity > 0) {
const liveNote = {
id: `note_${Date.now()}_${pitch}`,
pitch: pitch,
start_beat: currentBeat,
duration_beats: 0.25, // Default initial length until Note Off
velocity: velocity / 127.0
};
this.activeNotes.set(pitch, liveNote);
this.targetTrack.activeRecordingItem.source_data.notes.push(liveNote);
}
// Command 0x8: Note Off
else if (command === 0x8 || (command === 0x9 && velocity === 0)) {
if (this.activeNotes.has(pitch)) {
const note = this.activeNotes.get(pitch);
note.duration_beats = Math.max(0.125, currentBeat - note.start_beat);
this.activeNotes.delete(pitch);
}
}
}
```
### Step 3: Real-Time UI Canvas Render Loop
Add real-time item length expansion and live note drawing on the main canvas during recording:
```javascript
function renderLiveRecordingUI() {
if (!transport.isRecording) return;
const currentBar = transport.getCurrentBarPosition();
session.tracks.forEach(track => {
if (track.is_armed && track.activeRecordingItem) {
const item = track.activeRecordingItem;
// Expand item duration on timeline as playhead moves forward
item.duration_bars = Math.max(0.5, currentBar - item.start_bar);
// Draw item bounding box and active/completed MIDI note rectangles
drawTimelineItem(trackCanvasCtx, item);
}
});
requestAnimationFrame(renderLiveRecordingUI);
}
```
---
## 4. Checklist to Fix in Your Application
* [ ] Check if clicking top toolbar **Record + Play** sets `transport.isRecording = true`.
* [ ] Verify that Track 01 generates a temporary `activeRecordingItem` on record start.
* [ ] Confirm `requestAnimationFrame` is re-rendering the canvas continuously while transport is moving.
* [ ] Ensure incoming MIDI events on `MIDIIN2 (SE49)` push notes into Track 01's item note array rather than just playing the synth.
+367
View File
@@ -0,0 +1,367 @@
# TECHNICAL SPECIFICATION: AI MIDI GENERATOR TOOL & PROMPT PRESET SYSTEM
---
## 1. System Overview
The AI Copilot system integrated within the DAW provides two core capabilities:
* **AI Tool Call (Function Calling):** Receives user requests, triggers the automated music generation engine, and returns a list of `Tracks` and `MIDIItems` complying with the JSON schema structure for direct placement onto the DAW Timeline.
* **Prompt Template Engine (Preset Library):** Manages a prompt template directory categorized by genre, mood, and song structure. Upon detecting specific keywords (e.g., *"epic orchestra"*), the engine automatically looks up and expands the raw user prompt into a System Context Standard Prompt containing music theory definitions (pitch range, scales, rhythm patterns, chord progressions, and voicing) prior to dispatching to the LLM.
```text
+-----------------------------------------------------------------------------------+
| USER INTERFACE |
| |
| [ Prompt Bar / Copilot UI ] <---> [ Prompt Template Preset Manager (CRUD) ] |
| | | |
| | Input: "Write 8 bars of | Matched Preset: |
| | epic orchestra MIDI notes..." | "Epic Orchestra Intro Spec" |
| v v |
| +---------------------------------------------------------------------------+ |
| | Prompt Context Expander Engine | |
| +--------------------------------------------------+------------------------+ |
| | |
+------------------------------------------------------|----------------------------+
| Extended Prompt Payload
v
+-----------------------------------------------------------------------------------+
| AI LLM ENGINE |
| |
| Tool Calling Execution: `generate_multitrack_midi()` |
| Output: Structured JSON Payload (Multi-track 8-bar notes) |
+------------------------------------------------------|----------------------------+
| Validated JSON Output
v
+-----------------------------------------------------------------------------------+
| DAW CLIENT STATE ENGINE |
| |
| - Parses JSON Payload |
| - Spawns/Finds Tracks (Strings, Brass, Synth, Percussion, Drums) |
| - Injects `MIDIItem` into `Main Session` / `Section Store` |
| - Re-renders Canvas & Piano Roll UI |
+-----------------------------------------------------------------------------------+
```
---
## 2. Prompt Template & Preset Engine
To eliminate the need to re-type lengthy system instructions, the application provides a Preset Manager stored as JSON format in `LocalStorage` / `IndexedDB` on the client side or within the server database.
### 2.1 Prompt Preset Schema (`preset_schema.json`)
```json
{
"id": "preset_epic_orchestra_intro",
"name": "Epic Orchestra Intro (8 Bars)",
"keywords": ["epic orchestra", "epic orchestral", "hoành tráng", "nhạc phim epic"],
"category": "Orchestral / Film Score",
"default_bars": 8,
"default_bpm": 130,
"default_scale": "C Minor",
"system_instruction_template": "You are a professional Epic Orchestral film composer. Create a powerful, dramatic 8-bar intro composition.\nThe required structure to return via the `generate_multitrack_midi` tool consists of 5 tracks:\n1. Strings Ensemble: Plays staccato 16th notes in the low register (C2, G2) driving the rhythm (Ostinato).\n2. Brass Section: Plays the main swelling melodic theme (Horn/Trumpet swells) in the C3-C5 range.\n3. Epic Percussion / Taiko: Hits heavily on beats 1 and 3, featuring a snare roll accent at bars 4 and 8.\n4. Synth Bass/Pad: Holds smooth octave foundation layers (Legato).\n5. Orchestral Drums/Cymbals: Crashes on bar 1 and bar 5.\nEnsure the duration of each track is precisely 8 bars (32 beats).",
"is_user_defined": false,
"created_at": "2026-07-23T16:00:00Z"
}
```
### 2.2 Intent Detection & Auto-Expansion Flow
When a user submits a prompt message inside the AI Copilot UI:
```javascript
class PromptTemplateManager {
constructor() {
this.presets = [];
this.loadPresets();
}
// Loads preset list from LocalStorage or API
async loadPresets() {
const localData = localStorage.getItem('daw_ai_prompt_presets');
if (localData) {
this.presets = JSON.parse(localData);
} else {
this.presets = DEFAULT_PRESETS; // Developer default fallback templates
this.savePresets();
}
}
// Matches queries against preset keywords automatically
matchPreset(userQuery) {
const queryLower = userQuery.toLowerCase();
for (const preset of this.presets) {
const hasKeyword = preset.keywords.some(kw => queryLower.includes(kw.toLowerCase()));
if (hasKeyword) {
return preset;
}
}
return null; // Fallback to raw user prompt if no keyword matches
}
// CRUD methods for user-defined presets
saveUserPreset(presetObject) {
const index = this.presets.findIndex(p => p.id === presetObject.id);
if (index >= 0) {
this.presets[index] = presetObject;
} else {
this.presets.push(presetObject);
}
this.savePresets();
}
savePresets() {
localStorage.setItem('daw_ai_prompt_presets', JSON.stringify(this.presets));
}
}
```
---
## 3. Function Calling Specification
Defines the Tool/Function schema passed to the LLM API (OpenAI / Local LLM) to enforce strict structured JSON output.
### 3.1 Function Tool Schema (`tools_spec.json`)
```json
{
"type": "function",
"function": {
"name": "generate_multitrack_midi",
"description": "Generates multi-track MIDI data based on genre, bar duration, and requested instruments list.",
"parameters": {
"type": "object",
"properties": {
"composition_title": {
"type": "string",
"description": "Title of the musical piece (e.g., Epic Orchestra Intro 8-Bars)"
},
"bpm": {
"type": "integer",
"default": 120
},
"total_bars": {
"type": "integer",
"default": 8
},
"tracks": {
"type": "array",
"description": "Array of instrument tracks along with their corresponding MIDI notes",
"items": {
"type": "object",
"properties": {
"track_name": {
"type": "string",
"description": "Track name (e.g., String Ensemble, Epic Brass, Taiko Drums)"
},
"instrument_type": {
"type": "string",
"enum": ["STRINGS", "BRASS", "SYNTH", "PERCUSSION", "DRUMS"]
},
"notes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"pitch": {
"type": "integer",
"description": "MIDI note pitch from 0 to 127 (e.g., C4 = 60, C3 = 48)"
},
"start_beat": {
"type": "number",
"description": "Note start position in beats (from 0.0 to total_bars * 4.0)"
},
"duration_beats": {
"type": "number",
"description": "Note length in beats (e.g., Quarter note = 1.0, Eighth note = 0.5)"
},
"velocity": {
"type": "number",
"description": "Keypress velocity intensity from 0.0 to 1.0",
"default": 0.8
}
},
"required": ["pitch", "start_beat", "duration_beats", "velocity"]
}
}
},
"required": ["track_name", "instrument_type", "notes"]
}
}
},
"required": ["composition_title", "bpm", "total_bars", "tracks"]
}
}
}
```
---
## 4. AI Response Payload Example
Below is an example JSON payload returned by the AI following the execution of `generate_multitrack_midi` for an 8-bar Epic Orchestra request:
```json
{
"tool_call": "generate_multitrack_midi",
"result": {
"composition_title": "Epic Orchestra Intro",
"bpm": 130,
"total_bars": 8,
"tracks": [
{
"track_name": "String Ensemble (Staccato)",
"instrument_type": "STRINGS",
"notes": [
{ "pitch": 36, "start_beat": 0.0, "duration_beats": 0.25, "velocity": 0.9 },
{ "pitch": 36, "start_beat": 0.5, "duration_beats": 0.25, "velocity": 0.85 },
{ "pitch": 48, "start_beat": 1.0, "duration_beats": 0.25, "velocity": 0.95 },
{ "pitch": 36, "start_beat": 1.5, "duration_beats": 0.25, "velocity": 0.8 }
]
},
{
"track_name": "French Horns & Brass",
"instrument_type": "BRASS",
"notes": [
{ "pitch": 60, "start_beat": 0.0, "duration_beats": 2.0, "velocity": 0.95 },
{ "pitch": 63, "start_beat": 2.0, "duration_beats": 2.0, "velocity": 0.9 },
{ "pitch": 67, "start_beat": 4.0, "duration_beats": 4.0, "velocity": 1.0 }
]
},
{
"track_name": "Epic Synth Lead",
"instrument_type": "SYNTH",
"notes": [
{ "pitch": 72, "start_beat": 4.0, "duration_beats": 1.0, "velocity": 0.85 },
{ "pitch": 75, "start_beat": 5.0, "duration_beats": 1.0, "velocity": 0.85 }
]
},
{
"track_name": "Taiko & Percussion",
"instrument_type": "PERCUSSION",
"notes": [
{ "pitch": 36, "start_beat": 0.0, "duration_beats": 0.5, "velocity": 1.0 },
{ "pitch": 36, "start_beat": 2.0, "duration_beats": 0.5, "velocity": 0.95 },
{ "pitch": 38, "start_beat": 3.5, "duration_beats": 0.25, "velocity": 0.8 }
]
},
{
"track_name": "Orchestral Cymbals",
"instrument_type": "DRUMS",
"notes": [
{ "pitch": 49, "start_beat": 0.0, "duration_beats": 4.0, "velocity": 0.9 },
{ "pitch": 49, "start_beat": 16.0, "duration_beats": 4.0, "velocity": 1.0 }
]
}
]
}
}
```
---
## 5. DAW State Ingestion Logic
When the client receives the AI JSON payload, the `ingestAIGeneratedMIDI()` function executes the following pipeline:
1. Spawns or matches corresponding tracks within `main_session`.
2. Generates 8-bar `MIDIItem` objects containing the note lists.
3. Triggers UI canvas and timeline re-renders.
```javascript
function ingestAIGeneratedMIDI(aiPayload, sessionState) {
const { composition_title, bpm, total_bars, tracks } = aiPayload.result;
// 1. Update project BPM if specified
if (bpm) sessionState.metadata.bpm = bpm;
// 2. Iterate through generated tracks
tracks.forEach((aiTrack) => {
// Match existing track or instantiate a new one
let targetTrack = sessionState.main_session.tracks.find(
t => t.name.toLowerCase() === aiTrack.track_name.toLowerCase()
);
if (!targetTrack) {
targetTrack = {
id: `track_ai_${Date.now()}_${Math.random().toString(36).substr(2, 5)}`,
name: aiTrack.track_name,
type: "MIDI",
volume_db: 0.0,
pan: 0.0,
mute: false,
solo: false,
items: []
};
sessionState.main_session.tracks.push(targetTrack);
}
// 3. Create 8-bar MIDIItem
const newMidiItem = {
id: `item_ai_${Date.now()}_${Math.random().toString(36).substr(2, 5)}`,
name: `${composition_title} - ${aiTrack.track_name}`,
type: "MIDI_ITEM",
start_bar: 0.0, // Placed at Timeline start or active Playhead position
duration_bars: total_bars,
clip_start_offset_bars: 0.0,
source_data: {
total_buffer_bars: total_bars,
notes: aiTrack.notes.map((note, index) => ({
id: `note_ai_${Date.now()}_${index}`,
pitch: note.pitch,
start_beat: note.start_beat,
duration_beats: note.duration_beats,
velocity: note.velocity,
pan: 0.0
}))
}
};
// 4. Append Item to Track
targetTrack.items.push(newMidiItem);
});
// 5. Trigger Timeline & Piano Roll UI Re-render Event
window.dispatchEvent(new CustomEvent('DAW_STATE_UPDATED', { detail: sessionState }));
}
```
---
## 6. Prompt Preset Manager UI Layout
```text
+-------------------------------------------------------------------------------+
| AI PROMPT PRESET MANAGER [ + New ]|
+-------------------------------------------------------------------------------+
| SEARCH: [ epic orchestra ] FILTER: [ Orchestral v ]|
| |
| Preset Name Keywords matched Default Bars Actions |
| --------------------------------------------------------------------------- |
| [★] Epic Orchestra Intro epic, orchestra, tráng 8 Bars [Edit][Del] |
| [ ] Pop Piano Chords pop, piano, chord 4 Bars [Edit][Del] |
| [ ] Cyberpunk Synth Synth synth, synthwave, 80s 8 Bars [Edit][Del] |
| |
+-------------------------------------------------------------------------------+
| EDIT PRESET: Epic Orchestra Intro |
| |
| Keyword Triggers (comma-separated): |
| [ epic orchestra, hoành tráng, nhạc phim epic ] |
| |
| System Instruction / Music Rules: |
| +---------------------------------------------------------------------------+ |
| | Compose an 8-bar epic orchestral theme featuring Brass, staccato | |
| | Strings, Taiko percussion, and background Synth bass... | |
| +---------------------------------------------------------------------------+ |
| [ CANCEL ] [ SAVE ]
+-------------------------------------------------------------------------------+
```
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "sonicforge-studio", "name": "sonicforge-studio",
"private": true, "private": true,
"scripts": { "scripts": {
"build": "babel app/static/js/app.jsx --presets=@babel/preset-react -o app/static/js/app.precompiled.js" "build": "babel app/static/js/app.jsx --config-file ./babel.config.json -o app/static/js/app.precompiled.js"
}, },
"dependencies": { "dependencies": {
"@babel/cli": "^8.0.4", "@babel/cli": "^8.0.4",
+171
View File
@@ -0,0 +1,171 @@
# Kế hoạch cài đặt SoundFont / VSTi (32_SF_VSTi.md)
> Dựa trên hiện trạng codebase: `app/core/render_engine.py`, `app/core/vst_engine.py`, `tests/test_render_engine.py`, `app/api/v1/audio.py`, `requirements.txt` (có pedalboard, numpy, soundfile).
---
## 1. Server Backend (Python / FastAPI)
### 1.1 PluginManager (app/core/vst_engine.py — SỬA)
**Hiện trạng:** `vst_engine.py``render_midi_events_to_audio()` synth sine cơ bản. Cần class quản lý VST3.
**Công việc:**
- Thêm class `PluginManager`:
- `__init__(vst_dir="/opt/daw_engine/vst3", sf_dir="/opt/daw_engine/soundfonts")`
- `_scan_plugins()`: quét `.vst3`/`.so` trong `vst_dir`
- `_scan_soundfonts()`: quét `.sf2`/`.sf3` trong `sf_dir`
- `load_vst(name, preset_data=None)`: load VST3 bằng `pedalboard.VST3Plugin`
- `load_soundfont(path)`: load SF2 bằng `pyfluidsynth` (in-memory buffer)
- `list_available()`: trả về `{ vst_instruments: [...], soundfonts: [...] }`
- Import `pedalboard` + `pyfluidsynth` với `try/except`
- **requirements.txt:** thêm `mido>=1.3.0`, `pyfluidsynth>=1.3.0`
**File:** `app/core/vst_engine.py`
---
### 1.2 Render Engine (app/core/render_engine.py — SỬA)
**Hiện trạng:** `PythonRenderEngine` render audio clips + MIDI events. Cần VST3 + SoundFont path.
**Optimization 1 — In-memory SoundFont render (thay subprocess CLI):**
- Dùng `pyfluidsynth` C-bindings thay vì lệnh `fluidsynth` CLI qua subprocess
- Luồng render:
1. `fl = pyfluidsynth.FluidSynth(sample_rate=sr, gain=0.5)`
2. `fl.sfload(path_to_sf2)``fl.program_select(track_id, font_id, bank, preset)`
3. Render từng block MIDI event → numpy array bằng `fl.render_midi(midi_events, sr)`
4. Trả về in-memory buffer, không ghi file tạm
- Loại bỏ hoàn toàn subprocess CLI (tránh I/O disk, race condition, temp file collision)
**Optimization 2 — Sample Offset chính xác cho Pedalboard VST3:**
- Công thức quy đổi beat → sample offset:
```
SampleOffset = BeatPosition × (60 / BPM) × SampleRate
```
- Khi gọi `vst_plugin(array, sample_rate=sr, midi_messages=midi_list)`:
- Mỗi `pedalboard.MidiMessage` phải kèm `sample_offset` chính xác
- Chuyển từng MIDI event: `midi_events[].start_beat``sample_offset`
- Nếu không quy đổi, VSTi dồn toàn bộ nốt vào sample đầu ($0\text{ms}$) → sai nhịp
- Thêm hàm `_midi_events_to_messages(midi_events, bpm, sr) → list[MidiMessage]`
**Công việc:**
- Thêm `render_midi_track_with_vst(midi_events, vst_plugin, sr, bpm)`:
- Gọi `_midi_events_to_messages()` → quy đổi sang sample offset
- Khởi tạo VST3 từ `PluginManager.load_vst()`, process buffer
- Fallback về basic synth nếu VST không available
- Thêm `render_soundfont_track(midi_events, sf_path, sr, bpm)`:
- Dùng `pyfluidsynth.FluidSynth` in-memory
- `fl.render_midi()` → numpy array
- Sửa `render_session_container()` nhận `track["instrument"]` (VST/SF id) mỗi track
**File:** `app/core/render_engine.py`
---
### 1.3 API Endpoints (app/api/v1/plugins.py — MỚI + main.py — SỬA)
- **GET** `/api/v1/plugins/available``PluginManager().list_available()`
- **GET** `/api/v1/plugins/default-soundfonts` → scan `app/static/soundfonts/`
- **POST** `/api/v1/projects/render` → nhận ProjectSchema → render → return URL WAV
- Mount router trong `main.py`:
```python
from app.api.v1.plugins import router
app.include_router(router, prefix="/api/v1/plugins", tags=["plugins"])
```
**Optimization 3 — Validation Magic Bytes cho Upload SoundFont:**
- Endpoint upload `.sf2` không chỉ kiểm tra đuôi file
- Thêm FastAPI Dependency kiểm tra Header/Magic Bytes:
```
RIFF header (4 bytes: 0x52 0x49 0x46 0x46) + sfbk (4 bytes: 0x73 0x66 0x62 0x6B)
```
- Logic validate:
```python
def validate_sf2_header(data: bytes):
if len(data) < 12: return False
if data[0:4] != b'RIFF': return False
if data[8:12] != b'sfbk': return False
return True
```
- Từ chối upload nếu magic bytes không khớp → chặn file độc hại ngay từ API layer
**File mới:** `app/api/v1/plugins.py`
---
### 1.4 Dockerfile / Dependencies
- **requirements.txt:** thêm `mido>=1.3.0`, `pyfluidsynth>=1.3.0`
- **Dockerfile:** cài `fluidsynth` (shared lib cho pyfluidsynth), `libfluidsynth-dev`, `build-essential`
- Tạo `/opt/daw_engine/vst3/``/opt/daw_engine/soundfonts/`
- Tạo `app/static/soundfonts/` cho default SF
---
## 2. Client Frontend (JavaScript)
### 2.1 API Client (app/static/js/services/api.js — SỬA)
Thêm methods:
- `listPlugins()``GET /v1/plugins/available`
- `listDefaultSoundfonts()``GET /v1/plugins/default-soundfonts`
- `renderProject(data)``POST /v1/projects/render`
- `uploadSoundFont(file)``POST /v1/audio/upload-soundfont` (FormData)
---
### 2.2 SoundFont Player (app/static/js/services/soundfontPlayer.js — MỚI)
- `loadSoundFont(url)` → fetch `.sf2` → ArrayBuffer
- `initFluidSynth()` → khởi tạo Was m engine (fallback Web Audio API basic synth)
- `playNote(note, velocity, duration)` → play MIDI note
- Lưu user SF vào IndexedDB qua `window.SonicStorage`
- Global: `window.SonicSF`
---
### 2.3 Synth / Instrument Panel UI (index.html)
- **Instrument Selector:** dropdown chọn VST/SoundFont từ `listPlugins()`
- **SoundFont Upload:** drag & drop `.sf2` → IndexedDB
- **Preset Browser:** load `.vital`/`.fxp`/`.syx` cho VST đang chọn
- **Render to WAV button:** gọi `renderProject()` → progress → play
---
### 2.4 MIDI Track Support (index.html)
- Track model thêm: `type: 'audio' | 'midi'`, `midiEvents: []`, `instrumentId`
- WaveformLane vẽ MIDI notes (rects) thay waveform
- Double-click → piano roll editor (tham khảo `md/Piano_roll_UX.md`)
---
## 3. Tests
| Test | File | Mới/Sửa |
|---|---|---|
| PluginManager init + scan + load | `tests/test_vst_engine.py` | MỚI |
| Render MIDI track with VST (sample offset chính xác) | `tests/test_render_engine.py` | SỬA |
| Render MIDI track with SoundFont (pyfluidsynth in-memory) | `tests/test_render_engine.py` | SỬA |
| Render project mixed audio + MIDI | `tests/test_render_engine.py` | SỬA |
| API plugins/available/render | `tests/test_plugin_api.py` | MỚI |
| SF2 Magic Bytes validation (RIFF+sfbk) | `tests/test_plugin_api.py` | MỚI |
---
## 4. Thứ tự thực hiện
| Bước | Mô tả | Phụ thuộc |
|---|---|---|
| **B1** | Sửa `vst_engine.py`: PluginManager class | |
| **B2** | Thêm `mido`, `pyfluidsynth` vào `requirements.txt` | |
| **B3** | Tạo `app/api/v1/plugins.py`: 3 endpoints + SF2 magic bytes validate | B1 |
| **B4** | Mount router trong `main.py` | B3 |
| **B5** | Sửa `render_engine.py`: VST3 (sample offset) + SF (pyfluidsynth in-memory) | B1 |
| **B6** | Tạo `soundfontPlayer.js` | |
| **B7** | Sửa `api.js`: plugin/SF methods | |
| **B8** | UI Synth Panel + MIDI track (index.html) | B5-B7 |
| **B9** | Tests | B1-B5 |
| **B10** | Dockerfile: fluidsynth lib + VST dirs | |
+2
View File
@@ -12,3 +12,5 @@ jinja2>=3.1.2
httpx>=0.24.0 httpx>=0.24.0
jsonschema>=4.18.0 jsonschema>=4.18.0
pedalboard>=0.8.0 pedalboard>=0.8.0
mido>=1.3.0
pyfluidsynth>=1.3.0
+85
View File
@@ -0,0 +1,85 @@
import os
import json
import pytest
from fastapi.testclient import TestClient
from unittest.mock import patch, MagicMock
from app.main import app
from app.core.vst_engine import HAS_PEDALBOARD, HAS_PYFLUIDSYNTH
client = TestClient(app)
def get_admin_token():
resp = client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin123"})
if resp.status_code == 200:
return resp.json()["access_token"]
return None
class TestPluginAPI:
def test_list_plugins_requires_auth(self):
resp = client.get("/api/v1/plugins/available")
assert resp.status_code in (401, 403)
def test_list_plugins_authenticated(self):
token = get_admin_token()
if not token:
pytest.skip("Cannot get admin token")
resp = client.get("/api/v1/plugins/available", headers={"Authorization": f"Bearer {token}"})
assert resp.status_code == 200
data = resp.json()
assert "vst_instruments" in data
assert "soundfonts" in data
def test_list_default_soundfonts(self):
resp = client.get("/api/v1/plugins/default-soundfonts")
assert resp.status_code == 200
data = resp.json()
assert isinstance(data, list)
def test_render_project_invalid_json(self):
token = get_admin_token()
if not token:
pytest.skip("Cannot get admin token")
resp = client.post("/api/v1/plugins/render", headers={"Authorization": f"Bearer {token}"}, json={"project_json": {}})
# Should fail because project is empty, but API should return 500 or error
assert resp.status_code in (400, 422, 500)
def test_upload_soundfont_requires_auth(self):
resp = client.post("/api/v1/plugins/upload-soundfont")
assert resp.status_code in (401, 403, 422)
def test_upload_soundfont_invalid_magic(self):
token = get_admin_token()
if not token:
pytest.skip("Cannot get admin token")
# Upload a file with invalid magic bytes
fake_content = b'XXXX\x00\x00\x00\x00YYYY' * 100
resp = client.post(
"/api/v1/plugins/upload-soundfont",
headers={"Authorization": f"Bearer {token}"},
files={"file": ("fake.sf2", fake_content, "application/octet-stream")}
)
assert resp.status_code == 400
assert "Invalid SoundFont" in resp.json().get("detail", "")
def test_upload_soundfont_valid_magic(self):
token = get_admin_token()
if not token:
pytest.skip("Cannot get admin token")
# Upload a file with valid RIFF + sfbk magic
valid_content = b'RIFF\x00\x00\x00\x00sfbk' + b'\x00' * 200
resp = client.post(
"/api/v1/plugins/upload-soundfont",
headers={"Authorization": f"Bearer {token}"},
files={"file": ("test.sf2", valid_content, "application/octet-stream")}
)
# Should succeed (200) unless auth/permission issues
if resp.status_code == 200:
data = resp.json()
assert "id" in data
assert "size_bytes" in data
assert data["size_bytes"] == len(valid_content)
elif resp.status_code == 403:
pytest.skip("Permission denied for admin user")
+95
View File
@@ -0,0 +1,95 @@
import os
import json
import pytest
import numpy as np
from unittest.mock import patch, MagicMock
from app.core.vst_engine import PluginManager, midi_note_to_freq, render_midi_events_to_audio, HAS_PEDALBOARD
class TestPluginManager:
def test_init(self):
pm = PluginManager()
assert pm.vst_dir == "/opt/daw_engine/vst3"
assert pm.sf_dir == "/opt/daw_engine/soundfonts"
def test_midi_events_to_messages(self):
if not HAS_PEDALBOARD:
pytest.skip("pedalboard not available")
events = [{"note": 60, "start_beat": 0, "duration_beats": 1, "velocity": 100}]
bpm = 120
sr = 44100
msgs = PluginManager.midi_events_to_messages(events, bpm, sr)
beat_sec = 60.0 / 120
expected_note_on_offset = 0
expected_note_off_offset = int(beat_sec * sr)
# Check note_on message
assert msgs[0].sample_offset == expected_note_on_offset
assert msgs[0].note == 60
# Check note_off message
assert msgs[1].sample_offset == expected_note_off_offset
assert msgs[1].note == 60
def test_list_available_empty(self):
pm = PluginManager(vst_dir="/tmp/nonexistent_vst_dir_xyz", sf_dir="/tmp/nonexistent_sf_dir_xyz")
available = pm.list_available()
assert "vst_instruments" in available
assert "soundfonts" in available
assert available["vst_instruments"] == []
assert available["soundfonts"] == []
def test_validate_sf2_header_valid(self):
# Valid RIFF + sfbk header
valid = b'RIFF' + b'\x00' * 4 + b'sfbk' + b'\x00' * 100
assert PluginManager.validate_sf2_header(valid) is True
def test_validate_sf2_header_invalid_no_riff(self):
invalid = b'XXXX' + b'\x00' * 4 + b'sfbk' + b'\x00' * 100
assert PluginManager.validate_sf2_header(invalid) is False
def test_validate_sf2_header_invalid_no_sfbk(self):
invalid = b'RIFF' + b'\x00' * 4 + b'XXXX' + b'\x00' * 100
assert PluginManager.validate_sf2_header(invalid) is False
def test_validate_sf2_header_too_short(self):
assert PluginManager.validate_sf2_header(b'RIFF') is False
class TestMidiNoteToFreq:
def test_a4_440(self):
assert midi_note_to_freq(69) == 440.0
def test_c4(self):
# C4 = MIDI 60 = 261.63
freq = midi_note_to_freq(60)
assert abs(freq - 261.63) < 0.5
def test_note_zero_to_freq(self):
freq = midi_note_to_freq(0)
assert freq > 0 and freq < 10
class TestRenderMidiToAudio:
def test_render_single_note(self):
events = [{"note": 60, "start_beat": 0, "duration_beats": 1, "velocity": 100}]
audio = render_midi_events_to_audio(events, sr=44100, bpm=120)
assert audio.shape[0] == 2 # Stereo
assert audio.shape[1] > 0
# Should have non-zero samples
assert np.max(np.abs(audio)) > 0
def test_render_empty_events(self):
audio = render_midi_events_to_audio([], sr=44100, bpm=120)
assert audio.shape[0] == 2 # Stereo
# Empty events defaults to 2 seconds of silence (minimum length)
assert audio.shape[1] >= 0
def test_render_multiple_notes(self):
events = [
{"note": 60, "start_beat": 0, "duration_beats": 1, "velocity": 100},
{"note": 64, "start_beat": 1, "duration_beats": 1, "velocity": 80},
{"note": 67, "start_beat": 2, "duration_beats": 1, "velocity": 90},
]
audio = render_midi_events_to_audio(events, sr=44100, bpm=120)
assert audio.shape[0] == 2
assert audio.shape[1] > 44100 * 1
+135
View File
@@ -0,0 +1,135 @@
### [2026-07-25 06:57] Task: Fix Piano Roll runtime errors (rollBeats TDZ + handleCCMouseMove)
- **Tóm tắt thay đổi:** Sửa lỗi `Cannot access 'rollBeats' before initialization` bằng cách di chuyển khai báo `rollBeats`/`rollBeatsRef` lên trước `totalBeats`. Thêm hàm `handleCCMouseMove` bị thiếu và cập nhật `handleCCMouseDown` hỗ trợ Ctrl+Click paint velocity.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `cd /home/locpham/SonicForgeStudio && npm run build` — build passes.
---
### [2026-07-25 07:25] Task: 4 tính năng mới (Piano Roll zoom fill + Shift+scroll playhead + auto-scroll drag + SoundFont cache)
- **Tóm tắt thay đổi:** (1) Piano Roll zoom-out không còn màn hình đen — bars fill toàn bộ viewport. (2) Shift+scroll trong Piano Roll di chuyển playhead và play notes MIDI như fast-forward. (3) Khi drag section/MIDI/clip đến cạnh phải timeline, auto-scroll container. (4) Server-side cache FluidSynth instances + PluginManager singleton + list_soundfont_instruments cache.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/core/vst_engine.py`
- **Ghi chú/Test (nếu có):** `npm run build` pass. `vst_engine.py` thêm `load_soundfont_cached`, `release_soundfont`, `get_plugin_manager`, `_SF_INSTRUMENTS_CACHE` — refcount-based cache.
### [2026-07-25 07:25] Task: Fix auto-scroll + maxDuration tab isolation + 1-bar margin
- **Tóm tắt thay đổi:** (1) `maxDuration` dùng `activeTracks` + 4-bar buffer. (2) Cách ly MAIN vs SECTION-TAB. (3) Clip/section/MIDI drag/stretch/resize clamp 1-bar from right. (4) Clip drag + stretched clip dùng `updateActiveTracks`. (5) Stretched clip handler thêm auto-scroll.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 07:31] Task: Fix zoom cursor + right edge black + AI Copilot for Piano Roll
- **Tóm tắt thay đổi:** (1) Tách `drawWidth` (content) và `cssWidth` (CSS) — canvas coordinate system không còn bị sai khi zoom out. (2) Thêm `min-width` fill viewport, không còn mảng đen phải. (3) Thêm AI Copilot input + handler trong Piano Roll — gõ prompt tạo MIDI notes trực tiếp.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npm run build` pass.
### [2026-07-25 08:08] Task: Fix velocity draw position, shift+scroll play, canvas redraw deps
- **Tóm tắt thay đổi:** (1) CC lane velocity stem vẽ tại start note thay vì center. (2) Shift+scroll chỉ play note khi playhead vượt qua start_beat. (3) Thêm `viewWidth`/`viewBeats` vào dependency array grid canvas + CC canvas để auto-redraw khi resize/zoom.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` pass.
### [2026-07-25 08:31] Task: Fix AI Copilot tab scope + hide bottom panels + context menu Insert + autoscroll
- **Tóm tắt thay đổi:** (1) AI Copilot context dùng `activeTracks` thay `tracks` — chỉ tác động tab đang active. (2) Bottom panels mặc định ẩn. (3) Right-click track timeline có 4 mục Insert. (4) Thêm auto-scroll vào stretched clip resize. Sửa `insertSectionAtPlayhead` dùng `updateActiveTracks`.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 08:36] Task: Fix auto-scroll + maxDuration tab isolation + 1-bar margin
- **Tóm tắt thay đổi:** (1) `maxDuration` dùng `activeTracks` + 4-bar buffer. (2) Cách ly MAIN vs SECTION-TAB. (3) Clip/section/MIDI drag/stretch/resize clamp 1-bar from right. (4) Clip drag + stretched clip dùng `updateActiveTracks`. (5) Stretched clip handler thêm auto-scroll.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 08:53] Task: Add timeline left/right margin + dynamic buffer expansion
- **Tóm tắt thay đổi:** (1) PADDING_LEFT thay đổi từ 2s thành 2-bar margin ở cả WaveformLane và TempoTrackLane. (2) Thêm state `scrollBufferExtra` + auto-expand 4 bars khi auto-scroll gần đến rìa timeline. (3) `maxDuration` thêm `scrollBufferExtra` + 6 bars static buffer. (4) Clip drag/stretch dùng `activeTracks` + `updateActiveTracks` thay `tracks`/`setTracks`.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:04] Task: Fix left margin 1-beat, items always visible, bar/timeline sync
- **Tóm tắt thay đổi:** (1) PADDING_LEFT từ 2-bar xuống 1-beat ở cả WaveformLane và TempoTrackLane. (2) Drag clamp left ≥ 1 beat, right ≤ maxDuration - 1 bar. (3) Bar numbers + grid vẽ lại đồng bộ với scroll+zoom.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:08] Task: Fix Piano Roll CC lane always visible + resizable height
- **Tóm tắt thay đổi:** (1) `showCC` default `true` — CC lane luôn hiển thị. (2) Thêm `ccHeight` state + resize handle (drag top edge) — cho phép thay đổi độ cao CC lane. (3) Canvas dùng `ccHeight` thay hardcoded 80px.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:18] Task: Fix brush velocity isolation from CC lane
- **Tóm tắt thay đổi:** Thêm `brushVelocityRef` riêng biệt (0.8) — brush không còn bị ảnh hưởng bởi velocity paint từ CC lane.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:20] Task: Auto-follow dragged item in viewport
- **Tóm tắt thay đổi:** (1) Clip drag: sau mỗi update, nếu item ra khỏi vùng 120px viewport edge, tự scroll để item luôn thấy. (2) Section/MIDI drag: tương tự. (3) Section/MIDI resize: theo dõi edge đang kéo.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:23] Task: Sync bar ruler redraw during drag
- **Tóm tắt thay đổi:** (1) TempoTrackLane nhận `canvasRedrawCount` prop + deps — force redraw bar numbers khi drag. (2) Cả 3 drag handler increment `setCanvasRedrawCount` sau auto-follow scroll. (3) Sub-tab TempoTrackLane thêm `viewportWidth` + `scrollLeft` + `canvasRedrawCount` — đồng bộ ruler với timeline.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 09:30] Task: Fix bar numbers disappear past bar 12 during drag
- **Tóm tắt thay đổi:** TempoTrackLane đọc `scrollLeft` từ DOM (walk parent chain tìm scroll container) thay vì React prop — canvas luôn dùng scroll position thực tế, không lag do async state.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):**
### [2026-07-25 09:58] Task: Fix drag speed + bar numbers visibility + snap section items
- **Tóm tắt thay đổi:** (1) Xoá auto-follow phải (gây jump), chỉ giữ auto-follow trái 60px — scroll phải nhờ `autoScrollTimeline` mượt. (2) Thêm `PADDING_RIGHT` vào `tEnd` ở TempoTrackLane và WaveformLane — bar numbers + grid hiển thị đến sát rìa phải. (3) `if (localX < 0 || localX > drawWidth) continue;` — skip phần tử ngoài canvas. (4) Buffer 12 bars cho `maxDuration`. (5) Thêm `snapTime` vào section/MIDI item drag. (6) Thêm snap option `'4'` (1 bar). (7) Đồng bộ `PADDING_LEFT = barDuration` ở cả 2 lane.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npx babel app/static/js/app.jsx` — compile pass.
### [2026-07-25 10:08] Task: Merge tempo bar + timebar, beat-based ruler, snap grid
- **Tóm tắt thay đổi:** Thay thế seconds-based ruler (0.00s, 1.00s...) + TempoTrackLane bằng TimelineRuler (48px canvas) hiển thị bar:beat grid lines + snap sub-ticks + BPM label. Grid WaveformLane khớp ruler, snapTime() đã dùng grid spacing beat — snap items đúng vị trí ruler.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:35] Task: Restore time duration labels on TimelineRuler + TempoTrackLane
- **Tóm tắt thay đổi:** Thêm time duration labels (formatTimeSimple: "0.00s", "1.00s",...) vào TimelineRuler và TempoTrackLane. Hiển thị dạng màu cam (#ffb464) ở chân ruler, động điều chỉnh interval (1s/2s/5s/10s...) theo zoom level. Time labels đã bị mất từ commit ff44fd4 khi merge time bar + tempo bar thành TimelineRuler.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:39] Task: Sync TempoTrackLane leadIn + time labels on ruler top
- **Tóm tắt thay đổi:** Thêm `leadInMargin` prop vào TempoTrackLane để đồng bộ column-0 (time=0) với vị trí leadIn của TimelineRuler. Thêm CLIP_BUFFER vào TempoTrackLane. Di chuyển time duration labels trong TimelineRuler từ dưới đáy (y=40) lên phía trên (y=11).
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 11:09] Task: Fix TempoTrackLane scroll sync during drag auto-scroll
- **Tóm tắt thay đổi:** TempoTrackLane/WaveformLane/TimelineRuler hiện đọc scrollLeft trực tiếp từ DOM wrapper (canvases tìm parent scrollable) thay vì React prop. Tránh lệch vị trí khi auto-scroll nhanh trong clip drag do React state batching.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 11:04] Task: Fix PADDING_LEFT alignment + right-side bar rendering
- **Tóm tắt thay đổi:** Set PADDING_LEFT=0 trong TimelineRuler/WaveformLane/TempoTrackLane để grid column 0 khớp với clip position 0 (trước đây lệch 8px). TempoTrackLane dùng CLIP_BUFFER/zoom cho tEnd thay vì PADDING_LEFT để bar/beat vẽ đủ xa về phía phải khi scroll.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:58] Task: Fix clip/section snap to column 0
- **Tóm tắt thay đổi:** Xóa `Math.max(beatSec, ...)` trong clip drag (10325), section drag (10495), section resize (10576) — thay bằng `Math.max(0, ...)` để item có thể snap vào column 0.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:54] Task: Fix TempoTrackLane height + clamp selection at column 0
- **Tóm tắt thay đổi:** Fix TempoTrackLane canvas height = 40px cố định (thay vì parentElement.clientHeight gây tràn). Wrapper `h-10 shrink-0 overflow-hidden` để không đẩy track lanes lệch TCP. Clamp selLeft/selRight/ruler handlers >= 0 để selection không vào vùng âm.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:44] Task: Zero lead-in, time ruler + tempo track split layout
- **Tóm tắt thay đổi:** Xóa leadInMargin (về 0) — time 0 tại mép trái, chỉ còn 8px PADDING_LEFT. TimelineRuler đơn giản thành time-only ruler (40px, chỉ hiển thị time labels). Thêm TempoTrackLane ngay dưới (40px, bars/beats/BPM). Selection overlay top từ 48→80px. Đồng bộ WaveformLane PADDING_LEFT=8/zoom.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 10:18] Task: Lead-in margin, bar 0-indexed, grid clip fix
- **Tóm tắt thay đổi:** Thêm leadInMargin (1 bar) đẩy timeline content sang phải — bar 0, time 0s tại mép margin. Playhead bắt đầu từ đó. Bar numbering 0-indexed (bar 0 thay vì 1). Sửa CLIP_BUFFER từ 200px → max(400, barWidth+200) để bar number không bị che. Đồng bộ leadInMargin vào tStart/tEnd canvas + mouse handlers (ruler, clip drag, section resize).
- **Các file ảnh hưởng:** `app/static/js/app.jsx`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---
### [2026-07-25 11:32] Task: Redraw canvas on section item duration resize
- **Tóm tắt thay đổi:** Thêm `setCanvasRedrawCount(n => n + 1)` trong resize mousemove handler để WaveformLane redraw khi section/MIDI item duration thay đổi.
- **Các file ảnh hưởng:** `app/static/js/app.jsx`, `app/static/js/app.precompiled.js`
- **Ghi chú/Test (nếu có):** `npm run build` — build passes.
---