3dtours/SonicForgeStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: hiển thị ghost note trong piano roll tab

Browse Source
This commit is contained in:
3dtours
2026-07-27 17:26:13 +07:00
parent 53a27b5af4
commit d21091b69b
3 changed files with 546 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kilo/plans/1785145662134-midi-ghost-note-plan.md
+287
View File
@@ -0,0 +1,287 @@
# Plan: MIDI Ghost Notes + Dropdown Item Switcher + Session Sync Mode
## Overview
Three features built on each other:
1. **Dropdown** at tab title position listing ALL MIDI items across all tracks
2. **Item switching** — selected item becomes editable, all others become ghost notes
3. **Session sync mode toggle** — viewport aligns with session bars (ghost visible) or resets to bar 0 (isolated, no ghost)
---
## Key Design Decisions
### A. Ghost note scope = all items except the selected one
Not just "other tracks" — ALL MIDI items in `activeTracks` except the one matching `targetItemId` contribute ghost notes if overlapping.
### B. Two viewport modes, togglable
| Mode | Viewport origin | Ghost notes | Bar labels |
|------|----------------|-------------|------------|
| **Session Sync** (default) | `item.startTime / secondsPerBar` | Visible | `Bar N` (session-absolute) |
| **Isolated** | bar 0 | Hidden | `Bar N` (0-based) |
### C. Ghost notes computed (not persisted)
No schema changes. Extraction runs in `useMemo` inside `PianoRollTabEditor`.
### D. Hit-testing exclusion is automatic
Ghost notes are in separate `ghostLayers` state; mouse handlers only iterate `notes`.
---
## Files to Modify
| File | Change |
|------|--------|
| **NEW** `app/static/js/services/ghostNoteExtractor.js` | Extraction logic |
| `app/static/js/app.jsx` (~16113) | Pass `activeTracks` prop to `PianoRollTabEditor` |
| `app/static/js/app.jsx` (~4563-6070) | All PianoRollTabEditor changes below |
---
## Step-by-Step Implementation
### Step 1: Create `ghostNoteExtractor.js`
`app/static/js/services/ghostNoteExtractor.js`
```js
export function extractGhostLayers(activeTracks, targetTrackId, targetItemId, bpm)
```
Algorithm:
1. `secondsPerBeat = 60 / bpm`
2. Find `targetItem` across all tracks → `windowStartBeat = targetItem.startTime / secondsPerBeat`, `windowEndBeat = (targetItem.startTime + targetItem.duration) / secondsPerBeat`
3. Iterate ALL tracks, ALL MIDI items:
- Skip non-MIDI tracks (`!t.midiItems || !t.midiItems.length`)
- Skip muted tracks (`t.muted`)
- Skip item matching `targetItemId` (the active item)
4. For each candidate item:
- `itemStartBeat = item.startTime / secondsPerBeat`
- `itemEndBeat = (item.startTime + item.duration) / secondsPerBeat`
- Overlap test: `itemStartBeat < windowEndBeat && itemEndBeat > windowStartBeat`
- For each overlapping note:
- `noteAbsStart = itemStartBeat + note.start_beat`
- `noteAbsEnd = noteAbsStart + note.duration_beats`
- Clip: keep if `noteAbsStart < windowEndBeat && noteAbsEnd > windowStartBeat`
- `clampedDur = Math.min(noteAbsEnd, windowEndBeat) - Math.max(noteAbsStart, windowStartBeat)`
- Push: `{ id: ghost_${note.id}, pitch, relative_start_beat: noteAbsStart - windowStartBeat, duration_beats: clampedDur, velocity, original_track_name: t.name, original_track_color: t.color || '#888' }`
5. Group by track → `ghostLayers: [{ track_id, track_name, track_color, notes }]`
6. Return `ghostLayers`
### Step 2: Pass `activeTracks` to PianoRollTabEditor
At render site (~line 16113), add:
```js
activeTracks: activeTracks,
```
Add `activeTracks` to destructured props in `PianoRollTabEditor` function signature (~line 4563).
### Step 3: New state & derived data
Inside `PianoRollTabEditor` (~line 4570), after existing `React.useState` declarations:
```js
const [showGhostNotes, setShowGhostNotes] = React.useState(true);
const [sessionSyncMode, setSessionSyncMode] = React.useState(true);
// Compute all MIDI items for dropdown
const allMidiItems = React.useMemo(() => {
const result = [];
(activeTracks || []).forEach(t => {
if (!t.midiItems || !t.midiItems.length) return;
t.midiItems.forEach(m => {
result.push({ ...m, _trackId: t.id, _trackName: t.name });
});
});
return result;
}, [activeTracks]);
// Compute ghost layers
const ghostLayers = React.useMemo(() => {
if (!activeTracks || !st || !st.target_id) return [];
return extractGhostLayers(activeTracks, st.trackId, st.target_id, parseInt(bpm) || 120);
}, [activeTracks, st.trackId, st.target_id, bpm]);
// Compute session offset for bar labels
const secondsPerBar = (60.0 / (parseInt(bpm) || 120)) * 4;
const targetTrack = React.useMemo(
() => (activeTracks || []).find(t => t.id === st.trackId),
[activeTracks, st.trackId]
);
const activeTargetItem = React.useMemo(
() => targetTrack ? (targetTrack.midiItems || []).find(m => m.id === st.target_id) : null,
[targetTrack, st.target_id]
);
const sessionStartBar = sessionSyncMode && activeTargetItem
? (activeTargetItem.startTime / secondsPerBar) : 0;
```
### Step 4: Dropdown at tab title position
Replace the static title (line 5748-5752) with a dropdown:
```js
/* 1a. TAB TITLE DROPDOWN */
React.createElement("div", { className: "relative inline-block text-xs" },
React.createElement("select", {
value: st.target_id,
onChange: e => handleSwitchMidiItem(e.target.value),
className: "bg-zinc-800 border border-zinc-700 text-yellow-500 font-bold rounded px-1.5 py-0.5 outline-none focus:border-yellow-500 max-w-[200px]"
}, allMidiItems.map(m =>
React.createElement("option", {
key: m.id,
value: m.id
}, `${m._trackName} - ${m.name || 'MIDI'}`)
))
)
```
### Step 5: Switch handler function
Add before the return statement:
```js
const handleSwitchMidiItem = (itemId) => {
if (itemId === st.target_id) return;
// Save current notes first
onSaveNotes(st.id, st.trackId, st.target_id, notes);
// Find selected item
const match = allMidiItems.find(m => m.id === itemId);
if (!match) return;
// Update subTab state (triggers ghost re-compute via useMemo)
setSubTabs(prev => prev.map(s => s.id === st.id ? {
...s,
trackId: match._trackId,
target_id: match.id,
label: `Piano Roll: ${match.name || 'MIDI'}`,
notes: match.notes || [],
duration: match.duration || 4,
instrumentProgram: activeTracks.find(t => t.id === match._trackId)?.instrumentProgram,
instrumentName: activeTracks.find(t => t.id === match._trackId)?.instrumentName,
note_selection: [],
currentTime: 0,
} : s));
// Reset local state
setSelectedNoteIds([]);
setLoopStartBeat(null);
setLoopEndBeat(null);
};
```
### Step 6: Session sync toggle button
In toolbar (~line 5798, near CC toggle), add:
```js
/* Session sync mode toggle */
React.createElement("button", {
onClick: () => setSessionSyncMode(!sessionSyncMode),
className: `px-2 py-1 rounded text-xs ${sessionSyncMode ? 'bg-cyan-900/60 text-cyan-300 border border-cyan-700' : 'text-zinc-500 hover:text-zinc-300'}`,
title: sessionSyncMode ? "Session-synced mode (ghost visible)" : "Isolated mode (bar 0, no ghost)"
}, sessionSyncMode ? "🌐 Session" : "📋 Isolated")
```
And the Ghost toggle:
```js
React.createElement("button", {
onClick: () => setShowGhostNotes(!showGhostNotes),
disabled: !sessionSyncMode,
className: `px-2 py-1 rounded text-xs ${!sessionSyncMode ? 'opacity-30 cursor-not-allowed' : showGhostNotes ? 'bg-purple-900/60 text-purple-300 border border-purple-700' : 'text-zinc-500 hover:text-zinc-300'}`,
title: "Toggle ghost notes visibility"
}, "👻 Ghost")
```
Ghost toggle disabled in isolated mode (no ghost notes to show).
### Step 7: Bar labels with session offset
Modify `renderBarLabels()` (~line 5711-5734):
Replace `Bar ${bar}` with:
```js
const displayBar = bar + Math.floor(sessionStartBar);
`Bar ${displayBar}`
```
And the seek click handler:
```js
const barTime = (bar + Math.floor(sessionStartBar)) * 4 * beatSec;
```
### Step 8: Auto-scroll to session position
Add `useEffect`:
```js
React.useEffect(() => {
if (sessionSyncMode && gridScrollRef.current && activeTargetItem) {
const scrollTargetBeats = sessionStartBar * 4;
gridScrollRef.current.scrollLeft = scrollTargetBeats * pixelsPerBeat;
}
}, [sessionSyncMode, sessionStartBar, st.target_id, pixelsPerBeat]);
```
### Step 9: Ghost note canvas layer
In the note-drawing `useLayoutEffect` (~line 4834), insert **before** active note rendering:
```js
/* Layer 2: Ghost Notes */
if (showGhostNotes && sessionSyncMode && ghostLayers.length > 0) {
ghostLayers.forEach(layer => {
ctx.save();
ctx.globalAlpha = 0.25;
ctx.fillStyle = layer.track_color || '#888';
ctx.strokeStyle = layer.track_color || '#888';
layer.notes.forEach(note => {
const x = note.relative_start_beat * pixelsPerBeat;
const y = (127 - note.pitch) * NoteHeight;
const w = note.duration_beats * pixelsPerBeat;
const h = NoteHeight - 1;
ctx.fillRect(x, y, w, h);
ctx.setLineDash([2, 2]);
ctx.lineWidth = 1;
ctx.strokeRect(x, y, w, h);
ctx.setLineDash([]);
});
ctx.restore();
});
}
```
Add `showGhostNotes`, `sessionSyncMode`, `ghostLayers` to dependency array.
### Step 10: Ghost notes in CC Lane
Skip ghost notes in CC lane (only active notes). CC lane already only iterates `notes`, not `ghostLayers`. No changes needed.
---
## Edge Cases
- **Dropdown with single MIDI item**: Only one option, no ghost notes (nothing to ghost).
- **Item deleted while piano roll is open**: `handleSwitchMidiItem` fails gracefully (item not found → no-op). `ghostLayers` `useMemo` returns `[]`.
- **BPM change mid-edit**: All beat computations update via `useMemo`/React reactivity.
- **Session sync → Isolated switch**: Scroll resets to 0, bar labels change to 0-based, ghost notes disappear.
- **Isolated → Session sync switch**: Scroll jumps to session position, ghost notes reappear.
- **Color fallback**: Use track's `color` prop; if `#888` as default.
---
## Validation
1. Open MIDI item → dropdown shows all MIDI items across all tracks
2. Select different item from dropdown → active notes switch, ghost notes re-compute
3. Ghost notes from ALL non-selected items appear (same track + other tracks)
4. Ghost toggle hides/shows ghost notes (disabled in isolated mode)
5. Session sync mode shows correct bar labels (e.g. `Bar 2` if item starts at bar 2)
6. Isolated mode shows `Bar 0, 1, 2...` regardless of item's session position
7. Switching items in isolated mode: notes change, viewport stays at bar 0
8. Ghost notes cannot be clicked/dragged (excluded from hit-testing)
9. Muted tracks' MIDI items are excluded from ghost notes
10. Items outside target window are excluded from ghost notes
app/storage/sonicforge.db
BIN
View File
Binary file not shown.
md/40_MIDI_GHOSTNOTE.md
+259
View File
@@ -0,0 +1,259 @@
# TECHNICAL SPECIFICATION: GHOST NOTES FEATURE IN PIANO ROLL TAB
This document details the workflow and processing algorithms for opening any `MIDIItem` (for example, MIDI Item 1 spanning Bar 0 to Bar 4) on the main Timeline in a Piano Roll Tab, while extracting and displaying all notes from other tracks occupying the same time interval (Bar 0 - Bar 4) as Ghost Notes (faded reference notes that cannot be interactively edited).
---
## 1. DATA FLOW & MODULE ARCHITECTURE DIAGRAM
```text
[ USER DBL-CLICK ITEM 1 ]
(Track A, Bar 0 - 4)
|
v
[ 1. Context Extractor Engine ]
├── Target Item: Active Editing Item
├── Window Bounds: [Bar 0.0 -> Bar 4.0]
└── Scan All Other Tracks (B, C, D...)
|
v
[ 2. Overlap Filtering Algorithm ]
├── Track B (Bass): Item B1 (Bar 0 - 8) -> Overlap! Slice [Bar 0 -> 4]
├── Track C (Pads): Item C1 (Bar 2 - 6) -> Overlap! Slice [Bar 2 -> 4]
└── Track D (Lead): Item D1 (Bar 5 - 8) -> Out of bounds! Ignore
|
v
[ 3. Piano Roll State Store ]
├── activeItem: MIDI Item 1 (Full Edit Access)
└── ghostLayers: [ Track B Notes, Track C Notes ] (Read-Only)
|
v
[ 4. Multi-Layer Canvas Renderer ]
├── Layer 1: Background Grid & Pitch Keys
├── Layer 2: Ghost Notes (Opacity 25%, Muted Color, Pointer Events OFF)
└── Layer 3: Active Notes (Full Opacity, Drag/Drop/Resize Allowed)
```
---
## 2. TIME-WINDOW OVERLAP FILTERING ALGORITHM
To determine whether a `MIDIItem` on another track overlaps with the interval $[start\_bar, end\_bar]$ of MIDI Item 1, a 1D geometric condition is applied:
### Overlap Condition
Two time intervals $[A_{start}, A_{end}]$ and $[B_{start}, B_{end}]$ intersect if and only if:
$$B_{start} < A_{end} \quad \text{AND} \quad B_{end} > A_{start}$$
Where:
* $A_{start} = \text{Item1.start\_bar} = 0.0$
* $A_{end} = \text{Item1.start\_bar} + \text{Item1.duration\_bars} = 4.0$
```javascript
// app/static/js/services/ghostNoteExtractor.js
/**
* Extracts a list of Ghost Notes from other tracks within the specified Bar window
* @param {Object} sessionState - Full Main Session state object
* @param {string} targetTrackId - ID of the currently active editing Track (Track A)
* @param {number} windowStartBar - Start bar position of the active Item (e.g., 0.0)
* @param {number} windowDurationBars - Bar duration of the active Item (e.g., 4.0)
* @returns {Array} List of normalized Ghost Layers
*/
export function extractGhostLayers(sessionState, targetTrackId, windowStartBar, windowDurationBars) {
const windowEndBar = windowStartBar + windowDurationBars;
const timeSigNumerator = sessionState.metadata.time_signature_numerator || 4;
const ghostLayers = [];
// Iterate over all tracks in Main Session
sessionState.main_session.tracks.forEach((track) => {
// Ignore active editing track and non-MIDI or muted tracks
if (track.id === targetTrackId || track.type !== "MIDI" || track.mute) {
return;
}
const trackGhostNotes = [];
// Iterate over all items in target candidate tracks
track.items.forEach((item) => {
if (item.type !== "MIDI_ITEM") return;
const itemStartBar = item.start_bar;
const itemEndBar = item.start_bar + item.duration_bars;
// Evaluate Overlap Condition
if (itemStartBar < windowEndBar && itemEndBar > windowStartBar) {
const itemOffsetBar = item.clip_start_offset_bars || 0.0;
item.source_data.notes.forEach((note) => {
// Convert internal note start_beat to absolute timeline beats
const noteAbsoluteBeat = (itemStartBar * timeSigNumerator) + note.start_beat - (itemOffsetBar * timeSigNumerator);
const noteEndAbsoluteBeat = noteAbsoluteBeat + note.duration_beats;
const windowStartBeat = windowStartBar * timeSigNumerator;
const windowEndBeat = windowEndBar * timeSigNumerator;
// Retain notes truly within the visible bounds [windowStartBeat -> windowEndBeat]
if (noteAbsoluteBeat < windowEndBeat && noteEndAbsoluteBeat > windowStartBeat) {
trackGhostNotes.push({
id: `ghost_${note.id}`,
pitch: note.pitch,
// Convert beat position to relative coordinates of the Piano Roll Window (0 -> duration_beats)
relative_start_beat: noteAbsoluteBeat - windowStartBeat,
duration_beats: note.duration_beats,
velocity: note.velocity,
original_track_name: track.name,
original_track_color: track.color || "#888888"
});
}
});
}
});
if (trackGhostNotes.length > 0) {
ghostLayers.push({
track_id: track.id,
track_name: track.name,
track_color: track.color || "#6b7280",
notes: trackGhostNotes
});
}
});
return ghostLayers;
}
```
---
## 3. STATE STRUCTURE FOR PIANO ROLL TAB
When the user opens MIDI Item 1, the Tab Controller generates an isolated context payload for the Piano Roll:
```javascript
// Data Payload passed into the PianoRollTab Component
const pianoRollTabContext = {
tab_id: "tab_pianoroll_item_1",
title: "Piano Roll - MIDI Item 1",
type: "PIANO_ROLL_TAB",
parent_tab_id: "tab_main_session",
// 1. Target Item open for direct interactive editing
active_context: {
track_id: "track_A",
item_id: "item_1",
item_name: "MIDI Item 1",
start_bar: 0.0,
duration_bars: 4.0,
notes: [/* Original source MIDI notes of Item 1 */]
},
// 2. Read-only Ghost Layers displayed as background reference indicators
ghost_layers: [
{
track_id: "track_B_bass",
track_name: "Track B (Bass)",
track_color: "#3b82f6", // Green / Blue
notes: [
{ pitch: 36, relative_start_beat: 0.0, duration_beats: 4.0, velocity: 0.9 },
{ relative_start_beat: 4.0, pitch: 38, duration_beats: 4.0, velocity: 0.8 }
]
},
{
track_id: "track_C_pads",
track_name: "Track C (Pads)",
track_color: "#ec4899", // Pink
notes: [
{ pitch: 60, relative_start_beat: 8.0, duration_beats: 8.0, velocity: 0.6 }
]
}
]
};
```
---
## 4. MULTI-LAYER CANVAS RENDERER WORKFLOW
Inside the Piano Roll canvas rendering module, elements are drawn sequentially by z-index order to position Ghost Notes behind Active Notes:
```javascript
// app/static/js/views/pianoRollRenderer.js
export function renderPianoRollCanvas(ctx, canvasWidth, canvasHeight, viewState, activeItem, ghostLayers) {
const { zoomX, zoomY, scrollX, scrollY, noteHeight } = viewState;
// -----------------------------------------------------------------
// LAYER 1: BACKGROUND & GRID LINES
// -----------------------------------------------------------------
drawPianoGrid(ctx, canvasWidth, canvasHeight, viewState);
// -----------------------------------------------------------------
// LAYER 2: GHOST NOTES (BACKGROUND REFERENCE FROM OTHER TRACKS)
// -----------------------------------------------------------------
if (ghostLayers && ghostLayers.length > 0) {
ghostLayers.forEach((layer) => {
ctx.save();
// Set faded opacity (20% - 30% opacity)
ctx.globalAlpha = 0.25;
ctx.fillStyle = layer.track_color;
ctx.strokeStyle = layer.track_color;
layer.notes.forEach((note) => {
const x = (note.relative_start_beat - scrollX) * zoomX;
const y = (127 - note.pitch - scrollY) * noteHeight;
const w = note.duration_beats * zoomX;
const h = noteHeight - 1; // 1px border gap
// Draw Ghost note body (Dashed border or light stroke)
ctx.fillRect(x, y, w, h);
ctx.setLineDash([2, 2]); // Dashed lines indicating non-interactive status
ctx.lineWidth = 1;
ctx.strokeRect(x, y, w, h);
});
ctx.restore();
});
}
// -----------------------------------------------------------------
// LAYER 3: ACTIVE NOTES (EDITABLE NOTES FOR ITEM 1)
// -----------------------------------------------------------------
activeItem.notes.forEach((note) => {
ctx.save();
ctx.globalAlpha = 1.0; // 100% full opacity
const x = (note.start_beat - scrollX) * zoomX;
const y = (127 - note.pitch - scrollY) * noteHeight;
const w = note.duration_beats * zoomX;
const h = noteHeight - 1;
// Bright fill colors for active notes
ctx.fillStyle = note.selected ? "#f59e0b" : "#10b981"; // Orange if selected, green if default
ctx.fillRect(x, y, w, h);
ctx.strokeStyle = "#ffffff";
ctx.lineWidth = 1;
ctx.strokeRect(x, y, w, h);
ctx.restore();
});
}
```
---
## 5. POINTER INTERACTION RULES
To prevent accidental modification or drag interactions with Ghost Notes:
* **Hit-Testing Isolation:** On pointer events (`mousedown`, `mousemove`, or marquee selection), the Hit-Test algorithm queries `activeItem.notes` exclusively. `ghostLayers` items are omitted from event evaluation.
* **Toggle Ghost Notes Visibility:** The Piano Roll toolbar includes an Eye / Ghost icon button (`[👻 Ghost Notes]`) allowing operators to toggle the visibility of the reference background layer.
* **Live Synchronous Updates:** Modifying a note on Track B within the Main Session or an adjacent tab dispatches an `EVENT_SESSION_UPDATED` Event Bus signal. The active Piano Roll Tab receives this event and invokes `extractGhostLayers()` to dynamically redraw updated Ghost Note positions.