361 lines
15 KiB
Markdown
361 lines
15 KiB
Markdown
Here is the conversion of the document into a professional English Markdown format:
|
|
|
|
# ARCHITECTURAL, TECHNICAL, AND ALGORITHMIC SPECIFICATION
|
|
|
|
## Sub-Session System, Section Arrangement & Piano Roll Tab (Hybrid DAW)
|
|
|
|
This document details the technical solution for building a Hierarchical DAW Engine. This architecture enables nesting Sub-Sessions (Sections) inside the Main Session, alongside a Sub-Tab Editor system (including Piano Roll and Audio Sample Editor) to precisely edit MIDI and Audio Items.
|
|
|
|
---
|
|
|
|
### 0. Non-Breaking Modular Principles (Integration & Backward Compatibility)
|
|
|
|
To guarantee that new features do not disrupt the DAW's existing core logic and codebase, the entire extension architecture is designed according to these principles:
|
|
|
|
* **Extensibility & Encapsulation:**
|
|
* The current Session architecture serves directly as the **Project Root / Main Session**.
|
|
* `SectionItem`, `ItemMIDI`, and `ItemAudio` operate as **Polymorphic Item Types** inheriting from the existing base `Item` class/interface. Existing Item logic (e.g., drag-and-drop, timeline trimming) remains $100\%$ untouched.
|
|
|
|
|
|
* **Decoupled State Pipeline:**
|
|
* The logic governing the Playhead, Transport controls (Play/Pause/Stop), and the global Audio Context of the Main Session remains unmodified.
|
|
* **Nested Time Mapping** acts solely as an intermediate Transformation Layer when passing time coordinates down into Sub-Sessions. It does not overwrite or mutate the beat synchronization loop of the Main Timeline.
|
|
|
|
|
|
* **Plugin Style Architecture (Audio & MIDI Engine):**
|
|
* Synth Tracks, Audio Clip Processors, and Sub-Session Sub-Mix Buses plug into the existing AudioNode Graph as auxiliary nodes. They route directly back to the current Master Node without breaking pre-established Gain/Pan/FX pipelines.
|
|
|
|
|
|
|
|
---
|
|
|
|
### 1. Hierarchical Data Model
|
|
|
|
To support embedding Sessions within Sessions as well as isolated Clip/Sample-level editing, the data state model expands into an encapsulated Tree Graph structure.
|
|
|
|
```text
|
|
Project Root
|
|
├── Main Session (Root Session - Current Session Structure)
|
|
│ ├── Track 01 (Audio Track)
|
|
│ │ └── ItemAudio: "Vocals.wav" ──► [Opens Audio Sample Editor Sub-Tab]
|
|
│ ├── Track 02 (MIDI Track + Synth Engine)
|
|
│ │ └── ItemMIDI: "Melody_Main" ──► [Opens Piano Roll Sub-Tab]
|
|
│ └── Track 03 (Section Track - New Track Type)
|
|
│ └── Item: Section_A (Referencing SubSession_01)
|
|
│
|
|
├── Sub-Sessions Store (Auxiliary Memory Registry)
|
|
│ ├── SubSession_01 ("Verse 1")
|
|
│ │ ├── Computed Length: Dynamic Bars (Auto-calculated from longest Item)
|
|
│ │ ├── Track 1.1 (Audio Track)
|
|
│ │ │ └── ItemAudio: "Guitar_Riff.wav" ──► [Opens Audio Sample Editor Sub-Tab]
|
|
│ │ └── Track 1.2 (MIDI Track)
|
|
│ │ └── ItemMIDI: "Bassline" ─────────► [Opens Piano Roll Sub-Tab]
|
|
│ └── SubSession_02 ("Chorus")
|
|
│
|
|
└── Active Editor Views / Sub-Tabs (Isolated Editing Contexts)
|
|
├── Audio Sample Editor Sub-Tab (Edits Audio Clips from Main Session or Sub-Session)
|
|
└── Piano Roll Sub-Tab (Edits MIDI Items from Main Session or Sub-Session)
|
|
|
|
```
|
|
|
|
#### Detailed Data Schemas (JSON Specs)
|
|
|
|
**a. Schema: `NoteMIDI**`
|
|
|
|
```typescript
|
|
interface NoteMIDI {
|
|
id: string;
|
|
pitch: number; // 0 - 127 (Midi Note Number, e.g., 60 = C4)
|
|
startTick: number; // Time coordinate based on Pulses Per Quarter note (PPQ, e.g., 960 PPQ)
|
|
durationTicks: number;
|
|
velocity: number; // 0 - 127
|
|
selected?: boolean;
|
|
}
|
|
|
|
```
|
|
|
|
**b. Schema: `ItemMIDI` (Belongs to MIDI Track - Inherits from Base Item)**
|
|
|
|
```typescript
|
|
interface ItemMIDI {
|
|
id: string;
|
|
type: 'MIDI';
|
|
name: string;
|
|
parentSessionId: string; // Target Session ID (Main or Sub-Session)
|
|
startBar: number; // Start position on the Timeline (Bar)
|
|
lengthBars: number; // Item duration in Bars
|
|
offsetTick: number; // Internal trim offset
|
|
notes: NoteMIDI[]; // Array tracking MIDI Notes
|
|
}
|
|
|
|
```
|
|
|
|
**c. Schema: `ItemAudio` (Belongs to Audio Track - Inherits from Base Item)**
|
|
|
|
```typescript
|
|
interface ItemAudio {
|
|
id: string;
|
|
type: 'AUDIO';
|
|
name: string;
|
|
parentSessionId: string; // Target Session ID (Main or Sub-Session)
|
|
startBar: number;
|
|
lengthBars: number;
|
|
samplePath: string; // Audio file path or Buffer Key
|
|
sampleOffsetSec: number; // Playback start point offset (Trim In)
|
|
gain: number; // Clip Gain
|
|
pitchShiftSemi: number; // Pitch Shift (Semitones)
|
|
}
|
|
|
|
```
|
|
|
|
**d. Schema: `SectionItem` (Represents a Sub-Session inside the Main Session)**
|
|
|
|
```typescript
|
|
interface SectionItem {
|
|
id: string;
|
|
type: 'SECTION';
|
|
subSessionId: string; // Reference ID pointing to SubSession inside Memory Store
|
|
name: string;
|
|
startBar: number;
|
|
lengthBars: number; // Defaults to SubSession.computedLengthBars unless trimmed/cropped
|
|
loop: boolean; // Enables repetition if lengthBars > SubSession.computedLengthBars
|
|
}
|
|
|
|
```
|
|
|
|
**e. Schema: `Session` (Unified structure for both Main Session and Sub-Session)**
|
|
|
|
```typescript
|
|
interface Session {
|
|
id: string;
|
|
name: string;
|
|
isMain: boolean;
|
|
timeSignature: [number, number]; // e.g., [4, 4]
|
|
bpm: number;
|
|
tracks: Track[];
|
|
|
|
// Dynamically calculated derived state; never assigned manually
|
|
get computedLengthBars(): number;
|
|
}
|
|
|
|
```
|
|
|
|
---
|
|
|
|
### 2. Audio & Synth Engine Routing Architecture (Web Audio API)
|
|
|
|
For MIDI tracks to output audio, each is bound to an Instrument/Synth Instance. When a Section is placed onto the Main Session, all audio generated by its child tracks is bussed directly into the existing Gain/Pan matrix.
|
|
|
|
#### Audio Node Graph Diagram
|
|
|
|
```text
|
|
[MIDI Items] ──(Triggers)──► [Synth Engine / Soundfont / WebAssembly VSTi]
|
|
│
|
|
[Audio Items] ──(Buffer Source)──────────┤
|
|
▼
|
|
[Track Gain / Pan Node]
|
|
│
|
|
▼
|
|
[Sub-Session Sub-Mix Bus Node]
|
|
│
|
|
┌──────────────────────┴──────────────────────┐
|
|
▼ ▼
|
|
[Main Session Audio Graph] [Solo / Mute Logic]
|
|
(Current Audio Processing Logic)
|
|
│
|
|
▼
|
|
[Master Destination]
|
|
|
|
```
|
|
|
|
**Instrument Engine Processing Logic for MIDI Tracks:**
|
|
|
|
* **Virtual Instrument Binding:** Every MIDI Track instantiates a synthesis `AudioNode` (e.g., Web Audio API Soundfont Player, WebSynth JS, or WASM Synthesizer).
|
|
* **Dynamic Polyphony Engine:** As playback scans across MIDI Notes, the system triggers `noteOn(pitch, velocity, time)` and `noteOff(pitch, time)` events. These are scheduled ahead of time ($100\text{ms} - 200\text{ms}$ Lookahead) via the `AudioContext.currentTime` clock.
|
|
|
|
---
|
|
|
|
### 3. Tab UI Management & Event Processing Flow (Tab Navigation Stack)
|
|
|
|
The graphical interface expands on a Tab Manager & Navigation Stack model to handle isolated views (Views/Sub-tabs) for specific data entities.
|
|
|
|
```text
|
|
[ Tabs Bar ] ── [ Main Session ] │ [ Sub-Session: Verse 1 ] │ [ Piano Roll: Bassline ] │ [ Sample Edit: Vocals.wav ]
|
|
|
|
```
|
|
|
|
#### Interaction & Navigation Mechanics:
|
|
|
|
* **Opening a Sub-Session Tab:**
|
|
* *Action:* User double-clicks a `SectionItem` on a Main Track.
|
|
* *Result:*
|
|
* Instantiates a new Tab using `ID = SubSession.id`.
|
|
* Maps the Timeline Viewport rendering context to the SubSession.
|
|
* Enables adding, editing, or deleting child tracks (Audio & MIDI) within the Sub-Session boundary.
|
|
|
|
|
|
|
|
|
|
* **Opening the Piano Roll Sub-Tab:**
|
|
* *Action:* User double-clicks an `ItemMIDI` inside the Main Session OR a Sub-Session.
|
|
* *Result:*
|
|
* Instantiates a Sub-tab labeled: `Piano Roll - [Item Name]`.
|
|
* Caches context references: `{ itemId, parentSessionId }`.
|
|
* Passes the `ItemMIDI.notes` array directly into the Canvas/Piano Roll Grid.
|
|
* Any add/edit/delete actions executed on notes inside the Piano Roll instantly update the native `ItemMIDI` in the target Session via Mutable/Immutable References.
|
|
|
|
|
|
|
|
|
|
* **Opening the Audio Sample Editor Sub-Tab (Session Edit Audio Sample):**
|
|
* *Action:* User double-clicks OR right-clicks and selects "Edit" on an `ItemAudio` inside the Main Session or a Sub-Session.
|
|
* *Result:*
|
|
* Instantiates a Sub-tab labeled: `Audio Editor - [Clip Name]`.
|
|
* Loads the high-resolution Waveform of the target `ItemAudio` onto the sample editing Viewport.
|
|
* Provides access to tools: Trim start/end, Normalized Peak, Pitch Shift, Reverse, Fade In/Out, or DSP slicing.
|
|
* When clicking *Save / Apply Changes*: The system updates the `ItemAudio` attributes (or dispatches a DSP processing request to the Python Server for heavy tasks) and forces a visual refresh of the Clip on the Main Session / Sub-Session timeline.
|
|
|
|
|
|
|
|
|
|
|
|
#### Data Persistence & Dynamic Sub-Session Length Updates:
|
|
|
|
* Because JavaScript handles array/object data passing by **Reference**, modifications made to Notes in the Piano Roll Tab or Clips in the Audio Editor directly update the origin State of the corresponding Session.
|
|
* Any add/remove/move/stretch operation targeting an Item inside a Sub-session will immediately trigger the **Dynamic Length Recalculation** algorithm to update the temporal boundary of the Sub-Session.
|
|
|
|
---
|
|
|
|
### 4. Core Algorithms
|
|
|
|
#### Algorithm 1: Dynamic Sub-Session Length Calculation
|
|
|
|
Sub-sessions do not enforce rigid length constraints. Instead, they dynamically map their duration ($L_{\text{bars}}$) to match the furthest end-point of all encapsulated Items.
|
|
|
|
**Formula:**
|
|
Given a Sub-Session containing a list of $T$ tracks, where each track $t$ holds a list of $I_t$ items (Audio, MIDI, etc.):
|
|
|
|
|
|
$$\text{ItemEndBar}(item) = item.\text{startBar} + item.\text{lengthBars}$$
|
|
|
|
$$L_{\text{bars}} = \max_{t \in T} \left( \max_{i \in I_t} (\text{ItemEndBar}(i)) \right)$$
|
|
|
|
*If the Sub-session is entirely empty (contains no Items), $L_{\text{bars}}$ defaults to $1$ Bar (or the default duration of a single grid bar).*
|
|
|
|
```javascript
|
|
function calculateSubSessionLength(subSession) {
|
|
let maxEndBar = 1; // Minimum duration fallback for empty sub-sessions
|
|
|
|
for (const track of subSession.tracks) {
|
|
for (const item of track.items) {
|
|
const itemEndBar = item.startBar + item.lengthBars;
|
|
if (itemEndBar > maxEndBar) {
|
|
maxEndBar = itemEndBar;
|
|
}
|
|
}
|
|
}
|
|
|
|
return maxEndBar;
|
|
}
|
|
|
|
```
|
|
|
|
#### Algorithm 2: Nested Time Mapping
|
|
|
|
When the Main Session Playhead tracks time $T_{\text{main}}$ (seconds), the engine must calculate the relative time coordinate $T_{\text{sub}}$ inside the active Sub-Session.
|
|
|
|
**Formula:**
|
|
Assume:
|
|
|
|
* $S_{\text{bar}}$: The starting Bar of the Section Item on the Main Timeline.
|
|
* $L_{\text{bars}}$: The dynamically evaluated length of the root Sub-Session ($L_{\text{bars}} = \text{calculateSubSessionLength}(\text{SubSession})$).
|
|
* $BPM$: Beats Per Minute.
|
|
* $TimeSig$: Beats per Bar (e.g., 4 beats).
|
|
|
|
$$\text{SecondsPerBar} = \frac{60}{\text{BPM}} \times \text{TimeSig}$$
|
|
|
|
$$\text{OffsetSeconds} = (T_{\text{main}} - (S_{\text{bar}} - 1) \times \text{SecondsPerBar})$$
|
|
|
|
If `SectionItem.loop = true`:
|
|
|
|
|
|
$$T_{\text{sub}} = \text{OffsetSeconds} \pmod{L_{\text{bars}} \times \text{SecondsPerBar}}$$
|
|
|
|
If `SectionItem.loop = false`:
|
|
|
|
|
|
$$T_{\text{sub}} = \begin{cases} \text{OffsetSeconds} & \text{if } 0 \le \text{OffsetSeconds} \le (L_{\text{bars}} \times \text{SecondsPerBar}) \\ \text{undefined} & \text{if out of bounds} \end{cases}$$
|
|
|
|
#### Algorithm 3: Lookahead MIDI Scheduler
|
|
|
|
JavaScript's `setInterval` function lacks the temporal precision required for audio playback. We employ the **Web Audio Lookahead Scheduler** algorithm combined with Ticks $\rightarrow$ Seconds translation.
|
|
|
|
```javascript
|
|
const PPQ = 960; // 960 Pulses Per Quarter note (Standard MIDI resolution)
|
|
let nextNoteIndex = 0;
|
|
const scheduleAheadTime = 0.2; // 200ms Lookahead buffer
|
|
const lookaheadMs = 25; // Polling interval interval block (25ms)
|
|
|
|
function ticksToSeconds(ticks, bpm) {
|
|
const secondsPerQuarterNote = 60.0 / bpm;
|
|
return (ticks / PPQ) * secondsPerQuarterNote;
|
|
}
|
|
|
|
function scheduler(midiItem, audioCtx, currentPlayheadTime) {
|
|
// Extract notes mapped within [currentPlayheadTime, currentPlayheadTime + scheduleAheadTime]
|
|
while (nextNoteIndex < midiItem.notes.length) {
|
|
const note = midiItem.notes[nextNoteIndex];
|
|
const noteStartTimeSec = ticksToSeconds(note.startTick, currentBpm);
|
|
|
|
if (noteStartTimeSec >= currentPlayheadTime + scheduleAheadTime) {
|
|
break; // Note start bounds exceed the active Lookahead window
|
|
}
|
|
|
|
if (noteStartTimeSec >= currentPlayheadTime) {
|
|
// Calculate absolute scheduling time against the AudioContext Clock
|
|
const audioCtxStartTime = audioCtx.currentTime + (noteStartTimeSec - currentPlayheadTime);
|
|
const durationSec = ticksToSeconds(note.durationTicks, currentBpm);
|
|
|
|
// Fire the VSTi/Synth Engine
|
|
trackSynthEngine.playNote(note.pitch, note.velocity, audioCtxStartTime, durationSec);
|
|
}
|
|
nextNoteIndex++;
|
|
}
|
|
}
|
|
|
|
```
|
|
|
|
#### Algorithm 4: Grid Snapping & Quantization (Piano Roll)
|
|
|
|
When adding or dragging a MIDI Note in the Piano Roll Tab, the $X$ coordinate of the mouse cursor must snap to the nearest rhythmic grid boundary (1/4, 1/8, 1/16, 1/32 Note).
|
|
|
|
```javascript
|
|
function snapTickToGrid(rawTick, gridFraction, ppq) {
|
|
// gridFraction: 0.25 (1/4 note), 0.125 (1/8 note), 0.0625 (1/16 note)
|
|
const ticksPerGridStep = ppq * (gridFraction * 4);
|
|
|
|
// Snap rounding formula targeting the nearest grid boundary
|
|
const snappedTick = Math.round(rawTick / ticksPerGridStep) * ticksPerGridStep;
|
|
return Math.max(0, snappedTick);
|
|
}
|
|
|
|
```
|
|
|
|
---
|
|
|
|
### 5. Performance Optimization
|
|
|
|
* **Virtual Rendering for Piano Roll, Audio Sample Editor & Main Session:**
|
|
* Never render the entire array of MIDI Notes or total Audio Waveforms simultaneously into the HTML DOM.
|
|
* Mandatory use of HTML5 Canvas 2D / WebGL paired with **Virtual Viewport Rendering** (only drawing Notes/Samples situated within the active Viewport Rect boundary).
|
|
|
|
|
|
* **Audio Bouncing / Freezing (For Heavy Sections):**
|
|
* If a Sub-Session houses too many Tracks and VSTi plugins, causing CPU bottlenecks during Main Session playback:
|
|
* Enable the **"Freeze Section"** action: The Python backend processes the request, rendering that entire Sub-Session block into a single temporary Audio WAV file (Bounce to Disk).
|
|
* The Main Session then only processes one discrete Audio file instead of simultaneously calculating dozens of child tracks.
|
|
|
|
|
|
* **Immutable State & Undo/Redo Engine:**
|
|
* Project State management is handled via the Redux/Zustand pattern model.
|
|
* Every add/edit/delete operation applied to Notes on the Piano Roll or edits made to Audio Clips generates an Action that pushes to the `UndoStack`, supporting seamless `Ctrl + Z` shortcuts across every Sub-tab context. |