SonicForgeStudio/md/30_MIDI_SESSION.md

Here is the conversion of the document into a professional English Markdown format:

ARCHITECTURAL, TECHNICAL, AND ALGORITHMIC SPECIFICATION

Sub-Session System, Section Arrangement & Piano Roll Tab (Hybrid DAW)

This document details the technical solution for building a Hierarchical DAW Engine. This architecture enables nesting Sub-Sessions (Sections) inside the Main Session, alongside a Sub-Tab Editor system (including Piano Roll and Audio Sample Editor) to precisely edit MIDI and Audio Items.

---

0. Non-Breaking Modular Principles (Integration & Backward Compatibility)

To guarantee that new features do not disrupt the DAW's existing core logic and codebase, the entire extension architecture is designed according to these principles:

• Extensibility & Encapsulation:

• The current Session architecture serves directly as the Project Root / Main Session.

• SectionItem, ItemMIDI, and ItemAudio operate as Polymorphic Item Types inheriting from the existing base Item class/interface. Existing Item logic (e.g., drag-and-drop, timeline trimming) remains 100\% untouched.

• Decoupled State Pipeline:

• The logic governing the Playhead, Transport controls (Play/Pause/Stop), and the global Audio Context of the Main Session remains unmodified.

• Nested Time Mapping acts solely as an intermediate Transformation Layer when passing time coordinates down into Sub-Sessions. It does not overwrite or mutate the beat synchronization loop of the Main Timeline.

• Plugin Style Architecture (Audio & MIDI Engine):

• Synth Tracks, Audio Clip Processors, and Sub-Session Sub-Mix Buses plug into the existing AudioNode Graph as auxiliary nodes. They route directly back to the current Master Node without breaking pre-established Gain/Pan/FX pipelines.

---

1. Hierarchical Data Model

To support embedding Sessions within Sessions as well as isolated Clip/Sample-level editing, the data state model expands into an encapsulated Tree Graph structure.

Project Root
├── Main Session (Root Session - Current Session Structure)
│   ├── Track 01 (Audio Track)
│   │   └── ItemAudio: "Vocals.wav" ──► [Opens Audio Sample Editor Sub-Tab]
│   ├── Track 02 (MIDI Track + Synth Engine)
│   │   └── ItemMIDI: "Melody_Main" ──► [Opens Piano Roll Sub-Tab]
│   └── Track 03 (Section Track - New Track Type)
│       └── Item: Section_A (Referencing SubSession_01)
│
├── Sub-Sessions Store (Auxiliary Memory Registry)
│   ├── SubSession_01 ("Verse 1")
│   │   ├── Computed Length: Dynamic Bars (Auto-calculated from longest Item)
│   │   ├── Track 1.1 (Audio Track)
│   │   │   └── ItemAudio: "Guitar_Riff.wav" ──► [Opens Audio Sample Editor Sub-Tab]
│   │   └── Track 1.2 (MIDI Track)
│   │       └── ItemMIDI: "Bassline" ─────────► [Opens Piano Roll Sub-Tab]
│   └── SubSession_02 ("Chorus")
│
└── Active Editor Views / Sub-Tabs (Isolated Editing Contexts)
    ├── Audio Sample Editor Sub-Tab (Edits Audio Clips from Main Session or Sub-Session)
    └── Piano Roll Sub-Tab (Edits MIDI Items from Main Session or Sub-Session)

Detailed Data Schemas (JSON Specs)

**a. Schema: NoteMIDI**

interface NoteMIDI {
  id: string;
  pitch: number;      // 0 - 127 (Midi Note Number, e.g., 60 = C4)
  startTick: number;  // Time coordinate based on Pulses Per Quarter note (PPQ, e.g., 960 PPQ)
  durationTicks: number;
  velocity: number;   // 0 - 127
  selected?: boolean;
}

b. Schema: ItemMIDI (Belongs to MIDI Track - Inherits from Base Item)

interface ItemMIDI {
  id: string;
  type: 'MIDI';
  name: string;
  parentSessionId: string; // Target Session ID (Main or Sub-Session)
  startBar: number;        // Start position on the Timeline (Bar)
  lengthBars: number;      // Item duration in Bars
  offsetTick: number;      // Internal trim offset
  notes: NoteMIDI[];       // Array tracking MIDI Notes
}

c. Schema: ItemAudio (Belongs to Audio Track - Inherits from Base Item)

interface ItemAudio {
  id: string;
  type: 'AUDIO';
  name: string;
  parentSessionId: string; // Target Session ID (Main or Sub-Session)
  startBar: number;
  lengthBars: number;
  samplePath: string;      // Audio file path or Buffer Key
  sampleOffsetSec: number; // Playback start point offset (Trim In)
  gain: number;            // Clip Gain
  pitchShiftSemi: number;  // Pitch Shift (Semitones)
}

d. Schema: SectionItem (Represents a Sub-Session inside the Main Session)

interface SectionItem {
  id: string;
  type: 'SECTION';
  subSessionId: string;  // Reference ID pointing to SubSession inside Memory Store
  name: string;
  startBar: number;
  lengthBars: number;    // Defaults to SubSession.computedLengthBars unless trimmed/cropped
  loop: boolean;         // Enables repetition if lengthBars > SubSession.computedLengthBars
}

e. Schema: Session (Unified structure for both Main Session and Sub-Session)

interface Session {
  id: string;
  name: string;
  isMain: boolean;
  timeSignature: [number, number]; // e.g., [4, 4]
  bpm: number;
  tracks: Track[];
  
  // Dynamically calculated derived state; never assigned manually
  get computedLengthBars(): number; 
}

---

2. Audio & Synth Engine Routing Architecture (Web Audio API)

For MIDI tracks to output audio, each is bound to an Instrument/Synth Instance. When a Section is placed onto the Main Session, all audio generated by its child tracks is bussed directly into the existing Gain/Pan matrix.

Audio Node Graph Diagram

[MIDI Items] ──(Triggers)──► [Synth Engine / Soundfont / WebAssembly VSTi]
                                         │
[Audio Items] ──(Buffer Source)──────────┤
                                         ▼
                               [Track Gain / Pan Node]
                                         │
                                         ▼
                        [Sub-Session Sub-Mix Bus Node]
                                         │
                  ┌──────────────────────┴──────────────────────┐
                  ▼                                             ▼
       [Main Session Audio Graph]                      [Solo / Mute Logic]
       (Current Audio Processing Logic)
                  │
                  ▼
         [Master Destination]

Instrument Engine Processing Logic for MIDI Tracks:

• Virtual Instrument Binding: Every MIDI Track instantiates a synthesis AudioNode (e.g., Web Audio API Soundfont Player, WebSynth JS, or WASM Synthesizer).
• Dynamic Polyphony Engine: As playback scans across MIDI Notes, the system triggers noteOn(pitch, velocity, time) and noteOff(pitch, time) events. These are scheduled ahead of time (100\text{ms} - 200\text{ms} Lookahead) via the AudioContext.currentTime clock.

---

3. Tab UI Management & Event Processing Flow (Tab Navigation Stack)

The graphical interface expands on a Tab Manager & Navigation Stack model to handle isolated views (Views/Sub-tabs) for specific data entities.

[ Tabs Bar ] ── [ Main Session ] │ [ Sub-Session: Verse 1 ] │ [ Piano Roll: Bassline ] │ [ Sample Edit: Vocals.wav ]

Interaction & Navigation Mechanics:

• Opening a Sub-Session Tab:

• Action: User double-clicks a SectionItem on a Main Track.

• Result:

• Instantiates a new Tab using ID = SubSession.id.

• Maps the Timeline Viewport rendering context to the SubSession.

• Enables adding, editing, or deleting child tracks (Audio & MIDI) within the Sub-Session boundary.

• Opening the Piano Roll Sub-Tab:

• Action: User double-clicks an ItemMIDI inside the Main Session OR a Sub-Session.

• Result:

• Instantiates a Sub-tab labeled: Piano Roll - [Item Name].

• Caches context references: { itemId, parentSessionId }.

• Passes the ItemMIDI.notes array directly into the Canvas/Piano Roll Grid.

• Any add/edit/delete actions executed on notes inside the Piano Roll instantly update the native ItemMIDI in the target Session via Mutable/Immutable References.

• Opening the Audio Sample Editor Sub-Tab (Session Edit Audio Sample):

• Action: User double-clicks OR right-clicks and selects "Edit" on an ItemAudio inside the Main Session or a Sub-Session.

• Result:

• Instantiates a Sub-tab labeled: Audio Editor - [Clip Name].

• Loads the high-resolution Waveform of the target ItemAudio onto the sample editing Viewport.

• Provides access to tools: Trim start/end, Normalized Peak, Pitch Shift, Reverse, Fade In/Out, or DSP slicing.

• When clicking Save / Apply Changes: The system updates the ItemAudio attributes (or dispatches a DSP processing request to the Python Server for heavy tasks) and forces a visual refresh of the Clip on the Main Session / Sub-Session timeline.

Data Persistence & Dynamic Sub-Session Length Updates:

• Because JavaScript handles array/object data passing by Reference, modifications made to Notes in the Piano Roll Tab or Clips in the Audio Editor directly update the origin State of the corresponding Session.
• Any add/remove/move/stretch operation targeting an Item inside a Sub-session will immediately trigger the Dynamic Length Recalculation algorithm to update the temporal boundary of the Sub-Session.

---

4. Core Algorithms

Algorithm 1: Dynamic Sub-Session Length Calculation

Sub-sessions do not enforce rigid length constraints. Instead, they dynamically map their duration (L_{\text{bars}}) to match the furthest end-point of all encapsulated Items.

Formula: Given a Sub-Session containing a list of T tracks, where each track t holds a list of I_t items (Audio, MIDI, etc.):

\text{ItemEndBar}(item) = item.\text{startBar} + item.\text{lengthBars} L_{\text{bars}} = \max_{t \in T} \left( \max_{i \in I_t} (\text{ItemEndBar}(i)) \right)

If the Sub-session is entirely empty (contains no Items), L_{\text{bars}} defaults to 1 Bar (or the default duration of a single grid bar).

function calculateSubSessionLength(subSession) {
  let maxEndBar = 1; // Minimum duration fallback for empty sub-sessions

  for (const track of subSession.tracks) {
    for (const item of track.items) {
      const itemEndBar = item.startBar + item.lengthBars;
      if (itemEndBar > maxEndBar) {
        maxEndBar = itemEndBar;
      }
    }
  }

  return maxEndBar;
}

Algorithm 2: Nested Time Mapping

When the Main Session Playhead tracks time T_{\text{main}} (seconds), the engine must calculate the relative time coordinate T_{\text{sub}} inside the active Sub-Session.

Formula: Assume:

• S_{\text{bar}}: The starting Bar of the Section Item on the Main Timeline.
• L_{\text{bars}}: The dynamically evaluated length of the root Sub-Session (L_{\text{bars}} = \text{calculateSubSessionLength}(\text{SubSession})).
• BPM: Beats Per Minute.
• TimeSig: Beats per Bar (e.g., 4 beats).

\text{SecondsPerBar} = \frac{60}{\text{BPM}} \times \text{TimeSig} \text{OffsetSeconds} = (T_{\text{main}} - (S_{\text{bar}} - 1) \times \text{SecondsPerBar})

If SectionItem.loop = true:

T_{\text{sub}} = \text{OffsetSeconds} \pmod{L_{\text{bars}} \times \text{SecondsPerBar}}

If SectionItem.loop = false:

T_{\text{sub}} = \begin{cases} \text{OffsetSeconds} & \text{if } 0 \le \text{OffsetSeconds} \le (L_{\text{bars}} \times \text{SecondsPerBar}) \\ \text{undefined} & \text{if out of bounds} \end{cases}

Algorithm 3: Lookahead MIDI Scheduler

JavaScript's setInterval function lacks the temporal precision required for audio playback. We employ the Web Audio Lookahead Scheduler algorithm combined with Ticks \rightarrow Seconds translation.

const PPQ = 960; // 960 Pulses Per Quarter note (Standard MIDI resolution)
let nextNoteIndex = 0;
const scheduleAheadTime = 0.2; // 200ms Lookahead buffer
const lookaheadMs = 25;        // Polling interval interval block (25ms)

function ticksToSeconds(ticks, bpm) {
  const secondsPerQuarterNote = 60.0 / bpm;
  return (ticks / PPQ) * secondsPerQuarterNote;
}

function scheduler(midiItem, audioCtx, currentPlayheadTime) {
  // Extract notes mapped within [currentPlayheadTime, currentPlayheadTime + scheduleAheadTime]
  while (nextNoteIndex < midiItem.notes.length) {
    const note = midiItem.notes[nextNoteIndex];
    const noteStartTimeSec = ticksToSeconds(note.startTick, currentBpm);

    if (noteStartTimeSec >= currentPlayheadTime + scheduleAheadTime) {
      break; // Note start bounds exceed the active Lookahead window
    }

    if (noteStartTimeSec >= currentPlayheadTime) {
      // Calculate absolute scheduling time against the AudioContext Clock
      const audioCtxStartTime = audioCtx.currentTime + (noteStartTimeSec - currentPlayheadTime);
      const durationSec = ticksToSeconds(note.durationTicks, currentBpm);
      
      // Fire the VSTi/Synth Engine
      trackSynthEngine.playNote(note.pitch, note.velocity, audioCtxStartTime, durationSec);
    }
    nextNoteIndex++;
  }
}

Algorithm 4: Grid Snapping & Quantization (Piano Roll)

When adding or dragging a MIDI Note in the Piano Roll Tab, the X coordinate of the mouse cursor must snap to the nearest rhythmic grid boundary (1/4, 1/8, 1/16, 1/32 Note).

function snapTickToGrid(rawTick, gridFraction, ppq) {
  // gridFraction: 0.25 (1/4 note), 0.125 (1/8 note), 0.0625 (1/16 note)
  const ticksPerGridStep = ppq * (gridFraction * 4);
  
  // Snap rounding formula targeting the nearest grid boundary
  const snappedTick = Math.round(rawTick / ticksPerGridStep) * ticksPerGridStep;
  return Math.max(0, snappedTick);
}

---

5. Performance Optimization

• Virtual Rendering for Piano Roll, Audio Sample Editor & Main Session:

• Never render the entire array of MIDI Notes or total Audio Waveforms simultaneously into the HTML DOM.

• Mandatory use of HTML5 Canvas 2D / WebGL paired with Virtual Viewport Rendering (only drawing Notes/Samples situated within the active Viewport Rect boundary).

• Audio Bouncing / Freezing (For Heavy Sections):

• If a Sub-Session houses too many Tracks and VSTi plugins, causing CPU bottlenecks during Main Session playback:

• Enable the "Freeze Section" action: The Python backend processes the request, rendering that entire Sub-Session block into a single temporary Audio WAV file (Bounce to Disk).

• The Main Session then only processes one discrete Audio file instead of simultaneously calculating dozens of child tracks.

• Immutable State & Undo/Redo Engine:

• Project State management is handled via the Redux/Zustand pattern model.

• Every add/edit/delete operation applied to Notes on the Piano Roll or edits made to Audio Clips generates an Action that pushes to the UndoStack, supporting seamless Ctrl + Z shortcuts across every Sub-tab context.