# DESIGN SPECIFICATION & IMPLEMENTATION ROADMAP: FLOATING VIRTUAL MIDI KEYBOARD & CHORDS PANEL ENGINE

This document specifies the technical architecture, UI/UX design, data flow diagrams, and interaction matrices for integrating two core features into the DAW system:

1. **Floating Virtual MIDI Keyboard:** A draggable floating virtual keyboard window that receives input from computer QWERTY keys, supporting both live preview and real-time recording of MIDI notes directly into the Timeline / Piano Roll.
2. **Comprehensive Scale & Chords System / Insert Chords Panel:** A multi-genre chord theory engine, automated chord progression insertion panel, custom chord builder/storage, and internet/AI chord lookup system.

---

## I. SYSTEM ARCHITECTURE OVERVIEW

```text
┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                   CLIENT FRONTEND STUDIO                                        │
│                                                                                                 │
│   ┌─────────────────────────────────────────┐         ┌─────────────────────────────────────┐   │
│   │ Menu: Tools -> Virtual MIDI Keyboard    │         │ Menu: Insert -> Chords panel        │   │
│   │ Shortcut: [F2]                          │         │ Shortcut: [Shift + K]               │   │
│   └────────────────────┬────────────────────┘         └──────────────────┬──────────────────┘   │
└────────────────────────┼─────────────────────────────────────────────────┼──────────────────────┘
                         │                                                 │
                         ▼                                                 ▼
┌──────────────────────────────────────────────────┐     ┌────────────────────────────────────────┐
│ FLOATING VIRTUAL MIDI KEYBOARD (UI OVERLAY)      │     │ INSERT CHORDS PANEL (MODAL / SIDEBAR)   │
│ - Visual 25/49-Keybed Rendering                  │     │ - Style Catalog (Pop, Jazz, Epic...)   │
│ - QWERTY Key Mapping Engine                      │     │ - Custom Chord Builder & LocalStorage  │
│ - Octave Shift (-4 to +4) & Velocity Adjuster    │     │ - Internet / AI Chord Search Engine    │
└────────────────────────┬─────────────────────────┘     └─────────────────┬──────────────────────┘
                         │                                                 │
                         ▼                                                 ▼
┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                UNIFIED MIDI ROUTER & DISPATCHER                                 │
│ - Normalizes to `UnifiedMidiEvent`: { command, channel, pitch, velocity, timestamp }            │
└────────────────────────┬─────────────────────────────────────────────────┬──────────────────────┘
                         │                                                 │
                         ├─────────────────────────────────┐               │
                         ▼                                 ▼               ▼
┌──────────────────────────────────────────────────┐ ┌────────────────────────────────────────────┐
│ REAL-TIME SYNTH ENGINE (CLIENT WASM / BRIDGE)    │ │ CLIENT MIDI RECORDER & TIMELINE INGESTION  │
│ - Real-time Audio Preview (< 5ms Latency)        │ │ - Live Recording to active `MIDIItem`      │
│ - FluidSynth WASM / Native VSTi Bridge           │ │ - Direct Canvas Redraw on Piano Roll       │
└──────────────────────────────────────────────────┘ └────────────────────────────────────────────┘

```

---

## II. PART I: FLOATING VIRTUAL MIDI KEYBOARD SYSTEM

### 1. Activation & Floating Window Management

* **Activation Triggers:**
* **System Shortcut:** **F2** (Toggle On/Off).
* **Menu Bar:** **Tools ▾ -> Virtual MIDI Keyboard**.


* **Window Properties:**
* **Floating & Draggable:** Allows dynamic positioning across MAIN SESSION, SECTION-TAB, and PIANO ROLL TAB views.
* **Always-on-Top / Z-Index Isolation:** Renders above all Timeline Canvases without capturing global Transport hotkeys (**Space** for Play/Stop, **R** for Record).
* **State Persistence:** Saves coordinates ($x, y$), active toggle state, current octave offset, and default velocity to `localStorage`.



---

### 2. Virtual Keyboard User Interface Layout

```text
┌────────────────────────────────────────────────────────────────────────────────────────┐
│ 🎹 VIRTUAL MIDI KEYBOARD                                                       [–] [X] │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ [ Octave: -1 ] [ Octave: +1 ] │ Octave Shift: 4 (C4-C6) │ Velocity: [ 100 ] [Slider---|]  │
│ [ Channel: 1 ▾ ]              │ Transpose: 0 semitones  │ Scale Highlight: [ C Minor ▾ ] │
├────────────────────────────────────────────────────────────────────────────────────────┤
│  |   | | |   |   | | | | |   |   | | |   |   | | | | |   |   | | |   |   | | | | |   | │
│  |   |W| |E  |   |T| |Y| |U  |   |2| |3  |   |5| |6| |7  |   | | |   |   | | | | |   | │
│  |   |_| |_| |   |_| |_| |_| |   |_| |_| |   |_| |_| |_| |   |_| |_| |   |_| |_| |_| | │
│  | A | S | D | F | G | H | J | K | Q | W | E | R | T | Y | U | I | O | P |   |   |   | │
│  └───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┴───┘ │
└────────────────────────────────────────────────────────────────────────────────────────┘

```

#### Keyboard Control Bar Components:

* **Octave Down / Up (`[Oct -]`, `[Oct +]`):** Shifts pitch octave range (Shortcuts: **Shift + Z** / **Shift + X** or **[** / **]**).
* **Velocity Slider:** Adjusts note velocity from **1 to 127** (Default: **100**).
* **Target MIDI Channel:** Selects MIDI Channel from **1 to 16** (Defaults automatically to the currently selected/armed track's channel).
* **Scale Highlight Toggle:** Visual keybed guide highlighting keys that belong to the active musical scale.

---

### 3. Computer QWERTY Keyboard Mapping Matrix

Uses two rows of QWERTY keys to span two continuous octaves:

#### Lower Octave (Root Base):

| Computer Key | Relative Pitch | Note Name | Key Type |
| --- | --- | --- | --- |
| **Z** | $\text{Root} + 0$ | C | White Key |
| **S** | $\text{Root} + 1$ | C# / Db | Black Key |
| **X** | $\text{Root} + 2$ | D | White Key |
| **D** | $\text{Root} + 3$ | D# / Eb | Black Key |
| **C** | $\text{Root} + 4$ | E | White Key |
| **V** | $\text{Root} + 5$ | F | White Key |
| **G** | $\text{Root} + 6$ | F# / Gb | Black Key |
| **B** | $\text{Root} + 7$ | G | White Key |
| **H** | $\text{Root} + 8$ | G# / Ab | Black Key |
| **N** | $\text{Root} + 9$ | A | White Key |
| **J** | $\text{Root} + 10$ | A# / Bb | Black Key |
| **M** | $\text{Root} + 11$ | B | White Key |

#### Upper Octave ($+12$ Semitones):

| Computer Key | Relative Pitch | Note Name | Key Type |
| --- | --- | --- | --- |
| **Q** | $\text{Root} + 12$ | C (+1 Oct) | White Key |
| **2** | $\text{Root} + 13$ | C# (+1 Oct) | Black Key |
| **W** | $\text{Root} + 14$ | D (+1 Oct) | White Key |
| **3** | $\text{Root} + 15$ | D# (+1 Oct) | Black Key |
| **E** | $\text{Root} + 16$ | E (+1 Oct) | White Key |
| **R** | $\text{Root} + 17$ | F (+1 Oct) | White Key |
| **5** | $\text{Root} + 18$ | F# (+1 Oct) | Black Key |
| **T** | $\text{Root} + 19$ | G (+1 Oct) | White Key |
| **6** | $\text{Root} + 20$ | G# (+1 Oct) | Black Key |
| **Y** | $\text{Root} + 21$ | A (+1 Oct) | White Key |
| **7** | $\text{Root} + 22$ | A# (+1 Oct) | Black Key |
| **U** | $\text{Root} + 23$ | B (+1 Oct) | White Key |
| **I** | $\text{Root} + 24$ | C (+2 Oct) | White Key |

---

### 4. Audio Playback & Recording Event Pipeline

#### Key Press Handler (`KeyDown` Event):

1. Verifies `e.repeat` is false to prevent event flood on key holds.
2. If the active DOM element is a text input field (`<input>`, `<textarea>`), passes through and skips the piano handler.
3. Computes the absolute MIDI Pitch value:

$$Pitch = (Octave + 1) \times 12 + PitchOffset$$

4. Highlights the corresponding visual key on the Virtual Keyboard UI.
5. Dispatches the event to the `UnifiedMidiRouter`:

```javascript
unifiedMidiRouter.dispatchMidiEvent({
  command: 'NOTE_ON',
  channel: activeTrackChannel,
  pitch: calculatedPitch,
  velocity: currentVirtualVelocity,
  sourceType: 'VIRTUAL_KEYBOARD'
});

```

6. **Live Recording Mode (If Transport Record is ACTIVE):**
The `ClientMIDIRecorder` captures the `NOTE_ON` event, creates a note instance in the active `MIDIItem` on the armed track, and invokes a `requestAnimationFrame` loop to render real-time red/orange preview blocks on the Timeline and Piano Roll canvases.

#### Key Release Handler (`KeyUp` Event):

1. Removes key highlight from the Virtual Keyboard UI.
2. Dispatches a `NOTE_OFF` event to `UnifiedMidiRouter`.
3. `ClientMIDIRecorder` calculates total note duration:

$$Duration = Beat_{\text{release}} - Beat_{\text{press}}$$

and writes the finalized note data to the clip's state array.

---

## III. PART II: COMPREHENSIVE SCALE & CHORD SYSTEM & CHORDS PANEL

### 1. Chords Panel Trigger

* **Activation Triggers:**
* **Menu Bar:** **Insert ▾ -> Chords panel**.
* **System Shortcut:** **Shift + K** or clicking the 🎼 **Chords** icon on the Piano Roll Toolbar.


* **Display Format:**
* Centered modal window or sliding dockable panel positioned on the right side of the Piano Roll.



---

### 2. Style-Based Chord Catalog

Contains pre-coded chord progressions categorized by musical genre and emotional mood:

```text
┌────────────────────────────────────────────────────────────────────────────────────────┐
│ 🎼 INSERT CHORDS PANEL                                                         [X]     │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ [ Filter Style: Pop / Ballad ▾ ] [ Key: C ▾ ] [ Scale: Major ▾ ] [ Search: _______ ]  │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ PRESET PROGRESSIONS:                                                                   │
│ ┌────────────────────────────────────────────────────────────────────────────────────┐ │
│ │ 🌟 Pop Classic Emotional (I - V - vi - IV)                                         │ │
│ │    Chords: Cmaj -> Gmaj -> Am -> Fmaj | Voicing: Root Position | Rhythm: 1/2 Beat   │ │
│ │    [ Preview Sound ] [ Insert to Timeline ] [ Favorite ★ ]                         │ │
│ ├────────────────────────────────────────────────────────────────────────────────────┤ │
│ │ 🎷 Jazz Neo-Soul Smooth (ii7 - V7 - Imaj7 - VI7)                                   │ │
│ │    Chords: Dm7 -> G7 -> Cmaj7 -> A7 | Voicing: Drop-2 | Rhythm: Syncopated        │ │
│ │    [ Preview Sound ] [ Insert to Timeline ] [ Favorite ★ ]                         │ │
│ ├────────────────────────────────────────────────────────────────────────────────────┤ │
│ │ 🎬 Epic Film Score Rising (i - VI - III - VII)                                     │ │
│ │    Chords: Cm -> Ab -> Eb -> Bb | Voicing: Power Octaves | Rhythm: 8th Arp         │ │
│ │    [ Preview Sound ] [ Insert to Timeline ] [ Favorite ★ ]                         │ │
│ └────────────────────────────────────────────────────────────────────────────────────┘ │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ TAB: [ Preset Library ]  [ Custom Chord Builder ]  [ Internet / AI Search ]            │
└────────────────────────────────────────────────────────────────────────────────────────┘

```

#### Music Style Catalog Matrix:

| Genre / Style | Standard Progression (Roman Numerals) | Transposed Example (Root Key C) | Voicing & Rhythmic Characteristics |
| --- | --- | --- | --- |
| **Pop / Ballad** | $I - V - vi - IV$ | $C - G - Am - F$ | Sustained Whole Notes, Triad Voicings |
| **Jazz / Neo-Soul** | $ii^7 - V^7 - I^{\text{maj}7} - VI^7$ | $Dm^7 - G^7 - C^{\text{maj}7} - A^7$ | Drop-2 Voicings, Off-beat Syncopation |
| **Cinematic / Epic** | $i - VI - III - VII$ | $Cm - Ab - Eb - Bb$ | Low-octave bass root, Octave layering, 8th-note arpeggiation |
| **EDM / Future Bass** | $vi - IV - I - V$ | $Am - F - C - G$ | 16th-note synth stabs, Inverted voicings |
| **Lofi Chillhop** | $i^7 - iv^7 - v^7 - i^7$ | $Cm^7 - Fm^7 - Gm^7 - Cm^7$ | Add9 / Min7 Extensions, Micro-timing jitter |
| **R&B / Soul** | $I^{\text{maj}7} - iii^7 - IV^{\text{maj}7} - V^{13}$ | $C^{\text{maj}7} - Em^7 - F^{\text{maj}7} - G^{13}$ | Rolled Strumming, 9th/11th Extensions |

---

### 3. Custom Chord Builder & Storage

Enables users to create and store custom chord progressions persistently for subsequent sessions:

#### Custom Builder Input Fields:

* **Progression Name:** e.g., *"Sad Indie Pop Progression 2026"*.
* **Genre Category:** Dropdown selection or free-text tag.
* **Chord Tokens:**
* Roman Numeral input: `I - vi - IV - V`
* Direct Chord Name input: `Cmaj7 - Am9 - Fadd9 - G13`


* **Rhythm Pattern:** Whole Note (Sustained), Quarter Stabs, Arpeggiated, Strummed.

#### Persistence Engine:

* Persists to `localStorage` under the key `daw_user_custom_chords`.
* Dispatches a custom event `CUSTOM_CHORD_SAVED` to update the Chords Panel list in real time without refreshing the page.

---

### 4. Internet & AI Chord Search Engine

Allows users to look up chord progressions for any song or style query directly through an integrated AI Gateway or web endpoint:

```text
┌────────────────────────────────────────────────────────────────────────────────────────┐
│ 🔍 SEARCH CHORDS VIA INTERNET & AI GATEWAY                                              │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ Search: [ Hotel California - Eagles / Melancholy Lofi Progression...      ] [ Search ] │
├────────────────────────────────────────────────────────────────────────────────────────┤
│ AI / SEARCH RESULTS:                                                                   │
│                                                                                        │
│ 🎵 Song: Hotel California (Eagles) - Key: B Minor                                      │
│    Progression: Bm -> F#7 -> A -> E7 -> G -> D -> Em -> F#7                            │
│    AI Analysis: "Classic Flamenco-influenced rock progression using secondary dominants"│
│                                                                                        │
│    [ 🔊 Listen Preview ]  [ ➕ Save to My Library ]  [ 🎹 Insert to Piano Roll ]      │
└────────────────────────────────────────────────────────────────────────────────────────┘

```

#### Query Flow & Data Extraction:

1. User enters a song title or style description into the search bar.
2. The client submits a request to `/api/v1/ai/search-chords` or dispatches an AI Gateway call with the following Tool Schema:

```json
{
  "type": "function",
  "function": {
    "name": "search_or_generate_chords",
    "description": "Searches for existing chord progressions or generates custom progressions matching a target style.",
    "parameters": {
      "type": "object",
      "properties": {
        "song_or_style_title": { "type": "string" },
        "detected_key": { "type": "string" },
        "chords_list": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "chord_name": { "type": "string" },
              "roman_numeral": { "type": "string" },
              "pitches": { "type": "array", "items": { "type": "integer" } },
              "duration_beats": { "type": "number", "default": 4.0 }
            }
          }
        }
      }
    }
  }
}

```

3. The API/Backend returns structured MIDI pitch numbers ($0 \to 127$).
4. The client inserts the parsed notes directly into the Piano Roll Canvas.

---

### 5. Chord Insertion Engine

When clicking **[ Insert to Timeline / Piano Roll ]**:

#### Insertion Algorithm:

1. Let $Beat_{\text{insert}}$ represent the Playhead timestamp (or clicked beat index on the Piano Roll).
2. For each chord $C_k$ in the progression sequence:
* Compute starting beat position:



$$StartBeat(C_k) = Beat_{\text{insert}} + \sum_{i=0}^{k-1} Duration(C_i)$$

* Convert chord symbols to MIDI pitch arrays (e.g., $C^{\text{maj}7}$ at Key C4, where $\text{Pitch} = 60$):

$$Pitches(C^{\text{maj}7}) = [60, 64, 67, 71] \quad (C4, E4, G4, B4)$$

* Apply chosen Voicing/Inversion rules:
* **Root Position:** Standard ascending pitch order.
* **First Inversion:** Transpose root pitch up $+12$ semitones.
* **Drop-2 Voicing:** Transpose the second-highest note down $-12$ semitones.


* Generate `MIDINote` objects:

```javascript
pitches.forEach(pitch => {
  newNotes.push({
    id: `note_chord_${Date.now()}_${pitch}`,
    pitch: pitch,
    start_beat: startBeat(C_k),
    duration_beats: duration(C_k),
    velocity: 0.8,
    pan: 0.0
  });
});

```

3. **State Mutation & Canvas Redraw:**
* If an active `MIDIItem` is open: Injects `newNotes` into `item.source_data.notes`.
* Dispatches the `DAW_STATE_UPDATED` event to trigger an immediate canvas redraw across both the Piano Roll and Timeline views.



---

## IV. VERIFICATION & TEST PLAN (E2E CHECKLIST)

| Test Item | Action | Expected Result |
| --- | --- | --- |
| **1. Toggle Virtual Keyboard (F2)** | Press **F2** or select **Tools -> Virtual MIDI Keyboard**. | The Floating Keyboard window opens overlaid on the DAW view; pressing **F2** again closes/hides the window. |
| **2. QWERTY Preview Playback** | Press **Z, S, X, D, C, V, G, B** on the computer keyboard. | Virtual keys light up; audio from the active track's synth engine triggers immediately with latency **< 5ms**. |
| **3. Live Recording to Timeline** | Arm track with **[R]** $\rightarrow$ Click **Record + Play** on Transport $\rightarrow$ Play QWERTY keys. | Notes render in real time on the Timeline canvas; clicking **Stop** creates a finalized `MIDIItem` containing the recorded notes. |
| **4. Open Chords Panel** | Select **Insert -> Chords panel** (or press **Shift + K**). | Chords Panel opens displaying progression presets for Pop, Jazz, Epic, and EDM styles. |
| **5. Insert Progression into Piano Roll** | Select *Jazz Neo-Soul* progression $\rightarrow$ Click **[ Insert to Piano Roll ]**. | $Dm^7 - G^7 - C^{\text{maj}7} - A^7$ sequence inserts at the Playhead location with calculated durations. |
| **6. Custom Chord Builder** | Open Custom Builder tab $\rightarrow$ Enter `Cmaj7 - Am9 - Fadd9` $\rightarrow$ Click **Save**. | New progression saves to `localStorage` and appears under the user's custom catalog. |
| **7. AI / Online Chord Lookup** | Open Internet Search tab $\rightarrow$ Query *"Hotel California"* $\rightarrow$ Click **Search**. | Resolves progression ($Bm - F\#7 - A - E7...$) with preview playback and direct insertion actions. |