# TECHNICAL SPECIFICATION: AI MIDI GENERATOR TOOL & PROMPT PRESET SYSTEM

---

## 1. System Overview

The AI Copilot system integrated within the DAW provides two core capabilities:

* **AI Tool Call (Function Calling):** Receives user requests, triggers the automated music generation engine, and returns a list of `Tracks` and `MIDIItems` complying with the JSON schema structure for direct placement onto the DAW Timeline.
* **Prompt Template Engine (Preset Library):** Manages a prompt template directory categorized by genre, mood, and song structure. Upon detecting specific keywords (e.g., *"epic orchestra"*), the engine automatically looks up and expands the raw user prompt into a System Context Standard Prompt containing music theory definitions (pitch range, scales, rhythm patterns, chord progressions, and voicing) prior to dispatching to the LLM.

```text
+-----------------------------------------------------------------------------------+
|                                 USER INTERFACE                                    |
|                                                                                   |
|   [ Prompt Bar / Copilot UI ] <---> [ Prompt Template Preset Manager (CRUD) ]     |
|              |                                      |                             |
|              | Input: "Write 8 bars of             | Matched Preset:             |
|              |  epic orchestra MIDI notes..."       | "Epic Orchestra Intro Spec" |
|              v                                      v                             |
|   +---------------------------------------------------------------------------+   |
|   |                       Prompt Context Expander Engine                      |   |
|   +--------------------------------------------------+------------------------+   |
|                                                      |                            |
+------------------------------------------------------|----------------------------+
                                                       | Extended Prompt Payload
                                                       v
+-----------------------------------------------------------------------------------+
|                                   AI LLM ENGINE                                   |
|                                                                                   |
|   Tool Calling Execution: `generate_multitrack_midi()`                            |
|   Output: Structured JSON Payload (Multi-track 8-bar notes)                       |
+------------------------------------------------------|----------------------------+
                                                       | Validated JSON Output
                                                       v
+-----------------------------------------------------------------------------------+
|                                DAW CLIENT STATE ENGINE                            |
|                                                                                   |
|   - Parses JSON Payload                                                           |
|   - Spawns/Finds Tracks (Strings, Brass, Synth, Percussion, Drums)                 |
|   - Injects `MIDIItem` into `Main Session` / `Section Store`                      |
|   - Re-renders Canvas & Piano Roll UI                                             |
+-----------------------------------------------------------------------------------+

```

---

## 2. Prompt Template & Preset Engine

To eliminate the need to re-type lengthy system instructions, the application provides a Preset Manager stored as JSON format in `LocalStorage` / `IndexedDB` on the client side or within the server database.

### 2.1 Prompt Preset Schema (`preset_schema.json`)

```json
{
  "id": "preset_epic_orchestra_intro",
  "name": "Epic Orchestra Intro (8 Bars)",
  "keywords": ["epic orchestra", "epic orchestral", "hoành tráng", "nhạc phim epic"],
  "category": "Orchestral / Film Score",
  "default_bars": 8,
  "default_bpm": 130,
  "default_scale": "C Minor",
  "system_instruction_template": "You are a professional Epic Orchestral film composer. Create a powerful, dramatic 8-bar intro composition.\nThe required structure to return via the `generate_multitrack_midi` tool consists of 5 tracks:\n1. Strings Ensemble: Plays staccato 16th notes in the low register (C2, G2) driving the rhythm (Ostinato).\n2. Brass Section: Plays the main swelling melodic theme (Horn/Trumpet swells) in the C3-C5 range.\n3. Epic Percussion / Taiko: Hits heavily on beats 1 and 3, featuring a snare roll accent at bars 4 and 8.\n4. Synth Bass/Pad: Holds smooth octave foundation layers (Legato).\n5. Orchestral Drums/Cymbals: Crashes on bar 1 and bar 5.\nEnsure the duration of each track is precisely 8 bars (32 beats).",
  "is_user_defined": false,
  "created_at": "2026-07-23T16:00:00Z"
}

```

### 2.2 Intent Detection & Auto-Expansion Flow

When a user submits a prompt message inside the AI Copilot UI:

```javascript
class PromptTemplateManager {
    constructor() {
        this.presets = [];
        this.loadPresets();
    }

    // Loads preset list from LocalStorage or API
    async loadPresets() {
        const localData = localStorage.getItem('daw_ai_prompt_presets');
        if (localData) {
            this.presets = JSON.parse(localData);
        } else {
            this.presets = DEFAULT_PRESETS; // Developer default fallback templates
            this.savePresets();
        }
    }

    // Matches queries against preset keywords automatically
    matchPreset(userQuery) {
        const queryLower = userQuery.toLowerCase();
        for (const preset of this.presets) {
            const hasKeyword = preset.keywords.some(kw => queryLower.includes(kw.toLowerCase()));
            if (hasKeyword) {
                return preset;
            }
        }
        return null; // Fallback to raw user prompt if no keyword matches
    }

    // CRUD methods for user-defined presets
    saveUserPreset(presetObject) {
        const index = this.presets.findIndex(p => p.id === presetObject.id);
        if (index >= 0) {
            this.presets[index] = presetObject;
        } else {
            this.presets.push(presetObject);
        }
        this.savePresets();
    }

    savePresets() {
        localStorage.setItem('daw_ai_prompt_presets', JSON.stringify(this.presets));
    }
}

```

---

## 3. Function Calling Specification

Defines the Tool/Function schema passed to the LLM API (OpenAI / Local LLM) to enforce strict structured JSON output.

### 3.1 Function Tool Schema (`tools_spec.json`)

```json
{
  "type": "function",
  "function": {
    "name": "generate_multitrack_midi",
    "description": "Generates multi-track MIDI data based on genre, bar duration, and requested instruments list.",
    "parameters": {
      "type": "object",
      "properties": {
        "composition_title": {
          "type": "string",
          "description": "Title of the musical piece (e.g., Epic Orchestra Intro 8-Bars)"
        },
        "bpm": {
          "type": "integer",
          "default": 120
        },
        "total_bars": {
          "type": "integer",
          "default": 8
        },
        "tracks": {
          "type": "array",
          "description": "Array of instrument tracks along with their corresponding MIDI notes",
          "items": {
            "type": "object",
            "properties": {
              "track_name": {
                "type": "string",
                "description": "Track name (e.g., String Ensemble, Epic Brass, Taiko Drums)"
              },
              "instrument_type": {
                "type": "string",
                "enum": ["STRINGS", "BRASS", "SYNTH", "PERCUSSION", "DRUMS"]
              },
              "notes": {
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "pitch": {
                      "type": "integer",
                      "description": "MIDI note pitch from 0 to 127 (e.g., C4 = 60, C3 = 48)"
                    },
                    "start_beat": {
                      "type": "number",
                      "description": "Note start position in beats (from 0.0 to total_bars * 4.0)"
                    },
                    "duration_beats": {
                      "type": "number",
                      "description": "Note length in beats (e.g., Quarter note = 1.0, Eighth note = 0.5)"
                    },
                    "velocity": {
                      "type": "number",
                      "description": "Keypress velocity intensity from 0.0 to 1.0",
                      "default": 0.8
                    }
                  },
                  "required": ["pitch", "start_beat", "duration_beats", "velocity"]
                }
              }
            },
            "required": ["track_name", "instrument_type", "notes"]
          }
        }
      },
      "required": ["composition_title", "bpm", "total_bars", "tracks"]
    }
  }
}

```

---

## 4. AI Response Payload Example

Below is an example JSON payload returned by the AI following the execution of `generate_multitrack_midi` for an 8-bar Epic Orchestra request:

```json
{
  "tool_call": "generate_multitrack_midi",
  "result": {
    "composition_title": "Epic Orchestra Intro",
    "bpm": 130,
    "total_bars": 8,
    "tracks": [
      {
        "track_name": "String Ensemble (Staccato)",
        "instrument_type": "STRINGS",
        "notes": [
          { "pitch": 36, "start_beat": 0.0, "duration_beats": 0.25, "velocity": 0.9 },
          { "pitch": 36, "start_beat": 0.5, "duration_beats": 0.25, "velocity": 0.85 },
          { "pitch": 48, "start_beat": 1.0, "duration_beats": 0.25, "velocity": 0.95 },
          { "pitch": 36, "start_beat": 1.5, "duration_beats": 0.25, "velocity": 0.8 }
        ]
      },
      {
        "track_name": "French Horns & Brass",
        "instrument_type": "BRASS",
        "notes": [
          { "pitch": 60, "start_beat": 0.0, "duration_beats": 2.0, "velocity": 0.95 },
          { "pitch": 63, "start_beat": 2.0, "duration_beats": 2.0, "velocity": 0.9 },
          { "pitch": 67, "start_beat": 4.0, "duration_beats": 4.0, "velocity": 1.0 }
        ]
      },
      {
        "track_name": "Epic Synth Lead",
        "instrument_type": "SYNTH",
        "notes": [
          { "pitch": 72, "start_beat": 4.0, "duration_beats": 1.0, "velocity": 0.85 },
          { "pitch": 75, "start_beat": 5.0, "duration_beats": 1.0, "velocity": 0.85 }
        ]
      },
      {
        "track_name": "Taiko & Percussion",
        "instrument_type": "PERCUSSION",
        "notes": [
          { "pitch": 36, "start_beat": 0.0, "duration_beats": 0.5, "velocity": 1.0 },
          { "pitch": 36, "start_beat": 2.0, "duration_beats": 0.5, "velocity": 0.95 },
          { "pitch": 38, "start_beat": 3.5, "duration_beats": 0.25, "velocity": 0.8 }
        ]
      },
      {
        "track_name": "Orchestral Cymbals",
        "instrument_type": "DRUMS",
        "notes": [
          { "pitch": 49, "start_beat": 0.0, "duration_beats": 4.0, "velocity": 0.9 },
          { "pitch": 49, "start_beat": 16.0, "duration_beats": 4.0, "velocity": 1.0 }
        ]
      }
    ]
  }
}

```

---

## 5. DAW State Ingestion Logic

When the client receives the AI JSON payload, the `ingestAIGeneratedMIDI()` function executes the following pipeline:

1. Spawns or matches corresponding tracks within `main_session`.
2. Generates 8-bar `MIDIItem` objects containing the note lists.
3. Triggers UI canvas and timeline re-renders.

```javascript
function ingestAIGeneratedMIDI(aiPayload, sessionState) {
    const { composition_title, bpm, total_bars, tracks } = aiPayload.result;

    // 1. Update project BPM if specified
    if (bpm) sessionState.metadata.bpm = bpm;

    // 2. Iterate through generated tracks
    tracks.forEach((aiTrack) => {
        // Match existing track or instantiate a new one
        let targetTrack = sessionState.main_session.tracks.find(
            t => t.name.toLowerCase() === aiTrack.track_name.toLowerCase()
        );

        if (!targetTrack) {
            targetTrack = {
                id: `track_ai_${Date.now()}_${Math.random().toString(36).substr(2, 5)}`,
                name: aiTrack.track_name,
                type: "MIDI",
                volume_db: 0.0,
                pan: 0.0,
                mute: false,
                solo: false,
                items: []
            };
            sessionState.main_session.tracks.push(targetTrack);
        }

        // 3. Create 8-bar MIDIItem
        const newMidiItem = {
            id: `item_ai_${Date.now()}_${Math.random().toString(36).substr(2, 5)}`,
            name: `${composition_title} - ${aiTrack.track_name}`,
            type: "MIDI_ITEM",
            start_bar: 0.0, // Placed at Timeline start or active Playhead position
            duration_bars: total_bars,
            clip_start_offset_bars: 0.0,
            source_data: {
                total_buffer_bars: total_bars,
                notes: aiTrack.notes.map((note, index) => ({
                    id: `note_ai_${Date.now()}_${index}`,
                    pitch: note.pitch,
                    start_beat: note.start_beat,
                    duration_beats: note.duration_beats,
                    velocity: note.velocity,
                    pan: 0.0
                }))
            }
        };

        // 4. Append Item to Track
        targetTrack.items.push(newMidiItem);
    });

    // 5. Trigger Timeline & Piano Roll UI Re-render Event
    window.dispatchEvent(new CustomEvent('DAW_STATE_UPDATED', { detail: sessionState }));
}

```

---

## 6. Prompt Preset Manager UI Layout

```text
+-------------------------------------------------------------------------------+
| AI PROMPT PRESET MANAGER                                              [ + New ]|
+-------------------------------------------------------------------------------+
| SEARCH: [ epic orchestra                          ]  FILTER: [ Orchestral  v ]|
|                                                                               |
|  Preset Name                Keywords matched       Default Bars   Actions     |
|  ---------------------------------------------------------------------------  |
|  [★] Epic Orchestra Intro   epic, orchestra, tráng 8 Bars         [Edit][Del] |
|  [ ] Pop Piano Chords       pop, piano, chord      4 Bars         [Edit][Del] |
|  [ ] Cyberpunk Synth Synth  synth, synthwave, 80s  8 Bars         [Edit][Del] |
|                                                                               |
+-------------------------------------------------------------------------------+
| EDIT PRESET: Epic Orchestra Intro                                             |
|                                                                               |
| Keyword Triggers (comma-separated):                                           |
| [ epic orchestra, hoành tráng, nhạc phim epic                               ] |
|                                                                               |
| System Instruction / Music Rules:                                             |
| +---------------------------------------------------------------------------+ |
| | Compose an 8-bar epic orchestral theme featuring Brass, staccato          | |
| | Strings, Taiko percussion, and background Synth bass...                   | |
| +---------------------------------------------------------------------------+ |
|                                                              [ CANCEL ] [ SAVE ]
+-------------------------------------------------------------------------------+

```