3dtours/KGStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial public release.

Browse Source
This commit is contained in:
Xiaohan-Tian
2025-08-11 18:37:21 -07:00
commit de51967b49
186 changed files with 32322 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+26
View File
@@ -0,0 +1,26 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
res/
dist
dist-ssr
*.local
CLAUDE.md
# Editor directories and files
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
*.psd
.vscode/launch.json
Vendored
+19
View File
@@ -0,0 +1,19 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "chrome",
"request": "launch",
"name": "Launch Chrome against localhost",
"url": "http://localhost:5173",
"webRoot": "${workspaceFolder}",
"sourceMaps": true,
"sourceMapPathOverrides": {
"/@fs/*": "${webRoot}/*",
"webpack:///./~/*": "${webRoot}/node_modules/*",
"webpack://?:*/*": "${webRoot}/*"
},
"preLaunchTask": "npm: dev"
}
]
}
.vscode/tasks.json
Vendored
+28
View File
@@ -0,0 +1,28 @@
{
"version": "2.0.0",
"tasks": [
{
"type": "npm",
"script": "dev",
"label": "npm: dev",
"detail": "vite",
"isBackground": true,
"options": {
"shell": {
"executable": "/bin/zsh",
"args": ["-c", "source ~/.nvm/nvm.sh && nvm use 20 && npm run dev"]
}
},
"problemMatcher": {
"pattern": {
"regexp": "^.*$"
},
"background": {
"activeOnStart": true,
"beginsPattern": ".*",
"endsPattern": "ready in .*ms"
}
}
}
]
}
LICENSE
+135
View File
@@ -0,0 +1,135 @@
# License
Copyright 2025 Xiaohan Tian
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this project except in compliance with the License.
You may obtain a copy of the License at:
http://www.apache.org/licenses/LICENSE-2.0
## Additional Terms
1. **No Patent Applications**
You may not use this software or any included assets, in whole or in part, to file or assist in filing any patent application, whether directly or indirectly, in any jurisdiction.
2. **Attribution Requirement**
If you use this project in any public or commercial product, you must provide the following credit in a prominent location (such as an "About" screen or documentation):
> Powered by **K.G.Studio**
> © 2025 Xiaohan Tian
## Standard Terms
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an **"AS IS" BASIS**, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.
---
## Third-Party Components
This project includes resources from the following third-party projects.
Their original license notices **must be retained** in all copies or substantial portions of this project.
---
### 1. midi-js-soundfonts
```
Copyright (C) 2012 Benjamin Gleitzman (gleitz@mit.edu)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
```
---
### 2. FluidR3_GM SoundFont
```
Fluid (R3) SoundFont
Copyright (c) 2000-2002, 2008, 2013 Frank Wen <getfrank@gmail.com>
I hereby release Fluid under the MIT license, as described in COPYING.
Regardless the implications or stipulations of the MIT license, the wishes
and intent of the author are as follows:
1. This soundfont is provided for free, for unlimited use, and can be used
interchangably with other soundfonts for personal or commercial music
composition.
2. In regards to bundling Fluid with software or hardware for commercial
purposes (freeware with paid features, or paid completely), please
follow these guidelines so that Fluid will not be construed as a commodity.
-Your design must have openness for anyone to be able to load whatever
soundfont they want to use, and have full access to all the instruments
in the soundfont.
-You cannot lock access to Fluid by having people have to pay in order
to get access to more instrument banks or instrument banks associated
with specific tracks in the sequencer.
-If you are confused, please contact me.
3. If you use part of Fluid to make a new bank for public domain, document it
and credit me.
4. Realistically, I cannot police or enforce what actually happens to Fluid,
although I believe there are honorable people who have respected my wishes
to keep the spirit of what I wanted the work to be, and thank all who have
supported and used Fluid in their musical endeavors.
This package, of course, is the original Release 3 of Fluid.
Fluid was constructed in part from samples found in the public domain that I
edited/cleaned/remixed/programmed and largely from recordings of my own and
in conjunction with the people below who helped along the way:
Suren M. Seron
Scott Hanan
Steve Aupperle
Chris Gillman
Alex Taubr
Chris Prola
Andrew Klenk
Winfried Hubbe
Dylan
Tim
Gort
Uros Katic
Ethan Winer (http://www.ethanwiner.com)
I'd also like to thank Alexander Ryan, Dan Reagan, and Thomas Hammer for
hosting the project for so many years. Without them, distribution would have
been impossible.
And also Tobe Smithe, for bringing this to more people through Ubuntu.
It's obviously been a few years since the project, but its nice to see that
people are still enjoying my work and getting good use out of it. As always,
I'd like to hear some work done with Fluid so email me, or just email me to
say hello and tell me what is going on in the computer musician world.
Who knows, maybe I'll kick start this project again? ;)
```
---
### 3. System Prompt Structure (based on Cline)
Portions of the system prompt structure are derived from the Cline project,
licensed under the Apache License, Version 2.0.
Original project: [Cline](https://github.com/cline/cline)
Modifications: Substantial textual and functional changes have been made; only the general content structure remains.
---
Apache License Version 2.0, January 2004: http://www.apache.org/licenses/LICENSE-2.0
README.md
+154
View File
@@ -0,0 +1,154 @@
<div align="center">
<img src="./public/logo.png" alt="K.G.Studio Logo" width="160" />
</div>
## K.G.Studio — Browser-based DAW (Digital Audio Workstation)
K.G.Studio is a lightweight, modern DAW that runs entirely in the browser with **K.G.Studio Musician Assistant** at its core. It features realistic instrument playback via Tone.js samplers, a piano‑roll editor, track and region management with full undo/redo, project persistence to IndexedDB, a configurable settings panel, and an integrated AI assistant with tool execution.
**K.G.Studio Musician Assistant** is an AI assistance agent for harmony, arrangement, and note editing — but not full auto‑composition.
### Highlights
- **K.G.Studio Musician Assistant**: Chat with the LLM‑powered K.G.Studio Musician Assistant AI Agent; it can automatically execute tools to make music edits.
- **Multiple LLM providers**: OpenAI, Claude (via OpenRouter), Gemini (via OpenRouter), or OpenAI‑compatible (e.g., Ollama, OpenRouter).
- **Track & Region editing**: Add/reorder tracks, create/move/resize regions, copy/paste regions, and more.
- **Piano roll**: Create and edit notes with snapping/quantization support.
- **Real instruments**: Tone.js‑based Sampler with high‑quality FluidR3 soundfonts.
- **Undo/Redo everywhere**: Command pattern for tracks, regions, notes, and project properties
- **Persistence with privacy**: Save/load projects and configuration in your browser’s IndexedDB (on your device).
- **Export/Import**: Export your project as a MIDI file, or import a MIDI file into your project.
- **Settings**: LLM provider, AI agent custom instructions, app behavior, and more.
For a deeper technical overview, see `res/overview.md`.
## Getting Started
### Prerequisites
- Node.js 20.19.3+
### Install & Run
```bash
git clone https://github.com/Xiaohan-Tian/KGSP {your-local-path}
cd {your-local-path}
npm install
npm run dev
```
## Configuration
K.G.Studio loads defaults from `./public/config.json` (with an internal fallback) and persists user edits in the browser via `ConfigManager` + IndexedDB. IndexedDB is your browser’s own local database stored on your device; it does not leave your machine and is cleared if you clear this site’s data. Modify settings via the in‑app Settings panel.
- **General**
- LLM provider: OpenAI, or OpenAI‑compatible
- API keys and models for the selected provider
- OpenAI‑compatible base URL (for self‑hosted gateways)
- Soundfont base URL (CDN for instrument samples)
- **Behavior**
- Chatbox default open on startup
- **Templates**
- Custom instructions used by the AI assistant
### Connectivity & Privacy
- K.G.Studio is fully client‑side. No first‑party servers are required to run the app.
- All projects, configuration, and UI state are stored in your browser’s IndexedDB (on your device).
- Network access is only used for:
- Downloading instrument sound samples from the configured soundfont CDN
- Communicating with the LLM provider you select (e.g., OpenAI or OpenAI‑compatible services)
- Outside of the two cases above, the app functions locally. If you block those endpoints, the app still loads; instrument playback and AI features will not function until network access is restored.
- For security, when running from a non‑local host we do not persist your API key in IndexedDB (to reduce XSS risk). You’ll be prompted to enter it each time you start K.G.Studio.
## Using the App
1. Start the app and a default project will load. A default “Melody” MIDI track is ensured on load.
2. Add tracks, rename and reorder them. Click or drag in the track grid to create and arrange regions.
3. Add a region by double‑clicking or holding Ctrl/Cmd and clicking. Drag the edges to resize a region; drag the center to move it. Use Ctrl/Cmd+C and Ctrl/Cmd+V to copy/paste regions.
4. Open a MIDI region to edit notes in the Piano Roll window (click the pencil icon on the region’s top‑left corner, or click the Piano icon on the toolbar).
5. Double‑click or hold Ctrl/Cmd and click to create a note. Drag the edges to change its length, or drag the center to move it. Use box selection or hold Shift to multi‑select.
6. Quantize note length/position using the Quantize dropdowns at the top‑right of the piano roll.
7. Change instruments via the Instrument Selection panel; samples are downloaded on demand.
8. Save projects to your browser’s IndexedDB and load by name from the toolbar. Export your project as a MIDI file, or import a MIDI file into your project.
## Keyboard Shortcuts (defaults)
- Main
- Play/Pause: Space
- Undo / Redo: Ctrl/Cmd+Z / Ctrl/Cmd+Shift+Z
- Copy / Cut / Paste: Ctrl/Cmd+C / Ctrl/Cmd+X / Ctrl/Cmd+V
- Save: Ctrl/Cmd+S
- Hold to create region: Ctrl/Cmd
- Piano Roll
- Tools: Select (Q), Pencil (W)
- Hold to create note: Ctrl/Cmd
- Snap: 1(None), 2(1/4), 3(1/8), 4(1/16)
- Quantize Position: 5(1/4), 6(1/8), 7(1/16)
- Quantize Length: 8(1/4), 9(1/8), 0(1/16)
## AI Assistant
### Quick Start
- First, select an LLM provider in **Settings ⚙️ → General → LLM Provider**. If you are using OpenAI (e.g., GPT‑4o), get your API key from [**OpenAI**](https://platform.openai.com/account/api-keys) and paste it in **OpenAI → Key**.
- You can find the K.G.Studio Musician Assistant chatbox on the right. If you don't see it, you can click the Chat 🗨️ button on the toolbar.
- Type your message in the chatbox; press Enter to send, Shift+Enter to insert a new line.
- The agent will automatically process your request and invoke tools to make modifications scoped to the selected region. It may take one or more turns to complete a task.
- Note that the AI could make mistakes, so you should always check the result and make adjustments if necessary. You can also use undo/redo to revert the changes.
- Click the "+" button or `/clear` command to clear the chat history.
### Configuring Your LLM Provider
Navigate to **Settings ⚙️ → General → LLM Provider**. Depending on your chosen provider, you will need to supply the appropriate API Key and, if applicable, a custom base URL (for non-official OpenAI-compatible services such as Ollama, OpenRouter, etc.).
Note: due to CORS limitations with some providers, Google Gemini and Anthropic Claude are currently supported via OpenRouter only.
### Using OpenAI models
1. Obtain an OpenAI API Key from [**OpenAI**](https://platform.openai.com/account/api-keys). You may need to create an account and add a payment method to generate an API Key.
2. In **Settings ⚙️ → General → LLM Provider**, select **OpenAI** as your provider.
3. Enter your API Key in **OpenAI → Key**.
4. Select your preferred model from the **OpenAI → Model** dropdown. For optimal performance, we recommend `gpt‑4o`. (`gpt‑5` is still being evaluated.)
5. Optionally, choose whether to enable Flex Mode in **OpenAI → Flex Mode**. Flex Mode offers discounted pricing, but may result in slower response times or server-side errors.
### Using OpenRouter
OpenRouter is a platform that provides unified access to a wide range of language models—including free options—from various providers. This makes it easy to experiment and find the model that best suits your needs.
1. Obtain an API Key from [**OpenRouter**](https://openrouter.ai/keys). Registration is required; for paid models, a payment method may also be necessary.
2. In **Settings ⚙️ → General → LLM Provider**, select **OpenAI Compatible** as your provider.
3. Enter your API Key in **OpenAI Compatible Server → Key**.
4. Browse available models on the [**OpenRouter Models Page**](https://openrouter.ai/models). Use the "Prompt Pricing" filter to identify free models.
**Note:** Each model provider may have different data retention and privacy policies. Please review these policies before use.
5. Enter your chosen model name in **OpenAI Compatible Server → Model**. Recommended model series include:
- `Google: gemini-2.5-pro`
- `Anthropic: claude-4-sonnet`
- `DeepSeek: deepseek-r1` (free: `deepseek/deepseek-r1-0528:free`)
- `DeepSeek: deepseek-v3` (free: `deepseek/deepseek-chat-v3-0324:free`)
- `Qwen: qwen3-235b-a22b` (free: `qwen/qwen3-235b-a22b:free`)
6. Input the base URL `https://openrouter.ai/api/v1/chat/completions` **OpenAI Compatible Server → Base URL**.
### About the agent and LLM providers
Currently, based on our evaluation, OpenAI’s open‑source models (`gpt‑oss‑20b` and `gpt‑oss‑120b`) are not yet compatible with the current agent implementation; support is planned.
For security, when using K.G.Studio from a non‑local host, API keys are not persisted in IndexedDB; you will need to input your API key each time you start K.G.Studio.
K.G.Studio does not provide or host any of the models listed above, nor is it affiliated with any model provider. All data is stored locally on your device; K.G.Studio does not collect or transmit your data. You are solely responsible for any data you provide to third‑party model providers.
## Upcoming Features
- [ ] More instruments
- [ ] Support track control automations (e.g. sustain, volume, pan, etc.)
- [ ] Support MIDI control events (e.g. CC, pitch bend, etc.)
- [ ] Support WAV audio tracks
- [ ] MCP Support
- [ ] Add support for OpenAI's open source models (`gpt-oss-20b` and `gpt-oss-120b`)
- [ ] Automatically compact conversations when the context window runs low on space
## License
Licensed under the Apache License, Version 2.0, with additional terms (see `LICENSE`):
- No patent applications using this software or assets
- Attribution required when used in public/commercial products (“Powered by K.G.Studio”)
Third‑party notices (e.g., FluidR3_GM SoundFont, midi‑js‑soundfonts, and prompt structure notes) are included in `LICENSE`.
docs/USER_GUIDE.md
+179
View File
@@ -0,0 +1,179 @@
# K.G.Studio User Guide
A lightweight, browser‑based DAW with an AI Agent "K.G.Studio Musician Assistant". This guide walks you through setup, the interface, and everyday workflows.
## 1. Introduction
- K.G.Studio runs entirely in your browser. It provides track and region editing, a piano‑roll editor, realistic instrument playback (Tone.js samplers with FluidR3 soundfonts), robust undo/redo, and project persistence.
- K.G.Studio Musician Assistant can respond to natural language prompts and execute edit tools on your behalf.
## 2. System Requirements
- Tested on Chrome, Firefox, and Safari. It should work on other modern browsers, but those are not officially tested yet.
- macOS, Windows, or Linux.
- Network access is only needed for downloading instrument sound samples and contacting your selected LLM provider.
## 3. Quick Start
- **Use the hosted app: [K.G.Studio (kgaudiolab.github.io/kgstudio)](https://kgaudiolab.github.io/kgstudio)**
- Or clone and run locally:
```bash
# Make sure you have Node.js >= 20.19.3 installed
# Clone the repository
git clone https://github.com/KGAudioLab/KGStudio {your-local-path}
cd {your-local-path}
# Install dependencies
npm install
# Run the development server
npm run dev
```
- First run:
- A welcome message will be shown on the K.G.Studio Musician Assistant chatbox, please follow the instructions to configure the LLM provider.
- Your data is stored in your browser’s IndexedDB (on your device).
## 4. Data, Privacy, and Connectivity
- IndexedDB is your browser’s local database for this site; it never leaves your machine and is cleared if you clear the site’s data.
- Per‑origin storage: IndexedDB is not shared across different hosts/origins, across different browsers, or separate browser profiles. To move or share your work, use Export (KGStudio JSON) and Import on the destination.
- K.G.Studio is fully client‑side. It connects to the network only to:
- Download instrument soundfonts from the configured CDN
- Communicate with your chosen LLM provider (OpenAI or OpenAI‑compatible)
- API keys are not persisted when running from a non‑local host (to reduce XSS risk). You’ll be prompted to re‑enter them on each start in that scenario.
- Important: While K.G.Studio does not collect your data, different LLM providers may have different data‑retention policies. Review the policy of the provider you select before use.
## 5. User Interface Tour
### Toolbar
- Left
- Logo
- Project name (click to rename)
- Center (exact order)
- New, Load, Save, Export (dropdown), Import
- Undo, Redo
- Pointer tool, Pencil tool
- Copy, Paste, Delete
- Back to beginning, Play/Pause
- Piano button (open piano roll for the active/selected region)
- Right
- Transport readouts: current time | BPM | time signature | key signature
- Click time to set max bars, BPM to change BPM, time signature to change signature; key signature opens a dropdown.
- Settings (gear)
- Chat (speech bubble) — toggles the AI chatbox on the right
### Track Area
- Track info column
- Solo (S), Mute (M), volume, instrument button (piano icon)
- Settings button (to the right of the instrument button) with a Delete Track option
- Track grid
- Regions display as blocks with a header; the small pencil on the top‑left of a region opens its Piano Roll.
### Instrument Selection Panel
- Appears automatically on load/first track creation, or when clicking a track’s instrument button (to the right of “M”).
- Choose instrument groups and individual instruments; a large preview is shown.
- Stays open until you close it (X) or click the same track’s instrument button again.
### Chatbox (AI Assistant)
- Docked on the right; toggle from the toolbar.
- Press Enter to send; Shift+Enter for a new line.
### Status Bar and Loading Overlay
- Status messages appear at the bottom.
- A global loading overlay shows while soundfonts are downloading.
### Piano Roll Window
- Header: close (X) on the left; title (click to rename); tools (Select, Pencil); menus on the right: NO SNAP, Qua. Pos. (quantize start), Qua. Len. (quantize length).
- ESC closes the piano roll.
## 6. Projects
- New, Save (to IndexedDB), Load (from IndexedDB by name).
- Export: KGStudio JSON, MIDI.
- Import: KGStudio JSON (replaces current project state), MIDI (appends tracks/regions into the current project).
- Legacy compatibility: projects are upgraded to the latest structure on load.
## 7. Tracks
- Add, rename, and reorder tracks.
- Change instrument (instrument button in the track info row). Real instruments are powered by Tone.Sampler and FluidR3 soundfonts.
- Solo/Mute/Volume controls for quick mixing.
- Track settings (button to the right of the instrument) provides Delete Track.
## 8. Regions
- Create regions
- Pointer tool: double‑click; or hold Ctrl/Cmd and click
- Pencil tool: single‑click to create
- Move/resize by dragging the region or its edges; copy/paste with toolbar buttons or shortcuts.
- Delete selected regions from the toolbar delete button.
- Open Piano Roll via the region’s small pencil on the top‑left.
## 9. Piano Roll (MIDI Editing)
- Tools: Select vs Pencil.
- Create notes: double‑click or Ctrl/Cmd+click (Select); single‑click (Pencil).
- Move/resize notes by dragging; box‑select or Shift‑click for multi‑selection.
- NO SNAP menu controls snapping for create/move/resize.
- Quantize using Qua. Pos. (start) and Qua. Len. (length) menus.
- Close with X or ESC.
## 10. Playback and Transport
- Back to beginning; Play/Pause.
- Set playhead by clicking bar numbers in the main grid; in Piano Roll, click the header timeline (respects snapping).
- Edit time/BPM/time signature/key signature via the toolbar readouts.
## 11. Undo/Redo and Clipboard
- Undo/Redo available for tracks, regions, notes, and project properties.
- Copy/Paste works for regions in the main grid and notes in the piano roll.
- Use toolbar buttons or keyboard shortcuts.
## 12. Instruments and Soundfonts
- Instruments are organized by groups (e.g., Piano & Keyboards, Strings, Brass, Woodwind, Percussion Kit, Synthesizer).
- Playback uses high‑quality FluidR3 soundfonts via Tone.Sampler.
- If loading stalls, the overlay will time out; refresh to retry downloading.
## 13. Settings
- General: LLM provider (OpenAI or OpenAI‑compatible), API key, model, soundfont base URL.
- Behavior: chatbox default open at startup.
- Templates: custom instructions for the AI.
- Settings persist in IndexedDB (local); API keys are not persisted on non‑local hosts.
## 14. AI Agent "K.G.Studio Musician Assistant"
- Open the chat (toolbar). Describe goals in natural language (e.g., “Can you help me write a 4‑bar chord progression for the melody?”).
- The agent executes tools to edit your project. Actions are typically scoped to the selected region.
- Slash‑commands:
- `/clear` — clear chat history
- `/welcome` — show the welcome message
- Providers: OpenAI or OpenAI‑compatible (e.g., OpenRouter). Due to CORS, some providers are supported via OpenRouter.
- Reminder: LLM providers differ in data retention and usage policies. Check your provider’s policy and configure keys/models accordingly.
## 15. Keyboard Shortcuts (defaults)
- Global
- Play/Pause: Space
- Undo / Redo: Ctrl/Cmd+Z / Ctrl/Cmd+Shift+Z
- Copy / Cut / Paste: Ctrl/Cmd+C / Ctrl/Cmd+X / Ctrl/Cmd+V
- Save: Ctrl/Cmd+S
- Hold to create region: Ctrl/Cmd
- Piano Roll
- Tools: Select (Q), Pencil (W)
- Hold to create note: Ctrl/Cmd
- Snap: 1(None), 2(1/4), 3(1/8), 4(1/16)
- Quantize Position: 5(1/4), 6(1/8), 7(1/16)
- Quantize Length: 8(1/4), 9(1/8), 0(1/16)
## 16. Troubleshooting
- No sound: ensure soundfonts can download; check network; try refreshing.
- Chat/LLM errors: verify provider, base URL (if compatible), API key, and model name; check CORS if using third‑party gateways.
- MIDI import: only valid `.mid/.midi` files; malformed files will show an error.
- Performance: close unused panels; reduce concurrent soundfont loads.
## 17. FAQ
- Where are my projects saved? In your browser’s IndexedDB on your device.
- Can I use the app offline? Yes, core editing works offline; instruments and AI need network when first used.
- Does the app send my projects to a server? No. Only your LLM requests go to your chosen provider.
## 18. Glossary
- Project, Track, Region, Note, Snapping, Quantize, General MIDI (GM), IndexedDB, Soundfont, LLM.
## 19. Known Limitations
- Limited note editing (no velocity editing yet)
- No region duplication shortcut
- Export UI present; audio export not implemented
- No audio recording yet; limited effects processing
- No project browser UI (load by name)
## 20. Credits and Licenses
- Licensed under Apache 2.0 with additional terms (see `LICENSE`).
- Third‑party attributions: FluidR3_GM soundfont, midi‑js‑soundfonts, and prompt structure notes.
eslint.config.js
+27
View File
@@ -0,0 +1,27 @@
import js from '@eslint/js'
import globals from 'globals'
import reactHooks from 'eslint-plugin-react-hooks'
import reactRefresh from 'eslint-plugin-react-refresh'
import tseslint from 'typescript-eslint'
import { globalIgnores } from 'eslint/config'
export default tseslint.config([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
js.configs.recommended,
tseslint.configs.recommended,
reactHooks.configs['recommended-latest'],
reactRefresh.configs.vite,
],
languageOptions: {
ecmaVersion: 2020,
globals: globals.browser,
},
rules: {
'@typescript-eslint/no-unused-vars': 'warn', // Downgrade from error to warning
'no-unused-vars': 'warn' // Also set the base rule to warn
}
},
])
index.html
+16
View File
@@ -0,0 +1,16 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/png" href="/favicon-96x96.png" sizes="96x96" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<link rel="shortcut icon" href="/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>K.G.Studio</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
package-lock.json
Generated
+5231
View File
File diff suppressed because it is too large Load Diff
package.json
+39
View File
@@ -0,0 +1,39 @@
{
"name": "kgsp",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"lint": "eslint .",
"preview": "vite preview"
},
"dependencies": {
"class-transformer": "^0.5.1",
"idb": "^8.0.3",
"react": "^19.1.0",
"react-dom": "^19.1.0",
"react-icons": "^5.5.0",
"react-markdown": "^10.1.0",
"react-syntax-highlighter": "^15.6.1",
"reflect-metadata": "^0.2.2",
"remark-gfm": "^4.0.1",
"tone": "^15.1.22",
"zustand": "^5.0.6"
},
"devDependencies": {
"@eslint/js": "^9.29.0",
"@types/react": "^19.1.8",
"@types/react-dom": "^19.1.6",
"@types/react-syntax-highlighter": "^15.5.13",
"@vitejs/plugin-react": "^4.5.2",
"eslint": "^9.29.0",
"eslint-plugin-react-hooks": "^5.2.0",
"eslint-plugin-react-refresh": "^0.4.20",
"globals": "^16.2.0",
"typescript": "~5.8.3",
"typescript-eslint": "^8.34.1",
"vite": "^7.0.0"
}
}
public/apple-touch-icon.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 35 KiB

public/chat/custom_instructions_gpt-4o.md
+1
View File
@@ -0,0 +1 @@
When making chord progression for existing melody, you should be very careful to make sure the chords can match the notes in the melody at any given time, they should sound harmony.
public/chat/custom_instructions_qwen3-a3b-30b.md
+1
View File
@@ -0,0 +1 @@
When making chord progression for existing melody, you should be very careful to make sure the chords can match the notes in the melody at any given time, they should sound harmony.
public/chat/error_no_openai_compatible_base_url.md
+9
View File
@@ -0,0 +1,9 @@
### OpenAI Compatible Base URL Required
You selected OpenAI Compatible as your LLM Provider, but no Base URL is configured.
How to fix:
- Go to **Settings ⚙️ → General → OpenAI Compatible Server**
- Enter the **Base URL** for your provider (e.g., `https://openrouter.ai/api/v1/chat/completions` for OpenRouter, `http://localhost:11434/api/chat` for Ollama, etc.)
After updating your settings, try your request again.
public/chat/error_no_openai_compatible_model.md
+9
View File
@@ -0,0 +1,9 @@
### OpenAI Compatible Model Required
You selected OpenAI Compatible as your LLM Provider, but no Model is configured.
How to fix:
- Go to **Settings ⚙️ → General → OpenAI Compatible Server**
- Enter a **Model** (e.g., `qwen/qwen3-235b-a22b:free`, etc.)
After updating your settings, try your request again.
public/chat/error_no_openai_key.md
+9
View File
@@ -0,0 +1,9 @@
### OpenAI API Key Required
You selected OpenAI as your LLM Provider, but no API key is configured.
How to fix:
- Go to **Settings ⚙️ → General → OpenAI**
- Paste your API Key into **Key**
After updating your settings, try your request again.
public/chat/error_no_selected_region.md
+10
View File
@@ -0,0 +1,10 @@
### No Region Selected
To proceed, please select a MIDI region in the track view or open a region in the piano roll.
Tips:
- Double-click in a track to create a new MIDI region, then select it.
- Click an existing region to select it.
- Use the Piano button to open the active region in the piano roll.
After selecting a region, try your request again.
public/chat/welcome_again.md
+10
View File
@@ -0,0 +1,10 @@
## Welcome Back
Great to see you again! Your LLM provider appears to be configured. You can start chatting right away.
Tips:
- Use `/clear` anytime to reset the chat.
- Ask me to create tracks, regions, or MIDI notes, and I'll help orchestrate tool actions.
- Type `/welcome` to view this message again.
Have fun composing with K.G.Studio Musician Assistant.
public/chat/welcome_new.md
+52
View File
@@ -0,0 +1,52 @@
## Welcome
Welcome to **K.G.Studio Musician Assistant**—your AI-powered agent for music composition and arrangement. Harnessing advanced language models, I am here to help you create, arrange, and refine your musical ideas with ease and intelligence.
To get started, you will need to configure an LLM (Large Language Model) Provider in the application settings. Please follow the instructions below to ensure proper setup:
---
### Configuring Your LLM Provider
Navigate to **Settings ⚙️ → General → LLM Provider**. Depending on your chosen provider, you will need to supply the appropriate API Key and, if applicable, a custom base URL (for non-official OpenAI-compatible services such as Ollama, OpenRouter, etc.).
---
### Using the OpenAI GPT Model Series
1. Obtain an OpenAI API Key from [**OpenAI**](https://platform.openai.com/account/api-keys). You may need to create an account and add a payment method to generate an API Key.
2. In **Settings ⚙️ → General → LLM Provider**, select **OpenAI** as your provider.
3. Enter your API Key in **OpenAI → Key**.
4. Select your preferred model from the **OpenAI → Model** dropdown. For optimal performance, we recommend `gpt-5` or `gpt-4o`.
5. Optionally, choose whether to enable Flex Mode in **OpenAI → Flex Mode**. Flex Mode offers discounted pricing, but may result in slower response times.
---
### Using OpenRouter
OpenRouter is a platform that provides unified access to a wide range of language models—including free options—from various providers. This makes it easy to experiment and find the model that best suits your needs.
1. Obtain an API Key from [**OpenRouter**](https://openrouter.ai/keys). Registration is required; for paid models, a payment method may also be necessary.
2. In **Settings ⚙️ → General → LLM Provider**, select **OpenAI Compatible** as your provider.
3. Enter your API Key in **OpenAI Compatible Server → Key**.
4. Browse available models on the [**OpenRouter Models Page**](https://openrouter.ai/models). Use the "Prompt Pricing" filter to identify free models.
**Note:** Each model provider may have different data retention and privacy policies. Please review these policies before use.
5. Enter your chosen model name in **OpenAI Compatible Server → Model**. Recommended model series include:
- `Google: gemini-2.5-pro`
- `Anthropic: claude-4-sonnet`
- `DeepSeek: deepseek-r1` (free: `deepseek/deepseek-r1-0528:free`)
- `DeepSeek: deepseek-v3` (free: `deepseek/deepseek-chat-v3-0324:free`)
- `Qwen: qwen3-235b-a22b` (free: `qwen/qwen3-235b-a22b:free`)
6. Input the base URL `https://openrouter.ai/api/v1/chat/completions` of the OpenAI Compatible Server in **OpenAI Compatible Server → Base URL**.
---
### Note
Due to security reasons, if you are using K.G.Studio from a non-local hosted environment, we won't persist your API Key in your browser's IndexedDB in order to prevent potential XSS attack, you will need to input your API Key every time you start K.G.Studio.
### Disclaimer
K.G.Studio does not provide or host any of the models listed above, nor is it affiliated with any model provider. All data is stored locally on your device; K.G.Studio does not collect or transmit your data. You are solely responsible for any data you provide to third-party model providers.
We hope you enjoy using K.G.Studio Musician Assistant!
public/config.json
+60
View File
@@ -0,0 +1,60 @@
{
"general": {
"language": "en_us",
"llm_provider": "openai",
"openai": {
"api_key": "",
"flex": false,
"model": "gpt-4o"
},
"gemini": {
"api_key": "",
"model": "gemini-2.5-flash"
},
"claude": {
"api_key": "",
"model": "claude-sonnet-4-0"
},
"openai_compatible": {
"api_key": "",
"base_url": "",
"model": ""
},
"soundfont": {
"base_url": "https://cdn.jsdelivr.net/npm/soundfont-for-samplers/FluidR3_GM/"
}
},
"hotkeys": {
"main": {
"hold_to_create_region": "ctrl",
"play": "space",
"undo": "ctrl+z",
"redo": "ctrl+shift+z",
"copy": "ctrl+c",
"cut": "ctrl+x",
"paste": "ctrl+v",
"save": "ctrl+s"
},
"piano_roll": {
"select": "q",
"pencil": "w",
"hold_to_create_note": "ctrl",
"snap_none": "1",
"snap_1_4": "2",
"snap_1_8": "3",
"snap_1_16": "4",
"qua_pos_1_4": "5",
"qua_pos_1_8": "6",
"qua_pos_1_16": "7",
"qua_len_1_4": "8",
"qua_len_1_8": "9",
"qua_len_1_16": "0"
}
},
"chatbox": {
"default_open": true
},
"templates": {
"custom_instructions": ""
}
}
public/favicon-96x96.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.8 KiB

public/favicon.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

public/favicon.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 1.7 MiB

public/logo-kgaudiolab.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 137 KiB

public/logo.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 316 KiB

public/prompts/README.md
+15
View File
@@ -0,0 +1,15 @@
# System Prompt Attribution & License
This system prompt was created for **K.G.Studio**.
## License
The system prompt is licensed under the **Apache License, Version 2.0**.
You may not use this prompt, in whole or in part, to file or assist in filing any patent application.
Full license text can be found in `LICENSE` under the root directory.
## Attribution
Portions of the **structure and organization** of this prompt are derived from the **Cline** project, which is licensed under the Apache License, Version 2.0.
Original project: [Cline](https://github.com/cline/cline)
Modifications: Substantial textual and functional changes have been made; only the general content structure remains.
public/prompts/system.md
+448
View File
@@ -0,0 +1,448 @@
You are K.G.Studio Musician Assistant Agent, a highly skilled musician with extensive knowledge in music theory, composition, and production.
As an AI music agent, you possess comprehensive musical knowledge spanning:
- Classical harmony and counterpoint
- Jazz theory and improvisation
- Popular music structures and progressions
- World music traditions and scales
- Contemporary production techniques
- Orchestration and arrangement principles
You should leverage this extensive musical training to provide creative, musically intelligent solutions. While the MUSIC THEORY REFERENCE section provides quick reference material, your primary strength lies in your deep understanding of musical relationships, stylistic conventions, and compositional techniques. Apply this knowledge creatively to fulfill user requests with musical sophistication and artistic sensibility.
====
TOOL USE
You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. Note: The `attempt_completion` tool is an exception - after using it, the user's next message will be a new request rather than a tool result.
# Tool Use Formatting
Tool use is formatted using XML-style tags. The tool name is enclosed in opening and closing tags, and each parameter is similarly enclosed within its own set of tags. Here's the structure:
<tool_name>
<parameter1_name>value1</parameter1_name>
<parameter2_name>value2</parameter2_name>
...
</tool_name>
For example:
<read_music>
<start_beat>0</start_beat>
<length>8</length>
</read_music>
Always adhere to this format for the tool use to ensure proper parsing and execution.
# Tools
## read_music
Description: Read a given part of the music. The output is the selected part of the music in ABC notation. If there are multiple tracks, this tool will read all tracks with each track as a separate ABC notation section.
Parameters:
- start_beat: (required) The start beat of the region to read.
- length: (optional) The length of the region to read. If you want to read the entire music, you can omit this parameter.
Usage:
<read_music>
<start_beat>start from beat</start_beat>
<length>length of the region to read (optional)</length>
</read_music>
## remove_notes
Description: Remove notes from the given range in the current region.
Parameters:
- start_beat: (required) The start beat of the range to remove notes from.
- end_beat: (required) The end beat of the range to remove notes from.
Usage:
<remove_notes>
<start_beat>start from beat</start_beat>
<end_beat>end at beat</end_beat>
</remove_notes>
## add_notes
Description: Add notes to the current region.
Parameters:
- notes: (required) The notes to add. Each note is an XML object with the following properties:
- pitch (required): The pitch of the note.
- start_beat (required): The start beat of the note. The start beat is the absolute beat number of the note, not the relative beat number to the current region. for example, if you want to add a note at beat 6, regardless the current region starts from beat 0 or beat 4 or other beat, you should set the start_beat to 6.
- length (required): The length of the note.
Usage:
<add_notes>
<notes>
<note>
<pitch>pitch of the first note you want to add, e.g. C4</pitch>
<start_beat>start beat of the first note you want to add</start_beat>
<length>length of the first note you want to add</length>
</note>
<note>
<pitch>pitch of the second note you want to add, e.g. E4</pitch>
<start_beat>start beat of the second note you want to add</start_beat>
<length>length of the second note you want to add</length>
</note>
...
</notes>
</add_notes>
## attempt_completion
Description: After each tool use, the user will respond with the result of that tool use, i.e. if it succeeded or failed, along with any reasons for failure. Once you've received the results of tool uses and can confirm that the task is complete, use this tool to present the result of your work to the user.
IMPORTANT NOTE: This tool CANNOT be used until you've confirmed from the user that any previous tool uses were successful. Failure to do so will result in code corruption and system failure. Before using this tool, you must ask yourself in <thinking></thinking> tags if you've confirmed from the user that any previous tool uses were successful. If not, then DO NOT use this tool.
Parameters:
- comment: (required) The result of the task. Formulate this result in a way that is final and does not require further input from the user. Don't end your result with questions or offers for further assistance.
Usage:
<attempt_completion>
<comment>Your final result or comment here</comment>
</attempt_completion>
# Tool Use Examples
## Example 1: Requesting to read a part of the music from beat 0 to beat 8
<read_music>
<start_beat>0</start_beat>
<length>8</length>
</read_music>
## Example 2: Requesting to remove notes from the current region from beat 0 to beat 4
<remove_notes>
<start_beat>0</start_beat>
<end_beat>4</end_beat>
</remove_notes>
## Example 3: Requesting to add notes C4, D4, G4, E4 to the current region, starting at beat 0 and each one lasting 1 beat.
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>1</length>
</note>
<note>
<pitch>D4</pitch>
<start_beat>1</start_beat>
<length>1</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>2</start_beat>
<length>1</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>3</start_beat>
<length>1</length>
</note>
</notes>
</add_notes>
## Example 4: Requesting to add a chord containing C4, E4, G4 to the current region, starting at beat 0 and lasting 2 beats.
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
</notes>
</add_notes>
## Example 5: Requesting to complete current task with a final comment
<attempt_completion>
<comment>Completed the I-V-vi-IV chord progression in C major, each chord lasting 2 beats</comment>
</attempt_completion>
# Tool Use Guidelines
1. In <thinking> tags, assess what information you already have and what information you need to proceed with the task.
2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. It's critical that you think about each available tool and use the one that best fits the current step in the task.
3. If multiple actions are needed, use one tool at a time per message to accomplish the task iteratively, with each tool use being informed by the result of the previous tool use. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.
4. Formulate your tool use using the XML format specified for each tool.
5. After each tool use, the user will respond with the result of that tool use. This result will provide you with the necessary information to continue your task or make further decisions. This response may include:
- Information about whether the tool succeeded or failed, along with any reasons for failure.
- Music pieces in ABC notation if you have used the read_music tool.
- Any other relevant feedback or information related to the tool use.
Note: After using the `attempt_completion` tool, the user's response will be a new request rather than a tool result, as this tool marks the end of the current task.
6. ALWAYS wait for user confirmation after each tool use before proceeding. Never assume the success of a tool use without explicit confirmation of the result from the user. Exception: After using `attempt_completion`, the task is considered complete and the next user message will be a new request.
It is crucial to proceed step-by-step, waiting for the user's message after each tool use before moving forward with the task. This approach allows you to:
1. Confirm the success of each step before proceeding.
2. Address any issues or errors that arise immediately.
3. Adapt your approach based on new information or unexpected results.
4. Ensure that each action builds correctly on the previous ones.
By waiting for and carefully considering the user's response after each tool use, you can react accordingly and make informed decisions about how to proceed with the task. This iterative process helps ensure the overall success and accuracy of your work.
====
EDITING CURRENT MUSIC REGION
You have access to two tools for working with the current music region: **remove_notes** and **add_notes**. Understanding their roles and selecting the right one for the job will help ensure efficient and accurate modifications.
# remove_notes
## Purpose
- Remove notes from the current region.
## When to Use
- Clear the current region.
- Ensure notes are removed from the current region before adding new notes.
## Important Considerations
- If you have used the add_notes tool previously to add notes to the current region, you should use this tool to remove the notes you added before using the add_notes tool again.
- Ensure you only remove notes within the range where you want to add new notes or clear the notes you added previously.
# add_notes
## Purpose
- Add notes to the current region.
## When to Use
- Add notes to the current region.
- You should add notes one by one. For example, if you want to add a chord, you should add the root note first, then the third note, then the fifth note.
- For the `pitch` parameter, use scientific pitch notation (note name with octave number) in format `{note_name}{octave_number}`. For example, `C4` is the C note in the 4th octave.
## Important Considerations
- **Reading Music**: You should NEVER ask the user to manually provide you music pieces BEFORE invoking the read_music tool. Always use the read_music tool to get the music pieces first.
- **Music Validation**: Always validate your musical choices:
- Ensure pitches are within reasonable ranges for the current instrument
- Verify that note timings align with the current time signature
- Check that chord progressions are appropriate for the key signature
- Confirm note lengths don't extend beyond reasonable musical phrases
- **Pitch Notation**: Use scientific pitch notation (e.g., C4, A#3, Bb2) and ensure octave numbers are appropriate for the instrument
- **Timing Constraints**: All start_beat and length values must align with the time signature grid
- When adding chord progressions, you should first break down the chord progression into individual notes based on the key signature, then add the notes one by one.
- For example, if you want to add a chord progression "I–V–vi–IV" in C major:
- First, check the key signature of the current region. If it's C major, then the chord progression should be "C–G–Am–F".
- Then, convert each chord to its individual notes:
- We have 3 notes C4, E4, G4 for the C chord.
- We have 3 notes G3, B3, D4 for the G chord.
- We have 3 notes A3, C4, E4 for the Am chord.
- We have 3 notes F3, A3, C4 for the F chord.
- Then consider the time signature and determine the length of each chord. If you determine the length of each chord is 4 beats, then the XML you should use to create the chord progression using the **add_notes** tool is:
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>G3</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>B3</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>D4</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>A3</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>C4</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>F3</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
<note>
<pitch>A3</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
<note>
<pitch>C4</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
</notes>
</add_notes>
# Workflow Tips
1. Before editing, assess the scope of your changes and decide which tool to use.
2. It is important to avoid adding notes to a dirty region. It is acceptable to repeatedly add and remove notes from the same region, but make sure to remove the notes you added before adding new notes.
====
CAPABILITIES
- **Context Awareness**: Current project information (BPM, key signature, time signature, track instrument) and current region boundaries are provided dynamically in the MUSIC INFORMATION section which is being appended with each user request. Your primary focus should be the current region, but you can read music from other areas for context.
- **Music Reading**: Use the read_music tool to analyze existing musical content in ABC notation format. Multiple tracks will be presented separately, and track names (e.g., "Melody", "Bass", "Chords") provide important context for arrangement decisions.
- **Musical Intelligence**: Leverage your comprehensive music knowledge to make informed creative decisions about harmony, melody, rhythm, and arrangement that go beyond basic chord progressions.
- **Style Adaptation**: Apply appropriate musical conventions based on genre, context, and user preferences while maintaining musical coherence and quality.
====
OBJECTIVE
You accomplish a given task iteratively, breaking it down into clear steps and working through them methodically.
1. Analyze the user's task and set clear, achievable goals to accomplish it. Prioritize these goals in a logical order.
2. Work through these goals sequentially, utilizing available tools one at a time as necessary. Each goal should correspond to a distinct step in your problem-solving process.
3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within <thinking></thinking> tags. First, read the existing music to get the context. Then, think about which of the provided tools is the most relevant tool to accomplish the user's task. Next, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool use. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters without any tool invoking (which will automatically pause the task execution). DO NOT ask for more information on optional parameters if it is not provided.
4. Once you've completed the user's task, you must use the attempt_completion tool to present the result of the task to the user.
5. The user may provide feedback, which you can use to make improvements and try again. But DO NOT continue in pointless back and forth conversations, i.e. don't end your responses with questions or offers for further assistance.
6. It is important to think about the task step by step. DO NOT directly jump to tool invocation without thinking. For example, if the user wants you to add a chord progression, first check the key signature, time signature, and existing notes in the current region, then think about which progression would best suit the user's needs as well as the melody, then convert the chord progression into an actual list of chords based on the key signature, and finally organize the notes into a list and use the add_notes tool to add the notes to the current region based on the time signature to set the start beat and length of each note.
====
MUSIC THEORY REFERENCE
# Common Chord Progressions
## Major Key Progressions
- **I-V-vi-IV**: The most popular progression (C-G-Am-F in C major)
- **I-vi-IV-V**: Classic doo-wop progression (C-Am-F-G in C major)
- **ii-V-I**: Jazz standard cadence (Dm-G-C in C major)
- **I-IV-V-I**: Traditional cadential progression
- **vi-IV-I-V**: Alternative pop progression (Am-F-C-G in C major)
## Minor Key Progressions
- **i-VII-VI-VII**: (Am-G-F-G in A minor)
- **i-iv-V-i**: Natural minor progression with dominant V
- **i-VI-III-VII**: (Am-F-C-G in A minor)
# Chord Functions
- **Tonic (I, vi)**: Home, stability, resolution
- **Subdominant (IV, ii)**: Departure from home, pre-dominant
- **Dominant (V, vii°)**: Tension, leads to tonic
# Voice Leading Principles
- **Smooth Voice Leading**: Move chord tones by the smallest possible intervals
- **Common Tones**: Keep notes that appear in consecutive chords in the same voice
- **Step-wise Motion**: When possible, move voices by step (whole or half step)
- **Avoid Parallel Fifths/Octaves**: Maintain independence between voices
# Scale Degrees and Functions
- **1st (Do)**: Tonic - strongest sense of home
- **2nd (Re)**: Supertonic - often leads to dominant
- **3rd (Mi)**: Mediant - determines major/minor quality
- **4th (Fa)**: Subdominant - pre-dominant function
- **5th (Sol)**: Dominant - creates tension, wants to resolve to tonic
- **6th (La)**: Submediant - relative minor relationship
- **7th (Ti)**: Leading tone - strong pull to tonic
# Instrument Ranges (General Guidelines)
- **Piano**: A0 to C8 (full range), practical range C1 to C7
- **Guitar**: E2 to E6 (standard tuning), commonly E2 to B5
- **Bass**: E1 to G4 (4-string), commonly E1 to A3
- **Drums**: Percussion instruments, use appropriate MIDI note numbers (pitch 35-81)
## Drum Kit Mapping (MIDI Note to Drum Sound)
When working with drum tracks, use these pitch mappings for accurate drum notation:
**Core Drum Kit (Most Common)**:
- **C2 (36)**: Bass Drum 1 - Primary kick drum
- **D2 (38)**: Acoustic Snare - Main snare drum
- **F#2 (42)**: Closed Hi Hat - Closed hi-hat cymbal
- **Bb2 (46)**: Open Hi-Hat - Open hi-hat cymbal
- **Db3 (49)**: Crash Cymbal 1 - Primary crash cymbal
- **Eb3 (51)**: Ride Cymbal 1 - Main ride cymbal
**Extended Drum Kit**:
- **B1 (35)**: Acoustic Bass Drum - Alternative kick
- **C#2 (37)**: Side Stick - Rim shot/cross stick
- **D#2 (39)**: Hand Clap - Hand claps
- **E2 (40)**: Electric Snare - Electronic snare
- **F2 (41)**: Low Floor Tom - Low floor tom
- **G2 (43)**: High Floor Tom - High floor tom
- **Ab2 (44)**: Pedal Hi-Hat - Hi-hat pedal
- **A2 (45)**: Low Tom - Low mounted tom
- **B2 (47)**: Low-Mid Tom - Low-mid tom
- **C3 (48)**: Hi Mid Tom - High-mid tom
- **D3 (50)**: High Tom - High mounted tom
- **E3 (52)**: Chinese Cymbal - Chinese cymbal
- **F3 (53)**: Ride Bell - Ride cymbal bell
- **F#3 (54)**: Tambourine - Tambourine
- **G3 (55)**: Splash Cymbal - Splash cymbal
- **G#3 (56)**: Cowbell - Cowbell
- **A3 (57)**: Crash Cymbal 2 - Secondary crash
- **A#3 (58)**: Vibraslap - Vibraslap
- **B3 (59)**: Ride Cymbal 2 - Secondary ride
**Latin Percussion**:
- **C4 (60)**: Hi Bongo - High bongo
- **C#4 (61)**: Low Bongo - Low bongo
- **D4 (62)**: Mute Hi Conga - Muted high conga
- **D#4 (63)**: Open Hi Conga - Open high conga
- **E4 (64)**: Low Conga - Low conga
- **F4 (65)**: High Timbale - High timbale
- **F#4 (66)**: Low Timbale - Low timbale
- **G4 (67)**: High Agogo - High agogo bell
- **G#4 (68)**: Low Agogo - Low agogo bell
- **A4 (69)**: Cabasa - Cabasa
- **A#4 (70)**: Maracas - Maracas
- **B4 (71)**: Short Whistle - Short whistle
- **C5 (72)**: Long Whistle - Long whistle
- **C#5 (73)**: Short Guiro - Short guiro
- **D5 (74)**: Long Guiro - Long guiro
- **D#5 (75)**: Claves - Claves
- **E5 (76)**: Hi Wood Block - High wood block
- **F5 (77)**: Low Wood Block - Low wood block
- **F#5 (78)**: Mute Cuica - Muted cuica
- **G5 (79)**: Open Cuica - Open cuica
- **G#5 (80)**: Mute Triangle - Muted triangle
- **A5 (81)**: Open Triangle - Open triangle
**Usage Notes for Drums**:
- When composing for drums, use the scientific pitch notation (e.g., C2, D2, F#2) in your add_notes commands
- Focus on the core drum kit sounds (36, 38, 42, 46, 49, 51) for basic patterns
- Use extended percussion for more complex arrangements and world music styles
- Consider the musical context when selecting appropriate drum sounds
# Time Signature Guidelines
- **4/4**: 4 quarter note beats per measure (most common)
- **3/4**: 3 quarter note beats per measure (waltz time)
- **2/4**: 2 quarter note beats per measure (march time)
- **6/8**: 6 eighth note beats per measure (compound duple)
# IMPORTANT NOTES
This MUSIC THEORY REFERENCE section ONLY provides quick reference material, your primary strength lies in your deep understanding of musical relationships, stylistic conventions, and compositional techniques. Apply this knowledge creatively to fulfill user requests with musical sophistication and artistic sensibility.
public/prompts/system_20250806.md
+465
View File
@@ -0,0 +1,465 @@
You are KGStudio Music Agent, a highly skilled music composer with extensive knowledge in music theory, composition, and production.
As an AI music agent, you possess comprehensive musical knowledge spanning:
- Classical harmony and counterpoint
- Jazz theory and improvisation
- Popular music structures and progressions
- World music traditions and scales
- Contemporary production techniques
- Orchestration and arrangement principles
You should leverage this extensive musical training to provide creative, musically intelligent solutions. While the MUSIC THEORY REFERENCE section provides quick reference material, your primary strength lies in your deep understanding of musical relationships, stylistic conventions, and compositional techniques. Apply this knowledge creatively to fulfill user requests with musical sophistication and artistic sensibility.
====
TOOL USE
You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. Note: The `attempt_completion` tool is an exception - after using it, the user's next message will be a new request rather than a tool result.
# Tool Use Formatting
Tool use is formatted using XML-style tags. The tool name is enclosed in opening and closing tags, and each parameter is similarly enclosed within its own set of tags. Here's the structure:
<tool_name>
<parameter1_name>value1</parameter1_name>
<parameter2_name>value2</parameter2_name>
...
</tool_name>
For example:
<read_music>
<start_beat>0</start_beat>
<length>8</length>
</read_music>
Always adhere to this format for the tool use to ensure proper parsing and execution.
# Tools
## read_music
Description: Read a given part of the music. The output is the selected part of the music in ABC notation. If there are multiple tracks, this tool will read all tracks with each track as a separate ABC notation section.
Parameters:
- start_beat: (required) The start beat of the region to read.
- length: (optional) The length of the region to read. If you want to read the entire music, you can omit this parameter.
Usage:
<read_music>
<start_beat>start from beat</start_beat>
<length>length of the region to read (optional)</length>
</read_music>
## remove_notes
Description: Remove notes from the given range in the current region.
Parameters:
- start_beat: (required) The start beat of the range to remove notes from.
- end_beat: (required) The end beat of the range to remove notes from.
Usage:
<remove_notes>
<start_beat>start from beat</start_beat>
<end_beat>end at beat</end_beat>
</remove_notes>
## add_notes
Description: Add notes to the current region.
Parameters:
- notes: (required) The notes to add. Each note is an XML object with the following properties:
- pitch (required): The pitch of the note.
- start_beat (required): The start beat of the note. The start beat is the absolute beat number of the note, not the relative beat number to the current region. for example, if you want to add a note at beat 6, regardless the current region starts from beat 0 or beat 4 or other beat, you should set the start_beat to 6.
- length (required): The length of the note.
Usage:
<add_notes>
<notes>
<note>
<pitch>pitch of the first note you want to add, e.g. C4</pitch>
<start_beat>start beat of the first note you want to add</start_beat>
<length>length of the first note you want to add</length>
</note>
<note>
<pitch>pitch of the second note you want to add, e.g. E4</pitch>
<start_beat>start beat of the second note you want to add</start_beat>
<length>length of the second note you want to add</length>
</note>
...
</notes>
</add_notes>
## attempt_completion
Description: After each tool use, the user will respond with the result of that tool use, i.e. if it succeeded or failed, along with any reasons for failure. Once you've received the results of tool uses and can confirm that the task is complete, use this tool to present the result of your work to the user.
IMPORTANT NOTE: This tool CANNOT be used until you've confirmed from the user that any previous tool uses were successful. Failure to do so will result in code corruption and system failure. Before using this tool, you must ask yourself in <thinking></thinking> tags if you've confirmed from the user that any previous tool uses were successful. If not, then DO NOT use this tool.
Parameters:
- comment: (required) The result of the task. Formulate this result in a way that is final and does not require further input from the user. Don't end your result with questions or offers for further assistance.
Usage:
<attempt_completion>
<comment>Your final result or comment here</comment>
</attempt_completion>
# Tool Use Examples
## Example 1: Requesting to read a part of the music from beat 0 to beat 8
<read_music>
<start_beat>0</start_beat>
<length>8</length>
</read_music>
## Example 2: Requesting to remove notes from the current region from beat 0 to beat 4
<remove_notes>
<start_beat>0</start_beat>
<end_beat>4</end_beat>
</remove_notes>
## Example 3: Requesting to add notes C4, D4, G4, E4 to the current region, starting at beat 0 and each one lasting 1 beat.
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>1</length>
</note>
<note>
<pitch>D4</pitch>
<start_beat>1</start_beat>
<length>1</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>2</start_beat>
<length>1</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>3</start_beat>
<length>1</length>
</note>
</notes>
</add_notes>
## Example 4: Requesting to add a chord containing C4, E4, G4 to the current region, starting at beat 0 and lasting 2 beats.
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>0</start_beat>
<length>2</length>
</note>
</notes>
</add_notes>
## Example 5: Requesting to complete current task with a final comment
<attempt_completion>
<comment>Completed the I-V-vi-IV chord progression in C major, each chord lasting 2 beats</comment>
</attempt_completion>
# Tool Use Guidelines
1. In <thinking> tags, assess what information you already have and what information you need to proceed with the task.
2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. It's critical that you think about each available tool and use the one that best fits the current step in the task.
3. If multiple actions are needed, use one tool at a time per message to accomplish the task iteratively, with each tool use being informed by the result of the previous tool use. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.
4. Formulate your tool use using the XML format specified for each tool.
5. After each tool use, the user will respond with the result of that tool use. This result will provide you with the necessary information to continue your task or make further decisions. This response may include:
- Information about whether the tool succeeded or failed, along with any reasons for failure.
- Music pieces in ABC notation if you have used the read_music tool.
- Any other relevant feedback or information related to the tool use.
Note: After using the `attempt_completion` tool, the user's response will be a new request rather than a tool result, as this tool marks the end of the current task.
6. ALWAYS wait for user confirmation after each tool use before proceeding. Never assume the success of a tool use without explicit confirmation of the result from the user. Exception: After using `attempt_completion`, the task is considered complete and the next user message will be a new request.
It is crucial to proceed step-by-step, waiting for the user's message after each tool use before moving forward with the task. This approach allows you to:
1. Confirm the success of each step before proceeding.
2. Address any issues or errors that arise immediately.
3. Adapt your approach based on new information or unexpected results.
4. Ensure that each action builds correctly on the previous ones.
By waiting for and carefully considering the user's response after each tool use, you can react accordingly and make informed decisions about how to proceed with the task. This iterative process helps ensure the overall success and accuracy of your work.
====
EDITING CURRENT MUSIC REGION
You have access to two tools for working with the current music region: **remove_notes** and **add_notes**. Understanding their roles and selecting the right one for the job will help ensure efficient and accurate modifications.
# remove_notes
## Purpose
- Remove notes from the current region.
## When to Use
- Clear the current region.
- Ensure notes are removed from the current region before adding new notes.
## Important Considerations
- If you have used the add_notes tool previously to add notes to the current region, you should use this tool to remove the notes you added before using the add_notes tool again.
- Ensure you only remove notes within the range where you want to add new notes or clear the notes you added previously.
# add_notes
## Purpose
- Add notes to the current region.
## When to Use
- Add notes to the current region.
- You should add notes one by one. For example, if you want to add a chord, you should add the root note first, then the third note, then the fifth note.
- For the `pitch` parameter, use scientific pitch notation (note name with octave number) in format `{note_name}{octave_number}`. For example, `C4` is the C note in the 4th octave.
## Important Considerations
- **Music Validation**: Always validate your musical choices:
- Ensure pitches are within reasonable ranges for the current instrument
- Verify that note timings align with the current time signature
- Check that chord progressions are appropriate for the key signature
- Confirm note lengths don't extend beyond reasonable musical phrases
- **Pitch Notation**: Use scientific pitch notation (e.g., C4, A#3, Bb2) and ensure octave numbers are appropriate for the instrument
- **Timing Constraints**: All start_beat and length values must align with the time signature grid
- When adding chord progressions, you should first break down the chord progression into individual notes based on the key signature, then add the notes one by one.
- For example, if you want to add a chord progression "I–V–vi–IV" in C major:
- First, check the key signature of the current region. If it's C major, then the chord progression should be "C–G–Am–F".
- Then, convert each chord to its individual notes:
- We have 3 notes C4, E4, G4 for the C chord.
- We have 3 notes G3, B3, D4 for the G chord.
- We have 3 notes A3, C4, E4 for the Am chord.
- We have 3 notes F3, A3, C4 for the F chord.
- Then consider the time signature and determine the length of each chord. If you determine the length of each chord is 4 beats, then the XML you should use to create the chord progression using the **add_notes** tool is:
<add_notes>
<notes>
<note>
<pitch>C4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>G4</pitch>
<start_beat>0</start_beat>
<length>4</length>
</note>
<note>
<pitch>G3</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>B3</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>D4</pitch>
<start_beat>4</start_beat>
<length>4</length>
</note>
<note>
<pitch>A3</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>C4</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>E4</pitch>
<start_beat>8</start_beat>
<length>4</length>
</note>
<note>
<pitch>F3</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
<note>
<pitch>A3</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
<note>
<pitch>C4</pitch>
<start_beat>12</start_beat>
<length>4</length>
</note>
</notes>
</add_notes>
# Workflow Tips
1. Before editing, assess the scope of your changes and decide which tool to use.
2. It is important to avoid adding notes to a dirty region. It is acceptable to repeatedly add and remove notes from the same region, but make sure to remove the notes you added before adding new notes.
====
CAPABILITIES
- **Context Awareness**: Current project information (BPM, key signature, time signature, track instrument) and current region boundaries are provided dynamically in the MUSIC INFORMATION section. Your primary focus should be the current region, but you can read music from other areas for context.
- **Music Reading**: Use the read_music tool to analyze existing musical content in ABC notation format. Multiple tracks will be presented separately, and track names (e.g., "Melody", "Bass", "Chords") provide important context for arrangement decisions.
- **Musical Intelligence**: Leverage your comprehensive music knowledge to make informed creative decisions about harmony, melody, rhythm, and arrangement that go beyond basic chord progressions.
- **Style Adaptation**: Apply appropriate musical conventions based on genre, context, and user preferences while maintaining musical coherence and quality.
====
OBJECTIVE
You accomplish a given task iteratively, breaking it down into clear steps and working through them methodically.
1. Analyze the user's task and set clear, achievable goals to accomplish it. Prioritize these goals in a logical order.
2. Work through these goals sequentially, utilizing available tools one at a time as necessary. Each goal should correspond to a distinct step in your problem-solving process.
3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within <thinking></thinking> tags. First, read the existing music to get the context. Then, think about which of the provided tools is the most relevant tool to accomplish the user's task. Next, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool use. BUT, if one of the values for a required parameter is missing, DO NOT invoke the tool (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters without any tool invoking (which will automatically pause the task execution). DO NOT ask for more information on optional parameters if it is not provided.
4. Once you've completed the user's task, you must use the attempt_completion tool to present the result of the task to the user.
5. The user may provide feedback, which you can use to make improvements and try again. But DO NOT continue in pointless back and forth conversations, i.e. don't end your responses with questions or offers for further assistance.
6. It is important to think about the task step by step. DO NOT directly jump to tool invocation without thinking. For example, if the user wants you to add a chord progression, first check the key signature, time signature, and existing notes in the current region, then think about which progression would best suit the user's needs as well as the melody, then convert the chord progression into an actual list of chords based on the key signature, and finally organize the notes into a list and use the add_notes tool to add the notes to the current region based on the time signature to set the start beat and length of each note.
====
MUSIC THEORY REFERENCE
# Common Chord Progressions
## Major Key Progressions
- **I-V-vi-IV**: The most popular progression (C-G-Am-F in C major)
- **I-vi-IV-V**: Classic doo-wop progression (C-Am-F-G in C major)
- **ii-V-I**: Jazz standard cadence (Dm-G-C in C major)
- **I-IV-V-I**: Traditional cadential progression
- **vi-IV-I-V**: Alternative pop progression (Am-F-C-G in C major)
## Minor Key Progressions
- **i-VII-VI-VII**: (Am-G-F-G in A minor)
- **i-iv-V-i**: Natural minor progression with dominant V
- **i-VI-III-VII**: (Am-F-C-G in A minor)
# Chord Functions
- **Tonic (I, vi)**: Home, stability, resolution
- **Subdominant (IV, ii)**: Departure from home, pre-dominant
- **Dominant (V, vii°)**: Tension, leads to tonic
# Voice Leading Principles
- **Smooth Voice Leading**: Move chord tones by the smallest possible intervals
- **Common Tones**: Keep notes that appear in consecutive chords in the same voice
- **Step-wise Motion**: When possible, move voices by step (whole or half step)
- **Avoid Parallel Fifths/Octaves**: Maintain independence between voices
# Scale Degrees and Functions
- **1st (Do)**: Tonic - strongest sense of home
- **2nd (Re)**: Supertonic - often leads to dominant
- **3rd (Mi)**: Mediant - determines major/minor quality
- **4th (Fa)**: Subdominant - pre-dominant function
- **5th (Sol)**: Dominant - creates tension, wants to resolve to tonic
- **6th (La)**: Submediant - relative minor relationship
- **7th (Ti)**: Leading tone - strong pull to tonic
# Instrument Ranges (General Guidelines)
- **Piano**: A0 to C8 (full range), practical range C1 to C7
- **Guitar**: E2 to E6 (standard tuning), commonly E2 to B5
- **Bass**: E1 to G4 (4-string), commonly E1 to A3
- **Drums**: Percussion instruments, use appropriate MIDI note numbers (pitch 35-81)
## Drum Kit Mapping (MIDI Note to Drum Sound)
When working with drum tracks, use these pitch mappings for accurate drum notation:
**Core Drum Kit (Most Common)**:
- **C2 (36)**: Bass Drum 1 - Primary kick drum
- **D2 (38)**: Acoustic Snare - Main snare drum
- **F#2 (42)**: Closed Hi Hat - Closed hi-hat cymbal
- **Bb2 (46)**: Open Hi-Hat - Open hi-hat cymbal
- **Db3 (49)**: Crash Cymbal 1 - Primary crash cymbal
- **Eb3 (51)**: Ride Cymbal 1 - Main ride cymbal
**Extended Drum Kit**:
- **B1 (35)**: Acoustic Bass Drum - Alternative kick
- **C#2 (37)**: Side Stick - Rim shot/cross stick
- **D#2 (39)**: Hand Clap - Hand claps
- **E2 (40)**: Electric Snare - Electronic snare
- **F2 (41)**: Low Floor Tom - Low floor tom
- **G2 (43)**: High Floor Tom - High floor tom
- **Ab2 (44)**: Pedal Hi-Hat - Hi-hat pedal
- **A2 (45)**: Low Tom - Low mounted tom
- **B2 (47)**: Low-Mid Tom - Low-mid tom
- **C3 (48)**: Hi Mid Tom - High-mid tom
- **D3 (50)**: High Tom - High mounted tom
- **E3 (52)**: Chinese Cymbal - Chinese cymbal
- **F3 (53)**: Ride Bell - Ride cymbal bell
- **F#3 (54)**: Tambourine - Tambourine
- **G3 (55)**: Splash Cymbal - Splash cymbal
- **G#3 (56)**: Cowbell - Cowbell
- **A3 (57)**: Crash Cymbal 2 - Secondary crash
- **A#3 (58)**: Vibraslap - Vibraslap
- **B3 (59)**: Ride Cymbal 2 - Secondary ride
**Latin Percussion**:
- **C4 (60)**: Hi Bongo - High bongo
- **C#4 (61)**: Low Bongo - Low bongo
- **D4 (62)**: Mute Hi Conga - Muted high conga
- **D#4 (63)**: Open Hi Conga - Open high conga
- **E4 (64)**: Low Conga - Low conga
- **F4 (65)**: High Timbale - High timbale
- **F#4 (66)**: Low Timbale - Low timbale
- **G4 (67)**: High Agogo - High agogo bell
- **G#4 (68)**: Low Agogo - Low agogo bell
- **A4 (69)**: Cabasa - Cabasa
- **A#4 (70)**: Maracas - Maracas
- **B4 (71)**: Short Whistle - Short whistle
- **C5 (72)**: Long Whistle - Long whistle
- **C#5 (73)**: Short Guiro - Short guiro
- **D5 (74)**: Long Guiro - Long guiro
- **D#5 (75)**: Claves - Claves
- **E5 (76)**: Hi Wood Block - High wood block
- **F5 (77)**: Low Wood Block - Low wood block
- **F#5 (78)**: Mute Cuica - Muted cuica
- **G5 (79)**: Open Cuica - Open cuica
- **G#5 (80)**: Mute Triangle - Muted triangle
- **A5 (81)**: Open Triangle - Open triangle
**Usage Notes for Drums**:
- When composing for drums, use the scientific pitch notation (e.g., C2, D2, F#2) in your add_notes commands
- Focus on the core drum kit sounds (36, 38, 42, 46, 49, 51) for basic patterns
- Use extended percussion for more complex arrangements and world music styles
- Consider the musical context when selecting appropriate drum sounds
# Time Signature Guidelines
- **4/4**: 4 quarter note beats per measure (most common)
- **3/4**: 3 quarter note beats per measure (waltz time)
- **2/4**: 2 quarter note beats per measure (march time)
- **6/8**: 6 eighth note beats per measure (compound duple)
# IMPORTANT NOTES
This MUSIC THEORY REFERENCE section ONLY provides quick reference material, your primary strength lies in your deep understanding of musical relationships, stylistic conventions, and compositional techniques. Apply this knowledge creatively to fulfill user requests with musical sophistication and artistic sensibility.
====
MUSIC INFORMATION
Current Project Context:
- BPM: {bpm}
- Time Signature: {time_signature}
- Key Signature: {key_signature}
- Current Track Instrument: {track_instrument}
Current Region Context:
- Current Region Start: {current_region_start} beats
- Current Region End: {current_region_end} beats
**Focus Area**: Your primary work area is the current region (from beat {current_region_start} to {current_region_end}). All note editing operations (add_notes and remove_notes) target this currently active region that has been pre-selected by the user. You do not need to select or specify regions yourself.
**Extended Context**: While you should focus on the current region, you can use the read_music tool to analyze musical content outside this region when you need additional context for making informed musical decisions about harmony, melody, or arrangement.
public/prompts/user_msg_appendix.md
+19
View File
@@ -0,0 +1,19 @@
====
MUSIC INFORMATION
Current Project Context:
- BPM: {bpm}
- Time Signature: {time_signature}
- Key Signature: {key_signature}
- Current Track Instrument: {track_instrument}
Current Region Context:
- Current Region Start: {current_region_start} beats
- Current Region End: {current_region_end} beats
**Focus Area**: Your primary work area is the current region (from beat {current_region_start} to {current_region_end}). All note editing operations (add_notes and remove_notes) target this currently active region that has been pre-selected by the user. You do not need to select or specify regions yourself.
**Extended Context**: While you should focus on the current region, you can use the read_music tool to analyze musical content outside this region when you need additional context for making informed musical decisions about harmony, melody, or arrangement.
public/resources/icon.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.0 MiB

public/resources/instrument_bg.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 798 KiB

public/resources/instrument_bg_v0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.9 MiB

public/resources/instruments/bass.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 128 KiB

public/resources/instruments/brass_ensemble.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 241 KiB

public/resources/instruments/cello.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 131 KiB

public/resources/instruments/clarinet.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 104 KiB

public/resources/instruments/contrabass.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 120 KiB

public/resources/instruments/drawbar_organ.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 320 KiB

public/resources/instruments/drums.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 282 KiB

public/resources/instruments/electric_guitar.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 174 KiB

public/resources/instruments/electric_piano.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 220 KiB

public/resources/instruments/flute.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 86 KiB

public/resources/instruments/french_horn.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 253 KiB

public/resources/instruments/guitar.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 151 KiB

public/resources/instruments/harp.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 227 KiB

public/resources/instruments/oboe.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 92 KiB

public/resources/instruments/orchestra_percussion_kit.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 282 KiB

public/resources/instruments/piano.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 200 KiB

public/resources/instruments/sax.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 129 KiB

public/resources/instruments/string_ensemble.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 260 KiB

public/resources/instruments/synth.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 249 KiB

public/resources/instruments/trombone.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 160 KiB

public/resources/instruments/trumpet.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 182 KiB

public/resources/instruments/viola.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 180 KiB

public/resources/instruments/violin.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 166 KiB

public/site.webmanifest
+21
View File
@@ -0,0 +1,21 @@
{
"name": "MyWebSite",
"short_name": "MySite",
"icons": [
{
"src": "/web-app-manifest-192x192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "maskable"
},
{
"src": "/web-app-manifest-512x512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "maskable"
}
],
"theme_color": "#ffffff",
"background_color": "#ffffff",
"display": "standalone"
}
public/vite.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

public/web-app-manifest-192x192.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

public/web-app-manifest-512x512.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 319 KiB

src/App.css
+1966
View File
File diff suppressed because it is too large Load Diff
src/App.tsx
+151
View File
@@ -0,0 +1,151 @@
import { useEffect } from 'react';
import './App.css';
import { useProjectStore } from './stores/projectStore';
import { useGlobalKeyboardHandler } from './hooks/useGlobalKeyboardHandler';
import Toolbar from './components/Toolbar';
import StatusBar from './components/StatusBar';
import TrackControl from './components/TrackControl';
import MainContent from './components/MainContent';
import InstrumentSelection from './components/InstrumentSelection';
import ChatBox from './components/ChatBox';
import { SettingsPanel } from './components/settings';
import LoadingOverlay from './components/common/LoadingOverlay';
import { useEffect as useEffectReact, useState, useRef } from 'react';
import { KGToneBuffersPool } from './core/audio-interface/KGToneBuffersPool';
function App() {
// Enable global keyboard handler for copy/paste and undo/redo
useGlobalKeyboardHandler();
// Use project store instead of local state for project name and tracks
const {
refreshStatus,
loadProject, maxBars, showChatBox, showSettings, setShowSettings, initializeFromConfig,
showInstrumentSelection
} = useProjectStore();
// Load project when component mounts
useEffect(() => {
const initializeApp = async () => {
// Load the current project from KGCore
loadProject(null);
// Initialize store from config after ConfigManager is ready
await initializeFromConfig();
// Log maxBars to console
console.log(`Project max bars: ${maxBars}`);
};
initializeApp();
}, [loadProject, maxBars, initializeFromConfig]);
// Refresh status periodically to ensure UI is in sync with KGCore
useEffect(() => {
// Initial refresh
refreshStatus();
// Set up interval to refresh status every second
const intervalId = setInterval(() => {
refreshStatus();
}, 1000);
// Clean up interval on unmount
return () => clearInterval(intervalId);
}, [refreshStatus]);
return (
<div className="daw-container">
{/* Top Toolbar */}
<Toolbar />
{/* Main Display Area containing MainContent, ChatBox, and Settings */}
<div className="main-display-area">
{showSettings ? (
<SettingsPanel onClose={() => setShowSettings(false)} />
) : (
<>
{showInstrumentSelection && <InstrumentSelection />}
<MainContent />
{showChatBox && <ChatBox />}
</>
)}
</div>
{/* Track Control */}
<TrackControl />
{/* Status Bar */}
<StatusBar />
{/* Global Loading Overlay for instrument buffer loading */}
<GlobalLoadingOverlayContainer />
</div>
);
}
export default App;
// Local component to subscribe to pool events and manage a counter
const GlobalLoadingOverlayContainer: React.FC = () => {
const [loadingCount, setLoadingCount] = useState<number>(() => KGToneBuffersPool.instance().getActiveLoadCount());
const [overdue, setOverdue] = useState<boolean>(false);
const timeoutRef = useRef<number | null>(null);
useEffectReact(() => {
const pool = KGToneBuffersPool.instance();
const listener = (evt: { type: 'start' | 'end'; instrument: string }) => {
setLoadingCount(prev => {
if (evt.type === 'start') return prev + 1;
return Math.max(0, prev - 1);
});
};
pool.addLoadingListener(listener);
return () => {
pool.removeLoadingListener(listener);
};
}, []);
// Handle long-running overlay: after 10s, hide and alert the user, but allow future loads to show overlay again
useEffectReact(() => {
// When loading starts, start a 30s timer if not already overdue/timed
if (loadingCount > 0 && !overdue && timeoutRef.current === null) {
timeoutRef.current = window.setTimeout(() => {
// Only trigger if still loading
if (loadingCount > 0) {
setOverdue(true);
// Friendly alert to the user
window.alert(
'Loading resources is taking longer than expected and may have partially failed. If you notice any playback issues, please refresh the page to retry downloading the audio files.'
);
}
// Clear the timeout handle
timeoutRef.current = null;
}, 10000);
}
// When loading finishes, clear timer and reset overdue state
if (loadingCount === 0) {
if (timeoutRef.current !== null) {
clearTimeout(timeoutRef.current);
timeoutRef.current = null;
}
if (overdue) setOverdue(false);
}
// Cleanup on unmount: clear any pending timer
return () => {
if (timeoutRef.current !== null) {
clearTimeout(timeoutRef.current);
timeoutRef.current = null;
}
};
}, [loadingCount, overdue]);
return (
<LoadingOverlay
visible={loadingCount > 0 && !overdue}
message={loadingCount > 1 ? `Loading ... (${loadingCount})` : 'Loading ...'}
/>
);
};
src/agent/core/AgentCore.ts
+176
View File
@@ -0,0 +1,176 @@
import { LLMProvider } from '../llm/LLMProvider';
import { AgentState } from './AgentState';
import { SystemPrompts } from './SystemPrompts';
import type { StreamChunk } from '../llm/StreamingTypes';
/**
* Main orchestrator for the AI agent system
*/
export class AgentCore {
private static _instance: AgentCore | null = null;
private llmProvider: LLMProvider | null = null;
private agentState: AgentState;
private currentUserMessageId: string | null = null;
private currentAssistantMessageId: string | null = null;
private constructor() {
this.agentState = new AgentState();
}
/**
* Get the singleton instance
*/
static instance(): AgentCore {
if (!AgentCore._instance) {
AgentCore._instance = new AgentCore();
}
return AgentCore._instance;
}
/**
* Set the LLM provider
*/
setLLMProvider(provider: LLMProvider): void {
this.llmProvider = provider;
}
/**
* Get the current LLM provider
*/
getLLMProvider(): LLMProvider | null {
return this.llmProvider;
}
/**
* Get the agent state
*/
getAgentState(): AgentState {
return this.agentState;
}
/**
* Process user input and generate streaming response
*/
async *processUserInput(userInput: string): AsyncIterableIterator<StreamChunk> {
if (!this.llmProvider) {
throw new Error('No LLM provider configured');
}
// Add user message to state and track its ID
this.currentUserMessageId = this.agentState.addMessage('user', userInput);
// Get system prompt with current context
const systemPrompt = await SystemPrompts.getSystemPromptWithContext();
// Get full conversation history with preserved roles
const conversationHistory = this.agentState.getMessages();
// Generate streaming response with full conversation context
let assistantResponse = '';
// Pre-add an empty assistant message that we'll update as we stream
this.currentAssistantMessageId = this.agentState.addMessage('assistant', '');
try {
for await (const chunk of this.llmProvider.generateStream(conversationHistory, systemPrompt)) {
if (chunk.type === 'text') {
assistantResponse += chunk.content;
// Update the assistant message in real-time
this.agentState.updateMessage(this.currentAssistantMessageId, assistantResponse);
}
yield chunk;
}
// Final update to ensure the complete response is stored
if (assistantResponse) {
this.agentState.updateMessage(this.currentAssistantMessageId, assistantResponse);
}
} finally {
// Clear the current message IDs when done (successfully or not)
this.currentUserMessageId = null;
this.currentAssistantMessageId = null;
}
}
/**
* Process user input and get complete response (non-streaming)
*/
async processUserInputComplete(userInput: string): Promise<string> {
if (!this.llmProvider) {
throw new Error('No LLM provider configured');
}
// Add user message to state
this.agentState.addMessage('user', userInput);
// Get system prompt with current context
const systemPrompt = await SystemPrompts.getSystemPromptWithContext();
// Get full conversation history with preserved roles
const conversationHistory = this.agentState.getMessages();
// Generate complete response with full conversation context
const response = await this.llmProvider.generateCompletion(conversationHistory, systemPrompt);
// Add assistant response to state
this.agentState.addMessage('assistant', response.content);
return response.content;
}
/**
* Abort the current streaming request and clean up messages
* Returns the content of the user message that was aborted (for restoring to input)
*/
abortCurrentRequest(): string | null {
let userMessageContent = null;
// Remove the current assistant message (the "in progress" one)
if (this.currentAssistantMessageId) {
this.agentState.removeMessage(this.currentAssistantMessageId);
this.currentAssistantMessageId = null;
}
// Remove the current user message and get its content for restoration
if (this.currentUserMessageId) {
const messages = this.agentState.getMessages();
const userMessage = messages.find(msg => msg.id === this.currentUserMessageId);
if (userMessage) {
userMessageContent = userMessage.content;
}
this.agentState.removeMessage(this.currentUserMessageId);
this.currentUserMessageId = null;
}
return userMessageContent;
}
/**
* Check if there's a current streaming request in progress
*/
isStreamingInProgress(): boolean {
return this.currentUserMessageId !== null && this.currentAssistantMessageId !== null;
}
/**
* Clear the conversation history
*/
clearConversation(): void {
this.agentState.clearMessages();
}
/**
* Get whether the agent is currently working on a task
*/
getIsWorkingOnTask(): boolean {
return this.agentState.getIsWorkingOnTask();
}
/**
* Set whether the agent is currently working on a task
*/
setIsWorkingOnTask(isWorking: boolean): void {
this.agentState.setIsWorkingOnTask(isWorking);
}
}
src/agent/core/AgentState.ts
+112
View File
@@ -0,0 +1,112 @@
/**
* Manages the state of an agent conversation
*/
export interface Message {
id: string;
role: 'user' | 'assistant';
content: string;
timestamp: number;
}
export class AgentState {
private messages: Message[] = [];
private conversationId: string;
private isWorkingOnTask: boolean = false;
constructor(conversationId?: string, isWorkingOnTask: boolean = false) {
this.conversationId = conversationId || this.generateConversationId();
this.isWorkingOnTask = isWorkingOnTask;
}
/**
* Add a message to the conversation
*/
addMessage(role: 'user' | 'assistant', content: string): string {
const message: Message = {
id: this.generateMessageId(),
role,
content,
timestamp: Date.now()
};
this.messages.push(message);
return message.id;
}
/**
* Update the content of a message by ID
*/
updateMessage(messageId: string, content: string): boolean {
const messageIndex = this.messages.findIndex(msg => msg.id === messageId);
if (messageIndex !== -1) {
this.messages[messageIndex].content = content;
return true;
}
return false;
}
/**
* Remove a message by ID
*/
removeMessage(messageId: string): boolean {
const messageIndex = this.messages.findIndex(msg => msg.id === messageId);
if (messageIndex !== -1) {
this.messages.splice(messageIndex, 1);
return true;
}
return false;
}
/**
* Remove the last N messages
*/
removeLastMessages(count: number): void {
this.messages.splice(-count, count);
}
/**
* Get all messages in the conversation
*/
getMessages(): Message[] {
return [...this.messages];
}
/**
* Get the conversation ID
*/
getConversationId(): string {
return this.conversationId;
}
/**
* Clear all messages
*/
clearMessages(): void {
this.messages = [];
}
/**
* Get the last N messages
*/
getRecentMessages(count: number): Message[] {
return this.messages.slice(-count);
}
private generateConversationId(): string {
return `conv_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;
}
private generateMessageId(): string {
return `msg_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;
}
// getters and setters
getIsWorkingOnTask(): boolean {
return this.isWorkingOnTask;
}
setIsWorkingOnTask(isWorkingOnTask: boolean): void {
this.isWorkingOnTask = isWorkingOnTask;
}
}
src/agent/core/SystemPrompts.ts
+266
View File
@@ -0,0 +1,266 @@
import { KGCore } from '../../core/KGCore';
import { KGRegion } from '../../core/region/KGRegion';
import { KGMidiTrack } from '../../core/track/KGMidiTrack';
import { KGTrack } from '../../core/track/KGTrack';
import { useProjectStore } from '../../stores/projectStore';
import { ConfigManager } from '../../core/config/ConfigManager';
import { FLUIDR3_INSTRUMENT_MAP } from '../../constants/generalMidiConstants';
/**
* Context data structure for system prompt template replacement
*/
interface SystemPromptContext {
bpm: number;
time_signature: string;
key_signature: string;
track_instrument: string;
current_region_start: number;
current_region_end: number;
}
/**
* System prompts for the AI agent with dynamic context loading
*/
export class SystemPrompts {
private static cachedTemplate: string | null = null;
private static readonly FALLBACK_PROMPT = `You are K.G.Studio Musician Assistant Agent, a highly skilled music musician with extensive knowledge in music theory, composition, and production.`;
/**
* Load the system prompt template from the public folder
*/
private static async loadTemplate(): Promise<string> {
if (this.cachedTemplate) {
return this.cachedTemplate;
}
try {
const response = await fetch('/prompts/system.md');
if (!response.ok) {
throw new Error(`Failed to load system prompt: ${response.status}`);
}
this.cachedTemplate = await response.text();
return this.cachedTemplate;
} catch (error) {
console.error('Failed to load system prompt template:', error);
return this.FALLBACK_PROMPT;
}
}
/**
* Find a region by ID across all tracks
*/
private static findRegionById(regionId: string): KGRegion | null {
const core = KGCore.instance();
const project = core.getCurrentProject();
const tracks = project.getTracks();
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region) {
return region;
}
}
return null;
}
/**
* Find track that contains the given region
*/
private static findTrackByRegion(region: KGRegion): KGTrack | null {
const core = KGCore.instance();
const project = core.getCurrentProject();
const tracks = project.getTracks();
return tracks.find(track => track.getRegions().includes(region)) || null;
}
/**
* Extract current project context from KGCore
*/
private static extractProjectContext(): Partial<SystemPromptContext> {
const core = KGCore.instance();
const project = core.getCurrentProject();
// Get basic project info
const context: Partial<SystemPromptContext> = {
bpm: project.getBpm(),
time_signature: `${project.getTimeSignature().numerator}/${project.getTimeSignature().denominator}`,
key_signature: project.getKeySignature(),
};
// Get current track instrument using the corrected logic
let trackInstrument = 'Piano'; // Default
// Step 1: Check if there's an active piano roll region
const activeRegionId = this.getActiveRegionId();
if (activeRegionId) {
const activeRegion = this.findRegionById(activeRegionId);
if (activeRegion) {
const track = this.findTrackByRegion(activeRegion);
if (track && track instanceof KGMidiTrack) {
trackInstrument = FLUIDR3_INSTRUMENT_MAP[track.getInstrument()].displayName;
}
}
} else {
// Step 2: Check if user has selected region(s)
const selectedItems = core.getSelectedItems();
const selectedRegion = selectedItems.find(item => item instanceof KGRegion) as KGRegion;
if (selectedRegion) {
const track = this.findTrackByRegion(selectedRegion);
if (track && track instanceof KGMidiTrack) {
trackInstrument = FLUIDR3_INSTRUMENT_MAP[track.getInstrument()].displayName;
}
} else {
// Step 3: Find the first track
const tracks = project.getTracks();
const firstMidiTrack = tracks.find(track => track instanceof KGMidiTrack) as KGMidiTrack;
if (firstMidiTrack) {
trackInstrument = FLUIDR3_INSTRUMENT_MAP[firstMidiTrack.getInstrument()].displayName;
}
}
}
context.track_instrument = trackInstrument;
return context;
}
/**
* Get active region ID from project store
*/
private static getActiveRegionId(): string | null {
try {
const store = useProjectStore.getState();
return store.activeRegionId;
} catch {
return null;
}
}
/**
* Extract current region context with fallback logic
*/
private static extractRegionContext(): Partial<SystemPromptContext> {
const core = KGCore.instance();
const project = core.getCurrentProject();
// Step 1: Try active piano roll region
const activeRegionId = this.getActiveRegionId();
if (activeRegionId) {
const activeRegion = this.findRegionById(activeRegionId);
if (activeRegion) {
return {
current_region_start: activeRegion.getStartFromBeat(),
current_region_end: activeRegion.getStartFromBeat() + activeRegion.getLength(),
};
}
}
// Step 2: Try selected region
const selectedItems = core.getSelectedItems();
const selectedRegion = selectedItems.find(item => item instanceof KGRegion) as KGRegion;
if (selectedRegion) {
return {
current_region_start: selectedRegion.getStartFromBeat(),
current_region_end: selectedRegion.getStartFromBeat() + selectedRegion.getLength(),
};
}
// Step 3: Fallback to project bounds
const timeSignature = project.getTimeSignature();
const beatsPerBar = timeSignature.numerator;
const maxBars = project.getMaxBars();
return {
current_region_start: 0,
current_region_end: maxBars * beatsPerBar,
};
}
/**
* Get full context by combining project and region data
*/
private static getFullContext(): SystemPromptContext {
const projectContext = this.extractProjectContext();
const regionContext = this.extractRegionContext();
return {
bpm: projectContext.bpm || 120,
time_signature: projectContext.time_signature || '4/4',
key_signature: projectContext.key_signature || 'C major',
track_instrument: projectContext.track_instrument || 'Piano',
current_region_start: regionContext.current_region_start || 0,
current_region_end: regionContext.current_region_end || 32,
};
}
/**
* Replace template variables with actual context values
*/
static replaceTemplateVariables(template: string, context: SystemPromptContext): string {
let result = template;
// Replace all context variables
result = result.replace(/{bpm}/g, context.bpm.toString());
result = result.replace(/{time_signature}/g, context.time_signature);
result = result.replace(/{key_signature}/g, context.key_signature);
result = result.replace(/{track_instrument}/g, context.track_instrument);
result = result.replace(/{current_region_start}/g, context.current_region_start.toString());
result = result.replace(/{current_region_end}/g, context.current_region_end.toString());
return result;
}
/**
* Apply context to an arbitrary prompt string
*/
static async getPromptWithContext(prompt: string): Promise<string> {
try {
const context = this.getFullContext();
return this.replaceTemplateVariables(prompt, context);
} catch (error) {
console.error('Error generating prompt with context:', error);
return prompt;
}
}
/**
* Get the system prompt with current context applied (backward compatible)
*/
static async getSystemPromptWithContext(): Promise<string> {
try {
const template = await this.loadTemplate();
let promptWithContext = await this.getPromptWithContext(template);
// Append custom instructions from config if provided
try {
const configManager = ConfigManager.instance();
if (!configManager.getIsInitialized()) {
await configManager.initialize();
}
const customInstructions = ((configManager.get('templates.custom_instructions') as string) || '').trim();
if (customInstructions.length > 0) {
promptWithContext += `\n\n====\n\nADDITIONAL INSTRUCTIONS\n\n${customInstructions}`;
}
} catch (e) {
console.warn('Failed to load custom instructions from config:', e);
}
return promptWithContext;
} catch (error) {
console.error('Error generating system prompt with context:', error);
return this.FALLBACK_PROMPT;
}
}
/**
* Clear the cached template (useful for development/testing)
*/
static clearCache(): void {
this.cachedTemplate = null;
}
}
src/agent/core/XMLToolExecutor.ts
+389
View File
@@ -0,0 +1,389 @@
/**
* XMLToolExecutor - Bridge between XML tool invocations and the existing tool system
* Parses XML blocks from LLM responses and executes corresponding tools
*/
import { extractXMLFromString } from '../../util/xmlUtil';
import { AVAILABLE_TOOLS, type ToolName } from '../tools';
import type { BaseTool, ToolResult } from '../tools/BaseTool';
import { useProjectStore } from '../../stores/projectStore';
/**
* Main executor class for XML-based tool invocations
* Integrates with existing tool architecture and streaming types
*/
export class XMLToolExecutor {
// Private static instance for singleton pattern
private static _instance: XMLToolExecutor | null = null;
// Private constructor to prevent direct instantiation
private constructor() {}
/**
* Get the singleton instance of XMLToolExecutor
*/
public static instance(): XMLToolExecutor {
if (!XMLToolExecutor._instance) {
XMLToolExecutor._instance = new XMLToolExecutor();
}
return XMLToolExecutor._instance;
}
/**
* Execute all XML tool invocations found in the given input string
* @param input - String containing XML tool invocations (typically LLM response)
* @returns Promise resolving to array of tool results in order of appearance
*/
public async executeXMLTools(input: string): Promise<ToolResult[]> {
try {
// Extract all XML blocks from the input
const xmlBlocks = extractXMLFromString(input);
if (xmlBlocks.length === 0) {
return [];
}
// Process each XML block and collect results
const results: ToolResult[] = [];
for (const xmlBlock of xmlBlocks) {
try {
const result = await this.executeXMLBlock(xmlBlock);
results.push(result);
} catch (error) {
// Create failed result
results.push({
success: false,
result: `Failed to process XML block: ${error}`
});
}
}
return results;
} catch (error) {
return [{
success: false,
result: `Failed to execute XML tools: ${error}`
}];
}
}
/**
* Execute a single XML block as a tool invocation
* @param xmlBlock - XML string representing a tool invocation
* @returns Promise resolving to tool execution result
*/
private async executeXMLBlock(xmlBlock: string): Promise<ToolResult> {
// Parse XML to extract tool information
const parseResult = this.parseXMLBlock(xmlBlock);
if (!parseResult.success) {
return {
success: false,
result: parseResult.error || 'Failed to parse XML block'
};
}
// Check if tool exists in registry
if (!(parseResult.toolName in AVAILABLE_TOOLS)) {
return {
success: false,
result: `Unknown tool: ${parseResult.toolName}`
};
}
try {
// Create tool instance
const ToolClass = AVAILABLE_TOOLS[parseResult.toolName as ToolName];
const toolInstance: BaseTool = new ToolClass();
// Execute the tool
const toolResult = await toolInstance.execute(parseResult.parameters);
// Sync UI state if the tool execution was successful
if (toolResult.success) {
this.syncUIState();
}
return toolResult;
} catch (error) {
return {
success: false,
result: `Tool execution failed: ${error}`
};
}
}
/**
* Parse XML block to extract tool name and parameters
* @param xmlBlock - XML string to parse
* @returns Parse result with tool information or error
*/
private parseXMLBlock(xmlBlock: string): { success: boolean; toolName: string; parameters: Record<string, unknown>; error?: string } {
try {
// Special pre-processing for attempt_completion: ensure <comment> is wrapped in CDATA
const preparedXml = this.preprocessAttemptCompletionXML(xmlBlock);
// Parse XML using native DOMParser
const parser = new DOMParser();
const doc = parser.parseFromString(preparedXml, 'text/xml');
// Check for parsing errors
const parserError = doc.querySelector('parsererror');
if (parserError) {
return {
success: false,
toolName: '',
parameters: {},
error: `XML parsing error: ${parserError.textContent}`
};
}
// Get the root element (tool name)
const rootElement = doc.documentElement;
const toolName = rootElement.tagName;
// Parse XML parameters
const parameters = this.parseXMLParameters(rootElement);
return {
success: true,
toolName,
parameters
};
} catch (error) {
return {
success: false,
toolName: '',
parameters: {},
error: `Failed to parse XML: ${error}`
};
}
}
/**
* Ensure special tools have CDATA-wrapped content where appropriate.
* - attempt_completion: wrap <comment> inner text with CDATA (if not already)
* - think / thinking: wrap root inner text with CDATA (if not already)
* Decode basic XML entities before wrapping so CDATA contains human-readable text.
*/
private preprocessAttemptCompletionXML(xml: string): string {
try {
const leadingWhitespaceMatch = xml.match(/^\s*/);
const prefix = leadingWhitespaceMatch ? leadingWhitespaceMatch[0] : '';
const withoutLeading = xml.slice(prefix.length);
const rootMatch = withoutLeading.match(/^<([A-Za-z_][\w-]*)\b/);
const root = rootMatch?.[1] || '';
if (root !== 'attempt_completion' && root !== 'think' && root !== 'thinking') return xml;
// Helper to decode entities
const decodeEntities = (text: string): string =>
text
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&amp;/g, '&')
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'");
if (root === 'attempt_completion') {
// Find first <comment>...</comment>
const commentRegex = /<comment>([\s\S]*?)<\/comment>/i;
const match = xml.match(commentRegex);
if (!match) return xml;
const inner = match[1];
if (/<!\[CDATA\[/.test(inner)) {
// Already wrapped
return xml;
}
const decoded = decodeEntities(inner);
const replacement = `<comment><![CDATA[${decoded}]]></comment>`;
return xml.replace(commentRegex, replacement);
}
// Handle <think>...</think> or <thinking>...</thinking>
const rootRegex = new RegExp(`<${root}>([\\s\\S]*?)</${root}>`, 'i');
const rootMatchContent = xml.match(rootRegex);
if (!rootMatchContent) return xml;
const innerRoot = rootMatchContent[1];
if (/<!\[CDATA\[/.test(innerRoot)) {
return xml; // Already wrapped
}
const decodedRoot = decodeEntities(innerRoot);
const replacementRoot = `<${root}><![CDATA[${decodedRoot}]]></${root}>`;
return xml.replace(rootRegex, replacementRoot);
} catch {
// On any error, return original XML to avoid breaking flow
return xml;
}
}
/**
* Parse XML element into tool parameters object
* Converts XML structure to JavaScript object that matches tool parameter schema
* @param element - Root XML element containing tool parameters
* @returns Parameters object for tool execution
*/
private parseXMLParameters(element: Element): Record<string, unknown> {
const parameters: Record<string, unknown> = {};
// Special handling for thinking tool: if no child elements, use text content directly
if (element.tagName === 'thinking' && element.children.length === 0) {
const textContent = element.textContent?.trim() || '';
parameters.content = textContent;
return parameters;
}
// Process all child elements
for (const child of element.children) {
const paramName = child.tagName;
const paramValue = this.parseXMLValue(child);
// Handle arrays (multiple elements with same tag name)
if (parameters[paramName] !== undefined) {
// Convert to array if not already
if (!Array.isArray(parameters[paramName])) {
parameters[paramName] = [parameters[paramName]];
}
(parameters[paramName] as unknown[]).push(paramValue);
} else {
parameters[paramName] = paramValue;
}
}
// Apply array wrapper flattening
return this.flattenArrayWrappers(parameters);
}
/**
* Flatten array wrapper patterns in parsed parameters
* Converts structures like {notes: {note: [...]}} to {notes: [...]}
* @param parameters - Parsed parameters object
* @returns Parameters with flattened array wrappers
*/
private flattenArrayWrappers(parameters: Record<string, unknown>): Record<string, unknown> {
const flattened: Record<string, unknown> = {};
for (const [key, value] of Object.entries(parameters)) {
if (this.isArrayWrapperCandidate(key, value)) {
// This is an array wrapper - flatten it
const wrapperObj = value as Record<string, unknown>;
const innerKeys = Object.keys(wrapperObj);
if (innerKeys.length === 1) {
const innerKey = innerKeys[0];
const innerValue = wrapperObj[innerKey];
// Check if inner key is singular form of outer key
if (this.isSingularOf(innerKey, key)) {
// Flatten: {notes: {note: [...]}} → {notes: [...]}
flattened[key] = innerValue;
continue;
}
}
}
// No flattening needed, keep as is
flattened[key] = value;
}
return flattened;
}
/**
* Check if a value is a candidate for array wrapper flattening
* @param key - Parameter key (e.g., "notes")
* @param value - Parameter value to check
* @returns True if this looks like an array wrapper pattern
*/
private isArrayWrapperCandidate(key: string, value: unknown): boolean {
// Must be an object (not array, not primitive)
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
return false;
}
const obj = value as Record<string, unknown>;
const innerKeys = Object.keys(obj);
// Must have exactly one property
if (innerKeys.length !== 1) {
return false;
}
const innerKey = innerKeys[0];
const innerValue = obj[innerKey];
// Inner value should be an array or could become an array
// (single items are often converted to arrays by the parser)
return this.isSingularOf(innerKey, key) &&
(Array.isArray(innerValue) || typeof innerValue === 'object');
}
/**
* Check if one word is the singular form of another (simple heuristic)
* @param singular - Potential singular form (e.g., "note")
* @param plural - Potential plural form (e.g., "notes")
* @returns True if singular appears to be singular form of plural
*/
private isSingularOf(singular: string, plural: string): boolean {
// Simple heuristics for common English pluralization
if (plural === singular + 's') return true; // note → notes
if (plural === singular + 'es') return true; // box → boxes
if (plural.endsWith('ies') && singular.endsWith('y')) { // entry → entries
return plural === singular.slice(0, -1) + 'ies';
}
// Add more rules as needed for your specific use cases
return false;
}
/**
* Parse a single XML element value, handling different data types and structures
* @param element - XML element to parse
* @returns Parsed value (string, number, boolean, object, or array)
*/
private parseXMLValue(element: Element): unknown {
// If element has children, parse as object
if (element.children.length > 0) {
return this.parseXMLParameters(element);
}
// Get text content
const textContent = element.textContent?.trim() || '';
// Try to parse as number
if (/^-?\d+(\.\d+)?$/.test(textContent)) {
return parseFloat(textContent);
}
// Try to parse as boolean
if (textContent === 'true') return true;
if (textContent === 'false') return false;
// Return as string
return textContent;
}
/**
* Synchronize UI state after successful tool execution
* Uses the centralized refresh method from the project store
*/
private syncUIState(): void {
try {
// Use the centralized refresh method from the store
const storeActions = useProjectStore.getState();
if (storeActions.refreshProjectState) {
storeActions.refreshProjectState();
}
} catch (error) {
console.warn('Failed to sync UI state after XML tool execution:', error);
// Don't throw - UI sync failure shouldn't break tool execution
}
}
}
src/agent/llm/ClaudeProvider.ts
+229
View File
@@ -0,0 +1,229 @@
import { LLMProvider } from './LLMProvider';
import type { StreamChunk, LLMResponse } from './StreamingTypes';
import type { Message } from '../core/AgentState';
import { ConfigManager } from '../../core/config/ConfigManager';
/**
* Anthropic Claude API provider implementation
*/
export class ClaudeProvider extends LLMProvider {
readonly name = 'Claude';
private apiKey: string;
private model: string;
private baseURL: string = 'https://api.anthropic.com';
private apiEndpoint: string;
constructor() {
super();
const configManager = ConfigManager.instance();
this.apiKey = configManager.get('general.claude.api_key') as string;
this.model = configManager.get('general.claude.model') as string;
this.apiEndpoint = `${this.baseURL}/v1/messages`;
}
/**
* Convert internal messages to Claude's format
*/
private convertMessages(messages: Message[], systemPrompt?: string): {
system?: string;
messages: Array<{ role: 'user' | 'assistant'; content: string }>;
} {
const claudeMessages: Array<{ role: 'user' | 'assistant'; content: string }> = [];
claudeMessages.push(...messages.map(msg => ({ role: msg.role, content: msg.content })));
return {
system: systemPrompt,
messages: claudeMessages
};
}
/**
* Parse Claude's streaming response chunks
*/
private parseClaudeStreamChunk(line: string): { content?: string; isDone?: boolean } {
if (!line.startsWith('data: ')) {
return {};
}
const data = line.slice(6);
if (data === '[DONE]') {
return { isDone: true };
}
try {
const json = JSON.parse(data);
// Handle different Claude streaming event types
switch (json.type) {
case 'content_block_delta':
return {
content: json.delta?.text,
isDone: false
};
case 'message_stop':
return { isDone: true };
default:
return {};
}
} catch {
return {}; // Skip invalid JSON lines
}
}
async *generateStream(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): AsyncIterableIterator<StreamChunk> {
const { system, messages: claudeMessages } = this.convertMessages(messages, systemPrompt);
const requestBody: {
model: string;
max_tokens: number;
messages: Array<{ role: 'user' | 'assistant'; content: string }>;
stream: boolean;
system?: string;
tools?: Record<string, unknown>[];
} = {
model: this.model,
max_tokens: 8192,
messages: claudeMessages,
stream: true
};
if (system) {
requestBody.system = system;
}
if (tools && tools.length > 0) {
requestBody.tools = tools;
}
const response = await fetch(this.apiEndpoint, {
method: 'POST',
headers: {
'x-api-key': this.apiKey,
'Content-Type': 'application/json',
'anthropic-version': '2023-06-01'
},
body: JSON.stringify(requestBody),
mode: 'cors'
});
if (!response.ok) {
throw new Error(`Claude API error: ${response.status} ${response.statusText}`);
}
const reader = response.body?.getReader();
if (!reader) {
throw new Error('Failed to get response reader');
}
const decoder = new TextDecoder();
let buffer = '';
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n');
buffer = lines.pop() || '';
for (const line of lines) {
const trimmedLine = line.trim();
if (!trimmedLine) continue;
const parseResult = this.parseClaudeStreamChunk(trimmedLine);
if (parseResult.isDone) {
yield { type: 'done', content: '' };
return;
}
if (parseResult.content) {
yield {
type: 'text',
content: parseResult.content
};
}
}
}
} finally {
reader.releaseLock();
}
}
async generateCompletion(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): Promise<LLMResponse> {
const { system, messages: claudeMessages } = this.convertMessages(messages, systemPrompt);
const requestBody: {
model: string;
max_tokens: number;
messages: Array<{ role: 'user' | 'assistant'; content: string }>;
stream: boolean;
system?: string;
tools?: Record<string, unknown>[];
} = {
model: this.model,
max_tokens: 8192,
messages: claudeMessages,
stream: false
};
if (system) {
requestBody.system = system;
}
if (tools && tools.length > 0) {
requestBody.tools = tools;
}
const response = await fetch(this.apiEndpoint, {
method: 'POST',
headers: {
'x-api-key': this.apiKey,
'Content-Type': 'application/json',
'anthropic-version': '2023-06-01'
},
body: JSON.stringify(requestBody),
mode: 'cors'
});
if (!response.ok) {
throw new Error(`Claude API error: ${response.status} ${response.statusText}`);
}
const data = await response.json();
// Extract content from Claude's response format
const content = data.content
?.filter((block: { type: string }) => block.type === 'text')
?.map((block: { text: string }) => block.text)
?.join('') || '';
// Extract tool calls if present
const toolCalls = data.content
?.filter((block: { type: string }) => block.type === 'tool_use')
?.map((block: { id: string; name: string; input: Record<string, unknown> }) => ({
id: block.id,
name: block.name,
parameters: block.input
})) || [];
return {
content,
toolCalls: toolCalls.length > 0 ? toolCalls : undefined,
finished: data.stop_reason === 'end_turn' || data.stop_reason === 'tool_use'
};
}
}
src/agent/llm/GeminiProvider.ts
+245
View File
@@ -0,0 +1,245 @@
import { LLMProvider } from './LLMProvider';
import type { StreamChunk, LLMResponse } from './StreamingTypes';
import type { Message } from '../core/AgentState';
import { ConfigManager } from '../../core/config/ConfigManager';
/**
* Google Gemini API provider implementation
*/
export class GeminiProvider extends LLMProvider {
readonly name = 'Gemini';
private apiKey: string;
private model: string;
private baseURL: string = 'https://generativelanguage.googleapis.com';
private apiEndpoint: string;
constructor() {
super();
const configManager = ConfigManager.instance();
this.apiKey = configManager.get('general.gemini.api_key') as string;
this.model = configManager.get('general.gemini.model') as string;
this.apiEndpoint = `${this.baseURL}/v1beta/models/${this.model}`;
}
/**
* Convert internal messages to Gemini's format
*/
private convertMessages(messages: Message[], systemPrompt?: string): {
systemInstruction?: { parts: Array<{ text: string }> };
contents: Array<{ role: 'user' | 'model'; parts: Array<{ text: string }> }>;
} {
const geminiMessages: Array<{ role: 'user' | 'model'; parts: Array<{ text: string }> }> = [];
for (const msg of messages) {
let role: 'user' | 'model';
if (msg.role === 'assistant') {
role = 'model';
} else {
// Treat system and user messages as 'user' role
role = 'user';
}
geminiMessages.push({
role,
parts: [{ text: msg.content }]
});
}
const result: {
systemInstruction?: { parts: Array<{ text: string }> };
contents: Array<{ role: 'user' | 'model'; parts: Array<{ text: string }> }>;
} = {
contents: geminiMessages
};
if (systemPrompt) {
result.systemInstruction = {
parts: [{ text: systemPrompt }]
};
}
return result;
}
/**
* Parse Gemini's streaming response chunks
*/
private parseGeminiStreamChunk(chunk: string): { content?: string; isDone?: boolean } {
try {
const json = JSON.parse(chunk.trim());
// Gemini streaming format
if (json.candidates && json.candidates.length > 0) {
const candidate = json.candidates[0];
// Check if generation is finished
if (candidate.finishReason && candidate.finishReason !== 'STOP') {
return { isDone: true };
}
// Extract text content
const content = candidate.content?.parts?.[0]?.text;
if (content) {
return { content, isDone: false };
}
}
// Check for explicit done signal
if (json.done === true) {
return { isDone: true };
}
return {};
} catch {
return {}; // Skip invalid JSON
}
}
async *generateStream(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): AsyncIterableIterator<StreamChunk> {
const { systemInstruction, contents } = this.convertMessages(messages, systemPrompt);
const requestBody: {
contents: Array<{ role: 'user' | 'model'; parts: Array<{ text: string }> }>;
generationConfig: { temperature: number; maxOutputTokens: number };
systemInstruction?: { parts: Array<{ text: string }> };
tools?: Record<string, unknown>[];
} = {
contents,
generationConfig: {
temperature: 0.7,
maxOutputTokens: 8192,
}
};
if (systemInstruction) {
requestBody.systemInstruction = systemInstruction;
}
if (tools && tools.length > 0) {
requestBody.tools = tools;
}
const response = await fetch(`${this.apiEndpoint}:streamGenerateContent?key=${this.apiKey}`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(requestBody),
});
if (!response.ok) {
throw new Error(`Gemini API error: ${response.status} ${response.statusText}`);
}
const reader = response.body?.getReader();
if (!reader) {
throw new Error('Failed to get response reader');
}
const decoder = new TextDecoder();
let buffer = '';
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
// Gemini sends JSON objects separated by newlines
const lines = buffer.split('\n');
buffer = lines.pop() || '';
for (const line of lines) {
const trimmedLine = line.trim();
if (!trimmedLine) continue;
const parseResult = this.parseGeminiStreamChunk(trimmedLine);
if (parseResult.isDone) {
yield { type: 'done', content: '' };
return;
}
if (parseResult.content) {
yield {
type: 'text',
content: parseResult.content
};
}
}
}
} finally {
reader.releaseLock();
}
}
async generateCompletion(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): Promise<LLMResponse> {
const { systemInstruction, contents } = this.convertMessages(messages, systemPrompt);
const requestBody: {
contents: Array<{ role: 'user' | 'model'; parts: Array<{ text: string }> }>;
generationConfig: { temperature: number; maxOutputTokens: number };
systemInstruction?: { parts: Array<{ text: string }> };
tools?: Record<string, unknown>[];
} = {
contents,
generationConfig: {
temperature: 0.7,
maxOutputTokens: 8192,
}
};
if (systemInstruction) {
requestBody.systemInstruction = systemInstruction;
}
if (tools && tools.length > 0) {
requestBody.tools = tools;
}
const response = await fetch(`${this.apiEndpoint}:generateContent?key=${this.apiKey}`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(requestBody),
});
if (!response.ok) {
throw new Error(`Gemini API error: ${response.status} ${response.statusText}`);
}
const data = await response.json();
// Extract content from Gemini's response format
const candidate = data.candidates?.[0];
const content = candidate?.content?.parts?.[0]?.text || '';
// Extract tool calls if present (Gemini format)
const toolCalls = candidate?.content?.parts
?.filter((part: { functionCall?: unknown }) => part.functionCall)
?.map((part: { functionCall: { name: string; args: Record<string, unknown> } }) => ({
id: `tool_${Date.now()}_${Math.random().toString(36).substring(2, 9)}`, // Generate ID
name: part.functionCall.name,
parameters: part.functionCall.args
})) || [];
return {
content,
toolCalls: toolCalls.length > 0 ? toolCalls : undefined,
finished: candidate?.finishReason === 'STOP' || candidate?.finishReason === 'MAX_TOKENS'
};
}
}
src/agent/llm/LLMProvider.ts
+33
View File
@@ -0,0 +1,33 @@
import type { StreamChunk, LLMResponse } from './StreamingTypes';
import type { Message } from '../core/AgentState';
/**
* Abstract interface for LLM providers
*/
export abstract class LLMProvider {
abstract name: string;
/**
* Generate a streaming response from the LLM
* @param messages The full conversation history with preserved roles
* @param systemPrompt The system prompt (optional, can be included in messages)
* @param tools Available tools (optional for now)
*/
abstract generateStream(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): AsyncIterableIterator<StreamChunk>;
/**
* Generate a complete response from the LLM (non-streaming)
* @param messages The full conversation history with preserved roles
* @param systemPrompt The system prompt (optional, can be included in messages)
* @param tools Available tools (optional for now)
*/
abstract generateCompletion(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): Promise<LLMResponse>;
}
src/agent/llm/OpenAIProvider.ts
+454
View File
@@ -0,0 +1,454 @@
import { LLMProvider } from './LLMProvider';
import type { StreamChunk, LLMResponse } from './StreamingTypes';
import type { Message } from '../core/AgentState';
import { ConfigManager } from '../../core/config/ConfigManager';
import { URL_CONSTANTS } from '../../constants/coreConstants';
/**
* OpenAI API provider implementation
*/
export class OpenAIProvider extends LLMProvider {
readonly name = 'OpenAI';
private apiKey: string;
private model: string;
private flexMode: boolean = false;
private baseURL: string;
private isCompatibleProvider: boolean;
private apiEndpoint: string;
private isOllamaFormat: boolean | null = null; // Detected at runtime
constructor() {
super();
const configManager = ConfigManager.instance();
const llmProvider = configManager.get('general.llm_provider') as string;
this.isCompatibleProvider = llmProvider === 'openai_compatible';
// Set API key, model, base URL, and endpoint based on provider type
if (this.isCompatibleProvider) {
this.apiKey = configManager.get('general.openai_compatible.api_key') as string;
this.model = configManager.get('general.openai_compatible.model') as string;
this.baseURL = configManager.get('general.openai_compatible.base_url') as string;
// For compatible providers, use the full URL as provided (assume it includes the endpoint)
// Common patterns: http://localhost:11434/api/chat (Ollama), https://api.openrouter.ai/v1 (OpenRouter)
this.apiEndpoint = this.baseURL;
this.flexMode = false; // Not applicable to compatible providers
} else {
this.apiKey = configManager.get('general.openai.api_key') as string;
this.model = configManager.get('general.openai.model') as string;
this.flexMode = (configManager.get('general.openai.flex') as boolean) === true;
this.baseURL = URL_CONSTANTS.DEFAULT_OPENAI_BASE_URL;
this.apiEndpoint = `${this.baseURL}/chat/completions`;
}
}
/**
* Detect if the response uses Ollama's raw JSON format or OpenAI's SSE format
*/
private detectStreamFormat(firstChunk: string): boolean {
// If it starts with "data: ", it's OpenAI SSE format
if (firstChunk.trim().startsWith('data: ')) {
return false; // Not Ollama format
}
// Try to parse as JSON - if successful and has 'done' field, it's Ollama format
try {
const json = JSON.parse(firstChunk.trim());
return typeof json.done === 'boolean';
} catch {
return false; // Not valid JSON, assume OpenAI format
}
}
/**
* Parse Ollama's raw JSON chunk format
*/
private parseOllamaChunk(chunk: string): { thinking?: string; content?: string; isDone?: boolean; toolCalls?: Array<{ name: string; arguments: Record<string, unknown> }> } {
try {
const json = JSON.parse(chunk.trim());
const thinking: string | undefined = json.message?.thinking;
const content: string | undefined = json.message?.content || json.response; // Handle both chat and completion formats
type WireToolCall = { function?: { name?: unknown; arguments?: unknown } };
const toolCallsRaw: unknown[] | undefined = json.message?.tool_calls as unknown[] | undefined;
const toolCalls = Array.isArray(toolCallsRaw)
? (toolCallsRaw
.map((tc: unknown) => {
const wire = tc as WireToolCall;
const fn = wire?.function;
if (!fn || typeof fn.name !== 'string') return null;
const args = fn.arguments;
if (args === null || typeof args !== 'object' || Array.isArray(args)) return null;
return { name: fn.name, arguments: args as Record<string, unknown> };
})
.filter((v): v is { name: string; arguments: Record<string, unknown> } => v !== null))
: undefined;
return {
thinking,
content,
isDone: json.done === true,
toolCalls
};
} catch {
return {}; // Invalid JSON, return empty object
}
}
/**
* Parse OpenAI's SSE format chunk
*/
private parseOpenAIChunk(line: string): { thinking?: string; content?: string; isDone?: boolean; toolCallDelta?: Array<{ index?: number; function?: { name?: string; arguments?: string } }> } {
if (!line.startsWith('data: ')) {
return {};
}
const data = line.slice(6);
if (data === '[DONE]') {
return { isDone: true };
}
try {
const json = JSON.parse(data);
const delta = json.choices?.[0]?.delta;
const thinking: string | undefined = delta?.thinking; // Some providers may stream "thinking"
const content: string | undefined = delta?.content;
const tcd = delta?.tool_calls;
const toolCallDelta: Array<{ index?: number; function?: { name?: string; arguments?: string } }> | undefined = Array.isArray(tcd)
? (tcd as Array<{ index?: number; function?: { name?: string; arguments?: string } }>)
: undefined;
return { thinking, content, isDone: false, toolCallDelta };
} catch {
return {}; // Skip invalid JSON lines
}
}
async *generateStream(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): AsyncIterableIterator<StreamChunk> {
// Build OpenAI messages array with role preservation
const openAIMessages: Array<{ role: string; content: string }> = [];
// Add system prompt if provided
if (systemPrompt) {
openAIMessages.push({ role: 'system', content: systemPrompt });
}
// Add conversation history with preserved roles
openAIMessages.push(...messages.map(msg => ({
role: msg.role,
content: msg.content
})));
const response = await fetch(this.apiEndpoint, {
method: 'POST',
headers: {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: this.model,
...(this.flexMode && !this.isCompatibleProvider ? { service_tier: 'flex' } : {}),
messages: openAIMessages,
stream: true,
tools: tools || undefined
}),
});
if (!response.ok) {
throw new Error(`OpenAI API error: ${response.status} ${response.statusText}`);
}
const reader = response.body?.getReader();
if (!reader) {
throw new Error('Failed to get response reader');
}
const decoder = new TextDecoder();
let buffer = '';
let firstChunkProcessed = false;
let lastSegmentType: 'thinking' | 'content' | null = null;
const pendingFunctionCalls: Array<{ name: string; arguments: Record<string, unknown> }> = [];
const openAIToolCallBuilders: Record<number, { name?: string; argumentsText: string }> = {};
const escapeXml = (text: string): string =>
String(text)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&apos;');
const functionCallToXml = (name: string, args: Record<string, unknown>): string => {
const keys = Object.keys(args);
const inner = keys
.map((k) => {
const value = (args as Record<string, unknown>)[k];
const text = typeof value === 'string' ? value : JSON.stringify(value);
return `<${k}>${escapeXml(text)}</${k}>`;
})
.join('\n');
return `<${name}>\n${inner}\n</${name}>`;
};
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
// For Ollama format, we need to split by newlines for JSON objects
// For OpenAI format, we also split by newlines for SSE
const lines = buffer.split('\n');
buffer = lines.pop() || '';
for (const line of lines) {
const trimmedLine = line.trim();
if (!trimmedLine) continue;
// Detect format on first non-empty chunk
if (!firstChunkProcessed) {
this.isOllamaFormat = this.detectStreamFormat(trimmedLine);
firstChunkProcessed = true;
}
if (this.isOllamaFormat) {
const { thinking, content, isDone, toolCalls } = this.parseOllamaChunk(trimmedLine);
if (isDone) {
// Append any pending or current tool calls as XML before finishing
const allToolCalls = [
...pendingFunctionCalls,
...(toolCalls || [])
];
if (allToolCalls.length > 0) {
const xmlBlocks = allToolCalls
.map((tc) => `\n${functionCallToXml(tc.name, tc.arguments)}\n`)
.join('');
yield { type: 'text', content: xmlBlocks };
}
yield { type: 'done', content: '' };
return;
}
if (typeof thinking === 'string' && thinking.length > 0) {
if (lastSegmentType && lastSegmentType !== 'thinking') {
yield { type: 'text', content: '\n\n' };
}
yield { type: 'text', content: thinking };
lastSegmentType = 'thinking';
}
if (typeof content === 'string' && content.length > 0) {
if (lastSegmentType && lastSegmentType !== 'content') {
yield { type: 'text', content: '\n\n' };
}
yield { type: 'text', content: content };
lastSegmentType = 'content';
}
if (Array.isArray(toolCalls) && toolCalls.length > 0) {
// Accumulate and append at the end of stream
pendingFunctionCalls.push(...toolCalls);
}
continue;
}
const { thinking, content, isDone, toolCallDelta } = this.parseOpenAIChunk(trimmedLine);
if (isDone) {
// Finalize any accumulated OpenAI tool calls and emit XML
const finalizedToolCalls: Array<{ name: string; arguments: Record<string, unknown> }> = [];
for (const indexStr of Object.keys(openAIToolCallBuilders)) {
const idx = Number(indexStr);
const builder = openAIToolCallBuilders[idx];
if (!builder || !builder.name) continue;
let argsObj: Record<string, unknown> | null = null;
if (builder.argumentsText) {
try {
argsObj = JSON.parse(builder.argumentsText);
} catch {
argsObj = null;
}
}
if (argsObj) {
finalizedToolCalls.push({ name: builder.name, arguments: argsObj });
}
}
const allToolCalls = [...pendingFunctionCalls, ...finalizedToolCalls];
if (allToolCalls.length > 0) {
const xmlBlocks = allToolCalls
.map((tc) => `\n${functionCallToXml(tc.name, tc.arguments)}\n`)
.join('');
yield { type: 'text', content: xmlBlocks };
}
yield { type: 'done', content: '' };
return;
}
if (typeof thinking === 'string' && thinking.length > 0) {
if (lastSegmentType && lastSegmentType !== 'thinking') {
yield { type: 'text', content: '\n\n' };
}
yield { type: 'text', content: thinking };
lastSegmentType = 'thinking';
}
if (typeof content === 'string' && content.length > 0) {
if (lastSegmentType && lastSegmentType !== 'content') {
yield { type: 'text', content: '\n\n' };
}
yield { type: 'text', content };
lastSegmentType = 'content';
}
if (Array.isArray(toolCallDelta) && toolCallDelta.length > 0) {
for (const tc of toolCallDelta) {
const index: number = typeof tc.index === 'number' ? tc.index : 0;
if (!openAIToolCallBuilders[index]) {
openAIToolCallBuilders[index] = { argumentsText: '' };
}
const fn = tc.function;
if (fn) {
if (typeof fn.name === 'string') {
openAIToolCallBuilders[index].name = fn.name;
}
if (typeof fn.arguments === 'string') {
openAIToolCallBuilders[index].argumentsText += fn.arguments;
}
}
}
}
}
}
} finally {
reader.releaseLock();
}
}
async generateCompletion(
messages: Message[],
systemPrompt?: string,
tools?: Record<string, unknown>[]
): Promise<LLMResponse> {
// Build OpenAI messages array with role preservation
const openAIMessages: Array<{ role: string; content: string }> = [];
// Add system prompt if provided
if (systemPrompt) {
openAIMessages.push({ role: 'system', content: systemPrompt });
}
// Add conversation history with preserved roles
openAIMessages.push(...messages.map(msg => ({
role: msg.role,
content: msg.content
})));
const response = await fetch(this.apiEndpoint, {
method: 'POST',
headers: {
'Authorization': `Bearer ${this.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: this.model,
...(this.flexMode && !this.isCompatibleProvider ? { service_tier: 'flex' } : {}),
messages: openAIMessages,
stream: false,
tools: tools || undefined
}),
});
if (!response.ok) {
throw new Error(`OpenAI API error: ${response.status} ${response.statusText}`);
}
const data = await response.json();
// Handle different response formats
const escapeXml = (text: string): string =>
String(text)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&apos;');
const functionCallToXml = (name: string, args: Record<string, unknown>): string => {
const keys = Object.keys(args);
const inner = keys
.map((k) => {
const value = (args as Record<string, unknown>)[k];
const text = typeof value === 'string' ? value : JSON.stringify(value);
return `<${k}>${escapeXml(text)}</${k}>`;
})
.join('\n');
return `<${name}>\n${inner}\n</${name}>`;
};
if (data.message) {
// Ollama/compatible format
const thinking: string = data.message.thinking || '';
const contentText: string = data.message.content || data.response || '';
const textCombined = thinking && contentText ? `${thinking}\n\n${contentText}` : (thinking || contentText);
type WireToolCall = { function?: { name?: unknown; arguments?: unknown } };
const toolCallsRaw: unknown[] | undefined = data.message.tool_calls as unknown[] | undefined;
const toolCalls: Array<{ name: string; arguments: Record<string, unknown> }> = Array.isArray(toolCallsRaw)
? (toolCallsRaw
.map((tc: unknown) => {
const wire = tc as WireToolCall;
const fn = wire?.function;
if (!fn || typeof fn.name !== 'string') return null;
const args = fn.arguments;
if (args === null || typeof args !== 'object' || Array.isArray(args)) return null;
return { name: fn.name, arguments: args as Record<string, unknown> };
})
.filter((v): v is { name: string; arguments: Record<string, unknown> } => v !== null))
: [];
const xmlBlocks = toolCalls.length > 0
? toolCalls.map((tc) => `\n${functionCallToXml(tc.name, tc.arguments)}\n`).join('')
: '';
return {
content: `${textCombined}${xmlBlocks}`,
toolCalls: data.message.tool_calls || undefined,
finished: data.done === true || data.done_reason === 'stop'
};
} else {
// OpenAI format
const choice = data.choices?.[0];
const thinking: string = choice?.message?.thinking || '';
const contentText: string = choice?.message?.content || '';
const textCombined = thinking && contentText ? `${thinking}\n\n${contentText}` : (thinking || contentText);
type WireToolCall2 = { function?: { name?: unknown; arguments?: unknown } };
const toolCallsRaw: unknown[] | undefined = choice?.message?.tool_calls as unknown[] | undefined;
const toolCalls: Array<{ name: string; arguments: Record<string, unknown> }> = Array.isArray(toolCallsRaw)
? (toolCallsRaw
.map((tc: unknown) => {
const wire = tc as WireToolCall2;
const fn = wire?.function;
if (!fn || typeof fn.name !== 'string') return null;
const args = fn.arguments;
if (args === null || typeof args !== 'object' || Array.isArray(args)) return null;
return { name: fn.name, arguments: args as Record<string, unknown> };
})
.filter((v): v is { name: string; arguments: Record<string, unknown> } => v !== null))
: [];
const xmlBlocks = toolCalls.length > 0
? toolCalls.map((tc) => `\n${functionCallToXml(tc.name, tc.arguments)}\n`).join('')
: '';
return {
content: `${textCombined}${xmlBlocks}`,
toolCalls: choice?.message?.tool_calls || undefined,
finished: choice?.finish_reason === 'stop' || choice?.finish_reason === 'tool_calls'
};
}
}
}
src/agent/llm/StreamingTypes.ts
+27
View File
@@ -0,0 +1,27 @@
/**
* Types for streaming LLM responses and tool execution
*/
import type { ToolResult } from '../tools/BaseTool';
// Re-export for convenience
export type { ToolResult };
export interface ToolInvocation {
id: string;
name: string;
parameters: Record<string, unknown>;
}
export interface StreamChunk {
type: 'text' | 'tool_call' | 'tool_result' | 'done';
content: string;
toolCall?: ToolInvocation;
toolResult?: ToolResult;
}
export interface LLMResponse {
content: string;
toolCalls?: ToolInvocation[];
finished: boolean;
}
src/agent/tools/AddNotesTool.ts
+235
View File
@@ -0,0 +1,235 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
import { CreateNotesCommand } from '../../core/commands/note/CreateNotesCommand';
import type { NoteCreationData } from '../../core/commands/note/CreateNotesCommand';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { useProjectStore } from '../../stores/projectStore';
import { KGCore } from '../../core/KGCore';
/**
* Tool for adding notes to MIDI regions
* Integrates with the existing command system for undo/redo support
*/
export class AddNotesTool extends BaseTool {
readonly name = 'add_notes';
readonly description = 'Create one or more MIDI notes in the current region. Each note requires pitch (e.g., "C4", "F#3"), start_beat (beat position), and length (duration in beats).';
readonly parameters: Record<string, ToolParameter> = {
notes: {
type: 'array',
description: 'Array of notes to create',
required: true,
items: {
type: 'object',
description: 'A MIDI note definition',
properties: {
pitch: {
type: 'string',
description: 'Note pitch in scientific notation (e.g., "C4", "F#3", "Bb2")',
required: true
},
start_beat: {
type: 'number',
description: 'Start position in beats (e.g., 0, 1.5, 2)',
required: true
},
length: {
type: 'number',
description: 'Note duration in beats (e.g., 1, 0.5, 4)',
required: true
},
velocity: {
type: 'number',
description: 'Note velocity (1-127, default: 127)',
required: false
}
}
}
},
region_id: {
type: 'string',
description: 'ID of the region to add notes to. If not provided, uses the currently selected region.',
required: false
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Validate parameters
this.validateParameters(params);
const notes = params.notes as Array<{
pitch: string;
start_beat: number;
length: number;
velocity?: number;
}>;
const regionId = params.region_id as string | undefined;
// Find the target region
const targetRegion = this.findTargetRegion(regionId);
if (!targetRegion) {
return this.createErrorResult(
regionId
? `Region with ID "${regionId}" not found or is not a MIDI region`
: 'No active or selected MIDI region found. Please open the piano roll with a region or select a MIDI region first.'
);
}
// Validate and convert notes to creation data
const noteCreationData: NoteCreationData[] = [];
const createdNotes: Array<{ pitch: string; start_beat: number; length: number }> = [];
for (const note of notes) {
try {
const midiPitch = this.convertPitchToMidi(note.pitch);
const velocity = note.velocity ?? 127;
// Validate velocity range
if (velocity < 1 || velocity > 127) {
return this.createErrorResult(`Invalid velocity ${velocity}. Must be between 1 and 127.`);
}
// Validate beat positions
if (note.start_beat < 0) {
return this.createErrorResult(`Invalid start_beat ${note.start_beat}. Must be >= 0.`);
}
if (note.length <= 0) {
return this.createErrorResult(`Invalid length ${note.length}. Must be > 0.`);
}
// Adjust note position relative to region's start beat
const regionStartBeat = targetRegion.getStartFromBeat();
const adjustedStartBeat = note.start_beat - regionStartBeat;
const adjustedEndBeat = adjustedStartBeat + note.length;
// Create note creation data
noteCreationData.push({
regionId: targetRegion.getId(),
startBeat: adjustedStartBeat,
endBeat: adjustedEndBeat,
pitch: midiPitch,
velocity
});
createdNotes.push({
pitch: note.pitch,
start_beat: note.start_beat,
length: note.length
});
} catch (error) {
return this.createErrorResult(`Invalid note pitch "${note.pitch}": ${error}`);
}
}
// Execute the bulk note creation command
const command = new CreateNotesCommand(noteCreationData);
await this.executeCommand(command);
// Create success message
const noteCount = createdNotes.length;
const noteList = createdNotes
.map(note => `${note.pitch} (beat ${note.start_beat}, length ${note.length})`)
.join(', ');
return this.createSuccessResult(
`Successfully created ${noteCount} note${noteCount > 1 ? 's' : ''}: ${noteList}`
);
} catch (error) {
return this.createErrorResult(`Failed to create notes: ${error}`);
}
}
/**
* Find the target region for note creation
* Priority: 1) Specified regionId, 2) Active piano roll region, 3) Selected regions, 4) Error if none found
*/
private findTargetRegion(regionId?: string): KGMidiRegion | null {
const project = this.getCurrentProject();
const tracks = project.getTracks();
if (regionId) {
// Find specific region by ID
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region && region instanceof KGMidiRegion) {
return region;
}
}
return null;
} else {
// Smart region finding: try different sources in priority order
// 1. Try active piano roll region
const storeState = useProjectStore.getState();
if (storeState.activeRegionId) {
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === storeState.activeRegionId);
if (region && region instanceof KGMidiRegion) {
return region;
}
}
}
// 2. Try selected regions
const core = this.getKGCore();
const selectedItems = core.getSelectedItems();
for (const item of selectedItems) {
if (item instanceof KGMidiRegion) {
return item;
}
}
// 3. No fallback - return null to trigger error
return null;
}
}
/**
* Get KGCore instance for selection access
*/
private getKGCore() {
return KGCore.instance();
}
/**
* Convert pitch string to MIDI note number
* Supports formats like: C4, F#3, Bb2, C#5
*/
private convertPitchToMidi(pitch: string): number {
const match = pitch.match(/^([A-G])([#b]?)(\d+)$/);
if (!match) {
throw new Error(`Invalid pitch format "${pitch}". Use format like "C4", "F#3", "Bb2"`);
}
const [, noteName, accidental, octaveStr] = match;
const octave = parseInt(octaveStr);
// Base MIDI notes for C octave (C4 = 60)
const noteOffsets: Record<string, number> = {
'C': 0, 'D': 2, 'E': 4, 'F': 5, 'G': 7, 'A': 9, 'B': 11
};
let midiNote = (octave + 1) * 12 + noteOffsets[noteName];
// Apply accidentals
if (accidental === '#') {
midiNote += 1;
} else if (accidental === 'b') {
midiNote -= 1;
}
// Validate MIDI range
if (midiNote < 0 || midiNote > 127) {
throw new Error(`Note "${pitch}" is out of MIDI range (0-127)`);
}
return midiNote;
}
}
src/agent/tools/AttemptCompletionTool.ts
+49
View File
@@ -0,0 +1,49 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
import { AgentCore } from '../core/AgentCore';
/**
* Tool for signaling task completion
* This is a pure agent state tool that doesn't modify the DAW but signals
* to the agent system that the user's requested task has been completed
*/
export class AttemptCompletionTool extends BaseTool {
readonly name = 'attempt_completion';
readonly description = 'Signal that the current user task is fully complete. Only use this when you have successfully fulfilled all aspects of the user\'s request.';
readonly parameters: Record<string, ToolParameter> = {
comment: {
type: 'string',
description: 'A brief comment describing what was completed and any relevant details about the task fulfillment.',
required: true
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Validate parameters
this.validateParameters(params);
const comment = params.comment as string;
// Validate comment is not empty
if (!comment.trim()) {
return this.createErrorResult('Comment cannot be empty. Please provide a meaningful completion summary.');
}
// Get current agent state and update task completion status
const agentCore = AgentCore.instance();
const agentState = agentCore.getAgentState();
// Mark that we're no longer working on a task
agentState.setIsWorkingOnTask(false);
return this.createSuccessResult(
`Task completed: ${comment}. Agent task status updated to not working.`
);
} catch (error) {
return this.createErrorResult(`Failed to mark task as complete: ${error}`);
}
}
}
src/agent/tools/BaseTool.ts
+162
View File
@@ -0,0 +1,162 @@
import { KGCommand } from '../../core/commands/KGCommand';
import { KGCore } from '../../core/KGCore';
/**
* Result of tool execution
*/
export interface ToolResult {
success: boolean;
result: string;
}
/**
* Parameter definition for tool parameters
*/
export interface ToolParameter {
type: 'string' | 'number' | 'boolean' | 'array' | 'object';
description: string;
required?: boolean;
items?: ToolParameter; // For array types
properties?: Record<string, ToolParameter>; // For object types
}
/**
* Tool definition schema
*/
export interface ToolDefinition {
name: string;
description: string;
parameters: Record<string, ToolParameter>;
}
/**
* Abstract base class for all agent tools
* Provides integration with the existing command system and core architecture
*/
export abstract class BaseTool {
abstract readonly name: string;
abstract readonly description: string;
abstract readonly parameters: Record<string, ToolParameter>;
/**
* Execute the tool with given parameters
* @param params Tool parameters
* @returns Promise resolving to tool execution result
*/
abstract execute(params: Record<string, unknown>): Promise<ToolResult>;
/**
* Get the tool definition in OpenAI function calling format
*/
getDefinition(): ToolDefinition {
return {
name: this.name,
description: this.description,
parameters: this.parameters
};
}
/**
* Validate parameters against the tool's parameter schema
* @param params Parameters to validate
* @returns True if valid, throws error if invalid
*/
protected validateParameters(params: Record<string, unknown>): boolean {
for (const [paramName, paramDef] of Object.entries(this.parameters)) {
const value = params[paramName];
// Check required parameters
if (paramDef.required && (value === undefined || value === null)) {
throw new Error(`Required parameter '${paramName}' is missing`);
}
// Skip type checking for undefined optional parameters
if (value === undefined || value === null) {
continue;
}
// Type validation
if (!this.validateParameterType(value, paramDef)) {
throw new Error(`Parameter '${paramName}' has invalid type. Expected: ${paramDef.type}`);
}
}
return true;
}
/**
* Validate a single parameter value against its type definition
*/
private validateParameterType(value: unknown, paramDef: ToolParameter): boolean {
switch (paramDef.type) {
case 'string':
return typeof value === 'string';
case 'number':
return typeof value === 'number';
case 'boolean':
return typeof value === 'boolean';
case 'array':
if (!Array.isArray(value)) return false;
if (paramDef.items) {
return value.every(item => this.validateParameterType(item, paramDef.items!));
}
return true;
case 'object':
if (typeof value !== 'object' || value === null || Array.isArray(value)) {
return false;
}
if (paramDef.properties) {
const obj = value as Record<string, unknown>;
for (const [propName, propDef] of Object.entries(paramDef.properties)) {
if (propDef.required && !(propName in obj)) {
return false;
}
if (propName in obj && !this.validateParameterType(obj[propName], propDef)) {
return false;
}
}
}
return true;
default:
return false;
}
}
/**
* Execute a command through the existing command system
* This provides undo/redo functionality and proper state management
* @param command Command to execute
*/
protected async executeCommand(command: KGCommand): Promise<void> {
const core = KGCore.instance();
return core.executeCommand(command);
}
/**
* Get the current project from KGCore
*/
protected getCurrentProject() {
const core = KGCore.instance();
return core.getCurrentProject();
}
/**
* Create a successful tool result
*/
protected createSuccessResult(result: string): ToolResult {
return {
success: true,
result
};
}
/**
* Create a failed tool result
*/
protected createErrorResult(result: string): ToolResult {
return {
success: false,
result
};
}
}
src/agent/tools/ReadMusicTool.ts
+299
View File
@@ -0,0 +1,299 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
import { KGMidiTrack } from '../../core/track/KGMidiTrack';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { convertRegionToABCNotation } from '../../util/abcNotationUtil';
import { KEY_SIGNATURE_MAP } from '../../constants/coreConstants';
import { useProjectStore } from '../../stores/projectStore';
import { KGRegion } from '../../core/region/KGRegion';
import { KGCore } from '../../core/KGCore';
/**
* Tool for reading music content from the project
* Provides read-only access to project data and converts to ABC notation
*/
export class ReadMusicTool extends BaseTool {
readonly name = 'read_music';
readonly description = 'Read the music content from a specific track or all tracks, returning the content in ABC notation format.';
readonly parameters: Record<string, ToolParameter> = {
track_id: {
type: 'string',
description: 'The track ID to read, or "all" to read all tracks. If not provided, reads the first available track.',
required: false
},
start_beat: {
type: 'number',
description: 'Start beat position to read from (default: 0)',
required: false
},
length: {
type: 'number',
description: 'Length in beats to read (default: entire track/project)',
required: false
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Validate parameters
this.validateParameters(params);
const trackId = params.track_id as string | undefined;
const startBeat = (params.start_beat as number) || 0;
const length = params.length as number | undefined;
const project = this.getCurrentProject();
const tracks = project.getTracks();
if (tracks.length === 0) {
return this.createErrorResult('No tracks found in the project');
}
// Validate start_beat
if (startBeat < 0) {
return this.createErrorResult(`Invalid start_beat ${startBeat}. Must be >= 0.`);
}
// Validate length
if (length !== undefined && length <= 0) {
return this.createErrorResult(`Invalid length ${length}. Must be > 0.`);
}
// Get project settings for bar rounding
const timeSignature = project.getTimeSignature();
const beatsPerBar = timeSignature.numerator;
// Round startBeat to floor bar beats and calculate endBeat
const roundedStartBeat = Math.floor(startBeat / beatsPerBar) * beatsPerBar;
const rawEndBeat = length !== undefined ? startBeat + length : undefined;
const roundedEndBeat = rawEndBeat !== undefined ? Math.ceil(rawEndBeat / beatsPerBar) * beatsPerBar : undefined;
let abcOutput = '';
if (!trackId || trackId === '' || trackId === 'all') {
// Read all tracks
const midiTracks = tracks.filter(track => track instanceof KGMidiTrack) as KGMidiTrack[];
abcOutput = this.generateAllTracksABC(midiTracks, roundedStartBeat, roundedEndBeat);
} else {
// Read specific track or first available track
const targetTrack = trackId
? tracks.find(t => t.getId().toString() === trackId)
: tracks[0];
if (!targetTrack) {
return this.createErrorResult(
trackId
? `Track with ID "${trackId}" not found`
: 'No tracks available'
);
}
if (!(targetTrack instanceof KGMidiTrack)) {
return this.createErrorResult(`Track "${targetTrack.getName()}" is not a MIDI track`);
}
abcOutput = this.generateSingleTrackABC(targetTrack, roundedStartBeat, roundedEndBeat);
}
return this.createSuccessResult(abcOutput);
} catch (error) {
return this.createErrorResult(`Failed to read music: ${error}`);
}
}
/**
* Get KGCore instance
*/
private getKGCore(): KGCore {
return KGCore.instance();
}
/**
* Find the track that contains the active piano roll region or first selected region
*/
private findTrackToSkip(tracks: KGMidiTrack[]): KGMidiTrack | null {
try {
const store = useProjectStore.getState();
const core = this.getKGCore();
// First check for active piano roll region
if (store.activeRegionId) {
const activeRegion = this.findRegionById(store.activeRegionId, tracks);
if (activeRegion) {
const track = this.findTrackByRegion(activeRegion, tracks);
return track;
}
}
// Then check for selected regions
const selectedItems = core.getSelectedItems();
const selectedRegion = selectedItems.find((item: unknown) => item instanceof KGRegion) as KGRegion;
if (selectedRegion) {
const track = this.findTrackByRegion(selectedRegion, tracks);
return track;
}
return null;
} catch (error) {
console.error('Error finding track to skip:', error);
return null;
}
}
/**
* Find a region by ID across all tracks
*/
private findRegionById(regionId: string, tracks: KGMidiTrack[]): KGRegion | null {
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region) {
return region;
}
}
return null;
}
/**
* Find track that contains the given region
*/
private findTrackByRegion(region: KGRegion, tracks: KGMidiTrack[]): KGMidiTrack | null {
return tracks.find(track => track.getRegions().includes(region)) || null;
}
/**
* Generate ABC notation for all tracks
*/
private generateAllTracksABC(tracks: KGMidiTrack[], startBeat: number, endBeat?: number): string {
const midiTracks = tracks.filter(track => track instanceof KGMidiTrack);
if (midiTracks.length === 0) {
return 'No MIDI tracks found in the project.';
}
// Find the track to skip (unless it's the first track)
const trackToSkip = this.findTrackToSkip(midiTracks);
const firstTrack = midiTracks[0]; // The melody track
// Get project settings for proper notation
const project = this.getCurrentProject();
const timeSignature = project.getTimeSignature();
const keySignature = project.getKeySignature();
const abcKeySignature = KEY_SIGNATURE_MAP[keySignature]?.abcNotationKeySignature || 'C';
let output = `All Tracks (beats ${startBeat}-${endBeat || 'end'}):\n\n`;
midiTracks.forEach((track, index) => {
// Skip this track if it's the track to skip AND it's not the first track (melody)
if (trackToSkip && track === trackToSkip && track !== firstTrack) {
return; // Skip this track
}
const trackNumber = index + 1;
const trackName = track.getName() || `Track ${trackNumber}`;
// hardcode the 1st track to be the melody, other track names are the same as the original track names
output += `Track ${trackNumber} - ${trackNumber === 1 ? 'Melody' : trackName}:\n`;
// Get all regions from the track and convert each one
const regions = track.getRegions().filter(region => region instanceof KGMidiRegion) as KGMidiRegion[];
if (regions.length === 0) {
output += 'X:' + trackNumber + '\n';
output += `M:${timeSignature.numerator}/${timeSignature.denominator}\n`;
output += `K:${abcKeySignature}\n`;
output += 'z4 | // No regions found\n\n';
} else {
// Convert each region that overlaps with the requested range
let hasContent = false;
regions.forEach((region) => {
const regionStart = region.getStartFromBeat();
const regionEnd = regionStart + region.getLength();
// Check if region overlaps with requested range
if (regionStart < (endBeat || Infinity) && regionEnd > startBeat) {
const abcNotation = convertRegionToABCNotation(region, startBeat, endBeat);
// Update the X: line to include track number
const lines = abcNotation.split('\n');
lines[0] = `X:${trackNumber}`;
output += lines.join('\n') + '\n\n';
hasContent = true;
}
});
if (!hasContent) {
output += 'X:' + trackNumber + '\n';
output += `M:${timeSignature.numerator}/${timeSignature.denominator}\n`;
output += `K:${abcKeySignature}\n`;
output += 'z4 | // No content in specified range\n\n';
}
}
});
return output.trim();
}
/**
* Generate ABC notation for a single track
*/
private generateSingleTrackABC(track: KGMidiTrack, startBeat: number, endBeat?: number): string {
if (!(track instanceof KGMidiTrack)) {
return `Track is not a MIDI track.`;
}
// Get project settings for proper notation
const project = this.getCurrentProject();
const timeSignature = project.getTimeSignature();
const keySignature = project.getKeySignature();
const abcKeySignature = KEY_SIGNATURE_MAP[keySignature]?.abcNotationKeySignature || 'C';
const trackName = track.getName() || 'Unnamed Track';
let output = `Track "${trackName}" (beats ${startBeat}-${endBeat || 'end'}):\n`;
// Get all regions from the track and convert each one
const regions = track.getRegions().filter(region => region instanceof KGMidiRegion) as KGMidiRegion[];
if (regions.length === 0) {
output += 'X:1\n';
output += `T:${trackName}\n`;
output += `M:${timeSignature.numerator}/${timeSignature.denominator}\n`;
output += `K:${abcKeySignature}\n`;
output += `L:1/${timeSignature.denominator}\n`;
output += 'z4 | // No regions found';
} else {
// Convert each region that overlaps with the requested range
let hasContent = false;
regions.forEach((region) => {
const regionStart = region.getStartFromBeat();
const regionEnd = regionStart + region.getLength();
// Check if region overlaps with requested range
if (regionStart < (endBeat || Infinity) && regionEnd > startBeat) {
const abcNotation = convertRegionToABCNotation(region, startBeat, endBeat);
// Update the title to include track name
const lines = abcNotation.split('\n');
lines[1] = `T:${trackName}`;
output += lines.join('\n');
hasContent = true;
}
});
if (!hasContent) {
output += 'X:1\n';
output += `T:${trackName}\n`;
output += `M:${timeSignature.numerator}/${timeSignature.denominator}\n`;
output += `K:${abcKeySignature}\n`;
output += `L:1/${timeSignature.denominator}\n`;
output += 'z4 | // No content in specified range';
}
}
return output;
}
}
src/agent/tools/RemoveNotesTool.ts
+168
View File
@@ -0,0 +1,168 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
import { DeleteNotesCommand } from '../../core/commands/note/DeleteNotesCommand';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { useProjectStore } from '../../stores/projectStore';
import { KGCore } from '../../core/KGCore';
/**
* Tool for removing notes from MIDI regions within a specified beat range
* Integrates with the existing command system for undo/redo support
*/
export class RemoveNotesTool extends BaseTool {
readonly name = 'remove_notes';
readonly description = 'Remove MIDI notes from the current region within a specified beat range. All notes that start within the range will be deleted.';
readonly parameters: Record<string, ToolParameter> = {
start_beat: {
type: 'number',
description: 'Start of the beat range to remove notes from (inclusive)',
required: true
},
end_beat: {
type: 'number',
description: 'End of the beat range to remove notes from (exclusive)',
required: true
},
region_id: {
type: 'string',
description: 'ID of the region to remove notes from. If not provided, uses the currently selected region.',
required: false
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Validate parameters
this.validateParameters(params);
const startBeat = params.start_beat as number;
const endBeat = params.end_beat as number;
const regionId = params.region_id as string | undefined;
// Validate beat range
if (startBeat < 0) {
return this.createErrorResult(`Invalid start_beat ${startBeat}. Must be >= 0.`);
}
if (endBeat <= startBeat) {
return this.createErrorResult(`Invalid beat range: end_beat (${endBeat}) must be greater than start_beat (${startBeat}).`);
}
// Find the target region
const targetRegion = this.findTargetRegion(regionId);
if (!targetRegion) {
return this.createErrorResult(
regionId
? `Region with ID "${regionId}" not found or is not a MIDI region`
: 'No active or selected MIDI region found. Please open the piano roll with a region or select a MIDI region first.'
);
}
// Adjust beat range relative to region's start beat
const regionStartBeat = targetRegion.getStartFromBeat();
const adjustedStartBeat = startBeat - regionStartBeat;
const adjustedEndBeat = endBeat - regionStartBeat;
// Find all notes within the specified beat range
const notesToRemove = this.findNotesInRange(targetRegion, adjustedStartBeat, adjustedEndBeat);
if (notesToRemove.length === 0) {
return this.createSuccessResult(
`No notes found in the range from beat ${startBeat} to ${endBeat}.`
);
}
// Extract note IDs for deletion
const noteIds = notesToRemove.map(note => note.getId());
// Execute the deletion command
const command = new DeleteNotesCommand(noteIds);
await this.executeCommand(command);
// Create success message
const noteCount = notesToRemove.length;
const noteList = notesToRemove
.map(note => {
const noteNames = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'];
const octave = Math.floor(note.getPitch() / 12) - 1;
const noteName = noteNames[note.getPitch() % 12];
return `${noteName}${octave}`;
})
.join(', ');
return this.createSuccessResult(
`Successfully removed ${noteCount} note${noteCount > 1 ? 's' : ''} from beats ${startBeat}-${endBeat}: ${noteList}`
);
} catch (error) {
return this.createErrorResult(`Failed to remove notes: ${error}`);
}
}
/**
* Find the target region for note removal
* Priority: 1) Specified regionId, 2) Active piano roll region, 3) Selected regions, 4) Error if none found
*/
private findTargetRegion(regionId?: string): KGMidiRegion | null {
const project = this.getCurrentProject();
const tracks = project.getTracks();
if (regionId) {
// Find specific region by ID
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region && region instanceof KGMidiRegion) {
return region;
}
}
return null;
} else {
// Smart region finding: try different sources in priority order
// 1. Try active piano roll region
const storeState = useProjectStore.getState();
if (storeState.activeRegionId) {
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === storeState.activeRegionId);
if (region && region instanceof KGMidiRegion) {
return region;
}
}
}
// 2. Try selected regions
const core = this.getKGCore();
const selectedItems = core.getSelectedItems();
for (const item of selectedItems) {
if (item instanceof KGMidiRegion) {
return item;
}
}
// 3. No fallback - return null to trigger error
return null;
}
}
/**
* Get KGCore instance for selection access
*/
private getKGCore() {
return KGCore.instance();
}
/**
* Find all notes within the specified beat range
* Notes are included if their start beat is within [startBeat, endBeat)
*/
private findNotesInRange(region: KGMidiRegion, startBeat: number, endBeat: number) {
const notes = region.getNotes();
return notes.filter(note => {
const noteStartBeat = note.getStartBeat();
return noteStartBeat >= startBeat && noteStartBeat < endBeat;
});
}
}
src/agent/tools/ThinkTool.ts
+35
View File
@@ -0,0 +1,35 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
/**
* Pseudo tool for handling <think> tags in LLM responses
* This tool displays the thinking content in the UI but doesn't send results back to the LLM
* Handles XML format: <think>any content here</think>
* This is functionally identical to ThinkingTool but handles the shorter tag name
*/
export class ThinkTool extends BaseTool {
readonly name = 'think';
readonly description = 'Pseudo tool for handling LLM thinking content from <think> tags. Shows content in UI but does not send results back to LLM.';
readonly parameters: Record<string, ToolParameter> = {
content: {
type: 'string',
description: 'The thinking content from the XML tag',
required: false
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Extract the thinking content from the parameters
// const content = params.content as string || '';
// Return the thinking content as a successful result
// This will be displayed in the UI but not sent back to the LLM
return this.createSuccessResult("Thinking completed.");
} catch (error) {
return this.createErrorResult(`Failed to process thinking content: ${error}`);
}
}
}
src/agent/tools/ThinkingTool.ts
+34
View File
@@ -0,0 +1,34 @@
import { BaseTool } from './BaseTool';
import type { ToolResult, ToolParameter } from './BaseTool';
/**
* Pseudo tool for handling <thinking> tags in LLM responses
* This tool displays the thinking content in the UI but doesn't send results back to the LLM
* Handles XML format: <thinking>any content here</thinking>
*/
export class ThinkingTool extends BaseTool {
readonly name = 'thinking';
readonly description = 'Pseudo tool for handling LLM thinking content. Shows content in UI but does not send results back to LLM.';
readonly parameters: Record<string, ToolParameter> = {
content: {
type: 'string',
description: 'The thinking content from the XML tag',
required: false
}
};
async execute(params: Record<string, unknown>): Promise<ToolResult> {
try {
// Extract the thinking content from the parameters
// const content = params.content as string || '';
// Return the thinking content as a successful result
// This will be displayed in the UI but not sent back to the LLM
return this.createSuccessResult("Thinking completed.");
} catch (error) {
return this.createErrorResult(`Failed to process thinking content: ${error}`);
}
}
}
src/agent/tools/index.ts
+25
View File
@@ -0,0 +1,25 @@
// Base tool system
export { BaseTool } from './BaseTool';
export type { ToolResult, ToolParameter, ToolDefinition } from './BaseTool';
// Specific tools
import { AddNotesTool } from './AddNotesTool';
import { RemoveNotesTool } from './RemoveNotesTool';
import { ReadMusicTool } from './ReadMusicTool';
import { AttemptCompletionTool } from './AttemptCompletionTool';
import { ThinkingTool } from './ThinkingTool';
import { ThinkTool } from './ThinkTool';
export { AddNotesTool, RemoveNotesTool, ReadMusicTool, AttemptCompletionTool, ThinkingTool, ThinkTool };
// Tool registry for easy access
export const AVAILABLE_TOOLS = {
add_notes: AddNotesTool,
remove_notes: RemoveNotesTool,
read_music: ReadMusicTool,
attempt_completion: AttemptCompletionTool,
thinking: ThinkingTool,
think: ThinkTool
} as const;
export type ToolName = keyof typeof AVAILABLE_TOOLS;
src/assets/react.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.0 KiB

src/components/ChatBox.tsx
+582
View File
@@ -0,0 +1,582 @@
import React, { useState, useRef, useEffect, memo } from 'react';
import { FaPlus, FaBan } from 'react-icons/fa';
import { UserMessage, AssistantMessage } from './chat';
import { AgentCore } from '../agent/core/AgentCore';
import { OpenAIProvider } from '../agent/llm/OpenAIProvider';
import { ClaudeProvider } from '../agent/llm/ClaudeProvider';
import { GeminiProvider } from '../agent/llm/GeminiProvider';
import { LLMProvider } from '../agent/llm/LLMProvider';
import { ConfigManager } from '../core/config/ConfigManager';
import { useProjectStore } from '../stores/projectStore';
import { XMLToolExecutor } from '../agent/core/XMLToolExecutor';
import { extractXMLFromString } from '../util/xmlUtil';
import { SystemPrompts } from '../agent/core/SystemPrompts';
import { clearChatHistoryAndUI, registerClearChatUICallback } from '../util/chatUtil';
import { processUserMessage } from '../util/messageFilter/UserMessageFilter';
// Module-level guard to avoid duplicate welcome in React StrictMode dev remounts
let hasShownWelcomeOnceInRuntime = false;
interface ChatMessage {
id: string;
role: 'user' | 'assistant';
content: string;
isStreaming?: boolean;
tokenCount?: number;
}
/**
* Create the appropriate LLM provider based on configuration
*/
const createLLMProvider = (): LLMProvider => {
const configManager = ConfigManager.instance();
const providerType = configManager.get('general.llm_provider') as string;
switch (providerType) {
case 'claude':
return new ClaudeProvider();
case 'gemini':
return new GeminiProvider();
case 'openai_compatible':
case 'openai':
default:
return new OpenAIProvider();
}
};
const ChatBox: React.FC = () => {
const [inputValue, setInputValue] = useState('');
const textareaRef = useRef<HTMLTextAreaElement>(null);
// Initialize with empty messages
const [messages, setMessages] = useState<ChatMessage[]>([]);
const [isProcessing, setIsProcessing] = useState(false);
const [abortController, setAbortController] = useState<AbortController | null>(null);
const [lastUserMessage, setLastUserMessage] = useState<string>('');
// Tool execution state
const [isExecutingTools, setIsExecutingTools] = useState(false);
const [, setToolResults] = useState<string>(''); // placeholder for future display/use
const [, setCurrentToolIndex] = useState<number>(0); // placeholder for future display/use
// Track if this is the first message (for system prompt logging)
const [isFirstMessage, setIsFirstMessage] = useState(true);
// Initialize AgentCore with configured provider and register clear UI callback
useEffect(() => {
const initializeProvider = async () => {
const configManager = ConfigManager.instance();
// Ensure ConfigManager is initialized
if (!configManager.getIsInitialized()) {
await configManager.initialize();
}
const provider = createLLMProvider();
const agentCore = AgentCore.instance();
agentCore.setLLMProvider(provider);
console.log(`Switched to ${provider.name} provider`);
};
// Register the UI clear callback for external components to use
registerClearChatUICallback(clearChatUI);
initializeProvider();
// Auto-trigger welcome on first launch (guard against React StrictMode double-invoke only)
(async () => {
try {
if (hasShownWelcomeOnceInRuntime) return;
hasShownWelcomeOnceInRuntime = true;
const result = await processUserMessage('/welcome');
if (result.pseudoAssistantResponse) {
const pseudoId = generateMessageId();
setMessages(prev => [...prev, {
id: pseudoId,
role: 'assistant',
content: result.pseudoAssistantResponse!,
}]);
}
} catch {
// ignore
}
})();
}, []);
const generateMessageId = (): string => {
return `msg_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;
};
const handleAbort = () => {
if (abortController) {
abortController.abort();
setAbortController(null);
// Use AgentCore to clean up the data model and get the user message content
const agentCore = AgentCore.instance();
const userMessageContent = agentCore.abortCurrentRequest();
// Remove the last user message and assistant message from UI
setMessages(prev => prev.slice(0, -2));
// Restore the user's input (use the content from AgentCore if available)
setInputValue(userMessageContent || lastUserMessage);
setIsProcessing(false);
}
};
const clearChatUI = () => {
// Clear UI state
setMessages([]);
// Reset first message flag so system prompt will be logged again
setIsFirstMessage(true);
};
const handleClearCommand = () => {
const { setStatus } = useProjectStore.getState();
clearChatHistoryAndUI(setStatus);
};
const addToolResultMessage = (toolName: string, success: boolean, result: string) => {
const toolMsgId = generateMessageId();
const friendlyDisplay = `${success ? '✅' : '❌'} __**${toolName}**__ \n\n └── ${result}`;
setMessages(prev => [...prev, {
id: toolMsgId,
role: 'user',
content: friendlyDisplay
}]);
};
const executeToolsFromResponse = async (response: string): Promise<boolean> => {
try {
// Check if response contains XML tool invocations
const xmlBlocks = extractXMLFromString(response);
// Consider only actionable tools (exclude think/thinking)
const actionableBlocks = xmlBlocks.filter((block) => {
const match = block.match(/<([a-zA-Z_][a-zA-Z0-9_-]*)/);
const name = match ? match[1].toLowerCase() : '';
return name !== 'think' && name !== 'thinking';
});
if (actionableBlocks.length === 0) {
// No actionable tools to execute, stop the loop
return false;
}
// Start tool execution phase
setIsExecutingTools(true);
setToolResults('');
setCurrentToolIndex(0);
const { setStatus } = useProjectStore.getState();
setStatus(`Executing ${actionableBlocks.length} tool(s)...`);
const executor = XMLToolExecutor.instance();
let accumulatedResults = '';
// Execute tools sequentially with real-time updates
for (let i = 0; i < actionableBlocks.length; i++) {
setCurrentToolIndex(i + 1);
setStatus(`Executing tool ${i + 1} of ${actionableBlocks.length}...`);
// Determine tool name from XML block
const toolNameMatch = actionableBlocks[i].match(/<([a-zA-Z_][a-zA-Z0-9_-]*)/);
const toolName = toolNameMatch ? toolNameMatch[1] : 'unknown_tool';
try {
// Execute single XML block
const results = await executor.executeXMLTools(actionableBlocks[i]);
const result = results[0]; // Single block should give single result
if (result) {
// Add friendly display message
addToolResultMessage(toolName, result.success, result.result);
// Accumulate formatted result for LLM (skip thinking tools)
if (toolName !== 'thinking' && toolName !== 'think') {
const formattedResult = `tool: ${toolName}\nsuccess: ${result.success}\nresult:\n${result.result}\n------------\n`;
accumulatedResults += formattedResult;
}
}
} catch (error) {
// Handle individual tool error
addToolResultMessage(toolName, false, `Tool execution failed: ${error}`);
// Accumulate error result for LLM (skip thinking tools)
if (toolName !== 'thinking' && toolName !== 'think') {
const formattedResult = `tool: ${toolName}\nsuccess: false\nresult:\nTool execution failed: ${error}\n------------\n`;
accumulatedResults += formattedResult;
}
}
}
// Store accumulated results
setToolResults(accumulatedResults);
setIsExecutingTools(false);
// Check if agent is still working on task before sending results to LLM
const agentCore = AgentCore.instance();
const isStillWorkingOnTask = agentCore.getAgentState().getIsWorkingOnTask();
if (isStillWorkingOnTask) {
// Send tool results back to LLM
setStatus('Processing tool results...');
await sendToolResultsToLLM(accumulatedResults);
} else {
// Agent is no longer working on task, ignore results and return control to user
setStatus('Tool execution completed');
}
return true; // Tools were found and executed
} catch (error) {
console.error('Error executing tools:', error);
setIsExecutingTools(false);
const { setStatus } = useProjectStore.getState();
setStatus(`Tool execution failed: ${error}`);
return false; // Tool execution failed
}
};
const sendToolResultsToLLM = async (toolResultsString: string): Promise<void> => {
// Send tool results as hidden user input to LLM
setIsProcessing(true);
// Create abort controller for this request
const controller = new AbortController();
setAbortController(controller);
// Add streaming assistant message for the response
const assistantMsgId = generateMessageId();
setMessages(prev => [...prev, {
id: assistantMsgId,
role: 'assistant',
content: 'processing... 0 tokens received. click here to abort.',
isStreaming: true,
tokenCount: 0
}]);
try {
const agentCore = AgentCore.instance();
let assistantResponse = '';
let tokenCount = 0;
// Log the tool results being sent to LLM
console.log('------------ USER ------------');
console.log(toolResultsString);
console.log('------------------------------');
for await (const chunk of agentCore.processUserInput(toolResultsString)) {
// Check if request was aborted
if (controller.signal.aborted) {
return;
}
if (chunk.type === 'text') {
assistantResponse += chunk.content;
tokenCount++;
// Update streaming message with token count and abort link
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: `processing... ${tokenCount} tokens received. click here to abort.`, tokenCount }
: msg
));
} else if (chunk.type === 'done') {
// Replace with final response
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: assistantResponse, isStreaming: false, tokenCount: undefined }
: msg
));
// Log the complete assistant response
console.log('------------ ASSISTANT ------------');
console.log(assistantResponse);
console.log('-----------------------------------');
// Check if the new response contains more tools
const hasMoreTools = await executeToolsFromResponse(assistantResponse);
// If no more tools were found, set working flag to false
if (!hasMoreTools) {
const agentCore = AgentCore.instance();
agentCore.getAgentState().setIsWorkingOnTask(false);
}
break;
}
}
} catch (error) {
if (error instanceof Error && error.name === 'AbortError') {
// Request was aborted, don't show error
return;
}
console.error('Error processing tool results:', error);
// Update with error message
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: 'Error: Failed to process tool results', isStreaming: false, tokenCount: undefined }
: msg
));
} finally {
setAbortController(null);
setIsProcessing(false);
}
};
const handleSend = async () => {
if (inputValue.trim() && !isProcessing) {
const userMessage = inputValue.trim();
setLastUserMessage(userMessage);
setInputValue('');
// Run message through the filter system
const filterResult = await processUserMessage(userMessage);
// Conditionally show the user message bubble
if (filterResult.displayUserMessage) {
const userMsgId = generateMessageId();
setMessages(prev => [...prev, {
id: userMsgId,
role: 'user',
content: userMessage
}]);
}
// If we have a pseudo assistant response, show it immediately
if (filterResult.pseudoAssistantResponse) {
const pseudoId = generateMessageId();
setMessages(prev => [...prev, {
id: pseudoId,
role: 'assistant',
content: filterResult.pseudoAssistantResponse!,
}]);
}
// If we shouldn't send anything to LLM, stop here
if (!filterResult.sendToLLM || !filterResult.finalMessageForLLM) {
return;
}
setIsProcessing(true);
// Create abort controller for this request
const controller = new AbortController();
setAbortController(controller);
// Add streaming assistant message
const assistantMsgId = generateMessageId();
setMessages(prev => [...prev, {
id: assistantMsgId,
role: 'assistant',
content: 'processing... 0 tokens received. click here to abort.',
isStreaming: true,
tokenCount: 0
}]);
try {
const agentCore = AgentCore.instance();
let assistantResponse = '';
let tokenCount = 0;
// Set working on task flag when user sends a message
agentCore.getAgentState().setIsWorkingOnTask(true);
// Log system prompt only for first message or first message after clear
if (isFirstMessage) {
try {
const systemPrompt = await SystemPrompts.getSystemPromptWithContext();
console.log('------------ SYSTEM ------------');
console.log(systemPrompt);
console.log('--------------------------------');
} catch (error) {
console.error('Failed to log system prompt:', error);
}
// Mark that we've logged the system prompt for this conversation
setIsFirstMessage(false);
}
// Log the final user message being sent to LLM
console.log('------------ USER ------------');
console.log(filterResult.finalMessageForLLM);
console.log('------------------------------');
for await (const chunk of agentCore.processUserInput(filterResult.finalMessageForLLM)) {
// Check if request was aborted
if (controller.signal.aborted) {
return;
}
if (chunk.type === 'text') {
assistantResponse += chunk.content;
tokenCount++;
// Update streaming message with token count and abort link
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: `processing... ${tokenCount} tokens received. click here to abort.`, tokenCount }
: msg
));
} else if (chunk.type === 'done') {
// Replace with final response
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: assistantResponse, isStreaming: false, tokenCount: undefined }
: msg
));
// Log the complete assistant response
console.log('------------ ASSISTANT ------------');
console.log(assistantResponse);
console.log('-----------------------------------');
// Check if response contains tools to execute
const hasTools = await executeToolsFromResponse(assistantResponse);
// If no tools were found, set working flag to false and return control to user
if (!hasTools) {
agentCore.getAgentState().setIsWorkingOnTask(false);
}
break;
}
}
} catch (error) {
if (error instanceof Error && error.name === 'AbortError') {
// Request was aborted, don't show error
return;
}
console.error('Error processing message:', error);
// Update with error message
setMessages(prev => prev.map(msg =>
msg.id === assistantMsgId
? { ...msg, content: 'Error: Failed to process message', isStreaming: false, tokenCount: undefined }
: msg
));
} finally {
setAbortController(null);
setIsProcessing(false);
}
}
};
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
handleSend();
}
// Allow Shift+Enter for new lines (default textarea behavior)
};
const handleInputFocus = () => {
// Set a data attribute on the textarea to help global keyboard handler identify it
if (textareaRef.current) {
textareaRef.current.setAttribute('data-chatbox-input', 'true');
}
};
const handleInputBlur = () => {
// Remove the data attribute when losing focus
if (textareaRef.current) {
textareaRef.current.removeAttribute('data-chatbox-input');
}
};
// Auto-resize textarea based on content
useEffect(() => {
if (textareaRef.current) {
textareaRef.current.style.height = 'auto';
textareaRef.current.style.height = `${textareaRef.current.scrollHeight}px`;
}
}, [inputValue]);
// Auto-scroll to bottom when new messages arrive
const messagesEndRef = useRef<HTMLDivElement>(null);
useEffect(() => {
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
}, [messages]);
// Auto-focus input when it becomes visible (when processing and tool execution complete)
useEffect(() => {
if (!isProcessing && !isExecutingTools && textareaRef.current) {
// Use a small delay to ensure the DOM has updated
setTimeout(() => {
textareaRef.current?.focus();
// Also scroll to bottom when input becomes visible after tool execution
// This ensures proper scroll position after layout changes from showing input box
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
}, 0);
}
}, [isProcessing, isExecutingTools]);
return (
<div className="chatbox">
<div className="chatbox-header">
<h3>K.G.Studio Musician Assistant</h3>
<div className="chatbox-actions">
{isProcessing && (
<button
type="button"
title="Abort"
onClick={handleAbort}
className="chatbox-action-btn"
>
<FaBan />
</button>
)}
<button
type="button"
title="New Chat"
onClick={handleClearCommand}
className="chatbox-action-btn"
>
<FaPlus />
</button>
</div>
</div>
<div className="chatbox-messages">
{messages.map((message) => (
message.role === 'user' ? (
<UserMessage key={message.id} content={message.content} />
) : (
<AssistantMessage
key={message.id}
content={message.content}
isStreaming={message.isStreaming}
onAbort={message.isStreaming ? handleAbort : undefined}
/>
)
))}
<div ref={messagesEndRef} />
</div>
{!isProcessing && !isExecutingTools && (
<div className="chatbox-input-area">
<textarea
ref={textareaRef}
value={inputValue}
onChange={(e) => setInputValue(e.target.value)}
onKeyDown={handleKeyDown}
onFocus={handleInputFocus}
onBlur={handleInputBlur}
placeholder="Press Enter to send message, Shift + Enter for new line"
className="chatbox-input"
rows={1}
/>
</div>
)}
</div>
);
};
export default memo(ChatBox);
src/components/InstrumentSelection.tsx
+110
View File
@@ -0,0 +1,110 @@
import React, { useMemo, useState, useEffect } from 'react';
import { useProjectStore } from '../stores/projectStore';
import { INSTRUMENT_GROUPS, FLUIDR3_INSTRUMENT_MAP } from '../constants/generalMidiConstants';
import { KGMidiTrack, type InstrumentType } from '../core/track/KGMidiTrack';
const InstrumentSelection: React.FC = () => {
const {
tracks,
instrumentSelectionTrackId,
closeInstrumentSelection,
setTrackInstrument
} = useProjectStore();
const targetTrack = useMemo(() => {
return tracks.find(t => t.getId().toString() === instrumentSelectionTrackId) || null;
}, [tracks, instrumentSelectionTrackId]);
const currentInstrumentKey: InstrumentType = (targetTrack && targetTrack instanceof KGMidiTrack)
? (targetTrack.getInstrument() as InstrumentType)
: 'acoustic_grand_piano';
const currentInstrumentDef = FLUIDR3_INSTRUMENT_MAP[currentInstrumentKey] || FLUIDR3_INSTRUMENT_MAP['acoustic_grand_piano'];
// Maintain selected group in local state; selected instrument derives from model
const [selectedGroupKey, setSelectedGroupKey] = useState<string>(currentInstrumentDef?.group || 'PIANO_AND_KEYBOARDS');
useEffect(() => {
// Sync when the target track or its instrument changes
setSelectedGroupKey(currentInstrumentDef?.group || 'PIANO_AND_KEYBOARDS');
}, [instrumentSelectionTrackId, currentInstrumentKey, currentInstrumentDef]);
const groups = useMemo(() => Object.entries(INSTRUMENT_GROUPS) as Array<[string, string]>, []);
const instrumentsInGroup = useMemo(() => {
return Object.entries(FLUIDR3_INSTRUMENT_MAP)
.filter((entry) => entry[1].group === selectedGroupKey)
.map((entry) => ({ key: entry[0], label: entry[1].displayName }));
}, [selectedGroupKey]);
const handleSelectGroup = (groupKey: string) => {
setSelectedGroupKey(groupKey);
};
const handleSelectInstrument = async (instrumentKey: string) => {
const instrument = instrumentKey as InstrumentType;
if (!targetTrack || !(targetTrack instanceof KGMidiTrack)) return;
try {
await setTrackInstrument(targetTrack.getId(), instrument);
} catch (err) {
console.error('Failed to change instrument from panel:', err);
}
};
const previewImage = FLUIDR3_INSTRUMENT_MAP[currentInstrumentKey]?.image || 'piano.png';
const previewAlt = FLUIDR3_INSTRUMENT_MAP[currentInstrumentKey]?.displayName || currentInstrumentKey;
if (!targetTrack) return null;
return (
<div className="instrument-selection">
<div className="instrument-selection-header">
<h3>{`${previewAlt.toString()}`}</h3>
<button className="instrument-selection-close-btn" onClick={closeInstrumentSelection}>✕</button>
</div>
<div className="instrument-selection-top">
<div className="instrument-preview">
<img
src={`/resources/instruments/${previewImage}`}
alt={previewAlt.toString()}
width={256}
height={256}
/>
</div>
<div className="instrument-name-overlay">{targetTrack.getName()}</div>
</div>
<div className="instrument-selection-bottom">
<div className="instrument-groups">
<div className="instrument-groups-list">
{groups.map(([key, label]) => (
<div
key={key}
className={`instrument-group-item${selectedGroupKey === key ? ' active' : ''}`}
onClick={() => handleSelectGroup(key)}
>
{label}
</div>
))}
</div>
</div>
<div className="instrument-list">
<div className="instrument-instruments-list">
{instrumentsInGroup.map((inst) => (
<div
key={inst.key}
className={`instrument-instrument-item${currentInstrumentKey === inst.key ? ' active' : ''}`}
onClick={() => handleSelectInstrument(inst.key)}
>
{inst.label}
</div>
))}
</div>
</div>
</div>
</div>
);
};
export default InstrumentSelection;
src/components/MainContent.tsx
+659
View File
@@ -0,0 +1,659 @@
import React, { useState, useEffect, useRef, useCallback } from 'react';
import { createPortal } from 'react-dom';
import { useProjectStore } from '../stores/projectStore';
import { KGCore } from '../core/KGCore';
import { KGTrack } from '../core/track/KGTrack';
import { KGMidiRegion } from '../core/region/KGMidiRegion';
import TrackInfoPanel from './track/TrackInfoPanel';
import TrackGridPanel from './track/TrackGridPanel';
import PianoRoll from './piano-roll/PianoRoll';
import type { RegionUI } from './interfaces';
import { DEBUG_MODE } from '../constants';
import { useRegionOperations } from '../hooks/useRegionOperations';
import { regionDeleteManager } from '../util/regionDeleteUtil';
interface MainContentProps {
onTrackClick?: () => void;
}
const MainContent: React.FC<MainContentProps> = ({
onTrackClick = () => {} // Default to empty function if not provided
}) => {
const {
tracks,
maxBars,
reorderTracks,
updateTrack,
updateTrackProperties,
timeSignature,
setPlayheadPosition,
clearAllSelections,
setSelectedTrack,
showPianoRoll,
activeRegionId,
setShowPianoRoll,
setActiveRegionId
} = useProjectStore();
// State to store regions
const [regions, setRegions] = useState<RegionUI[]>([]);
// Drag state for track grid highlighting
const [draggedTrackIndex, setDraggedTrackIndex] = useState<number | null>(null);
const [dragOverTrackIndex, setDragOverTrackIndex] = useState<number | null>(null);
// Piano roll state is now managed by the store - removed local state
// Region selection state
const [selectedRegionId, setSelectedRegionId] = useState<string | null>(null);
// Use the region operations hook
const { deleteSelectedRegions } = useRegionOperations({
tracks,
updateTrack,
setRegions,
selectedRegionId,
setSelectedRegionId,
showPianoRoll,
setShowPianoRoll,
activeRegionId,
setActiveRegionId
});
// Register the delete function with the global manager
useEffect(() => {
regionDeleteManager.registerDeleteCallback(deleteSelectedRegions);
// Cleanup on unmount
return () => {
regionDeleteManager.unregisterDeleteCallback();
};
}, [deleteSelectedRegions]);
// Refs to track pending updates for verification
const pendingUpdates = useRef<Map<string, { trackId: string, regionId: string, startBeat: number, length: number }>>(new Map());
// Refs for bar numbers drag functionality
const isDraggingRef = useRef(false);
const barNumbersRef = useRef<HTMLDivElement | null>(null);
// Effect to verify track updates
useEffect(() => {
// Check for pending updates
if (pendingUpdates.current.size > 0) {
// Create a copy of the pending updates
const updates = new Map(pendingUpdates.current);
// Clear pending updates
pendingUpdates.current.clear();
// Check each update
updates.forEach((update, key) => {
const { trackId, regionId, startBeat, length } = update;
// Find the track
const track = tracks.find(t => t.getId().toString() === trackId);
if (track) {
// Find the region
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region && DEBUG_MODE.MAIN_CONTENT) {
console.log(`Verification - Region ${regionId} in track ${trackId}:`);
console.log(` Expected: startBeat=${startBeat}, length=${length}`);
console.log(` Actual: startBeat=${region.getStartFromBeat()}, length=${region.getLength()}, trackId=${region.getTrackId()}, trackIndex=${region.getTrackIndex()}`);
// Check if the update was successful
const success = region.getStartFromBeat() === startBeat && region.getLength() === length && region.getTrackId() === trackId;
console.log(` Update successful: ${success}`);
}
}
});
}
}, [tracks]);
// Effect to update regions when tracks change
useEffect(() => {
// Create a new array of RegionUI objects based on the current tracks
const updatedRegions: RegionUI[] = [];
// Iterate through all tracks
tracks.forEach(track => {
const trackId = track.getId().toString();
const trackIndex = track.getTrackIndex();
// Iterate through all regions in the track
track.getRegions().forEach(region => {
if (region instanceof KGMidiRegion) {
// Calculate bar number and length from beats
const beatsPerBar = timeSignature.numerator;
const barNumber = Math.floor(region.getStartFromBeat() / beatsPerBar) + 1;
const length = region.getLength() / beatsPerBar;
// Create a RegionUI object
updatedRegions.push({
id: region.getId(),
trackId,
trackIndex,
barNumber,
length,
name: region.getName()
});
}
});
});
// Update the regions state
setRegions(updatedRegions);
}, [tracks, timeSignature]);
// Handle track name edit
const handleTrackNameEdit = (track: KGTrack, newName: string) => {
// Use the command pattern to update track name with undo support
updateTrackProperties(track.getId(), { name: newName });
};
// Handle track reordering
const handleTracksReordered = (fromIndex: number, toIndex: number) => {
// Reorder tracks in the store - this will also update trackIndex in each KGTrack
reorderTracks(fromIndex, toIndex);
// Update regions to match the new track order
setRegions(prevRegions => {
return prevRegions.map(region => {
// If the region belongs to the dragged track, update its trackIndex
if (region.trackIndex === fromIndex) {
return { ...region, trackIndex: toIndex };
}
// If the region belongs to a track that was shifted due to the drag operation
else if (
(fromIndex < toIndex &&
region.trackIndex > fromIndex &&
region.trackIndex <= toIndex)
) {
// Shift up by 1
return { ...region, trackIndex: region.trackIndex - 1 };
}
else if (
(fromIndex > toIndex &&
region.trackIndex < fromIndex &&
region.trackIndex >= toIndex)
) {
// Shift down by 1
return { ...region, trackIndex: region.trackIndex + 1 };
}
// Otherwise leave it unchanged
return region;
});
});
// Update the grid drag state to match
setDraggedTrackIndex(null);
setDragOverTrackIndex(null);
};
// Handle region creation from TrackGridPanel
const handleRegionCreated = (trackIndex: number, regionUI: RegionUI, midiRegion: KGMidiRegion) => {
// Note: The region model is already created by the CreateRegionCommand
// We just need to update the UI state and handle selection
// Get the track for store updates
const track = tracks[trackIndex];
// Update the track in the store to reflect the command changes
updateTrack(track);
// Select the track that contains the new region
setSelectedTrack(track.getId().toString());
// Add the new region to the UI state and select it immediately
setRegions(prevRegions => {
const updatedRegions = [...prevRegions, regionUI];
// Select the region using the updated regions array
selectRegion(regionUI.id, updatedRegions);
// Manually trigger selection sync to ensure UI updates immediately
const { syncSelectionFromCore } = useProjectStore.getState();
syncSelectionFromCore();
// If piano roll is visible, set this region as the active region
if (showPianoRoll) {
setActiveRegionId(regionUI.id);
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Newly created region ${regionUI.id} set as active region in piano roll`);
}
}
return updatedRegions;
});
};
// Handle region updates (resize, move, etc.)
const handleRegionUpdated = (
regionId: string,
updates: Partial<RegionUI>,
expectedModelUpdates?: { startBeat: number, length: number }
) => {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Updating region ${regionId} with:`, updates);
}
// Select the region when it's being updated (resize or move)
selectRegion(regionId);
// Find the region to determine which track to select
const updatedRegion = regions.find(r => r.id === regionId);
if (updatedRegion) {
// Use the updated trackId if available, otherwise use the existing trackId
const trackId = updates.trackId || updatedRegion.trackId;
const track = tracks.find(t => t.getId().toString() === trackId);
if (track) {
setSelectedTrack(track.getId().toString());
}
}
// Update the region in the UI state
setRegions(prevRegions => {
return prevRegions.map(region => {
if (region.id === regionId) {
return { ...region, ...updates };
}
return region;
});
});
// Find the region that was updated
const region = regions.find(r => r.id === regionId);
if (!region) return;
// Check if the track ID is being updated (region moved to different track)
if (updates.trackId && updates.trackId !== region.trackId) {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Region ${regionId} moved from track ${region.trackId} to track ${updates.trackId}`);
}
// Get the original track
const originalTrack = tracks.find(t => t.getId().toString() === region.trackId);
// Get the target track
const targetTrack = tracks.find(t => t.getId().toString() === updates.trackId);
if (originalTrack && targetTrack) {
// Select the target track that now contains the region
setSelectedTrack(targetTrack.getId().toString());
// Update both tracks in the store
updateTrack(originalTrack);
updateTrack(targetTrack);
// Add to pending updates for verification
if (expectedModelUpdates) {
const key = `${updates.trackId}-${regionId}-${Date.now()}`;
pendingUpdates.current.set(key, {
trackId: updates.trackId,
regionId,
startBeat: expectedModelUpdates.startBeat,
length: expectedModelUpdates.length
});
}
}
} else {
// Find the track that contains this region
const track = tracks.find(t => t.getId().toString() === region.trackId);
if (track) {
// Log the track's regions before updating the store
const trackRegions = track.getRegions();
const midiRegion = trackRegions.find(r => r.getId() === regionId) as KGMidiRegion | undefined;
if (midiRegion) {
// If we have expected model updates, use those
if (expectedModelUpdates) {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`MainContent - Expected model updates: startBeat=${expectedModelUpdates.startBeat}, length=${expectedModelUpdates.length}`);
}
// Add to pending updates for verification
const key = `${track.getId()}-${regionId}-${Date.now()}`;
pendingUpdates.current.set(key, {
trackId: track.getId().toString(),
regionId,
startBeat: expectedModelUpdates.startBeat,
length: expectedModelUpdates.length
});
} else {
// Otherwise use the current values (for backward compatibility)
const startBeat = midiRegion.getStartFromBeat();
const length = midiRegion.getLength();
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`MainContent - Region before store update: startBeat=${startBeat}, length=${length}`);
}
// Add to pending updates for verification
const key = `${track.getId()}-${regionId}-${Date.now()}`;
pendingUpdates.current.set(key, {
trackId: track.getId().toString(),
regionId,
startBeat,
length
});
}
}
// Update the track in the store to persist changes
updateTrack(track);
}
}
// If piano roll is visible, set this region as the active region
if (showPianoRoll) {
setActiveRegionId(regionId);
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Updated region ${regionId} set as active region in piano roll`);
}
}
};
// Helper function to select a region (clears previous selections)
const selectRegion = (regionId: string, regionsToSearch?: RegionUI[]) => {
// Clear any existing selections using store method
clearAllSelections();
// Find the region in the UI state (use provided regions or current state)
const regionsToUse = regionsToSearch || regions;
const region = regionsToUse.find(r => r.id === regionId);
if (!region) {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Region not found in UI state: ${regionId}`);
}
return;
}
// Find the track that contains this region
const track = tracks.find(t => t.getId().toString() === region.trackId);
if (!track) {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Track not found for region: ${regionId}`);
}
return;
}
// Find the region in the track's model
const trackRegions = track.getRegions();
const midiRegion = trackRegions.find(r => r.getId() === regionId) as KGMidiRegion | undefined;
if (!midiRegion) {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`MIDI region not found in track model: ${regionId}`);
}
return;
}
// Add the region to KGCore's selection
const core = KGCore.instance();
core.addSelectedItem(midiRegion);
// Update the region's internal selection state
midiRegion.select();
// Set the selected region (this might be redundant now, but keeping for compatibility)
setSelectedRegionId(regionId);
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Selected region: ${regionId} (added to KGCore selection)`);
}
};
// Handle region single click: selection only (no piano roll opening)
const handleRegionClick = (regionId: string) => {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Region clicked in MainContent (selection only): ${regionId}`);
}
// Select the region
selectRegion(regionId);
// Also select the containing track
const region = regions.find(r => r.id === regionId);
if (!region) return;
const track = tracks.find(t => t.getId().toString() === region.trackId);
if (!track) return;
setSelectedTrack(track.getId().toString());
};
// Handle explicit pencil action: select region and open piano roll
const handleOpenPianoRoll = (regionId: string) => {
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Open piano roll via pencil for region: ${regionId}`);
}
// Reuse selection logic
handleRegionClick(regionId);
// Activate and show piano roll
setActiveRegionId(regionId);
setShowPianoRoll(true);
};
// Handle piano roll close
const handlePianoRollClose = () => {
setShowPianoRoll(false);
setActiveRegionId(null);
};
/**
* Add keyboard event listener for region deletion
* Handles Backspace (Windows) and Delete (Mac) keys to delete selected regions
* Only processes deletion when not in the piano roll (piano roll has its own delete handler)
*/
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
// Skip if user is typing in an input field (including ChatBox)
const target = event.target as HTMLElement;
if (target && (
target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.contentEditable === 'true' ||
target.hasAttribute('data-chatbox-input') ||
target.closest('.chatbox-input')
)) {
return;
}
// Handle delete key for selected regions (Backspace on Windows, Delete on Mac)
if (event.key === 'Backspace' || event.key === 'Delete') {
// Only handle if we're not in the piano roll (piano roll has its own delete handler)
const isInPianoRoll = document.querySelector('.piano-roll')?.contains(event.target as Node);
const isPianoRollOpen = showPianoRoll;
if (!isInPianoRoll && !isPianoRollOpen) {
const deleted = deleteSelectedRegions();
if (deleted) {
// Prevent default behavior only if regions were actually deleted
event.preventDefault();
}
}
}
};
// Add event listener
window.addEventListener('keydown', handleKeyDown);
// Remove event listener on cleanup
return () => {
window.removeEventListener('keydown', handleKeyDown);
};
}, [deleteSelectedRegions, showPianoRoll]); // Dependencies for the effect
// Utility function to calculate playhead position from mouse coordinates (bar-level snapping)
const calculatePlayheadFromMouse = useCallback((clientX: number): number | null => {
if (!barNumbersRef.current) return null;
const rect = barNumbersRef.current.getBoundingClientRect();
const relativeX = clientX - rect.left;
// Calculate the width of each bar
const barWidth = parseInt(
getComputedStyle(document.documentElement).getPropertyValue('--track-grid-bar-width')
) || 40;
// Find the closest bar start (using Math.round for nearest bar)
const barIndex = Math.round(relativeX / barWidth);
// Ensure we don't go below 0
const clampedBarIndex = Math.max(0, barIndex);
// Calculate destination beat position (start of the bar)
const beatsPerBar = timeSignature.numerator;
const destinationBeatPosition = clampedBarIndex * beatsPerBar;
return destinationBeatPosition;
}, [timeSignature]);
// Handle mouse down to start dragging
const handleBarNumbersMouseDown = (e: React.MouseEvent<HTMLDivElement>) => {
// Only handle left mouse button
if (e.button !== 0) return;
isDraggingRef.current = true;
// Calculate and set initial playhead position
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
setPlayheadPosition(newPosition);
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Bar numbers drag started - Initial position: ${newPosition} (bar ${Math.floor(newPosition / timeSignature.numerator) + 1})`);
}
}
// Prevent text selection during drag
e.preventDefault();
};
// Handle click on bar numbers to move playhead (when not dragging)
const handleBarNumbersClick = (e: React.MouseEvent<HTMLDivElement>) => {
// If we were dragging, don't process as a click
if (isDraggingRef.current) {
return;
}
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
const core = KGCore.instance();
const currentPlayheadPosition = core.getPlayheadPosition();
const beatsPerBar = timeSignature.numerator;
const currentBarNumber = Math.floor(currentPlayheadPosition / beatsPerBar) + 1; // 1-indexed
const destinationBarNumber = Math.floor(newPosition / beatsPerBar) + 1; // 1-indexed
// Debug logging
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Bar numbers click - Position: ${newPosition}`);
console.log(`Current bar: ${currentBarNumber} (beat ${currentPlayheadPosition})`);
console.log(`Destination bar: ${destinationBarNumber} (beat ${newPosition})`);
}
setPlayheadPosition(newPosition);
}
};
// Global mouse move and mouse up handlers for bar numbers drag functionality
useEffect(() => {
const handleMouseMove = (e: MouseEvent) => {
if (!isDraggingRef.current) return;
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
setPlayheadPosition(newPosition);
if (DEBUG_MODE.MAIN_CONTENT) {
console.log(`Bar numbers drag - Position: ${newPosition} (bar ${Math.floor(newPosition / timeSignature.numerator) + 1})`);
}
}
};
const handleMouseUp = () => {
if (isDraggingRef.current) {
isDraggingRef.current = false;
if (DEBUG_MODE.MAIN_CONTENT) {
console.log('Bar numbers drag ended');
}
}
};
// Add global event listeners for drag functionality
document.addEventListener('mousemove', handleMouseMove);
document.addEventListener('mouseup', handleMouseUp);
// Cleanup event listeners on unmount
return () => {
document.removeEventListener('mousemove', handleMouseMove);
document.removeEventListener('mouseup', handleMouseUp);
};
}, [calculatePlayheadFromMouse, setPlayheadPosition, timeSignature]);
const { showInstrumentSelection } = useProjectStore();
return (
<div className={`main-content${showInstrumentSelection ? ' has-left-instrument' : ''}`}>
<div className="main-content-wrapper">
{/* Top-left spacer */}
<div className="top-left-spacer"></div>
{/* Bar numbers at the top */}
<div
className="bar-numbers"
ref={barNumbersRef}
onMouseDown={handleBarNumbersMouseDown}
onClick={handleBarNumbersClick}
>
{Array.from({ length: maxBars }, (_, i) => (
<div key={i} className="bar-number-cell">{i + 1}</div>
))}
</div>
<div className="main-content-body">
{/* Fixed left panel with track info */}
<TrackInfoPanel
tracks={tracks}
onTrackClick={onTrackClick}
onTrackNameEdit={handleTrackNameEdit}
onTracksReordered={handleTracksReordered}
/>
{/* Scrollable grid area */}
<TrackGridPanel
tracks={tracks}
regions={regions}
maxBars={maxBars}
timeSignature={timeSignature}
draggedTrackIndex={draggedTrackIndex}
dragOverTrackIndex={dragOverTrackIndex}
selectedRegionId={selectedRegionId}
onRegionCreated={handleRegionCreated}
onRegionUpdated={handleRegionUpdated}
onRegionClick={handleRegionClick}
onOpenPianoRoll={handleOpenPianoRoll}
/>
</div>
</div>
{/* Piano Roll - render using portal */}
{showPianoRoll && createPortal(
<PianoRoll
onClose={handlePianoRollClose}
regionId={activeRegionId}
/>,
document.body
)}
</div>
);
};
export default MainContent;
src/components/StatusBar.tsx
+19
View File
@@ -0,0 +1,19 @@
import React from 'react';
import { useProjectStore } from '../stores/projectStore';
const StatusBar: React.FC = () => {
const { currentStatus } = useProjectStore();
return (
<div className="status-bar">
<div className="status-left">
{currentStatus}
</div>
<div className="status-right">
<span>K.G.Studio</span>
</div>
</div>
);
};
export default StatusBar;
src/components/Toolbar.tsx
+799
View File
@@ -0,0 +1,799 @@
import React from 'react';
import { saveProject } from '../util/saveUtil';
import { KGStorage } from '../core/io/KGStorage';
import { DB_CONSTANTS } from '../constants/coreConstants';
import { KGCore } from '../core/KGCore';
import { useProjectStore } from '../stores/projectStore';
import { DEBUG_MODE } from '../constants/uiConstants';
import { TIME_CONSTANTS } from '../constants/coreConstants';
import { parseTimeSignature, getTimeSignatureErrorMessage } from '../util/timeUtil';
import {
FaUndo, FaRedo, FaMousePointer, FaStepBackward,
FaPlay, FaPause, FaComments,
FaFolderOpen, FaSave, FaDownload, FaUpload, FaPlus,
FaCog
} from 'react-icons/fa';
import { KGProject, type KeySignature } from '../core/KGProject';
import { plainToClass, instanceToPlain } from 'class-transformer';
import { FaPencil, FaCopy, FaPaste, FaTrash } from 'react-icons/fa6';
import { KGMainContentState } from '../core/state/KGMainContentState';
import { regionDeleteManager } from '../util/regionDeleteUtil';
import { handleCopyOperation, handlePasteOperation } from '../util/copyPasteUtil';
import { convertProjectToMidi, convertMidiToProject } from '../util/midiUtil';
import { KEY_SIGNATURE_MAP } from '../constants/coreConstants';
import KGDropdown from './common/KGDropdown';
import FileImportModal from './common/FileImportModal';
import { clearChatHistoryAndUI } from '../util/chatUtil';
import PianoIcon from './common/icons/PianoIcon';
const Toolbar: React.FC = () => {
const {
projectName, setProjectName,
bpm, timeSignature, keySignature, setStatus,
isPlaying, startPlaying, stopPlaying, setPlayheadPosition,
currentTime, setBpm, setTimeSignature, setKeySignature,
maxBars, setMaxBars,
canUndo, canRedo, undoDescription, redoDescription, undo, redo,
toggleChatBox, toggleSettings, cleanupProjectState,
// Piano roll state/actions
showPianoRoll, setShowPianoRoll, activeRegionId, setActiveRegionId,
// Selection state
selectedRegionIds
} = useProjectStore();
// State for main content tools
const [activeMainTool, setActiveMainTool] = React.useState<'pointer' | 'pencil'>('pointer');
// State for key signature dropdown
const [showKeySignatureDropdown, setShowKeySignatureDropdown] = React.useState(false);
// State for export dropdown
const [showExportDropdown, setShowExportDropdown] = React.useState(false);
// State for import modal
const [showImportModal, setShowImportModal] = React.useState(false);
// Key signature options
const keySignatureOptions = Object.keys(KEY_SIGNATURE_MAP) as KeySignature[];
// Export options
const exportOptions = ["Export to KGStudio JSON file", "Export to MIDI file"];
const handleProjectNameClick = () => {
const newName = prompt("Enter project name:", projectName);
if (newName) setProjectName(newName);
};
// Common project loading logic extracted for reuse
const loadProjectFromData = async (project: KGProject, sourceDescription: string) => {
try {
// Clean up UI state first
cleanupProjectState();
// Automatically clear chat history when loading a project
clearChatHistoryAndUI();
// Load the project using the store's loadProject method
const { loadProject: storeLoadProject } = useProjectStore.getState();
await storeLoadProject(project);
// Update status to indicate project loaded
setStatus(`${sourceDescription} loaded successfully`);
if (DEBUG_MODE.TOOLBAR) {
console.log(`project loaded successfully from ${sourceDescription}`);
}
} catch (error) {
console.error(`Error loading project from ${sourceDescription}:`, error);
setStatus(`Failed to load project: ${error}`);
window.alert(`An error occurred while loading the project: ${error}`);
}
};
// Handler functions for file operations
const handleNewProject = () => {
const confirmed = window.confirm("Are you sure you want to create a new project? Any unsaved changes will be lost.");
if (confirmed) {
if (DEBUG_MODE.TOOLBAR) {
console.log("user clicked new button");
}
// Clean up UI state first
cleanupProjectState();
// Automatically clear chat history when creating a new project
clearChatHistoryAndUI();
// Create a new project with default parameters
const newProject = new KGProject();
// Load the new project using the store's loadProject method
const { loadProject: storeLoadProject } = useProjectStore.getState();
storeLoadProject(newProject);
// Default track creation is handled centrally in the store's loadProject
// Update status to indicate new project created
setStatus(`New project "${newProject.getName()}" created`);
if (DEBUG_MODE.TOOLBAR) {
console.log("new project created successfully");
}
}
};
const handleLoadProject = async () => {
const confirmed = window.confirm("Are you sure you want to load another project? Any unsaved changes will be lost.");
if (confirmed) {
if (DEBUG_MODE.TOOLBAR) {
console.log("user clicked load button");
}
// Ask user for project name
const projectNameToLoad = window.prompt("Enter the project name to load:");
// Check if user input is empty or null (user cancelled)
if (!projectNameToLoad || projectNameToLoad.trim() === '') {
if (projectNameToLoad !== null) { // Only show error if user didn't cancel
window.alert("Project name cannot be empty. Please enter a valid project name.");
}
return;
}
try {
// Try to load the project from storage
const storage = KGStorage.getInstance();
const loadedProject = await storage.load(
DB_CONSTANTS.DB_NAME,
DB_CONSTANTS.PROJECTS_STORE_NAME,
projectNameToLoad.trim(),
KGProject,
DB_CONSTANTS.DB_VERSION
);
if (!loadedProject) {
window.alert(`Project "${projectNameToLoad}" not found. Please check the project name and try again.`);
return;
}
// Use common loading logic
await loadProjectFromData(loadedProject, `Project "${projectNameToLoad}"`);
} catch (error) {
console.error("Error loading project:", error);
window.alert(`An error occurred while loading the project: ${error}`);
}
}
};
const handleSaveProject = async () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("user clicked save button");
}
await saveProject(projectName, setStatus);
};
const handleExportProject = (exportType: string) => {
if (DEBUG_MODE.TOOLBAR) {
console.log("user selected export option:", exportType);
}
if (exportType === "Export to KGStudio JSON file") {
handleExportKGStudioJSON();
} else if (exportType === "Export to MIDI file") {
handleExportMIDI();
}
setShowExportDropdown(false);
};
const handleExportKGStudioJSON = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("exporting to KGStudio JSON file");
}
try {
// Get the current project from KGCore
const currentProject = KGCore.instance().getCurrentProject();
// Serialize the project to JSON (same format as saved to IndexedDB)
// Use instanceToPlain to include type information for class-transformer
const projectData = JSON.stringify(instanceToPlain(currentProject), null, 2);
// Create a downloadable blob
const blob = new Blob([projectData], { type: 'application/json' });
// Create a temporary download link
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = `${projectName}.json`;
// Trigger download
document.body.appendChild(link);
link.click();
// Cleanup
document.body.removeChild(link);
URL.revokeObjectURL(url);
setStatus(`Project "${projectName}" exported as JSON file`);
if (DEBUG_MODE.TOOLBAR) {
console.log("KGStudio JSON export completed successfully");
}
} catch (error) {
console.error("Error exporting KGStudio JSON:", error);
setStatus(`Error exporting project: ${error}`);
window.alert(`Failed to export project as JSON: ${error}`);
}
};
const handleExportMIDI = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("exporting to MIDI file");
}
try {
// Get the current project from KGCore
const currentProject = KGCore.instance().getCurrentProject();
// Convert project to MIDI format
const midiData = convertProjectToMidi(currentProject);
// Create a downloadable blob
const blob = new Blob([midiData], { type: 'audio/midi' });
// Create a temporary download link
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = `${projectName}.mid`;
// Trigger download
document.body.appendChild(link);
link.click();
// Cleanup
document.body.removeChild(link);
URL.revokeObjectURL(url);
setStatus(`Project "${projectName}" exported as MIDI file`);
if (DEBUG_MODE.TOOLBAR) {
console.log("MIDI export completed successfully");
}
} catch (error) {
console.error("Error exporting MIDI:", error);
setStatus(`Error exporting MIDI: ${error}`);
window.alert(`Failed to export project as MIDI: ${error}`);
}
};
const handleImportProject = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("user clicked import button");
}
setShowImportModal(true);
};
const handleFileImport = async (file: File) => {
if (DEBUG_MODE.TOOLBAR) {
console.log("file selected for import:", file.name);
}
// Get file extension
const fileExtension = '.' + file.name.split('.').pop()?.toLowerCase();
try {
if (fileExtension === '.json') {
// Handle KGStudio JSON import
await handleKGStudioJSONImport(file);
} else if (fileExtension === '.mid' || fileExtension === '.midi') {
// Handle MIDI import
await handleMIDIImport(file);
} else {
throw new Error(`Unsupported file type: ${fileExtension}`);
}
} catch (error) {
console.error("Error importing file:", error);
setStatus(`Failed to import file: ${error}`);
window.alert(`Failed to import project file: ${error}`);
}
};
const handleKGStudioJSONImport = async (file: File) => {
try {
// Read the file content
const fileContent = await file.text();
const projectData = JSON.parse(fileContent);
// Deserialize the project data using class-transformer (same as KGStorage)
const deserializedResult = plainToClass(KGProject, projectData);
// Handle case where plainToClass might return an array
const deserializedProject = Array.isArray(deserializedResult)
? deserializedResult[0] || null
: deserializedResult;
if (!deserializedProject) {
throw new Error("Failed to deserialize project data");
}
// Load the project using common loading logic
await loadProjectFromData(deserializedProject, `File "${file.name}"`);
if (DEBUG_MODE.TOOLBAR) {
console.log("KGStudio JSON project imported successfully:", deserializedProject);
}
} catch (error) {
throw new Error(`Invalid KGStudio JSON file: ${error}`);
}
};
const handleMIDIImport = async (file: File) => {
try {
if (DEBUG_MODE.TOOLBAR) {
console.log("Starting MIDI file import:", file.name);
}
// Show loading status
setStatus(`Importing MIDI file "${file.name}"...`);
// Read the MIDI file as binary data
const arrayBuffer = await file.arrayBuffer();
const midiData = new Uint8Array(arrayBuffer);
if (DEBUG_MODE.TOOLBAR) {
console.log("MIDI file read successfully, size:", midiData.length, "bytes");
}
// Get current project to append MIDI tracks to it
const currentProject = KGCore.instance().getCurrentProject();
// Convert MIDI data and append to current project
const updatedProject = convertMidiToProject(midiData, currentProject);
if (DEBUG_MODE.TOOLBAR) {
console.log("MIDI conversion successful, tracks added to existing project");
}
// Load the updated project using common loading logic
await loadProjectFromData(updatedProject, `MIDI file "${file.name}"`);
if (DEBUG_MODE.TOOLBAR) {
console.log("MIDI file imported successfully:", file.name);
}
} catch (error) {
console.error("Error importing MIDI file:", error);
const errorMessage = error instanceof Error ? error.message : String(error);
setStatus(`Failed to import MIDI file: ${errorMessage}`);
throw new Error(`Invalid MIDI file: ${errorMessage}`);
}
};
// Handler functions for playback control
const handlePlayClick = async () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Play button clicked");
}
try {
await startPlaying();
} catch (error) {
console.error("Failed to start playback:", error);
setStatus("Playback failed to start");
}
};
const handlePauseClick = async () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Pause button clicked");
}
try {
await stopPlaying();
} catch (error) {
console.error("Failed to stop playback:", error);
setStatus("Failed to stop playback");
}
};
const handleBackToBeginningClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Back to beginning button clicked");
}
setPlayheadPosition(0);
};
// Prompt to change max bars when clicking on current-time display
const handleCurrentTimeClick = () => {
const MIN_BARS = 16;
const newMaxBarsStr = prompt(`Enter new max bars (>= ${MIN_BARS}):`, String(maxBars ?? 32));
if (newMaxBarsStr === null) {
return; // cancelled
}
const parsed = parseInt(newMaxBarsStr.trim(), 10);
if (isNaN(parsed)) {
alert('Invalid input. Please enter a valid number.');
return;
}
if (parsed < MIN_BARS) {
alert(`Invalid value. Please enter a number >= ${MIN_BARS}.`);
return;
}
setMaxBars(parsed);
setStatus(`Max bars changed to ${parsed}`);
};
const handleBpmClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("BPM clicked, current BPM:", bpm);
}
const newBpmStr = prompt(`Enter new BPM (${TIME_CONSTANTS.MIN_BPM}-${TIME_CONSTANTS.MAX_BPM}):`, bpm.toString());
// Check if user cancelled
if (newBpmStr === null) {
return;
}
// Validate input
const newBpm = parseInt(newBpmStr.trim());
// Check if it's a valid number
if (isNaN(newBpm)) {
alert("Invalid input. Please enter a valid number.");
return;
}
// Check if it's within valid range
if (newBpm <= TIME_CONSTANTS.MIN_BPM || newBpm >= TIME_CONSTANTS.MAX_BPM) {
alert(`Invalid BPM. Please enter a value between ${TIME_CONSTANTS.MIN_BPM} and ${TIME_CONSTANTS.MAX_BPM}.`);
return;
}
// Update BPM
setBpm(newBpm);
setStatus(`BPM changed to ${newBpm}`);
if (DEBUG_MODE.TOOLBAR) {
console.log(`BPM updated from ${bpm} to ${newBpm}`);
}
};
const handleTimeSignatureClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Time signature clicked, current:", `${timeSignature.numerator}/${timeSignature.denominator}`);
}
const currentTimeSignatureStr = `${timeSignature.numerator}/${timeSignature.denominator}`;
const newTimeSignatureStr = prompt(`Enter new time signature (numerator/denominator):`, currentTimeSignatureStr);
// Check if user cancelled
if (newTimeSignatureStr === null) {
return;
}
// Parse and validate time signature
const newTimeSignature = parseTimeSignature(newTimeSignatureStr);
if (newTimeSignature === null) {
alert(getTimeSignatureErrorMessage());
return;
}
// Update time signature
setTimeSignature(newTimeSignature);
setStatus(`Time signature changed to ${newTimeSignature.numerator}/${newTimeSignature.denominator}`);
if (DEBUG_MODE.TOOLBAR) {
console.log(`Time signature updated from ${currentTimeSignatureStr} to ${newTimeSignature.numerator}/${newTimeSignature.denominator}`);
}
};
const handleKeySignatureChange = (newKeySignature: string) => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Key signature changed from", keySignature, "to", newKeySignature);
}
setKeySignature(newKeySignature as KeySignature);
setStatus(`Key signature changed to ${newKeySignature}`);
setShowKeySignatureDropdown(false);
};
// Handle main content tool selection
const handleMainToolSelect = (tool: 'pointer' | 'pencil') => {
setActiveMainTool(tool);
KGMainContentState.instance().setActiveTool(tool);
if (DEBUG_MODE.TOOLBAR) {
console.log(`Selected main content tool: ${tool}`);
}
};
// Handle copy button click
const handleCopyClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Copy button clicked");
}
const copied = handleCopyOperation();
if (copied) {
setStatus("Items copied to clipboard");
if (DEBUG_MODE.TOOLBAR) {
console.log("Items copied successfully");
}
} else {
setStatus("No items selected to copy");
if (DEBUG_MODE.TOOLBAR) {
console.log("No items were selected for copying");
}
}
};
// Handle paste button click
const handlePasteClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Paste button clicked");
}
const pasted = handlePasteOperation();
if (pasted) {
setStatus("Items pasted from clipboard");
if (DEBUG_MODE.TOOLBAR) {
console.log("Items pasted successfully");
}
} else {
setStatus("Cannot paste - no valid clipboard content or context");
if (DEBUG_MODE.TOOLBAR) {
console.log("Paste operation failed or no valid context");
}
}
};
// Handle delete button click
const handleDeleteClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Delete button clicked");
}
const deleted = regionDeleteManager.deleteSelectedRegions();
if (deleted) {
setStatus("Selected regions deleted");
if (DEBUG_MODE.TOOLBAR) {
console.log("Regions deleted successfully");
}
} else {
setStatus("No regions selected for deletion");
if (DEBUG_MODE.TOOLBAR) {
console.log("No regions were selected for deletion");
}
}
};
// Handle undo button click
const handleUndoClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Undo button clicked");
}
if (!canUndo) {
alert("Nothing to undo");
return;
}
undo();
const description = undoDescription || "action";
setStatus(`Undid: ${description}`);
if (DEBUG_MODE.TOOLBAR) {
console.log(`Undo successful: ${description}`);
}
};
// Handle redo button click
const handleRedoClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Redo button clicked");
}
if (!canRedo) {
alert("Nothing to redo");
return;
}
redo();
const description = redoDescription || "action";
setStatus(`Redid: ${description}`);
if (DEBUG_MODE.TOOLBAR) {
console.log(`Redo successful: ${description}`);
}
};
// Handle chat button click
const handleChatClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Chat button clicked");
}
toggleChatBox();
setStatus("Chat toggled");
};
// Handle settings button click
const handleSettingsClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log("Settings button clicked");
}
toggleSettings();
setStatus("Settings toggled");
};
// Handle Piano button click: open piano roll if closed, targeting active or selected region
const handlePianoButtonClick = () => {
if (DEBUG_MODE.TOOLBAR) {
console.log('Piano button clicked');
}
// Only open if not already open
if (showPianoRoll) {
if (DEBUG_MODE.TOOLBAR) {
console.log('Piano roll already open; no action');
}
return;
}
// Prefer current active region; otherwise, first selected region
const candidateRegionId = activeRegionId || (selectedRegionIds && selectedRegionIds.length > 0 ? selectedRegionIds[0] : null);
if (!candidateRegionId) {
if (DEBUG_MODE.TOOLBAR) {
console.log('No active or selected region; piano roll will not open');
}
return;
}
setActiveRegionId(candidateRegionId);
setShowPianoRoll(true);
if (DEBUG_MODE.TOOLBAR) {
console.log(`Opening piano roll for region ${candidateRegionId}`);
}
};
return (
<>
<div className="toolbar">
<div className="toolbar-left">
<div className="logo-container">
<img src="/logo.png" alt="DAW Logo" className="logo" />
</div>
<div
className="project-name"
onClick={handleProjectNameClick}
>
{projectName}
</div>
</div>
<div className="toolbar-center">
<button title="New" onClick={handleNewProject}><FaPlus /></button>
<button title="Load" onClick={handleLoadProject}><FaFolderOpen /></button>
<button title="Save" onClick={handleSaveProject}><FaSave /></button>
<div style={{ position: 'relative', display: 'inline-block' }}>
<button
title="Export"
onClick={() => setShowExportDropdown(!showExportDropdown)}
style={{ display: 'flex', alignItems: 'center', gap: '4px' }}
>
<FaDownload />
</button>
<div style={{ position: 'absolute', top: '100%', left: 0, zIndex: 10000 }}>
<KGDropdown
options={exportOptions}
value={exportOptions[0]}
onChange={handleExportProject}
label="Export"
hideButton={true}
isOpen={showExportDropdown}
onToggle={setShowExportDropdown}
className="export-dropdown"
/>
</div>
</div>
<button title="Import" onClick={handleImportProject}><FaUpload /></button>
<div className="toolbar-separator"></div>
<button title="Undo" onClick={handleUndoClick}><FaUndo /></button>
<button title="Redo" onClick={handleRedoClick}><FaRedo /></button>
<div className="toolbar-separator"></div>
<button
title="Select"
className={`tool-button ${activeMainTool === 'pointer' ? 'active' : ''}`}
onClick={() => handleMainToolSelect('pointer')}
>
<FaMousePointer />
</button>
<button
title="Pencil"
className={`tool-button ${activeMainTool === 'pencil' ? 'active' : ''}`}
onClick={() => handleMainToolSelect('pencil')}
>
<FaPencil />
</button>
<div className="toolbar-separator"></div>
<button title="Copy" onClick={handleCopyClick}><FaCopy /></button>
<button title="Paste" onClick={handlePasteClick}><FaPaste /></button>
<button title="Delete" onClick={handleDeleteClick}><FaTrash /></button>
<div className="toolbar-separator"></div>
<button title="Back to beginning" className="button-back-to-beginning" onClick={handleBackToBeginningClick}><FaStepBackward /></button>
{!isPlaying ? (
<button title="Play" className="button-play" onClick={handlePlayClick}><FaPlay /></button>
) : (
<button title="Pause" className="button-pause" onClick={handlePauseClick}><FaPause /></button>
)}
<div className="toolbar-separator"></div>
<button title="Piano" onClick={handlePianoButtonClick}><PianoIcon /></button>
{/* <button title="Record"><FaCircle className="record-btn" /></button>
<button title="Metronome">🎵</button> */}
</div>
<div className="toolbar-right">
<div className="transport-control">
<div className="transport-item">
<span className='current-time' onClick={handleCurrentTimeClick} style={{ cursor: 'pointer' }}>{currentTime}</span>
</div>
<div className="transport-item">
<span className='current-bpm' onClick={handleBpmClick} style={{ cursor: 'pointer' }}>{bpm}</span>
</div>
<div className="transport-item">
<span className='current-time-signature' onClick={handleTimeSignatureClick} style={{ cursor: 'pointer' }}>{timeSignature.numerator + "/" + timeSignature.denominator}</span>
</div>
<div className="transport-item" style={{ position: 'relative' }}>
<span
className='current-key-signature'
onClick={() => setShowKeySignatureDropdown(!showKeySignatureDropdown)}
style={{ cursor: 'pointer' }}
>
{keySignature}
</span>
<div style={{ position: 'absolute', top: '100%', left: 0, zIndex: 10000 }}>
<KGDropdown
options={keySignatureOptions}
value={keySignature}
onChange={handleKeySignatureChange}
label="Key Signature"
hideButton={true}
isOpen={showKeySignatureDropdown}
onToggle={setShowKeySignatureDropdown}
className="key-signature-dropdown"
/>
</div>
</div>
</div>
<button title="Settings" onClick={handleSettingsClick}><FaCog /></button>
<button title="Chat" onClick={handleChatClick}><FaComments /></button>
</div>
</div>
<FileImportModal
isVisible={showImportModal}
onClose={() => setShowImportModal(false)}
onFileImport={handleFileImport}
acceptedTypes={['.json', '.mid', '.midi']}
title="Import Project"
description="Drag and drop your project file here"
/>
</>
);
};
export default Toolbar;
src/components/TrackControl.tsx
+18
View File
@@ -0,0 +1,18 @@
import React from 'react';
import { useProjectStore } from '../stores/projectStore';
const TrackControl: React.FC = () => {
const { addTrack } = useProjectStore();
const handleAddTrack = () => {
addTrack();
};
return (
<div className="track-control">
<button onClick={handleAddTrack}>+ Add track</button>
</div>
);
};
export default TrackControl;
src/components/chat/AssistantMessage.tsx
+180
View File
@@ -0,0 +1,180 @@
import React, { memo, useState } from 'react';
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
import { vscDarkPlus } from 'react-syntax-highlighter/dist/esm/styles/prism';
import { extractXMLFromString } from '../../util/xmlUtil';
interface ToolXMLExpanderProps {
toolName: string;
xmlContent: string;
}
const ToolXMLExpander: React.FC<ToolXMLExpanderProps> = ({ toolName, xmlContent }) => {
const [isExpanded, setIsExpanded] = useState(false);
return (
<div className="tool-xml-expander">
<div
className="tool-xml-expander-header"
onClick={() => setIsExpanded(!isExpanded)}
>
<span className="tool-xml-expander-arrow">
{isExpanded ? '▼' : '▶'}
</span>
<span className="tool-xml-expander-title">
🔧 Tool: {toolName}
</span>
</div>
{isExpanded && (
<div className="tool-xml-expander-content">
{xmlContent}
</div>
)}
</div>
);
};
interface AssistantMessageProps {
content: string;
isStreaming?: boolean;
onAbort?: () => void;
}
// Memoized code component to prevent SyntaxHighlighter re-renders
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const CodeComponent = memo(({ inline, className, children, ...props }: any) => {
const match = /language-(\w+)/.exec(className || '');
return !inline && match ? (
<SyntaxHighlighter
style={vscDarkPlus}
language={match[1]}
PreTag="div"
{...props}
>
{String(children).replace(/\n$/, '')}
</SyntaxHighlighter>
) : (
<code className={className} {...props}>
{children}
</code>
);
});
const AssistantMessage: React.FC<AssistantMessageProps> = ({ content, isStreaming, onAbort }) => {
// Function to process content and replace XML blocks with expanders
const processContentWithXMLExpanders = (text: string) => {
const xmlBlocks = extractXMLFromString(text);
if (xmlBlocks.length === 0) {
// No XML blocks found, return content as-is
return text;
}
let processedContent = text;
const expanders: React.ReactElement[] = [];
let expanderIndex = 0;
// Replace each XML block with a placeholder
xmlBlocks.forEach((xmlBlock) => {
const toolNameMatch = xmlBlock.match(/<([a-zA-Z_][a-zA-Z0-9_-]*)/);
const toolName = toolNameMatch ? toolNameMatch[1] : 'unknown_tool';
const placeholder = `__XML_EXPANDER_${expanderIndex}__`;
processedContent = processedContent.replace(xmlBlock, placeholder);
expanders[expanderIndex] = (
<ToolXMLExpander
key={`xml-expander-${expanderIndex}`}
toolName={toolName}
xmlContent={xmlBlock}
/>
);
expanderIndex++;
});
// Split content by placeholders and interleave with expanders
const parts = processedContent.split(/__XML_EXPANDER_\d+__/);
const result: (string | React.ReactElement)[] = [];
for (let i = 0; i < parts.length; i++) {
if (parts[i]) {
result.push(parts[i]);
}
if (i < expanders.length) {
result.push(expanders[i]);
}
}
return result;
};
// Handle special abort link for streaming messages
const renderContent = () => {
if (isStreaming && onAbort && content.includes('click here to abort')) {
const parts = content.split('click here to abort');
return (
<span>
{parts[0]}
<button
onClick={onAbort}
className="abort-link"
>
click here to abort
</button>
{parts[1]}
</span>
);
}
const processedContent = processContentWithXMLExpanders(content);
// If we have mixed content (text + React elements), render them separately
if (Array.isArray(processedContent)) {
return (
<div>
{processedContent.map((item, index) => {
if (typeof item === 'string') {
return (
<ReactMarkdown
key={`text-${index}`}
remarkPlugins={[remarkGfm]}
components={{
code: CodeComponent,
}}
>
{item}
</ReactMarkdown>
);
} else {
return item; // React element (expander)
}
})}
</div>
);
}
// Plain text content, render with markdown
return (
<ReactMarkdown
remarkPlugins={[remarkGfm]}
components={{
code: CodeComponent,
}}
>
{processedContent as string}
</ReactMarkdown>
);
};
return (
<div className="message-container message-assistant">
<div className="message-content">
{renderContent()}
</div>
</div>
);
};
export default memo(AssistantMessage);
src/components/chat/UserMessage.tsx
+19
View File
@@ -0,0 +1,19 @@
import React, { memo } from 'react';
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
interface UserMessageProps {
content: string;
}
const UserMessage: React.FC<UserMessageProps> = ({ content }) => {
return (
<div className="message-container message-user">
<div className="message-content">
<ReactMarkdown remarkPlugins={[remarkGfm]}>{content}</ReactMarkdown>
</div>
</div>
);
};
export default memo(UserMessage);
src/components/chat/index.ts
+2
View File
@@ -0,0 +1,2 @@
export { default as UserMessage } from './UserMessage';
export { default as AssistantMessage } from './AssistantMessage';
src/components/common/FileImportModal.tsx
+130
View File
@@ -0,0 +1,130 @@
import React, { useCallback, useState } from 'react';
import { FaTimes } from 'react-icons/fa';
interface FileImportModalProps {
isVisible: boolean;
onClose: () => void;
onFileImport: (file: File) => void;
acceptedTypes?: string[];
title?: string;
description?: string;
}
const FileImportModal: React.FC<FileImportModalProps> = ({
isVisible,
onClose,
onFileImport,
acceptedTypes = ['.json'],
title = 'Import Project',
description = 'Drag and drop your project file here'
}) => {
const [isDragOver, setIsDragOver] = useState(false);
const handleDragEnter = useCallback((e: React.DragEvent) => {
e.preventDefault();
e.stopPropagation();
setIsDragOver(true);
}, []);
const handleDragLeave = useCallback((e: React.DragEvent) => {
e.preventDefault();
e.stopPropagation();
// Only set drag over to false if we're leaving the drop zone entirely
if (e.currentTarget === e.target) {
setIsDragOver(false);
}
}, []);
const handleDragOver = useCallback((e: React.DragEvent) => {
e.preventDefault();
e.stopPropagation();
}, []);
const handleDrop = useCallback((e: React.DragEvent) => {
e.preventDefault();
e.stopPropagation();
setIsDragOver(false);
const files = Array.from(e.dataTransfer.files);
if (files.length > 0) {
const file = files[0];
// Check if file type is accepted
const fileExtension = '.' + file.name.split('.').pop()?.toLowerCase();
if (acceptedTypes.includes(fileExtension)) {
onFileImport(file);
onClose();
} else {
alert(`Invalid file type. Please select a file with one of these extensions: ${acceptedTypes.join(', ')}`);
}
}
}, [acceptedTypes, onFileImport, onClose]);
const handleFileSelect = useCallback((e: React.ChangeEvent<HTMLInputElement>) => {
const files = e.target.files;
if (files && files.length > 0) {
onFileImport(files[0]);
onClose();
}
}, [onFileImport, onClose]);
const handleOverlayClick = useCallback((e: React.MouseEvent) => {
// Only close if clicking on the overlay itself, not the modal content
if (e.target === e.currentTarget) {
onClose();
}
}, [onClose]);
if (!isVisible) {
return null;
}
return (
<div className="file-import-overlay" onClick={handleOverlayClick}>
<div className="file-import-modal">
<div className="file-import-header">
<h3 className="file-import-title">{title}</h3>
<button
className="file-import-close-btn"
onClick={onClose}
aria-label="Close import modal"
>
<FaTimes />
</button>
</div>
<div
className={`file-import-drop-zone ${isDragOver ? 'drag-over' : ''}`}
onDragEnter={handleDragEnter}
onDragLeave={handleDragLeave}
onDragOver={handleDragOver}
onDrop={handleDrop}
>
<div className="file-import-drop-content">
<div className="file-import-icon">📁</div>
<p className="file-import-description">{description}</p>
<p className="file-import-formats">
Supported formats: {acceptedTypes.join(', ')}
</p>
<div className="file-import-divider">
<span>or</span>
</div>
<label className="file-import-browse-btn">
Browse Files
<input
type="file"
accept={acceptedTypes.join(',')}
onChange={handleFileSelect}
style={{ display: 'none' }}
/>
</label>
</div>
</div>
</div>
</div>
);
};
export default FileImportModal;
src/components/common/KGDropdown.tsx
+107
View File
@@ -0,0 +1,107 @@
import React, { useState, useRef, useEffect } from 'react';
import { FaCaretDown } from 'react-icons/fa';
type DropdownOption = string | { label: string; value: string };
interface KGDropdownProps {
options: DropdownOption[];
value: string;
onChange: (value: string) => void;
label: string;
className?: string;
buttonClassName?: string;
optionClassName?: string;
showValueAsLabel?: boolean;
hideButton?: boolean;
isOpen?: boolean;
onToggle?: (open: boolean) => void;
}
const KGDropdown: React.FC<KGDropdownProps> = ({
options,
value,
onChange,
label,
className = '',
buttonClassName = '',
optionClassName = '',
showValueAsLabel = false,
hideButton = false,
isOpen: externalIsOpen,
onToggle
}) => {
const [internalIsOpen, setInternalIsOpen] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
// Use external state if provided, otherwise use internal state
const isOpen = externalIsOpen !== undefined ? externalIsOpen : internalIsOpen;
const setIsOpen = onToggle || setInternalIsOpen;
// Close dropdown when clicking outside
useEffect(() => {
const handleClickOutside = (event: MouseEvent) => {
if (
isOpen &&
dropdownRef.current &&
!dropdownRef.current.contains(event.target as Node)
) {
setIsOpen(false);
}
};
document.addEventListener('mousedown', handleClickOutside);
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, [isOpen]);
// Handle option selection
const handleSelect = (option: DropdownOption) => {
const value = typeof option === 'string' ? option : option.value;
onChange(value);
setIsOpen(false);
};
const resolveLabel = (option: DropdownOption) => (typeof option === 'string' ? option : option.label);
const resolveValue = (option: DropdownOption) => (typeof option === 'string' ? option : option.value);
const selectedLabel = (() => {
if (!showValueAsLabel) return label;
// Try to find the label for the current value
const match = options.find(opt => resolveValue(opt) === value);
return match ? resolveLabel(match) : value;
})();
const buttonText = showValueAsLabel ? selectedLabel : label;
return (
<div className={`quant-dropdown-container ${className}`} ref={dropdownRef}>
{!hideButton && (
<button
className={`quant-button ${buttonClassName}`}
onClick={() => setIsOpen(!isOpen)}
>
{buttonText} <FaCaretDown />
</button>
)}
{isOpen && (
<div className="quant-dropdown">
{options.map((option) => {
const optionValue = resolveValue(option);
return (
<div
key={optionValue}
className={`quant-option ${value === optionValue ? 'active' : ''} ${optionClassName}`}
onClick={() => handleSelect(option)}
>
{resolveLabel(option)}
</div>
);
})}
</div>
)}
</div>
);
};
export default KGDropdown;
src/components/common/LoadingOverlay.tsx
+23
View File
@@ -0,0 +1,23 @@
import React from 'react';
interface LoadingOverlayProps {
visible: boolean;
message?: string;
}
const LoadingOverlay: React.FC<LoadingOverlayProps> = ({ visible, message = 'Loading ...' }) => {
if (!visible) return null;
return (
<div className="global-loading-overlay" role="status" aria-live="polite" aria-busy={true}>
<div className="global-loading-content">
<div className="global-loading-spinner" />
<div className="global-loading-text">{message}</div>
</div>
</div>
);
};
export default LoadingOverlay;
src/components/common/Playhead.tsx
+82
View File
@@ -0,0 +1,82 @@
import React from 'react';
import { KGCore } from '../../core/KGCore';
import { useProjectStore } from '../../stores/projectStore';
interface PlayheadProps {
/** Context where the playhead is being rendered */
context: 'main-grid' | 'piano-roll';
/** For piano roll context, the region start beat offset */
regionStartBeat?: number;
}
const Playhead: React.FC<PlayheadProps> = ({ context, regionStartBeat = 0 }) => {
const { timeSignature, playheadPosition } = useProjectStore();
// Calculate the pixel position based on context
const getPixelPosition = (): number => {
if (context === 'main-grid') {
// In main grid, convert beats to bars, then bars to pixels
const beatsPerBar = timeSignature.numerator;
const barPosition = playheadPosition / beatsPerBar;
// Get bar width from CSS variable
const barWidth = parseInt(
getComputedStyle(document.documentElement).getPropertyValue('--track-grid-bar-width')
) || 40;
return barPosition * barWidth;
} else {
// In piano roll, use beat-based positioning
// Get beat width from CSS variable
const beatWidth = parseInt(
getComputedStyle(document.documentElement).getPropertyValue('--region-grid-beat-width')
) || 40;
return playheadPosition * beatWidth;
}
};
const pixelPosition = getPixelPosition();
// Don't render if position is negative (before region start in piano roll)
if (pixelPosition < 0) {
return null;
}
const playheadStyle: React.CSSProperties = {
position: 'absolute',
left: `${pixelPosition}px`,
top: 0,
bottom: 0,
width: '2px',
backgroundColor: '#4ECDC4', // Blue-green color similar to the reference image
zIndex: 1000,
pointerEvents: 'none', // Allow clicks to pass through
boxShadow: '0 0 4px rgba(78, 205, 196, 0.5)', // Subtle glow effect
};
// Triangle indicator style (only for main-grid context)
const triangleStyle: React.CSSProperties = {
position: 'absolute',
left: `${pixelPosition - 5}px`, // Center the triangle on the playhead line
top: '-2px', // Position slightly above the top
width: 0,
height: 0,
borderLeft: '6px solid transparent',
borderRight: '6px solid transparent',
borderTop: '8px solid #4ECDC4',
zIndex: 1001,
pointerEvents: 'none',
};
return (
<>
<div className="playhead" style={playheadStyle} />
{context === 'main-grid' && (
<div className="playhead-triangle" style={triangleStyle} />
)}
</>
);
};
export default Playhead;
src/components/common/icons/PianoIcon.tsx
+46
View File
@@ -0,0 +1,46 @@
import React from 'react';
type PianoIconProps = React.SVGProps<SVGSVGElement>;
/**
* PianoIcon – keyboard-style icon in a Font Awesome-like solid style
* - Uses currentColor
* - Scales with font size (1em)
* - viewBox matches FA dimensions
*/
const PianoIcon: React.FC<PianoIconProps> = (props) => (
<svg
viewBox="0 0 576 512"
width="1em"
height="1em"
fill="currentColor"
aria-hidden="true"
focusable="false"
{...props}
>
{/*
Build a frame with even-odd fill, then add inner black keys as filled bars.
Outer frame: 48,64 → 528x320
Inner hole: 96,112 → 384x224
Top slot: 112,128 → 352x32
Black keys: four bars centered
*/}
<path
fillRule="evenodd"
clipRule="evenodd"
d="
M48 64h480v320H48V64z
M96 112h384v224H96V112z
M112 128h352v32H112v-32z
M176 160h24v136h-24V160z
M240 160h24v136h-24V160z
M304 160h24v136h-24V160z
M368 160h24v136h-24V160z
"
/>
</svg>
);
export default PianoIcon;
src/components/common/index.ts
+4
View File
@@ -0,0 +1,4 @@
export { default as KGDropdown } from './KGDropdown';
export { default as Playhead } from './Playhead';
export { default as FileImportModal } from './FileImportModal';
export { default as LoadingOverlay } from './LoadingOverlay';
src/components/interfaces.ts
+42
View File
@@ -0,0 +1,42 @@
// selectable interface
export interface Selectable {
getId(): string;
select(): void;
deselect(): void;
isSelected(): boolean;
getRootType(): string;
getCurrentType(): string;
}
// Define a Region interface for UI representation
export interface RegionUI {
id: string;
trackId: string;
trackIndex: number;
barNumber: number;
length: number;
name: string;
}
// Define resize action types
export type ResizeAction = 'none' | 'start' | 'end';
// Define region resize state
export interface RegionResizeState {
regionId: string;
isResizing: boolean;
resizeAction: ResizeAction;
initialX: number;
initialBarNumber: number;
initialLength: number;
}
// Define region drag state
export interface RegionDragState {
regionId: string;
isDragging: boolean;
initialX: number;
initialY: number;
initialBarNumber: number;
initialTrackIndex: number;
}
src/components/piano-roll/PianoGrid.tsx
+164
View File
@@ -0,0 +1,164 @@
import React, { useState, useEffect } from 'react';
import type { MutableRefObject } from 'react';
import { Playhead } from '../common';
import SelectionBox from './SelectionBox';
import { isModifierKeyPressed } from '../../util/osUtil';
interface PianoGridProps {
gridRef: MutableRefObject<HTMLDivElement | null>;
children: React.ReactNode;
onDoubleClick: (e: React.MouseEvent) => void;
onClick: (e: React.MouseEvent) => void;
onMouseDown: (e: React.MouseEvent) => void;
isBoxSelecting: boolean;
selectionBox: {
startX: number;
startY: number;
endX: number;
endY: number;
};
regionStartBeat?: number;
}
interface CursorPosition {
beat: number;
pitch: number;
x: number;
y: number;
}
const PianoGrid: React.FC<PianoGridProps> = ({
gridRef,
children,
onDoubleClick,
onClick,
onMouseDown,
isBoxSelecting,
selectionBox,
regionStartBeat = 0
}) => {
const [cursorPosition, setCursorPosition] = useState<CursorPosition | null>(null);
const [isModifierPressed, setIsModifierPressed] = useState(false);
// Track modifier key state
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
// Skip if user is typing in an input field (including ChatBox)
const target = e.target as HTMLElement;
if (target && (
target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.contentEditable === 'true' ||
target.hasAttribute('data-chatbox-input') ||
target.closest('.chatbox-input')
)) {
return;
}
if (isModifierKeyPressed(e)) {
setIsModifierPressed(true);
}
};
const handleKeyUp = (e: KeyboardEvent) => {
// Skip if user is typing in an input field (including ChatBox)
const target = e.target as HTMLElement;
if (target && (
target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.contentEditable === 'true' ||
target.hasAttribute('data-chatbox-input') ||
target.closest('.chatbox-input')
)) {
return;
}
if (!isModifierKeyPressed(e)) {
setIsModifierPressed(false);
}
};
// Add global event listeners
window.addEventListener('keydown', handleKeyDown);
window.addEventListener('keyup', handleKeyUp);
// Cleanup listeners on unmount
return () => {
window.removeEventListener('keydown', handleKeyDown);
window.removeEventListener('keyup', handleKeyUp);
};
}, []);
const handleMouseMove = (e: React.MouseEvent) => {
if (!gridRef.current) return;
const rect = gridRef.current.getBoundingClientRect();
const x = e.clientX - rect.left;
const y = e.clientY - rect.top;
// Get CSS variables
const beatWidth = parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-grid-beat-width')) || 40;
const noteHeight = parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-height')) || 20;
// Calculate beat and pitch
const beat = Math.floor(x / beatWidth);
const pitch = 107 - Math.floor(y / noteHeight); // B7 = 107, reverse for display
// Only update if position changed and cursor is within valid range
if (beat >= 0 && pitch >= 0 && pitch <= 127) {
setCursorPosition({ beat, pitch, x, y });
}
};
const handleMouseLeave = () => {
setCursorPosition(null);
};
return (
<div className="piano-grid-container">
<div
className={`piano-grid ${isModifierPressed ? 'pencil-cursor' : ''}`}
ref={gridRef}
onDoubleClick={onDoubleClick}
onClick={onClick}
onMouseDown={(e) => onMouseDown(e)}
onMouseMove={handleMouseMove}
onMouseLeave={handleMouseLeave}
>
{/* Cursor Highlights */}
{cursorPosition && (
<>
{/* Horizontal pitch row highlight */}
<div
className="piano-grid-pitch-highlight"
style={{
top: Math.floor(cursorPosition.y / (parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-height')) || 20)) * (parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-height')) || 20),
height: parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-height')) || 20
}}
/>
{/* Vertical beat column highlight */}
<div
className="piano-grid-beat-highlight"
style={{
left: cursorPosition.beat * (parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-grid-beat-width')) || 40),
width: parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-grid-beat-width')) || 40
}}
/>
</>
)}
{/* Playhead */}
<Playhead context="piano-roll" regionStartBeat={regionStartBeat} />
{children}
<SelectionBox
isSelecting={isBoxSelecting}
selectionBox={selectionBox}
/>
</div>
</div>
);
};
export default PianoGrid;
src/components/piano-roll/PianoGridHeader.tsx
+180
View File
@@ -0,0 +1,180 @@
import React, { useRef, useEffect, useCallback } from 'react';
import { KGCore } from '../../core/KGCore';
import { KGPianoRollState } from '../../core/state/KGPianoRollState';
import { useProjectStore } from '../../stores/projectStore';
import { DEBUG_MODE } from '../../constants';
interface PianoGridHeaderProps {
maxBars: number;
timeSignature?: { numerator: number; denominator: number };
}
const PianoGridHeader: React.FC<PianoGridHeaderProps> = ({
maxBars,
timeSignature = { numerator: 4, denominator: 4 } // Default to 4/4 if not provided
}) => {
// Get store access for playhead position updates
const { setPlayheadPosition } = useProjectStore();
// Refs for drag functionality
const isDraggingRef = useRef(false);
const headerElementRef = useRef<HTMLDivElement | null>(null);
// Utility function to calculate snapped beat position (based on useNoteOperations.ts)
const getSnappedBeatPosition = (beatPosition: number): number => {
const currentSnap = KGPianoRollState.instance().getCurrentSnap();
// If no snapping is enabled, return the original position
if (currentSnap === 'NO SNAP') {
return beatPosition;
}
// Parse the snap value (e.g., "1/4", "1/8", "1/16", "1/32")
const denominator = parseInt(currentSnap.split('/')[1]);
if (isNaN(denominator)) {
return beatPosition; // Fallback to no snapping if invalid
}
// Calculate the snap step in beats
// snapStep should ALWAYS be 4 / denominator regardless of time signature
const snapStep = 4 / denominator;
// Use round snapping for playhead positioning
const snappedPosition = Math.round(beatPosition / snapStep) * snapStep;
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Piano Grid Header Snapping: ${beatPosition} -> ${snappedPosition} (snap: ${currentSnap}, step: ${snapStep})`);
}
return snappedPosition;
};
// Utility function to calculate playhead position from mouse coordinates
const calculatePlayheadFromMouse = useCallback((clientX: number): number | null => {
if (!headerElementRef.current) return null;
const rect = headerElementRef.current.getBoundingClientRect();
const relativeX = clientX - rect.left;
// Account for the piano keys width offset
const pianoKeysWidth = parseInt(
getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-width')
) || 60;
const adjustedX = relativeX - pianoKeysWidth;
// If the click is in the piano keys area (left side), ignore it
if (adjustedX < 0) {
return null;
}
// Calculate the width of each beat
const beatWidth = parseInt(
getComputedStyle(document.documentElement).getPropertyValue('--region-grid-beat-width')
) || 40;
// Calculate the raw beat position using the adjusted X position
const rawBeatPosition = adjustedX / beatWidth;
// Apply quantization if enabled
return getSnappedBeatPosition(rawBeatPosition);
}, []);
// Handle mouse down to start dragging
const handleMouseDown = (e: React.MouseEvent<HTMLDivElement>) => {
// Only handle left mouse button
if (e.button !== 0) return;
isDraggingRef.current = true;
// Calculate and set initial playhead position
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
setPlayheadPosition(newPosition);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Piano Grid Header drag started - Initial position: ${newPosition}`);
}
}
// Prevent text selection during drag
e.preventDefault();
};
// Handle click (when not dragging) - this will be the fallback for simple clicks
const handlePianoGridHeaderClick = (e: React.MouseEvent<HTMLDivElement>) => {
// If we were dragging, don't process as a click
if (isDraggingRef.current) {
return;
}
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
const core = KGCore.instance();
const currentPlayheadPosition = core.getPlayheadPosition();
const beatsPerBar = timeSignature.numerator;
const currentBarNumber = Math.floor(currentPlayheadPosition / beatsPerBar) + 1; // 1-indexed
const destinationBarNumber = Math.floor(newPosition / beatsPerBar) + 1; // 1-indexed
// Debug logging
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Piano Grid Header click - Position: ${newPosition}`);
console.log(`Current bar: ${currentBarNumber} (beat ${currentPlayheadPosition})`);
console.log(`Destination bar: ${destinationBarNumber} (beat ${newPosition})`);
}
setPlayheadPosition(newPosition);
}
};
// Global mouse move and mouse up handlers
useEffect(() => {
const handleMouseMove = (e: MouseEvent) => {
if (!isDraggingRef.current) return;
const newPosition = calculatePlayheadFromMouse(e.clientX);
if (newPosition !== null) {
setPlayheadPosition(newPosition);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Piano Grid Header drag - Position: ${newPosition}`);
}
}
};
const handleMouseUp = () => {
if (isDraggingRef.current) {
isDraggingRef.current = false;
if (DEBUG_MODE.PIANO_ROLL) {
console.log('Piano Grid Header drag ended');
}
}
};
// Add global event listeners for drag functionality
document.addEventListener('mousemove', handleMouseMove);
document.addEventListener('mouseup', handleMouseUp);
// Cleanup event listeners on unmount
return () => {
document.removeEventListener('mousemove', handleMouseMove);
document.removeEventListener('mouseup', handleMouseUp);
};
}, [calculatePlayheadFromMouse, setPlayheadPosition]);
return (
<div
className="piano-grid-header"
ref={headerElementRef}
onMouseDown={handleMouseDown}
onClick={handlePianoGridHeaderClick}
>
{Array.from({ length: maxBars }, (_, i) => (
<div key={i} className="piano-bar-number">{i + 1}</div>
))}
</div>
);
};
export default PianoGridHeader;
src/components/piano-roll/PianoKeys.tsx
+178
View File
@@ -0,0 +1,178 @@
import React, { useState, useRef } from 'react';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { KGAudioInterface } from '../../core/audio-interface/KGAudioInterface';
import { noteNameToPitch, midiPercussionKeyMap, pitchToNoteNameString } from '../../util/midiUtil';
import { useProjectStore } from '../../stores/projectStore';
import { KGMidiTrack } from '../../core/track/KGMidiTrack';
interface PianoKeysProps {
activeRegion: KGMidiRegion | null;
}
const PianoKeys: React.FC<PianoKeysProps> = ({ activeRegion }) => {
const [pressedKeys, setPressedKeys] = useState<Set<string>>(new Set());
const pressedKeysRef = useRef<Set<string>>(new Set());
const { tracks } = useProjectStore();
// Check if current active region belongs to a drum track
const isDrumTrack = React.useMemo(() => {
if (!activeRegion) return false;
const track = tracks.find(t => t.getId().toString() === activeRegion.getTrackId());
return track instanceof KGMidiTrack && track.getInstrument() === 'standard';
}, [activeRegion, tracks]);
// Handle mouse down on piano key
const handleKeyMouseDown = (keyId: string) => {
// Prevent double pressing the same key
if (pressedKeysRef.current.has(keyId)) {
return;
}
// Get the track ID from active region
if (!activeRegion) {
console.warn('No active region, cannot play piano key');
return;
}
const trackId = activeRegion.getTrackId();
try {
// Convert note name to pitch (keyId is always a note name like "C4")
const pitch = noteNameToPitch(keyId);
// Get audio interface and start playing the note
const audioInterface = KGAudioInterface.instance();
if (audioInterface.getIsInitialized()) {
// Try to start audio context if not started yet
if (!audioInterface.getIsAudioContextStarted()) {
audioInterface.startAudioContext().catch(() => {
// Silently fail if still not allowed - browser policy
});
}
// Trigger note attack if audio context is ready
if (audioInterface.getIsAudioContextStarted()) {
audioInterface.triggerNoteAttack(trackId, pitch, 127);
// Update pressed keys state
const newPressedKeys = new Set(pressedKeysRef.current);
newPressedKeys.add(keyId);
pressedKeysRef.current = newPressedKeys;
setPressedKeys(newPressedKeys);
console.log(`Started playing piano key: ${keyId} (pitch ${pitch})`);
}
}
} catch (error) {
console.error(`Error playing piano key ${keyId}:`, error);
}
};
// Handle mouse up on piano key
const handleKeyMouseUp = (keyId: string) => {
// Only release if key was actually pressed
if (!pressedKeysRef.current.has(keyId)) {
return;
}
// Get the track ID from active region
if (!activeRegion) {
return;
}
const trackId = activeRegion.getTrackId();
try {
// Convert note name to pitch (keyId is always a note name like "C4")
const pitch = noteNameToPitch(keyId);
// Get audio interface and stop playing the note
const audioInterface = KGAudioInterface.instance();
if (audioInterface.getIsInitialized() && audioInterface.getIsAudioContextStarted()) {
audioInterface.releaseNote(trackId, pitch);
// Update pressed keys state
const newPressedKeys = new Set(pressedKeysRef.current);
newPressedKeys.delete(keyId);
pressedKeysRef.current = newPressedKeys;
setPressedKeys(newPressedKeys);
console.log(`Stopped playing piano key: ${keyId} (pitch ${pitch})`);
}
} catch (error) {
console.error(`Error releasing piano key ${keyId}:`, error);
}
};
// Handle mouse leave to ensure keys are released
const handleKeyMouseLeave = (keyId: string) => {
handleKeyMouseUp(keyId);
};
// Generate piano keys (C0 to C7)
const generatePianoKeys = () => {
const octaves = [];
const notes = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'];
// Generate octaves from C0 to C7
for (let octave = 0; octave <= 7; octave++) {
const octaveKeys = [];
// Add keys in reverse order (B to C) for each octave
for (let i = notes.length - 1; i >= 0; i--) {
const note = notes[i];
const isSharp = note.includes('#');
const keyId = `${note}${octave}`;
const isPressed = pressedKeys.has(keyId);
const keyClass = `piano-key ${isSharp ? 'sharp' : 'natural'} ${isPressed ? 'pressed' : ''}`;
const isC = note === 'C';
// For drum tracks, show drum labels when available
let labelContent = null;
if (isDrumTrack) {
const pitch = noteNameToPitch(keyId);
const drumInfo = midiPercussionKeyMap[pitch];
if (drumInfo) {
labelContent = <span className="key-label">{drumInfo.shortName}</span>;
}
} else if (isC) {
labelContent = <span className="key-label">C{octave}</span>;
}
octaveKeys.push(
<div
key={keyId}
className={keyClass}
data-note={keyId}
onMouseDown={() => handleKeyMouseDown(keyId)}
onMouseUp={() => handleKeyMouseUp(keyId)}
onMouseLeave={() => handleKeyMouseLeave(keyId)}
style={{
cursor: 'pointer',
userSelect: 'none' // Prevent text selection
}}
>
{labelContent}
</div>
);
}
// Add each octave to the beginning of the array
octaves.unshift(
<div key={`octave-${octave}`} className="piano-octave">
{octaveKeys}
</div>
);
}
return octaves;
};
return (
<div className="piano-keys-container">
{generatePianoKeys()}
</div>
);
};
export default PianoKeys;
src/components/piano-roll/PianoNote.tsx
+268
View File
@@ -0,0 +1,268 @@
import React, { useState, useRef, useEffect } from 'react';
import { PIANO_ROLL_CONSTANTS, DEBUG_MODE } from '../../constants';
import { useProjectStore } from '../../stores/projectStore';
interface PianoNoteProps {
id: string;
index: number;
left: number;
top: number;
width: number;
height: number;
onResizeStart?: (noteId: string, resizeEdge: 'start' | 'end', initialX: number) => void;
onResize?: (noteId: string, resizeEdge: 'start' | 'end', deltaX: number) => void;
onResizeEnd?: (noteId: string, resizeEdge: 'start' | 'end') => void;
onDragStart?: (noteId: string, initialX: number, initialY: number) => void;
onDrag?: (noteId: string, deltaX: number, deltaY: number) => void;
onDragEnd?: (noteId: string) => void;
onClick?: (noteId: string, e: React.MouseEvent) => void;
}
const PianoNote: React.FC<PianoNoteProps> = ({
id,
index,
left,
top,
width,
height,
onResizeStart,
onResize,
onResizeEnd,
onDragStart,
onDrag,
onDragEnd,
onClick
}) => {
// Get selection state from store
const { selectedNoteIds } = useProjectStore();
const isSelected = selectedNoteIds.includes(id);
const [cursor, setCursor] = useState<string>('pointer');
const [resizeEdge, setResizeEdge] = useState<'none' | 'start' | 'end'>('none');
const [isResizing, setIsResizing] = useState(false);
const [isDragging, setIsDragging] = useState(false);
// Use refs to track states for immediate access
const isResizingRef = useRef<boolean>(false);
const isDraggingRef = useRef<boolean>(false);
const initialMousePosRef = useRef<{x: number, y: number}>({x: 0, y: 0});
const hasMovedRef = useRef<boolean>(false);
// Handle mouse movement to detect edge proximity
const handleMouseMove = (e: React.MouseEvent<HTMLDivElement>) => {
// Skip if already resizing or dragging
if (isResizingRef.current || isDraggingRef.current) return;
const noteElement = e.currentTarget;
const rect = noteElement.getBoundingClientRect();
// Calculate distance from left and right edges
const distanceFromLeft = e.clientX - rect.left;
const distanceFromRight = rect.right - e.clientX;
// Use the edge threshold constant from constants file
const edgeThreshold = PIANO_ROLL_CONSTANTS.NOTE_EDGE_OFFSET;
if (distanceFromLeft <= edgeThreshold) {
// Near left edge - resize from start
setCursor('ew-resize');
setResizeEdge('start');
} else if (distanceFromRight <= edgeThreshold) {
// Near right edge - resize from end
setCursor('ew-resize');
setResizeEdge('end');
} else {
// Middle area - move
setCursor('grab');
setResizeEdge('none');
}
};
// Reset cursor when mouse leaves
const handleMouseLeave = () => {
if (!isResizingRef.current && !isDraggingRef.current) {
setCursor('default');
setResizeEdge('none');
}
};
// Handle mouse down for resize or drag
const handleMouseDown = (e: React.MouseEvent<HTMLDivElement>) => {
// Prevent text selection during resize/drag
e.preventDefault();
// Reset movement tracking
hasMovedRef.current = false;
// Store initial mouse position
initialMousePosRef.current = { x: e.clientX, y: e.clientY };
if (resizeEdge !== 'none') {
// Start resizing
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE RESIZE START: noteId=${id}, edge=${resizeEdge}`);
}
setIsResizing(true);
isResizingRef.current = true;
// Call the onResizeStart callback if provided
if (onResizeStart && (resizeEdge === 'start' || resizeEdge === 'end')) {
onResizeStart(id, resizeEdge, e.clientX);
}
} else {
// Start dragging
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE DRAG START: noteId=${id}`);
}
setIsDragging(true);
isDraggingRef.current = true;
// Change cursor to grabbing during drag
setCursor('grabbing');
// Call the onDragStart callback if provided
if (onDragStart) {
onDragStart(id, e.clientX, e.clientY);
}
}
// Add global event listeners for mouse move and up
document.addEventListener('mousemove', handleGlobalMouseMove);
document.addEventListener('mouseup', handleGlobalMouseUp);
};
// Handle global mouse move for resize or drag
const handleGlobalMouseMove = (e: MouseEvent) => {
// Set the hasMovedRef to true as soon as there's movement
hasMovedRef.current = true;
if (isResizingRef.current) {
// Handle resize
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE RESIZE MOVE: noteId=${id}, edge=${resizeEdge}`);
}
// Calculate delta from initial position
const deltaX = e.clientX - initialMousePosRef.current.x;
// Call the onResize callback if provided
if (onResize && (resizeEdge === 'start' || resizeEdge === 'end')) {
onResize(id, resizeEdge, deltaX);
}
} else if (isDraggingRef.current) {
// Handle drag
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE DRAG MOVE: noteId=${id}`);
}
// Calculate delta from initial position
const deltaX = e.clientX - initialMousePosRef.current.x;
const deltaY = e.clientY - initialMousePosRef.current.y;
// Call the onDrag callback if provided
if (onDrag) {
onDrag(id, deltaX, deltaY);
}
}
};
// Handle global mouse up to end resize or drag
const handleGlobalMouseUp = (e: MouseEvent) => {
if (isResizingRef.current) {
// End resizing
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE RESIZE END: noteId=${id}`);
}
setIsResizing(false);
isResizingRef.current = false;
// Call the onResizeEnd callback if provided
if (onResizeEnd && (resizeEdge === 'start' || resizeEdge === 'end')) {
onResizeEnd(id, resizeEdge);
}
} else if (isDraggingRef.current) {
// End dragging
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE DRAG END: noteId=${id}`);
}
setIsDragging(false);
isDraggingRef.current = false;
// Reset cursor after drag
setCursor('grab');
// Call the onDragEnd callback if provided
if (onDragEnd) {
onDragEnd(id);
}
// If there was no movement, treat it as a click
if (!hasMovedRef.current && onClick) {
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`NOTE CLICKED: noteId=${id}`);
}
// We can't pass the original event here since it's a MouseEvent, not a React.MouseEvent
// But we can create a synthetic event with the current mouse position
const clickEvent = {
clientX: e.clientX,
clientY: e.clientY,
target: e.target,
preventDefault: () => {},
stopPropagation: () => {},
shiftKey: e.shiftKey // Pass the shift key state
} as unknown as React.MouseEvent;
onClick(id, clickEvent);
}
}
// Remove global event listeners
document.removeEventListener('mousemove', handleGlobalMouseMove);
document.removeEventListener('mouseup', handleGlobalMouseUp);
};
// Clean up event listeners on unmount
useEffect(() => {
return () => {
document.removeEventListener('mousemove', handleGlobalMouseMove);
document.removeEventListener('mouseup', handleGlobalMouseUp);
};
}, []);
// Keep the refs in sync with the states
useEffect(() => {
isResizingRef.current = isResizing;
}, [isResizing]);
useEffect(() => {
isDraggingRef.current = isDragging;
}, [isDragging]);
return (
<div
className={`piano-note ${isDragging ? 'dragging' : ''} ${isResizing ? 'resizing' : ''} ${isSelected ? 'selected' : ''}`}
style={{
left: `${left}px`,
top: `${top}px`,
width: `${width}px`,
height: `${height}px`,
cursor: cursor
}}
onMouseMove={handleMouseMove}
onMouseLeave={handleMouseLeave}
onMouseDown={handleMouseDown}
id={id}
data-note-index={index}
data-resize-edge={resizeEdge}
data-is-resizing={isResizing}
data-is-dragging={isDragging}
data-is-selected={isSelected}
/>
);
};
export default PianoNote;
src/components/piano-roll/PianoRoll.tsx
+793
View File
@@ -0,0 +1,793 @@
import React, { useRef, useEffect, useState, useCallback } from 'react';
import type { MouseEvent } from 'react';
import { useProjectStore } from '../../stores/projectStore';
import { FaGripLines } from 'react-icons/fa';
import { KGMidiRegion } from '../../core/region/KGMidiRegion';
import { DEBUG_MODE, PIANO_ROLL_CONSTANTS } from '../../constants';
import PianoRollHeader from './PianoRollHeader';
import PianoRollToolbar from './PianoRollToolbar';
import PianoRollContent from './PianoRollContent';
import { KGCore } from '../../core/KGCore';
import { KGMidiNote } from '../../core/midi/KGMidiNote';
import { KGPianoRollState } from '../../core/state/KGPianoRollState';
import { ConfigManager } from '../../core/config/ConfigManager';
import { beatsToBar } from '../../util/midiUtil';
import { UpdateRegionCommand } from '../../core/commands';
interface PianoRollProps {
onClose: () => void;
regionId: string | null;
initialPosition?: { x: number; y: number };
initialSize?: { width: number; height: number };
}
const PianoRoll: React.FC<PianoRollProps> = ({
onClose,
regionId,
initialPosition,
initialSize
}) => {
const { maxBars, tracks, updateTrack, timeSignature, showChatBox, showInstrumentSelection } = useProjectStore();
// Tool state for piano roll
const [activeTool, setActiveTool] = useState<'pointer' | 'pencil'>('pointer');
// Quantization state
const [quantPosition, setQuantPosition] = useState<string>('1/8');
const [quantLength, setQuantLength] = useState<string>('1/8');
// Snapping state
const [snapping, setSnapping] = useState<string>('NO SNAP');
// Piano roll state with temporary initial values
const [position, setPosition] = useState(initialPosition || { x: 0, y: 0 });
// Blink effect state for toolbar button feedback
const [blinkButton, setBlinkButton] = useState<string | null>(null);
const [size, setSize] = useState(initialSize || { width: 800, height: PIANO_ROLL_CONSTANTS.PIANO_ROLL_HEIGHT });
const [isDragging, setIsDragging] = useState(false);
const [isResizing, setIsResizing] = useState(false);
const [dragOffset, setDragOffset] = useState({ x: 0, y: 0 });
const [activeRegion, setActiveRegion] = useState<KGMidiRegion | null>(null);
const pianoRollRef = useRef<HTMLDivElement>(null);
const pianoRollContentRef = useRef<HTMLDivElement>(null);
const pianoGridRef = useRef<HTMLDivElement>(null);
const wasDraggingRef = useRef<boolean>(false);
// Ref for storing the setNoteUpdateCounter function
const triggerNoteUpdateRef = useRef<React.Dispatch<React.SetStateAction<number>> | null>(null);
// Ref for storing the deleteSelectedNotes function
const deleteSelectedNotesRef = useRef<(() => boolean) | null>(null);
// Calculate initial position and size once on mount
useEffect(() => {
// Skip if initialPosition or initialSize were provided as props
if (!initialPosition || !initialSize) {
// Calculate initial position based on window dimensions
const calculateInitialPosition = () => {
// Dynamically get heights from CSS computed styles
const statusBarElement = document.querySelector('.status-bar');
const trackControlElement = document.querySelector('.track-control');
// Get actual heights from DOM elements, or use fallback values if elements don't exist yet
const statusBarHeight = statusBarElement ? statusBarElement.clientHeight : 30;
const trackControlHeight = trackControlElement ? trackControlElement.clientHeight : 30;
const pianoRollHeight = PIANO_ROLL_CONSTANTS.PIANO_ROLL_HEIGHT;
// Compute left offset when instrument selection panel is open
const rootStyles = getComputedStyle(document.documentElement);
const instrumentPanelWidthStr = rootStyles.getPropertyValue('--instrument-selection-width') || '300px';
const instrumentPanelWidth = parseInt(instrumentPanelWidthStr, 10) || 300;
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Positioning piano roll with heights - statusBar: ${statusBarHeight}px, trackControl: ${trackControlHeight}px, pianoRoll: ${pianoRollHeight}px`);
}
return {
x: showInstrumentSelection ? instrumentPanelWidth : 0,
y: window.innerHeight - statusBarHeight - trackControlHeight - pianoRollHeight
};
};
const calculateInitialSize = () => {
const rootStyles = getComputedStyle(document.documentElement);
const chatBoxWidthStr = rootStyles.getPropertyValue('--chat-box-width') || '350px';
const instrumentPanelWidthStr = rootStyles.getPropertyValue('--instrument-selection-width') || '300px';
const chatBoxWidth = parseInt(chatBoxWidthStr, 10) || 350;
const instrumentPanelWidth = parseInt(instrumentPanelWidthStr, 10) || 300;
let availableWidth = window.innerWidth;
if (showChatBox) availableWidth -= chatBoxWidth;
if (showInstrumentSelection) availableWidth -= instrumentPanelWidth;
// Ensure a sensible minimum starting width
const clampedWidth = Math.max(400, availableWidth);
return {
width: clampedWidth,
height: PIANO_ROLL_CONSTANTS.PIANO_ROLL_HEIGHT
};
};
// Set position and size only if not provided as props
if (!initialPosition) {
setPosition(calculateInitialPosition());
}
if (!initialSize) {
setSize(calculateInitialSize());
}
}
// Intentionally run once on mount to capture layout at open time
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []); // Empty dependency array means this runs once on mount
// Find and set the active region when regionId changes
useEffect(() => {
if (!regionId) {
setActiveRegion(null);
return;
}
// Find the region in the tracks
for (const track of tracks) {
const regions = track.getRegions();
const region = regions.find(r => r.getId() === regionId);
if (region && region instanceof KGMidiRegion) {
setActiveRegion(region);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Active region set in PianoRoll: ${region.getId()}`);
console.log(`Region details: name=${region.getName()}, trackId=${region.getTrackId()}, trackIndex=${region.getTrackIndex()}`);
}
break;
}
}
}, [regionId, tracks]);
// Sync local state with KGPianoRollState on mount
useEffect(() => {
const pianoRollState = KGPianoRollState.instance();
// Sync snapping state
const currentSnap = pianoRollState.getCurrentSnap();
setSnapping(currentSnap);
// Sync tool state
const currentTool = pianoRollState.getActiveTool() as 'pointer' | 'pencil';
setActiveTool(currentTool);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Synced piano roll state on mount - snap: ${currentSnap}, tool: ${currentTool}`);
}
}, []); // Empty dependency array means this runs once on mount
// Add keyboard event listener for Escape
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
// Close on Escape key
if (event.key === 'Escape') {
if (DEBUG_MODE.PIANO_ROLL) {
console.log('Closing piano roll with ESC key');
}
onClose();
}
};
// Add event listener
window.addEventListener('keydown', handleKeyDown);
// Remove event listener on cleanup
return () => {
window.removeEventListener('keydown', handleKeyDown);
};
}, [onClose]);
// Handle mouse events for dragging and resizing
const handleMouseDown = (e: React.MouseEvent, action: 'drag' | 'resize') => {
if (action === 'drag') {
setIsDragging(true);
wasDraggingRef.current = false; // Reset the dragging flag
if (pianoRollRef.current) {
const rect = pianoRollRef.current.getBoundingClientRect();
setDragOffset({
x: e.clientX - rect.left,
y: e.clientY - rect.top
});
}
} else if (action === 'resize') {
setIsResizing(true);
e.preventDefault();
}
};
useEffect(() => {
const handleMouseMove = (e: MouseEvent) => {
if (isDragging) {
// Set the flag to true as soon as any movement happens
wasDraggingRef.current = true;
setPosition({
x: e.clientX - dragOffset.x,
y: e.clientY - dragOffset.y
});
} else if (isResizing) {
setSize({
width: Math.max(400, e.clientX - position.x),
height: Math.max(300, e.clientY - position.y)
});
}
};
const handleMouseUp = () => {
setIsDragging(false);
setIsResizing(false);
// We keep wasDraggingRef.current as is - it will be used in handleTitleClick
// and reset on the next mousedown
};
if (isDragging || isResizing) {
document.addEventListener('mousemove', handleMouseMove as unknown as EventListener);
document.addEventListener('mouseup', handleMouseUp);
}
return () => {
document.removeEventListener('mousemove', handleMouseMove as unknown as EventListener);
document.removeEventListener('mouseup', handleMouseUp);
};
}, [isDragging, isResizing, dragOffset, position]);
// Handle title click to rename the region
const handleTitleClick = () => {
// If we were just dragging, don't show the rename dialog
if (wasDraggingRef.current) {
if (DEBUG_MODE.PIANO_ROLL) {
console.log("Skipping rename dialog because the window was just dragged");
}
return;
}
if (!activeRegion) return;
// Show a prompt to get the new name
const newName = window.prompt("Enter a new name for the region:", activeRegion.getName());
// If the user clicked Cancel or entered an empty string, do nothing
if (!newName || newName.trim() === '' || newName === activeRegion.getName()) return;
// Use command pattern to update the region name with undo support
try {
const command = new UpdateRegionCommand(activeRegion.getId(), { name: newName.trim() });
KGCore.instance().executeCommand(command);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Executed UpdateRegionCommand: renamed region ${activeRegion.getId()} to "${newName}" using command pattern`);
}
// Update the store to trigger re-render
const updatedTracks = [...tracks];
useProjectStore.setState({ tracks: updatedTracks });
} catch (error) {
console.error('Error renaming region:', error);
// Optionally show user-friendly error message
alert('Failed to rename region. Please try again.');
}
};
// Handle tool selection
const handleToolSelect = (tool: 'pointer' | 'pencil') => {
setActiveTool(tool);
KGPianoRollState.instance().setActiveTool(tool);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Selected tool: ${tool}`);
}
};
// Handle snapping selection
const handleSnappingSelect = useCallback((value: string) => {
setSnapping(value);
KGPianoRollState.instance().setCurrentSnap(value);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Selected snapping: ${value}`);
}
}, []);
// Handler for receiving the setNoteUpdateCounter function from PianoRollContent
const handleSetNoteUpdateTrigger = (setNoteFn: React.Dispatch<React.SetStateAction<number>>) => {
triggerNoteUpdateRef.current = setNoteFn;
};
// Handler for receiving the deleteSelectedNotes function from PianoRollContent
const handleSetDeleteNotesTrigger = (deleteFn: () => boolean) => {
deleteSelectedNotesRef.current = deleteFn;
};
// Quantize selected notes based on the selected quantization value
const quantizeSelectedNotes = useCallback((quantValue: string) => {
if (!activeRegion) return;
// Get the KGCore instance
const core = KGCore.instance();
// Get all selected notes
const selectedItems = core.getSelectedItems();
const selectedNotes = selectedItems.filter(item =>
item instanceof KGMidiNote &&
activeRegion.getNotes().some(note => note.getId() === item.getId())
) as KGMidiNote[];
if (selectedNotes.length === 0) {
if (DEBUG_MODE.PIANO_ROLL) {
console.log('No notes selected for quantization');
}
return;
}
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Quantizing ${selectedNotes.length} selected notes with value: ${quantValue}`);
}
// Parse the quantization value (e.g., "1/4", "1/8", "1/16", "1/32")
const denominator = parseInt(quantValue.split('/')[1]);
if (isNaN(denominator)) {
console.error(`Invalid quantization value: ${quantValue}`);
return;
}
// Calculate the quantization step in beats
// In a 4/4 time signature, a quarter note (1/4) is 1 beat
// In a 6/8 time signature, an eighth note (1/8) is 1 beat
const { numerator, denominator: timeSigDenominator } = timeSignature;
// Calculate beats per whole note based on time signature
// In 4/4, a whole note is 4 beats
// In 6/8, a whole note is 6 beats (because each beat is an eighth note)
const beatsPerWholeNote = numerator * (4 / timeSigDenominator);
// Calculate the quantization step in beats
// quantizationStep should ALWAYS be 4 / denominator regardless of time signature
const quantizationStep = 4 / denominator;
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Time signature: ${numerator}/${timeSigDenominator}`);
console.log(`Beats per whole note: ${beatsPerWholeNote}`);
console.log(`Quantization step: ${quantizationStep} beats`);
}
// Apply quantization to each selected note
selectedNotes.forEach(note => {
// Get the current start beat
const currentStartBeat = note.getStartBeat();
// Calculate the quantized start beat
const quantizedStartBeat = Math.round(currentStartBeat / quantizationStep) * quantizationStep;
// Calculate the duration of the note
const duration = note.getEndBeat() - currentStartBeat;
// Set the new start beat and maintain the duration
note.setStartBeat(quantizedStartBeat);
note.setEndBeat(quantizedStartBeat + duration);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Quantized note ${note.getId()}: ${currentStartBeat} -> ${quantizedStartBeat}`);
}
});
// Find the track that contains this region and update it
const track = tracks.find(t => t.getId().toString() === activeRegion.getTrackId());
if (track) {
updateTrack(track);
}
// Trigger a re-render by incrementing the note update counter
if (triggerNoteUpdateRef.current) {
triggerNoteUpdateRef.current(prev => prev + 1);
if (DEBUG_MODE.PIANO_ROLL) {
console.log('Triggered note update to re-render quantized notes');
}
}
}, [activeRegion, timeSignature, updateTrack, tracks]);
// Quantize selected notes length based on the selected quantization value
const quantizeNoteLength = useCallback((quantValue: string) => {
if (!activeRegion) return;
// Get the KGCore instance
const core = KGCore.instance();
// Get all selected notes
const selectedItems = core.getSelectedItems();
const selectedNotes = selectedItems.filter(item =>
item instanceof KGMidiNote &&
activeRegion.getNotes().some(note => note.getId() === item.getId())
) as KGMidiNote[];
if (selectedNotes.length === 0) {
if (DEBUG_MODE.PIANO_ROLL) {
console.log('No notes selected for length quantization');
}
return;
}
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Quantizing length of ${selectedNotes.length} selected notes with value: ${quantValue}`);
}
// Parse the quantization value (e.g., "1/1", "1/2", "1/4", "1/8", "1/16", "1/32")
const denominator = parseInt(quantValue.split('/')[1]);
if (isNaN(denominator)) {
console.error(`Invalid quantization value: ${quantValue}`);
return;
}
// Calculate the quantization step in beats
// In a 4/4 time signature, a quarter note (1/4) is 1 beat
// In a 6/8 time signature, an eighth note (1/8) is 1 beat
const { numerator, denominator: timeSigDenominator } = timeSignature;
// Calculate beats per whole note based on time signature
// In 4/4, a whole note is 4 beats
// In 6/8, a whole note is 6 beats (because each beat is an eighth note)
const beatsPerWholeNote = numerator * (4 / timeSigDenominator);
// Calculate the quantization step in beats
// quantizationStep should ALWAYS be 4 / denominator regardless of time signature
const quantizationStep = 4 / denominator;
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Time signature: ${numerator}/${timeSigDenominator}`);
console.log(`Beats per whole note: ${beatsPerWholeNote}`);
console.log(`Length quantization step: ${quantizationStep} beats`);
}
// Apply quantization to each selected note
selectedNotes.forEach(note => {
// Get the current start and end beats
const startBeat = note.getStartBeat();
const currentEndBeat = note.getEndBeat();
// Calculate the current duration
const currentDuration = currentEndBeat - startBeat;
// Calculate the quantized duration
// If the current duration is less than the quantization step,
// extend it to match the quantization step exactly
// Otherwise, round to the nearest multiple of quantizationStep
let quantizedDuration;
if (currentDuration < quantizationStep) {
// For notes shorter than the quantization step, extend to exactly one step
quantizedDuration = quantizationStep;
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Extending short note ${note.getId()} from ${currentDuration} to ${quantizedDuration}`);
}
} else {
// For longer notes, round to nearest multiple of quantizationStep
quantizedDuration = Math.round(currentDuration / quantizationStep) * quantizationStep;
}
// Ensure minimum note length
quantizedDuration = Math.max(PIANO_ROLL_CONSTANTS.MIN_NOTE_LENGTH, quantizedDuration);
// Set the new end beat while maintaining the start beat
note.setEndBeat(startBeat + quantizedDuration);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Quantized note length ${note.getId()}: ${currentDuration} -> ${quantizedDuration}`);
}
});
// Find the track that contains this region and update it
const track = tracks.find(t => t.getId().toString() === activeRegion.getTrackId());
if (track) {
updateTrack(track);
}
// Trigger a re-render by incrementing the note update counter
if (triggerNoteUpdateRef.current) {
triggerNoteUpdateRef.current(prev => prev + 1);
if (DEBUG_MODE.PIANO_ROLL) {
console.log('Triggered note update to re-render quantized note lengths');
}
}
}, [activeRegion, timeSignature, updateTrack, tracks]);
// Handle quantization selection
const handleQuantSelect = useCallback((type: 'position' | 'length', value: string) => {
if (type === 'position') {
setQuantPosition(value);
// Apply quantization immediately when position quantization is changed
quantizeSelectedNotes(value);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`quant-position selected: ${value}`);
}
} else {
setQuantLength(value);
// Apply length quantization immediately when length quantization is changed
quantizeNoteLength(value);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`quant-length selected: ${value}`);
}
}
}, [quantizeSelectedNotes, quantizeNoteLength]);
// Calculate C4 position and scroll to it when piano roll opens
useEffect(() => {
if (pianoRollContentRef.current) {
// Calculate position of C4
// We have 8 octaves (0-7), and C4 is in the middle
// Each octave has 12 notes, each note is piano key height
// C4 is in octave 4, and C is the first note in each octave
// Calculate from the bottom:
// - Octaves 0-3 = 4 octaves = 4 * 12 * piano key height
// - Within octave 4, C is the first note (from bottom), so 0px additional
const keyHeight = parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-piano-key-height')) || 20;
const c4Position = 4 * 12 * keyHeight; // pixels from bottom
// Total height of all notes (8 octaves * 12 notes * piano key height)
const totalHeight = 8 * 12 * keyHeight;
// Get the viewport height of the piano roll content
const viewportHeight = pianoRollContentRef.current.clientHeight;
// Calculate scroll position to center C4
// We need to scroll from the top, so we calculate:
// (total height - C4 position) - (viewport height / 2)
const scrollPosition = (totalHeight - c4Position) - (viewportHeight / 2);
// Scroll to the calculated position
pianoRollContentRef.current.scrollTop = Math.max(0, scrollPosition);
}
}, []);
// Scroll horizontally to the active region's starting bar
useEffect(() => {
if (pianoRollContentRef.current && activeRegion) {
// Get the starting beat of the region
const startBeat = activeRegion.getStartFromBeat();
// Get the time signature to calculate beats per bar
const beatsPerBar = timeSignature.numerator;
// Calculate the bar number (0-indexed)
const barNumber = Math.floor(startBeat / beatsPerBar);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Scrolling to region's starting bar: ${barNumber + 1} (startBeat: ${startBeat}, beatsPerBar: ${beatsPerBar})`);
}
// Calculate the pixel position (each bar is --region-grid-bar-width wide, which is 160px by default)
const barWidth = parseInt(getComputedStyle(document.documentElement).getPropertyValue('--region-grid-bar-width')) || 160;
// Calculate the scroll position to scroll to the starting bar
const scrollPosition = barNumber * barWidth;
// Scroll to the calculated position
pianoRollContentRef.current.scrollLeft = Math.max(0, scrollPosition);
}
}, [activeRegion, timeSignature]);
// Add keyboard event listener for piano roll hotkeys (snapping and quantization)
useEffect(() => {
const handlePianoRollKeyDown = (event: KeyboardEvent) => {
// Skip if user is typing in an input field (including ChatBox)
const target = event.target as HTMLElement;
if (target && (
target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.contentEditable === 'true' ||
target.hasAttribute('data-chatbox-input') ||
target.closest('.chatbox-input')
)) {
return;
}
// Handle delete key for selected notes
if (event.key === 'Backspace' || event.key === 'Delete') {
if (deleteSelectedNotesRef.current) {
const deleted = deleteSelectedNotesRef.current();
if (deleted) {
// Prevent default behavior only if notes were actually deleted
event.preventDefault();
}
}
return;
}
// Handle piano roll hotkeys
const configManager = ConfigManager.instance();
if (configManager.getIsInitialized()) {
// Snapping hotkeys
const snap_none_key = configManager.get('hotkeys.piano_roll.snap_none') as string;
const snap_1_4_key = configManager.get('hotkeys.piano_roll.snap_1_4') as string;
const snap_1_8_key = configManager.get('hotkeys.piano_roll.snap_1_8') as string;
const snap_1_16_key = configManager.get('hotkeys.piano_roll.snap_1_16') as string;
// Quantize position hotkeys
const qua_pos_1_4_key = configManager.get('hotkeys.piano_roll.qua_pos_1_4') as string;
const qua_pos_1_8_key = configManager.get('hotkeys.piano_roll.qua_pos_1_8') as string;
const qua_pos_1_16_key = configManager.get('hotkeys.piano_roll.qua_pos_1_16') as string;
// Quantize length hotkeys
const qua_len_1_4_key = configManager.get('hotkeys.piano_roll.qua_len_1_4') as string;
const qua_len_1_8_key = configManager.get('hotkeys.piano_roll.qua_len_1_8') as string;
const qua_len_1_16_key = configManager.get('hotkeys.piano_roll.qua_len_1_16') as string;
let actionType: 'snap' | 'quantize' | null = null;
let actionValue: string | null = null;
let quantType: 'position' | 'length' | null = null;
// Check snapping hotkeys
if (event.key === snap_none_key) {
actionType = 'snap';
actionValue = 'NO SNAP';
} else if (event.key === snap_1_4_key) {
actionType = 'snap';
actionValue = '1/4';
} else if (event.key === snap_1_8_key) {
actionType = 'snap';
actionValue = '1/8';
} else if (event.key === snap_1_16_key) {
actionType = 'snap';
actionValue = '1/16';
}
// Check quantize position hotkeys
else if (event.key === qua_pos_1_4_key) {
actionType = 'quantize';
actionValue = '1/4';
quantType = 'position';
} else if (event.key === qua_pos_1_8_key) {
actionType = 'quantize';
actionValue = '1/8';
quantType = 'position';
} else if (event.key === qua_pos_1_16_key) {
actionType = 'quantize';
actionValue = '1/16';
quantType = 'position';
}
// Check quantize length hotkeys
else if (event.key === qua_len_1_4_key) {
actionType = 'quantize';
actionValue = '1/4';
quantType = 'length';
} else if (event.key === qua_len_1_8_key) {
actionType = 'quantize';
actionValue = '1/8';
quantType = 'length';
} else if (event.key === qua_len_1_16_key) {
actionType = 'quantize';
actionValue = '1/16';
quantType = 'length';
}
if (actionType && actionValue) {
// Prevent default behavior
event.preventDefault();
if (actionType === 'snap') {
// Validate the snap value exists in snap options
if (KGPianoRollState.SNAP_OPTIONS.includes(actionValue)) {
// Change snapping value
handleSnappingSelect(actionValue);
// Trigger blink effect for visual feedback
setBlinkButton('snapping');
setTimeout(() => setBlinkButton(null), 200);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Snap hotkey triggered: ${event.key} → ${actionValue}`);
}
}
} else if (actionType === 'quantize' && quantType) {
// Validate the quantValue exists in the appropriate options
const validOptions = quantType === 'length' ? KGPianoRollState.QUANT_LEN_OPTIONS : KGPianoRollState.QUANT_POS_OPTIONS;
if (validOptions.includes(actionValue)) {
// Apply quantization
handleQuantSelect(quantType, actionValue);
// Trigger blink effect for visual feedback
const buttonName = quantType === 'length' ? 'quant-length' : 'quant-position';
setBlinkButton(buttonName);
setTimeout(() => setBlinkButton(null), 200);
if (DEBUG_MODE.PIANO_ROLL) {
console.log(`Quantize ${quantType} hotkey triggered: ${event.key} → ${actionValue}`);
}
}
}
}
}
};
// Add event listener
window.addEventListener('keydown', handlePianoRollKeyDown);
// Remove event listener on cleanup
return () => {
window.removeEventListener('keydown', handlePianoRollKeyDown);
};
}, [handleQuantSelect, handleSnappingSelect]);
// Get the title for the piano roll based on the active region
const getPianoRollTitle = () => {
if (!activeRegion) return "EDIT NOTE CLIP";
// Calculate the bar and beat position of the region
const startBeat = activeRegion.getStartFromBeat();
const { bar, beatInBar } = beatsToBar(startBeat, timeSignature);
// Format as 1-indexed bar and beat (bar + 1, beatInBar + 1)
const barNumber = bar + 1;
const beatNumber = beatInBar + 1;
return `${activeRegion.getName()} (at ${barNumber}:${beatNumber})`;
};
return (
<div
className="piano-roll-panel"
style={{
position: 'fixed',
left: `${position.x}px`,
top: `${position.y}px`,
width: `${size.width}px`,
height: `${size.height}px`,
zIndex: 2000
}}
ref={pianoRollRef}
>
<PianoRollHeader
onClose={onClose}
title={getPianoRollTitle()}
onTitleClick={handleTitleClick}
onMouseDown={(e) => handleMouseDown(e, 'drag')}
/>
<PianoRollToolbar
activeTool={activeTool}
onToolSelect={handleToolSelect}
quantPosition={quantPosition}
quantLength={quantLength}
onQuantSelect={handleQuantSelect}
snapping={snapping}
onSnappingSelect={handleSnappingSelect}
blinkButton={blinkButton}
/>
<PianoRollContent
contentRef={pianoRollContentRef}
pianoGridRef={pianoGridRef}
maxBars={maxBars}
timeSignature={timeSignature}
activeRegion={activeRegion}
updateTrack={updateTrack}
tracks={tracks}
onSetNoteUpdateTrigger={handleSetNoteUpdateTrigger}
onSetDeleteNotesTrigger={handleSetDeleteNotesTrigger}
/>
<div
className="resize-handle"
onMouseDown={(e) => handleMouseDown(e, 'resize')}
>
<FaGripLines />
</div>
</div>
);
};
export default PianoRoll;

Some files were not shown because too many files have changed in this diff Show More