docs: updated README.md, added K.G.One Music Studio information
This commit is contained in:
@@ -2,14 +2,28 @@
|
||||
<img src="./public/logo.png" alt="K.G.Studio Logo" width="160" />
|
||||
</div>
|
||||
|
||||
## K.G.Studio — A Browser-based DAW with AI Assistant
|
||||
# K.G.Studio — A Browser-based DAW with AI Assistant
|
||||
|
||||
## New!
|
||||
|
||||
> ### ✦ [**K.G.One Music Studio is available now.**](https://github.com/KGAudioLab/K.G.One) ✦ <br />
|
||||
> [**K.G.One Music Studio**](https://github.com/KGAudioLab/K.G.One) is a fully local, open-source integrated platform built around **K.G.Studio** (this project). It bundles [ACE-Step 1.5](https://github.com/ace-step/ACE-Step-1.5) for full-song generation, [Foundation-1](https://huggingface.co/RoyalCities/Foundation-1) for audio clip and MIDI loop generation, and [python-audio-separator (UVR5)](https://github.com/nomadkaraoke/python-audio-separator) for stem separation — bringing GPU-accelerated AI music generation directly into your browser-based production workflow.
|
||||
|
||||
## What is K.G.Studio?
|
||||
|
||||
K.G.Studio is a lightweight, modern DAW that runs entirely in the browser with **K.G.Studio Musician Assistant** at its core. It features realistic instrument playback via Tone.js samplers, a piano‑roll editor, track and region management with full undo/redo, project persistence to IndexedDB, a configurable settings panel, and an integrated AI assistant with tool execution.
|
||||
|
||||
**K.G.Studio Musician Assistant** is an AI assistance agent for harmony, arrangement, and note editing — but not full auto‑composition.
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/KGOne-Demo-GIF.gif" alt="K.G.One Logo" width="640" />
|
||||
</div>
|
||||
|
||||
> Note: Full-Song Generation Feature, Audio Clip Generation Feature, and Stem Separation Feature requires [**K.G.One Music Studio**](https://github.com/KGAudioLab/K.G.One) integration.
|
||||
|
||||
## Latest Updates
|
||||
|
||||
- **2026.04.24**: Added [**K.G.One Music Studio**](https://github.com/KGAudioLab/K.G.One) integration! When K.G.Studio connects to a local K.G.One server, the **K.G.One Music Generator** panel (magic wand button ✦ in the toolbar) becomes available with three AI-powered tools: **Full Song Generation** (powered by ACE-Step 1.5 — generate full-length songs from text prompts), **Clip Generation** (powered by Foundation-1 — generate instrument clips and MIDI loops from text), and **Stem Separation** (powered by python-audio-separator — split any audio into vocals, instrumentals, and more). Generated audio and MIDI can be previewed instantly and dragged directly onto your tracks. K.G.One runs entirely on your own machine (Windows/Linux, CUDA GPU required); see the [K.G.One repository](https://github.com/KGAudioLab/K.G.One) for setup instructions.
|
||||
- **2026.04.11**: Migrated project storage from IndexedDB to OPFS (Origin Private File System) with a folder-based structure for better media file handling. Added audio track support with WAV/MP3 import, playback, looping, and non-destructive region trimming. Added bounce-to-WAV/MP3 export via offline rendering.
|
||||
- **2026.04.05**: Migrated the AI agent from XML-based tool calling to native OpenAI SDK function calling for improved reliability and compatibility. Added new LLM model options including GPT-5.4 series.
|
||||
- **2026.01.23**: Implemented seamless loop playback! Drag on the bar numbers to set loop range, or toggle loop mode with the Loop button in the toolbar. Loop playback uses `Tone.js`'s native looping for sample-accurate, gap-free looping.
|
||||
@@ -52,10 +66,12 @@ This project investigates how AI-human collaboration can enhance creative music-
|
||||
- [Click here to get a free OpenRouter API Key](https://openrouter.ai/keys) (you may need an OpenRouter account).
|
||||
- In **Settings ⚙️ → General → LLM Provider**, select **OpenAI Compatible**.
|
||||
- In **OpenAI Compatible Server → Key**, paste your key. (Note: on non‑localhost, your key isn't persisted by default for security; you can enable "Persist API Keys on Non-Localhost" in Settings to persist them, though this may increase XSS risk.)
|
||||
- In **OpenAI Compatible Server → Model**, enter `qwen/qwen3.6-plus:free`. (Note: this is a free model [qwen3.6-plus](https://openrouter.ai/qwen/qwen3.6-plus:free); non‑free models may require billing; free model providers may collect your data, check the model page for details; this project is not affiliated with OpenRouter or any model provider.)
|
||||
- In **OpenAI Compatible Server → Model**, enter `openai/gpt-oss-120b:free`. (Note: this is a free model; non‑free models may require billing; free model providers may collect your data, check the model page for details; this project is **not** affiliated with OpenRouter or any model provider.)
|
||||
- In **OpenAI Compatible Server → Base URL**, enter `https://openrouter.ai/api/v1`.
|
||||
|
||||
(Alternatively, you can use the official OpenAI API, other OpenAI‑compatible services, or your own hosted LLM server. e.g., Ollama, vLLM)
|
||||
**Tips:**
|
||||
- You can also use the official OpenAI API, other OpenAI-compatible services, or a self-hosted LLM server (e.g., Ollama, vLLM). Note that model quality varies — not all models perform equally well for music editing tasks. For local hosting, we recommend `qwen3.5-35b-a3b` as a good balance between generation quality and hardware requirements.
|
||||
- If you have an active subscription with OpenAI or another LLM provider, you can use [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) to run a local proxy server that routes requests through your existing subscription, without needing a separate API key.
|
||||
|
||||
### Basic DAW operations
|
||||
- Double‑click (or hold Ctrl/Cmd and click) on a track to create a region.
|
||||
@@ -220,7 +236,10 @@ OpenRouter is a platform that provides unified access to a wide range of languag
|
||||
5. Enter your chosen model name in **OpenAI Compatible Server → Model**. Recommended model series include:
|
||||
- `Anthropic: Claude Sonnet 4.6` (`anthropic/claude-sonnet-4.6`: [Link](https://openrouter.ai/anthropic/claude-sonnet-4.6)) — best balance of quality and cost for the Claude series
|
||||
- `Qwen: Qwen3.5-35B-A3B` (`qwen/qwen3.5-35b-a3b`: [Link](https://openrouter.ai/qwen/qwen3.5-35b-a3b)) — recommended open source model
|
||||
- `Qwen: Qwen3.6 Plus` (FREE MODEL: `qwen/qwen3.6-plus:free`: [Link](https://openrouter.ai/qwen/qwen3.6-plus:free)) — recommended free model; note that free model providers may collect your data, check the model page for details
|
||||
- `Qwen: Qwen3-Next-80B-A3B` (FREE MODEL: `qwen/qwen3-next-80b-a3b-instruct:free`: [Link](https://openrouter.ai/qwen/qwen3-next-80b-a3b-instruct:free)) — recommended free model
|
||||
- `OpenAI: GPT-OSS 120B` (FREE MODEL: `openai/gpt-oss-120b:free`: [Link](https://openrouter.ai/openai/gpt-oss-120b:free)) — recommended free model
|
||||
- Note: free model providers may collect your data; check the model page for details before use
|
||||
- Note: free model availability changes frequently — for the latest free options, visit the [OpenRouter Models Page](https://openrouter.ai/models) and use the **Prompt Pricing** filter to find currently free models
|
||||
6. Input the base URL `https://openrouter.ai/api/v1` **OpenAI Compatible Server → Base URL**.
|
||||
|
||||
### About the agent and LLM providers
|
||||
@@ -231,6 +250,53 @@ For security, when using K.G.Studio from a non‑local host, API keys are not pe
|
||||
|
||||
K.G.Studio does not provide or host any of the models listed above, nor is it affiliated with any model provider. All data is stored locally on your device; K.G.Studio does not collect or transmit your data. You are solely responsible for any data you provide to third‑party model providers.
|
||||
|
||||
## K.G.One Music Generator
|
||||
|
||||
> **Requires [K.G.One Music Studio](https://github.com/KGAudioLab/K.G.One) integration.** The K.G.One Music Generator panel is only available when K.G.Studio is connected to a running K.G.One server. See the [K.G.One repository](https://github.com/KGAudioLab/K.G.One) for setup instructions.
|
||||
|
||||
The **K.G.One Music Generator** panel provides three GPU-accelerated AI tools for music generation and audio processing. Click the **✦ (magic wand)** button in the toolbar to open it. The panel is mutually exclusive with the AI Assistant chatbox — opening one will close the other.
|
||||
|
||||
> **Note:** The first time you use each tool, the server needs to load the corresponding AI model, which can take 60 seconds or longer depending on your hardware. Switching between tabs may also trigger a model reload.
|
||||
|
||||
### Full Song Generation
|
||||
|
||||
Generate a complete, full-length song from a text description and optional lyrics. Powered by [ACE-Step 1.5](https://github.com/ace-step/ACE-Step-1.5).
|
||||
|
||||
- In the **Full Song** tab, enter a **Caption** describing the desired style, mood, tempo, instrumentation, and structure in natural language. For example: `Genre: Eurodance, 90s dance-pop, upbeat electronic. Tempo: ~130 BPM. Instrumentation: driving kick drum, eurodance bassline...`
|
||||
- Optionally, enter **Lyrics**. Use `[Intro]`, `[Verse]`, `[Chorus]`, `[Bridge]` tags to mark sections. Check **Instrumental** to skip vocals entirely.
|
||||
- Click **Generate Song**. A progress indicator shows the generation stage and percentage in real time.
|
||||
- Once complete, a preview player appears. Drag it onto an **audio track** to import the song as a region.
|
||||
- Dropping onto a MIDI track is not supported for full song generation.
|
||||
- **Advanced Settings** (expandable): Inference Steps, Guidance Scale, Seed, and Thinking (CoT metadata generation).
|
||||
|
||||
### Clip Generation
|
||||
|
||||
Generate short instrument clips and MIDI loops from text descriptions. Powered by [Foundation-1](https://huggingface.co/RoyalCities/Foundation-1).
|
||||
|
||||
- In the **Clip** tab, enter a **Prompt** describing the clip using comma-separated tags covering instrument family, sub-type, timbre, effects, length, BPM, and key. For example: `Gritty, Acid, Bassline, 303, Synth Lead, FM, Sub, High Reverb, 8 Bars, 140 BPM, E minor`
|
||||
- Optionally, enter a **Negative Prompt** to steer the generation away from unwanted characteristics (e.g. `distortion, noise`).
|
||||
- Select **Bars**: 4 or 8. The BPM and key signature are pre-filled from your project settings and can be adjusted in **Advanced Settings**.
|
||||
- Click **Generate Clip**. Once complete, a preview player appears with a drag handle on the left and a download button on the right.
|
||||
- **To import**: drag the player onto a track on the timeline.
|
||||
- Drop onto an **audio track** to import as a WAV audio region (recommended).
|
||||
- Drop onto a **MIDI track** to import as a MIDI region. Note that the MIDI is transcribed from the audio and may not be perfectly accurate.
|
||||
- **Advanced Settings** (expandable): Note, Scale, BPM, Steps, CFG Scale, Seed (`-1` for random), Sampler Type, Sigma Min/Max, and CFG Rescale.
|
||||
|
||||
### Stem Separation
|
||||
|
||||
Split an existing audio region into individual stems (e.g. vocals, instruments, drums). Powered by [python-audio-separator (UVR5)](https://github.com/nomadkaraoke/python-audio-separator).
|
||||
|
||||
- **Select an audio region** on the timeline before opening this tab. The selected region and its track name are shown at the top of the **Separator** tab. Only audio regions are supported — MIDI regions cannot be separated.
|
||||
- Choose a **Separation Model**:
|
||||
- **Vocal and Instrument (Medium Accuracy)** (`UVR-MDX-NET-Inst_HQ_3`) — fast two-stem split (vocal / instrumental).
|
||||
- **Vocal and Instrument (High Accuracy)** (`MDX23C-8KFFT-InstVoc_HQ`) — higher-quality two-stem split, slower.
|
||||
- **Vocal, Drums, Bass, Guitar, Piano, and Others** (`htdemucs_6s`) — full six-stem separation.
|
||||
- Click **Separate Stems**. If the selected region has a clip start offset or is trimmed, the audio is automatically sliced to match the region range before uploading.
|
||||
- Once complete, each stem appears as a labelled preview player with a drag handle. You can preview each stem individually before importing.
|
||||
- **To import stems**:
|
||||
- Drag each stem player individually onto an **audio track** to place it where you want.
|
||||
- Or click **Import All Stems to Timeline** to create one new audio track per stem automatically, positioned immediately below the source track and aligned to the same start beat as the original region. This is a single undoable operation.
|
||||
|
||||
## Upcoming Features
|
||||
|
||||
Feature priorities might change.
|
||||
@@ -240,7 +306,7 @@ Feature priorities might change.
|
||||
- [X] Intelligent Chord Assistant with functional harmony guidance (T/S/D)
|
||||
- [ ] Support track control automations (e.g. sustain, volume, pan, etc.)
|
||||
- [ ] Support MIDI control events (e.g. CC, pitch bend, etc.)
|
||||
- [ ] Support WAV audio tracks
|
||||
- [X] Support WAV audio tracks
|
||||
- [ ] Filters and effects
|
||||
- [ ] MCP Support
|
||||
- [X] Add support for OpenAI's open source models (`gpt-oss-20b` and `gpt-oss-120b`)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 8.9 MiB |
Reference in New Issue
Block a user