commit 349ad168d470fc9e059d4ccba8478010cdc6ef53 Author: Xiaohan-Tian Date: Tue Apr 14 19:19:45 2026 -0700 feat: initial commit diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..aec2466 --- /dev/null +++ b/.gitignore @@ -0,0 +1,42 @@ +# --- Cloned submodules (restored by init.bat) --- +ace-step/ +foundation1/ +separator/ + +# --- Python environments --- +.venv/ + +# --- Generated and uploaded files --- +outputs/ +uploads/ + +# --- Python cache --- +__pycache__/ +*.py[cod] +*.pyo +*.pyd + +# --- Distribution / packaging --- +*.egg-info/ +dist/ +build/ + +# --- Environment variables --- +.env +.env.* + +# --- Editor / OS --- +.vscode/ +.idea/ +*.DS_Store +Thumbs.db + +# --- Docs --- +docs/ + +# --- Claude --- +.claude/ +CLAUDE.md + +# --- Utility Scripts --- +_write_*.py diff --git a/README.md b/README.md new file mode 100644 index 0000000..ee1d2ae --- /dev/null +++ b/README.md @@ -0,0 +1,633 @@ +# K.G.One + +A unified REST API gateway that exposes two AI music-generation models behind a single consistent interface. + +| Service | Model | Output | Port | +|---------|-------|--------|------| +| **Full-song** | [ACE-Step 1.5](https://github.com/ace-step/ACE-Step-1.5) | Full-length music (MP3/WAV/FLAC) | 8001 (internal) | +| **Clip** | [Foundation-1](https://huggingface.co/RoyalCities/Foundation-1) | Short instrument clips — WAV **and** MIDI | 8002 (internal) | +| **Separator** | [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator) | Separated stems (Vocals, Instrumental, etc.) as MP3 | CLI (no port) | +| **Gateway** | K.G.One | Routes all requests, enforces GPU mutex | **8000** (public) | + +Because all services require a GPU, only one is active at a time. You explicitly switch via `POST /v1/models/load` before generating or separating. + +--- + +## Requirements + +| Requirement | Notes | +|-------------|-------| +| Windows 10/11 | `init.bat` is Windows-only; Linux/macOS support can be added | +| NVIDIA GPU | CUDA required for both models | +| [Git](https://git-scm.com/downloads) | For cloning sub-projects | +| [uv](https://docs.astral.sh/uv/getting-started/installation/) | Python environment manager | +| Python 3.10 | Required by Foundation-1; ACE-Step works with 3.10+ | + +--- + +## Setup + +### 1. Initialize + +Run `init.bat` from the project root. It will: + +1. Read pinned commit hashes from `submodules.json` +2. Clone ACE-Step 1.5 into `ace-step/` +3. Clone Foundation-1 into `foundation1/` +4. Create three isolated Python environments: + - `.venv` — the gateway (fastapi, httpx) + - `ace-step/.venv` — ACE-Step and its CUDA dependencies + - `foundation1/.venv` — Foundation-1 and its dependencies (scipy==1.8.1) +5. Create output directories under `outputs/` + +```bat +init.bat +``` + +> **Note:** ACE-Step downloads large CUDA packages. Expect 10–20 minutes on the first run. + +### 2. Download model weights + +**ACE-Step** downloads weights automatically on first start via its built-in model downloader. + +**Foundation-1** downloads from HuggingFace on first start (handled by `get_pretrained_model`). Alternatively, set environment variables to point to a local checkpoint: + +```bat +set FOUNDATION1_CKPT_PATH=C:\path\to\foundation1.safetensors +set FOUNDATION1_CONFIG_PATH=C:\path\to\model_config.json +``` + +### 3. Start the gateway + +```bat +.venv\Scripts\python.exe main.py +``` + +The gateway starts on `http://localhost:8000`. Interactive API docs are available at `http://localhost:8000/docs`. + +### 4. Load a model and generate + +```bash +# Load Foundation-1 +curl -X POST http://localhost:8000/v1/models/load \ + -H "Content-Type: application/json" \ + -d '{"model": "clip"}' + +# Submit a generation +curl -X POST http://localhost:8000/v1/clip/generate \ + -H "Content-Type: application/json" \ + -d '{"prompt": "Piano, Rhodes, Warm, 8 Bars, 120 BPM, C major", "bars": 8, "bpm": 120}' +``` + +### Upgrading a pinned dependency + +Edit the `commit` field in `submodules.json`, delete the corresponding subfolder, then re-run `init.bat`. + +--- + +## Project structure + +``` +K.G.One/ +├── submodules.json # Pinned commits — source of truth for dependency versions +├── init.bat # Windows bootstrap script +├── pyproject.toml # Gateway Python project +├── main.py # Gateway FastAPI application (port 8000) +├── services/ +│ ├── model_manager.py # GPU mutex — starts/stops sub-service subprocesses +│ ├── acestep_client.py # ACE-Step connection config +│ ├── foundation1_client.py# Foundation-1 connection config +│ └── separator_runner.py # Runs audio-separator CLI per-request, manages tasks +├── foundation1_server/ +│ └── server.py # Foundation-1 FastAPI wrapper (port 8002) +├── ace-step/ # Cloned by init.bat — ACE-Step 1.5 source +├── foundation1/ # Cloned by init.bat — RC-stable-audio-tools source +├── separator/ # Cloned by init.bat — python-audio-separator source + venv +├── outputs/ +│ ├── clip/ # Foundation-1 generated WAV + MIDI files +│ ├── fullsong/ # (reserved for ACE-Step output references) +│ └── separator/ # Separated stem MP3 files +└── uploads/ + └── separator/ # Temporary upload storage (auto-deleted after processing) +``` + +--- + +## API Reference + +### System + +| Method | URL | Description | +|--------|-----|-------------| +| `GET` | `/health` | Gateway health check | +| `POST` | `/v1/models/load` | Load a model onto the GPU (unloads the active one first) | +| `GET` | `/v1/models/status` | Return the currently active model | + +--- + +#### `GET /health` + +**Response** +```json +{ + "status": "ok", + "active_model": "clip" +} +``` + +`active_model` is `null` when no model is loaded. + +--- + +#### `POST /v1/models/load` + +Loads a model onto the GPU. If a different model is currently active, it is shut down first. + +For `"fullsong"` and `"clip"` this call **blocks** until the sub-service reports healthy (model weights loaded). Expect 30–120 seconds on first run. + +For `"separator"` it only terminates the currently running model to free VRAM — no persistent process is started. Returns immediately. + +**Request** + +| Field | Type | Required | Values | +|-------|------|----------|--------| +| `model` | string | yes | `"clip"`, `"fullsong"`, or `"separator"` | + +```json +{ "model": "separator" } +``` + +**Response** +```json +{ + "active_model": "separator", + "status": "ready" +} +``` + +**Error — unknown model (400)** +```json +{ "detail": "Unknown model 'foo'. Must be 'fullsong', 'clip', or 'separator'." } +``` + +--- + +#### `GET /v1/models/status` + +**Response** +```json +{ + "active_model": "fullsong" +} +``` + +--- + +### Full-song generation (ACE-Step 1.5) + +> All `/v1/fullsong/*` endpoints return HTTP 503 if `fullsong` is not the active model. + +| Method | URL | Description | +|--------|-----|-------------| +| `POST` | `/v1/fullsong/generate` | Submit a full-song generation task | +| `GET` | `/v1/fullsong/result/{task_id}` | Poll task status and retrieve result | +| `GET` | `/v1/fullsong/audio?path={path}` | Download a generated audio file | + +--- + +#### `POST /v1/fullsong/generate` + +Proxied to ACE-Step's `/release_task`. Accepts the full ACE-Step generation parameter set. + +**Request** (key fields — see [ACE-Step API docs](https://github.com/ace-step/ACE-Step-1.5/blob/main/docs/en/API.md) for the complete spec) + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `caption` | string | required | Musical style/description | +| `lyrics` | string | `""` | Song lyrics (`[verse]`, `[chorus]` tags supported) | +| `duration` | number | `60` | Duration in seconds | +| `instrumental` | boolean | `false` | Generate without vocals | +| `bpm` | number | `null` | Target BPM (null = auto) | +| `inference_steps` | integer | `8` | Diffusion steps (8 = turbo, 50 = full quality) | +| `guidance_scale` | number | `7.0` | Classifier-free guidance scale | +| `seed` | integer | `-1` | `-1` for random | +| `audio_format` | string | `"mp3"` | `"mp3"`, `"wav"`, `"flac"`, `"opus"` | + +```json +{ + "caption": "upbeat electronic dance, synthesizer, four-on-the-floor kick, 128 BPM", + "lyrics": "[verse]\nLights are flashing\nBeats are crashing\n[chorus]\nDance all night", + "duration": 90, + "instrumental": false, + "inference_steps": 8, + "guidance_scale": 7.0, + "seed": -1, + "audio_format": "mp3" +} +``` + +**Response** +```json +{ + "data": { + "task_id": "a3f2c1d8-9e4b-4a7f-b012-3c5d6e7f8a9b", + "status": "queued", + "queue_position": 1 + }, + "code": 200, + "error": null, + "timestamp": 1744300000000 +} +``` + +--- + +#### `GET /v1/fullsong/result/{task_id}` + +Poll until `status` is `"finished"`. Recommended interval: 2–5 seconds. + +**Path parameter:** `task_id` from the generate response. + +**Response — pending** +```json +{ + "data": [ + { + "task_id": "a3f2c1d8-9e4b-4a7f-b012-3c5d6e7f8a9b", + "status": "running", + "progress": 0.4 + } + ], + "code": 200 +} +``` + +**Response — finished** +```json +{ + "data": [ + { + "task_id": "a3f2c1d8-9e4b-4a7f-b012-3c5d6e7f8a9b", + "status": "finished", + "audio_path": "/tmp/acestep/outputs/a3f2c1d8.mp3", + "duration": 90.2, + "bpm": 128 + } + ], + "code": 200 +} +``` + +Use `audio_path` as the `path` query parameter when calling `/v1/fullsong/audio`. + +--- + +#### `GET /v1/fullsong/audio?path={path}` + +Download the generated audio file. Returns binary audio data. + +**Query parameter:** `path` — the `audio_path` value from the result response. + +**Response:** Binary audio file (`audio/mpeg`, `audio/wav`, etc. depending on format). + +--- + +### Clip generation (Foundation-1) + +> All `/v1/clip/*` endpoints return HTTP 503 if `clip` is not the active model. + +Foundation-1 generates short instrument clips (4 or 8 bars) from a structured text prompt, producing both a WAV audio file and a MIDI transcription simultaneously. + +| Method | URL | Description | +|--------|-----|-------------| +| `POST` | `/v1/clip/generate` | Submit a clip generation task | +| `GET` | `/v1/clip/result/{task_id}` | Poll task status and retrieve file URLs | +| `GET` | `/v1/clip/audio/{filename}` | Download the generated WAV file | +| `GET` | `/v1/clip/midi/{filename}` | Download the generated MIDI file | + +--- + +#### `POST /v1/clip/generate` + +**Request** + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `prompt` | string | required | Comma-separated descriptor tags (see prompt guide below) | +| `negative_prompt` | string | `""` | Tags to avoid | +| `bars` | integer | `4` | Clip length: `4` or `8` | +| `bpm` | integer | `140` | Tempo in BPM (e.g. 100, 110, 120, 128, 130, 140, 150) | +| `note` | string | `"C"` | Root note: `A` through `G#` | +| `scale` | string | `"minor"` | `"major"` or `"minor"` | +| `steps` | integer | `75` | Diffusion steps (1–500; 75 is a good balance) | +| `cfg_scale` | number | `7.0` | Classifier-free guidance (0–25) | +| `seed` | integer | `-1` | `-1` for random | +| `sampler_type` | string | `"dpmpp-2m-sde"` | Sampler algorithm | +| `sigma_min` | number | `0.03` | Minimum noise sigma | +| `sigma_max` | number | `500.0` | Maximum noise sigma | +| `cfg_rescale` | number | `0.0` | CFG rescale factor (0–1) | + +**Prompt format** + +Foundation-1 prompts are structured tag lists. Key components: + +``` +[Instrument family], [Sub-type], [Timbre descriptors], [FX], [Bars], [BPM], [Key] +``` + +Example prompts: +- `"Piano, Rhodes Piano, Warm, Bright, Lush, 8 Bars, 120 BPM, C major"` +- `"Bass, FM Bass, Acid, Gritty, Thick, 8 Bars, 140 BPM, E minor"` +- `"Synth, Wavetable Synth, Pad, Wide, Silky, 4 Bars, 128 BPM, A minor"` + +```json +{ + "prompt": "Bass, FM Bass, Acid, Gritty, Wide, Thick, 8 Bars, 140 BPM, E minor", + "bars": 8, + "bpm": 140, + "note": "E", + "scale": "minor", + "steps": 75, + "cfg_scale": 7.0, + "seed": -1, + "sampler_type": "dpmpp-2m-sde" +} +``` + +**Response** +```json +{ + "task_id": "b7e3a921-4f1c-4d8e-a023-9d6c5e8f1b2a" +} +``` + +--- + +#### `GET /v1/clip/result/{task_id}` + +Poll until `status` is `"complete"`. Recommended interval: 2–5 seconds. + +**Path parameter:** `task_id` from the generate response. + +**Response — pending / running** +```json +{ + "task_id": "b7e3a921-4f1c-4d8e-a023-9d6c5e8f1b2a", + "status": "running", + "error": null +} +``` + +**Response — complete** +```json +{ + "task_id": "b7e3a921-4f1c-4d8e-a023-9d6c5e8f1b2a", + "status": "complete", + "wav_url": "/v1/clip/audio/Bass_FM_Bass_Acid_140BPM_E_minor_42.wav", + "midi_url": "/v1/clip/midi/Bass_FM_Bass_Acid_140BPM_E_minor_42.mid" +} +``` + +**Response — error** +```json +{ + "task_id": "b7e3a921-4f1c-4d8e-a023-9d6c5e8f1b2a", + "status": "error", + "error": "Generation failed — check server logs." +} +``` + +--- + +#### `GET /v1/clip/audio/{filename}` + +Download the generated WAV file (32 kHz stereo). + +**Path parameter:** `filename` — the filename portion of `wav_url` from the result response. + +**Response:** Binary WAV file (`audio/wav`). + +```bash +curl http://localhost:8000/v1/clip/audio/Bass_FM_Bass_Acid_140BPM_E_minor_42.wav \ + --output clip.wav +``` + +--- + +#### `GET /v1/clip/midi/{filename}` + +Download the MIDI transcription derived from the generated audio (via [basic-pitch](https://github.com/spotify/basic-pitch)). + +**Path parameter:** `filename` — the filename portion of `midi_url` from the result response. + +**Response:** Binary MIDI file (`audio/midi`). + +```bash +curl http://localhost:8000/v1/clip/midi/Bass_FM_Bass_Acid_140BPM_E_minor_42.mid \ + --output clip.mid +``` + +--- + +### Stem separation (python-audio-separator) + +> All `/v1/separator/*` endpoints return HTTP 503 if `separator` is not the active model. + +Separates an uploaded audio file into individual stems (vocals, instrumental, etc.) using UVR models. Outputs are always MP3. + +| Method | URL | Description | +|--------|-----|-------------| +| `POST` | `/v1/separator/separate` | Upload audio + select model → task ID | +| `GET` | `/v1/separator/result/{task_id}` | Poll task status and retrieve output filenames | +| `GET` | `/v1/separator/download/{filename}` | Download a separated stem file | + +--- + +#### `POST /v1/separator/separate` + +Accepts a `multipart/form-data` body. + +**Form fields** + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `file` | file | yes | Audio file to separate (MP3, WAV, FLAC, …) | +| `model_filename` | string | yes | One of the three supported models (see below) | + +**Supported models** + +| `model_filename` | Stems produced | +|-----------------|----------------| +| `UVR-MDX-NET-Inst_HQ_3.onnx` | 2 — Vocals, Instrumental | +| `MDX23C-8KFFT-InstVoc_HQ.ckpt` | 2 — Vocals, Instrumental | +| `htdemucs_6s.yaml` | 6 — bass, drums, guitar, other, piano, vocals | + +Models are downloaded automatically by `audio-separator` on first use. + +```bash +curl -X POST http://localhost:8000/v1/separator/separate \ + -F "file=@song.mp3" \ + -F "model_filename=UVR-MDX-NET-Inst_HQ_3.onnx" +``` + +**Response** +```json +{ "task_id": "c4e2f891-3b1a-4d7e-b023-8e5f6a9c2d1b" } +``` + +--- + +#### `GET /v1/separator/result/{task_id}` + +Poll until `status` is `"complete"`. Recommended interval: 2–5 seconds. Separation typically takes 10–60 seconds depending on file length and model. + +**Response — running** +```json +{ + "task_id": "c4e2f891-3b1a-4d7e-b023-8e5f6a9c2d1b", + "status": "running" +} +``` + +**Response — complete** +```json +{ + "task_id": "c4e2f891-3b1a-4d7e-b023-8e5f6a9c2d1b", + "status": "complete", + "files": [ + "c4e2f891_(Instrumental)_UVR-MDX-NET-Inst_HQ_3.mp3", + "c4e2f891_(Vocals)_UVR-MDX-NET-Inst_HQ_3.mp3" + ] +} +``` + +**Response — error** +```json +{ + "task_id": "c4e2f891-3b1a-4d7e-b023-8e5f6a9c2d1b", + "status": "error", + "error": "Separation failed — check server logs." +} +``` + +--- + +#### `GET /v1/separator/download/{filename}` + +Download a stem MP3 file. `filename` is one of the entries from the `files` list in the result response. + +**Response:** Binary MP3 file (`audio/mpeg`) with `Content-Disposition: attachment`. + +```bash +curl "http://localhost:8000/v1/separator/download/c4e2f891_(Vocals)_UVR-MDX-NET-Inst_HQ_3.mp3" \ + --output vocals.mp3 +``` + +--- + +## Typical workflows + +### Generate a full song + +```bash +# 1. Load ACE-Step +curl -X POST http://localhost:8000/v1/models/load \ + -H "Content-Type: application/json" \ + -d '{"model": "fullsong"}' + +# 2. Submit generation +TASK=$(curl -s -X POST http://localhost:8000/v1/fullsong/generate \ + -H "Content-Type: application/json" \ + -d '{ + "caption": "lo-fi hip hop, mellow piano, soft drums, vinyl crackle", + "duration": 120, + "instrumental": true, + "inference_steps": 8, + "audio_format": "mp3" + }' | python -c "import sys,json; print(json.load(sys.stdin)['data']['task_id'])") + +# 3. Poll until finished +curl http://localhost:8000/v1/fullsong/result/$TASK + +# 4. Download (using audio_path from step 3 result) +curl "http://localhost:8000/v1/fullsong/audio?path=/tmp/acestep/outputs/$TASK.mp3" \ + --output song.mp3 +``` + +### Generate a MIDI + WAV clip + +```bash +# 1. Load Foundation-1 +curl -X POST http://localhost:8000/v1/models/load \ + -H "Content-Type: application/json" \ + -d '{"model": "clip"}' + +# 2. Submit generation +curl -s -X POST http://localhost:8000/v1/clip/generate \ + -H "Content-Type: application/json" \ + -d '{ + "prompt": "Keys, Rhodes Piano, Warm, Lush, 8 Bars, 90 BPM, D major", + "bars": 8, "bpm": 90, "note": "D", "scale": "major", "steps": 75 + }' +# => {"task_id": "b7e3a921-..."} + +# 3. Poll +curl http://localhost:8000/v1/clip/result/b7e3a921-... +# => {"status": "complete", "wav_url": "/v1/clip/audio/Keys_Rhodes_...", "midi_url": "..."} + +# 4. Download both files +curl http://localhost:8000/v1/clip/audio/Keys_Rhodes_Piano_Warm_Lush_42.wav --output clip.wav +curl http://localhost:8000/v1/clip/midi/Keys_Rhodes_Piano_Warm_Lush_42.mid --output clip.mid +``` + +### Separate stems from an audio file + +```bash +# 1. Load separator (terminates any active model, frees VRAM) +curl -X POST http://localhost:8000/v1/models/load \ + -H "Content-Type: application/json" \ + -d '{"model": "separator"}' + +# 2. Upload file and submit separation +TASK=$(curl -s -X POST http://localhost:8000/v1/separator/separate \ + -F "file=@song.mp3" \ + -F "model_filename=UVR-MDX-NET-Inst_HQ_3.onnx" | python -c "import sys,json; print(json.load(sys.stdin)['task_id'])") + +# 3. Poll until complete +curl http://localhost:8000/v1/separator/result/$TASK +# => {"status": "complete", "files": ["...(Vocals)...", "...(Instrumental)..."]} + +# 4. Download stems +curl "http://localhost:8000/v1/separator/download/...(Vocals)....mp3" --output vocals.mp3 +curl "http://localhost:8000/v1/separator/download/...(Instrumental)....mp3" --output instrumental.mp3 +``` + +### Switch between models + +```bash +# Foundation-1 is active — switch to ACE-Step +curl -X POST http://localhost:8000/v1/models/load \ + -H "Content-Type: application/json" \ + -d '{"model": "fullsong"}' +# Foundation-1 subprocess is terminated, ACE-Step starts. Blocks until healthy. +``` + +--- + +## Error reference + +| HTTP Status | Meaning | +|-------------|---------| +| `400` | Bad request (e.g. unknown model name) | +| `404` | Task ID or file not found | +| `503` | Requested model is not currently loaded, or sub-service is unreachable | + +**503 body when wrong model is active:** +```json +{ + "detail": { + "error": "Model 'clip' is not loaded. POST /v1/models/load first.", + "active_model": "fullsong" + } +} +``` diff --git a/cleanup.bat b/cleanup.bat new file mode 100644 index 0000000..f4eae46 --- /dev/null +++ b/cleanup.bat @@ -0,0 +1,79 @@ +@echo off +setlocal + +echo ========================================== +echo KGOne Cleanup Script (Windows) +echo ========================================== +echo. +echo This will permanently delete: +echo .venv\ (gateway Python environment) +echo ace-step\ (submodule + venv + model checkpoints) +echo foundation1\ (submodule + venv + model weights) +echo separator\ (submodule + venv) +echo outputs\ (all generated audio and MIDI files) +echo uploads\ (all uploaded audio files) +echo. +echo Re-run init.bat to restore everything except outputs and uploads. +echo. + +set /p CONFIRM=Type YES to confirm: +if /i not "%CONFIRM%"=="YES" ( + echo Cancelled. + exit /b 0 +) +echo. + +if exist ".venv" ( + echo Removing .venv ... + rmdir /s /q ".venv" + echo Done. +) else ( + echo .venv not found, skipping. +) + +if exist "ace-step" ( + echo Removing ace-step ... + rmdir /s /q "ace-step" + echo Done. +) else ( + echo ace-step not found, skipping. +) + +if exist "foundation1" ( + echo Removing foundation1 ... + rmdir /s /q "foundation1" + echo Done. +) else ( + echo foundation1 not found, skipping. +) + +if exist "separator" ( + echo Removing separator ... + rmdir /s /q "separator" + echo Done. +) else ( + echo separator not found, skipping. +) + +if exist "outputs" ( + echo Removing outputs ... + rmdir /s /q "outputs" + echo Done. +) else ( + echo outputs not found, skipping. +) + +if exist "uploads" ( + echo Removing uploads ... + rmdir /s /q "uploads" + echo Done. +) else ( + echo uploads not found, skipping. +) + +echo. +echo ========================================== +echo Cleanup complete. +echo Run init.bat to set up again. +echo ========================================== +echo. diff --git a/foundation1_server/server.py b/foundation1_server/server.py new file mode 100644 index 0000000..83296ad --- /dev/null +++ b/foundation1_server/server.py @@ -0,0 +1,244 @@ +"""Foundation-1 REST API wrapper server. + +Wraps RC-stable-audio-tools' generate_cond() behind FastAPI endpoints. +Runs on port 8002. Managed as a subprocess by the KGOne gateway. + +Environment variables: + FOUNDATION1_PRETRAINED_NAME HuggingFace model name (default: RoyalCities/Foundation-1) + FOUNDATION1_CKPT_PATH Path to local .safetensors checkpoint (overrides pretrained name) + FOUNDATION1_CONFIG_PATH Path to local model_config.json (required if CKPT_PATH is set) + FOUNDATION1_SERVER_PORT Port to listen on (default: 8002) +""" + +from __future__ import annotations + +import asyncio +import json +import logging +import os +import sys +import time +import uuid +from concurrent.futures import ThreadPoolExecutor +from contextlib import asynccontextmanager +from pathlib import Path +from typing import Optional + +import uvicorn +from fastapi import FastAPI, HTTPException +from fastapi.responses import FileResponse +from pydantic import BaseModel + +# ── path setup ──────────────────────��──────────────────────────────��────────── +# Must happen BEFORE importing stable_audio_tools — gradio.py opens config.json +# relative to CWD at import time. + +ROOT_DIR = Path(__file__).parent.parent.resolve() +FOUNDATION1_DIR = ROOT_DIR / "foundation1" +OUTPUT_DIR = ROOT_DIR / "outputs" / "clip" + +if not FOUNDATION1_DIR.exists(): + raise RuntimeError( + f"Foundation-1 repo not found at {FOUNDATION1_DIR}. " + "Run init.bat first to clone the submodules." + ) + +os.chdir(FOUNDATION1_DIR) +sys.path.insert(0, str(FOUNDATION1_DIR)) +OUTPUT_DIR.mkdir(parents=True, exist_ok=True) + +# ── imports from stable_audio_tools (CWD is now FOUNDATION1_DIR) ───────────── +import stable_audio_tools.interface.gradio as _gradio_module # noqa: E402 +from stable_audio_tools.interface.gradio import generate_cond, load_model # noqa: E402 + +# Override the output directory from the one baked into config.json +_gradio_module.output_directory = str(OUTPUT_DIR) + +# ── logging ────────────────────────────────────────────────────────────────���── +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s %(levelname)s %(name)s: %(message)s", +) +logger = logging.getLogger(__name__) + +# ── configuration ───────────────────────────────────────────────────────────── +PRETRAINED_NAME = os.environ.get("FOUNDATION1_PRETRAINED_NAME", "RoyalCities/Foundation-1") +CKPT_PATH = os.environ.get("FOUNDATION1_CKPT_PATH") +CONFIG_PATH = os.environ.get("FOUNDATION1_CONFIG_PATH") +PORT = int(os.environ.get("FOUNDATION1_SERVER_PORT", "8002")) + +# Default local paths written by init.bat's snapshot_download step. +# The HuggingFace repo names the file Foundation_1.safetensors (not model.safetensors), +# so get_pretrained_model() would fail — we bypass it by using the local path directly. +_DEFAULT_LOCAL_DIR = ROOT_DIR / "foundation1" / "models" / "RoyalCities-Foundation-1" +_DEFAULT_CKPT = _DEFAULT_LOCAL_DIR / "Foundation_1.safetensors" +_DEFAULT_CONFIG = _DEFAULT_LOCAL_DIR / "model_config.json" + +# ── in-memory task store ────────────────────────────────────────────────────── +_tasks: dict[str, dict] = {} +_executor = ThreadPoolExecutor(max_workers=1) # one generation at a time (single GPU) + + +# ── app lifecycle ───────────────────────────────────────────────────────────── + +@asynccontextmanager +async def lifespan(app: FastAPI): + logger.info("Loading Foundation-1 model...") + _load_foundation1_model() + logger.info("Foundation-1 model ready.") + yield + logger.info("Shutting down Foundation-1 server.") + _gradio_module.model = None + + +def _load_foundation1_model() -> None: + import torch + + device = "cuda" if torch.cuda.is_available() else "cpu" + logger.info("Using device: %s", device) + + # Resolve checkpoint: env var override > local download > pretrained_name fallback + ckpt = Path(CKPT_PATH) if CKPT_PATH else (_DEFAULT_CKPT if _DEFAULT_CKPT.exists() else None) + config = Path(CONFIG_PATH) if CONFIG_PATH else (_DEFAULT_CONFIG if _DEFAULT_CONFIG.exists() else None) + + if ckpt and config: + with open(config) as f: + model_config = json.load(f) + load_model(model_config=model_config, model_ckpt_path=str(ckpt), device=device) + logger.info("Loaded local checkpoint: %s", ckpt) + else: + # Last resort: let stable_audio_tools attempt its own download. + # Note: get_pretrained_model() expects model.safetensors; this will fail unless + # the upstream repo is updated to match that name. + logger.warning( + "Local weights not found at %s — falling back to pretrained_name download. " + "Run init.bat to pre-download weights.", + _DEFAULT_CKPT, + ) + load_model(pretrained_name=PRETRAINED_NAME, device=device) + logger.info("Loaded pretrained model: %s", PRETRAINED_NAME) + + +app = FastAPI( + title="Foundation-1 Server", + description="Wraps Foundation-1 (RC-stable-audio-tools) for REST API use.", + version="0.1.0", + lifespan=lifespan, +) + + +# ── request model ───────────────────────────────────────────────────────────── + +class GenerateRequest(BaseModel): + prompt: str + negative_prompt: str = "" + bars: int = 4 + bpm: int = 140 + note: str = "C" + scale: str = "minor" + steps: int = 75 + cfg_scale: float = 7.0 + seed: int = -1 + sampler_type: str = "dpmpp-2m-sde" + sigma_min: float = 0.03 + sigma_max: float = 500.0 + cfg_rescale: float = 0.0 + + +# ── routes ──────────────────────────────────────────────────────────────────── + +@app.get("/health") +async def health(): + return {"status": "ok", "model_loaded": _gradio_module.model is not None} + + +@app.post("/generate") +async def generate(req: GenerateRequest): + if _gradio_module.model is None: + raise HTTPException(503, "Model not loaded") + + task_id = str(uuid.uuid4()) + _tasks[task_id] = {"status": "pending", "created_at": time.time()} + + loop = asyncio.get_event_loop() + loop.run_in_executor(_executor, _run_generate, task_id, req) + + return {"task_id": task_id} + + +def _run_generate(task_id: str, req: GenerateRequest) -> None: + _tasks[task_id]["status"] = "running" + try: + wav_path, _spectrograms, _piano_roll, midi_path = generate_cond( + prompt=req.prompt, + negative_prompt=req.negative_prompt or None, + bars=req.bars, + bpm=req.bpm, + note=req.note, + scale=req.scale, + steps=req.steps, + cfg_scale=req.cfg_scale, + seed=req.seed, + sampler_type=req.sampler_type, + sigma_min=req.sigma_min, + sigma_max=req.sigma_max, + cfg_rescale=req.cfg_rescale, + ) + _tasks[task_id].update( + { + "status": "complete", + "wav_filename": Path(wav_path).name if wav_path else None, + "midi_filename": Path(midi_path).name if midi_path else None, + } + ) + except Exception: + logger.exception("Generation failed for task %s", task_id) + _tasks[task_id]["status"] = "error" + _tasks[task_id]["error"] = "Generation failed — check server logs." + + +@app.get("/result/{task_id}") +async def get_result(task_id: str): + task = _tasks.get(task_id) + if task is None: + raise HTTPException(404, "Task not found") + + status = task["status"] + if status != "complete": + return {"task_id": task_id, "status": status, "error": task.get("error")} + + return {"task_id": task_id, "status": "complete"} + + +@app.get("/audio/{task_id}") +async def serve_audio(task_id: str): + task = _tasks.get(task_id) + if task is None: + raise HTTPException(404, "Task not found") + filename = task.get("wav_filename") + if not filename: + raise HTTPException(404, "No audio file for this task") + path = OUTPUT_DIR / filename + if not path.is_file(): + raise HTTPException(404, "Audio file not found on disk") + return FileResponse(str(path), media_type="audio/wav") + + +@app.get("/midi/{task_id}") +async def serve_midi(task_id: str): + task = _tasks.get(task_id) + if task is None: + raise HTTPException(404, "Task not found") + filename = task.get("midi_filename") + if not filename: + raise HTTPException(404, "No MIDI file for this task") + path = OUTPUT_DIR / filename + if not path.is_file(): + raise HTTPException(404, "MIDI file not found on disk") + return FileResponse(str(path), media_type="audio/midi") + + +# ── entry point ────────────────────────────��────────────────────────────────���─ + +if __name__ == "__main__": + uvicorn.run(app, host="0.0.0.0", port=PORT, log_level="info") diff --git a/init.bat b/init.bat new file mode 100644 index 0000000..23bb0c2 --- /dev/null +++ b/init.bat @@ -0,0 +1,339 @@ +@echo off +setlocal enabledelayedexpansion + +echo ========================================== +echo KGOne Initialization Script (Windows) +echo ========================================== +echo. + +set "ROOT=%~dp0" +cd /d "%ROOT%" + +echo Checking prerequisites... + +where git >nul 2>&1 +if errorlevel 1 ( + echo ERROR: git not found. Install from https://git-scm.com/downloads + exit /b 1 +) + +where uv >nul 2>&1 +if errorlevel 1 ( + echo ERROR: uv not found. Install from https://docs.astral.sh/uv/ + exit /b 1 +) + +echo git ... OK +echo uv ... OK +echo. + +:: --------------------------------------------------------- +echo [1/6] Reading submodules.json... +:: --------------------------------------------------------- + +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.'ace-step'.url | Out-File -Encoding ASCII '%TEMP%\kg_acestep_url.txt' -NoNewline" +if errorlevel 1 ( + echo ERROR: Failed to read submodules.json + exit /b 1 +) +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.'ace-step'.commit | Out-File -Encoding ASCII '%TEMP%\kg_acestep_commit.txt' -NoNewline" +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.foundation1.url | Out-File -Encoding ASCII '%TEMP%\kg_f1_url.txt' -NoNewline" +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.foundation1.commit | Out-File -Encoding ASCII '%TEMP%\kg_f1_commit.txt' -NoNewline" +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.'python-audio-separator'.url | Out-File -Encoding ASCII '%TEMP%\kg_sep_url.txt' -NoNewline" +powershell -NoProfile -Command "$j = Get-Content 'submodules.json' | ConvertFrom-Json; $j.'python-audio-separator'.commit | Out-File -Encoding ASCII '%TEMP%\kg_sep_commit.txt' -NoNewline" + +set /p ACESTEP_URL=<"%TEMP%\kg_acestep_url.txt" +set /p ACESTEP_COMMIT=<"%TEMP%\kg_acestep_commit.txt" +set /p FOUNDATION1_URL=<"%TEMP%\kg_f1_url.txt" +set /p FOUNDATION1_COMMIT=<"%TEMP%\kg_f1_commit.txt" +set /p SEP_URL=<"%TEMP%\kg_sep_url.txt" +set /p SEP_COMMIT=<"%TEMP%\kg_sep_commit.txt" +del "%TEMP%\kg_acestep_url.txt" "%TEMP%\kg_acestep_commit.txt" "%TEMP%\kg_f1_url.txt" "%TEMP%\kg_f1_commit.txt" "%TEMP%\kg_sep_url.txt" "%TEMP%\kg_sep_commit.txt" >nul 2>&1 + +if "%ACESTEP_URL%"=="" ( + echo ERROR: Could not read from submodules.json + exit /b 1 +) + +echo ACE-Step URL : %ACESTEP_URL% +echo ACE-Step commit: %ACESTEP_COMMIT% +echo Foundation-1 URL : %FOUNDATION1_URL% +echo Foundation-1 commit: %FOUNDATION1_COMMIT% +echo audio-separator URL : %SEP_URL% +echo audio-separator commit: %SEP_COMMIT% +echo. + +:: --------------------------------------------------------- +echo [2/6] Setting up ACE-Step... +:: --------------------------------------------------------- + +if exist "ace-step\.git" goto :acestep_fetch +echo Cloning ACE-Step (may take several minutes)... +git clone "%ACESTEP_URL%" ace-step +if errorlevel 1 ( + echo ERROR: Failed to clone ACE-Step + exit /b 1 +) +goto :acestep_checkout + +:acestep_fetch +echo Repository exists. Fetching refs... +git -C ace-step fetch --quiet origin + +:acestep_checkout +echo Checking out pinned commit %ACESTEP_COMMIT%... +git -C ace-step checkout %ACESTEP_COMMIT% --quiet +if errorlevel 1 ( + echo ERROR: Could not checkout ACE-Step commit %ACESTEP_COMMIT% + exit /b 1 +) +echo ACE-Step OK. +echo. + +:: --------------------------------------------------------- +echo [3/6] Setting up Foundation-1... +:: --------------------------------------------------------- + +if exist "foundation1\.git" goto :f1_fetch +echo Cloning Foundation-1... +git clone "%FOUNDATION1_URL%" foundation1 +if errorlevel 1 ( + echo ERROR: Failed to clone Foundation-1 + exit /b 1 +) +goto :f1_checkout + +:f1_fetch +echo Repository exists. Fetching refs... +git -C foundation1 fetch --quiet origin + +:f1_checkout +echo Checking out pinned commit %FOUNDATION1_COMMIT%... +git -C foundation1 checkout %FOUNDATION1_COMMIT% --quiet +if errorlevel 1 ( + echo ERROR: Could not checkout Foundation-1 commit %FOUNDATION1_COMMIT% + exit /b 1 +) +echo Foundation-1 OK. +echo. + +:: --------------------------------------------------------- +echo Setting up python-audio-separator... +:: --------------------------------------------------------- + +if exist "separator\.git" goto :sep_fetch +echo Cloning python-audio-separator... +git clone "%SEP_URL%" separator +if errorlevel 1 ( + echo ERROR: Failed to clone python-audio-separator + exit /b 1 +) +goto :sep_checkout + +:sep_fetch +echo Repository exists. Fetching refs... +git -C separator fetch --quiet origin + +:sep_checkout +echo Checking out pinned commit %SEP_COMMIT%... +git -C separator checkout %SEP_COMMIT% --quiet +if errorlevel 1 ( + echo ERROR: Could not checkout python-audio-separator commit %SEP_COMMIT% + exit /b 1 +) +echo python-audio-separator OK. +echo. + +:: --------------------------------------------------------- +echo [4/6] Setting up Python environments... +:: --------------------------------------------------------- + +:: Gateway venv +if exist ".venv\Scripts\python.exe" goto :gateway_venv_done +echo Creating gateway venv... +uv venv .venv +if errorlevel 1 ( + echo ERROR: Failed to create gateway venv + exit /b 1 +) +uv pip install --python .venv "fastapi>=0.110.0" "uvicorn[standard]>=0.27.0" "httpx>=0.27.0" "python-multipart>=0.0.9" +if errorlevel 1 ( + echo ERROR: Failed to install gateway dependencies + exit /b 1 +) +echo Gateway venv OK. +goto :acestep_venv_start +:gateway_venv_done +echo Gateway venv already exists, skipping. + +:: ACE-Step venv +:acestep_venv_start +if exist "ace-step\.venv\Scripts\python.exe" goto :acestep_venv_done +echo Creating ACE-Step venv... +echo NOTE: Downloads large CUDA packages. May take 10-20 minutes. +pushd ace-step +uv venv .venv +if errorlevel 1 ( + popd + echo ERROR: Failed to create ACE-Step venv + exit /b 1 +) +uv pip install --python .venv -e . +if errorlevel 1 ( + popd + echo ERROR: Failed to install ACE-Step dependencies + exit /b 1 +) +popd +echo ACE-Step venv OK. +goto :f1_venv_start +:acestep_venv_done +echo ACE-Step venv already exists, skipping. + +:: Foundation-1 venv +:f1_venv_start +if exist "foundation1\.venv\Scripts\python.exe" goto :f1_venv_done +echo Creating Foundation-1 venv (requires Python 3.10)... +uv venv --python 3.10 foundation1\.venv +if errorlevel 1 ( + echo WARNING: Python 3.10 not found. Falling back to default Python. + uv venv foundation1\.venv + if errorlevel 1 ( + echo ERROR: Failed to create Foundation-1 venv + exit /b 1 + ) +) +echo Installing PyTorch 2.5.1 (CUDA 12.1 wheels)... +echo NOTE: This downloads ~2 GB. May take several minutes. +uv pip install --python foundation1\.venv torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu121 +if errorlevel 1 ( + echo ERROR: Failed to install PyTorch + exit /b 1 +) +echo Installing pinned numpy... +uv pip install --python foundation1\.venv "numpy==1.23.5" +if errorlevel 1 ( + echo ERROR: Failed to install numpy + exit /b 1 +) +echo Installing Foundation-1 dependencies... +uv pip install --python foundation1\.venv -e foundation1 +if errorlevel 1 ( + echo ERROR: Failed to install Foundation-1 dependencies + exit /b 1 +) +uv pip install --python foundation1\.venv "fastapi>=0.110.0" "uvicorn[standard]>=0.27.0" +if errorlevel 1 ( + echo ERROR: Failed to install Foundation-1 server dependencies + exit /b 1 +) +echo Pinning setuptools for pkg_resources compatibility... +uv pip install --python foundation1\.venv "setuptools<70" +if errorlevel 1 ( + echo ERROR: Failed to pin setuptools + exit /b 1 +) +echo Foundation-1 venv OK. +goto :sep_venv_start +:f1_venv_done +echo Foundation-1 venv already exists, skipping. + +:: Separator venv +:sep_venv_start +if exist "separator\.venv\pyvenv.cfg" goto :sep_venv_done +echo Creating separator venv... +echo NOTE: Downloads CUDA packages. May take several minutes. +uv venv separator\.venv +if errorlevel 1 ( + echo ERROR: Failed to create separator venv + exit /b 1 +) +uv pip install --python separator\.venv -e "separator[gpu]" +if errorlevel 1 ( + echo ERROR: Failed to install audio-separator + exit /b 1 +) +uv pip uninstall --python separator\.venv torch torchvision +uv pip install --python separator\.venv torch==2.11.0+cu126 torchvision==0.26.0+cu126 --index-url https://download.pytorch.org/whl/cu126 +if errorlevel 1 ( + echo ERROR: Failed to install PyTorch for separator + exit /b 1 +) +uv pip install --python separator\.venv --force-reinstall onnxruntime-gpu +if errorlevel 1 ( + echo ERROR: Failed to install onnxruntime-gpu + exit /b 1 +) +echo Separator venv OK. +goto :venvs_done +:sep_venv_done +echo Separator venv already exists, skipping. + +:venvs_done +echo. + +:: --------------------------------------------------------- +echo [5/6] Creating output directories... +:: --------------------------------------------------------- +if not exist "outputs\clip" mkdir "outputs\clip" +if not exist "outputs\fullsong" mkdir "outputs\fullsong" +if not exist "outputs\separator" mkdir "outputs\separator" +if not exist "uploads\separator" mkdir "uploads\separator" +if not exist "foundation1\generations" mkdir "foundation1\generations" +if not exist "foundation1\models" mkdir "foundation1\models" +echo Done. +echo. + +:: --------------------------------------------------------- +echo [6/6] Downloading model weights... +:: --------------------------------------------------------- + +:: ACE-Step weights - mirrors what the API server does on first initialize: +:: ensure_model_downloaded('acestep-v15-turbo', checkpoints/) +:: ensure_model_downloaded('vae', checkpoints/) +:: Auto-selects HuggingFace vs ModelScope by pinging Google (or ACESTEP_DOWNLOAD_SOURCE env var). +if exist "ace-step\checkpoints\acestep-v15-turbo\model.safetensors" goto :acestep_weights_done +echo Downloading ACE-Step checkpoints (may take several minutes)... +ace-step\.venv\Scripts\python.exe -c "from acestep.api.model_download import ensure_model_downloaded; import os; ckpt=os.path.join(os.getcwd(),'ace-step','checkpoints'); ensure_model_downloaded('acestep-v15-turbo', ckpt); ensure_model_downloaded('vae', ckpt)" +if errorlevel 1 ( + echo ERROR: ACE-Step weight download failed + exit /b 1 +) +:acestep_weights_done +echo ACE-Step weights OK. +echo. + +:: Foundation-1 weights (huggingface_hub is already installed as a dep of stable_audio_tools) +if exist "foundation1\models\RoyalCities-Foundation-1\Foundation_1.safetensors" goto :f1_weights_done +echo Downloading Foundation-1 weights (approx 1.7 GB)... +foundation1\.venv\Scripts\python.exe -c "from huggingface_hub import snapshot_download; snapshot_download('RoyalCities/Foundation-1', local_dir='foundation1/models/RoyalCities-Foundation-1')" +if errorlevel 1 ( + echo ERROR: Foundation-1 weight download failed + exit /b 1 +) +:f1_weights_done +echo Foundation-1 weights OK. +echo. + +:: Separator models are downloaded automatically by audio-separator on first use. +echo Separator models will be downloaded on first use (handled by audio-separator). +echo. + +echo ========================================== +echo Initialization complete! +echo ========================================== +echo. +echo Start the gateway: +echo .venv\Scripts\python.exe main.py +echo. +echo Then load a model: +echo POST http://localhost:8000/v1/models/load +echo Body: {"model": "clip"} -- Foundation-1 (MIDI + WAV) +echo Body: {"model": "fullsong"} -- ACE-Step 1.5 (full songs) +echo. +echo Or separate stems (no model load needed): +echo POST http://localhost:8000/v1/separator/separate +echo Body: multipart/form-data file=