feat: add audio track support with WAV/MP3 playback and looping

Introduce KGAudioTrack and KGAudioRegion as new track/region types
alongside existing MIDI tracks. Users can import WAV, MP3, OGG, FLAC,
and AAC files into audio tracks with full playback, looping, and
solo/mute/volume support.

New core models:
- KGAudioTrack (extends KGTrack with TrackType.Wave)
- KGAudioRegion (extends KGRegion with audio file reference)
- KGAudioPlayerBus (Tone.Gain + ToneBufferSource-based playback)
- KGAudioFileStorage (OPFS media/ directory for binary audio files)

New commands:
- AddAudioTrackCommand (create audio track with player bus)
- ImportAudioCommand (import audio file, create region at playhead,
auto-expand maxBars)

Playback features:
- Audio regions scheduled via Tone.Transport alongside MIDI events
- Loop support with duration capping at loop boundaries
- Resume safety offset to prevent ToneBufferSource timing misses
- Cross-track buffer copy when moving audio regions between tracks
- Proper buffer cleanup on track deletion and project switch

UI changes:
- Separate "+ MIDI" and "+ Audio" buttons in top-left spacer
- Audio track shows upload button instead of instrument selector
- FileImportModal reused for audio file drag-and-drop upload
- Real waveform rendering on audio region canvas
- Green color scheme for audio regions (vs blue for MIDI)
- Pencil icon and resize handles hidden for audio regions
- Cross-type region moves blocked (MIDI ↔ Audio)
- Piano roll blocked for audio regions
- Audio region deletion support

Infrastructure:
- Project structure version bumped to 4 (no-op migration)
- class-transformer discriminators updated for new subtypes
- RemoveTrackCommand updated to clean up audio player buses
This commit is contained in:
Xiaohan-Tian
2026-04-09 21:54:54 -07:00
parent de2979178e
commit fc84edfc59
23 changed files with 1406 additions and 100 deletions
+72
View File
@@ -0,0 +1,72 @@
import { OPFS_CONSTANTS } from '../../constants/coreConstants';
/**
* KGAudioFileStorage — Utility for storing and loading audio files
* in the OPFS media/ directory alongside project data.
*/
export class KGAudioFileStorage {
/**
* Store an audio file in the project's media/ directory.
*/
public static async storeAudioFile(
projectName: string,
fileId: string,
file: File
): Promise<void> {
const mediaDir = await KGAudioFileStorage.getMediaDir(projectName);
const fileHandle = await mediaDir.getFileHandle(fileId, { create: true });
const writable = await fileHandle.createWritable();
await writable.write(await file.arrayBuffer());
await writable.close();
console.log(`Stored audio file ${fileId} (${file.size} bytes) for project "${projectName}"`);
}
/**
* Load an audio file as ArrayBuffer from the project's media/ directory.
*/
public static async loadAudioFile(
projectName: string,
fileId: string
): Promise<ArrayBuffer> {
const mediaDir = await KGAudioFileStorage.getMediaDir(projectName);
const fileHandle = await mediaDir.getFileHandle(fileId);
const file = await fileHandle.getFile();
return file.arrayBuffer();
}
/**
* Delete an audio file from the project's media/ directory.
*/
public static async deleteAudioFile(
projectName: string,
fileId: string
): Promise<void> {
try {
const mediaDir = await KGAudioFileStorage.getMediaDir(projectName);
await mediaDir.removeEntry(fileId);
console.log(`Deleted audio file ${fileId} from project "${projectName}"`);
} catch (error) {
console.warn(`Failed to delete audio file ${fileId}:`, error);
}
}
/**
* Generate a unique audio file ID preserving the original file extension.
*/
public static generateAudioFileId(originalFileName: string): string {
const ext = originalFileName.split('.').pop()?.toLowerCase() || 'wav';
const timestamp = Date.now();
const random = Math.random().toString(36).substring(2, 10);
return `audio_${timestamp}_${random}.${ext}`;
}
/**
* Get the media directory handle for a project.
*/
private static async getMediaDir(projectName: string): Promise<FileSystemDirectoryHandle> {
const root = await navigator.storage.getDirectory();
const projectsDir = await root.getDirectoryHandle(OPFS_CONSTANTS.ROOT_DIR);
const projectDir = await projectsDir.getDirectoryHandle(projectName);
return projectDir.getDirectoryHandle(OPFS_CONSTANTS.MEDIA_DIR, { create: true });
}
}