Add LM Backend selector (PT vs VLLM) to reduce VRAM usage
PT backend uses ~1.6 GB VRAM vs ~9.2 GB for VLLM, making it accessible on lower-end GPUs. Adds dropdown in Advanced Settings defaulting to PT.
This commit is contained in:
@@ -232,6 +232,7 @@ export interface GenerationParams {
|
||||
lmTopK?: number;
|
||||
lmTopP?: number;
|
||||
lmNegativePrompt?: string;
|
||||
lmBackend?: 'pt' | 'vllm';
|
||||
|
||||
// Expert Parameters
|
||||
referenceAudioUrl?: string;
|
||||
|
||||
Reference in New Issue
Block a user