diff --git a/17_AI_SCAN.md b/17_AI_SCAN.md new file mode 100644 index 0000000..48eb24a --- /dev/null +++ b/17_AI_SCAN.md @@ -0,0 +1,277 @@ +# Technical Specification: AI Loop Scanning System & Fade-Free Zero-Crossing Slicing + +This document specifies the software architecture, digital signal processing (DSP) algorithms, and API design required to integrate AI-driven automated loop scanning and perfect, fade-free audio slicing (Zero-Crossing Aligned Slicing) without boundary transition effects (Fade-In/Fade-Out). + +--- + +## 1. Feature 1: AI Loop Scan & Automated Marker Labeling + +This feature allows users to quickly scan an audio track (driven by backend AI/DSP) to detect segments with the highest rhythmic or musical periodicity (e.g., drum loops, chord progressions, vocal loops) and automatically map both boundaries using the timeline marker system. + +```text + AI LOOP SCAN PROCESSING FLOW +┌───────────────────┐ 1. Send File ID ┌────────────────────────┐ +│ Frontend Client ├──────────────────────►│ Backend FastAPI Server │ +│ (Click "AI Scan") │◄──────────────────────┤ (Celery Task Worker) │ +└───────────────────┘ 4. Return timestamps└───────────┬────────────┘ + ▲ [t_start, t_end] │ + │ ▼ + │ 2. Analyze Audio Features + │ (Self-Similarity Matrix) + │ │ + │ ▼ + └───────── (Pin Markers automatically) ◄ 3. Snap to Zero-Crossing + +``` + +### 1.1. Workflow + +1. The user selects an audio track within the Main Session and clicks the *AI Scan* button on the AI Panel. +2. The frontend dispatches a request containing the track's `file_id` to the backend gateway. +3. The backend initiates an asynchronous Celery Task, leveraging the `librosa` acoustic processing library to extract spectral feature matrices (Chromagram/Mel-spectrogram) and search for target loop boundaries exhibiting the highest recurrence correlation. +4. Once the optimal loop region $[t_{\text{start}}, t_{\text{end}}]$ is calculated, the backend executes a Zero-Crossing Alignment routine to precisely shift both boundaries to the nearest index where the signal amplitude reaches exactly zero. +5. The processed absolute timestamps $[t'_{\text{start}}, t'_{\text{end}}]$ are returned to the client. The frontend dynamically instantiates and renders timeline markers pinned directly onto that track lane. + +--- + +## 2. Feature 2: AI Analysis & AI Cut (Fade-Free) + +When cutting an audio segment at arbitrary time markers, if a slice intersects a high-amplitude point (non-zero), the continuous physical phase of the waveform is abruptly broken (Jump discontinuity). This generates a sharp, vertical step in the amplitude waveform graph, translating mechanically into an audible, harsh popping or ticking artifact ("click" or "pop") through speakers. + +Standard or basic DAW systems mitigate this issue by adding an ultra-short linear fade envelope (Fade-In/Fade-Out) spanning roughly $5\text{ ms} \rightarrow 10\text{ ms}$. However, this masking method dampens the physical attack phase (transients) of the sound field, which is severely destructive to sharp, high-impact hits such as kick drums or snares. + +The perfect architecture is a **Fade-Free AI Cut**. It dynamically calculates the closest hardware zero-crossing indices—where the acoustic wave amplitude passes through the central horizontal timeline axis ($0\text{V}$ absolute silence)—and executes the audio slice precisely at those coordinates. + +```text + WAVEFORM TIMELINE & FADE-FREE AI CUT PROCESS + Amplitude + ▲ + +1.0 ┼ / \ / \ + │ / \ / \ + │ User-defined/ \ / \ User-defined + │ selection marker \ / \ selection marker + 0.0 ┼───────○─────────────○─────○─────────○───────► Time Axis + │ / \ / \ / \ / \ + │ / \ / \ / \ / \ + -1.0 ┼────/──────\──────/─────○─────\───/─────\─ + ▲ + │ [ AI CUTS EXACTLY HERE ] + │ Amplitude = 0 (Sound is silent) + │ Absolute zero Click/Pop anomalies! + +``` + +### 2.1. Workflow + +1. The user left-clicks and drags a time selection window $[T_{\text{start}}, T_{\text{end}}]$ across the target Waveform Lane. +2. The user clicks **AI Analysis**: The backend calculates and shifts both bounding coordinates slightly to align with physical zero-crossing sample indices ($T'_{\text{start}}$ and $T'_{\text{end}}$), instantly refreshing the highlighted overlay on the screen viewport. +3. The user clicks **AI Cut**: The engine slices the raw binary sample stream from index $T'_{\text{start}}$ to $T'_{\text{end}}$ straight inside RAM, generates a new track row directly underneath, and drops the cut clip onto it. The asset remains un-rendered and pure, with absolutely no volume fade multi-stage nodes applied. + +--- + +## 3. Mathematical Zero-Crossing Optimization Algorithm (DSP Math) + +Let $x[n]$ represent a single-channel discrete sample array containing mono audio amplitudes ($0$ mapping to the left track lane channel). At the target sample index address $n_{\text{target}}$ derived from the user's raw timeline click event, the engine establishes a symmetrical boundary scanning window of size $W$ (typically set to a $50\text{ ms}$ horizontal time width): + +$$n_{\text{start}} = n_{\text{target}} - \frac{W \cdot f_s}{2}, \quad n_{\text{end}} = n_{\text{target}} + \frac{W \cdot f_s}{2}$$ + +Where $f_s$ tracks the absolute project Sample Rate hardware clock (e.g., $44100\text{ Hz}$). + +### 3.1. Physical Phase Inversion Condition (Zero-Crossing Condition) + +The optimization loop evaluates all internal sample index integers $i \in [n_{\text{start}}, n_{\text{end}}]$ that satisfy the algebraic sign-inversion condition rule: + +$$x[i] \cdot x[i+1] \le 0$$ + +### 3.2. Optimization Criterion + +Among all matching coordinate entries captured by the boundary condition filter, the algorithm targets the specific index $i_{\text{best}}$ that minimizes the spatial sample offset relative to the operator's input selection address ($n_{\text{target}}$): + +$$i_{\text{best}} = \arg\min_{i} \left\vert{} i - n_{\text{target}} \right\vert{}$$ + +At coordinate point $i_{\text{best}}$, the immediate signal amplitude approaches zero ($x[i_{\text{best}}] \approx 0$). Slicing at this address ensures absolute physical phase continuity when the audio stream is partitioned or unlinked. + +--- + +## 4. Python Backend Implementation Manual (Docker Celery DSP Worker) + +This prototype Python module (`core/ai_dsp_engine.py`) runs on the backend Celery worker environment to execute automated loop indexing and fade-free zero-crossing slicing: + +```python +import numpy as np +import librosa + +class AIDSPEngine: + @staticmethod + def find_exact_zero_crossing(y: np.ndarray, sr: int, target_time: float, window_ms: float = 50.0) -> float: + """ + Locates the absolute nearest physical zero-crossing sample index to target_time (seconds). + Returns the optimized timeline index position in seconds where amplitude hits 0. + """ + target_sample = int(target_time * sr) + window_samples = int((window_ms / 1000.0) * sr) + + # Define symmetrical horizontal boundary window + start_idx = max(0, target_sample - window_samples // 2) + end_idx = min(len(y) - 2, target_sample + window_samples // 2) + + y_segment = y[start_idx:end_idx] + + # DSP Condition logic tracking sign inversion: y[i] * y[i+1] <= 0 + zero_crossings = np.where(y_segment[:-1] * y_segment[1:] <= 0)[0] + + if len(zero_crossings) == 0: + # Fallback: if no sign change occurs, return the absolute minimum sample inside the viewport + abs_min_idx = np.argmin(np.abs(y_segment)) + return float((abs_min_idx + start_idx) / sr) + + # Translate local segment array address back to absolute buffer coordinates + absolute_crossings = zero_crossings + start_idx + + # Isolate the crossing point closest to the raw target_sample baseline + distances = np.abs(absolute_crossings - target_sample) + best_sample_idx = absolute_crossings[np.argmin(distances)] + + return float(best_sample_idx / sr) + + @classmethod + def scan_best_loop_regions(cls, y: np.ndarray, sr: int, min_duration: float = 2.0, max_duration: float = 8.0) -> list: + """ + Evaluates spectral Self-Similarity Matrices (Recurrence plots) to extract + the most musically periodic and cohesive loop segments within the track. + """ + # 1. Compute harmonic structural properties via Chroma Constant-Q Transform + chroma = librosa.feature.chroma_cqt(y=y, sr=sr) + + # 2. Compile the Self-Similarity Matrix (Cosine Recurrence Plot) + # This maps global structural recurrence profiles across runtime frame vectors + from sklearn.metrics.pairwise import cosine_similarity + ssm = cosine_similarity(chroma.T, chroma.T) + + num_frames = ssm.shape[0] + hop_length = 512 + frame_duration = hop_length / sr + + best_score = -1.0 + best_loop = (0.0, 4.0) # Fallback baseline setup to target initial 4 seconds + + # Scan sub-diagonals to track high-density recurring correlation coefficients + # Diagonals parallel to the main identity path flag strict periodic cycles + min_frames = int(min_duration / frame_duration) + max_frames = int(max_duration / frame_duration) + + for lag in range(min_frames, min_frames * 4): # Trace delay frames matching typical 1-2 measure blocks + if lag >= num_frames: + break + # Accumulate mean recurrence indices across the active sub-diagonal line + score = np.mean(np.diagonal(ssm, offset=lag)) + if score > best_score: + best_score = score + # Map optimized chronological boundaries + start_frame = 0 + end_frame = min(num_frames - 1, start_frame + lag) + + t_start = start_frame * frame_duration + t_end = end_frame * frame_duration + + best_loop = (t_start, t_end) + + # 3. Lock boundaries to precise physical zero-crossings to prevent transient click noise + t_start_zero = cls.find_exact_zero_crossing(y, sr, best_loop[0]) + t_end_zero = cls.find_exact_zero_crossing(y, sr, best_loop[1]) + + return [{"start_time": t_start_zero, "end_time": t_end_zero, "score": float(best_score)}] + + @classmethod + def slice_and_copy_with_zero_crossing( + cls, + y: np.ndarray, + sr: int, + start_time: float, + end_time: float + ) -> tuple: + """ + Slices an audio data array from start_time to end_time using zero-crossing alignment. + Strictly bypasses linear or exponential fade configurations. + """ + # Align bounding start and termination boundaries directly to zero-amplitude addresses + t_start_zero = cls.find_exact_zero_crossing(y, sr, start_time) + t_end_zero = cls.find_exact_zero_crossing(y, sr, end_time) + + sample_start = int(t_start_zero * sr) + sample_end = int(t_end_zero * sr) + + # Squeeze out raw buffer array slice without applying any destructive envelope modifiers + y_sliced = np.copy(y[sample_start:sample_end]) + + return y_sliced, t_start_zero, t_end_zero + +``` + +--- + +## 5. Serialized API Data Transfer Protocols + +During data exchange cycles initiated over the AI Panel UI layer, the client application communicates with the FastAPI routing layer via the following structured JSON payloads: + +### 5.1. API 1: AI Loop Scanning (POST `/api/v1/audio/ai-scan`) + +* **Request Payload (Client $\rightarrow$ Server):** + +```json +{ + "track_id": "1", + "file_id": "creak_forest_raw.wav", + "min_loop_duration": 2.0, + "max_loop_duration": 6.0 +} + +``` + +* **Response Payload (Server $\rightarrow$ Client):** + +```json +{ + "success": true, + "track_id": "1", + "suggested_loops": [ + { + "start_time": 1.4589, + "end_time": 5.4592, + "score": 0.892 + } + ] +} + +``` + +*(Upon parsing this response, the frontend layout engine executes an automated marker rendering pass, pinning visual handles precisely at `start_time` and `end_time`).* + +### 5.2. API 2: Fade-Free AI Slicing (POST `/api/v1/audio/ai-cut`) + +* **Request Payload (Client $\rightarrow$ Server):** + +```json +{ + "source_track_id": "1", + "file_id": "creak_forest_raw.wav", + "selection_start": 3.120, + "selection_end": 7.450 +} + +``` + +* **Response Payload (Server $\rightarrow$ Client):** + +```json +{ + "success": true, + "output_file_id": "ai_cut_creak_forest_3.1s.wav", + "aligned_start": 3.1192, + "aligned_end": 7.4504, + "duration": 4.3312 +} + +``` + +*(The frontend automatically builds a new track row layout right below the baseline channel, mapping the received `output_file_id` block to mount perfectly at the real-world timeline timestamp indicated by `aligned_start`).* \ No newline at end of file diff --git a/app/core/auth.py b/app/core/auth.py index 24f6e99..b31d2a4 100644 --- a/app/core/auth.py +++ b/app/core/auth.py @@ -60,7 +60,7 @@ def seed_admin(): conn = get_db_connection() cursor = conn.cursor() - default_pwd = os.getenv("DEFAULT_ADMIN_PASSWORD", "admin123").strip() + default_pwd = (os.getenv("DEFAULT_ADMIN_PASSWORD") or "admin123").strip() hashed_pwd = hash_password(default_pwd) now = time.time() @@ -79,9 +79,9 @@ def seed_admin(): """, (admin_id,)) conn.commit() else: - # Guarantee admin account password hash matches default_pwd if must_change_password is true or hash doesn't match - if row["must_change_password"] or not verify_password(default_pwd, row["hashed_password"]): - cursor.execute("UPDATE users SET hashed_password = ? WHERE id = ?", (hashed_pwd, row["id"])) + # Kiểm tra và sửa password admin mặc định nếu cần + if not verify_password(default_pwd, row["hashed_password"]): + cursor.execute("UPDATE users SET hashed_password = ?, must_change_password = 1 WHERE id = ?", (hashed_pwd, row["id"])) conn.commit() conn.close() diff --git a/app/main.py b/app/main.py index d2172a3..a25a1cd 100644 --- a/app/main.py +++ b/app/main.py @@ -18,6 +18,10 @@ os.makedirs(settings.PROCESSED_DIR, exist_ok=True) app = FastAPI(title="SonicForge API Engine") +from fastapi.middleware.gzip import GZipMiddleware + +app.add_middleware(GZipMiddleware, minimum_size=500) + app.add_middleware( CORSMiddleware, allow_origins=["*"], diff --git a/app/storage/sonicforge.db b/app/storage/sonicforge.db index 7e8c4a9..cb3f2f8 100644 Binary files a/app/storage/sonicforge.db and b/app/storage/sonicforge.db differ diff --git a/app/templates/index.html b/app/templates/index.html index 1d4bd15..c626c77 100644 --- a/app/templates/index.html +++ b/app/templates/index.html @@ -1600,10 +1600,10 @@ {/* Tools Section - passthrough từ main session */}
@@ -1683,30 +1683,30 @@ {/* Transport Controls */}
@@ -1898,15 +1898,24 @@ const [newPassword, setNewPassword] = useState(''); const [error, setError] = useState(''); const [loading, setLoading] = useState(false); - useEffect(() => { if (mode) setActiveTab(mode); }, [mode]); + useEffect(() => { + if (mode) setActiveTab(mode); + if (mode === 'force_change' && !oldPassword) { + setOldPassword('admin123'); + } + }, [mode]); const handleSubmit = async (e) => { e.preventDefault(); setError(''); setLoading(true); try { if (activeTab === 'login') { const targetUsername = username.trim() || 'admin'; - const res = await window.SonicAPI.login(targetUsername, password.trim()); + const targetPwd = password.trim() || 'admin123'; + const res = await window.SonicAPI.login(targetUsername, targetPwd); localStorage.setItem('sonic_token', res.access_token); localStorage.setItem('sonic_user', JSON.stringify(res.user)); + if (res.user && res.user.must_change_password) { + setOldPassword(targetPwd); + } onSuccess(res.user, res.access_token); } else if (activeTab === 'register') { const res = await window.SonicAPI.register(username.trim(), email.trim(), password.trim()); @@ -1922,7 +1931,7 @@ onSuccess(user, res.access_token); } } catch (err) { - setError(err.message || 'Thao tác không thành công'); + setError(err.message || (activeTab === 'login' ? 'Tài khoản hoặc mật khẩu không chính xác. (Nếu bạn đã đổi mật khẩu trước đó, vui lòng nhập mật khẩu mới mà bạn đã tạo)' : 'Thao tác không thành công')); } finally { setLoading(false); } }; const isForceMode = activeTab === 'force_change'; @@ -1980,6 +1989,15 @@ + {activeTab === 'login' && !isForceMode && ( + + )} {!isForceMode && (
@@ -2333,7 +2351,7 @@ }; // ── Tab System (LOOP_EDITOR_2.md §1) ── - const [activeTab, setActiveTab] = useState('subtab_1'); + const [activeTab, setActiveTab] = useState('main'); const [subTabSelectedNodeTime, setSubTabSelectedNodeTime] = useState(null); const [subTabNormVal, setSubTabNormVal] = useState(0); const [subTabGainVal, setSubTabGainVal] = useState(100); @@ -2790,6 +2808,10 @@ useEffect(() => { const handler = (e) => { + // Bypass global hotkeys when typing inside input/textarea/contentEditable elements + if (e.target && (e.target.tagName === 'INPUT' || e.target.tagName === 'TEXTAREA' || e.target.isContentEditable)) { + return; + } const ctrl = e.ctrlKey || e.metaKey; const alt = e.altKey; @@ -5762,7 +5784,7 @@ ) : ( @@ -5782,7 +5804,7 @@ className={`px-1.5 py-0.5 rounded text-[10px] border transition ${ showAIConfig ? 'bg-purple-900 text-purple-200 border-purple-700' : 'bg-zinc-800 text-zinc-400 border-zinc-700 hover:text-zinc-200' }`}> - +
@@ -5797,7 +5819,7 @@ ? 'text-cyan-400 border-cyan-500 bg-zinc-800/50' : 'text-zinc-500 border-transparent hover:text-zinc-300 hover:bg-zinc-800/30' }`}> - Main Session + Main Session {subTabs.map(st => (
@@ -5807,13 +5829,13 @@ ? 'text-amber-400 border-amber-500 bg-zinc-800/50' : 'text-zinc-500 border-transparent hover:text-zinc-300 hover:bg-zinc-800/30' }`}> - + {st.label}
))} @@ -5823,7 +5845,7 @@ {showAIConfig && (
- Cấu hình cổng kết nối API + Cấu hình cổng kết nối API
@@ -5866,17 +5888,17 @@
- +
+
+
@@ -5938,7 +5967,7 @@ } }} className="w-7 h-7 flex items-center justify-center bg-zinc-800 hover:bg-zinc-700 text-zinc-200 rounded border border-zinc-700 transition" - title="Quay lại đầu"> + title="Quay lại đầu"> + title="Đầu vùng chọn"> + title="Stop"> + title="Cuối vùng chọn"> + title="Đến cuối">
Snap loadFileOnTrack(track.id, e.target.files[0])} /> - - - + + +
handleTrackResizeMouseDown(e, track.id)} className="absolute bottom-0 left-0 right-0 h-1 cursor-ns-resize z-30 hover:bg-cyan-500/50 transition-colors" onClick={e => e.stopPropagation()} />
); - })} + }) + )}
@@ -6402,7 +6432,7 @@ {track.buffer && (
+ className="px-1.5 py-0.5 bg-zinc-900/95 text-zinc-300 rounded text-[9px] flex items-center gap-1 border border-zinc-700/50"> Cắt
)}
handleTrackResizeMouseDown(e, track.id)} @@ -6413,7 +6443,7 @@
{ if (draggedClipRef.current) { setHoveredTrackId(addNewTrack()); } }} onClick={addNewTrack}> - Kéo clip xuống hoặc Click tạo Track + Kéo clip xuống hoặc Click tạo Track
{selectionMode !== 'local' && selLeft !== null && selRight !== null && selRight > selLeft && (
Sub-Tab
{vTrack ? ( @@ -6493,11 +6523,11 @@
+ className={`px-2 py-0.5 text-[14px] rounded border font-bold ${st.isLooping ? 'bg-amber-700 text-white border-amber-500' : 'bg-zinc-800 text-zinc-400 border-zinc-700'}`}> Loop: setSubTabs(prev => prev.map(s => s.id === st.id ? {...s, loopCount: Math.max(0, parseInt(e.target.value) || 0)} : s))} className="w-12 bg-black text-amber-300 text-[14px] px-1 py-0.5 rounded border border-zinc-700 text-center font-mono" title="Loop count" /> @@ -6538,11 +6568,11 @@
@@ -6672,28 +6702,28 @@ - Scroll: Zoom + Scroll: Zoom | - Ctrl+Scroll: Playhead + Ctrl+Scroll: Playhead @@ -6708,28 +6738,28 @@
@@ -6738,39 +6768,39 @@
e.stopPropagation()}>
@@ -6781,11 +6811,11 @@ {/* ── Toast ── */} {toastMessage && (
- + }`}> {toastMessage.text}
)}