3dtours/SonicForgeStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: add UI image to README.md

Browse Source
This commit is contained in:
3dtours
2026-07-21 18:25:03 +07:00
parent d5143b440a
commit 9de9ae965f
30 changed files with 2 additions and 6156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
10_SNAP_TOOL.md
-165
View File
@@ -1,165 +0,0 @@
# Technical Specification: Grid Snapping System (Grid Snapping Specification)
This document defines the graphical user interface design and coordinate/signal processing algorithms required to build a synchronized grid snapping feature across both the Web Frontend and the Dockerized Python Desktop Backend.
---
## 1. Toolbar UI Design Upgrade
The toolbar layout has been expanded to double its physical vertical height. This increase in interactive space allows for larger navigation buttons and the integration of a dedicated Snap controller.
```text
+───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────+
| [Pro Toolbar - Height: 64px] |
| +──────────+ +──────────+ +──────────+ +──────────+ +───────────────────────────────────────────────────────────────+ |
| | Cut | | Copy | | Paste | | Snap: | | Transport Monitor | |
| | [Ctrl+X]| | [Ctrl+C]| | [Ctrl+V]| | [1/4 ▼] | | [Tempo: 120 BPM] [Time Signature: 4/4] [Bar:Beat 1.3.00] | |
| +──────────+ +──────────+ +──────────+ +──────────+ +───────────────────────────────────────────────────────────────+ |
+───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────+
```
### 1.1. Snap Dropdown Configuration
* **Placement:** Located immediately following the *Paste* button on the primary toolbar row.
* **UI Syntax:** `Snap: <Dropdown_Widget>`
* **Dropdown Option Matrix:**
* `free`: Disables snapping; allows unrestricted pixel-by-pixel dragging.
* `1`: Snaps to the beginning of each complete measure (Whole Bar / 1/1).
* `1/2`: Divides the bar into 2 subdivisions (Half Note).
* `1/4`: Divides the bar into 4 subdivisions (Quarter Note / 1 Beat).
* `1/8`: Divides the bar into 8 subdivisions (Eighth Note).
* `1/16`: Divides the bar into 16 subdivisions (Sixteenth Note).
* `1/32`: Divides the bar into 32 subdivisions (Thirty-second Note).
---
## 2. DSP Grid Math: Grid Subdivision & Time Interval Calculations
To calculate the absolute temporal duration between individual grid lanes, the time metrics must be derived dynamically from the project's master tempo.
Let:
* $B$ be the project tempo (Beats Per Minute, e.g., $120\text{ BPM}$).
* $T_{\text{beat}}$ be the duration of a single beat (seconds).
* $T_{\text{bar}}$ be the duration of a full measure/bar (seconds)—assuming a standard $4/4$ time signature (4 beats per bar).
The foundational constants are established as follows:
$$T_{\text{beat}} = \frac{60}{B} \quad (\text{seconds})$$
$$T_{\text{bar}} = 4 \times T_{\text{beat}} = \frac{240}{B} \quad (\text{seconds})$$
*Example:* At a tempo of $120\text{ BPM}$, a single $1\text{ Bar}$ measure spans exactly $2.0\text{ seconds}$.
### 2.1. Determining Grid Time Interval Modifiers ($\Delta t$)
Based on the user's active choice inside the snap dropdown selection ($S \in \{\text{free}, 1, 1/2, 1/4, 1/8, 1/16, 1/32\}$), the exact grid time step $\Delta t$ (seconds) is mapped as follows:
$$\Delta t = \begin{cases} 0 & S = \text{free} \\ T_{\text{bar}} & S = 1 \\ \frac{T_{\text{bar}}}{2} & S = 1/2 \\ \frac{T_{\text{bar}}}{4} = T_{\text{beat}} & S = 1/4 \\ \frac{T_{\text{bar}}}{8} & S = 1/8 \\ \frac{T_{\text{bar}}}{16} & S = 1/16 \\ \frac{T_{\text{bar}}}{32} & S = 1/32 \end{cases}$$
---
## 3. Snapping Coordinate Calculation
When a user executes a drag-and-drop event on an audio clip, or updates a timeline marker position, the system continuously converts raw cursor values into aligned coordinates.
```text
Grid Line 1 (k * dt) Grid Line 2 ((k+1) * dt)
│ │
├───────────────○─────────────┤
▲
│ [ Cursor dragging action ]
Raw Time (t_raw)
│
▼ [ Apply Snap round() function ]
────────────────┼─────────────►
Snapped Time (t_snapped)
```
### 3.1. Pixel to Snap-Time Translation Workflow
1. Intercept the actual client-side horizontal cursor position $X_{\text{raw}}$ (pixels).
2. Convert it into a raw timeline duration metric $t_{\text{raw}}$ (seconds) utilizing the current scaling zoom factor $Z$ (pixels/second):
$$t_{\text{raw}} = \frac{X_{\text{raw}}}{Z}$$
3. Apply the rounding constraint formula to lock the raw timestamp to the absolute nearest grid marker:
$$t_{\text{snapped}} = \begin{cases} t_{\text{raw}} & S = \text{free} \\ \text{round}\left( \frac{t_{\text{raw}}}{\Delta t} \right) \times \Delta t & S \neq \text{free} \end{cases}$$
4. Map the snapped timeline index $t_{\text{snapped}}$ back to the layout canvas system coordinates to paint the element at its snapped visual boundary:
$$X_{\text{snapped}} = t_{\text{snapped}} \times Z$$
---
## 4. Python Porting Blueprint
When translating this architectural logic into a desktop Python core environment using frameworks like PyQt6, the snapping evaluations are tied directly into the tracking loop inside the `mouseMoveEvent` handler.
```python
# [PYTHON PORTING BLUEPRINT] - Integrating the Snap algorithm into Python UI layer
import numpy as np
class AudioSnapEngine:
def __init__(self, bpm: float = 120.0):
self.bpm = bpm
def calculate_grid_step(self, snap_option: str) -> float:
"""
Calculates the grid's target duration step (seconds) based on Tempo and Snap selection.
"""
if snap_option == "free":
return 0.0
# 1 Bar in a standard 4/4 signature equals 240 / BPM seconds
t_bar = 240.0 / self.bpm
fraction_map = {
"1": 1.0,
"1/2": 2.0,
"1/4": 4.0,
"1/8": 8.0,
"1/16": 16.0,
"1/32": 32.0
}
division = fraction_map.get(snap_option, 4.0)
return float(t_bar / division)
def snap_time(self, raw_time_seconds: float, snap_option: str) -> float:
"""
Hard-clamps a raw timestamp to the nearest grid milestone. Prevents negative index overflows.
"""
dt = self.calculate_grid_step(snap_option)
if dt == 0.0:
return max(0.0, raw_time_seconds)
# Find the nearest integer index k of the target grid lane: raw_time / dt
k = round(raw_time_seconds / dt)
snapped_time = k * dt
return max(0.0, snapped_time)
```
---
## 5. Visual Grid Alignment
To maintain an intuitive environment for multi-channel editing, whenever a snapping constraint value other than `free` is engaged:
* The rendering engine overlays thin, low-contrast vertical grid lines (`rgba(255, 255, 255, 0.05)`) over the background profile of every active Waveform Lane.
* These marker lines are projected onto every timeline axis point that satisfies a whole multiple increment of $\Delta t$.
* Displaying these alignment indicators ensures that users can visually anticipate bounding snapping positions before releasing their mouse track buttons.
11_REFACTOR_UI.md
-164
View File
@@ -1,164 +0,0 @@
# Technical Specification: Advanced UI Refactoring & Clip Editing Mechanics
This document defines the improved user interface design and advanced audio interaction algorithms to standardize frontend development and porting workflows into a containerized Python DAW application running on Docker.
---
## 1. UI Refactoring Specification
### 1.1. Resolving TCP Horizontal Scroll Overflows (Horizontal Scroll Isolation)
* **Symptom:** When scrolling horizontally across the Timeline, waveform or grid canvas elements incorrectly render on top of the left Track Control Panel (TCP) region.
* **Refactoring Solution:** Enforce strict visual separation using Flexbox constraints. The master arrangement window (Workspace) is divided into two physically adjacent columns with completely isolated presentation variables:
```css
.tcp-column {
width: 300px;
flex-shrink: 0;
position: relative;
z-index: 30; /* Ensures columns stay stacked on top */
background-color: #262626; /* Solid, opaque color mask */
overflow: hidden;
}
.timeline-viewport {
flex: 1 1 0%;
position: relative;
z-index: 10;
overflow-x: auto;
overflow-y: hidden; /* Restricts column to independent horizontal scrolling */
}
```
### 1.2. Prominent Shortcut Labels
* **Graphical Standard:** Scale up text components displaying key combination hints within system dropdown menus and right-click Context Menus.
* **Layout Mapping Properties:**
* Keyboard shortcut font sizing: Scaled up from 10px to 12px (`text-[12px]`).
* Weight property: Configured to `font-semibold`.
* High-contrast color palette: Replace low-contrast gray strings with vivid purple (`text-purple-400` / `#c084fc`) or neon amber (`text-amber-400` / `#fbbf24`) that pop cleanly over the dark `#1e1e1e` canvas backdrop.
* Structural alignment: Push shortcut labels directly to the right edge of the context window (`ml-auto pl-8`).
### 1.3. Enlarged & Centered Toolbar
* **Layout Adjustment:** Primary editing triggers (Cut, Copy, Paste, Snap) are scaled up to $1.5\times$ their legacy sizing boundaries (button height locked at 40px).
* **Viewport Placement:** Move the button group into the center cluster on the same horizontal row plane as the ruler axis (positioned immediately to the left of the Time Ruler). This ensures the sound engineer's focus safely encapsulates macro controls alongside timeline visuals.
---
## 2. Timeline Mechanics & Advanced Clip Editing
### 2.1. Clip Delete vs. Track Delete Logic
The environment explicitly segregates asset deletions from track configurations to protect project layout structures:
* **Clip Erasure (`Delete` Key):** When a user triggers `Delete` or `Backspace` keys while an active Audio Clip segment is selected, the application drops the graphical boundary and unloads its corresponding sample sequence from the timeline. The containing track channel remains safely intact.
* **Track Disassembly (TCP Delete Button):** Add a compact red trash bin icon (`w-4 h-4 text-red-500 hover:text-red-400`) into the right edge profile of every TCP block. Engaging this trigger purges the entire track lane along with all embedded clip blocks out of the project.
### 2.2. Preserve Selection Border Resize
* **Legacy Behavior:** Clicking or interacting directly with selection handles accidentally flags a focus reset, clearing the bôi màu canvas overlay.
* **Preserve & Scale System:**
* When hovering the mouse near the explicit left or right edge boundaries of an active selection zone (within a $\pm 5\text{ px}$ tolerance window), the cursor style changes to `ew-resize`.
* Triggering a mouse drag updates, expands, or shrinks selection markers continuously without clearing the overlay mask.
* **Escape Loop Hook (Clear Focus):** The colored selection range is unmapped if and only if the user executes a `Ctrl + Click` shortcut interaction over an empty, unpopulated quadrant outside the selection bounds.
### 2.3. Sub-tab Sandboxing
When a user highlights a clip portion and triggers "Edit in Sub-tab" or double-clicks a targeted audio asset clip:
1. **Buffer Extraction:** The system maps a non-destructive copy of the target sub-region's audio slice into memory buffers.
2. **Tab Instantiation:** Appends a temporary document window onto the global Tab container bar (e.g., `Tab: Sample_Edit_1`).
3. **Automated Insertion:** Instantiates a single empty track channel workspace inside the tab context and drops the cloned audio segment at the absolute root milestone ($t = 0.0\text{ s}$). Editors evaluate local actions here before clicking *Apply* to pass the updated data payload back to the main session track.
### 2.4. Zero-Crossing Filter Tool & AI Cut
Automated crossfade calculation mechanics to eliminate popping anomalies during clip slicing:
1. A user selects a timeline region and hits the *AI Analysis* utility.
2. The server processes the audio block using NumPy arrays to locate phase inversion milestones (where amplitude values cross from negative to positive indices or vice versa) closest to the selection boundary vectors:
$$x[i] \cdot x[i+1] \le 0$$
3. The engine moves the actual slice boundaries to match these optimized zero-crossing sample addresses ($t'_{\text{start}}$ and $t'_{\text{end}}$).
4. Upon clicking *AI Cut*, the underlying engine runs the physical audio slice at the perfect sample indices, duplicates the segment, and appends it to a freshly populated track lane added right below the source track.
### 2.5. Track Height Resizing
* **Interaction:** Users can hover over the dividing line between two track lanes on either the left TCP column or the right Waveform viewport (the cursor scales to `ns-resize`).
* **Drag-and-Drop Mapping:** Dragging downward expands the specific track lane vertical ceiling (up to an upper bound of $200\text{ px}$), magnifying waveform amplitude layouts for precision edits. Dragging upward reduces the height dimension (down to a lower ceiling of $48\text{ px}$) for macro project navigation.
### 2.6. Time-Stretching & Speed Math
Alters the playback rate (*Speed*) of audio clips directly from the interactive timeline view:
```text
[ Right Boundary Drag Interaction ]
Alt + Left-Click & Drag the right border outwards (Expand)
|<────────────────── Original Clip ──────────────────>|
+─────────────────+─────────────────────────────────────────────────────+──────────+
| Track Waveform | ███████████████████████████████████████████████████ | |
+─────────────────+─────────────────────────────────────────────────────+──────────+
▲ ▲
│ │
│ ▼ [ Expand Rightward ]
+─────────────────+────────────────────────────────────────────────────────────────+
| Track Waveform | █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ |
+─────────────────+────────────────────────────────────────────────────────────────+
│ │
│ Visual Speed Tag: "Speed: 50%" │
|<────────────────────────── D' ─────────────────────────────>|
```
* **Modifier Binding:** Hold down the `Alt` key, left-click the rightmost bounding handle of a clip, and drag the boundary left or right.
* **Speed Ratio Formula ($S$):** Let $D$ represent the native unscaled duration value of the clip block (seconds), and $D'$ map to the modified duration value generated post-drag (seconds). The calculation for the updated target playback rate percentage ($S$) follows:
$$S = \frac{D}{D'} \times 100\%$$
* **Display Modifiers:**
* *Expanding rightward ($D' > D$):* Yields $S < 100\%$, meaning playback velocity drops (deceleration). Depending on DSP choices, pitches can either remain locked or drop proportionally.
* *Compressing leftward ($D' < D$):* Yields $S > 100\%$, accelerating the playback engine velocity through the clip.
* **Visual Metadata Tag:** A bright yellow text overlay displaying the calculated playback velocity percentage (e.g., `Speed: 75.0%` or `Speed: 120.5%`) is pinned directly to the upper-left boundary of the audio clip container.
---
## 3. Python Porting Manual (PyQt6 / PySide6)
When writing execution blocks for time-stretching and audio rate modulations onto the Python backend server layers, leverage standard scientific audio packages such as `numpy` or `rubberband` to scale signal arrays without warping Phase layouts:
```python
# [PYTHON PORTING BLUEPRINT] - Acoustic Time-Stretching Velocity Algorithm
import numpy as np
import librosa
def stretch_audio_clip_speed(y: np.ndarray, sr: int, speed_ratio: float) -> np.ndarray:
"""
Stretches or compresses a NumPy audio signal array using the target speed_ratio factor.
speed_ratio = 0.5 slows down velocity by half (expanding physical layout width by 2x).
speed_ratio = 2.0 doubles velocity (compressing physical layout width by half).
"""
if speed_ratio == 1.0:
return y
# Phase Vocoder approach via Librosa to alter speed while locking pitch (Pitch-preserving stretch):
# y_stretched = librosa.effects.time_stretch(y, rate=speed_ratio)
# Linear Resampling approach (Alters pitch along with velocity - vinyl style deceleration):
num_samples_new = int(len(y) / speed_ratio)
y_resampled = np.interp(
np.linspace(0, len(y) - 1, num_samples_new),
np.arange(len(y)),
y
)
return y_resampled.astype(np.float32)
```
12_SUBTAB.md
-287
View File
@@ -1,287 +0,0 @@
# Technical Specification: Sandbox Isolation & Sub-Tab DSP Editing Algorithms
This document defines the processing workflow design and digital signal processing (DSP) algorithms dedicated to localized clip editing within a temporary isolated document workspace (Sub-tab).
---
## 1. Sandbox Splicing Workflow
When a user highlights a time region on the Main Tab and triggers "Edit in Sub-tab" or presses the edit keyboard shortcut:
```text
[ MAIN TAB - MULTITRACK ]
Track 01: ───[█████ Selected Segment █████]───
│
▼ (Copy to Clipboard Buffer)
[ KHỔI TẠO TAB TẠM THỜI (SUB-TAB) ]
- Instantiates a single Track (Height bounds: 48px - 200px via ns-resize)
- Timeline Ruler axis resets to t = 0.0s
│
▼ (Automated Insertion - Auto-Paste)
Track 01 (Sub-tab): [█████ Isolated Segment █████] at t = 0s
```
* **Extract Buffer:** The underlying engine extracts the binary sample array (`Float32Array`) of the highlighted region from the active track, caching it securely into the application's clipboard buffer memory.
* **Sandbox Environment Initialization:**
* Appends a temporary document window onto the global Tab Bar (e.g., `Tab: sẤit tiá...n` or `Sub_Edit_1`).
* Focuses the viewport down into the sandboxed tab. Here, a single standalone track lane is drawn, mapping the timeline ruler scale to start at $t = 0.0\text{ s}$ up to the absolute duration limit ($T_{\text{clip}}$) of the extracted audio asset.
* **Track Height Resizing:**
* Hovering the cursor over the lower layout bounding path of the track lane changes the style configuration to `ns-resize`.
* Dragging downward expands the vertical height ceiling (up to an upper bound of $200\text{ px}$), maximizing the waveform amplitude drawing path for precision clip editing. Dragging upward compresses the physical row dimensions (down to a lower constraint of $48\text{ px}$) to protect screen space.
* **Auto-Paste Routine:** The framework automates the insertion sequence, dropping the cached array block onto the root index milestone ($t = 0.0\text{ s}$) inside the isolated single-track layer.
---
## 2. Apply & Sync-Back Workflow
When an editor finishes processing steps inside the sandbox workspace and engages the *Apply* action:
```text
[ SUB-TAB - AUDIO SANDBOX ]
y_sub = [█████ Edited Waveform █████]
│
▼ (Click "Apply" - Trigger Overwrite)
[ MAIN TAB - ORIGINAL TRACK ]
Track 01: ───[█████ Overwritten Segment █████]─── at t = t_start
│
(Sub-tab remains open, Undo Stack kept)
▼
[User presses Undo (Ctrl+Z) inside Sub-tab to iterate]
y_sub = [█████ Rollbacked Waveform █████]
│
(Click "Apply" again)
▼
Track 01: ───[█████ Corrected Segment █████]──── at t = t_start
```
### 2.1. Target Mapping & Metadata Linkage
Throughout its lifecycle, each sub-tab persistently locks standard metadata records linking back to the origin source elements:
* `parent_track_id`: Unique identifier referencing the primary source track on the Main Tab.
* `parent_clip_id`: Unique identifier tracking the original source audio clip.
* `t_start` (seconds): The exact historical start time position of the sliced block on the Main Tab timeline view.
* `original_duration` (seconds): The baseline temporal duration of the region prior to modification.
### 2.2. In-place Overwrite & Splicing
* **Edited Buffer Extraction:** The system reads the active sample sequence from the sub-tab ($y_{\text{sub}}$) along with its updated duration boundary $T_{\text{sub}}$ (which fluctuates if time-stretching or rate scaling actions have occurred).
* **Main Session Integration:**
1. The core route mapper checks for the matching `parent_track_id` parameter on the Main Tab.
2. Purges the legacy audio segment stretching from $t_{\text{start}}$ through $t_{\text{start}} + T_{\text{original}}$.
3. Splices the updated signal array $y_{\text{sub}}$ precisely at the historical insertion index $t_{\text{start}}$.
4. **Micro-crossfade:** Executes a ultra-fast crossfade envelope ($10\text{ ms}$) across both the initial and terminating splice boundaries. Blending adjacent files prevents phase cancellation or signal breakage that manifests as transient clicks/pops.
* **Visual Update Tracking:** Commands the canvas engine to redraw the waveform visualization grid for the origin track lane inside the Main Tab view.
### 2.3. Persistence for Iterative Editing
* **Tab Lifetime:** Engaging the *Apply* trigger propagates data back to the primary environment but does **not** close down the active sub-tab view.
* **Undo Stack Isolation:** The tracking loop containing the localized *Undo/Redo History Stack* inside the sub-tab sandbox remains entirely preserved.
* **Iterative Loop Workflow:**
1. If monitoring the Main Tab arrangement uncovers an audio anomaly, the user switches focus back to the Sub-tab workspace.
2. Pressing `Ctrl + Z` (Undo) rollbacks the localized signal to its earlier state.
3. The editor runs separate DSP actions.
4. Hitting *Apply* overwrites the updated audio slice over the same target coordinates on the Main Tab.
* **Explicit Destruction Hook:** The sandboxed tab structure is only unmapped when the user clicks the explicit close icon ($\times$) on the horizontal tab bar.
---
## 3. Sub-Tab DSP Algorithm Specification
Editing operations executed inside the sub-tab environment calculate discrete changes over the amplitude sample arrays ($x[n]$). These map to Web Audio API routines on the client layer and standard NumPy/SciPy audio arrays on the Dockerized backend.
### 3.1. Time-Stretching & Speed Math
Alters the duration bounds of the audio clip with optional pitch-shifting linking logic:
* **Pitch-preserving Time-stretching:** Utilizes the Phase Vocoder method to analyze the Short-Time Fourier Transform (STFT) of the signal, shifts spectral frames across the frequency domain, and reconstructs the audio via the Inverse Short-Time Fourier Transform (ISTFT) to align with a new playback velocity ratio $S$:
$$S = \frac{D}{D'} \times 100\%$$
*Where:* $D$ corresponds to the legacy unscaled duration (seconds), and $D'$ maps to the updated value post-resizing (executed by holding down the `Alt` key and dragging the right boundary handle).
* **Resampling (Pitch-shifting Speed Scale):** Runs a standard linear interpolation algorithm to resample the core data array size:
$$x_{\text{new}}[m] = x\left[ \frac{m \cdot D}{D'} \right]$$
### 3.2. Peak Normalization
Amplifies the signal scale uniformly across the active block until the single maximum absolute sample peak reaches a specified ceiling parameter $A_{\text{target}}$ (typically locked at $1.0$ or $0\text{ dBFS}$):
1. Evaluate the absolute maximum peak within the array bounds:
$$A_{\text{max}} = \max_{n=0}^{N-1} \vert x[n] \vert$$
2. Compute the static gain multiplier constant $G$:
$$G = \frac{A_{\text{target}}}{A_{\text{max}}}$$
3. Multiply the entire audio array values by $G$:
$$x_{\text{norm}}[n] = x[n] \cdot G$$
### 3.3. Volume Gain Adjustment (dB Scaling)
1. Capture the decibel variance target ($\Delta \text{dB}$).
2. Translate the logarithmic value into a standard linear scalar multiplier variable $G_{\text{linear}}$:
$$G_{\text{linear}} = 10^{\frac{\Delta \text{dB}}{20}}$$
3. Apply the gain multiplier directly into the sample values:
$$x_{\text{gained}}[n] = x[n] \cdot G_{\text{linear}}$$
### 3.4. Pitch Shifting
Shifts the fundamental frequencies of the signal up or down by a specific number of semitones ($n$) while keeping the temporal duration value completely intact.
* **Frequency Transposition Ratio ($F_{\text{ratio}}$):**
$$F_{\text{ratio}} = 2^{\frac{n}{12}}$$
* **DSP Processing Pipeline:** Employs either a Pitch Synchronous Overlap and Add (PSOLA) routine or a spectral Phase Vocoder to expand/compress the frequency components, then passes the array into a time-stretching step to return the physical track length to its source metric $T_{\text{clip}}$.
### 3.5. Linear Fade-In & Fade-Out Curves
Applies a linear fading envelope over the boundaries of the audio data block.
* **Linear Fade-In Envelope** (Across a duration bound of $L_{\text{fade}}$ samples):
$$x_{\text{fade}}[n] = x[n] \cdot \left( \frac{n}{L_{\text{fade}}} \right) \quad \text{for } 0 \le n < L_{\text{fade}}$$
* **Linear Fade-Out Envelope** (Across the final trailing $L_{\text{fade}}$ samples):
$$x_{\text{fade}}[N - 1 - n] = x[N - 1 - n] \cdot \left( \frac{n}{L_{\text{fade}}} \right) \quad \text{for } 0 \le n < L_{\text{fade}}$$
### 3.6. Array Splitting & Merging
* **Split at Position ($n_{\text{cut}}$):** Unlinks a single sample block $x[n]$ of size $N$ into two separate independent sub-arrays:
$$x_1[n] = x[n] \quad (0 \le n < n_{\text{cut}})$$
$$x_2[n] = x[n + n_{\text{cut}}] \quad (0 \le n < N - n_{\text{cut}})$$
* **Merge Segments:** Concatenates separate sample sequences end-to-end. The stitching logic runs a $10\text{ ms}$ micro-crossfade overlay envelope at the junction to smooth out phase gaps that prompt click artifacts.
---
## 4. Python Backend Implementation Manual
This prototype Python class (`core/sub_tab_dsp.py`) handles the sandboxed operations and includes the crossfaded structural splicing algorithm designed to run inside the Docker engine:
```python
import numpy as np
import scipy.signal as signal
import librosa
class SubTabDSPEngine:
@staticmethod
def change_speed(y: np.ndarray, sr: int, speed_ratio: float, preserve_pitch: bool = True) -> np.ndarray:
"""
Alters the playback velocity (Time-Stretching) of a NumPy signal array.
"""
if speed_ratio == 1.0:
return y
if preserve_pitch:
return librosa.effects.time_stretch(y, rate=speed_ratio)
else:
num_samples_new = int(len(y) / speed_ratio)
return signal.resample(y, num_samples_new)
@staticmethod
def normalize(y: np.ndarray, target_db: float = 0.0) -> np.ndarray:
"""
Performs Peak Normalization on an array to scale it to the target decibel value.
"""
target_amplitude = 10.0 ** (target_db / 20.0)
max_amplitude = np.max(np.abs(y))
if max_amplitude == 0:
return y
gain = target_amplitude / max_amplitude
return y * gain
@staticmethod
def merge_back_to_parent(
parent_track_audio: np.ndarray,
sr: int,
edited_sub_audio: np.ndarray,
start_seconds: float,
original_duration_seconds: float
) -> np.ndarray:
"""
Splices the modified audio segment from the Sub-tab back into the parent track array.
Applies a 10ms micro-crossfade at the boundaries to eliminate pop/click noise.
"""
start_sample = int(start_seconds * sr)
original_samples_len = int(original_duration_seconds * sr)
edited_samples_len = len(edited_sub_audio)
crossfade_samples = int(0.01 * sr) # 10ms crossfade window
# 1. Allocate the target output array dimension bounds
new_total_len = len(parent_track_audio) - original_samples_len + edited_samples_len
output_audio = np.zeros(new_total_len, dtype=np.float32)
# 2. Extract leading unedited block
output_audio[:start_sample] = parent_track_audio[:start_sample]
# 3. Stitch the modified audio payload
output_audio[start_sample:start_sample + edited_samples_len] = edited_sub_audio
# 4. Extract trailing unedited block
post_start_original = start_sample + original_samples_len
post_start_new = start_sample + edited_samples_len
output_audio[post_start_new:] = parent_track_audio[post_start_original:]
# 5. Execute micro-crossfade across the initial splice junction
if start_sample > crossfade_samples:
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
# Smooth 10ms interpolation overlay
output_audio[start_sample : start_sample + crossfade_samples] = (
edited_sub_audio[:crossfade_samples] * fade_in_ramp +
parent_track_audio[start_sample : start_sample + crossfade_samples] * fade_out_ramp
)
# 6. Execute micro-crossfade across the trailing splice junction
if post_start_new + crossfade_samples < len(output_audio):
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
output_audio[post_start_new : post_start_new + crossfade_samples] = (
parent_track_audio[post_start_original : post_start_original + crossfade_samples] * fade_in_ramp +
edited_sub_audio[-crossfade_samples:] * fade_out_ramp
)
return output_audio
```
13_SUBTAB_EDITOR.md
-233
View File
@@ -1,233 +0,0 @@
# Technical Specification: Advanced Editing Toolset & Volume Automation Envelope on Sub-Tab
This document defines the interactive layout design and signal processing algorithms for the advanced localized editing toolset contained within the isolated temporary document workspace (Sub-tab).
---
## 1. Target Selection Scope
The toolset within the Sub-tab environment supports two target operational boundaries:
* **Global Clip:** When no specific timeline selection highlighted mask is present, all active DSP effects apply uniformly across the entire length of the extracted Audio Clip.
* **Selected Range:** When an explicit timeline segment $[T_{\text{start}}, T_{\text{end}}]$ is highlighted by the user, DSP routines calculate changes exclusively inside those boundaries. Splice junctions automatically compute crossfades to mitigate transient click/pop anomalies.
---
## 2. Ruler-Based Tools
These utilities display as intuitive, linear slider scales (Sliders/Rulers) embedded in the top toolbar row:
```text
[ Normalize: |======o======| 0 dB ] [ Gain: |====o====| +3 dB ] [ Pitch: |==o==| -2 Semi ]
```
### 2.1. Peak Normalization
* **UI Layout:** A slide scale control allowing users to configure target amplitude thresholds variable from $-12\text{ dBFS}$ down to $0\text{ dBFS}$.
* **DSP Math Algorithm:** Locate the maximum absolute peak amplitude value $A_{\text{max}}$ within the targeted area, then multiply all active samples by a static scalar gain multiplier $G$:
$$G = \frac{10^{\frac{\text{Target\_dB}}{20}}}{A_{\text{max}}}$$
### 2.2. Volume Up / Down (Quick Gain)
* **UI Layout:** A linear sliding ruler modulating the overall absolute gain structure of the focused segment.
* **Operational Range:** Adjustable from $-\infty\text{ dB}$ (complete mute attenuation) up to $+12\text{ dB}$ of linear amplification.
### 2.3. Pitch Shifting
* **UI Layout:** A calibrated slider modifying the project's fundamental frequencies discrete in semitones or cents.
* **Operational Range:** Boundaries map from $-12\text{ semitones}$ (one octave down) to $+12\text{ semitones}$ (one octave up).
* **DSP Engine Routine:** Employs a spectral Phase Vocoder to shift frequencies without affecting the physical, real-time duration layout of the segment.
---
## 3. Graph-Based Fades
Fading curves overlay graphically directly onto the highlighted waveform canvas region, enabling precise boundary attenuation adjustments:
```text
Linear Fade-In Exponential Fade-Out
+───────────────────────────+ +───────────────────────────+
| /███████████████| |███████████\ |
| / ███████████████| |███████████ \ |
| / ███████████████| |███████████ \___ |
| / ███████████████| |███████████ \______|
+───────────────────────────+ +───────────────────────────+
|<──────── Fade-In ────────>| |<─────── Fade-Out ────────>|
```
* **Fade-In:** Multiplies an ascending amplitude ramp from $0.0$ to $1.0$ at the starting index profile of the selection region. Users can toggle between **Linear** or **Exponential** curves to achieve a smoother, more psychoacoustically natural volume build-up.
* **Fade-Out:** Multiplies a descending amplitude decay ramp from $1.0$ down to $0.0$ at the trailing boundary edge of the selection range.
---
## 4. Ruler Percentage Stretch Tool
A dedicated percentage metric scale control (`Ruler %`) sitting on the control toolbar dictates time-stretching and playback velocity parameters:
```text
[ Speed Stretch %: |========o========| 100% (Native) ] -> Range: 50% (Half Speed) - 200% (Double Speed)
```
* **Interaction Mapping:** Users drag the percentage slider node or hold down the `Alt` key and drag the rightmost boundary edge of the clip along the horizontal axis to change this scale metric.
* **Sync Formula:** Let $D$ map to the unscaled native duration value, and $D'$ map to the target modified duration footprint. The resulting structural playback speed ratio percentage ($S$) is given by:
$$S = \frac{D}{D'} \times 100\%$$
* **UI Representation:** A bright yellow text metadata indicator (e.g., `Speed: 85.3%`) is rendered at the top-left section of the audio clip bounding boundary.
---
## 5. Ultra-Zoom & Zero-Crossing Alignment
To facilitate precision structural slicing at sample-level resolutions, the sub-tab canvas allows microscopic viewport expansion:
```text
MICRO VIEWPORT ZOOM (ULTRA ZOOM-IN)
+─────────────────────────────────────────────────────────────────+
| Waveform renders discrete contiguous sample nodes explicitly |
| ○ (Sample i) |
| / \ |
| ─────────────────/───\─────────────────────────────► 0V Axis |
| \ ○ (Sample i+2) |
| \ / |
| \_○ (Sample i+1 - Zero-Crossing Point)|
+─────────────────────────────────────────────────────────────────+
```
* **Upper Viewport Scaling Limit:** Allows zooming in up to an extreme lower threshold of $2000\text{ pixels/second}$. At this zoom metric, layout compilation transitions away from downsampled peak profiles (Peak Waveform) to render actual discrete **sample nodes** interconnected by fine lines.
* **Zero-Line Snapping Logic:** When establishing selection boundaries, the tracking loop automatically snaps the horizontal selection cursor coordinate to the nearest available sample address exhibiting an algebraic phase inversion (sign change):
$$x[i] \cdot x[i+1] \le 0$$
---
## 6. Top Duration Timeline
Directly above the isolated sub-tab waveform canvas lane, a dedicated horizontal measuring ruler tracks clip timing data:
```text
| 0:00.000 | 0:01.000 | 0:02.000 | 0:03.000 | 0:04.000 (Duration: 4.152s)
+───────────────────────────────────────────────────────────────────────────────────────+
| [==================== VÙNG QUÉT CHỌN (RANGE SELECTION) ====================] |
+───────────────────────────────────────────────────────────────────────────────────────+
```
* **Total Duration Monitoring:** Renders the absolute, precise time extent of the isolated audio block in the right-hand corner of the timeline ruler layout (e.g., `Duration: 12.450s`).
* **Duration Selection Drag:** Left-clicking and dragging horizontally inside this top duration bar defines a highlighted selection overlay window. This range indicator automatically projects down into the waveform lane underneath.
---
## 7. Bottom Transport Panel
A prominent master transport toolbar occupies the bottom row layout of the sub-tab layout to manage audio playback monitoring:
```text
+───────────────────────────────────────────────────────────────────────────+
| [Back to Start] [Play] [Pause] [Stop] | Loop Sequence: [X] |
+───────────────────────────────────────────────────────────────────────────+
```
* **Back to Start:** Instantly updates the regional playhead time parameter back to the absolute starting point ($t = 0.0\text{ s}$).
* **Play / Pause / Stop:** Drives regional audio engine playback loops restricted entirely to the data buffers allocated inside the current sub-tab workspace.
* **Loop Toggle:** Toggles continuous cycle loops over the highlighted section or the whole clip.
---
## 8. Volume Automation Envelope (Pen Tool)
This advanced timeline automation layer allows audio designers to draw custom gain curves over the background waveform graphics.
```text
VOLUME AUTOMATION ENVELOPE (PEN TOOL)
+3 dB ──────────────────────────────────────────────────────────────
\ Node 1 Node 3
\ ○ ○
0 dB ───\────/─\─────────────────────────────────────/─\─────────── (0 dB Unity Gain Axis)
\ / \ / \
\/ \ / \
○ \_______________________________/ \________
Node 2 Node 4
-30 dB ──────────────────────────────────────────────────────────────
|<─────────────────── Horizontal Axis (Time) ─────────────────────>|
```
### 8.1. Pen Tool Interaction Mechanics
* **Activation:** Clicking the designated Pen Tool icon in the control panel modifies the pointer device presentation into a drawing crosshair or pencil graphic.
* **Envelope Initialization:** Activating the Pen Tool generates a solid horizontal neon green line representing $0\text{ dB}$ (Unity Gain) across the track workspace, acting as the baseline master axis.
* **Drawing Automation Curves:**
* Left-clicking anywhere along this line creates an adjustable anchor point (**Control Node**).
* Dragging an initialized control node upward increases signal amplitude (up to a maximal ceiling boundary of $+3\text{ dB}$).
* Dragging a control node downward reduces signal amplitude (down to a lower attenuation floor of $-30\text{ dB}$).
* The graphics framework automatically updates straight vector paths between sequential nodes utilizing simple linear interpolation.
### 8.2. DSP Volume Envelope Math
Given two chronologically adjacent drawn points $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the targeted instantaneous decibel gain variable $V_{\text{dB}}(t)$ at an arbitrary time index $t$ ($t_1 \le t \le t_2$) matches the following linear equation:
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
This decibel value must be translated into a standard linear gain scalar coefficient $G_{\text{linear}}(t)$ to multiply it into the core audio sample stream values:
$$G_{\text{linear}}(t) = 10^{\frac{V_{\text{dB}}(t)}{20}}$$
$$x_{\text{automation}}[n] = x[n] \cdot G_{\text{linear}}\left( \frac{n}{\text{Sample Rate}} \right)$$
---
## 9. Python Porting Manual (Docker Server Platform)
When translating these graphical volume automation envelope features to a desktop PyQt6 interface or an asynchronous Celery Docker worker pipeline, the standard scientific function `numpy.interp` handles array vector scaling processing loops:
```python
import numpy as np
def apply_volume_automation_envelope(y: np.ndarray, sr: int, nodes: list) -> np.ndarray:
"""
Applies a user-drawn volume automation envelope onto an acoustic signal NumPy array.
nodes: A list of point dictionaries, e.g., [{"time": 0.0, "db": 0.0}, {"time": 2.5, "db": -12.0}, ...]
"""
if not nodes:
return y
# Sort envelope nodes chronologically by time axis
nodes = sorted(nodes, key=lambda x: x["time"])
# 1. Map node variables into distinct coordinates arrays
node_times = np.array([node["time"] for node in nodes])
node_dbs = np.array([node["db"] for node in nodes])
# Hard-clamp boundary constraints matching the operational floor [-30.0dB, +3.0dB]
node_dbs = np.clip(node_dbs, -30.0, 3.0)
# 2. Evaluate absolute timeline timestamps for every index position inside the signal array
total_samples = len(y)
sample_times = np.arange(total_samples) / sr
# 3. Linearly interpolate localized decibel thresholds across every single sample step
interpolated_dbs = np.interp(sample_times, node_times, node_dbs, left=node_dbs[0], right=node_dbs[-1])
# 4. Map logarithmic values into standard linear gain scale arrays
linear_gains = 10.0 ** (interpolated_dbs / 20.0)
# 5. Multiply the raw amplitude vector array by the linear gain modifier mask
return y * linear_gains
```
14_FIX_SUBTAB.md
-283
View File
@@ -1,283 +0,0 @@
# Technical Specification: Advanced Editing Toolset & Graph-Based Continuous Waveform Painting on Sub-Tab
This document defines the interactive layout design, the configuration of the toolbar button arrays, and the signal processing routines for compiling a Graph-based Continuous Waveform graph optimized for the microscopic viewports inside the isolated temporary document workspace (Sub-tab), referencing the structural paradigms of `image_5ec2e5.png` and `image_5ec363.png`.
---
## 1. Target Selection Scope
The toolset within the Sub-tab environment supports two target operational boundaries:
* **Global Clip:** When no specific timeline selection highlighted mask is present, all active DSP effects apply uniformly across the entire length of the extracted Audio Clip.
* **Selected Range:** When an explicit timeline segment $[T_{\text{start}}, T_{\text{end}}]$ is highlighted by the user, DSP routines calculate changes exclusively inside those boundaries. Splice junctions automatically compute crossfades to mitigate transient click/pop anomalies.
---
## 2. Ruler-Based Tools
These utilities display as intuitive, linear slider scales (Sliders/Rulers) embedded in the top toolbar row:
```text
[ Normalize: |======o======| 0 dB ] [ Gain: |====o====| +3 dB ] [ Pitch: |==o==| -2 Semi ]
```
### 2.1. Peak Normalization
* **UI Layout:** A slide scale control allowing users to configure target amplitude thresholds variable from $-12\text{ dBFS}$ down to $0\text{ dBFS}$.
* **DSP Math Algorithm:** Locate the maximum absolute peak amplitude value $A_{\text{max}}$ within the targeted area, then multiply all active samples by a static scalar gain multiplier $G$:
$$G = \frac{10^{\frac{\text{Target\_dB}}{20}}}{A_{\text{max}}}$$
### 2.2. Volume Up / Down (Quick Gain)
* **UI Layout:** A linear sliding ruler modulating the overall absolute gain structure of the focused segment.
* **Operational Range:** Adjustable from $-\infty\text{ dB}$ (complete mute attenuation) up to $+12\text{ dB}$ of linear amplification.
### 2.3. Pitch Shifting
* **UI Layout:** A calibrated slider modifying the project's fundamental frequencies discrete in semitones or cents.
* **Operational Range:** Boundaries map from $-12\text{ semitones}$ (one octave down) to $+12\text{ semitones}$ (one octave up).
* **DSP Engine Routine:** Employs a spectral Phase Vocoder to shift frequencies without affecting the physical, real-time duration layout of the segment.
---
## 3. Graph-Based Fades
Fading curves overlay graphically directly onto the highlighted waveform canvas region, enabling precise boundary amplitude adjustments:
```text
Linear Fade-In Exponential Fade-Out
+───────────────────────────+ +───────────────────────────+
| /███████████████| |███████████\ |
| / ███████████████| |███████████ \ |
| / ███████████████| |███████████ \___ |
| / ███████████████| |███████████ \______|
+───────────────────────────+ +───────────────────────────+
|<──────── Fade-In ────────>| |<─────── Fade-Out ────────>|
```
* **Fade-In:** Multiplies an ascending amplitude ramp from $0.0$ to $1.0$ at the starting index profile of the selection region. Users can toggle seamlessly between **Linear** or **Exponential** curves to achieve a smoother, more psychoacoustically natural volume build-up.
* **Fade-Out:** Multiplies a descending amplitude decay ramp from $1.0$ down to $0.0$ at the trailing boundary edge of the selection range.
---
## 4. Ruler Percentage Stretch Tool
A dedicated percentage metric scale control (`Ruler %`) sitting on the control toolbar dictates time-stretching and playback velocity parameters:
```text
[ Speed Stretch %: |========o========| 100% (Native) ] -> Range: 50% - 200%
```
* **Interaction Mapping:** Users drag the percentage slider node or hold down the `Alt` key and drag the rightmost boundary edge of the clip along the horizontal axis to change this scale metric.
* **Sync Formula:** Let $D$ map to the unscaled native duration value, and $D'$ map to the target modified duration footprint. The resulting structural playback speed ratio percentage ($S$) is given by:
$$S = \frac{D}{D'} \times 100\%$$
---
## 5. Continuous Graph-Based Waveform Painting & Microscopic Viewports
The waveform graph inside the Sub-tab is compiled as a unified, continuous line vector (Continuous Line Graph) that flows seamlessly along the timeline axis, mapping the literal physical phase displacements of the underlying audio signal.
### 5.1. Logarithmic Amplitude Axis Grid Layout
Following the professional paradigm established in `image_5ec2e5.png`, the waveform painting canvas is divided by a symmetrical layout grid reflecting both positive and negative polarity limits of the central horizontal axis:
```text
+6.0 dB ───────────────────────────────────────────────────────────────────
~ ~ ~ ~ ~ ~ ~ ~ (Sub-division Grid Line) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
-6.0 dB ───────────────────────────────────────────────────────────────────
\ / \ / \
-Inf dB ─○───────────/───────────────○───────────/───────────────○───────── (Zero-Line Axis)
\ / \ / \
-6.0 dB ───────────────────────────────────────────────────────────────────
+6.0 dB ───────────────────────────────────────────────────────────────────
```
* **Visual Bounding Thresholds:**
* **Central Zero Axis (-Inf. dB):** Maps the absolute baseline $0\text{V}$ electrical reference (complete absence of audio signal / absolute silence).
* **Symmetrical Decibel Grids:** Project accurate scale metrics tracking normalized peak levels (the inner $-6.0\text{ dB}$ sub-grid marks a $50.1\%$ amplitude ceiling, while the outermost physical frame boundary aligns to $+6.0\text{ dB}$ or $0\text{ dBFS}$).
### 5.2. Standard Workspace View vs. Ultra Zoom Viewport Scaling
The drawing engine dynamically hot-swaps its rendering calculations (Rendering Routine) depending on the active pixel compression metric $Z$ (pixels/second):
* **Standard View Mode ($Z < 500\text{ pixels/second}$):** The system deploys a structural peak compression layout algorithm (**Peak Waveform**—as referenced in `image_5ec2e5.png`). It connects the maximum absolute upper peak bounding indices (Max) with the lower minimum value ranges (Min) passing through a common pixel column into a unified vector line, generating an organic, aliases-free continuous waveform silhouette.
* **Micro Viewport Zoom-In ($Z \ge 500\text{ pixels/second}$—as referenced in `image_5ec363.png`):** Once viewport stretching scales past this threshold, the framework transitions into a **Single Continuous Sine Polyline** loop. Chronologically sequential acoustic sample addresses ($x[i]$, $x[i+1]$) map as discrete vector coordinate indices bound together by thin lines (using sharp smooth polyline vectors or linear/cubic spline interpolation loops), charting pristine, individual sinusoidal phases explicitly.
### 5.3. Zero-Crossing Alignment within Ultra Zoom Viewports
When performing rapid cursor tracking edits (Scrub/Drag Selection), the alignment routine locks the selection boundary marker coordinates onto the nearest baseline sample offset exhibiting a complete algebraic phase conversion (sign inversion):
$$x[i] \cdot x[i+1] \le 0$$
---
## 6. Top Duration Timeline
Directly above the isolated sub-tab waveform canvas lane, a dedicated horizontal measuring ruler tracks clip timing data:
```text
| 0:00.000 | 0:01.000 | 0:02.000 | 0:03.000 | 0:04.000 (Duration: 4.152s)
+───────────────────────────────────────────────────────────────────────────────────────+
| [==================== VÙNG QUÉT CHỌN (RANGE SELECTION) ====================] |
+───────────────────────────────────────────────────────────────────────────────────────+
```
* **Total Duration Monitoring:** Renders the absolute, precise time extent of the isolated audio block in the right-hand corner of the timeline ruler layout (e.g., `Duration: 12.450s`).
* **Duration Selection Drag:** Left-clicking and dragging horizontally inside this top duration bar defines a highlighted selection overlay window. This range indicator automatically projects down into the waveform lane underneath.
---
## 7. Bottom Transport Panel & Master Tools
A comprehensive control framework containing expanded navigation buttons and deep session processing controls anchors the bottom row of the sub-tab environment, matching the layout structure in `image_5ec2e5.png`:
```text
+─────────────────────────────────────────────────────────────────────────────────────────────+
| [● Rec] [◀◀ Back] [▶ Play] [|| Pause] [■ Stop] | Rate: |====o====| 0.00 | Loop: [X] |
|---------------------------------------------------------------------------------------------|
| [Volume Pencil Tool] [AI Analysis Tool] | Active Asset: linh_ngua_powerup.wav |
+─────────────────────────────────────────────────────────────────────────────────────────────+
```
### 7.1. Functional Mapping Matrix:
* **Record (● Red Indicator):** Drives live microphone capture sequences targeted straight into the isolated sub-tab data matrix.
* **Back (◀◀ Rewind):** Resets the timeline playhead position index back to the absolute starting point ($t = 0.0\text{ s}$).
* **Play / Pause / Stop:** Coordinates low-latency runtime audio execution tracking locked onto the sub-tab's RAM cache blocks.
* **Rate Slider:** Adjusts the global monitoring playback pitch speed metrics in real time without overwriting source asset length (calibrated step ranges variable from `-1.00` scaling up to `+1.00`).
* **Loop Toggle:** Toggles continuous cycle loops over the highlighted section or the whole clip.
* **Volume Pencil Tool:** Engages the drawing framework to map point nodes for automated amplitude envelopes.
* **AI Analysis Tool:** Instructs the dockerized engine to evaluate rhythmic transient markers and pitch tracking grids.
---
## 8. Volume Automation Envelope (Pen Tool)
This advanced timeline automation layer allows audio designers to draw custom gain curves over the background waveform graphics.
```text
VOLUME AUTOMATION ENVELOPE (PEN TOOL)
+3 dB ──────────────────────────────────────────────────────────────
\ Node 1 Node 3
\ ○ ○
0 dB ───\────/─\─────────────────────────────────────/─\─────────── (0 dB Unity Gain Axis)
\ / \ / \
\/ \ / \
○ \_______________________________/ \________
Node 2 Node 4
-30 dB ──────────────────────────────────────────────────────────────
|<─────────────────── Horizontal Axis (Time) ─────────────────────>|
```
### 8.1. Pen Tool Interaction Mechanics
* **Activation:** Clicking the designated Pencil Tool icon in the control panel modifies the pointer device presentation into a pencil graphic.
* **Envelope Initialization:** Activating the Pen Tool generates a solid horizontal neon green line representing $0\text{ dB}$ (Unity Gain) across the track workspace, acting as the baseline master axis.
* **Drawing Automation Curves:**
* Left-clicking anywhere along this line creates an adjustable anchor point (**Control Node**).
* Dragging an initialized control node upward increases signal amplitude (up to a maximal ceiling boundary of $+3\text{ dB}$).
* Dragging a control node downward reduces signal amplitude (down to a lower attenuation floor of $-30\text{ dB}$).
* The graphics framework automatically updates straight vector paths between sequential nodes utilizing simple linear interpolation.
### 8.2. DSP Volume Envelope Math
Given two chronologically adjacent drawn points $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the targeted instantaneous decibel gain variable $V_{\text{dB}}(t)$ at an arbitrary time index $t$ ($t_1 \le t \le t_2$) matches the following linear equation:
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
This decibel value must be translated into a standard linear gain scalar coefficient $G_{\text{linear}}(t)$ to multiply it into the core audio sample stream values:
$$G_{\text{linear}}(t) = 10^{\frac{\text{V}_{\text{dB}}(t)}{20}}$$
$$x_{\text{automation}}[n] = x[n] \cdot G_{\text{linear}}\left( \frac{n}{\text{Sample Rate}} \right)$$
---
## 9. Porting Guidelines for Python Desktop Layouts (PyQt6 QPainter Context)
When translating the polyline vector engine and the symmetrical decibel gridding lines into a containerized desktop application using the native `QPainter` canvas inside PyQt6, leveraging a structured `QPainterPath` prevents rendering lag when mapping high-density signal segments:
```python
# [PYTHON PORTING BLUEPRINT] - Continuous Polyline Waveform Rendering via QPainterPath
from PyQt6.QtGui import QPainter, QPainterPath, QPen, QColor
from PyQt6.QtCore import QPointF, Qt
import numpy as np
def paint_continuous_waveform_path(painter: QPainter, rect_width: int, rect_height: int, y: np.ndarray, zoom_level: float):
"""
Renders a unified continuous single polyline path tracing absolute physical signal transitions.
y: A 1D NumPy float32 array tracking raw sample amplitudes bounded within [-1.0, 1.0].
zoom_level: The scale allocation mapping physical drawing pixels per second of audio data.
"""
if len(y) == 0:
return
painter.setRenderHint(QPainter.RenderHint.Antialiasing, True)
mid_y = rect_height / 2.0
# 1. Compile background Decibel reference grids (-6.0 dB, -Inf. dB, -6.0 dB)
grid_pen = QPen(QColor(45, 45, 45), 1, Qt.PenStyle.DashLine)
painter.setPen(grid_pen)
# A threshold of -6.0 dB maps approximately to an absolute scalar amplitude index of 0.501
y_6db_top = mid_y - (0.501 * (rect_height * 0.42))
y_6db_bottom = mid_y + (0.501 * (rect_height * 0.42))
painter.drawLine(0, int(y_6db_top), rect_width, int(y_6db_top))
painter.drawLine(0, int(y_6db_bottom), rect_width, int(y_6db_bottom))
# Paint the absolute Zero-Line horizontal center axis (-Inf. dB)
center_pen = QPen(QColor(60, 60, 60), 1, Qt.PenStyle.SolidLine)
painter.setPen(center_pen)
painter.drawLine(0, int(mid_y), rect_width, int(mid_y))
# 2. Initialize the Continuous Vector Polyline Route Layout Block
wave_path = QPainterPath()
wave_pen = QPen(QColor(100, 149, 237), 1.2, Qt.PenStyle.SolidLine) # Professional Cornflower Blue
painter.setPen(wave_pen)
# Map raw buffer indexes into structural coordinate pixels
start_point_set = False
for x_pixel in range(rect_width):
# Translate current canvas pixel offset back to timeline seconds metrics
time_at_pixel = x_pixel / zoom_level
# Calculate target array element offset
sample_index = int(time_at_pixel * 44100) # Assuming project sample rate baseline at 44.1kHz
if sample_index >= len(y):
break
amplitude = y[sample_index]
y_pixel = mid_y + (amplitude * (rect_height * 0.42))
if not start_point_set:
wave_path.moveTo(float(x_pixel), y_pixel)
start_point_set = True
else:
wave_path.lineTo(float(x_pixel), y_pixel)
# Draw the continuous vector polyline overlay onto the viewport canvas
painter.drawPath(wave_path)
```
15_GRAPH_EDIT.md
-322
View File
@@ -1,322 +0,0 @@
# Technical Specification: Implementing Volume, Fades & Panning Envelope Arrays on Audio Signals
This document defines the mathematical models, data flow diagrams (Audio Node Graph), and execution source code required to apply interactive graphical curves (Volume Automation, Fades, and Panning Automation) into the real-time digital signal processing pipeline on the Frontend and offline file export rendering on the Dockerized Python Backend.
---
## 1. Multi-stage Audio Node Graph
To simultaneously compute all three graphical configurations over the audio stream without precipitating phase cancellation or signal latency anomalies, the environment builds an explicit downstream node connection graph:
```text
┌─────────────────────────┐
│ AudioBufferSourceNode │ --> Streams the native original raw buffer array
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ GainNode (Automation) │ --> Modulates Volume dynamically via multi-point automation arrays
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ StereoPannerNode │ --> Transposes the Stereo Image (L/R Balance Automation trajectory)
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ GainNode (Fades) │ --> Multiplies bounding Fade-In and Fade-Out curves
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ AudioContext.destination│ --> Routes processed signal to hardware device outputs (Speakers/Headphones)
└─────────────────────────┘
```
---
## 2. Mathematical Formulations for Modulators
### 2.1. Multi-Point Volume Automation Curves
The vertical coordinate axis $Y$ of the volume points plots decibel thresholds bounded from $-30\text{ dB}$ to $+3\text{ dB}$. Prior to applying multipliers onto the signal, the logarithmic values must be translated into a standard linear scalar gain coefficient $G_{\text{linear}}$:
$$G_{\text{linear}}(t) = 10^{\frac{V_{\text{dB}}(t)}{20}}$$
At an arbitrary timeline timestamp $t$ residing between two chronologically adjacent control nodes $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the target volume attenuation value is computed via standard linear interpolation:
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
### 2.2. Constant-Power Stereo Panning
To ensure that when a user shifts the audio image toward the Left ($L$) or Right ($R$) perimeter channels, the cumulative output sound energy emitted by the drivers does not collapse (avoiding a volume drop at the absolute horizontal center axis—known as the *Center Dip* anomaly), the system implements the **Constant-Power Panning Law**.
Let $p(t) \in [-1.0, 1.0]$ map to the explicit panning index at timestamp $t$ (where $-1.0$ represents a hard-left channel displacement, $0.0$ marks absolute center, and $+1.0$ dictates a hard-right channel boundary).
Convert the raw linear panning factor $p(t)$ into a circular panning sweep angle coordinate $\theta(t) \in [0, \pi/2]$:
$$\theta(t) = \frac{p(t) + 1}{2} \cdot \frac{\pi}{2}$$
Calculate the independent amplitude scalar gains for the Left channel ($g_L$) and the Right channel ($g_R$) elements:
$$g_L(t) = \cos(\theta(t)), \quad g_R(t) = \sin(\theta(t))$$
*Mathematical Proof:* The total sound field energy remains perfectly preserved under all operational transformations because:
$$g_L(t)^2 + g_R(t)^2 = \cos^2(\theta(t)) + \sin^2(\theta(t)) = 1.0$$
### 2.3. Fade Curves (Fade-In & Fade-Out)
Fading shapes are driven by a trigonometric Cosine equation framework to build organic, smooth amplitude transitions at the structural boundary zones of the audio asset:
* **Fade-In Curve** (Across an introductory duration window of $L_{\text{fade}}$ seconds):
$$f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade}}} \right)}{2} \quad \text{for } 0 \le t < L_{\text{fade}}$$
* **Fade-Out Curve** (Across a trailing termination window of $L_{\text{fade}}$ seconds):
$$f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade}})}{L_{\text{fade}}} \right)}{2} \quad \text{for } T_{\text{max}} - L_{\text{fade}} \le t \le T_{\text{max}}$$
---
## 3. Client-Side Runtime Integration (Web Audio API - Live Playback Modulator)
This JavaScript module sets up the physical Web Audio node graphs and automates parameters directly matching the real-time audio thread clocks:
```javascript
/**
* Configures a real-time audio node processing graph with parameter automation.
* @param {AudioContext} audioCtx - The active Web Audio runtime context instance.
* @param {AudioBuffer} audioBuffer - Decoded original raw target audio source asset.
* @param {number} startTime - Global time position index marking where playback initiates (seconds).
* @param {Array} volumeNodes - Automation point layout maps: [{time: 0.5, db: -3.0}, ...].
* @param {Array} panningNodes - Panning position layout maps: [{time: 1.2, pan: -0.5}, ...].
* @param {object} fadeConfig - Bounding fade time constants: {fadeInLen: 0.5, fadeOutLen: 0.8}.
*/
function playTrackWithAutomation(audioCtx, audioBuffer, startTime, volumeNodes, panningNodes, fadeConfig) {
// 1. Instantiate the Global Audio Source Buffer Node
const sourceNode = audioCtx.createBufferSource();
sourceNode.buffer = audioBuffer;
// 2. Instantiate the Gain Node managing Volume Automation tracking loops
const volumeGainNode = audioCtx.createGain();
// Establish baseline default state variables at Unity Gain (0 dB)
volumeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
// Map timeline automations for custom Volume Node trajectories
if (volumeNodes && volumeNodes.length > 0) {
// Purge legacy scheduled values to safely overwrite parameters
volumeGainNode.gain.cancelScheduledValues(audioCtx.currentTime);
volumeNodes.forEach(node => {
const timeOffset = startTime + node.time;
const linearGain = Math.pow(10, node.db / 20); // Map logarithmic dB thresholds to linear multipliers
volumeGainNode.gain.linearRampToValueAtTime(linearGain, audioCtx.currentTime + node.time);
});
}
// 3. Instantiate the StereoPannerNode for Panning Automation structures
const pannerNode = audioCtx.createStereoPanner();
pannerNode.pan.setValueAtTime(0.0, audioCtx.currentTime); // Standard initialization locked at Center
// Map timeline automations for Panning Node trajectories
if (panningNodes && panningNodes.length > 0) {
pannerNode.pan.cancelScheduledValues(audioCtx.currentTime);
panningNodes.forEach(node => {
// Enforce rigid clipping bounds to keep panning factors inside [-1.0, 1.0]
const clampedPan = Math.max(-1.0, Math.min(1.0, node.pan));
pannerNode.pan.linearRampToValueAtTime(clampedPan, audioCtx.currentTime + node.time);
});
}
// 4. Instantiate the Gain Node dedicated to boundary Fades
const fadeGainNode = audioCtx.createGain();
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
const duration = audioBuffer.duration;
// Calculate and schedule introductory Fade-In values
if (fadeConfig.fadeInLen > 0) {
fadeGainNode.gain.setValueAtTime(0.0, audioCtx.currentTime);
fadeGainNode.gain.linearRampToValueAtTime(1.0, audioCtx.currentTime + fadeConfig.fadeInLen);
}
// Calculate and schedule terminating Fade-Out values
if (fadeConfig.fadeOutLen > 0) {
const fadeOutStart = duration - fadeConfig.fadeOutLen;
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime + fadeOutStart);
fadeGainNode.gain.linearRampToValueAtTime(0.0, audioCtx.currentTime + duration);
}
// 5. Connect the physical downstream structural audio pipeline
sourceNode.connect(volumeGainNode);
volumeGainNode.connect(pannerNode);
pannerNode.connect(fadeGainNode);
fadeGainNode.connect(audioCtx.destination);
// 6. Drive hardware execution loops
sourceNode.start(0);
return { sourceNode, volumeGainNode, pannerNode, fadeGainNode };
}
```
---
## 4. Server-Side Execution Engine (Dockerized Python Engine - NumPy Processing)
When a user triggers an *Apply* action or an offline *Export* script, the frontend dispatches serialized JSON configuration models down to the Python backend framework. The signal processing architecture uses high-efficiency vectorized loops inside NumPy to multiply envelope modulators straight onto raw multi-channel float data:
```python
import numpy as np
class DSPAudioModulator:
@staticmethod
def apply_automation_and_panning(
y_raw: np.ndarray,
sr: int,
volume_points: list, # [{"time": 0.5, "db": -6.0}, ...]
panning_points: list, # [{"time": 1.0, "pan": -0.7}, ...]
fade_in_sec: float = 0.0,
fade_out_sec: float = 0.0
) -> np.ndarray:
"""
Applies multi-point volume envelopes, constant-power panning, and trigonometric fades
directly onto a 1D (Mono) or 2D (Stereo) acoustic NumPy signal array.
Input: y_raw maps to the raw sound array (Mono/Stereo matrix bounded inside [-1.0, 1.0]).
Output: y_processed yields a 2D interleaved Stereo NumPy array (2, N) with baked modulations.
"""
total_samples = y_raw.shape[-1] if len(y_raw.shape) > 1 else len(y_raw)
duration_sec = total_samples / sr
# 1. Guarantee Stereo geometry dimensions (2 discrete channels) for Panning operations
if len(y_raw.shape) == 1:
# For Mono arrays, clone sample metrics symmetrically to Left/Right matrices
y_stereo = np.vstack((y_raw, y_raw))
else:
y_stereo = np.copy(y_raw)
# 2. Allocate Envelope Mask arrays matching total track samples limits
volume_envelope = np.ones(total_samples, dtype=np.float32)
pan_envelope = np.zeros(total_samples, dtype=np.float32) # Default initialization: Center (0.0)
# 3. Compile the Volume Envelope using linear interpolation bounds across nodes
if volume_points and len(volume_points) > 0:
# Enforce strict chronological sorting down the timeline axis
points = sorted(volume_points, key=lambda x: x["time"])
# Pad introductory bounds if the initial point coordinate sits past t = 0.0s
if points[0]["time"] > 0:
first_gain = 10.0 ** (points[0]["db"] / 20.0)
idx_end = int(points[0]["time"] * sr)
volume_envelope[:idx_end] = first_gain
for i in range(len(points) - 1):
p1, p2 = points[i], points[i+1]
idx_start = int(p1["time"] * sr)
idx_end = int(p2["time"] * sr)
gain_start = 10.0 ** (p1["db"] / 20.0)
gain_end = 10.0 ** (p2["db"] / 20.0)
# Linearly interpolate vector increments between adjacent anchor positions
volume_envelope[idx_start:idx_end] = np.linspace(gain_start, gain_end, idx_end - idx_start)
# Pad trailing bounds from the final milestone extending through end-of-file
if points[-1]["time"] < duration_sec:
last_gain = 10.0 ** (points[-1]["db"] / 20.0)
idx_start = int(points[-1]["time"] * sr)
volume_envelope[idx_start:] = last_gain
# 4. Compile the Panning Envelope using linear interpolation bounds across nodes
if panning_points and len(panning_points) > 0:
points = sorted(panning_points, key=lambda x: x["time"])
if points[0]["time"] > 0:
pan_envelope[:int(points[0]["time"] * sr)] = points[0]["pan"]
for i in range(len(points) - 1):
p1, p2 = points[i], points[i+1]
idx_start = int(p1["time"] * sr)
idx_end = int(p2["time"] * sr)
pan_envelope[idx_start:idx_end] = np.linspace(p1["pan"], p2["pan"], idx_end - idx_start)
if points[-1]["time"] < duration_sec:
pan_envelope[int(points[-1]["time"] * sr):] = points[-1]["pan"]
# 5. Apply Trigonometric Cosine Fade-In / Fade-Out functions onto the Volume Envelope mask
if fade_in_sec > 0:
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
x_fade = np.linspace(0.0, np.pi, fade_in_samples)
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
volume_envelope[:fade_in_samples] *= cosine_ramp
if fade_out_sec > 0:
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
x_fade = np.linspace(0.0, np.pi, fade_out_samples)
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
volume_envelope[-fade_out_samples:] *= cosine_ramp
# 6. Bake Volume Envelope matrices onto the Left and Right discrete audio paths
y_stereo[0, :] *= volume_envelope
y_stereo[1, :] *= volume_envelope
# 7. Apply Constant-Power Stereo Panning allocations
# Map panning metrics range [-1.0, 1.0] onto angular radians field array [0, pi/2]
theta_envelope = ((pan_envelope + 1.0) / 2.0) * (np.pi / 2.0)
# Evaluate localized amplitude coefficients for physical channels split
gain_left = np.cos(theta_envelope)
gain_right = np.sin(theta_envelope)
# Multiply scaling factors directly across corresponding discrete matrices
y_stereo[0, :] *= gain_left
y_stereo[1, :] *= gain_right
return y_stereo
```
---
## 5. Viewport Coordinate Mapping & Data Serialization Protocols
As users drag and adjust coordinate anchors over the visual drawing Canvas, mouse-event pixel coordinates are continuously calculated and mapped into absolute physical values to preserve data matching between frontend layouts and backend signal arrays:
```text
[ GRAPH CANVAS VIEWPORT COORDINATES ] [ REAL-WORLD SYSTEM PHENOMENA VALUES ]
x (pixel) ─────────────────────────────────────► Timeline position t (seconds) = x / zoom_level
y (pixel) ─ (Volume: Center axis maps to 0dB) ─► db = ( (h - y) / h_half ) * range_db
y (pixel) ─ (Panning: Center axis maps to 0) ──► pan = ( (h_half - y) / h_half ) -> Bounded [-1.0, 1.0]
```
### Serialized API Data Transfer Model (Standard JSON Package Syntax)
```json
{
"track_id": "1",
"fades": {
"fade_in_sec": 0.500,
"fade_out_sec": 1.200
},
"volume_automation": [
{ "time": 0.000, "db": 0.0 },
{ "time": 1.450, "db": 3.0 },
{ "time": 3.820, "db": -12.5 },
{ "time": 6.000, "db": 0.0 }
],
"panning_automation": [
{ "time": 0.000, "pan": 0.0 },
{ "time": 2.100, "pan": -0.8 },
{ "time": 4.500, "pan": 0.8 },
{ "time": 6.000, "pan": 0.0 }
]
}
```
16_FIX_GRAPH.md
-167
View File
@@ -1,167 +0,0 @@
# Technical Specification: Mapping Matrix & Interactive Graphical Rendering Algorithms
This document defines the interactive real-time non-linear curves system showcased.
---
## 1. UI Element Mapping Matrix
To upgrade the interface workflow from Image 1 to Image 2, a 1-to-1 mapping of graphical components is executed based on the structural breakdown below:
| --- | --- | --- |
| **Horizontal blue bar at the top** (Contains a node chain representing the default $0\text{ dB}$ volume level) | Multi-point peach-colored automation spine (**Automation Spline**) overlaying the waveform viewport area. | * **Double-click** anywhere along the spline to generate a new control node.<br>
<br>
<br>* **Click & Drag** a node vertically to scale Volume (Gain), or horizontally to adjust its chronological time position. |
| **"FI" text label** in the upper-left corner | Deep red arched **Fade-In Bezier Curve** smoothing the volume transition from $0\%$ up to $100\%$. | * **Click & Hold** the "FI" handle and drag rightward to increase the target Fade-In length ($L_{\text{fade\_in}}$). This action automatically projects a smooth curve overlay on top of the waveform graphic. |
| **"FO" text label** in the upper-right corner | Deep red arched **Fade-Out Bezier Curve** decaying the volume envelope from $100\%$ down to $0\%$ at the end of the clip boundary. | * **Click & Hold** the "FO" handle and drag leftward to increase the target Fade-Out length ($L_{\text{fade\_out}}$). The inverse curve automatically stretches or compresses based on the active dragging cursor coordinates. |
| **"VOL" button** in the lower-right corner | **Graphical Envelope Mode Switcher** (Toggles automation layer matrices). | * **Click** to hot-swap between multiple interactive graphs: Volume (VOL) (peach curve), Panning (PAN) (L/R Stereo Image automation trajectory), or FX Send grids. |
---
## 2. Non-Linear Graphical Curve Rendering Algorithms (Image 2)
### 2.1. Multi-Point Volume Automation Curves (Smooth Monotone Spline)
To ensure the interpolating paths connecting the peach-colored nodes in Image 2 are curved smoothly without generating sharp angular peaks, the framework runs a **Monotone Cubic Hermite Spline** interpolation algorithm.
Given two chronologically consecutive control nodes $P_a(x_a, y_a)$ and $P_b(x_b, y_b)$, an arbitrary absolute timeline position $x$ is normalized into a relative horizontal index interval $t$:
$$t = \frac{x - x_a}{x_b - x_a} \quad (0 \le t \le 1)$$
The target interpolated amplitude value $y(x)$ at position $x$ is evaluated using the cubic polynomial equation:
$$y(x) = (2t^3 - 3t^2 + 1)y_a + (t^3 - 2t^2 + t)h \cdot m_a + (-2t^3 + 3t^2)y_b + (t^3 - t^2)h \cdot m_b$$
Where: $h = x_b - x_a$, and $m_a, m_b$ correspond to the localized slopes (tangents) computed from adjacent surrounding node coordinates. This constraint ensures strict monotonicity to eliminate graphical or mathematical overshoot anomalies.
### 2.2. Fade Curve Contours (Fade-In & Fade-Out)
The physical curvature profile of the two deep red envelopes in Image 2 is evaluated using a trigonometric Cosine S-Curve or a 3rd-order Cubic Bezier equation framework:
* **Trigonometric Cosine Fade-In Curve** (Across a duration bound of $L_{\text{fade\_in}}$ seconds):
$$f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade\_in}}} \right)}{2} \quad \left( 0 \le t \le L_{\text{fade\_in}} \right)$$
* **Trigonometric Cosine Fade-Out Curve** (Across a trailing termination window of $L_{\text{fade\_out}}$ seconds):
$$f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade\_out}})}{L_{\text{fade\_out}}} \right)}{2} \quad \left( T_{\text{max}} - L_{\text{fade\_out}} \le t \le T_{\text{max}} \right)$$
---
## 3. Client-Side Runtime Integration (HTML5 Canvas Engine)
To make the static canvas layer from Image 1 respond fluidly to drag gestures like the interactive system in Image 2, the painting routine segregates graphic elements into distinct presentation layers, driven inside a low-latency `requestAnimationFrame` render loop:
```javascript
/**
* Renders non-linear Fade-In and Fade-Out curves over the Waveform canvas viewport.
* @param {CanvasRenderingContext2D} ctx - Target 2D rendering canvas context.
* @param {number} width - Total physical viewport tracking pixel width.
* @param {number} height - Total physical viewport tracking pixel height.
* @param {number} fadeInSec - Bounding target Fade-In duration in seconds.
* @param {number} fadeOutSec - Bounding target Fade-Out duration in seconds.
* @param {number} zoom - Current layout pixel compression scaling factor (pixels/second).
*/
function drawFadeCurves(ctx, width, height, fadeInSec, fadeOutSec, zoom) {
const fadeInWidth = fadeInSec * zoom;
const fadeOutWidth = fadeOutSec * zoom;
const midY = height / 2;
ctx.strokeStyle = '#800000'; // Professional dark deep red hue theme
ctx.lineWidth = 1.8;
// 1. Compile the non-linear Fade-In curve polyline
if (fadeInWidth > 0) {
ctx.beginPath();
for (let x = 0; x <= fadeInWidth; x++) {
const ratio = x / fadeInWidth;
// Apply trigonometric cosine to map curved vertical y-coordinates
const amp = (1 - Math.cos(Math.PI * ratio)) / 2;
const y = height - (amp * height); // Apply envelope tracking from bottom to top
if (x === 0) ctx.moveTo(x, height);
else ctx.lineTo(x, y);
}
ctx.stroke();
}
// 2. Compile the non-linear Fade-Out curve polyline
if (fadeOutWidth > 0) {
ctx.beginPath();
const startX = width - fadeOutWidth;
for (let x = 0; x <= fadeOutWidth; x++) {
const ratio = x / fadeOutWidth;
const amp = (1 + Math.cos(Math.PI * ratio)) / 2;
const y = height - (amp * height);
if (x === 0) ctx.moveTo(startX + x, 0);
else ctx.lineTo(startX + x, y);
}
ctx.stroke();
}
}
```
---
## 4. Server-Side DSP Automation Processing (Dockerized Python Engine)
When an operator commits tracking edits via the Frontend client layer, the mapped coordinates are encoded as a serialized JSON package and transferred down to the FastAPI server gateway. The Python core layer runs performance-optimized, vectorized array loops inside NumPy to multiply envelope filters straight into the raw source data buffer matrices:
```python
import numpy as np
class DSPAutomationProcessor:
@staticmethod
def apply_curves_to_samples(
y: np.ndarray,
sr: int,
fade_in_sec: float,
fade_out_sec: float,
automation_points: list # [{"time": 0.5, "db": -3.0}, ...]
) -> np.ndarray:
"""
Bakes multi-point Volume Automation splines and non-linear fade curves
directly onto a raw acoustic sample NumPy array.
"""
total_samples = len(y)
duration_sec = total_samples / sr
# 1. Initialize the baseline Gain Envelope at Unity Gain (1.0 or 0 dB)
gain_envelope = np.ones(total_samples, dtype=np.float32)
# 2. Evaluate Volume Automation scaling paths (Peach-colored nodes in Image 2)
if automation_points and len(automation_points) > 0:
points = sorted(automation_points, key=lambda x: x["time"])
xp = [p["time"] for p in points]
fp = [10.0 ** (p["db"] / 20.0) for p in points] # Map decibel factors to linear scalars
# Linearly interpolate point values quickly across the full timeline width
times = np.linspace(0, duration_sec, total_samples)
gain_envelope = np.interp(times, xp, fp)
# 3. Multiply the introductory Fade-In envelope (Cosine transition mask at Image 2 boundary)
if fade_in_sec > 0:
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
x_fade = np.linspace(0, np.pi, fade_in_samples)
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
gain_envelope[:fade_in_samples] *= cosine_ramp
# 4. Multiply the trailing Fade-Out envelope (Cosine decay mask at Image 2 boundary)
if fade_out_sec > 0:
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
x_fade = np.linspace(0, np.pi, fade_out_samples)
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
gain_envelope[-fade_out_samples:] *= cosine_ramp
# 5. Execute vectorized element-wise multiplication into raw audio values
return y * gain_envelope
```
17_AI_SCAN.md
-277
View File
@@ -1,277 +0,0 @@
# Technical Specification: AI Loop Scanning System & Fade-Free Zero-Crossing Slicing
This document specifies the software architecture, digital signal processing (DSP) algorithms, and API design required to integrate AI-driven automated loop scanning and perfect, fade-free audio slicing (Zero-Crossing Aligned Slicing) without boundary transition effects (Fade-In/Fade-Out).
---
## 1. Feature 1: AI Loop Scan & Automated Marker Labeling
This feature allows users to quickly scan an audio track (driven by backend AI/DSP) to detect segments with the highest rhythmic or musical periodicity (e.g., drum loops, chord progressions, vocal loops) and automatically map both boundaries using the timeline marker system.
```text
AI LOOP SCAN PROCESSING FLOW
┌───────────────────┐ 1. Send File ID ┌────────────────────────┐
│ Frontend Client ├──────────────────────►│ Backend FastAPI Server │
│ (Click "AI Scan") │◄──────────────────────┤ (Celery Task Worker) │
└───────────────────┘ 4. Return timestamps└───────────┬────────────┘
▲ [t_start, t_end] │
│ ▼
│ 2. Analyze Audio Features
│ (Self-Similarity Matrix)
│ │
│ ▼
└───────── (Pin Markers automatically) ◄ 3. Snap to Zero-Crossing
```
### 1.1. Workflow
1. The user selects an audio track within the Main Session and clicks the *AI Scan* button on the AI Panel.
2. The frontend dispatches a request containing the track's `file_id` to the backend gateway.
3. The backend initiates an asynchronous Celery Task, leveraging the `librosa` acoustic processing library to extract spectral feature matrices (Chromagram/Mel-spectrogram) and search for target loop boundaries exhibiting the highest recurrence correlation.
4. Once the optimal loop region $[t_{\text{start}}, t_{\text{end}}]$ is calculated, the backend executes a Zero-Crossing Alignment routine to precisely shift both boundaries to the nearest index where the signal amplitude reaches exactly zero.
5. The processed absolute timestamps $[t'_{\text{start}}, t'_{\text{end}}]$ are returned to the client. The frontend dynamically instantiates and renders timeline markers pinned directly onto that track lane.
---
## 2. Feature 2: AI Analysis & AI Cut (Fade-Free)
When cutting an audio segment at arbitrary time markers, if a slice intersects a high-amplitude point (non-zero), the continuous physical phase of the waveform is abruptly broken (Jump discontinuity). This generates a sharp, vertical step in the amplitude waveform graph, translating mechanically into an audible, harsh popping or ticking artifact ("click" or "pop") through speakers.
Standard or basic DAW systems mitigate this issue by adding an ultra-short linear fade envelope (Fade-In/Fade-Out) spanning roughly $5\text{ ms} \rightarrow 10\text{ ms}$. However, this masking method dampens the physical attack phase (transients) of the sound field, which is severely destructive to sharp, high-impact hits such as kick drums or snares.
The perfect architecture is a **Fade-Free AI Cut**. It dynamically calculates the closest hardware zero-crossing indices—where the acoustic wave amplitude passes through the central horizontal timeline axis ($0\text{V}$ absolute silence)—and executes the audio slice precisely at those coordinates.
```text
WAVEFORM TIMELINE & FADE-FREE AI CUT PROCESS
Amplitude
▲
+1.0 ┼ / \ / \
│ / \ / \
│ User-defined/ \ / \ User-defined
│ selection marker \ / \ selection marker
0.0 ┼───────○─────────────○─────○─────────○───────► Time Axis
│ / \ / \ / \ / \
│ / \ / \ / \ / \
-1.0 ┼────/──────\──────/─────○─────\───/─────\─
▲
│ [ AI CUTS EXACTLY HERE ]
│ Amplitude = 0 (Sound is silent)
│ Absolute zero Click/Pop anomalies!
```
### 2.1. Workflow
1. The user left-clicks and drags a time selection window $[T_{\text{start}}, T_{\text{end}}]$ across the target Waveform Lane.
2. The user clicks **AI Analysis**: The backend calculates and shifts both bounding coordinates slightly to align with physical zero-crossing sample indices ($T'_{\text{start}}$ and $T'_{\text{end}}$), instantly refreshing the highlighted overlay on the screen viewport.
3. The user clicks **AI Cut**: The engine slices the raw binary sample stream from index $T'_{\text{start}}$ to $T'_{\text{end}}$ straight inside RAM, generates a new track row directly underneath, and drops the cut clip onto it. The asset remains un-rendered and pure, with absolutely no volume fade multi-stage nodes applied.
---
## 3. Mathematical Zero-Crossing Optimization Algorithm (DSP Math)
Let $x[n]$ represent a single-channel discrete sample array containing mono audio amplitudes ($0$ mapping to the left track lane channel). At the target sample index address $n_{\text{target}}$ derived from the user's raw timeline click event, the engine establishes a symmetrical boundary scanning window of size $W$ (typically set to a $50\text{ ms}$ horizontal time width):
$$n_{\text{start}} = n_{\text{target}} - \frac{W \cdot f_s}{2}, \quad n_{\text{end}} = n_{\text{target}} + \frac{W \cdot f_s}{2}$$
Where $f_s$ tracks the absolute project Sample Rate hardware clock (e.g., $44100\text{ Hz}$).
### 3.1. Physical Phase Inversion Condition (Zero-Crossing Condition)
The optimization loop evaluates all internal sample index integers $i \in [n_{\text{start}}, n_{\text{end}}]$ that satisfy the algebraic sign-inversion condition rule:
$$x[i] \cdot x[i+1] \le 0$$
### 3.2. Optimization Criterion
Among all matching coordinate entries captured by the boundary condition filter, the algorithm targets the specific index $i_{\text{best}}$ that minimizes the spatial sample offset relative to the operator's input selection address ($n_{\text{target}}$):
$$i_{\text{best}} = \arg\min_{i} \left\vert{} i - n_{\text{target}} \right\vert{}$$
At coordinate point $i_{\text{best}}$, the immediate signal amplitude approaches zero ($x[i_{\text{best}}] \approx 0$). Slicing at this address ensures absolute physical phase continuity when the audio stream is partitioned or unlinked.
---
## 4. Python Backend Implementation Manual (Docker Celery DSP Worker)
This prototype Python module (`core/ai_dsp_engine.py`) runs on the backend Celery worker environment to execute automated loop indexing and fade-free zero-crossing slicing:
```python
import numpy as np
import librosa
class AIDSPEngine:
@staticmethod
def find_exact_zero_crossing(y: np.ndarray, sr: int, target_time: float, window_ms: float = 50.0) -> float:
"""
Locates the absolute nearest physical zero-crossing sample index to target_time (seconds).
Returns the optimized timeline index position in seconds where amplitude hits 0.
"""
target_sample = int(target_time * sr)
window_samples = int((window_ms / 1000.0) * sr)
# Define symmetrical horizontal boundary window
start_idx = max(0, target_sample - window_samples // 2)
end_idx = min(len(y) - 2, target_sample + window_samples // 2)
y_segment = y[start_idx:end_idx]
# DSP Condition logic tracking sign inversion: y[i] * y[i+1] <= 0
zero_crossings = np.where(y_segment[:-1] * y_segment[1:] <= 0)[0]
if len(zero_crossings) == 0:
# Fallback: if no sign change occurs, return the absolute minimum sample inside the viewport
abs_min_idx = np.argmin(np.abs(y_segment))
return float((abs_min_idx + start_idx) / sr)
# Translate local segment array address back to absolute buffer coordinates
absolute_crossings = zero_crossings + start_idx
# Isolate the crossing point closest to the raw target_sample baseline
distances = np.abs(absolute_crossings - target_sample)
best_sample_idx = absolute_crossings[np.argmin(distances)]
return float(best_sample_idx / sr)
@classmethod
def scan_best_loop_regions(cls, y: np.ndarray, sr: int, min_duration: float = 2.0, max_duration: float = 8.0) -> list:
"""
Evaluates spectral Self-Similarity Matrices (Recurrence plots) to extract
the most musically periodic and cohesive loop segments within the track.
"""
# 1. Compute harmonic structural properties via Chroma Constant-Q Transform
chroma = librosa.feature.chroma_cqt(y=y, sr=sr)
# 2. Compile the Self-Similarity Matrix (Cosine Recurrence Plot)
# This maps global structural recurrence profiles across runtime frame vectors
from sklearn.metrics.pairwise import cosine_similarity
ssm = cosine_similarity(chroma.T, chroma.T)
num_frames = ssm.shape[0]
hop_length = 512
frame_duration = hop_length / sr
best_score = -1.0
best_loop = (0.0, 4.0) # Fallback baseline setup to target initial 4 seconds
# Scan sub-diagonals to track high-density recurring correlation coefficients
# Diagonals parallel to the main identity path flag strict periodic cycles
min_frames = int(min_duration / frame_duration)
max_frames = int(max_duration / frame_duration)
for lag in range(min_frames, min_frames * 4): # Trace delay frames matching typical 1-2 measure blocks
if lag >= num_frames:
break
# Accumulate mean recurrence indices across the active sub-diagonal line
score = np.mean(np.diagonal(ssm, offset=lag))
if score > best_score:
best_score = score
# Map optimized chronological boundaries
start_frame = 0
end_frame = min(num_frames - 1, start_frame + lag)
t_start = start_frame * frame_duration
t_end = end_frame * frame_duration
best_loop = (t_start, t_end)
# 3. Lock boundaries to precise physical zero-crossings to prevent transient click noise
t_start_zero = cls.find_exact_zero_crossing(y, sr, best_loop[0])
t_end_zero = cls.find_exact_zero_crossing(y, sr, best_loop[1])
return [{"start_time": t_start_zero, "end_time": t_end_zero, "score": float(best_score)}]
@classmethod
def slice_and_copy_with_zero_crossing(
cls,
y: np.ndarray,
sr: int,
start_time: float,
end_time: float
) -> tuple:
"""
Slices an audio data array from start_time to end_time using zero-crossing alignment.
Strictly bypasses linear or exponential fade configurations.
"""
# Align bounding start and termination boundaries directly to zero-amplitude addresses
t_start_zero = cls.find_exact_zero_crossing(y, sr, start_time)
t_end_zero = cls.find_exact_zero_crossing(y, sr, end_time)
sample_start = int(t_start_zero * sr)
sample_end = int(t_end_zero * sr)
# Squeeze out raw buffer array slice without applying any destructive envelope modifiers
y_sliced = np.copy(y[sample_start:sample_end])
return y_sliced, t_start_zero, t_end_zero
```
---
## 5. Serialized API Data Transfer Protocols
During data exchange cycles initiated over the AI Panel UI layer, the client application communicates with the FastAPI routing layer via the following structured JSON payloads:
### 5.1. API 1: AI Loop Scanning (POST `/api/v1/audio/ai-scan`)
* **Request Payload (Client $\rightarrow$ Server):**
```json
{
"track_id": "1",
"file_id": "creak_forest_raw.wav",
"min_loop_duration": 2.0,
"max_loop_duration": 6.0
}
```
* **Response Payload (Server $\rightarrow$ Client):**
```json
{
"success": true,
"track_id": "1",
"suggested_loops": [
{
"start_time": 1.4589,
"end_time": 5.4592,
"score": 0.892
}
]
}
```
*(Upon parsing this response, the frontend layout engine executes an automated marker rendering pass, pinning visual handles precisely at `start_time` and `end_time`).*
### 5.2. API 2: Fade-Free AI Slicing (POST `/api/v1/audio/ai-cut`)
* **Request Payload (Client $\rightarrow$ Server):**
```json
{
"source_track_id": "1",
"file_id": "creak_forest_raw.wav",
"selection_start": 3.120,
"selection_end": 7.450
}
```
* **Response Payload (Server $\rightarrow$ Client):**
```json
{
"success": true,
"output_file_id": "ai_cut_creak_forest_3.1s.wav",
"aligned_start": 3.1192,
"aligned_end": 7.4504,
"duration": 4.3312
}
```
*(The frontend automatically builds a new track row layout right below the baseline channel, mapping the received `output_file_id` block to mount perfectly at the real-world timeline timestamp indicated by `aligned_start`).*
18_ZOOM_IN.md
-214
View File
@@ -1,214 +0,0 @@
# Technical Specification: Ultra-Zoom & Sample-Level Waveform Rendering (Sample-Level Waveform Zoom)
This document defines the technical solution, data flow schema, and graphical optimization algorithms across both the Frontend (HTML5 Canvas) and Backend (Python / Docker) to implement an Ultra-Zoom Waveform feature. This architecture renders discrete sample nodes interconnected by a continuous line vector for absolute Zero-Crossing alignment, referencing the design principles.
---
## 1. What is Sample-Level Zoom?
When displaying an audio waveform at a macro scale (Zoom Out), a single pixel column on the display represents hundreds or thousands of acoustic samples ($N$ samples/pixel). Consequently, the engine deploys a Peak Waveform algorithm that connects the maximum (Max) and minimum (Min) amplitude values within that segment using vertical lines.
However, when a operator scales the viewport magnification beyond a specific threshold (e.g., a zoom ratio of $Z \ge 100,000\text{ pixels/second}$):
* A single discrete audio sample occupies a large horizontal footprint on the display (e.g., $5 \rightarrow 15\text{ pixels/sample}$).
* The rendering engine must hot-swap its routine from standard vertical peak columns to a **Continuous Polyline with Sample Nodes** loop. Every discrete acoustic sample $x[n]$ is mapped as an independent circle node, with chronologically adjacent nodes joined by a smooth continuous path.
---
## 2. Frontend Layout Architecture (HTML5 Canvas & Web Audio API)
To render thousands of vector coordinate indices fluidly during rapid zooming and scrolling/dragging gestures without locking up the browser thread (Freeze UI), the system integrates the following memory pipeline:
```text
VIEWPORT SLICING ENGINE
┌────────────────────────────────────────────────────────────────────────┐
│ [ Web Audio Buffer (Full track - Millions of raw sample values) ] │
│ │ │
│ ▼ (Extract visible boundary region only) │
│ [ Visible Sample Array (Restricted to ~200 - 1,000 samples in view) ] │
│ │ │
│ ▼ (High-speed GPU-accelerated Canvas draw) │
│ [ HTML5 Canvas Render: ctx.arc() & ctx.lineTo() ] ──► Screen Viewport │
└────────────────────────────────────────────────────────────────────────┘
```
### 2.1. Viewport Slicing Technique
The rendering engine must never iterate through the total sample length of the audio file during a drawing pass. The slice generator isolates only the data segments that correspond directly to the physical visible screen dimensions (visible viewport boundary):
* **Visible Starting Timestamp:**
$$T_{\text{start}} = \frac{\text{scrollLeft}}{\text{Zoom}}$$
* **Visible Terminating Timestamp:**
$$T_{\text{end}} = \frac{\text{scrollLeft} + W_{\text{viewport}}}{\text{Zoom}}$$
* **Starting Array Index Offset:**
$$n_{\text{start}} = \lfloor T_{\text{start}} \times f_s \rfloor$$
* **Terminating Array Index Offset:**
$$n_{\text{end}} = \lceil T_{\text{end}} \times f_s \rceil$$
### 2.2. Sample Node Graph Canvas Algorithm
For every absolute sample index $x[i]$ contained within the sliced viewport interval $[n_{\text{start}}, n_{\text{end}}]$, the coordinate translation layer maps the raw data into physical pixel coordinates $(X, Y)$ on the Canvas:
$$X_i = \left( \frac{i}{f_s} \right) \times \text{Zoom} - \text{scrollLeft}$$
$$Y_i = \text{mid}_Y + x[i] \cdot \left( \text{height} \times 0.42 \right)$$
*Where:* $\text{mid}_Y$ maps the horizontal center zero axis (-Inf. dB line), and $x[i] \in [-1.0, 1.0]$ tracks the floating-point sample amplitude value.
### JavaScript Redraw Core Script (React / JS Context)
```javascript
function drawSampleLevelWaveform(ctx, canvasWidth, canvasHeight, audioBuffer, scrollLeft, zoom) {
const data = audioBuffer.getChannelData(0); // Query Left channel data stream
const fs = audioBuffer.sampleRate;
const midY = canvasHeight / 2;
const ampHeight = canvasHeight * 0.42; // Clamps drawing ceiling bounds to 84% of total height
// 1. Viewport Slicing Matrix Execution
const tStart = scrollLeft / zoom;
const tEnd = (scrollLeft + canvasWidth) / zoom;
const nStart = Math.max(0, Math.floor(tStart * fs));
const nEnd = Math.min(data.length, Math.ceil(tEnd * fs));
ctx.clearRect(0, 0, canvasWidth, canvasHeight);
// Set up standard studio charcoal theme background canvas
ctx.fillStyle = '#1e1e1e';
ctx.fillRect(0, 0, canvasWidth, canvasHeight);
// Overlay symmetrical decibel gridding lines (-6.0 dB, -Inf, -6.0 dB)
ctx.strokeStyle = 'rgba(255, 255, 255, 0.08)';
ctx.lineWidth = 1;
[-0.501, 0, 0.501].forEach(val => {
const y = midY + (val * ampHeight);
ctx.beginPath();
ctx.moveTo(0, y);
ctx.lineTo(canvasWidth, y);
ctx.stroke();
});
// 2. Continuous Vector Polyline Redraw Configuration
ctx.strokeStyle = '#5bc0be'; // Professional sleek light cyan accent theme
ctx.lineWidth = 1.5;
ctx.beginPath();
let isFirst = true;
for (let i = nStart; i < nEnd; i++) {
const xPixel = (i / fs) * zoom - scrollLeft;
const yPixel = midY + (data[i] * ampHeight);
if (isFirst) {
ctx.moveTo(xPixel, yPixel);
isFirst = false;
} else {
ctx.lineTo(xPixel, yPixel);
}
}
ctx.stroke();
// 3. Highlight Discrete Sample Nodes (Luminous node nodes circles)
ctx.fillStyle = '#6ee7b7'; // Vivid green emerald node color
for (let i = nStart; i < nEnd; i++) {
const xPixel = (i / fs) * zoom - scrollLeft;
const yPixel = midY + (data[i] * ampHeight);
// Render point node indicators if the physical pixel delta spacing is >= 4px (Prevents GPU thread thrashing)
const nextXPixel = ((i + 1) / fs) * zoom - scrollLeft;
if (nextXPixel - xPixel >= 4) {
ctx.beginPath();
ctx.arc(xPixel, yPixel, 2, 0, 2 * Math.PI);
ctx.fill();
}
}
}
```
---
## 3. Backend Architecture (Python / NumPy / Docker)
When an operator triggers editing transformations, loop boundary indexing (AI Scan Loops), or an AI Cut on the user interface, precise timestamp scalars (seconds) are pushed to the backend stack. The FastAPI routing layer and Celery task worker process the input metrics via NumPy using sample-accurate precision to eliminate clicking audio defects.
### 3.1. High-Performance Vectorized Zero-Crossing Analysis via NumPy
This algorithm targets the exact index offset location where an algebraic sign-inversion occurs (crossing the absolute 0 baseline) closest to the user's cursor selection coordinate:
```python
import numpy as np
def find_exact_zero_crossing_sample(y: np.ndarray, sr: int, target_time: float, search_window_ms: float = 40.0) -> int:
"""
Scans the signal buffer matrix to extract the exact sample index where amplitude
crosses the absolute 0 axis closest to target_time. Mitigates signal phase fracture.
"""
target_sample = int(target_time * sr)
window_samples = int((search_window_ms / 1000.0) * sr)
# Establish local window limits
start_idx = max(0, target_sample - window_samples // 2)
end_idx = min(len(y) - 2, target_sample + window_samples // 2)
y_segment = y[start_idx:end_idx]
# Vectorized loop matching physical phase boundaries: y[i] * y[i+1] <= 0
# This evaluates ultra-fast directly on NumPy's optimized underlying C-layer
zero_crossings = np.where(y_segment[:-1] * y_segment[1:] <= 0)[0]
if len(zero_crossings) == 0:
# Fallback: if no phase inversion is detected (extended silence), return the minimum absolute sample value
abs_min_idx = np.argmin(np.abs(y_segment))
return start_idx + abs_min_idx
# Translate the localized coordinate index back to global absolute buffer sample indices
absolute_crossings = zero_crossings + start_idx
# Isolate the index that maps closest to the original physical target_sample address
distances = np.abs(absolute_crossings - target_sample)
best_sample_index = absolute_crossings[np.argmin(distances)]
return int(best_sample_index)
```
### 3.2. Fade-Free Zero-Crossing Splicing Workflow
Once the exact boundary indices ($N_{\text{start\_zero}}$, $N_{\text{end\_zero}}$) are located using the zero-crossing analyzer:
1. **Slicing Operation:**
```python
y_cut = y[N_start_zero : N_end_zero]
```
2. **Merging & Track Insertion:** The sliced audio block is appended straight into the signal array of the destination track. Because both the initial and terminating boundaries of the cut segment are locked perfectly to a theoretical value of $0\text{V}$, splicing this array into any other silent segment preserves absolute physical phase continuity.
3. **Bypassing Fade Modulators:** The physical transient profiles (**Transients**) of percussive assets (Kick Drums, Snares, Claps) remain $100\%$ unwarped. This completely preserves the crisp, punchy acoustic characteristics of the source audio data.
---
## 4. Performance Optimization Manual
* **Double Buffering (Offscreen Canvas Rendering Canvas):** Under extreme magnification scales, client-side horizontal scrolling modifications (`onScroll`) trigger continuous drawing passes. To mitigate visual performance drop, the vector graphs should map onto an un-rendered buffer area (**Offscreen Canvas**) before executing a single block copy to the viewport canvas using the command `ctx.drawImage()`. This eliminates screen tearing or viewport flickering.
* **Throttle Rendering Threads:** Wrap interface redraw handlers inside an explicit `requestAnimationFrame()` loop. This throttles the drawing passes to synchronize exactly with the screen hardware refresh rate metrics (typically $60\text{Hz}$ or $120\text{Hz}$), which avoids drawing redundant frames when CPU threads are under heavy loads handling audio decoding.
---
Giúp bạn tìm hiểu thêm về cấu trúc này, bạn có muốn khám phá sâu hơn khía cạnh nào không?
* **Optimizing Audio Codecs:** Cách tối ưu cấu trúc lưu trữ và nén dữ liệu nhị phân khi truyền tải mảng mảng số lớn giữa Docker Server và Web Client.
* **PyQt6 High-Frequency Redraw:** Thiết lập vòng lặp vẽ đồ thị `QPainter` đa luồng trên ứng dụng Desktop Python mà không bị treo hàng đợi Event Loop.
* **Cubic Spline Interpolation:** Công thức toán học nội suy mượt nâng cao thay thế cho đường thẳng tuyến tính (Linear Polyline) để bo cong sóng âm mịn hơn.
19_DESCRIPT.md
-72
View File
@@ -1,72 +0,0 @@
Here is the complete document converted into a clean, professionally formatted Markdown layout, with fully optimized math expressions and standardized structures:
# Analysis & Bug Fix Guide: Hybrid DSP Architecture & Shift+Click Selection Algorithms
This document clarifies the execution boundaries of real-time audio monitoring (Real-time Preview) between the workstation (Client) and the server (Docker Server). It exposes the root cause of the "Shift + Click" selection range failure and provides a direct solution on the client-side.
---
## 1. Technical Q&A (Zoom-In & Hybrid Model)
### 1.1. Is it necessary to process audio vectors directly on the Client machine like Reaper or Sound Forge?
* **Answer:** Absolutely necessary ($100\%$) for **Visual Rendering**.
* **Reason:** When zooming deeply to observe individual granular phase fluctuations (**Sample Nodes**), the browser must have direct access to the raw binary array (`Float32Array`) stored in the client's RAM.
* **The Pitfall of Server-side Rendering:** If a "server-side render and push image" approach is used, the system will suffer from image blurring and network latency ($100\text{ms} \rightarrow 500\text{ms}$) during high-speed zooming or scrubbing. Decoding the file once via the Web Audio API (`AudioContext.decodeAudioData()`) on the Frontend is the industry-standard DAW solution to unlock instantaneous vector rendering at $60\text{ FPS} \rightarrow 120\text{ FPS}$ directly inside the browser.
### 1.2. Can a hybrid web application match the performance of a native desktop application?
Yes, it can execute seamlessly provided there is a clean, structured separation of roles (**Symmetrical Hybrid Separation**):
* **Client (HTML5/Web Audio/WASM):** Handles low-latency user interface interactions. This includes reading sample arrays to paint waveforms, tracking the playhead line, defining selection ranges, and driving real-time preview monitoring filters using Web Audio Nodes or WebAssembly.
* **Server (Dockerized Python):** Executes heavy rendering blocks and exports studio-grade master files. This includes multi-track mixdowns, loading genuine VST3 plugin chains via a C++ core framework (e.g., `Pedalboard`), and processing complex AI models. When changes occur, the frontend simply dispatches a lightweight JSON configuration package (**Metadata**) back to the server for asynchronous rendering, bypassing audio streaming bottlenecks.
---
## 2. Root Causes of the "Shift + Click Selection" Defect
Many AI Code Agents fail or struggle when programming this interaction loop because of several fundamental flaws:
* **Audio Waveforms on Canvas Lack DOM Nodes:** Unlike standard HTML texts where double-clicking or `Shift + Click` selections can be tracked natively between text tags, audio waveforms are flattened onto a raw `<canvas>` element. Mouse clicks only return physical pixel coordinates ($X$). Agents frequently omit the coordinate translation logic needed to map pixels back into absolute timeline seconds:
$$T = \frac{X_{\text{pixel}} + \text{scrollLeft}}{\text{Zoom}}$$
* **Event Listener Collision:** In DAW workflows, the primary mouse-down trigger (`onMouseDown`) over a track lane handles multiple overlapping roles: updating playhead placement, dragging audio clips, dragging perimeters for time-stretching, and dragging to create selection windows. When a user executes a `Shift + Click` interaction, if default behaviors are not explicitly blocked via `e.preventDefault()` and `e.stopPropagation()`, the system misinterprets the gesture as a playhead reset or a clip drag event, instantly destroying the existing selection.
* **Missing Anchor Point Tracking:** For `Shift + Click` to scale a region properly, the application must persistently cache an **Anchor Point** variable in memory:
* **Click 1 (Initial Focus):** Sets the bounding anchor milestone (e.g., $T_{\text{start}}$).
* **Shift + Click 2 (Extension):** Locks the anchor milestone and assigns a new dynamic timestamp parameter ($T_{\text{end}}$) to the secondary click coordinate.
---
## 3. Shift + Click Interaction Selection Algorithm
This interaction sequence is implemented by intercepting the state of the modifier parameter `e.shiftKey` inside the click handler logic for both the track lanes (localized selection—Local) and the timeline ruler (global master selection—Global).
### 3.1. Mouse Event Control Logic Schema
```text
[ MOUSE PRESS EVENT ON CANVAS / RULER ]
│
┌───────────────┴───────────────┐
▼ (Is Shift Key Active?) ▼ (Shift Key Inactive)
[ SHIFT + CLICK LOGIC ] [ STANDARD CLICK LOGIC ]
- Lock the existing Anchor point - Instantiate a new Anchor = Click Time
- Map new Click Time = End Time - Prepare Drag state for new region draw
- Refresh selection overlay color - Update Playhead location
```
### 3.2. Implementation Blueprint
Update your `index.html` source script with the following event mapping rules:
#### **At the Waveform Lane Viewport:**
When a mouse press is detected, evaluate `e.shiftKey`. If `true`, lock the initial boundary position from the existing selection (`localSelLeft`) as the anchor point. If no selection is present, fallback to the current playhead position (`currentTime`). Then, assign the calculated timeline position of the new click event to override the secondary boundary marker (`localSelectionEnd`).
#### **At the Time Ruler Track:**
Mirror the exact same bounding logic block to manage the global system selection layer (`selectionStart` and `selectionEnd`), enabling users to stretch or compress the global transport loop constraints efficiently.
1_DEVELOP_PLAN.md
-280
View File
@@ -1,280 +0,0 @@
# Kế Hoạch Chi Tiết: Dockerized Music Processing Server & SonicForge Studio
Tài liệu này trình bày giải pháp kiến trúc tổng thể, lựa chọn công nghệ, thuật toán xử lý tín hiệu số (DSP/AI) ở cả phía Client (Web Audio API) và Server (Python/Celery), thiết kế API, cấu trúc thư mục hợp nhất, cấu hình Docker và lộ trình triển khai hoàn chỉnh.
---
## 1. Kiến Trúc Hệ Thống Tổng Thể (System Architecture)
Hệ thống được thiết kế theo mô hình **Lai (Hybrid Client-Server)**:
* **Client-side (SonicForge Studio):** Đảm nhận các tác vụ tương tác thời gian thực, trực quan hóa sóng âm, nghe thử đa kênh (multi-track playback), tính toán điểm dừng mềm (micro-fades), mô phỏng điểm Zero-crossing gần nhất và xuất bản trực tiếp định dạng WAV nhẹ.
* **Server-side (FastAPI Gateway & Celery Workers):** Đảm nhận các tác vụ phân tích cấu trúc phức tạp (AI Beat tracking, tách nguồn Vocal/Instrumental bằng Demucs) và các phiên xử lý hàng loạt khối lượng lớn (Batch-editing/Rendering) tệp tin âm thanh độ phân giải cao.
```text
[ TRÌNH DUYỆT NGƯỜI DÙNG (CLIENT-SIDE) ]
┌──────────────────────────────────────────────────────────────────────┐
│ React UI - SonicForge Studio │
│ ┌────────────────────────┐ ┌────────────────────────┐ │
│ │ Web Audio Engine │ │ Client-side DSP Engine│ │
│ │ (Multi-track Playback, │ │ - Zero-Crossing Snap │ │
│ │ Gain Control, Fades) │ │ - Offline Mixdown WAV │ │
│ └───────────▲────────────┘ └───────────▲────────────┘ │
└──────────────┼───────────────────────────────────────┼───────────────┘
│ (Upload Audio / Trả kết quả) │ (JSON API Config)
▼ ▼
┌──────────────────────────────────────────────────────────────────────┐
│ MÁY CHỦ BẢN TIN (SERVER-SIDE DOCKER) │
│ │
│ [FastAPI Gateway] <──(Check Task Status)── [Redis Broker / Backend] │
│ │ ▲ │
│ ├─► (Đẩy tác vụ nặng) │ │
│ ▼ │ │
│ [Celery Workers (Audio DSP & AI Engine)] ─────────┘ │
│ │ (Đọc/Ghi dữ liệu) │
│ ▼ │
│ [Shared Volume (Audio Files / Storage)] │
└──────────────────────────────────────────────────────────────────────┘
```
---
## 2. Đề Xuất Công Nghệ & Thư Viện
### 2.1. Phía Giao Diện (Frontend & Client-side DSP)
* **Thư viện lõi:** React 18 (UI quản lý trạng thái, Track, Clip và Marker), Tailwind CSS (Giao diện đáp ứng tối ưu hóa không gian tối), Lucide Icons (Biểu tượng chức năng).
* **Công cụ xử lý âm thanh:** Web Audio API (Giải mã `AudioBuffer` trực tiếp, quản lý luồng định tuyến âm thanh thông qua `AudioContext`, `GainNode`, `AnalyserNode`).
* **Offline Processing:** `OfflineAudioContext` (Hòa âm đa kênh tốc độ cao ngay trên bộ nhớ trình duyệt để kết xuất không trễ).
* **Visualization:** HTML5 Canvas API (Vẽ đồ thị sóng âm waveform động cho từng clip dựa theo tọa độ thời gian thực và phổ tần số âm thanh Master Output).
### 2.2. Phía Máy Chủ (Backend & Server-side DSP)
* **Ngôn ngữ chính:** Python 3.11+ kết hợp FastAPI phục vụ static files và APIs.
* **Thư viện phân tích & xử lý:**
* `librosa`: Phân tích Tempo (BPM), Beat-tracking chính xác, xác định cấu trúc nhịp ($Bars$).
* `pydub` & `FFmpeg`: Cắt ghép tệp gốc ở tầng nhị phân, thay đổi cao độ, âm lượng ($dB$), tạo dải chuyển tiếp fade-in/fade-out chuyên nghiệp.
* `scipy` / `numpy`: Phân tích mảng số (array processing) trên đồ thị sóng âm gốc để đồng bộ điểm Zero-crossing tinh chỉnh.
---
## 3. Giải Pháp Kỹ Thuật & Thuật Toán Đồng Bộ
### 3.1. Phân Tích Nhịp & Đồng Bộ Nhịp Client-Server
Máy chủ chạy thuật toán Dynamic Programming của `librosa` để trích xuất mốc nhịp gốc. Kết quả trả về cấu trúc JSON chứa danh sách các điểm phách (beats) và khuôn nhạc ($bars$).
Giao diện Client tiếp nhận JSON này, chuyển dịch sang hệ tọa độ Pixels dựa trên tỷ lệ thu phóng (Zoom Level):
$$\text{X Position (px)} = \text{Time (seconds)} \times \text{Zoom Level (px/sec)}$$
### 3.2. Thuật Toán Tìm Điểm Zero-Crossing (Không Tiếng "Click" Âm Thanh)
Để triệt tiêu các xung âm đột ngột gây ra tiếng "click/pop" khi ghép nối hoặc lặp (loop) âm thanh, cả Client và Server đều áp dụng cơ chế căn lề Zero-Crossing.
**Thuật toán toán học:** Tìm vị trí mẫu $i$ sao cho tích của hai mẫu liên tiếp nhỏ hơn hoặc bằng $0$ (biên độ đổi dấu từ dương sang âm hoặc ngược lại):
$$x[i] \cdot x[i+1] \le 0$$
* **Tại Client:** Để tối ưu hiệu năng kéo thả Marker cắt ngay trên trình duyệt, Client-side JS quét mảng kênh 0 (`Float32Array` từ `AudioBuffer`) trong dải thời gian lân cận điểm kéo thả khoảng $\pm 50\text{ms}$:
$$\text{Vùng quét (mẫu)} = [\text{Target Sample} - (0.05 \times \text{Sample Rate}), \text{Target Sample} + (0.05 \times \text{Sample Rate})]$$
Marker sẽ tự động được dính ("snap") vào điểm Zero-Crossing gần nhất.
* **Tại Server:** Khi nhận request API cắt ghép từ Client dưới dạng giây (seconds), module Python `dsp_utils.py` sử dụng `numpy` thực hiện phép dò tương tự trên tệp audio gốc chất lượng cao trước khi ghi đĩa.
### 3.3. Giải Thuật Fade Nhẹ Tự Động (Micro-Fading)
Khi thực hiện thao tác Cắt (Split) hoặc Ghép (Merge) đa đoạn trên một Track, hệ thống tự động chèn dải Micro-Fade thời lượng cực ngắn ($50\text{ms}$) tại điểm cắt để triệt tiêu vĩnh viễn nhiễu sóng tần số cao.
---
## 4. Thiết Kế RESTful API Endpoints Hợp Nhất
| Method | Endpoint | Description | Request/Response |
| --- | --- | --- | --- |
| **GET** | `/` | Trả về giao diện Web Editor (Tệp tin tệp tĩnh `index.html`). | HTML Response |
| **POST** | `/api/v1/audio/upload` | Người dùng upload tệp tin nhạc lên máy chủ. Trả về `file_id`. | `file: UploadFile` |
| **GET** | `/api/v1/audio/tasks/{task_id}` | Kiểm tra trạng thái phân tích nhịp từ Celery (BPM, Beats, Bars). | JSON |
| **POST** | `/api/v1/audio/edit` | Thực hiện cắt ghép nâng cao và lưu kết quả trên server. | JSON Payload |
| **GET** | `/api/v1/audio/download/{file_id}` | Tải tệp tin kết quả cuối cùng từ Server. | Binary Stream |
#### Cấu trúc JSON Request cho Endpoint `/api/v1/audio/edit` (Đồng bộ hóa trực tiếp từ Client):
```json
{
"file_id": "original_uuid_1234.wav",
"cut_start_ms": 12000,
"cut_end_ms": 24000,
"zero_crossing_align": true,
"loop_count": 4,
"fade_in_ms": 1000,
"fade_out_ms": 1500,
"volume_change_db": 3.5
}
```
---
## 5. Cấu Trúc Thư Mục Dự Án Toàn Diện
```text
music-processing-server/
│
├── docker-compose.yml
├── Dockerfile
├── requirements.txt
│
├── app/
│ ├── __init__.py
│ ├── main.py # FastAPI Gateway (Chạy API & Phục vụ index.html)
│ ├── config.py # Biến môi trường & cấu hình hệ thống
│ │
│ ├── templates/ # Thư mục lưu trữ mã nguồn giao diện
│ │ └── index.html # File Giao diện SonicForge Studio React/Web Audio API
│ │
│ ├── api/ # Quản lý Router và Endpoints
│ │ ├── __init__.py
│ │ └── v1/
│ │ ├── audio.py # API tải lên/phục vụ tệp tĩnh âm thanh
│ │ └── tasks.py # Trạng thái tác vụ bất đồng bộ
│ │
│ ├── core/ # Các module xử lý lõi (DSP)
│ │ ├── __init__.py
│ │ ├── analyzer.py # Phân tích BPM, Beats, Bars (Librosa)
│ │ ├── dsp_utils.py # Thuật toán Zero-crossing, Fade nâng cao
│ │ └── audio_editor.py # Cắt, ghép, loop, volume (Pydub)
│ │
│ ├── tasks/ # Celery worker tasks
│ │ ├── __init__.py
│ │ └── worker.py # Định nghĩa tác vụ nền
│ │
│ └── storage/ # Thư mục lưu trữ volume chung
│ ├── uploads/ # Nhạc gốc tải lên
│ └── processed/ # Nhạc đầu ra sau kết xuất
```
---
## 6. Cấu Hình Docker & Docker Compose Phục Vụ Cả Frontend & Backend
### 6.1. File Dockerfile
Sử dụng base-image `python-slim`, tích hợp đầy đủ thư viện đồ họa và xử lý âm thanh `FFmpeg` và `libsndfile1`.
```dockerfile
FROM python:3.11-slim
# Thiết lập thư mục làm việc
WORKDIR /app
# Cài đặt các thư viện hệ thống cần thiết (FFmpeg, libsndfile)
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
libsndfile1 \
build-essential \
&& rm -rf /var/lib/apt/lists/*
# Sao chép và cài đặt Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Sao chép mã nguồn bao gồm cả thư mục templates chứa index.html
COPY . .
# Tạo thư mục chứa file nhạc và cấp quyền ghi
RUN mkdir -p /app/storage/uploads /app/storage/processed && chmod -R 777 /app/storage
# Mặc định mở port 8000 cho FastAPI
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
```
### 6.2. Phục Vụ Giao Diện Từ `app/main.py`
Để tránh lỗi phân tách nguồn gốc tên miền (CORS) khi chạy riêng lẻ, FastAPI sẽ đóng vai trò phục vụ trực tiếp tệp giao diện tĩnh:
```python
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from fastapi.staticfiles import StaticFiles
import os
app = FastAPI(title="SonicForge API Engine")
# Mount thư mục lưu trữ nhạc để Client có thể stream trực tiếp
app.mount("/static/audio", StaticFiles(directory="app/storage"), name="audio")
@app.get("/", response_class=HTMLResponse)
async def get_index():
index_path = os.path.join("app", "templates", "index.html")
with open(index_path, "r", encoding="utf-8") as file:
return HTMLResponse(content=file.read(), status_code=200)
```
---
## 7. Giải Thuật Kết Xuất WAV Đa Định Dạng (WAV Encoder 8/16/24-bit)
Khi người dùng thực hiện xuất bản nhạc trực tiếp ở giao diện Frontend, họ có thể lựa chọn xuất tệp WAV với độ sâu bit và tần số lấy mẫu chỉ định. Mã nguồn Frontend mã hóa trực tiếp thông qua lớp ghi nhị phân RIFF WAVE:
* **8-bit PCM (Lo-Fi):** Biểu diễn dạng số nguyên không dấu (Unsigned Integer), dải giá trị $[0, 255]$.
$$\text{Sample}_{8\text{-bit}} = \text{round}((\text{FloatSample} + 1.0) \times 127.5)$$
* **16-bit PCM (Chuẩn CD):** Biểu diễn dạng số nguyên có dấu (Signed Integer), dải giá trị $[-32768, 32767]$.
$$\text{Sample}_{16\text{-bit}} = \text{round}(\text{FloatSample} \times 32767)$$
* **24-bit PCM (HD Audio):** Biểu diễn dạng số nguyên có dấu, 3 bytes dữ liệu $[-8388608, 8388607]$.
$$\text{Sample}_{24\text{-bit}} = \text{round}(\text{FloatSample} \times 8388607)$$
---
## 8. Kế Hoạch Triển Khai Chi Tiết (5-Week Roadmap)
### Tuần 1: Khởi Tạo Môi Trường Docker & Giao Diện Tĩnh (Static Serving)
* [ ] Thiết lập cấu trúc thư mục dự án thống nhất.
* [ ] Viết file `Dockerfile` và `docker-compose.yml` để liên kết FastAPI, Redis và Celery.
* [ ] Đưa tệp `index.html` của trình biên tập SonicForge Studio vào thư mục `app/templates` và cấu hình Endpoint `/` để kiểm tra khả năng phục vụ giao diện và tải file kéo thả.
### Tuần 2: Xây Dựng Audio Engine Tại Client (Web Audio API)
* [ ] Hoàn thiện cơ chế vẽ đồ thị sóng âm động dựa trên canvas cho các tệp âm thanh tải lên tự do từ máy người dùng.
* [ ] Kiểm thử cơ chế tính toán điểm Zero-crossing trực tiếp bằng Javascript để gán Marker thông minh (Snap-to-zero).
* [ ] Hiện thực hóa các nút điều khiển: volume của từng track độc lập, tắt tiếng (Mute), solo, dải Fade-in/Fade-out cho từng clip.
### Tuần 3: Hoàn Thiện Core DSP Phía Backend (Python Processing)
* [ ] Hiện thực hóa thuật toán phân tích nhịp bằng librosa (`analyzer.py`), trả về danh sách phách ($beats$) và khuôn nhạc ($bars$) định dạng JSON.
* [ ] Phát triển công cụ chỉnh sửa `audio_editor.py` phía máy chủ để thực thi việc cắt, ghép, loop với độ dài lớn, chuyển đổi cao độ và xuất tệp tin chất lượng cao bằng pydub.
### Tuần 4: Tích Hợp Bất Đồng Bộ (Client-Server Sync)
* [ ] Liên kết các nút điều khiển trên Frontend để sinh mã cấu hình JSON API động gửi tới máy chủ FastAPI.
* [ ] Cài đặt Celery worker xử lý tác vụ nặng ở background và trả về tiến độ trực tiếp cho Frontend hiển thị thông qua trạng thái tác vụ của Redis Result Backend.
* [ ] Tích hợp tính năng Gộp Track (Merge) sử dụng `OfflineAudioContext` của Client kết hợp với API Render đa kênh của Server.
### Tuần 5: Kiểm Thử Âm Học & Tối Ưu Hóa (Testing & Optimization)
* [ ] Kiểm thử hiện tượng giật/trễ tiếng (Audio Glitch / Pop) bằng cách ghép nối ngẫu nhiên các đoạn nhạc và kiểm thử hiệu năng tối ưu của cả hai tầng Zero-Crossing (Web Audio vs FFmpeg/Pydub).
* [ ] Tối ưu hóa bộ nhớ đệm RAM trong Docker khi xử lý song song các tệp âm thanh có dung lượng lớn.
* [ ] Đóng gói và nghiệm thu dự án.
20_CLIENT_SIDE.md
-209
View File
@@ -1,209 +0,0 @@
# Technical Directive Manual & Architectural Standards: SonicForge Studio
This document serves as the supreme and mandatory technical standard for all AI Code Agents engaged in the development, maintenance, or refactoring of the SonicForge Studio codebase. The directives below are established to completely eliminate arbitrary inferences (hallucinations), ensuring the mathematical integrity of Digital Signal Processing (DSP) and professional-grade DAW graphical layouts.
---
## 1. Critical Directives for AI Agents
* **No Arbitrary Rewrites:** Absolutely do not alter the foundational architecture of waveform rendering loops, marker anchor management systems, or Web Audio API routing networks unless explicitly instructed.
* **Preserve DSP Math:** Symmetrically retain all trigonometric equations, Cubic Hermite Splines, Constant-Power Panning constraints, and zero-crossing detection routines within source files. A structural deviation of even a single sample ($1\text{ sample}$) constitutes a critical production failure.
* **Strict UI Alignment:** All graphical modulations must cleanly conform to specified spatial layout grids, dimensions, and hex color tokens.
* **Zero Spurious Scrollbars:** Prevent internal horizontal scrollbar generation inside the left Track Control Panel (TCP) container at all costs.
---
## 2. UI & Layout Refactoring Specifications
To eliminate vertical row desynchronization and layout overlaps during timeline scrubbing or zooming operations, all rendering passes must strictly conform to the following nested architecture:
### 2.1. Unified Row Layout — Fixing Vertical Misalignment
* **Strict Grid Containment:** Independent scrolling columns for track controls and waveforms are strictly prohibited.
* **Row Lock:** Every unique channel track must be bundled inside a single parent **Unified Track Row** container framework (Flex Row or Grid Row) enforcing a rigid vertical constraint ($H = 96\text{ px}$).
* **Single Scrollbar Mandate:** The layout must expose exactly one global vertical scrollbar on the far right of the viewport container. This scrollbar controls the entire track stack workspace simultaneously, forcing the TCP decks and waveform canvas viewports to slide along the $Y$-axis in perfect physical synchronization.
### 2.2. Graphical Overlap Containment Mechanics
* **TCP Isolation:** The left TCP channel block requires a rigid width lock at $300\text{ px}$, `flex-shrink: 0`, and a solid background color (`background-color: #262626`). It must be explicitly configured with `overflow: hidden` to block internal horizontal overflow scrollbars.
* **Z-Index Layering:** Assign an elevated layout layer profile (`position: relative`, `z-index: 20`) to the TCP column. When the right timeline area scrolls horizontally to the left, all waveform graphics, grid line divisions, and the absolute playback playhead line must scroll seamlessly beneath the solid TCP masking layer.
### 2.3. Dynamic Min-Zoom Constraint Specification
* **Viewport Boundary Alignment:** When executing a macro zoom-out operation, the comprehensive project arrangement length—stretching from $0.00\text{ s}$ out to the termination milestone ($T_{\text{max}}$)—must fit perfectly within the visible horizontal frame width ($W_{\text{viewport}}$).
* **Dynamic Bounds Calculation:** The layout manager must dynamically calculate the bounding minimum scale factor ($Z_{\text{min}}$) before updating drawing buffers:
$$Z_{\text{min}} = \frac{W_{\text{viewport}}}{T_{\text{max}}}$$
* **Clamping Rule:** Under no circumstances can the active zoom factor $Z$ drop below the $Z_{\text{min}}$ threshold. Enforcing this clamping boundary blocks the generation of dead black voids on the right side of shorter clips and prevents spurious scrollbar scaling artifacts.
---
## 3. Microscopic Viewport Waveform Painting (Ultra-Zoom Render Modes)
Whenever a user zooms deeply onto the timeline canvas to analyze microscopic phase movements, the canvas engine automatically swaps its calculation loop routines based on the instantaneous visible sample density profile ($\text{samplesPerPixel}$):
```text
SAMPLES PER PIXEL DENSITY SPECTRUM
[Samples/px ≥ 4] ──────────────────────► Peak Waveform (Symmetrical Vertical Min/Max bars)
[1.5 ≤ Samples/px < 4] ────────────────► Continuous Polyline (Light Cyan Sine Path)
[Samples/px < 1.5] ────────────────────► Discrete Sample Nodes (Green Emerald Nodes + Polyline)
```
### 3.1. Peak Compression Mode ($\text{samplesPerPixel} \ge 4$ — `image_5ec2e5.png`)
* **Waveform Envelopes:** Renders a high-density, symmetrical downsampled waveform graphic. The engine reads localized segment buffers to connect absolute maximum (Max) and minimum (Min) sample peaks passing through identical pixel columns using clean vertical line strokes.
### 3.2. Single Continuous Polyline & Node Mode ($\text{samplesPerPixel} < 4$)
* **Continuous Polyline:** Transitions away from vertical peak columns to compile a fine, anti-aliased single continuous vector polyline tracking raw values in professional cornflower blue (`#5bc0be`). The translation maps absolute sample addresses to physical drawing coordinates $(X_i, Y_i)$:
$$X_i = \left( \frac{i}{f_s} \right) \times Z - \text{scrollLeft}, \quad Y_i = \text{mid}_Y + x[i] \cdot \left( \text{height} \times 0.42 \right)$$
* **Discrete Sample Nodes ($\text{samplesPerPixel} < 1.5$):** Overlays luminous green emerald circle markers (`#6ee7b7`) with a rigid radius $r = 2\text{ px}$ directly centered over every sample index coordinate $(X_i, Y_i)$. To prevent GPU thread thrashing and rendering lag, point nodes are only drawn if the horizontal pixel spacing between adjacent nodes satisfies a $\ge 4\text{ px}$ width threshold.
* **Logarithmic Amplitude Grid:** Projects thin, low-contrast background horizontal marker grids to establish clear visible decibel tracking boundaries: a positive upper peak grid at $+6.0\text{ dB}$ (or $0\text{ dBFS}$), a true horizontal identity zero-line axis at $-\infty\text{ dB}$ ($0\text{V}$ absolute silence), and a negative lower sub-grid line at $-6.0\text{ dB}$.
---
## 4. Selection Ranges & Modifier Input Mechanics
### 4.1. Persistent Anchor Point Tracking Refs
* **State Preservation:** To ensure that horizontal selection boundaries are never discarded or cleared when UI frameworks trigger background state refresh cycles, the coordinate calculation loops must persistently cache initial interaction milestones inside non-reactive memory Refs:
* *Main Session Workspace:* Employs `localSelectionAnchorRef` to monitor channel track selections, and `rulerAnchorRef` to track global time loops on the ruler.
* *Sub-Tab Sandbox Workspace:* Locks anchor coordinate data inside `subTabAnchorRef`.
### 4.2. Shift + Click Selection Range Adjustment Algorithm
When intercepting a primary mouse-down event (`onMouseDown`) where the `Shift` modifier is explicitly engaged (`e.shiftKey === true`), the tracking framework must execute the following sequence:
1. **Event Interception:** Immediately call `e.preventDefault()` and `e.stopPropagation()`. This blocks the thread, halting automatic playhead relocation or clip dragging sequences.
2. **Anchor Extraction:** Extract the absolute timestamp cached inside the target workspace Ref ($T_{\text{anchor}}$). If the reference object is unpopulated, write the active playback playhead timestamp (`currentTime`) to act as the fallback anchor milestone.
3. **Boundary Translation:** Convert the new cursor coordinate column pixel position into absolute timeline seconds to define the moving boundary marker ($T_{\text{end}}$).
4. **Range Construction:** Update the highlighted selection envelope parameters to encapsulate the full calculated interval:
$$\text{Selection Range} = [\min(T_{\text{anchor}}, T_{\text{end}}), \max(T_{\text{anchor}}, T_{\text{end}})]$$
### 4.3. Transport Loop Constraints & Escape Hook
* **Strict Loop Lock:** When a selection window $[T_{\text{start}}, T_{\text{end}}]$ is engaged alongside loop playback mode, the transport playhead can never drift past $T_{\text{end}}$. Upon reaching the $T_{\text{end}}$ index, the audio thread must instantly trigger an immediate, gapless reset back to $T_{\text{start}}$.
* **Escape Hook:** To clear selection boundaries and return the engine to standard non-repeating tracking, the user executes a `Ctrl + Click` shortcut combo over an unpopulated workspace area. Once the selection ranges are nullified, pressing the `Spacebar` drives continuous, linear playback past the old loop constraints.
---
## 5. Non-Linear Graphical Automation Envelopes
The application upgrades static, linear layout components using the following signal processing algorithms:
### 5.1. Volume Automation Spline (Monotone Cubic Hermite Spline)
To connect peach-colored volume nodes smoothly without inducing artificial overshoot peaks, the system runs a 3rd-order monotone cubic interpolation framework:
$$y(t) = (2t^3 - 3t^2 + 1)y_1 + (t^3 - 2t^2 + t)h \cdot m_1 + (-2t^3 + 3t^2)y_2 + (t^3 - t^2)h \cdot m_2$$
Where $h = t_2 - t_1$, and the localized tangents ($m_1, m_2$) are evaluated via the Fritsch-Carlson configuration method to preserve strict mathematical monotonicity across the curve.
### 5.2. Boundary Fade Contours (Trigonometric Cosine S-Curve)
The physical curvature profile of the deep red fade envelopes is derived via trigonometric functions to protect structural transient integrity at the clips boundaries:
$$f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade}}} \right)}{2}, \quad f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade}})}{L_{\text{fade}}} \right)}{2}$$
### 5.3. Constant-Power Stereo Panning Law
To eliminate spatial perceived volume collapse (*Center Dip*) when moving signals across Left ($L$) and Right ($R$) drivers, the cumulative output sound field energy must remain perfectly preserved at unity ($1.0$) across all panning trajectories:
$$\theta(t) = \frac{p(t) + 1}{2} \cdot \frac{\pi}{2}, \quad g_L(t) = \cos(\theta(t)), \quad g_R(t) = \sin(\theta(t))$$
---
## 6. Isolated Sandbox Sub-Tab Workspace & Synchronization
When a user double-clicks an audio clip asset or highlights a segment and selects "Edit in Sub-tab", the application triggers a specialized editing sandbox pipeline:
### 6.1. Sandbox Isolation Flow
* **Buffer Isolation:** The application isolates a non-destructive copy of the targeted sample slice (`Audio Sub-segment Buffer`) into memory and spawns a distinct standalone document editor window. The timeline measuring ruler inside this sub-tab resets completely to map $t = 0.0\text{ s}$ at its origin.
* **Row Scale Adjustments:** Users drag the bottom perimeter boundary of the single track lane (`ns-resize` style handle) to dynamically alter height constraints between a lower boundary of $48\text{ px}$ and an upper boundary of $200\text{ px}$ for precision envelope drawing.
### 6.2. Core Toolbar Sliders Widget Matrix
* **Normalize Ceiling:** Evaluates the signal array to scale the single maximum absolute sample peak exactly up to user-specified decibel thresholds variable from $-12\text{ dBFS}$ to $0\text{ dBFS}$.
* **Gain & Pitch Modulation:** Adjusts macro channel decibel levels and transposes fundamental vocal or instrument frequencies using an integrated Phase Vocoder algorithm.
* **Speed Stretch Slider (%):** Drives time-stretching operations visuals directly from the timeline layer by holding the `Alt` modifier key and dragging the rightmost bounding clip handle. A bright yellow metadata text string (e.g., `Speed: 75.0%`) renders at the upper-left boundary of the audio clip container:
$$S = \frac{D_{\text{original}}}{D_{\text{stretched}}} \times 100\%$$
### 6.3. Volume Pencil Automation Tool
Activating the Pencil drawing utility overlays a solid horizontal neon green line representing $0\text{ dB}$ (Unity Gain) across the track axis. Users left-click to drop custom vector control points, dragging node handles upward to boost signal gains (up to $+3\text{ dB}$) or downward to attenuate track volume (down to $-30\text{ dB}$).
### 6.4. Crossfaded In-Place Overwrite Core Loop (Apply & Sync-Back)
Clicking the *Apply* action pushes the processed sample buffer array back into the primary multitrack mixing arrangement canvas. To prevent wave phase breakage that precipitates popping artifacts, the splicing engine bakes an ultra-fast linear crossfade envelope ($w = 10\text{ ms}$) across both the initial and trailing splice boundaries:
$$\text{Output}(t) = (1 - \alpha(t)) \cdot \text{Original}(t) + \alpha(t) \cdot \text{Edited}(t - T_{\text{start}})$$
---
## 7. Automated AI Loop Scanning & Fade-Free Slicing
### 7.1. Chromagram-Driven AI Loop Indexing
The system processes raw track files using an asynchronous Celery worker script that compiles a **Self-Similarity Matrix (SSM)** derived from spectral Chroma audio features. The algorithm locates areas showcasing the highest recurrence metrics (e.g., drum grooves, chord loops) and automatically maps matching timeline markers onto the user interface canvas views.
### 7.2. Sample-Accurate Phase Inversion Slicing (Fade-Free AI Cut)
Artificially introducing volume fade envelopes to mask clicking anomalies during macro audio cuts is strictly prohibited due to its destructive impact on percussive transient impact waves. The system must natively locate the absolute closest physical zero-crossing address where the signal array crosses the zero baseline (absolute silent index):
$$x[i] \cdot x[i+1] \le 0$$
Once both clip perimeters are hard-aligned to true zero-amplitude sample offsets, the engine slices the raw binary array inside RAM and generates a new track row directly below, dropping the processed clip onto it at the exact optimized time coordinates.
---
## 8. Dockerized Python Server Deployment & Architecture
### 8.1. Headless JUCE C++ VST/VSTi Audio Rendering Pipeline
To ensure that containerized Python workflows can initialize and instantiate VST3 processing nodes and virtual instruments compiled via C++ (`JUCE framework`) under Linux environments without triggering X11 display linkage initialization crashes, the underlying systems architecture must embed and initialize a virtual display frame buffer (`Xvfb`):
```dockerfile
# Dockerfile snippet installing core graphical rendering dependencies and Xvfb
RUN apt-get update && apt-get install -y \
libgl1-mesa-glx libglu1-mesa libasound2 libjack-jackd2-0 \
libfreetype6 libfontconfig1 libx11-6 libxext6 libxrandr2 \
xvfb \
&& rm -rf /var/lib/apt/lists/*
ENV DISPLAY=:99
CMD ["sh", "-c", "Xvfb :99 -screen 0 1024x768x16 & python app/main.py"]
```
### 8.2. RBAC Security, Disk Quotas, and Feature Flags Configuration
* **First-Login Security Control (Enforced Password Reset):** System administrator accounts are initialized using parameters parsed from environment strings (`DEFAULT_ADMIN_PASSWORD`). The identity route mapper assigns a strict boolean cờ `must_change_password = True` value, which intercepts all subsequent incoming client API audio processing requests and returns a `HTTP 403 Forbidden` error loop until a secure password overwrite is completed.
* **Storage Allocation Constraints (Admin Quotas):** The gateway layer embeds a resource allocation supervisor tracking storage disk boundaries ($S_{\text{limit}}$). It aggregates the byte sizes of active array blocks before certifying a file upload sequence:
$$S_{\text{used}} + S_{\text{new}} \le S_{\text{limit}}$$
* **Feature Flags Management:** Administrators can dynamically enable or disable advanced server-side runtime pipelines (such as high-fidelity 24-bit WAV mixdown rendering or automated AI track generation) via modifications to global database flag keys.
21_CLIENT_PRE.md
-193
View File
@@ -1,193 +0,0 @@
Dưới đây là toàn bộ nội dung tài liệu đặc tả kiến trúc xử lý âm thanh chuyên nghiệp cấp độ Desktop trên Client-Side đã được chuyển đổi sang định dạng Markdown chuẩn, tối ưu hóa các khối mã nguồn (`text`, `cpp`), căn chỉnh bảng biểu, sơ đồ luồng ASCII và các công thức toán học dạng LaTeX:
# Đặc Tả Kiến Trúc: Xử Lý Âm Thanh Chuyên Nghiệp Cấp Độ Desktop Trên Client-Side
Tài liệu này đặc tả kiến trúc hệ thống, giải pháp công nghệ và các thuật toán xử lý tín hiệu số (DSP) để xây dựng bộ máy biên tập âm thanh chuyên nghiệp (*Audio Editor Engine*) hoạt động độc lập và hiệu năng cao ngay trên máy trạm (*Client-side*) tương tự như Sound Forge hay Adobe Audition, sử dụng nền tảng HTML5, Web Audio API nâng cao, WebAssembly (WASM) và `SharedArrayBuffer`.
---
## 1. Sơ Đồ Kiến Trúc Lõi (Client-Side Audio Engine Architecture)
Để đạt được hiệu năng xử lý không độ trễ và không gây nghẽn luồng giao diện (*UI Main Thread*), hệ thống bắt buộc phải tách biệt hoàn toàn ba lớp luồng thực thi:
```text
┌────────────────────────────────────────────────────────────────────────┐
│ MAIN THREAD (UI / REACT) │
│ - Render giao diện Canvas, Sliders, Rulers, Waveform. │
│ - Nhận tương tác phím/chuột (Shift+Click, Drag, Zoom). │
│ - Giao tiếp bất đồng bộ qua MessagePort / Worker PostMessage. │
└───────────────────┬────────────────────────────────▲───────────────────┘
│ │
│ SharedArrayBuffer / Atomics │ SharedArrayBuffer / Atomics
▼ │
┌────────────────────────────────────────────────────┴───────────────────┐
│ AUDIO WORKLET THREAD (LOW-LATENCY AUDIO RENDERING) │
│ - Thực thi luồng xử lý âm thanh thời gian thực (Audio Graph). │
│ - Đọc/Ghi mảng Ring Buffer (Shared Memory) không khóa (Lock-free). │
│ - Gọi trực tiếp lõi xử lý DSP viết bằng WebAssembly (C++/Rust). │
└────────────────────────────────────────────────────────────────────────┘
```
---
## 2. Các Công Nghệ Cốt Lõi Trên Client-Side
### 2.1. Web Audio API Nâng Cao (`AudioContext` & `AudioWorklet`)
* **Hạn chế của API cũ:** Các nút xử lý mặc định (`ScriptProcessorNode`) chạy trực tiếp trên Main Thread, gây ra hiện tượng giật lag âm thanh (*audio glitching/pop*) bất cứ khi nào trình duyệt thực hiện tính toán UI hoặc render đồ họa nặng.
* **Giải pháp chuẩn DAW:** Sử dụng `AudioWorklet`. Trình duyệt sẽ khởi tạo một luồng xử lý riêng biệt có độ ưu tiên thời gian thực (*Real-time Priority Thread*) tách biệt hoàn toàn khỏi luồng dựng hình UI.
### 2.2. WebAssembly (WASM) — Bộ Máy DSP Hiệu Năng Tiệm Cận Native
* **Vai trò:** JavaScript không có kiểu dữ liệu tối ưu và tốc độ thực thi các vòng lặp mẫu nhanh bằng các ngôn ngữ có biên dịch biên độ thấp. WebAssembly cho phép đưa các thư viện xử lý âm thanh C++ hoặc Rust (như FFmpeg, SoX, Superpowered, hoặc JUCE DSP) chạy trực tiếp trong trình duyệt với hiệu năng đạt mức $90\% \rightarrow 95\%$ so với phần mềm máy tính.
* **Quy trình hoạt động:** Giải mã tệp WAV nhị phân vào bộ nhớ Heap của WASM (*WASM Linear Memory*). Luồng C++ sẽ xử lý toán học trực tiếp trên các con trỏ bộ nhớ này thông qua kiểu dữ liệu mảng float 32-bit (`Float32Array`).
### 2.3. `SharedArrayBuffer` & `Atomics` — Chia Sẻ Bộ Nhớ Không Khóa
* **Vấn đề luồng:** Việc chuyển dữ liệu lớn (Hàng chục Megabytes dữ liệu âm thanh) giữa Main Thread và AudioWorklet Thread bằng lệnh `postMessage` thông thường sẽ gây ra độ trễ sao chép dữ liệu (*Serialization Latency*) và tăng rác bộ nhớ (*Garbage Collection overhead*).
* **Giải pháp:** Sử dụng `SharedArrayBuffer`. Cả hai luồng UI và AudioWorklet cùng truy cập vào một vùng nhớ RAM vật lý duy nhất. Sử dụng thư viện `Atomics` để đồng bộ hóa và ghi nhận trạng thái con trỏ phát nhạc (*Playhead position*) một cách an toàn và không gây nghẽn luồng xử lý (*Lock-free Ring Buffer*).
---
## 3. Các Thuật Toán DSP Chuyên Sâu Cần Port Sang Client-Side
Để đạt được chất lượng xử lý của Sound Forge và Audition, hệ thống phải thực hiện các thuật toán tín hiệu số trực tiếp trên mảng dữ liệu $x[n]$ ở Client-side:
### 3.1. Phân Tích Phổ Tần Số Thời Gian Thực (Fast Fourier Transform — FFT)
Để hiển thị biểu đồ phổ (*Spectrogram*) và thực hiện biên tập tần số (*Spectral Editing*) như Adobe Audition, ta chuyển đổi tín hiệu từ miền thời gian sang miền tần số bằng phép biến đổi Fourier nhanh (FFT) bậc $N$ (thường chọn $N = 2048$ hoặc $N = 4096$ mẫu):
$$X(f) = \sum_{n=0}^{N-1} x[n] \cdot e^{-i 2 \pi f n / N}$$
* **Tối ưu hóa:** Sử dụng thư viện WASM FFT (như KissFFT hoặc FFTW biên dịch sang WASM) để thực hiện tính toán song song bằng tập lệnh Vector hóa SIMD (*Single Instruction, Multiple Data*) của CPU máy khách.
### 3.2. Thuật Toán Co Giãn Thời Gian & Dịch Cao Độ (Phase Vocoder)
Để thực hiện tính năng thay đổi tốc độ (*Stretch*) mà không đổi cao độ (*Pitch*), hoặc dịch giọng (*Pitch shifting*) mà không đổi thời lượng:
* **Phân tích:** Thực hiện biến đổi Fourier thời gian ngắn (STFT) với cửa sổ Hanning chồng chập $75\%$ (*Overlap-Add*):
$$w[n] = 0.5 \cdot \left(1 - \cos\left(\frac{2\pi n}{N-1}\right)\right)$$
* **Dịch chuyển pha:** Tính toán sự sai lệch pha $\Delta \Phi$ giữa các khung (*frames*) liên tiếp để xác định tần số tức thời và thực hiện bù pha (*Phase Resynthesis*) theo tỷ lệ co giãn $S$:
$$S = \frac{\text{Duration}_{\text{new}}}{\text{Duration}_{\text{original}}}$$
* **Tổng hợp:** Tái thiết lập tín hiệu bằng thuật toán biến đổi ngược (ISTFT) và phương pháp cộng chồng chập (OLA — *Overlap-Add*) để tạo ra tệp âm thanh trơn tru, không bị méo dạng hay giật tiếng.
### 3.3. Thuật Toán Lọc Méo Tiếng & Compressor Động (Dynamics Processing)
Lập trình thuật toán Compressor/Limiter để kiểm soát biên độ đỉnh của tín hiệu tự động bằng cách tính toán mốc năng lượng RMS trung bình của cửa sổ tín hiệu:
$$x_{\text{RMS}} = \sqrt{\frac{1}{M}\sum_{k=0}^{M-1} x[n-k]^2}$$
Hệ số khuếch đại Gain áp dụng $G(t)$ được tính toán động dựa trên các tham số Threshold ($T_{\text{dB}}$), Ratio ($R$), Attack ($t_A$) và Release ($t_R$):
$$G_{\text{target}}(t) = \begin{cases} 0 & x_{\text{dB}} \le T_{\text{dB}} \\ (T_{\text{dB}} - x_{\text{dB}}) \cdot \left(1 - \frac{1}{R}\right) & x_{\text{dB}} > T_{\text{dB}} \end{cases}$$
---
## 4. Giải Pháp Biên Tập Không Phá Hủy (Non-Destructive Editing VFS)
Các phần mềm chuyên nghiệp không chỉnh sửa trực tiếp vào file WAV gốc trong suốt quá trình làm việc để tránh làm giảm chất lượng hoặc tiêu tốn RAM. Ta áp dụng kiến trúc Hệ thống tệp ảo phi tuyến (*Virtual Non-Linear File System - VFS*):
```text
[ Tệp âm thanh gốc trong RAM ] ──────────────────────────────────────────┐
│
[ Bảng chỉ mục liên kết phân đoạn (Non-Destructive Edit List - EDL) ] │
├── Phân đoạn 1: Đọc từ giây 0s -> 3.5s ──────────────────────────────┼─► [ Kết xuất ra Loa / Master ]
├── Phân đoạn 2: [SILENCE / KHOẢNG LẶNG] độ dài 1.2s │
└── Phân đoạn 3: Đọc từ giây 15s -> 22.4s (Đã đảo ngược - Reverse) ┘
```
### 4.1. Cơ chế hoạt động:
* Khi người dùng thực hiện lệnh Cut, Paste, Delete, hệ thống không xóa hay di chuyển bất kỳ byte dữ liệu nào trong mảng AudioBuffer gốc.
* Hệ thống chỉ cập nhật một danh sách chỉ mục bao gồm các đối tượng con trỏ định vị (*Edit Decision List - EDL*):
```json
[
{ "source_buffer_id": "track_1", "start_sample": 0, "length": 176400, "playback_rate": 1.0 },
{ "source_buffer_id": "silence", "start_sample": 0, "length": 44100, "playback_rate": 1.0 },
{ "source_buffer_id": "track_1", "start_sample": 882000, "length": 220500, "playback_rate": -1.0 }
]
```
* **Lợi ích:** Thao tác Undo/Redo diễn ra tức thời (*Instantaneous*) và tốn $0\text{ ms}$ bất kể tệp âm thanh dài hàng tiếng đồng hồ, do hệ thống chỉ cập nhật mảng JSON EDL siêu nhẹ mà không phải tính toán mảng mẫu nhị phân thô.
---
## 5. Hiện Thực Hóa Mã Nguồn DSP Chạy Trên WASM Client-Side
Dưới đây là thiết kế mã nguồn C++ mẫu (`core/dsp_engine.cpp`) được tối ưu hóa cao để biên dịch sang WebAssembly thông qua bộ dịch Emscripten, thực hiện xử lý âm thanh không độ trễ trực tiếp trong AudioWorklet trên trình duyệt:
```cpp
#include <emscripten.h>
#include <cmath>
#include <vector>
// Sử dụng EMSCRIPTEN_KEEPALIVE để giữ hàm khi biên dịch sang WASM
extern "C" {
/**
* Thuật toán áp dụng Volume Gain và Panning Hằng Số Năng Lượng (Constant-Power)
* Thao tác trực tiếp trên vùng nhớ RAM tuyến tính của WASM (WASM Linear Memory)
*/
EMSCRIPTEN_KEEPALIVE
void process_audio_block(
float* input_l, // Con trỏ kênh trái đầu vào
float* input_r, // Con trỏ kênh phải đầu vào
float* output_l, // Con trỏ kênh trái đầu ra
float* output_r, // Con trỏ kênh phải đầu ra
int block_size, // Kích thước khối (thường mặc định 128 mẫu trong Web Audio)
float volume_db, // Độ lớn âm lượng điều chỉnh (dB)
float pan // Vị trí panning từ -1.0 (Trái) đến 1.0 (Phải)
) {
// 1. Quy đổi dB sang hệ số nhân tuyến tính
float gain = powf(10.0f, volume_db / 20.0f);
// 2. Thuật toán Constant-Power Panning Law
// Quy đổi pan từ [-1.0, 1.0] sang góc quét theta [0, pi/2]
float theta = ((pan + 1.0f) / 2.0f) * (M_PI / 2.0f);
float gain_l = cosf(theta) * gain;
float gain_r = sinf(theta) * gain;
// 3. Thực thi tính toán vector hóa siêu tốc (SIMD-capable loop)
#pragma clang loop vectorize(enable)
for (int i = 0; i < block_size; ++i) {
output_l[i] = input_l[i] * gain_l;
output_r[i] = input_r[i] * gain_r;
}
}
}
```
---
## 6. Lộ Trình Triển Khai Chuyển Đổi Sang Client-Side (WASM DSP Pipeline)
Để dịch chuyển dự án từ mô hình xử lý nặng ở Server sang Client-side Audio Engine chuyên nghiệp, chúng ta triển khai theo 4 bước sau:
```text
[ GIAI ĐOẠN 1 ] ──► Tách biệt luồng UI và luồng Audio bằng AudioWorklet.
[ GIAI ĐOẠN 2 ] ──► Biên dịch các thư viện DSP C++/Rust sang WebAssembly (.wasm).
[ GIAI ĐOẠN 3 ] ──► Triển khai bảng chỉ mục EDL để hỗ trợ Undo/Redo phi tuyến tức thời.
[ GIAI ĐOẠN 4 ] ──► Tận dụng WebGL/WebGPU để kết xuất đồ thị sóng & spectrogram bằng GPU.
```
### 1. Triển khai AudioWorklet Node
Thay thế hoàn toàn bộ đệm vẽ cũ bằng cách đăng ký một `AudioWorkletProcessor` chạy trên luồng phụ để liên tục nạp dữ liệu và cấp phát tín hiệu nghe thử thời gian thực mà không làm nghẽn giao diện.
### 2. Biên dịch WASM Toolchain
Sử dụng Emscripten SDK để biên dịch mã nguồn C++ của các hiệu ứng (Reverb, Delay, Phase Vocoder) thành tệp `.wasm`. Frontend tải bất đồng bộ tệp này khi khởi chạy ứng dụng và ánh xạ trực tiếp vùng nhớ RAM tuyến tính của WASM vào luồng âm học của trình duyệt.
### 3. Tích hợp WebGL/WebGPU Render Sóng Âm
Thay vì thực hiện vẽ lại Canvas bằng CPU Main Thread thông qua Context 2D truyền thống (thường gây lag khi zoom sâu), chúng ta chuyển các tọa độ đỉnh mẫu sang bộ nhớ của GPU và sử dụng WebGL/WebGPU để kết xuất vectơ sóng âm ở tần số quét $60\text{ Hz} \rightarrow 120\text{ Hz}$ cực kỳ mượt mà tương tự như Sound Forge.
22_CLIENT_DESK.md
-188
View File
@@ -1,188 +0,0 @@
# Kế Hoạch Triển Khai Kỹ Thuật: Dockerized Music Processing Server & SonicForge Studio
Kế hoạch này đặc tả lộ trình triển khai, kiểm thử và đồng bộ hóa hai lõi động cơ: Động cơ Web Audio Client-side (nghe thử thời gian thực, tương tác đồ họa) và Động cơ Python Docker Server-side (xử lý VST/VSTi, render chất lượng cao, quản lý phân quyền và hạn mức lưu trữ Quota).
---
## 1. GIAI ĐOẠN 1: ĐỒNG BỘ ĐỒ HỌA & XỬ LÝ SÓNG ÂM KHÔNG TRỄ
Mục tiêu là đưa mảng nhị phân thô (`Float32Array`) vào bộ nhớ RAM của Client để vẽ đồ thị siêu thu phóng mượt mà và thực thi bắt sự kiện bôi đen vùng chọn.
### 1.1. Các Tác Vụ Phía Frontend (HTML5/React)
* **[ ] Vẽ Sóng Đa Thang Đo (Multi-Scale Waveform):**
* Tích hợp thuật toán hoán đổi đồ họa trong `index.html`.
* Khi zoom xa ($Z < 500$ px/s): Vẽ dải bao đỉnh (Peak Waveform).
* Khi siêu thu phóng ($Z \ge 500$ px/s): Vẽ đường cong hình sin đơn tuyến (Continuous Polyline) và các chấm mẫu tròn (Sample Nodes, bán kính $r = 2\text{ px}$) tại các tọa độ mẫu chính xác.
* **[ ] Vẽ Lưới Trục Decibel:** Dựng rõ rệt các vạch lưới ngang màu tối phân chia mốc biên độ: vạch dương +6.0 dB, vạch trung tâm -Inf. dB (Zero-Line), và vạch biên âm -6.0 dB.
* **[ ] Khóa Điểm Neo Shift+Click:**
* Triển khai React Ref độc lập `localSelectionAnchorRef` để khóa điểm nhấp chuột đầu tiên.
* Khi người dùng nhấp Shift+Click lần 2, tính toán dải phủ màu cục bộ trên duy nhất track đang hoạt động trong khoảng $[\min(T_{\text{anchor}}, T_{\text{end}}), \max(T_{\text{anchor}}, T_{\text{end}})]$.
* Chặn đứng sự kiện click playhead hoặc kéo clip khi có phím Shift được nhấn.
* **[ ] Hủy Vòng Lặp (Escape Loop):** Hỗ trợ tổ hợp `Ctrl + Click` chuột vào vùng trống ngoài dải chọn để hủy mốc neo, nhấn Spacebar phát nhạc tuyến tính vượt quá mốc lặp cũ.
### 1.2. Các Tác Vụ Phía Backend (Python / NumPy)
* **[ ] Port Thuật Toán Dò Zero-Crossing:** Viết hàm dò tìm điểm đổi dấu vật lý trong tệp `app/core/dsp_utils.py` bằng toán tử NumPy vector hóa để tối ưu hóa tốc độ:
$$x[i] \cdot x[i+1] \le 0$$
---
## 2. GIAI ĐOẠN 2: CHỈNH SỬA PHI TUYẾN TRÊN SUB-TAB CÔ LẬP
Thiết lập môi trường làm việc cô lập (Sandbox) cho phép người dùng click đúp vào Clip để mở một Tab phụ biên tập chi tiết không ảnh hưởng đến bản phối chính.
### 2.1. Quy Trình Trích Xuất & Thước Đo
* **[ ] Sandbox Splicing:** Khi double-click vào Clip, Frontend trích xuất mảng mẫu phụ (Sub-segment Buffer) và tạo một tab biên tập độc lập. Đặt lại thước đo thời gian Ruler của Tab này chạy từ $t = 0.0\text{ s}$.
* **[ ] Tương Tác Slider Thước Đo:** Dựng 4 thanh kéo ngang điều hướng:
* *Normalize Ceiling:* Trần chuẩn hóa từ $-12\text{ dBFS}$ đến $0\text{ dBFS}$.
* *Gain (dB) & Pitch Shift (Semitones):* Khuếch đại biên độ và dịch giọng.
* *Speed Stretch (%):* Co giãn thời lượng clip trực quan bằng cách nhấn giữ `Alt` rồi kéo biên phải của Clip. Hiển thị nhãn màu vàng `Speed: 75.0%`.
* **[ ] Bút Vẽ Volume (Pencil Tool):** Kích hoạt cây bút vẽ để hiển thị đường thẳng lục sáng mốc $0\text{ dB}$. Cho phép người dùng nhấp tạo các nút thắt điều khiển (Control Nodes) và kéo tăng ($+3\text{ dB}$) hoặc kéo giảm ($-30\text{ dB}$).
### 2.2. Hòa Mạng Apply & Merge Back Phía Server
* **[ ] Bộ Lọc Micro-Crossfade:** Khi người dùng nhấn Apply, dữ liệu đã chỉnh sửa được đồng bộ ngược lại dòng phối chính. FastAPI Server chạy Celery task áp dụng bộ lọc mờ biên Micro-crossfade có độ rộng $w = 10\text{ ms}$ tại hai đầu điểm ráp nối để triệt tiêu tiếng click/pop.
---
## 3. GIAI ĐOẠN 3: ĐỊNH TUYẾN MIDI, PLUGIN VST/VSTI & MIXER
Tích hợp bộ soạn thảo MIDI Piano Roll, nạp nhạc cụ ảo, hiệu ứng và điều phối âm lượng đa kênh.
### 3.1. MIDI Items & Piano Roll Editor
* **[ ] Piano Roll Canvas:** Thiết lập giao diện lưới nốt nhạc có trục đứng $Y$ biểu diễn cao độ từ $0 \rightarrow 127$ (phím piano) và trục ngang $X$ biểu diễn lưới phách (Beats) đồng bộ với Tempo.
* **[ ] Thao Tác Lưới:** Cho phép nhấp chuột để thêm nốt nhạc, click chuột phải/nhấp đúp để xóa nốt, kéo hai đầu để thay đổi độ dài (`duration_beats`).
### 3.2. Động Cơ Định Tuyến VST / VSTi Trên Docker Linux
* **[ ] Nạp VSTi (Nhạc cụ ảo):** Cấu hình thư viện `pedalboard` ở Python Backend để nạp các tệp tin `.vst3` nhạc cụ ảo trên Linux, tiếp nhận sự kiện MIDI từ Piano Roll, tổng hợp âm và xuất ra mảng NumPy Stereo.
* **[ ] Nạp VST Effects (EQ/Reverb):** Hỗ trợ ghim chuỗi hiệu ứng nối tiếp gộp cả Stock WASM và Native VST3.
* **[ ] Giao Diện Mixer Panel Đa Kênh:** Dựng bảng mixer ở đáy màn hình hiển thị Master Bus, Track Audio, Track MIDI và Track FX Send/Return. Mỗi track có thước đo tín hiệu (Level Meter) dao động thời gian thực.
---
## 4. GIAI ĐOẠN 4: HỆ THỐNG PHÂN QUYỀN, QUOTA & ADMIN CONTROL
Xây dựng lớp bảo mật bảo vệ tài nguyên ổ đĩa máy chủ, quản lý người dùng và cờ tính năng (Feature Flags).
### 4.1. Phân Quyền & Quản Lý Quota
* **[ ] Bắt Buộc Đổi Mật Khẩu Lần Đầu (First-Time Login):**
* Khi tài khoản Admin/User được khởi tạo với mật khẩu mặc định từ môi trường Docker, hệ thống đặt cờ `must_change_password = True` trong database SQL.
* Middleware của FastAPI sẽ chặn đứng mọi yêu cầu xử lý nhạc, ép người dùng thực hiện đổi mật khẩu ở lần đăng nhập đầu tiên mới mở khóa hệ thống.
* **[ ] Admin Quotas:** Tích hợp bộ kiểm soát hạn mức dung lượng ổ đĩa lưu trữ ($S_{\text{limit}}$). Python sẽ tính toán tổng kích thước mảng nhị phân trước khi cho phép tải tệp lên:
$$S_{\text{used}} + S_{\text{new}} \le S_{\text{limit}}$$
* **[ ] Feature Flags:** Hỗ trợ Admin bật/tắt nóng các tính năng cao cấp (như xuất bản WAV 24-bit, AI generation) thông qua bảng cấu hình DB.
### 4.2. Cấu Hinh Headless JUCE VST Rendering
* **[ ] Docker Xvfb Display:** Bổ sung cấu hình màn hình ảo Xvfb (X Virtual Framebuffer) vào tệp Dockerfile để container nạp thành công các VST3 nhạc cụ và hiệu ứng biên dịch bằng C++ (JUCE framework) trên Linux mà không bị lỗi crash liên kết X11.
---
## 5. KIỂM THỬ XÁC MINH DANH TÍNH
| Mô-đun kiểm thử | Phương pháp thực thi | Tiêu chuẩn đạt (KPI) |
| --- | --- | --- |
| **Kiểm thử Zoom & Sóng** | Phóng to tối đa một bài nhạc $44.1\text{ kHz}$. | Nhìn thấy rõ hạt mẫu tròn màu xanh và dải lưới Decibel đối xứng. |
| **Kiểm thử Shift+Click** | Bôi chọn cục bộ và Master Loop trên thước Ruler. | Nhấn Spacebar lặp mượt mà, nhấn `Ctrl+Click` để hủy dải chọn. |
| **Kiểm thử Zero-Crossing** | Cắt lát nhạc bằng AI Cut ở mốc giây lẻ. | Tệp WAV kết xuất không có bất kỳ tiếng lách tách (click/pop) nào. |
| **Kiểm thử Docker VSTi** | Gửi chuỗi MIDI nốt và nạp một Virtual Synth VST3. | Kết xuất thành công tệp WAV Stereo có âm thanh nhạc cụ ảo. |
| **Kiểm thử Bảo Mật Auth** | Đăng nhập tài khoản mặc định và gọi API Mix nhạc. | Hệ thống trả về lỗi HTTP 403 Forbidden bắt buộc đổi mật khẩu. |
| **Kiểm thử Quota** | Cố tình tải lên tệp âm thanh nặng vượt giới hạn. | Trả về lỗi *Dung lượng lưu trữ vượt quá giới hạn Quota của bạn.* |
---
## Kế hoạch Cấu hình Dockerfile Hợp nhất (Có Xvfb Headless)
Để chuẩn bị môi trường chạy thật cho động cơ xử lý âm thanh bản địa (Native DSP) tích hợp VSTi/VST3 C++ thông qua Python Pedalboard, tệp tin `Dockerfile` của dự án bắt buộc phải được thiết lập màn hình ảo Xvfb để tránh crash liên kết đồ họa:
```dockerfile
# Sử dụng Python 3.11 làm nền tảng
FROM python:3.11-slim
# Cài đặt các gói thư viện đồ hoạ và asound bắt buộc đối với JUCE / VST3 Linux
RUN apt-get update && apt-get install -y \
libgl1-mesa-glx \
libglu1-mesa \
libasound2 \
libjack-jackd2-0 \
libfreetype6 \
libfontconfig1 \
libx11-6 \
libxext6 \
libxinerama1 \
libxrandr2 \
libxcursor1 \
xvfb \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# Thiết lập biến môi trường hiển thị cho X11 ảo
ENV DISPLAY=:99
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Khởi chạy Xvfb ảo ở cổng :99 trước khi kích hoạt FastAPI / Celery
CMD ["sh", "-c", "Xvfb :99 -screen 0 1024x768x16 & python app/main.py"]
```
23_FIX_ZOOM.md
-87
View File
@@ -1,87 +0,0 @@
# Technical Analysis: Canvas Dimension Overflow During Ultra-Zoom
This document analyzes the root cause of the graphical failure that occurs when users perform an ultra-zoom operation on audio files of varying durations (1 second versus over 5 seconds). This anomaly leads to a rendering crash and turns the entire track lane completely blank white at maximum zoom levels.
---
## 1. The Root Cause: Browser Canvas Dimension Limits
This phenomenon is not a standard programming logic error, but rather a physical hardware limitation of modern web browsers (Chrome, Firefox, Safari) when interacting with the GPU (Graphics Card).
### 1.1. Physical Canvas Width Calculation Formula
In traditional DAW user interface architectures, the actual physical width of a waveform lane, $W_{\text{canvas}}$ (measured in pixels), is calculated dynamically based on the clip duration, $T_{\text{clip}}$ (seconds), and the zoom scale factor, $Z$ (pixels/second):
$$W_{\text{canvas}} = T_{\text{clip}} \times Z$$
### 1.2. Browser Maximum Canvas Size Constraints ($W_{\text{limit}}$)
To optimize performance, browsers leverage the GPU for hardware acceleration, managing the `<canvas>` element as a specialized GPU Texture mapping block. Consequently, each browser and operating system sets an absolute maximum physical size boundary for the canvas element ($W_{\text{limit}}$).
This maximum ceiling typically ranges within:
* $16,384\text{ px}$ (on mobile devices or lower-end configurations).
* $32,768\text{ px}$ (on modern desktop browsers).
If the calculated width of the canvas exceeds this physical hardware ceiling ($W_{\text{canvas}} > W_{\text{limit}}$):
* The browser fails to allocate additional graphical memory or texture space.
* The underlying WebGL core or Canvas 2D Rendering Context suffers an immediate **Context Loss**.
* The entire display region of the canvas collapses and reverts to its default uninitialized hardware state: becoming completely blank white or entirely transparent.
---
## 2. Analysis of the Variance Between 1-Second and 5-Second Files
Assume an operator triggers an ultra-zoom action to an extreme deep magnification level of $Z = 10,000\text{ pixels/second}$ to monitor discrete sample node metrics:
### 2.1. Ultra-Short Audio Files (1 Second)
Applying the width calculation formula:
$$W_{\text{canvas\_1s}} = 1.0\text{ s} \times 10,000\text{ px/s} = 10,000\text{ px}$$
* **Result:** Because $10,000\text{ px} < 32,768\text{ px}$ (safely below the maximum hardware threshold), the browser allocates the texture memory cache perfectly. Users can zoom in completely to view discrete green sample nodes cleanly rendered on top of smooth sinusoidal phases.
### 2.2. Longer Audio Files (e.g., 5 Seconds or 10 Seconds)
Applying the width calculation formula at the identical zoom factor of $Z = 10,000\text{ px/s}$:
$$W_{\text{canvas\_5s}} = 5.0\text{ s} \times 10,000\text{ px/s} = 50,000\text{ px}$$
$$W_{\text{canvas\_10s}} = 10.0\text{ s} \times 10,000\text{ px/s} = 100,000\text{ px}$$
* **Result:** Both evaluated dimensions ($50,000\text{ px}$ and $100,000\text{ px}$) **drastically exceed the maximum boundary constraint** ($W_{\text{limit}} = 32,768\text{ px}$) enforced by the GPU.
* The instant the user pushes the magnification past this safety threshold, the browser overloads its hardware texture buffer, drops the canvas rendering context, and flashes the entire track lane into a **blank white void** (destroying waveform lines and grid displays entirely).
---
## 3. The Fixed-Viewport Canvas Architecture Solution
To eliminate this memory overflow anomaly permanently and allow users to zoom in infinitely across multi-hour audio files without encountering blank screen crashes, the layout engine must abandon the paradigm of scaling the physical canvas element width to match the audio clip length.
### Professional DAW Solution: **Viewport-Only Canvas Architecture**
```text
EDITOR VIEWPORT SCREEN (Fixed Width: 1200px)
|<────────────────────────── Physical Canvas Viewport ──────────────────────────>|
+────────────────────────────────────────────────────────────────────────────────+
| Waveform is painted dynamically based on the scrollLeft offset |
| |
| [ Render localized sample slice from RAM ] |
| |
+────────────────────────────────────────────────────────────────────────────────+
```
1. **Rigid Canvas Sizing Constraints:**
The physical dimension width of the `<canvas>` tag must never be allowed to stretch according to zoom ratios. It must remain strictly locked to match the exact visible horizontal window boundary of the user's viewport (e.g., $W_{\text{canvas}} = W_{\text{viewport}} \approx 1200\text{ px}$).
2. **Intelligent Slicing Redraw (Slicing Render):**
When a horizontal navigation event occurs (`scrollLeft`), the engine avoids shifting the physical canvas layout. Instead, it alters the offset index of the sample array queried for the drawing loop:
* **Visible Window Starting Index:** $T_{\text{start}} = \frac{\text{scrollLeft}}{Z}$
* **Visible Window Terminating Index:** $T_{\text{end}} = \frac{\text{scrollLeft} + W_{\text{viewport}}}{Z}$
The layout routine isolates only the localized sample chunk mapping within the interval $[T_{\text{start}}, T_{\text{end}}]$ straight from client-side RAM, rendering it directly over the fixed $1200\text{ px}$ canvas envelope.
* **Absolute Advantages:** Because the canvas physical size is permanently pinned to a lightweight display footprint ($1200\text{ px}$), the system **consumes a minimal, static fraction of GPU memory**. It can never exceed hardware boundaries, permanently eradicating the blank track lane bug and enabling a fluid $120\text{ FPS}$ refresh cycle regardless of total audio track length.
24_CURSOR_ZOOM.md
-174
View File
@@ -1,174 +0,0 @@
# Technical Specification: Playhead-Centering Zoom Algorithm
This document specifies the playhead drifting phenomenon during zoom operations and provides the architectural solutions, mathematical formulations, and source code prototypes required to lock the playback cursor as a static physical anchor point on the screen throughout timeline magnification updates.
---
## 1. Visual Symptom & Playhead Drifting Analysis
In standard digital audio workstation (DAW) graphical user interfaces, when an operator executes a mouse wheel zoom gesture (Zoom In/Out), the layout layout engine defaults to treating the leftmost physical pixel coordinate ($0$) of the timeline as the boundary axis for scaling.
### 1.1. Visual Failure Manifestations:
* **During Zoom In:** The red playback cursor (Playhead) positioned at a specific timestamp (e.g., $4.00\text{ s}$) is rapidly shifted toward the right perimeter of the viewport until it flies completely out of view.
* **During Zoom Out:** The playhead is abruptly snapped back toward the left perimeter of the screen viewport.
* **Consequence:** The sound engineer is forced to continuously adjust the horizontal scrollbar (`scrollLeft`) to find the playhead location, severely breaking the workflow during detail editing blocks.
### 1.2. Target Layout State (Playhead-Centering Zoom):
Throughout mouse-driven zoom updates at any scale:
* The playback cursor (Playhead) must act as a static physical anchor point locked to its exact pixel position relative to the visible browser window viewport.
* The multi-channel waveform graphics must stretch or compress symmetrically around the vertical axis of the playback cursor.
---
## 2. Mathematical Modeling for Playhead Anchoring
To guarantee that the on-screen placement of the cursor maps identically before and after a modification to the viewport magnification ratio, we establish a system of equations conserving the pixel coordinates of the playhead.
### 2.1. Operational Variables Mapping:
* $t_{\text{playhead}}$ (seconds): The instantaneous runtime clock position of the playhead (e.g., $4.00\text{ s}$).
* $Z_{\text{current}}$ (px/s): The initial timeline horizontal scaling zoom factor before resizing.
* $Z_{\text{new}}$ (px/s): The target timeline horizontal scaling zoom factor after resizing.
* $S_{\text{current}}$ (pixels): The current initial horizontal scroll offset (`scrollLeft`) of the timeline view.
* $S_{\text{new}}$ (pixels): The target adjusted horizontal scroll offset calculated to overwrite the container state.
* $X_{\text{viewport}}$ (pixels): The physical offset tracking the distance from the left edge of the screen viewport container to the playhead rendering path line.
### 2.2. Coordinate Conservation Formula
The absolute spatial coordinate of the playhead on the global arrangement timeline maps to:
$$X_{\text{absolute}} = t_{\text{playhead}} \times Z$$
The actual visible screen viewport placement of the cursor before executing the zoom factor modification evaluates to:
$$X_{\text{viewport}} = (t_{\text{playhead}} \times Z_{\text{current}}) - S_{\text{current}}$$
To lock the playhead directly to its coordinate position post-zoom ($Z_{\text{new}}$), the variable value $X_{\text{viewport}}$ must remain strictly unchanged:
$$X_{\text{viewport}} = (t_{\text{playhead}} \times Z_{\text{new}}) - S_{\text{new}}$$
Solving the equation systems to calculate the target adjusted scroll offset parameter $S_{\text{new}}$:
$$S_{\text{new}} = (t_{\text{playhead}} \times Z_{\text{new}}) - X_{\text{viewport}}$$
Substituting the initial definition statement of $X_{\text{viewport}}$ back into the calculation loop:
$$S_{\text{new}} = (t_{\text{playhead}} \times Z_{\text{new}}) - \left( (t_{\text{playhead}} \times Z_{\text{current}}) - S_{\text{current}} \right)$$
Compiling the final optimized mathematical reduction model:
$$S_{\text{new}} = S_{\text{current}} + t_{\text{playhead}} \times (Z_{\text{new}} - Z_{\text{current}})$$
*Physical Property Significance:* The calculated target scrollbar position equals the current scroll offset augmented by the absolute coordinate displacement of the playhead triggered by the variance across magnification scales.
---
## 3. Frontend Client Integration Blueprint (React / HTML5)
This mathematical alignment routine is tied directly into the primary mouse `wheel` event handler capturing timeline zoom interactions inside the main `index.html` structure:
```javascript
// Timeline wheel interaction handling segment capturing Playhead-anchored Zoom
const handleTimelineZoom = (e) => {
// Restrict zoom loops exclusively to situations where Ctrl (or Cmd) modifiers are engaged
if (!e.ctrlKey) return;
e.preventDefault();
const timelineWrapper = timelineWrapperRef.current;
if (!timelineWrapper) return;
// 1. Capture absolute layout dimensions before updating state variables
const scrollLeftCurrent = timelineWrapper.scrollLeft;
const zoomCurrent = zoom; // Maps to Z_current
const playheadTime = currentTime; // Maps to t_playhead
// 2. Evaluate target zoom ratio step updates (Enforces fluid scaling profiles)
const zoomFactor = e.deltaY > 0 ? 0.9 : 1.1;
let zoomNew = zoomCurrent * zoomFactor;
// Rigidly clamp calculation bounds within safe operating limits
const minZoomLimit = viewportWidth / maxDuration;
const maxZoomLimit = 2000; // Mitigates graphical memory canvas texture crashes
zoomNew = Math.max(minZoomLimit, Math.min(maxZoomLimit, zoomNew));
// 3. Apply the conservation formula to calculate S_new scroll offsets
const scrollLeftNew = scrollLeftCurrent + playheadTime * (zoomNew - zoomCurrent);
// 4. Propagate updated values synchronously down to State queues and the DOM
setZoom(zoomNew);
// Defer scroll alignment to requestAnimationFrame to execute right as Canvas buffers redraw
requestAnimationFrame(() => {
timelineWrapper.scrollLeft = scrollLeftNew;
});
};
```
---
## 4. Desktop Application Integration Manual (Python PyQt6 / PySide6)
When porting this layout algorithm to a containerized Python desktop context, capture the native `wheelEvent` tracking loop of the underlying `QGraphicsView` or `QScrollArea` layout wrapper:
```python
# [PYTHON PORTING BLUEPRINT] - Lock-step Playhead Zoom tracking over PyQt6 QGraphicsView
from PyQt6.QtWidgets import QGraphicsView, QScrollBar
from PyQt6.QtCore import Qt
class ProAudioTimelineView(QGraphicsView):
def __init__(self, parent=None):
super().__init__(parent)
self.playhead_time_seconds = 4.0 # Maps to t_playhead parameter
self.zoom_level = 100.0 # Maps to Z_current constant (pixels/second)
def wheelEvent(self, event):
# Inspect for active hardware keyboard ControlModifier keys
if event.modifiers() & Qt.KeyboardModifier.ControlModifier:
event.accept()
# 1. Capture absolute workspace metrics before calculating adjustments
h_scrollbar = self.horizontalScrollBar()
scroll_current = h_scrollbar.value() # Maps to S_current
zoom_current = self.zoom_level
t_playhead = self.playhead_time_seconds
# 2. Evaluate target scaling ratio increments
angle_delta = event.angleDelta().y()
zoom_factor = 1.1 if angle_delta > 0 else 0.9
zoom_new = max(10.0, min(2000.0, zoom_current * zoom_factor))
# 3. Apply the coordinate conservation model to isolate scroll_new offsets
scroll_new = scroll_current + t_playhead * (zoom_new - zoom_current)
# 4. Overwrite parameters and prompt vector updates on the QPainter surface
self.zoom_level = zoom_new
self.update_timeline_graphics() # Invokes the multi-channel waveform redraw routines
# Commit updated scroll values immediately to lock playhead layout tracking
h_scrollbar.setValue(int(scroll_new))
else:
# Drop down to default native vertical/horizontal scroll handling patterns
super().wheelEvent(event)
```
---
## 5. UI Operational State Comparison
Based on the verified structural architecture of the system layout:
* **Baseline Initial State:** Audio waveform paths render at standard macro scaling bounds (evaluating approximately to a few hundred pixel columns per second of timeline data). The distinct vertical red playback cursor path line tracking the $4.00\text{ s}$ clock milestone renders centered in the visible workspace view.
* **Post Maximum Zoom-In State:** Symmetrical audio waveform data lines stretch horizontally to their maximum viewport scaling boundaries (exposing granular peak structures explicitly). By executing the conservation equations defined in Section 2.2, the horizontal scroll container shifts rightward, keeping the red cursor line locked to its absolute pixel column coordinate on the screen instead of letting it slip past the viewport limits.
This technical spec document establishes the supreme design token rules for compiling and verifying zooming workflows on the arrangement canvas.
25_ZOOM.md
-121
View File
@@ -1,121 +0,0 @@
Here is the translation of the document into English Markdown format:
# Geometric Analysis: Progressive Center Drift During Asymmetrical Zoom & Pre-Roll Gutter Solution
This document analyzes the mathematical root cause of center drift during zoom operations at asymmetric timeline markers (e.g., zooming at $1\text{ s}$ drifts drastically compared to $5\text{ s}$ on a $10\text{ s}$ total track length). It also provides a structural solution using boundary margins (**Pre-roll/Post-roll Gutter**) to lock the absolute anchor point in all interaction scenarios.
---
## 1. Mathematical Proof: Why Zooming at $1\text{ s}$ Drifts Further Than $5\text{ s}$
This visual discrepancy is not caused by random calculation precision errors, but is the mathematical result of boundary clamping (**Scroll Left Clamping**).
### 1.1. Conservation Equation for Mouse/Playhead Anchor Points
To preserve the visual location of time marker $t$ at pixel coordinate $X_{\text{viewport}}$ relative to the display before and after changing the zoom scale factor ($Z_{\text{current}} \rightarrow Z_{\text{new}}$), the required horizontal scroll offset $S_{\text{new}}$ (`scrollLeft`) must satisfy:
$$S_{\text{new}} = (t \times Z_{\text{new}}) - X_{\text{viewport}}$$
### 1.2. Scenario Analysis: Zooming Out at $X_{\text{viewport}} = 300\text{ px}$ (Cursor at Screen Center)
Assume the timeline is zoomed out significantly, reducing the zoom ratio down to $Z_{\text{new}} = 100\text{ px/second}$.
#### Scenario A: Operator zooms at the central symmetrical coordinate $t = 5.0\text{ s}$
Applying the target scroll position calculation:
$$S_{\text{new}} = (5.0 \times 100) - 300 = 500 - 300 = +200\text{ px}$$
* **Result:** Because $+200\text{ px} \ge 0$, the scroll position resides safely within physical boundary limits. The browser sets `scrollLeft = 200` smoothly. The $5.0\text{ s}$ point remains locked at position $300\text{ px}$ on the screen with a spatial drift of $0\text{ px}$.
#### Scenario B: Operator zooms at an asymmetrical coordinate near the left edge $t = 1.0\text{ s}$
Applying the target scroll position calculation:
$$S_{\text{new}} = (1.0 \times 100) - 300 = 100 - 300 = -200\text{ px}$$
* **Critical Issue:** Browsers and operating hardware cannot execute negative scroll values ($scrollLeft < 0$), instantly **clamping the horizontal scroll position at the minimum boundary $S_{\text{clamped}} = 0\text{ px}$**.
* Due to this clamping, the actual on-screen rendering coordinate of the $1.0\text{ s}$ milestone drifts to:
$$X_{\text{viewport\_actual}} = (1.0 \times 100) - 0 = 100\text{ px}$$
* **Visual Discrepancy:** The $1.0\text{ s}$ marker, which should remain stationary at coordinate $300\text{ px}$, is **pulled to the left to coordinate $100\text{ px}$** (resulting in a spatial shift of $200\text{ px}$).
> **Geometric Principle:** The smaller the zoom anchor timestamp $t$ (the closer it sits to the left boundary), the more likely the required scroll position $S_{\text{new}}$ drops below zero to be clamped at $0$, increasing visual waveform displacement during zoom-out operations.
---
## 2. Professional DAW Solution: Pre-Roll & Post-Roll Gutters
To permanently eliminate this behavior and give SonicForge Studio a professional zoom experience similar to Reaper or Adobe Audition, apply a **Pre-roll & Post-roll Gutter (Boundary Margins)**.
```text
|<─────────────────── Actual Timeline Scroll Width ───────────────────>|
+──────────────────────────┬───────────────────────────────────────────+
| [ Pre-roll Gutter ] │ 0:00.000 (Actual music start time) |
| (Width: W_viewport) │ |
| (scrollLeft can run here)│ [ Waveform and track grid start here... ]|
+──────────────────────────┴───────────────────────────────────────────+
▲
│ [ 1.0s anchor point remains 100% stationary here ]
│ Because the scrollbar is allowed to retreat negatively into the gutter!
```
1. **Enabling Visual Negative Scrolling:** Instead of starting the timeline canvas at pixel coordinate $0\text{ px}$ (corresponding to $0.0\text{ s}$), prepend an empty padding region (**Gutter**) equal to the full viewport width $W_{\text{viewport}}$ (e.g., $1200\text{ px}$) before the $0.0\text{ s}$ mark.
2. **Updated Coordinate Mapping Formula:**
The physical pixel coordinate $X$ of timestamp $t$ on the Canvas includes the offset padding:
$$X_t = (t \times Z) + W_{\text{pre\_roll}}$$
3. **Unclamped Scroll Conservation Equation:**
When zooming at any asymmetrical timestamp (including $0.1\text{ s}$ or $0.0\text{ s}$):
$$S_{\text{new}} = (t \times Z_{\text{new}}) + W_{\text{pre\_roll}} - X_{\text{viewport}}$$
* Because $W_{\text{pre\_roll}}$ is added, $S_{\text{new}}$ remains greater than $0$ during standard zoom-out actions, eliminating the clamp at $0$. Your $1.0\text{ s}$ timestamp or playhead stays stationary, the waveform graphics scale symmetrically, and the $0.0\text{ s}$ mark smoothly recedes toward the center of the viewport, exposing a subtle, professional dark gray pre-roll gutter area in front of the track.
---
## 3. Implementing the Boundary Lock Algorithm in Source Code
Below is the upgraded mouse wheel zoom event handler for `index.html`, incorporating pre-roll margin compensation:
```javascript
const handleTimelineZoomWithGutter = (e) => {
if (!e.ctrlKey) return;
e.preventDefault();
const timelineWrapper = timelineWrapperRef.current;
if (!timelineWrapper) return;
const rect = timelineWrapper.getBoundingClientRect();
const mouseXInViewport = e.clientX - rect.left;
// Pre-roll gutter padding equal to half the viewport width to allow scrolling past 0s
const preRollPadding = rect.width / 2;
const scrollLeftCurrent = timelineWrapper.scrollLeft;
const zoomCurrent = zoom;
const anchorTime = (scrollLeftCurrent + mouseXInViewport - preRollPadding) / zoomCurrent;
const zoomFactor = e.deltaY > 0 ? 0.9 : 1.1;
let zoomNew = zoomCurrent * zoomFactor;
// Apply zoom constraints
zoomNew = Math.max(minZoom, Math.min(2000, zoomNew));
// Calculate new scroll offset preserving the anchor point under the cursor
const scrollLeftNew = (anchorTime * zoomNew) + preRollPadding - mouseXInViewport;
// Update state
setZoom(zoomNew);
requestAnimationFrame(() => {
timelineWrapper.scrollLeft = scrollLeftNew;
});
};
```
This upgrade enables SonicForge Studio to achieve zero-latency, sample-accurate zooming with studio-grade anchor locking!
26_ZOOM_HTML.md
-77
View File
@@ -1,77 +0,0 @@
## 💡 Nguyên lý tính toán đúng (Zoom to Mouse Pointer)
Để điểm dưới con trỏ chuột đứng yên tại đúng vị trí đó sau khi zoom, bạn cần giữ nguyên **tỷ lệ thời gian (time ratio)** tại điểm con trỏ chuột so với chiều rộng hiện tại của vùng hiển thị (Viewport).
### **Công thức chuyển đổi:**
Giả sử thanh cuộn (Scrollbar) có vị trí xả hiện tại là `scrollLeft`:
1. **Tìm điểm thời gian tương đối tại vị trí chuột ($T_{mouse}$):**
$$T_{mouse} = \text{scrollLeft} + X_{mouse\_in\_canvas}$$
2. **Tính tỷ lệ zoom mới ($S_{new} / S_{old}$):**
$$\text{ratio} = \frac{\text{scale}_{new}}{\text{scale}_{old}}$$
3. **Cập nhật vị trí cuộn mới (`scrollLeft_{new}`):**
$$\text{scrollLeft}_{new} = (T_{mouse} \times \text{ratio}) - X_{mouse\_in\_canvas}$$
---
## 🛠️ Code mẫu ngắn gọn (Pure JS / Canvas)
Dưới đây là đoạn code lắng nghe sự kiện `wheel` (lăn chuột) trên Waveform Canvas/Container để xử lý zoom đúng chuẩn các phần mềm DAW:
```javascript
const container = document.getElementById('waveform-container');
let pixelsPerSecond = 100; // Tỉ lệ Zoom ban đầu (mức Zoom)
container.addEventListener('wheel', (e) => {
// Chỉ thực hiện zoom khi giữ phím Ctrl (hoặc bạn có thể bỏ condition này nếu muốn lăn chuột là zoom)
if (!e.ctrlKey) return;
e.preventDefault();
// 1. Lấy vị trí con trỏ chuột so với viền trái của Waveform Container (Viewport)
const rect = container.getBoundingClientRect();
const mouseX = e.clientX - rect.left;
// 2. Tính tọa độ thời gian (giây) tại điểm con trỏ chuột đang chỉ vào
const currentScrollLeft = container.scrollLeft;
const timeAtMouse = (currentScrollLeft + mouseX) / pixelsPerSecond;
// 3. Tính tỉ lệ zoom mới (Phóng to / Thu nhỏ)
const zoomFactor = e.deltaY < 0 ? 1.2 : 0.8; // Lăn lên = phóng to, lăn xuống = thu nhỏ
const newPixelsPerSecond = Math.max(10, Math.min(2000, pixelsPerSecond * zoomFactor));
// 4. Cập nhật tỉ lệ zoom mới vào ứng dụng
pixelsPerSecond = newPixelsPerSecond;
// (Thực hiện render lại Waveform với pixelsPerSecond mới tại đây)
renderWaveform();
// 5. CẬP NHẬT SCROLLBAR: Cuộn lại sao cho điểm 'timeAtMouse' vẫn nằm đúng ở 'mouseX'
container.scrollLeft = (timeAtMouse * pixelsPerSecond) - mouseX;
}, { passive: false });
```
---
## 📌 Nhắc nhở thêm nếu dùng thư viện:
* **Nếu bạn dùng Canvas thuần:** Đảm bảo hàm `renderWaveform()` vẽ lại waveform dựa theo `pixelsPerSecond` mới trước khi cập nhật `container.scrollLeft`.
* **Nếu bạn đang dùng `wavesurfer.js`:** Thư viện này đã hỗ trợ sẵn logic này, bạn chỉ cần dùng method:
```javascript
wavesurfer.zoom(newPxPerSec);
```
*(Nếu WaveSurfer bản cũ bị trôi, bạn áp dụng lại công thức tính `scrollLeft` ở trên sau khi gọi lệnh `zoom()`)*.
27_ZOOM_TECH.md
-186
View File
@@ -1,186 +0,0 @@
# Giải pháp Virtual Viewport Rendering cho Waveform Zoom
Phương pháp này sử dụng kỹ thuật **Virtual Viewport Rendering** (Rendering theo vùng nhìn).
### Cơ chế hoạt động:
1. **Thanh cuộn ảo (Virtual Scrollbar):** Duy trì một thẻ `div` ẩn (hoặc gán chiều rộng cho container) bằng chiều rộng lý thuyết của toàn bộ file audio khi zoom. Nhưng **Canvas thực tế thì luôn cố định chiều rộng bằng khung nhìn (Viewport)**.
2. **Xử lý phần ẩn:** Các phần ngoài khung nhìn sẽ **không được vẽ/render lên Canvas**. Dữ liệu âm thanh gốc (`Audio Buffer` / `Array Data`) vẫn nằm nguyên trong bộ nhớ (RAM/JS Array), không bị ảnh hưởng.
3. **Khi Zoom Out:** Tính toán lại khoảng thời gian `[startTime, endTime]` rộng hơn, lấy mảng dữ liệu sample tương ứng trong khoảng đó và vẽ đè lại lên Canvas.
---
## 1. Kiến trúc tổng quan
```text
[ Toàn bộ Audio Buffer trong Memory: 0s ----------------------> 180s ]
| Khung nhìn |
v (Canvas Fixed) v
[startTime ------------> endTime]
```
---
## 2. Mã nguồn triển khai (Pure HTML5 & JS)
Đoạn code bên dưới minh họa cơ chế zoom chính xác tại vị trí con trỏ chuột mà không sợ quá tải Canvas hay nhảy vị trí:
```html
<!DOCTYPE html>
<html lang="vi">
<head>
<meta charset="UTF-8">
<style>
#viewport {
width: 800px; /* Chiều rộng khung nhìn cố định */
height: 150px;
overflow-x: auto; /* Hiện thanh cuộn */
position: relative;
background: #1e1e1e;
}
/* Container giả lập chiều rộng thực tế để tạo thanh cuộn */
#virtual-content {
height: 1px;
pointer-events: none;
}
/* Canvas cố định vị trí luôn đè theo khung nhìn */
#waveform-canvas {
position: sticky;
left: 0;
top: 0;
width: 800px;
height: 150px;
display: block;
}
</style>
</head>
<body>
<div id="viewport">
<div id="virtual-content"></div>
<canvas id="waveform-canvas" width="800" height="150"></canvas>
</div>
<script>
// --- GIẢ LẬP DỮ LIỆU AUDIO (Audio Buffer / Sample Data) ---
const AUDIO_DURATION = 60; // Audio dài 60 giây
const SAMPLE_RATE = 100; // 100 samples/giây
const audioSamples = new Float32Array(AUDIO_DURATION * SAMPLE_RATE);
// Tạo sóng âm giả lập
for (let i = 0; i < audioSamples.length; i++) {
audioSamples[i] = Math.sin(i * 0.05) * 0.8;
}
// --- KHAI BÁO BIẾN TRẠNG THÁI ---
const viewport = document.getElementById('viewport');
const virtualContent = document.getElementById('virtual-content');
const canvas = document.getElementById('waveform-canvas');
const ctx = canvas.getContext('2d');
const VIEWPORT_WIDTH = 800;
const VIEWPORT_HEIGHT = 150;
let pixelsPerSecond = 100; // Mức zoom ban đầu (100px = 1s)
// --- HÀM 1: CHỈ VẼ PHẦN HIỂN THỊ TRONG KHUNG NHÌN ---
function renderVisibleWaveform() {
// 1. Cập nhật độ dài ảo cho thanh cuộn
const totalWidth = AUDIO_DURATION * pixelsPerSecond;
virtualContent.style.width = `${totalWidth}px`;
// 2. Xác định khoảng thời gian đang nằm trong khung nhìn (Viewport)
const scrollLeft = viewport.scrollLeft;
const startTime = scrollLeft / pixelsPerSecond;
const endTime = (scrollLeft + VIEWPORT_WIDTH) / pixelsPerSecond;
// 3. Xóa Canvas cũ
ctx.clearRect(0, 0, VIEWPORT_WIDTH, VIEWPORT_HEIGHT);
ctx.fillStyle = '#00ffcc';
// 4. Lấy các sample âm thanh tương ứng trong khoảng [startTime, endTime]
const startSampleIndex = Math.floor(startTime * SAMPLE_RATE);
const endSampleIndex = Math.ceil(endTime * SAMPLE_RATE);
// 5. Vẽ đúng các sample này lên Canvas (Vẽ từ x = 0 đến VIEWPORT_WIDTH)
const middleY = VIEWPORT_HEIGHT / 2;
for (let i = startSampleIndex; i < endSampleIndex; i++) {
if (i < 0 || i >= audioSamples.length) continue;
// Thời gian của sample này
const sampleTime = i / SAMPLE_RATE;
// Tọa độ X trên Canvas cố định (đã trừ đi scrollLeft)
const x = (sampleTime * pixelsPerSecond) - scrollLeft;
// Chiều cao cột sóng âm
const amplitude = audioSamples[i] * (VIEWPORT_HEIGHT / 2);
ctx.fillRect(x, middleY - amplitude / 2, 2, amplitude);
}
}
// --- HÀM 2: LẮNG NGHE SỰ KIỆN CUỘN VÀ ZOOM ---
// Khi người dùng kéo thanh cuộn
viewport.addEventListener('scroll', () => {
renderVisibleWaveform();
});
// Khi người dùng lăn chuột để ZOOM tại điểm con trỏ
viewport.addEventListener('wheel', (e) => {
e.preventDefault();
// Tọa độ chuột trong khung nhìn Viewport
const rect = viewport.getBoundingClientRect();
const mouseX = e.clientX - rect.left;
// Tính thời điểm (giây) ngay bên dưới con trỏ chuột
const currentScrollLeft = viewport.scrollLeft;
const timeAtMouse = (currentScrollLeft + mouseX) / pixelsPerSecond;
// Hệ số Zoom (Phóng to / Thu nhỏ tùy ý)
const zoomFactor = e.deltaY < 0 ? 1.15 : 1 / 1.15;
// Giới hạn zoom out tối thiểu (vừa vặn khung nhìn) và zoom in tối đa
const minPxPerSec = VIEWPORT_WIDTH / AUDIO_DURATION;
const maxPxPerSec = 50000; // Có thể zoom sâu mà không sợ vỡ DOM
const newPixelsPerSecond = Math.max(minPxPerSec, Math.min(maxPxPerSec, pixelsPerSecond * zoomFactor));
if (newPixelsPerSecond === pixelsPerSecond) return;
// Cập nhật mức zoom mới
pixelsPerSecond = newPixelsPerSecond;
// Cập nhật lại vị trí thanh cuộn sao cho điểm timeAtMouse vẫn nằm đúng ở mouseX
viewport.scrollLeft = (timeAtMouse * pixelsPerSecond) - mouseX;
// Vẽ lại
renderVisibleWaveform();
}, { passive: false });
// Khởi tạo lần đầu
renderVisibleWaveform();
</script>
</body>
</html>
```
---
## 3. Các điểm quan trọng giúp giải quyết bài toán
* **Thẻ Canvas cố định (`position: sticky`):**
Dù zoom $10\times, 100\times$ hay $1000\times$, chiều rộng Canvas không thay đổi (luôn là `800px`). Điều này giúp tránh hoàn toàn việc vượt giới hạn chiều rộng của trình duyệt (`max canvas width limit`).
* **Khôi phục dữ liệu khi Zoom Out:**
Khi zoom out, `pixelsPerSecond` giảm xuống. Hàm `renderVisibleWaveform()` tự động mở rộng khoảng `[startTime, endTime]` (ví dụ từ 2 giây thành 30 giây). Mảng `audioSamples` gốc trong RAM vẫn giữ nguyên, hàm vẽ chỉ cần duyệt mảng rộng hơn và vẽ lại lên Canvas.
* **Tối ưu hiệu năng:**
Hệ thống chỉ tốn tài nguyên GPU/CPU để render đúng số lượng cột sóng xuất hiện trong khung nhìn thay vì vẽ toàn bộ file audio dài.
2_UI_PLAN.md
-182
View File
@@ -1,182 +0,0 @@
# Tài Liệu Đặc Tả Giao Diện (UI Blueprint): SonicForge Studio Pro DAW
Tài liệu này đặc tả chi tiết cấu trúc, bố cục, thông số thiết kế và cơ chế tương tác của giao diện SonicForge Studio (trong tệp `index.html`). Tài liệu được biên soạn nhằm mục đích hướng dẫn lập trình viên dựng lại (port) giao diện này sang các ứng dụng Python một cách chính xác bằng các thư viện như PyQt6 / PySide6 (`QGraphicsView`), Flet (Flutter for Python), hoặc Reflex / NiceGUI.
---
## 1. Thông Số Thiết Kế Hệ Thống (Design Tokens)
Giao diện SonicForge Studio sử dụng phong cách *Dark Charcoal Studio* tối giản, chuyên nghiệp và có độ tương phản cao nhằm giảm mỏi mắt khi làm việc trong thời gian dài.
| Thuộc tính | Giá trị màu HEX | Ánh xạ mã màu Tailwind | Mô tả ứng dụng |
| --- | --- | --- | --- |
| **Trần nền chính** | `#111111` | `bg-zinc-950` / `bg-[#111111]` | Nền của lưới dòng thời gian (Timeline grid) |
| **Nền ứng dụng** | `#1e1e1e` | `bg-zinc-900` / `bg-[#1e1e1e]` | Nền tổng thể của toàn bộ cửa sổ phần mềm |
| **Nền bảng điều khiển** | `#262626` | `bg-zinc-800` / `bg-[#262626]` | Nền của TCP (bên trái) và Footer (bên dưới) |
| **Nền thanh công cụ** | `#2e2e2e` | `bg-[#2e2e2e]` | Nền của thanh Header và thước đo thời gian (Ruler) |
| **Màu nhấn chính** | `#ef4444` | `text-red-500` / `bg-red-500` | Con trỏ phát nhạc (Playhead), núm âm lượng chủ |
| **Màu nhấn phụ** | `#06b6d4` | `text-cyan-500` / `bg-cyan-500` | Trạng thái chọn track hoạt động (Active select border) |
| **Màu vùng chọn** | `#f59e0b` (Alpha: 0.1) | `bg-amber-500/10` / `border-amber-500` | Khung phủ vùng chọn thời gian (Selection Overlay) |
| **Kiểu chữ (Font)** | `Inter, sans-serif` | - | Sử dụng font Sans-serif không chân, thanh mẫu |
---
## 2. Bản Vẽ Bố Cục Không Gian (Layout Architecture)
Cửa sổ làm việc được chia làm 3 khu vực chính theo chiều dọc màn hình (*Vertical Stack Layout*):
```text
+-----------------------------------------------------------------------+
| [1] HEADER & AI CONFIG DRAWER (Chiều cao cố định: 44px) |
+-----------------------------------------------------------------------+
| [2] WORKSPACE (Chiều cao linh hoạt - Fill Remaining Space) |
| +---------------------------+-------------------------------------+ |
| | [2.A] TRACK CONTROL PANEL | [2.B] TIMELINE LANES & RULER | |
| | (Width: 300px) | (Horizontal & Vertical Scroll)| |
| | | | |
| | - Track ID & Name | - Sticky Time Ruler (top-0) | |
| | - Solo (S) & Mute (M) | - Stacked Waveform Canvases | |
| | - Volume Rotary Knob | - Interactive Selection Overlay | |
| | - Upload Local/Synth | - Global Playhead Line (Vertical) | |
| | | | |
| +---------------------------+-------------------------------------+ |
+-----------------------------------------------------------------------+
| [3] FOOTER PANEL & TRANSPORT (Chiều cao cố định: 176px) |
+-----------------------------------------------------------------------+
| [4] UTILITY STATUS BAR (Chiều cao cố định: 24px) |
+-----------------------------------------------------------------------+
```
---
## 3. Đặc Tả Chi Tiết Thành Phần (Widget Specification)
### 2.A. Bảng Điều Khiển Kênh (Track Control Panel - TCP)
* **Chiều rộng:** Cố định 300px.
* **Cấu trúc:** Chứa danh sách các Track được xếp dọc.
* **Mỗi hàng (Track Header):** Chiều cao 96px (Đồng bộ tuyệt đối với chiều cao Waveform Lane bên phải).
* **Chỉ số (Index):** Số thứ tự track dạng Monospace (ví dụ: `01`, `02`).
* **Trạng thái màu:** Đèn LED tròn hiển thị màu đặc trưng của track (ví dụ: lục, lam, tím).
* **Tên tệp tin:** Nhãn chữ có tính năng thu gọn tự động (`truncate`).
* **Nút Mute (M):** Khi bật sẽ hiển thị nền đỏ (`bg-red-950` / `text-red-400`).
* **Nút Solo (S):** Khi bật sẽ hiển thị nền hổ phách (`bg-amber-950` / `text-amber-400`).
* **Volume Rotary Knob:** Thiết kế dạng núm vặn xoay tròn 2D.
* *Cơ chế hoạt động:* Nhấp và giữ chuột trên núm, kéo chuột lên trên để tăng Volume (vặn cùng chiều kim đồng hồ, giới hạn quay $+135^\circ$), kéo chuột xuống dưới để giảm Volume (vặn ngược chiều kim đồng hồ, giới hạn quay $-135^\circ$).
* **Nút Upload:** Nút bấm cục bộ mở hộp thoại chọn tệp âm thanh trên máy.
### 2.B. Khu Vực Dòng Thời Gian (Timeline & Waveform Lanes)
* **Chiều rộng:** Trải rộng chiếm toàn bộ phần màn hình còn lại.
* **Thước Đo Thời Gian (Ruler):**
* Chiều cao 32px (`sticky top-0`), luôn hiển thị ở trên cùng kể cả khi cuộn dọc.
* Hiển thị vạch chia độ theo từng giây dựa trên mức độ thu phóng (zoom). Định dạng thời gian: `M:SS.mmm`.
* **Các Làn Sóng Âm (Waveform Lanes):**
* Chiều cao mỗi làn: Cố định 96px (tươngương với chiều cao của Track Header bên trái).
* Sử dụng một đối tượng Canvas để vẽ đồ thị sóng âm thời gian thực. Sóng âm vẽ đối xứng qua trục nằm ngang chính giữa.
* **Vùng Chọn Phủ (Selection Overlay):**
* Một phân vùng bán trong suốt màu hổ phách (`bg-amber-500/10`) bao quanh khoảng thời gian được chọn.
* Ranh giới phía trên bắt đầu từ mép dưới thước Ruler (`top-8` hay 32px), kéo dài xuống tận đáy của toàn bộ các track để không đè và che khuất dải sóng âm phía trên.
* Hai đầu biên có thanh nắm (Handles) màu cam (`w-3`) để co dãn vùng chọn bằng cách kéo chuột (`EW-resize`).
* **Đường Chỉ Con Trỏ (Global Playhead Line):**
* Một đường kẻ đứng màu đỏ (`w-[2px] bg-red-500`) chạy dọc từ trên xuống dưới, ghim một tam giác đỏ nhỏ ở đỉnh thước Ruler.
---
## 4. Đặc Tả Cơ Chế Tương Tác & Lập Trình (Interactivity Specs)
Để chuyển giao chính xác sang Python (ví dụ sử dụng PyQt6 hoặc Soundfile/Numpy), cần lập trình chính xác các cơ chế điều khiển sau:
### A. Thuật Toán Phóng To/Thu Nhỏ Theo Vị Trí Chuột (Mouse-Anchored Zoom)
* **Sự kiện kích hoạt:** Cuộn chuột (`wheel`) trên vùng Timeline.
* **Cơ chế:**
1. Ghi nhận vị trí hoành độ `mouseX` của con trỏ chuột đối với khung chứa.
2. Xác định mốc thời gian tuyệt đối tại điểm chuột đang chỉ:
$$\text{anchorTime} = \frac{\text{mouseX} + \text{scrollLeft}}{\text{currentZoom}}$$
3. Cập nhật tỷ lệ zoom mới:
$$\text{newZoom} = \text{currentZoom} \times \text{zoomFactor}$$
*(Ràng buộc: $\text{minZoom} \le \text{newZoom} \le 2000\text{ px/s}$)*
4. Sau khi vẽ lại đồ thị, tính toán và gán lại vị trí thanh cuộn ngang để khóa điểm âm thanh dưới chuột đứng im:
$$\text{newScrollLeft} = (\text{anchorTime} \times \text{newZoom}) - \text{mouseX}$$
### B. Cơ Chế Di Chuyển Đầu/Cuối Vùng Chọn (Selection Resize & Move)
* **Kéo giãn (Resize):**
* Khi nhấp giữ chuột vào Handle trái: Cập nhật `selectionStart` tương ứng với vị trí chuột nhưng khống chế không được vượt quá `selectionEnd`.
* Khi nhấp giữ chuột vào Handle phải: Cập nhật `selectionEnd` nhưng không được nhỏ hơn `selectionStart`.
* **Dịch chuyển (Move):**
* Khi nhấp vào vùng lòng trong của dải chọn (màu cam nhạt), ghi nhận khoảng cách thời gian giữa hai đầu ($\Delta t = \text{end} - \text{start}$).
* Khi di chuột sang trái/phải, tịnh tiến đồng thời cả `start` và `end` một lượng tương đương mà vẫn giữ nguyên độ rộng $\Delta t$.
### C. Logic Lặp Không Độ Trễ (Seamless Looping Logic)
* Khi playhead chạy đến mốc `selectionEnd`, hệ thống phát nhạc phải kích hoạt nhảy ngay lập tức về mốc phát `selectionStart` ở tầng Audio Thread để tránh hiện tượng vấp hoặc trễ nhịp âm thanh.
---
## 5. Bản Đồ Ánh Xạ Sang Thư Viện Python (Python GUI Mapping)
Nếu bạn lựa chọn phát triển ứng dụng máy để bàn (Desktop Application) bằng Python, dưới đây là bảng tham chiếu các lớp Widget tương đương trong thư viện PyQt6 / PySide6:
| Thành phần giao diện (React/HTML) | Thành phần tương đương trong PyQt6 / PySide6 | Phương thức xử lý / Ghi chú |
| --- | --- | --- |
| **Workspace Scroll Container** | `QScrollArea` | Cho phép cuộn đứng đồng bộ cả TCP và Waveforms. |
| **Track Control Panel (TCP)** | `QVBoxLayout` chứa các `QWidget` | Layout xếp dọc, cố định chiều rộng bằng `.setFixedWidth(300)`. |
| **Ruler & Waveform Lanes** | `QGraphicsView` & `QGraphicsScene` | Thích hợp nhất để vẽ đồ thị vectơ sóng âm, playhead line và selection block nhờ khả năng vẽ hai tầng bộ đệm (*Double-buffering*) tốc độ cao. |
| **Waveform Canvas Painter** | `QPainter.drawPath()` / `QPainterPath` | Chuyển đổi dữ liệu mẫu thô (`numpy.ndarray`) thành một tập hợp các đường thẳng đứng biểu diễn biên độ đỉnh âm học (*Peak Waveform*). |
| **Volume Knob Control** | `QDial` hoặc Custom `QWidget` | Tùy biến sự kiện `mouseMoveEvent` để tính toán góc xoay núm âm lượng. |
| **Time Formatter** | Hàm định dạng chuỗi Python | `f"{minutes:02d}:{seconds:02d}.{milliseconds:03d}"` |
---
## 6. Sơ Đồ Cấu Trúc JSON Phục Vụ Port API
Khi Frontend tương tác và nhấn nút **AI Cut & New Track**, cấu hình dải chọn sẽ được đóng gói và gửi thẳng về Dockerized Python API theo định dạng chuẩn sau:
```json
{
"session_id": "pro_session_active",
"source_track_id": "1",
"selection": {
"start_seconds": 2.458,
"end_seconds": 7.892
},
"dsp_actions": {
"zero_crossing_align": true,
"apply_fades_ms": 50,
"volume_db_change": 0.0
},
"export_format": "wav"
}
```
3_LOOP_MAKER.md
-132
View File
@@ -1,132 +0,0 @@
Dưới đây là toàn bộ nội dung tài liệu đặc tả kỹ thuật đã được chuyển đổi sang định dạng Markdown chuẩn, tối ưu hóa các khối mã nguồn (`python`, `text`), căn chỉnh bảng biểu và định dạng các công thức toán học LaTeX:
# Đặc Tả Kỹ Thuật: Hành Động Chọn Vùng & Cơ Chế Phát Lặp (Solo vs. Master Loop)
Tài liệu này đặc tả cơ chế tương tác chuột và logic phát âm thanh tương ứng đối với hai vùng chọn: Chọn cục bộ trên Track (*Local Selection*) và Chọn toàn cục trên Timeline (*Global Selection*) dựa trên thiết kế chuẩn DAW trong hình `image_dfffa3.png`.
Mục tiêu là cung cấp thuật toán xử lý sự kiện chuột (`MouseEvent`) để lập trình viên chuyển đổi (*port*) trực tiếp sang Python (ví dụ sử dụng `QGraphicsView` hoặc `QWidget` tùy biến trong PyQt6/PySide6).
---
## 1. Bản Vẽ Phân Cấp Vùng Tương Tác (Layout Hitboxes)
Dựa trên hình `image_dfffa3.png`, khu vực biên tập bên phải được phân chia thành 2 hitbox tương tác chuột chính yếu:
```text
+-------------------------------------------------------------------------+
| [VÙNG A] TIMELINE RULER (Thước đo thời gian trên cùng - Mũi tên đỏ chỉ) |
+-------------------------------------------------------------------------+
| [VÙNG B] CÁC LÀN TRACK LANE (Xếp chồng dọc) |
| - Track 01 Waveform Area (Hitbox cục bộ 1) |
| - Track 02 Waveform Area (Hitbox cục bộ 2) |
| - Track 03 Waveform Area (Hitbox cục bộ 3) |
+-------------------------------------------------------------------------+
```
---
## 2. Đặc Tả Tương Tác 1: Chọn Cục Bộ & Phát Lặp Độc Lập (Local Track Selection & Solo Loop)
### 2.1. Hành động người dùng (User Action)
* **Click chuột đơn (Mouse Press):** Người dùng nhấp chuột vào một điểm bất kỳ trên dạng sóng của một track (Ví dụ: Track 01).
* *Hệ quả:* Track đó lập tức được highlight (*Active State*), các track khác chuyển sang trạng thái chờ.
* **Kéo chuột (Mouse Drag):** Click và nhấn giữ chuột, kéo sang trái hoặc phải trên phần hiển thị sóng âm của track đó.
* *Hệ quả:* Tạo ra một vùng chọn thời gian giới hạn bởi điểm nhấn đầu và điểm thả chuột cuối.
### 2.2. Cơ chế phát lặp (Loop Playback Logic)
* **Chế độ phát:** Khi nhấn nút *Play Loop*, hệ thống chỉ phát lặp duy nhất đoạn nhạc nằm trong vùng chọn của track đang active.
* **Xử lý Audio Engine ở Python:**
* Tự động kích hoạt cơ chế Mute tạm thời cho tất cả các track khác, hoặc kích hoạt nhanh chế độ Solo cho track đang active trong luồng phát âm thanh.
* Giới hạn mốc thời gian phát trong khoảng $T_{\text{start}}$ đến $T_{\text{end}}$. Khi kim phát phát chạm $T_{\text{end}}$, nhảy ngay lập tức về $T_{\text{start}}$ mà không dừng phát các luồng khác (nếu có).
### 2.3. Ánh xạ mã sự kiện Python (PyQt6 / PySide6 Concept)
```python
# Giả lập xử lý sự kiện trong lớp TrackWaveformWidget(QWidget)
def mousePressEvent(self, event):
if event.button() == Qt.MouseButton.LeftButton:
# 1. Kích hoạt chọn track hiện hành
self.parent_session.set_active_track(self.track_id)
# 2. Ghi nhận điểm mốc thời gian bắt đầu
self.drag_start_time = self.pixel_to_time(event.position().x())
self.is_dragging = True
def mouseMoveEvent(self, event):
if self.is_dragging:
current_time = self.pixel_to_time(event.position().x())
# Cập nhật vùng chọn cục bộ (Local Selection Range)
self.parent_session.update_local_selection(
track_id=self.track_id,
start=min(self.drag_start_time, current_time),
end=max(self.drag_start_time, current_time)
)
```
---
## 3. Đặc Tả Tương Tác 2: Chọn Toàn Cục & Phát Lặp Đa Kênh (Global Timeline Selection & Master Loop)
### 3.1. Hành động người dùng (User Action)
* **Định vị:** Di chuyển con trỏ chuột lên trên cùng, vào thanh chứa Timeline Ruler (Vị trí mũi tên màu đỏ trong hình `image_dfffa3.png`).
* **Kéo chuột (Mouse Drag):** Click chuột vào thanh Ruler và kéo sang trái/phải.
* *Hệ quả:* Một khung màu cam nhạt (*Selection Overlay*) xuất hiện và kéo dọc toàn bộ chiều cao màn hình xuyên qua tất cả các track từ trên xuống dưới (như hiển thị thực tế trong hình `image_dfffa3.png`).
### 3.2. Cơ chế phát lặp (Loop Playback Logic)
* **Chế độ phát:** Khi nhấn nút *Play Loop*, hệ thống sẽ phát đồng thời tất cả các track (*Master Playback*) nhưng giới hạn vòng lặp đồng bộ chỉ nằm trong dải thời gian được bôi màu.
* **Xử lý Audio Engine ở Python:**
* Giữ nguyên trạng thái Mute/Solo hiện tại của các track (không tự động tắt tiếng các track khác).
* Đồng bộ hóa pha của tất cả các luồng phát. Khi Playhead chạm mốc kết thúc vùng chọn $T_{\text{end}}$, toàn bộ các nguồn phát (*Audio Sources*) đang hoạt động đều phải nhảy đồng bộ về vị trí $T_{\text{start}}$.
### 3.3. Ánh xạ mã sự kiện Python (PyQt6 / PySide6 Concept)
```python
# Giả lập xử lý sự kiện trong lớp TimelineRulerWidget(QWidget)
def mousePressEvent(self, event):
if event.button() == Qt.MouseButton.LeftButton:
# Ghi nhận mốc thời gian toàn cục ban đầu
self.global_drag_start = self.pixel_to_time(event.position().x())
self.is_dragging_global = True
# Hủy bỏ các vùng chọn cục bộ (nếu có) để ưu tiên chế độ chọn tổng thể
self.parent_session.clear_all_local_selections()
def mouseMoveEvent(self, event):
if self.is_dragging_global:
current_time = self.pixel_to_time(event.position().x())
# Cập nhật vùng chọn toàn cục xuyên suốt tất cả các làn track
self.parent_session.set_global_selection(
start=min(self.global_drag_start, current_time),
end=max(self.global_drag_start, current_time)
)
```
---
## 4. Tóm Tắt Sự Khác Biệt Giữa 2 Trạng Trạng Thái
Để đảm bảo hệ thống không bị xung đột khi xử lý đa luồng âm thanh trên Python Docker, bộ điều phối âm thanh (*Audio Coordinator*) cần tuân thủ bảng logic sau:
| Thuộc tính | Tương tác trên Track (Local) | Tương tác trên Ruler (Global) |
| --- | --- | --- |
| **Phạm vi hiển thị vùng chọn** | Chỉ hiển thị hoặc sáng rõ tại track đang active. | Kéo dài thẳng đứng, phủ qua tất cả các track (như hình `image_dfffa3.png`). |
| **Phạm vi phát âm thanh** | **Solo Playback:** Chỉ phát duy nhất âm thanh của track được chọn. | **Master Playback:** Phát tất cả các track cùng lúc (không tự động Solo/Mute). |
| **Điểm lặp (Loop Target)** | $T_{\text{start}} \rightarrow T_{\text{end}}$ của riêng track hoạt động. | $T_{\text{start}} \rightarrow T_{\text{end}}$ toàn cục đồng bộ tất cả các track. |
| **Phục vụ AI Cut** | Chỉ cắt tệp tin của riêng track được chọn để đưa sang track mới. | Cho phép render/mixdown gộp tất cả các track trong dải chọn thành tệp mới. |
4_LOOP_EDITOR.md
-144
View File
@@ -1,144 +0,0 @@
Dưới đây là toàn bộ nội dung tài liệu đặc tả kỹ thuật đã được chuyển đổi sang định dạng Markdown chuẩn, tối ưu hóa các khối mã nguồn (`python`, `text`), căn chỉnh bảng biểu, sơ đồ luồng ASCII và các công thức toán học dạng LaTeX:
# Đặc Tả Kỹ Thuật: Biên Tập Cục Bộ, Cơ Chế Tab Tạm Thời & Hoàn Tác (Undo/Redo)
Tài liệu này phân tích chi tiết cơ chế tương tác đồ họa và xử lý tín hiệu âm thanh dựa trên giao diện DAW chuẩn hóa trong hình `image_e076cb.png`. Mục tiêu là cung cấp tài liệu thiết kế hệ thống và giải thuật để port trực tiếp sang ứng dụng Python chạy trên Docker.
---
## 1. Phân Tích Trạng Thái Track Hoạt Động (Active Track State) & Khung Vùng Chọn Cục Bộ
Dựa trên hình `image_e076cb.png`, hệ thống sử dụng cơ chế *Local Waveform Selection* (Chọn vùng cục bộ trên từng kênh) thay vì phủ bóng toàn bộ các kênh trên dòng thời gian.
### 1.1. Trạng thái Track Active
* **Hành động:** Khi người dùng click chuột vào vùng hiển thị của một Track (ví dụ: Track 1), hệ thống sẽ gán trạng thái `ACTIVE` cho track đó.
* **Hiển thị hình ảnh:**
* Nền của Track active sẽ chuyển sang màu xám sáng (`#2a2a2a` hoặc `#333333`), trong khi các track không active ở trạng thái chờ với màu tối hơn (`#181818`).
* Toàn bộ đường viền quanh track được highlight nhẹ bằng một viền sáng mờ.
### 1.2. Khung Chọn Cục Bộ (Local Selection Highlight)
* **Quy luật hiển thị:** Khung màu sáng (Overlay màu xám bạc trong hình `image_e076cb.png`) chỉ được vẽ đè lên dạng sóng (Waveform) của riêng track đang active, giới hạn trục ngang từ $T_{\text{start}}$ đến $T_{\text{end}}$.
* **Ràng buộc đồ họa:** Các track nằm dưới (ví dụ: Track 2) sẽ hoàn toàn không bị phủ bóng xám, dù nằm cùng khoảng thời gian $T_{\text{start}} \rightarrow T_{\text{end}}$.
> **Khai báo an toàn khi Port sang Python (Tránh Crash):**
> * Luôn kiểm tra tính hợp lệ của mốc thời gian: $0 \le T_{\text{start}} < T_{\text{end}} \le T_{\text{max}}$.
> * Chặn lỗi vượt quá giới hạn mảng mẫu (*Index Out of Bounds*) khi ánh xạ từ Pixel sang mẫu âm thanh số:
>
>
> $$\text{Sample}_{\text{start}} = \text{clamp}(0, \lfloor T_{\text{start}} \times \text{Sample Rate} \rfloor, \text{Total Samples})$$
>
>
---
## 2. Quy Trình Biên Tập Trong Tab Tạm Thời (Temporary Edit Tab Workflow)
Đây là tính năng biên tập không phá hủy (*Non-destructive*) nâng cao, cho phép cô lập phân đoạn âm thanh để xử lý chuyên sâu trước khi gộp lại vào bản phối chính.
```text
[Bản Phối Chính] ──► Chọn đoạn (T_start -> T_end) ──► Nhấn "Edit in Temp Tab"
│
┌────────────────────────────────────────────────────────────┘
▼
[Khởi tạo Tab Tạm Thời]
├── Trích xuất mảng mẫu phụ (Audio Sub-segment Buffer)
├── Hiển thị dạng sóng cô lập (Thời gian chạy từ 0 đến T_duration)
├── Người dùng thực hiện các hiệu ứng: Reverse, Gain, Pitch Shift, Fade...
└── Nhấn "Áp dụng (Apply)"
│
▼
[Hòa nhập lại Bản Phối]
├── Tính toán khớp Zero-crossing tại hai đầu biên ghép nối.
├── Áp dụng hiệu ứng mờ biên (Micro-crossfades) để chống tiếng Click/Pop.
└── Thay thế mảng mẫu mới vào vị trí cũ và dọn dẹp Tab tạm.
```
### 2.1. Trích xuất sang Tab Tạm Thời (Export to Temporary Tab)
Khi người dùng chọn vùng trên Track Active và nhấn "Edit in Temp Tab", hệ thống sẽ tách đoạn âm thanh này thành một thực thể đệm độc lập (`Sub-segment AudioBuffer`).
* Một tab mới (Ví dụ: `Tab: sẤit tiá...n` trong hình `image_e076cb.png`) xuất hiện ngay phía trên dòng thời gian.
* Trong tab này, trục thời gian của Ruler sẽ được đặt lại (*Reset*) bắt đầu từ `00:00:00.000` cho đến độ dài của đoạn được cắt:
$$T_{\text{duration}} = T_{\text{end}} - T_{\text{start}}$$
### 2.2. Hòa nhập lại Track Chính (Apply & Merge Back)
Khi người dùng hoàn tất chỉnh sửa trên Tab tạm và nhấn *Apply*, hệ thống Python/Docker Backend thực hiện quy trình DSP ghép nối sau để tránh hiện tượng vấp âm (*Click/Pop*):
1. **Tìm điểm Zero-Crossing lân cận:** Hệ thống tự động dịch nhẹ mốc nối $T_{\text{start}}$ và $T_{\text{end}}$ một vài mẫu ($5 \rightarrow 10$ samples) để đảm bảo biên độ tại điểm ghép nối bằng $0$.
2. **Áp dụng Micro-Crossfade:** Tạo một cửa sổ chuyển tiếp cực ngắn ($w = 10\text{ ms}$) giữa file gốc và file sửa đổi tại điểm ráp nối để triệt tiêu hoàn toàn sự thay đổi đột ngột của pha:
$$\text{Final}_{\text{audio}}(t) = (1 - \alpha(t)) \cdot \text{Original}(t) + \alpha(t) \cdot \text{Edited}(t - T_{\text{start}})$$
*Trong đó:* $\alpha(t) = \frac{t - T_{\text{start}}}{w}$ với $T_{\text{start}} \le t \le T_{\text{start}} + w$.
---
## 3. Cơ Chế Đồng Bộ Hóa Con Trỏ Phát Nhạc (Playhead Tracking)
* **Hành vi tương tác:** Khi phát nhạc (*Play*), kim phát nhạc (*Playhead line* màu đỏ) phải di chuyển liên tục, mượt mà dọc theo trục ngang của dạng sóng.
* **Thuật toán đồng bộ hóa (Client-Server):**
* Tốc độ di chuyển của Playhead dựa trên thời gian thực tế của luồng phát âm thanh (`AudioContext.currentTime` ở Client hoặc đồng hồ xung của card âm thanh phía Server).
* Vị trí hoành độ $X$ (Pixel) của con trỏ tại thời điểm $t$ được tính bằng công thức:
$$X(t) = t \times \text{Zoom Level}$$
* **Khi Loop hoạt động:** Khi $t \ge T_{\text{end}}$, luồng âm thanh lập tức chuyển hướng phát về $T_{\text{start}}$, đồng thời biến thời gian hiển thị con trỏ được đặt lại ngay lập tức: $t = T_{\text{start}}$ mà không dừng luồng phần cứng.
---
## 4. Kiến Trúc Hoàn Tác & Làm Lại (Undo / Redo Engine: Ctrl-Z & Ctrl-Y)
Để đảm bảo hiệu năng tối ưu trên Docker Server (tránh việc lưu đi lưu lại các tệp tin WAV nặng hàng trăm Megabytes vào bộ nhớ), hệ thống sử dụng Kiến trúc Hoàn tác dựa trên Delta (*State Delta-based Undo/Redo*).
### 4.1. Cấu trúc lưu trữ lịch sử (History Stack Node)
Mỗi hành động của người dùng (Cắt, ghép, thay đổi volume, fade, chỉnh sửa trong tab tạm) được đóng gói thành một đối tượng `ActionNode`:
```python
import time
class ActionNode:
def __init__(self, action_type: str, track_id: str):
self.action_type = action_type # 'SPLIT', 'VOLUME_CHANGE', 'TEMP_TAB_EDIT', etc.
self.track_id = track_id
self.timestamp = time.time()
# Lưu thông tin delta để khôi phục thay vì lưu cả file nhạc
self.before_state = {} # Trạng thái trước khi sửa
self.after_state = {} # Trạng thái sau khi sửa
```
### 4.2. Logic Hoàn tác (Undo - `Ctrl + Z`)
Khi người dùng nhấn tổ hợp phím `Ctrl + Z`:
1. Lấy hành động mới nhất từ *Undo Stack*.
2. Thực thi hàm nghịch đảo của hành động đó để đưa track về trạng thái `before_state`.
3. Đẩy hành động này sang *Redo Stack* để có thể làm lại.
4. Vẽ lại dạng sóng trên Canvas tương ứng.
### 4.3. Logic Làm lại (Redo - `Ctrl + Y`)
Khi người dùng nhấn tổ hợp phím `Ctrl + Y`:
1. Lấy hành động mới nhất từ *Redo Stack*.
2. Áp dụng trạng thái `after_state` lên track đích.
3. Đẩy ngược hành động này về lại *Undo Stack*.
4. Cập nhật đồ họa hiển thị.
### 4.4. Quản lý bộ nhớ tối ưu (Garbage Collection)
* Giới hạn kích thước tối đa của Stack hoàn tác (Ví dụ: tối đa 30 hành động) để tránh tràn bộ nhớ RAM của Docker Container.
* Các đoạn âm thanh bị thay thế bởi thao tác chỉnh sửa sẽ được lưu trữ dưới dạng các tệp nhị phân tạm thời (`.tmp`) trong thư mục `/app/storage/temp/` và tự động dọn dẹp khi phiên làm việc (Session) kết thúc.
5_LOOP_EDITOR_2.md
-130
View File
@@ -1,130 +0,0 @@
# Technical Specification: Tabbed Interface, Configurations & Strict Looping Constraints
This document details the software design specification for the tabbed multi-project structure, top-level application configurations, and low-latency transport loop engine boundaries for *SonicForge Studio*, referencing the professional DAW layout in `image_e0e462.png`.
---
## 1. Multi-Tab Architecture: Main vs. Sub (Temporary) Tabs
As illustrated in `image_e0e462.png` (indicated by the red arrows pointing to the tab bar), the application supports multiple active document spaces running in parallel.
```text
+---------------------------------------------------------------------------------+
| File Edit View Insert Track Options Actions Extensions Help |
+---------------------------------------------------------------------------------+
| [*Main_Session.rpp] | [Sub_Tab_Isolated_Edit] | |
+-----------------------------------------+---------------------------------------+
| | |
| [Main Tab: Multitrack Workspace] | [Sub Tab: Isolated Sample Editor] |
| - Multi-channel arrangements | - Destructive audio processing |
| - Level mixing & panning | - Focus on selected sub-region |
| - Real-time plugin chains | - Apply specialized DSP / AI |
| | |
+-----------------------------------------+---------------------------------------+
```
### 1.1. Main Tab (Multitrack Mixing)
* **Scope:** Hosts the global arrangement canvas with multiple tracks stacked vertically.
* **Function:** Used for complex operations including track leveling, master mixdowns, track synchronization, and timeline-based multi-channel volume automation.
### 1.2. Sub Tab (Temporary Isolated Clip Editor)
* **Scope:** A sandbox workspace containing only the isolated audio buffer extracted from a specific track's selection clip.
* **Function:** Contains standard sample-level editing tools (trimming, phase inversion, amplification, and precision AI noise reduction).
* **Workflow Sync:**
* Modifying data inside the Sub Tab operates on a temporary audio buffer.
* Clicking *Apply* triggers a non-destructive or destructive overwrite back into the Main Tab's parent track at the exact source offset coordinates.
---
## 2. Top-Level Menu Bar & Application Configurations
To support both basic user preferences and deep AI/system configurations, a global Menu Bar is placed at the absolute top of the frame (matching the menu path: `File` `Edit` `View` `Insert` `Item` `Track` `Options` `Actions` `Extensions` `Help` in `image_e0e462.png`).
### 2.1. Configuration Architecture
These menus map directly to local configurations and server APIs on the Docker backend:
* **File:** Session operations (*New*, *Open*, *Save Session*) and Offline Audio Mixdown export settings (*Sample Rate*, *Bit-Depth*, *Format*).
* **Options -> Audio Device Settings:** Defines client/server hardware routing, sample buffer frame size ($64 \rightarrow 512$ samples) to control playback latency, and audio API endpoints (*ASIO*, *CoreAudio*, *ALSA*).
* **Options -> AI Integration Settings:**
* *Endpoint Configuration:* Sets API Gateway URLs (OpenAI-compatible server endpoint).
* *Authentication:* API keys, model parameters, and target model configurations (e.g., `gpt-4o-mini`, local `ollama` endpoints).
* **Actions -> Admin Dashboard:** Admin-only access panel to manage user accounts, disk quota limits ($S_{\text{limit}}$), active socket connections, and toggle system Feature Flags.
---
## 3. Playhead Tracking & Looping Synchronicities
The transport engine must manage low-latency coordinate translations to ensure the playhead red line exactly mirrors the hardware audio clocks during loop operations.
### 3.1. Looping Playhead Movement Logic
* **Seamless Loop Synchronization:** When loop play is triggered, the playhead coordinates on the visual timeline must instantly align with the active audio buffers.
* **Immediate Reset on Cycle:** When the current audio timestamp $t$ reaches the loop end point $T_{\text{end}}$, the playhead must immediately reset to the loop start point $T_{\text{start}}$ without lagging or disappearing from the viewport.
* **Visual Refresh Coordination:** The browser animation loop (`requestAnimationFrame`) or Python GUI timer must query the audio hardware clock directly, avoiding UI-driven clock drift:
$$t_{\text{playhead}} = T_{\text{start}} + \left( (t_{\text{system}} - t_{\text{trigger}}) \pmod{T_{\text{end}} - T_{\text{start}}} \right)$$
---
## 4. Strict Playhead Looping Constraints & Escape Mechanism
```text
Strict Loop State Locked (Spacebar toggles within boundary)
+-------------------------------------------------+
| |
▼ | (Loop Repeat)
[ T_start ] ==============> [ Playhead (t) ] ======> [ T_end ]
│
│ (User clicks outside selection region)
▼
[ Escape Loop Triggered ]
│
▼ (Press SPACEBAR)
[ Linear Playback Active ] ===> Playhead continues past T_end indefinitely
```
### 4.1. Strict Boundary Constraint (Active Loop)
When a time selection $[T_{\text{start}}, T_{\text{end}}]$ is active and looping is turned on, the playhead is locked to the interval:
$$t \in [T_{\text{start}}, T_{\text{end}}]$$
Under no circumstances can the playhead drift past $T_{\text{end}}$. If the audio thread finishes rendering the buffer slice corresponding to $T_{\text{end}}$, it must seamlessly jump back to $T_{\text{start}}$.
### 4.2. Escape Loop Mechanism
To leave the loop and return to continuous, non-repeating playback, the user must perform the following actions:
1. **Clear Selection Focus:** The user clicks outside the selection box on an empty area of the timeline ruler or track lane.
2. **Deregister Selection Boundaries:** The variables $T_{\text{start}}$ and $T_{\text{end}}$ are cleared (set to `null` or $0$ and $T_{\text{max}}$ respectively).
3. **Resume Linear Playback:** Pressing the `Spacebar` key triggers the transport to play continuously through and past the old boundary marker.
### 4.3. Keyboard Mapping Matrix (Python GUI Translation)
```python
# PyQt6 / PySide6 Key Event Hook Simulation
def keyPressEvent(self, event):
if event.key() == Qt.Key.Key_Space:
if self.transport.is_playing:
self.transport.pause()
else:
# If selection was cleared, it continues linear playback past T_end
if self.session.selection_cleared:
self.transport.play_linear(from_time=self.playhead.current_time)
else:
self.transport.play_looped(
start=self.session.selection_start,
end=self.session.selection_end
)
```
6_NEW_PLAN.md
-1146
View File
File diff suppressed because it is too large Load Diff
7_FIX_CLIP.md
-132
View File
@@ -1,132 +0,0 @@
# Technical Specification: Bug Fix for Clip Visual Stretching
This document analyzes the root cause and defines the waveform painting algorithm to fix the bug where a short audio clip is incorrectly stretched to fill the viewport when pasted onto a new track, referencing the real-world visual analysis in `image_f05ca7.png`.
---
## 1. Root Cause
Based on `image_f05ca7.png`, the error occurs because the track lane's canvas render logic utilizes the total viewport width ($W_{\text{viewport}}$) as the bounding milestone to distribute and draw the entire sample count of the buffer.
* **Bug Mechanism:** The system treats the clip's start point as $0$ and the clip's end point as the end of the screen, completely ignoring the clip's actual duration ($T_{\text{clip}}$) and starting time coordinates ($t_{\text{offset}}$) of the pasted segment.
* **Consequence:** The short clip is stretched with an incorrect display frequency, falling completely out of sync with the global Time Ruler at the top.
---
## 2. Technical Solution: Coordinate System Alignment
To display the audio clip at its correct duration and position, every clip on the timeline must be managed using two core attributes:
* **$t_{\text{offset}}$ (seconds):** The timeline insertion position where the clip starts (the position of the playhead at the moment of pasting).
* **$T_{\text{clip}}$ (seconds):** The actual duration of the sliced audio file ($T_{\text{clip}} = \text{samples} / \text{sample\_rate}$).
```text
Global Timeline
+───────────────────────────────────────────────────────────────────────────+
│ │
│ Track 01: [█████████████████████████████████████████████████████████] │
│ │
│ Track 02: [██████████████] <--- Render only this range │
│ ▲ ▲ │
│ │ │ │
│ t_offset t_offset + T_clip │
+───────────────────────────────────────────────────────────────────────────+
```
### 2.1. Pixel Mapping Formula
Let $Z$ be the current zoom level (the number of display pixels per second of audio). The initial rendering coordinate and the physical width of the clip on the Canvas must strictly follow these formulas:
* **Starting rendering coordinate ($X_{\text{start}}$):**
$$X_{\text{start}} = t_{\text{offset}} \times Z$$
* **Physical width of the waveform ($W_{\text{clip}}$):**
$$W_{\text{clip}} = T_{\text{clip}} \times Z$$
---
## 3. Safe Waveform Render Algorithm (Python & JS)
The Render Loop must exclusively calculate and draw peak amplitudes within the bound stretching from $X_{\text{start}}$ to $X_{\text{start}} + W_{\text{clip}}$. Any pixel region outside this interval must be painted with an empty background color (transparent or the track's dark background theme).
### 3.1. Pseudo-code
```python
def render_track_lane(canvas_width, zoom_level, track_clip):
# 1. Compute rendering boundaries based on synchronization formulas
x_start = track_clip.offset_seconds * zoom_level
w_clip = track_clip.duration_seconds * zoom_level
x_end = x_start + w_clip
# 2. Initialize empty background
initialize_background(0, canvas_width)
# 3. Scan the pixel array and only render within the active clip segment
for x in range(0, canvas_width):
if x < x_start or x > x_end:
# Paint empty background color for region with no data
draw_background_pixel(x)
else:
# Map current pixel x coordinate back to sample index in Buffer
time_in_clip = (x - x_start) / zoom_level
sample_index = int(time_in_clip * track_clip.sample_rate)
# Calculate amplitude peak and draw symmetrical vertical line
amplitude_peak = get_peak_amplitude(track_clip.buffer, sample_index)
draw_waveform_vertical_line(x, amplitude_peak)
```
---
## 4. Guarding Array Boundaries on Python Docker Server
When a user triggers a cut/paste operation on the Frontend, the JSON data structure dispatched to the Python Server must explicitly specify the destination paste coordinates to prevent index calculation errors:
```json
{
"action": "paste_clip",
"source_clip": {
"clip_id": "clip_abc123",
"duration_seconds": 24.150,
"sample_rate": 44100
},
"destination": {
"track_id": "02",
"paste_at_seconds": 15.300
}
}
```
On the Python backend (utilizing `pydub` or `numpy`), the binary array insertion is executed at the exact time milestone by zero-padding the preceding segment to align perfectly:
```python
import numpy as np
def insert_clip_to_track_array(track_array: np.ndarray, sr: int, clip_array: np.ndarray, paste_sec: float) -> np.ndarray:
"""
Inserts clip_array into track_array at paste_sec without stretching the signal.
"""
paste_sample = int(paste_sec * sr)
clip_length = len(clip_array)
# Create a new array with a length covering the entire pasted segment
required_length = max(len(track_array), paste_sample + clip_length)
output_array = np.zeros(required_length, dtype=np.float32)
# Copy original track data over
output_array[0:len(track_array)] = track_array
# Overwrite the new clip at the precise real-time coordinate position
output_array[paste_sample:paste_sample + clip_length] = clip_array
return output_array
```
8_TRACK_ZOOM.md
-125
View File
@@ -1,125 +0,0 @@
# Technical Specification: Minimum Zoom Constraint Specification
This document defines the algorithm and graphical rendering mechanics (Rendering Logic) to solve the following problem: When zooming out to the absolute minimum, the audio waveforms of all tracks must fit perfectly within the horizontal width of the Editor viewport, as realistically illustrated in `image_f0c1e0.png`.
---
## 1. Current State & Design Problem Analysis
Based on `image_f0c1e0.png`, when a user performs a zoom-out operation to the absolute minimum limit:
* **Visual Requirement:** The entire audio range from the starting point ($0.00\text{ s}$) to the termination point ($T_{\text{max}}$) must be captured completely within the "Display Width" ($W_{\text{viewport}}$) of the screen.
* **Desired Outcomes:**
* No redundant horizontal scrollbars appear underneath the timeline.
* No massive black voids (dead space) exist on the right side if the track duration is shorter than the viewport bounding container.
* All audio tracks are scaled down synchronously in physical size to fully display their respective waveforms from start to finish.
```text
Editor Viewport Width (W_viewport)
|<───────────────────────────────────────────────────────────────────────────────────>|
+─────────────────────────────────────────────────────────────────────────────────────+
| Ruler: 0:00 0:10 0:20 0:30 0:40 0:50 1:00 |
+─────────────────────────────────────────────────────────────────────────────────────+
| Track 1: [███████████████████████████████████████████████████████████████████████] |
| |
| Track 2: [███████████████████████████████████████████████████████████████████████] |
+─────────────────────────────────────────────────────────────────────────────────────+
```
---
## 2. Dynamic Min-Zoom Calculation Algorithm
To guarantee that the waveforms always fit perfectly even when the user resizes the browser window (or a Python application window), the minimum boundary zoom value ($Z_{\text{min}}$) must be evaluated dynamically.
Let:
* **$W_{\text{viewport}}$ (pixels):** The actual horizontal visible width of the timeline container viewport.
* **$T_{\text{max}}$ (seconds):** The maximum duration among all active tracks residing on the timeline.
* **$Z$ (pixels/second):** The current zoom scale factor (Zoom Level—the number of physical pixels representing $1$ second of audio).
The bounding minimum zoom level ($Z_{\text{min}}$) is determined by the formula:
$$Z_{\text{min}} = \frac{W_{\text{viewport}}}{T_{\text{max}}}$$
### 2.1. Zoom Level Constraints
Throughout mouse wheel interaction events triggered to adjust $Z$, the system must check bounds and strictly clamp the value within a safe operating spectrum:
$$Z_{\text{clipped}} = \text{clamp}(Z_{\text{min}}, Z_{\text{target}}, Z_{\text{max}})$$
*Where:*
* **$Z_{\text{max}}$:** The upper zoom-in boundary limit (e.g., fixed at $2000\text{ pixels/s}$ to eliminate canvas rendering memory overflow vulnerabilities).
* **$Z_{\text{min}}$:** The dynamic lower zoom-out boundary limit (re-calculated based on fluctuations of $W_{\text{viewport}}$ and $T_{\text{max}}$).
---
## 3. Synchronized Implementation Manual (Frontend JS & Python Porting)
### 3.1. Client-side Integration (JavaScript / React)
Utilize a `ResizeObserver` to systematically recompute $Z_{\text{min}}$ as soon as the user scales the browser viewport:
```javascript
// Initialize element targeting for the Timeline container frame
const timelineWrapper = document.getElementById('timeline-wrapper');
const resizeObserver = new ResizeObserver(entries => {
for (let entry of entries) {
const viewportWidth = entry.contentRect.width;
// Compute dynamic Z_min boundary condition
const computedMinZoom = viewportWidth / maxDuration;
// Update state and instantly clamp current zoom so it doesn't fall below Z_min
setZoom(prevZoom => {
const nextZoom = Math.max(computedMinZoom, prevZoom);
return nextZoom;
});
}
});
resizeObserver.observe(timelineWrapper);
```
### 3.2. Server-side / Desktop App Integration (Python PyQt6 / PySide6)
When porting this layout system and mathematical constraint model to a Python desktop application context, hook into the `resizeEvent` method of the `QWidget` class to handle container adjustments:
```python
from PyQt6.QtWidgets import QWidget
from PyQt6.QtCore import QSize
class TimelineContainerWidget(QWidget):
def __init__(self, parent=None):
super().__init__(parent)
self.max_duration_seconds = 60.0 # Track duration benchmark (seconds)
self.current_zoom = 100.0 # Current scale metric (pixels/second)
self.max_zoom = 2000.0 # Strict upper ceiling for zoom-in operations
def resizeEvent(self, event):
"""
Intercepts the window/widget resizing event to refresh the minimum zoom bounds.
"""
viewport_width = self.width()
# 1. Evaluate the dynamic Z_min constraint from the new physical container width
min_zoom = float(viewport_width) / self.max_duration_seconds
# 2. Hard clamp the active zoom level to prevent dropping underneath min_zoom
if self.current_zoom < min_zoom:
self.current_zoom = min_zoom
# 3. Request a graphical redraw of the waveform lanes
self.update_waveform_painter()
super().resizeEvent(event)
def update_waveform_painter(self):
# Triggers the QPainter paintEvent routine redraw execution block
self.update()
```
9.1_FIX_TRACKUI.md
-142
View File
@@ -1,142 +0,0 @@
# Bug Fix Specification: Resolving Critical Row Desynchronization
This document analyzes the root cause and provides a permanent structural solution to eliminate the vertical row desynchronization and internal horizontal scrolling artifacts occurring between the left Track Control Panel (TCP) and the right waveform lanes, based on the real-world visual analysis
---
## 1. Visual Symptom Analysis
The layout engine is suffering from two critical alignment failures indicated by the red arrows:
```text
[ LEFT COLUMN - TCP PANEL ] [ RIGHT COLUMN - TIMELINE GRID ]
┌──────────────────────────────┐ ┌──────────────────────────────────────────────┐
│ ... Track 05, 06 (Aligned) │ ══════════ │ Waveform 05, 06 (Aligned) │
├──────────────────────────────┤ ├──────────────────────────────────────────────┤
│ 07 Track 3 (Channel Header) │ [MISALIGNED]│ [EMPTY BLACK DEAD SPACE] (Lower red arrow) │
│ [Junk horizontal scrollbar] │ ◄────────── │ ◄── Caused by Waveform 07 dropping height to 0│
│ (Upper red arrow) │ ├──────────────────────────────────────────────┤
├──────────────────────────────┤ │ Waveform 07 (Pushed down to Track 08's row) │
│ 08 Track 3 │ ══════════ │ ... │
└──────────────────────────────┘ └──────────────────────────────────────────────┘
```
### 1.1. Defect Index 1: Spurious Internal Horizontal Scrollbar (Upper Red Arrow)
* **Symptom:** A small gray horizontal scrollbar emerges directly beneath Track 07 within the left TCP column.
* **Root Cause:** The container wrapper for the left TCP column enforces a rigid bounding layout (`fixed width` or missing an explicit `overflow-x: hidden` safety attribute). When inner structural components (such as long text labels, Mute/Solo clusters, or upload file actions) expand horizontally, the browser generates a local scrollbar. This automatically inflates the effective physical height of the left Track 07 by roughly $12\text{ px} \rightarrow 16\text{ px}$.
### 1.2. Defect Index 2: Vertical Row Desynchronization & Dead Black Space (Lower Red Arrow)
* **Symptom:** On the right column (Timeline), a massive horizontal empty black gap disrupts the grid layout where Waveform 07 ought to sit. Consequently, all matching waveforms for Track 07 and Track 08 are offset downward, falling entirely out of phase with their corresponding control headers on the left.
* **Root Cause:** The system evaluates the target height ($H$) of the left TCP container independently from the right Waveform Lane. When the left Track 07 column expands due to the rendering of the junk scrollbar, the right canvas lane does not dynamically adapt. This triggers a cumulative pixel error along the vertical axis ($Y$), producing progressive, severe desynchronization downstream (the lower the tracks sit, the worse the alignment drifts).
---
## 2. Structural Correction Blueprint
To prevent this layout defect from recurring—especially when porting the interface to desktop Python using PyQt/PySide—the system must completely decouple from independent height calculations and embrace a **Unified Row Layout** model.
### 2.1. Standardized HTML / Tailwind CSS Architecture Blueprint
Instead of splitting the page tree layout into two isolated columns (`Col1: [TCP1, TCP2, TCP3]` and `Col2: [Wave1, Wave2, Wave3]`), the application must encapsulate each matching TCP and Waveform pair within a shared, unified row wrapper:
```html
<!-- Wrap the entire track stack inside a single vertical scroll container -->
<div class="flex-1 overflow-y-auto bg-[#111111]">
<!-- UNIFIED TRACK ROW (Enforces strict shared-row geometry) -->
<div class="flex h-[96px] w-full min-w-max border-b border-[#141414]">
<!-- Left Side: TCP (Fixed width; absolute containment of horizontal overflows) -->
<div class="w-[300px] shrink-0 bg-[#262626] p-2.5 overflow-hidden flex flex-col justify-between">
<!-- TCP Control Content Elements Go Here -->
</div>
<!-- Right Side: Waveform Lane (Flexibly fills remaining browser canvas viewport) -->
<div class="flex-1 relative overflow-hidden">
<!-- Waveform Canvas Engine -->
</div>
</div>
<!-- Add additional track rows duplicating the exact structural envelope above... -->
</div>
```
### Architectural Advantages:
* Because both the control deck and the waveform graphic share an identical row container wrapper (`flex row` or `grid row`), any arbitrary height fluctuation on the TCP side (due to text zoom-in behaviors or overflow glitches) will instantly force the right waveform canvas view to mirror the $100\%$ row scale change.
* Only one master vertical scrollbar exists on the outer perimeter window to slide all rows simultaneously.
---
## 3. Prevention Guidelines for Python Porting (PyQt6 / PySide6)
If you attempt to design this DAW interface inside a containerized Python Docker application by leveraging two separate `QScrollArea` nodes for the TCP track column and the timeline canvas, you will inevitably trigger this row alignment defect due to timing delays or scroll tracking errors (`scrollEvent` mismatch).
### 3.1. Secure Layout Architecture Using Python QWidget
Implement a nested widget strategy to securely bind the horizontal axes together at all times:
```python
from PyQt6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QScrollArea
from PyQt6.QtCore import Qt
class ProDAWArrangeWindow(QWidget):
def __init__(self):
super().__init__()
self.main_layout = QVBoxLayout(self)
self.main_layout.setContentsMargins(0, 0, 0, 0)
self.main_layout.setSpacing(0)
# 1. Instantiate a single, unified QScrollArea for the absolute Workspace
self.workspace_scroll = QScrollArea()
self.workspace_scroll.setWidgetResizable(True)
self.workspace_scroll.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOn)
self.workspace_scroll.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
# 2. Outer container hosting the multi-channel rows
self.container_widget = QWidget()
self.container_layout = QVBoxLayout(self.container_widget)
self.container_layout.setContentsMargins(0, 0, 0, 0)
self.container_layout.setSpacing(0)
self.container_layout.setAlignment(Qt.AlignmentFlag.AlignTop)
self.workspace_scroll.setWidget(self.container_widget)
self.main_layout.addWidget(self.workspace_scroll)
def add_track(self, track_id: str):
"""
Appends a unified track row utilizing QHBoxLayout with a rigid physical height constraint.
"""
track_row = QWidget()
track_row.setFixedHeight(96) # Lock physical pixel height constraints for the row
row_layout = QHBoxLayout(track_row)
row_layout.setContentsMargins(0, 0, 0, 0)
row_layout.setSpacing(0)
# Left Panel: Track Control Panel (Enforces a strict rigid width constraint)
tcp_widget = QWidget()
tcp_widget.setFixedWidth(300)
# tcp_widget.setup_ui(...)
# Right Panel: Waveform Canvas Viewport
waveform_widget = QWidget()
# waveform_widget.setup_canvas(...)
# Combine both widgets into the layout block to guarantee row lock
row_layout.addWidget(tcp_widget)
row_layout.addWidget(waveform_widget)
self.container_layout.addWidget(track_row)
```
### 3.2. Concrete Advantages for Python Docker Environments
* **Zero Alignment Variance:** Row alignment is entirely guaranteed at the OS-level layout engine, bypassing desynchronization issues caused by asynchronous rendering cycles or UI latency.
* **Streamlined UI Pipelines:** The environment tracking mechanisms hook into a single scrollbar, reducing memory usage and optimizing the drawing threads for the Docker X11 Server or WebRTC stream pipelines when projecting graphics down to the client.
9.2_UI_FIX.md
-133
View File
@@ -1,133 +0,0 @@
# Bug Fix Specification: Resolving Layout Overlaps & Synchronized Scroll Management (Scroll & Overlap Fix)
This document defines the technical solution to completely eliminate two critical layout overlap defects occurring during timeline scrolling operations, based on the real-world visual analysis.
---
## 1. Visual Overlap Analysis
Based on the graphical evidence, the system is experiencing user interface overlap (clipping) defects at two positions indicated by the red arrows:
### 1.1. Defect Index 1: Playhead and Grid Lines Overflowing Over the TCP
* **Symptom:** The red playback cursor (Playhead) and the vertical time grid markers (Ruler/Grid lines) render on top of the left Track Control Panel (TCP) during horizontal scrolling.
* **Root Cause:** The Timeline bounding container lacks an independent visual clipping boundary (`overflow: hidden`) relative to the TCP column. Alternatively, the TCP lacks a sufficient rendering layer tier (`z-index`) and a solid background color, which allows absolute-positioned elements from the Timeline to float over the TCP stack.
### 1.2. Defect Index 2: Horizontal Scrollbar Overflowing Underneath the TCP Base
* **Symptom:** The global horizontal scrollbar at the bottom of the viewport extends across the lower quadrant of the TCP all the way to the far left edge of the screen.
* **Root Cause:** The absolute outermost parent container wrapping both the TCP and the Timeline has been assigned horizontal scrolling properties, or the Timeline column is not physically isolated (as adjacent flex columns) from the TCP section.
---
## 2. Structural Architecture: Synchronized Dual-Column Viewports (Split-Container Sync)
To permanently resolve these defects, the workspace must completely isolate the two main columns into distinct physical viewports while linking their vertical scroll movements using JavaScript or UI event signals:
```text
[ MASTER WORKSPACE - flex h-full overflow-hidden ]
┌──────────────────────────────┬──────────────────────────────────────────────┐
│ [LEFT COLUMN - TCP PANEL] │ [RIGHT COLUMN - TIMELINE SCROLL VIEWPORT] │
│ - Width: 300px (Fixed) │ - flex-1 │
│ - overflow: hidden │ - overflow-x: auto (Isolated Horiz. Scroll) │
│ - z-index: 20 (Layer Top) │ - overflow-y: auto (Isolated Vert. Scroll) │
│ - bg: #262626 (Solid Solid) │ - z-index: 10 │
│ │ │
│ ┌──────────────────────────┐ │ ┌──────────────────────────────────────────┐ │
│ │ TCP Track 01 │ │ │ Waveform Track 01 │ │
│ ├──────────────────────────┤ │ ├──────────────────────────────────────────┤ │
│ │ TCP Track 02 │ │ │ Waveform Track 02 │ │
│ └──────────────────────────┘ │ └──────────────────────────────────────────┘ │
└──────────────────────────────┴──────────────────────────────────────────────┘
▲ │
│ [JS Vertical Scroll Sync Link] │
└──────────────────────────────────────▼
tcpContainer.scrollTop = timelineContainer.scrollTop
```
### 2.1. Rendering Priority and Containment Rules
* **TCP Panel:** Configured with `position: relative`, `z-index: 20`, and a solid `background-color: #262626`. Consequently, when the Timeline viewport scrolls horizontally to the left, all waveform vectors and the absolute playhead path automatically scroll beneath the TCP panel layer, masking them perfectly from view.
* **Timeline Wrapper:** Positioned immediately adjacent to the TCP column, utilizing `overflow-x: auto` and `overflow-y: auto`. The horizontal scrollbar will strictly begin rendering at coordinate $x = 300\text{ px}$ stretching rightward, preventing it from clipping the bottom area of the TCP.
---
## 3. Mouse Wheel Interaction Mechanics
The translation of scrolling gestures across the timeline canvas depends on the following hardware modifier key bindings:
### A. Standard Mouse Wheel Rotation (Vertical Scroll)
* **User Action:** The user rotates the mouse wheel up or down while hovering over the Timeline area.
* **Result:** The layout executes native vertical scrolling. The browser triggers the `onScroll` event listener loop, and the synchronization script immediately maps the offset values:
$$\text{scrollTop}_{\text{TCP}} = \text{scrollTop}_{\text{Timeline}}$$
This forces both columns to move up and down in absolute physical alignment.
### B. Shift + Mouse Wheel Rotation (Horizontal Scroll)
* **User Action:** The user holds down the `Shift` key while rotating the mouse wheel up or down.
* **Result:** The system intercepts the input and cross-routes vertical scrolling vectors into the horizontal scroll register:
$$\text{scrollLeft}_{\text{Timeline}} \mathrel{+}= \Delta y$$
The Timeline viewport shifts horizontally left or right, letting the editor browse across different segments of the arrangement timeline.
---
## 4. Porting Guidelines for Python Docker Applications (PyQt6 / PySide6)
When porting this split-container layout blueprint to a containerized Python desktop application, instantiate two independent `QScrollArea` nodes positioned side by side within a horizontal layout (`QHBoxLayout`), then connect their vertical scrollbar signals (`verticalScrollBar`):
```python
from PyQt6.QtWidgets import QWidget, QHBoxLayout, QScrollArea, QVBoxLayout
from PyQt6.QtCore import Qt
class SyncedDAWWorkspace(QWidget):
def __init__(self):
super().__init__()
layout = QHBoxLayout(self)
layout.setContentsMargins(0, 0, 0, 0)
layout.setSpacing(0)
# 1. Initialize the Left TCP Scroll Area (Enforce absolute scrollbar concealment)
self.tcp_scroll = QScrollArea()
self.tcp_scroll.setFixedWidth(300)
self.tcp_scroll.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
self.tcp_scroll.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
self.tcp_scroll.setWidgetResizable(True)
# 2. Initialize the Right Timeline Scroll Area (Enable bidirection scroll mapping)
self.timeline_scroll = QScrollArea()
self.timeline_scroll.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOn)
self.timeline_scroll.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOn)
self.timeline_scroll.setWidgetResizable(True)
layout.addWidget(self.tcp_scroll)
layout.addWidget(self.timeline_scroll)
# 3. VERTICAL SYNC BINDING: Redirect Timeline scrolling directly to the TCP axis
self.timeline_scroll.verticalScrollBar().valueChanged.connect(
self.tcp_scroll.verticalScrollBar().setValue
)
def eventFilter(self, obj, event):
"""
Intercepts WheelEvents on the Timeline view to handle Shift + Horizontal Scrolling.
"""
if obj == self.timeline_scroll.viewport() and event.type() == event.Type.Wheel:
if event.modifiers() & Qt.KeyboardModifier.ShiftModifier:
# Convert vertical wheel delta into horizontal scroll offset step increments
num_degrees = event.angleDelta().y() / 8
num_steps = num_degrees / 15
self.timeline_scroll.horizontalScrollBar().setValue(
self.timeline_scroll.horizontalScrollBar().value() - num_steps * 30
)
return True # Halt event propagation as it is now fully handled
return super().eventFilter(obj, event)
```
9_TRACK_UI.md
-191
View File
@@ -1,191 +0,0 @@
# Technical Specification: Multi-Channel Layout Synchronization & Scroll Management (Unified DAW Layout & Sync Scroll)
This document analyzes and defines the structural hierarchy of the graphical user interface based on the real-world interface analysis. This specification serves to guide Frontend interface programming and porting to a Python Desktop application running inside a Docker container.
---
## 1. Structural Wireframe
Based on the visual analysis, the layout composition is split into vertically static and dynamic zones:
```text
+───────────────────────────────────────────────────────────────────────────────+
| [ZONE A - STATIC] HEADER ZONE (Sticky - Permanently fixed when scrolling down)|
| +────────────────────+──────────────────────────────────────────────────────+ |
| | Channels & Tools | Time Ruler Scale | |
| |--------------------|------------------------------------------------------| |
| | Tempo Track Header | Tempo Grid Lane (120 BPM) | |
| +────────────────────+──────────────────────────────────────────────────────+ |
+───────────────────────────────────────────────────────────────────────────────+
| [ZONE B - DYNAMIC] TRACKS SCROLL WORKSPACE (Synchronized vertical scroll) |
| +────────────────────+──────────────────────────────────────────────────────+ |
| | TCP - Track 01 | Waveform Lane - Track 01 | |
| | TCP - Track 02 | Waveform Lane - Track 02 | |
| | TCP - Track 03 | Waveform Lane - Track 03 | |
| | ... | ... | |
| +────────────────────+──────────────────────────────────────────────────────+ |
+───────────────────────────────────────────────────────────────────────────────+ ▲
│ [Vertical Scrollbar]
│ (Single unified scroll)
▼
```
---
## 2. Layout Specifications
### 2.1. Fixed Header Zone (Green Border Area - Sticky Header)
* **Visual Scope:** Encompasses the toolbar, the time ruler scale, and the Tempo Track Lane (indicated by the green bounding border in `image_fbbd4e.png`).
* **Graphical Sticky Behavior:**
* When a user adds dozens of tracks and scrolls downward, this entire zone must remain anchored to the top of the screen and is not permitted to slide out of view.
* This ensures that users can continuously track the Ruler Seconds and the master project tempo (Tempo BPM) while editing tracks located deeper down the timeline.
### 2.2. Absolute Horizontal Row Alignment (Red Border Area - Row Alignment)
* **Interaction Scope:** The exact matching pair consisting of the left Track Control Panel (TCP) and the right Waveform Lane of the same track (e.g., Track 4 inside the red border of `image_fbbd4e.png`).
* **Row Alignment Rules:**
* The corresponding TCP and Waveform Lane must have identical heights ($H = 96\text{ px}$).
* These two elements must be wrapped within a single parent row container (`Flex Row` or `Grid Row`) to guarantee that during vertical scrolling, both move simultaneously along the exact same vertical axis coordinate ($Y$).
* Row misalignment must be strictly avoided (e.g., situations where the Track 4 TCP sits higher or lower than the Track 4 Waveform lane).
### 2.3. Single Vertical Scrollbar Mandate
* **Issue to Avoid:** Separating the TCP into an independent scrollable column and the Timeline into another independent scrollable column. Doing so leads to scroll-position desynchronization errors when a user drags the scrollbar.
* **Design Standard:**
* Only a single unified Vertical Scrollbar is permitted to appear on the absolute far right of the application window (as directed by the two red arrows in `image_fbbd4e.png`).
* This vertical scrollbar moves the entire dynamic wrapper (**Tracks Scroll Workspace**), scrolling both TCPs and Waveform Lanes up or down in sync.
---
## 3. Implementation Guide
### 3.1. Web Frontend Integration (HTML / Tailwind CSS)
To group everything into one scrollbar while keeping the Tempo Track anchored at the top, use `position: sticky` and wrap the dynamic track list inside a single container:
```html
<!-- Main Container (Entire Editor Wrapper) -->
<div class="flex flex-col h-full overflow-hidden">
<!-- [ZONE A] Top Anchored Sticky Header Zone -->
<div class="sticky top-0 z-40 bg-[#242424] border-b border-[#141414] shrink-0">
<!-- Toolbar & Time Ruler -->
<div class="h-8 flex">
<div class="w-[300px] border-r border-zinc-900 px-4 flex items-center">CHANNELS</div>
<div class="flex-1 relative h-full">...Ruler Numbers...</div>
</div>
<!-- Tempo Track (Green Border Area) -->
<div class="h-[44px] flex border-t border-zinc-800 bg-[#212121]">
<div class="w-[300px] border-r border-zinc-900 px-4 flex items-center justify-between">
<span class="font-bold text-zinc-400">Tempo Track</span>
<span class="bg-zinc-800 px-1.5 py-0.5 rounded text-[10px]">120 BPM</span>
</div>
<div class="flex-1">...Tempo Grid Lines...</div>
</div>
</div>
<!-- [ZONE B] Dynamic Track Workspace (Single global vertical scrollbar on the far right) -->
<div class="flex-1 overflow-y-auto bg-[#1a1a1a]">
<div class="flex flex-col divide-y divide-[#141414]">
<!-- Track Row Container (Absolute Horizontal Row Alignment) -->
<div class="h-[96px] flex hover:bg-zinc-800/20 transition-colors">
<!-- Left: TCP -->
<div class="w-[300px] border-r border-zinc-900 p-2.5 flex-shrink-0">
...Controls (Mute, Solo, Volume, File Name)...
</div>
<!-- Right: Waveform Lane -->
<div class="flex-1 relative overflow-hidden">
...Waveform Canvas...
</div>
</div>
<!-- Add more track rows repeating the structure above... -->
</div>
</div>
</div>
```
### 3.2. Desktop App Integration (Python PyQt6)
When engineering this user interface using the Qt framework in Python, utilize a `QScrollArea` to encapsulate a `QWidget` managed by a layout of rows to control the single scrollbar behavior:
```python
from PyQt6.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QScrollArea, QLabel
from PyQt6.QtCore import Qt
class MasterDAWWidget(QWidget):
def __init__(self):
super().__init__()
self.main_layout = QVBoxLayout(self)
self.main_layout.setContentsMargins(0, 0, 0, 0)
self.main_layout.setSpacing(0)
# 1. Initialize Fixed Header (Toolbar, Ruler, Tempo)
self.header_widget = QWidget()
self.header_widget.setFixedHeight(76) # 32px Ruler + 44px Tempo
self.setup_header_ui()
self.main_layout.addWidget(self.header_widget)
# 2. Initialize Scroll Area for dynamic track rows
self.scroll_area = QScrollArea()
self.scroll_area.setWidgetResizable(True)
# Force a single vertical scrollbar on the far right
self.scroll_area.setVerticalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOn)
self.scroll_area.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
# Widget container hosting the track list inside the Scroll Area
self.tracks_container = QWidget()
self.tracks_layout = QVBoxLayout(self.tracks_container)
self.tracks_layout.setContentsMargins(0, 0, 0, 0)
self.tracks_layout.setSpacing(0)
self.tracks_layout.setAlignment(Qt.AlignmentFlag.AlignTop)
self.scroll_area.setWidget(self.tracks_container)
self.main_layout.addWidget(self.scroll_area)
def add_track_row(self, track_id, track_name):
"""
Appends a new track row. Uses QHBoxLayout to lock the TCP and Waveform Lane
into absolute horizontal sync within the row.
"""
row_widget = QWidget()
row_widget.setFixedHeight(96) # Rigid constraint for the entire row
row_layout = QHBoxLayout(row_widget)
row_layout.setContentsMargins(0, 0, 0, 0)
row_layout.setSpacing(0)
# Left: Track Control Panel (TCP)
tcp_widget = QWidget()
tcp_widget.setFixedWidth(300)
# Setup TCP UI components...
row_layout.addWidget(tcp_widget)
# Right: Waveform Lane
waveform_widget = QWidget()
# Setup Waveform Canvas Painter...
row_layout.addWidget(waveform_widget)
self.tracks_layout.addWidget(row_widget)
```
---
## 4. Layout Architecture Advantages
* **Fluid User Experience:** Eliminates row-stuttering or scrolling layout shifts between the control panels and audio visuals when a user scrolls through long track stacks rapidly.
* **Flawless Python Porting Compatibility:** By wrapping the TCP and the Waveform Canvas inside a common row (`QHBoxLayout` in Qt or `Flex Row` in Web), the core widget tree hierarchy remains incredibly lean. This design removes the need to write custom coordinate bridging code to bind two separate scroll engines together.
* **Clean Interface Aesthetics:** Safely protects the pixel rendering mapping ratios of the fixed time grids at the top, precisely matching the professional DAW interface conventions observed
README.md
+2
View File
@@ -4,6 +4,8 @@
SonicForge Studio là một hệ thống xử lý âm thanh chuyên nghiệp kết hợp giao diện Web Audio API phía client với công cụ DSP/AI mạnh mẽ trên server (Python/Celery). SonicForge Studio là một hệ thống xử lý âm thanh chuyên nghiệp kết hợp giao diện Web Audio API phía client với công cụ DSP/AI mạnh mẽ trên server (Python/Celery).
![SonicForge Studio UI](./app/images/SonicForgeUI.png)
## 🎯 Tính Năng Chính ## 🎯 Tính Năng Chính
### Client-side (Web Audio API) ### Client-side (Web Audio API)