Compare commits
15 Commits
8363a46499
...
32fab9d366
| Author | SHA1 | Date | |
|---|---|---|---|
| 32fab9d366 | |||
| b1f9659f06 | |||
| b39bfbc1bc | |||
| e1b6f47ad0 | |||
| 2894fae19b | |||
| 4744b2c485 | |||
| dce104221d | |||
| 60dce0135e | |||
| 66b0811f55 | |||
| da923a7a79 | |||
| acee68e8f5 | |||
| 95dc9346ef | |||
| 615e0e8530 | |||
| d2e0010d88 | |||
| a6ec14b38b |
@@ -0,0 +1,164 @@
|
||||
# Technical Specification: Advanced UI Refactoring & Clip Editing Mechanics
|
||||
|
||||
This document defines the improved user interface design and advanced audio interaction algorithms to standardize frontend development and porting workflows into a containerized Python DAW application running on Docker.
|
||||
|
||||
---
|
||||
|
||||
## 1. UI Refactoring Specification
|
||||
|
||||
### 1.1. Resolving TCP Horizontal Scroll Overflows (Horizontal Scroll Isolation)
|
||||
|
||||
* **Symptom:** When scrolling horizontally across the Timeline, waveform or grid canvas elements incorrectly render on top of the left Track Control Panel (TCP) region.
|
||||
* **Refactoring Solution:** Enforce strict visual separation using Flexbox constraints. The master arrangement window (Workspace) is divided into two physically adjacent columns with completely isolated presentation variables:
|
||||
|
||||
```css
|
||||
.tcp-column {
|
||||
width: 300px;
|
||||
flex-shrink: 0;
|
||||
position: relative;
|
||||
z-index: 30; /* Ensures columns stay stacked on top */
|
||||
background-color: #262626; /* Solid, opaque color mask */
|
||||
overflow: hidden;
|
||||
}
|
||||
.timeline-viewport {
|
||||
flex: 1 1 0%;
|
||||
position: relative;
|
||||
z-index: 10;
|
||||
overflow-x: auto;
|
||||
overflow-y: hidden; /* Restricts column to independent horizontal scrolling */
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
### 1.2. Prominent Shortcut Labels
|
||||
|
||||
* **Graphical Standard:** Scale up text components displaying key combination hints within system dropdown menus and right-click Context Menus.
|
||||
* **Layout Mapping Properties:**
|
||||
* Keyboard shortcut font sizing: Scaled up from 10px to 12px (`text-[12px]`).
|
||||
* Weight property: Configured to `font-semibold`.
|
||||
* High-contrast color palette: Replace low-contrast gray strings with vivid purple (`text-purple-400` / `#c084fc`) or neon amber (`text-amber-400` / `#fbbf24`) that pop cleanly over the dark `#1e1e1e` canvas backdrop.
|
||||
* Structural alignment: Push shortcut labels directly to the right edge of the context window (`ml-auto pl-8`).
|
||||
|
||||
|
||||
|
||||
### 1.3. Enlarged & Centered Toolbar
|
||||
|
||||
* **Layout Adjustment:** Primary editing triggers (Cut, Copy, Paste, Snap) are scaled up to $1.5\times$ their legacy sizing boundaries (button height locked at 40px).
|
||||
* **Viewport Placement:** Move the button group into the center cluster on the same horizontal row plane as the ruler axis (positioned immediately to the left of the Time Ruler). This ensures the sound engineer's focus safely encapsulates macro controls alongside timeline visuals.
|
||||
|
||||
---
|
||||
|
||||
## 2. Timeline Mechanics & Advanced Clip Editing
|
||||
|
||||
### 2.1. Clip Delete vs. Track Delete Logic
|
||||
|
||||
The environment explicitly segregates asset deletions from track configurations to protect project layout structures:
|
||||
|
||||
* **Clip Erasure (`Delete` Key):** When a user triggers `Delete` or `Backspace` keys while an active Audio Clip segment is selected, the application drops the graphical boundary and unloads its corresponding sample sequence from the timeline. The containing track channel remains safely intact.
|
||||
* **Track Disassembly (TCP Delete Button):** Add a compact red trash bin icon (`w-4 h-4 text-red-500 hover:text-red-400`) into the right edge profile of every TCP block. Engaging this trigger purges the entire track lane along with all embedded clip blocks out of the project.
|
||||
|
||||
### 2.2. Preserve Selection Border Resize
|
||||
|
||||
* **Legacy Behavior:** Clicking or interacting directly with selection handles accidentally flags a focus reset, clearing the bôi màu canvas overlay.
|
||||
* **Preserve & Scale System:**
|
||||
* When hovering the mouse near the explicit left or right edge boundaries of an active selection zone (within a $\pm 5\text{ px}$ tolerance window), the cursor style changes to `ew-resize`.
|
||||
* Triggering a mouse drag updates, expands, or shrinks selection markers continuously without clearing the overlay mask.
|
||||
|
||||
|
||||
* **Escape Loop Hook (Clear Focus):** The colored selection range is unmapped if and only if the user executes a `Ctrl + Click` shortcut interaction over an empty, unpopulated quadrant outside the selection bounds.
|
||||
|
||||
### 2.3. Sub-tab Sandboxing
|
||||
|
||||
When a user highlights a clip portion and triggers "Edit in Sub-tab" or double-clicks a targeted audio asset clip:
|
||||
|
||||
1. **Buffer Extraction:** The system maps a non-destructive copy of the target sub-region's audio slice into memory buffers.
|
||||
2. **Tab Instantiation:** Appends a temporary document window onto the global Tab container bar (e.g., `Tab: Sample_Edit_1`).
|
||||
3. **Automated Insertion:** Instantiates a single empty track channel workspace inside the tab context and drops the cloned audio segment at the absolute root milestone ($t = 0.0\text{ s}$). Editors evaluate local actions here before clicking *Apply* to pass the updated data payload back to the main session track.
|
||||
|
||||
### 2.4. Zero-Crossing Filter Tool & AI Cut
|
||||
|
||||
Automated crossfade calculation mechanics to eliminate popping anomalies during clip slicing:
|
||||
|
||||
1. A user selects a timeline region and hits the *AI Analysis* utility.
|
||||
2. The server processes the audio block using NumPy arrays to locate phase inversion milestones (where amplitude values cross from negative to positive indices or vice versa) closest to the selection boundary vectors:
|
||||
|
||||
$$x[i] \cdot x[i+1] \le 0$$
|
||||
|
||||
|
||||
3. The engine moves the actual slice boundaries to match these optimized zero-crossing sample addresses ($t'_{\text{start}}$ and $t'_{\text{end}}$).
|
||||
4. Upon clicking *AI Cut*, the underlying engine runs the physical audio slice at the perfect sample indices, duplicates the segment, and appends it to a freshly populated track lane added right below the source track.
|
||||
|
||||
### 2.5. Track Height Resizing
|
||||
|
||||
* **Interaction:** Users can hover over the dividing line between two track lanes on either the left TCP column or the right Waveform viewport (the cursor scales to `ns-resize`).
|
||||
* **Drag-and-Drop Mapping:** Dragging downward expands the specific track lane vertical ceiling (up to an upper bound of $200\text{ px}$), magnifying waveform amplitude layouts for precision edits. Dragging upward reduces the height dimension (down to a lower ceiling of $48\text{ px}$) for macro project navigation.
|
||||
|
||||
### 2.6. Time-Stretching & Speed Math
|
||||
|
||||
Alters the playback rate (*Speed*) of audio clips directly from the interactive timeline view:
|
||||
|
||||
```text
|
||||
[ Right Boundary Drag Interaction ]
|
||||
Alt + Left-Click & Drag the right border outwards (Expand)
|
||||
|<────────────────── Original Clip ──────────────────>|
|
||||
+─────────────────+─────────────────────────────────────────────────────+──────────+
|
||||
| Track Waveform | ███████████████████████████████████████████████████ | |
|
||||
+─────────────────+─────────────────────────────────────────────────────+──────────+
|
||||
▲ ▲
|
||||
│ │
|
||||
│ ▼ [ Expand Rightward ]
|
||||
+─────────────────+────────────────────────────────────────────────────────────────+
|
||||
| Track Waveform | █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ |
|
||||
+─────────────────+────────────────────────────────────────────────────────────────+
|
||||
│ │
|
||||
│ Visual Speed Tag: "Speed: 50%" │
|
||||
|<────────────────────────── D' ─────────────────────────────>|
|
||||
|
||||
```
|
||||
|
||||
* **Modifier Binding:** Hold down the `Alt` key, left-click the rightmost bounding handle of a clip, and drag the boundary left or right.
|
||||
* **Speed Ratio Formula ($S$):** Let $D$ represent the native unscaled duration value of the clip block (seconds), and $D'$ map to the modified duration value generated post-drag (seconds). The calculation for the updated target playback rate percentage ($S$) follows:
|
||||
|
||||
$$S = \frac{D}{D'} \times 100\%$$
|
||||
|
||||
|
||||
* **Display Modifiers:**
|
||||
* *Expanding rightward ($D' > D$):* Yields $S < 100\%$, meaning playback velocity drops (deceleration). Depending on DSP choices, pitches can either remain locked or drop proportionally.
|
||||
* *Compressing leftward ($D' < D$):* Yields $S > 100\%$, accelerating the playback engine velocity through the clip.
|
||||
|
||||
|
||||
* **Visual Metadata Tag:** A bright yellow text overlay displaying the calculated playback velocity percentage (e.g., `Speed: 75.0%` or `Speed: 120.5%`) is pinned directly to the upper-left boundary of the audio clip container.
|
||||
|
||||
---
|
||||
|
||||
## 3. Python Porting Manual (PyQt6 / PySide6)
|
||||
|
||||
When writing execution blocks for time-stretching and audio rate modulations onto the Python backend server layers, leverage standard scientific audio packages such as `numpy` or `rubberband` to scale signal arrays without warping Phase layouts:
|
||||
|
||||
```python
|
||||
# [PYTHON PORTING BLUEPRINT] - Acoustic Time-Stretching Velocity Algorithm
|
||||
import numpy as np
|
||||
import librosa
|
||||
|
||||
def stretch_audio_clip_speed(y: np.ndarray, sr: int, speed_ratio: float) -> np.ndarray:
|
||||
"""
|
||||
Stretches or compresses a NumPy audio signal array using the target speed_ratio factor.
|
||||
speed_ratio = 0.5 slows down velocity by half (expanding physical layout width by 2x).
|
||||
speed_ratio = 2.0 doubles velocity (compressing physical layout width by half).
|
||||
"""
|
||||
if speed_ratio == 1.0:
|
||||
return y
|
||||
|
||||
# Phase Vocoder approach via Librosa to alter speed while locking pitch (Pitch-preserving stretch):
|
||||
# y_stretched = librosa.effects.time_stretch(y, rate=speed_ratio)
|
||||
|
||||
# Linear Resampling approach (Alters pitch along with velocity - vinyl style deceleration):
|
||||
num_samples_new = int(len(y) / speed_ratio)
|
||||
y_resampled = np.interp(
|
||||
np.linspace(0, len(y) - 1, num_samples_new),
|
||||
np.arange(len(y)),
|
||||
y
|
||||
)
|
||||
return y_resampled.astype(np.float32)
|
||||
|
||||
```
|
||||
+287
@@ -0,0 +1,287 @@
|
||||
# Technical Specification: Sandbox Isolation & Sub-Tab DSP Editing Algorithms
|
||||
|
||||
This document defines the processing workflow design and digital signal processing (DSP) algorithms dedicated to localized clip editing within a temporary isolated document workspace (Sub-tab).
|
||||
|
||||
---
|
||||
|
||||
## 1. Sandbox Splicing Workflow
|
||||
|
||||
When a user highlights a time region on the Main Tab and triggers "Edit in Sub-tab" or presses the edit keyboard shortcut:
|
||||
|
||||
```text
|
||||
[ MAIN TAB - MULTITRACK ]
|
||||
Track 01: ───[█████ Selected Segment █████]───
|
||||
│
|
||||
▼ (Copy to Clipboard Buffer)
|
||||
[ KHỔI TẠO TAB TẠM THỜI (SUB-TAB) ]
|
||||
- Instantiates a single Track (Height bounds: 48px - 200px via ns-resize)
|
||||
- Timeline Ruler axis resets to t = 0.0s
|
||||
│
|
||||
▼ (Automated Insertion - Auto-Paste)
|
||||
Track 01 (Sub-tab): [█████ Isolated Segment █████] at t = 0s
|
||||
|
||||
```
|
||||
|
||||
* **Extract Buffer:** The underlying engine extracts the binary sample array (`Float32Array`) of the highlighted region from the active track, caching it securely into the application's clipboard buffer memory.
|
||||
* **Sandbox Environment Initialization:**
|
||||
* Appends a temporary document window onto the global Tab Bar (e.g., `Tab: sẤit tiá...n` or `Sub_Edit_1`).
|
||||
* Focuses the viewport down into the sandboxed tab. Here, a single standalone track lane is drawn, mapping the timeline ruler scale to start at $t = 0.0\text{ s}$ up to the absolute duration limit ($T_{\text{clip}}$) of the extracted audio asset.
|
||||
|
||||
|
||||
* **Track Height Resizing:**
|
||||
* Hovering the cursor over the lower layout bounding path of the track lane changes the style configuration to `ns-resize`.
|
||||
* Dragging downward expands the vertical height ceiling (up to an upper bound of $200\text{ px}$), maximizing the waveform amplitude drawing path for precision clip editing. Dragging upward compresses the physical row dimensions (down to a lower constraint of $48\text{ px}$) to protect screen space.
|
||||
|
||||
|
||||
* **Auto-Paste Routine:** The framework automates the insertion sequence, dropping the cached array block onto the root index milestone ($t = 0.0\text{ s}$) inside the isolated single-track layer.
|
||||
|
||||
---
|
||||
|
||||
## 2. Apply & Sync-Back Workflow
|
||||
|
||||
When an editor finishes processing steps inside the sandbox workspace and engages the *Apply* action:
|
||||
|
||||
```text
|
||||
[ SUB-TAB - AUDIO SANDBOX ]
|
||||
y_sub = [█████ Edited Waveform █████]
|
||||
│
|
||||
▼ (Click "Apply" - Trigger Overwrite)
|
||||
[ MAIN TAB - ORIGINAL TRACK ]
|
||||
Track 01: ───[█████ Overwritten Segment █████]─── at t = t_start
|
||||
│
|
||||
(Sub-tab remains open, Undo Stack kept)
|
||||
▼
|
||||
[User presses Undo (Ctrl+Z) inside Sub-tab to iterate]
|
||||
y_sub = [█████ Rollbacked Waveform █████]
|
||||
│
|
||||
(Click "Apply" again)
|
||||
▼
|
||||
Track 01: ───[█████ Corrected Segment █████]──── at t = t_start
|
||||
|
||||
```
|
||||
|
||||
### 2.1. Target Mapping & Metadata Linkage
|
||||
|
||||
Throughout its lifecycle, each sub-tab persistently locks standard metadata records linking back to the origin source elements:
|
||||
|
||||
* `parent_track_id`: Unique identifier referencing the primary source track on the Main Tab.
|
||||
* `parent_clip_id`: Unique identifier tracking the original source audio clip.
|
||||
* `t_start` (seconds): The exact historical start time position of the sliced block on the Main Tab timeline view.
|
||||
* `original_duration` (seconds): The baseline temporal duration of the region prior to modification.
|
||||
|
||||
### 2.2. In-place Overwrite & Splicing
|
||||
|
||||
* **Edited Buffer Extraction:** The system reads the active sample sequence from the sub-tab ($y_{\text{sub}}$) along with its updated duration boundary $T_{\text{sub}}$ (which fluctuates if time-stretching or rate scaling actions have occurred).
|
||||
* **Main Session Integration:**
|
||||
1. The core route mapper checks for the matching `parent_track_id` parameter on the Main Tab.
|
||||
2. Purges the legacy audio segment stretching from $t_{\text{start}}$ through $t_{\text{start}} + T_{\text{original}}$.
|
||||
3. Splices the updated signal array $y_{\text{sub}}$ precisely at the historical insertion index $t_{\text{start}}$.
|
||||
4. **Micro-crossfade:** Executes a ultra-fast crossfade envelope ($10\text{ ms}$) across both the initial and terminating splice boundaries. Blending adjacent files prevents phase cancellation or signal breakage that manifests as transient clicks/pops.
|
||||
|
||||
|
||||
* **Visual Update Tracking:** Commands the canvas engine to redraw the waveform visualization grid for the origin track lane inside the Main Tab view.
|
||||
|
||||
### 2.3. Persistence for Iterative Editing
|
||||
|
||||
* **Tab Lifetime:** Engaging the *Apply* trigger propagates data back to the primary environment but does **not** close down the active sub-tab view.
|
||||
* **Undo Stack Isolation:** The tracking loop containing the localized *Undo/Redo History Stack* inside the sub-tab sandbox remains entirely preserved.
|
||||
* **Iterative Loop Workflow:**
|
||||
1. If monitoring the Main Tab arrangement uncovers an audio anomaly, the user switches focus back to the Sub-tab workspace.
|
||||
2. Pressing `Ctrl + Z` (Undo) rollbacks the localized signal to its earlier state.
|
||||
3. The editor runs separate DSP actions.
|
||||
4. Hitting *Apply* overwrites the updated audio slice over the same target coordinates on the Main Tab.
|
||||
|
||||
|
||||
* **Explicit Destruction Hook:** The sandboxed tab structure is only unmapped when the user clicks the explicit close icon ($\times$) on the horizontal tab bar.
|
||||
|
||||
---
|
||||
|
||||
## 3. Sub-Tab DSP Algorithm Specification
|
||||
|
||||
Editing operations executed inside the sub-tab environment calculate discrete changes over the amplitude sample arrays ($x[n]$). These map to Web Audio API routines on the client layer and standard NumPy/SciPy audio arrays on the Dockerized backend.
|
||||
|
||||
### 3.1. Time-Stretching & Speed Math
|
||||
|
||||
Alters the duration bounds of the audio clip with optional pitch-shifting linking logic:
|
||||
|
||||
* **Pitch-preserving Time-stretching:** Utilizes the Phase Vocoder method to analyze the Short-Time Fourier Transform (STFT) of the signal, shifts spectral frames across the frequency domain, and reconstructs the audio via the Inverse Short-Time Fourier Transform (ISTFT) to align with a new playback velocity ratio $S$:
|
||||
|
||||
$$S = \frac{D}{D'} \times 100\%$$
|
||||
|
||||
|
||||
|
||||
*Where:* $D$ corresponds to the legacy unscaled duration (seconds), and $D'$ maps to the updated value post-resizing (executed by holding down the `Alt` key and dragging the right boundary handle).
|
||||
* **Resampling (Pitch-shifting Speed Scale):** Runs a standard linear interpolation algorithm to resample the core data array size:
|
||||
|
||||
$$x_{\text{new}}[m] = x\left[ \frac{m \cdot D}{D'} \right]$$
|
||||
|
||||
|
||||
|
||||
### 3.2. Peak Normalization
|
||||
|
||||
Amplifies the signal scale uniformly across the active block until the single maximum absolute sample peak reaches a specified ceiling parameter $A_{\text{target}}$ (typically locked at $1.0$ or $0\text{ dBFS}$):
|
||||
|
||||
1. Evaluate the absolute maximum peak within the array bounds:
|
||||
|
||||
$$A_{\text{max}} = \max_{n=0}^{N-1} \vert x[n] \vert$$
|
||||
|
||||
|
||||
2. Compute the static gain multiplier constant $G$:
|
||||
|
||||
$$G = \frac{A_{\text{target}}}{A_{\text{max}}}$$
|
||||
|
||||
|
||||
3. Multiply the entire audio array values by $G$:
|
||||
|
||||
$$x_{\text{norm}}[n] = x[n] \cdot G$$
|
||||
|
||||
|
||||
|
||||
### 3.3. Volume Gain Adjustment (dB Scaling)
|
||||
|
||||
1. Capture the decibel variance target ($\Delta \text{dB}$).
|
||||
2. Translate the logarithmic value into a standard linear scalar multiplier variable $G_{\text{linear}}$:
|
||||
|
||||
$$G_{\text{linear}} = 10^{\frac{\Delta \text{dB}}{20}}$$
|
||||
|
||||
|
||||
3. Apply the gain multiplier directly into the sample values:
|
||||
|
||||
$$x_{\text{gained}}[n] = x[n] \cdot G_{\text{linear}}$$
|
||||
|
||||
|
||||
|
||||
### 3.4. Pitch Shifting
|
||||
|
||||
Shifts the fundamental frequencies of the signal up or down by a specific number of semitones ($n$) while keeping the temporal duration value completely intact.
|
||||
|
||||
* **Frequency Transposition Ratio ($F_{\text{ratio}}$):**
|
||||
|
||||
$$F_{\text{ratio}} = 2^{\frac{n}{12}}$$
|
||||
|
||||
|
||||
* **DSP Processing Pipeline:** Employs either a Pitch Synchronous Overlap and Add (PSOLA) routine or a spectral Phase Vocoder to expand/compress the frequency components, then passes the array into a time-stretching step to return the physical track length to its source metric $T_{\text{clip}}$.
|
||||
|
||||
### 3.5. Linear Fade-In & Fade-Out Curves
|
||||
|
||||
Applies a linear fading envelope over the boundaries of the audio data block.
|
||||
|
||||
* **Linear Fade-In Envelope** (Across a duration bound of $L_{\text{fade}}$ samples):
|
||||
|
||||
$$x_{\text{fade}}[n] = x[n] \cdot \left( \frac{n}{L_{\text{fade}}} \right) \quad \text{for } 0 \le n < L_{\text{fade}}$$
|
||||
|
||||
|
||||
* **Linear Fade-Out Envelope** (Across the final trailing $L_{\text{fade}}$ samples):
|
||||
|
||||
$$x_{\text{fade}}[N - 1 - n] = x[N - 1 - n] \cdot \left( \frac{n}{L_{\text{fade}}} \right) \quad \text{for } 0 \le n < L_{\text{fade}}$$
|
||||
|
||||
|
||||
|
||||
### 3.6. Array Splitting & Merging
|
||||
|
||||
* **Split at Position ($n_{\text{cut}}$):** Unlinks a single sample block $x[n]$ of size $N$ into two separate independent sub-arrays:
|
||||
|
||||
$$x_1[n] = x[n] \quad (0 \le n < n_{\text{cut}})$$
|
||||
|
||||
|
||||
$$x_2[n] = x[n + n_{\text{cut}}] \quad (0 \le n < N - n_{\text{cut}})$$
|
||||
|
||||
|
||||
* **Merge Segments:** Concatenates separate sample sequences end-to-end. The stitching logic runs a $10\text{ ms}$ micro-crossfade overlay envelope at the junction to smooth out phase gaps that prompt click artifacts.
|
||||
|
||||
---
|
||||
|
||||
## 4. Python Backend Implementation Manual
|
||||
|
||||
This prototype Python class (`core/sub_tab_dsp.py`) handles the sandboxed operations and includes the crossfaded structural splicing algorithm designed to run inside the Docker engine:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
import scipy.signal as signal
|
||||
import librosa
|
||||
|
||||
class SubTabDSPEngine:
|
||||
@staticmethod
|
||||
def change_speed(y: np.ndarray, sr: int, speed_ratio: float, preserve_pitch: bool = True) -> np.ndarray:
|
||||
"""
|
||||
Alters the playback velocity (Time-Stretching) of a NumPy signal array.
|
||||
"""
|
||||
if speed_ratio == 1.0:
|
||||
return y
|
||||
|
||||
if preserve_pitch:
|
||||
return librosa.effects.time_stretch(y, rate=speed_ratio)
|
||||
else:
|
||||
num_samples_new = int(len(y) / speed_ratio)
|
||||
return signal.resample(y, num_samples_new)
|
||||
|
||||
@staticmethod
|
||||
def normalize(y: np.ndarray, target_db: float = 0.0) -> np.ndarray:
|
||||
"""
|
||||
Performs Peak Normalization on an array to scale it to the target decibel value.
|
||||
"""
|
||||
target_amplitude = 10.0 ** (target_db / 20.0)
|
||||
max_amplitude = np.max(np.abs(y))
|
||||
|
||||
if max_amplitude == 0:
|
||||
return y
|
||||
|
||||
gain = target_amplitude / max_amplitude
|
||||
return y * gain
|
||||
|
||||
@staticmethod
|
||||
def merge_back_to_parent(
|
||||
parent_track_audio: np.ndarray,
|
||||
sr: int,
|
||||
edited_sub_audio: np.ndarray,
|
||||
start_seconds: float,
|
||||
original_duration_seconds: float
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Splices the modified audio segment from the Sub-tab back into the parent track array.
|
||||
Applies a 10ms micro-crossfade at the boundaries to eliminate pop/click noise.
|
||||
"""
|
||||
start_sample = int(start_seconds * sr)
|
||||
original_samples_len = int(original_duration_seconds * sr)
|
||||
edited_samples_len = len(edited_sub_audio)
|
||||
crossfade_samples = int(0.01 * sr) # 10ms crossfade window
|
||||
|
||||
# 1. Allocate the target output array dimension bounds
|
||||
new_total_len = len(parent_track_audio) - original_samples_len + edited_samples_len
|
||||
output_audio = np.zeros(new_total_len, dtype=np.float32)
|
||||
|
||||
# 2. Extract leading unedited block
|
||||
output_audio[:start_sample] = parent_track_audio[:start_sample]
|
||||
|
||||
# 3. Stitch the modified audio payload
|
||||
output_audio[start_sample:start_sample + edited_samples_len] = edited_sub_audio
|
||||
|
||||
# 4. Extract trailing unedited block
|
||||
post_start_original = start_sample + original_samples_len
|
||||
post_start_new = start_sample + edited_samples_len
|
||||
output_audio[post_start_new:] = parent_track_audio[post_start_original:]
|
||||
|
||||
# 5. Execute micro-crossfade across the initial splice junction
|
||||
if start_sample > crossfade_samples:
|
||||
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
|
||||
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
|
||||
|
||||
# Smooth 10ms interpolation overlay
|
||||
output_audio[start_sample : start_sample + crossfade_samples] = (
|
||||
edited_sub_audio[:crossfade_samples] * fade_in_ramp +
|
||||
parent_track_audio[start_sample : start_sample + crossfade_samples] * fade_out_ramp
|
||||
)
|
||||
|
||||
# 6. Execute micro-crossfade across the trailing splice junction
|
||||
if post_start_new + crossfade_samples < len(output_audio):
|
||||
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
|
||||
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
|
||||
|
||||
output_audio[post_start_new : post_start_new + crossfade_samples] = (
|
||||
parent_track_audio[post_start_original : post_start_original + crossfade_samples] * fade_in_ramp +
|
||||
edited_sub_audio[-crossfade_samples:] * fade_out_ramp
|
||||
)
|
||||
|
||||
return output_audio
|
||||
|
||||
```
|
||||
@@ -0,0 +1,233 @@
|
||||
# Technical Specification: Advanced Editing Toolset & Volume Automation Envelope on Sub-Tab
|
||||
|
||||
This document defines the interactive layout design and signal processing algorithms for the advanced localized editing toolset contained within the isolated temporary document workspace (Sub-tab).
|
||||
|
||||
---
|
||||
|
||||
## 1. Target Selection Scope
|
||||
|
||||
The toolset within the Sub-tab environment supports two target operational boundaries:
|
||||
|
||||
* **Global Clip:** When no specific timeline selection highlighted mask is present, all active DSP effects apply uniformly across the entire length of the extracted Audio Clip.
|
||||
* **Selected Range:** When an explicit timeline segment $[T_{\text{start}}, T_{\text{end}}]$ is highlighted by the user, DSP routines calculate changes exclusively inside those boundaries. Splice junctions automatically compute crossfades to mitigate transient click/pop anomalies.
|
||||
|
||||
---
|
||||
|
||||
## 2. Ruler-Based Tools
|
||||
|
||||
These utilities display as intuitive, linear slider scales (Sliders/Rulers) embedded in the top toolbar row:
|
||||
|
||||
```text
|
||||
[ Normalize: |======o======| 0 dB ] [ Gain: |====o====| +3 dB ] [ Pitch: |==o==| -2 Semi ]
|
||||
|
||||
```
|
||||
|
||||
### 2.1. Peak Normalization
|
||||
|
||||
* **UI Layout:** A slide scale control allowing users to configure target amplitude thresholds variable from $-12\text{ dBFS}$ down to $0\text{ dBFS}$.
|
||||
* **DSP Math Algorithm:** Locate the maximum absolute peak amplitude value $A_{\text{max}}$ within the targeted area, then multiply all active samples by a static scalar gain multiplier $G$:
|
||||
|
||||
$$G = \frac{10^{\frac{\text{Target\_dB}}{20}}}{A_{\text{max}}}$$
|
||||
|
||||
|
||||
|
||||
### 2.2. Volume Up / Down (Quick Gain)
|
||||
|
||||
* **UI Layout:** A linear sliding ruler modulating the overall absolute gain structure of the focused segment.
|
||||
* **Operational Range:** Adjustable from $-\infty\text{ dB}$ (complete mute attenuation) up to $+12\text{ dB}$ of linear amplification.
|
||||
|
||||
### 2.3. Pitch Shifting
|
||||
|
||||
* **UI Layout:** A calibrated slider modifying the project's fundamental frequencies discrete in semitones or cents.
|
||||
* **Operational Range:** Boundaries map from $-12\text{ semitones}$ (one octave down) to $+12\text{ semitones}$ (one octave up).
|
||||
* **DSP Engine Routine:** Employs a spectral Phase Vocoder to shift frequencies without affecting the physical, real-time duration layout of the segment.
|
||||
|
||||
---
|
||||
|
||||
## 3. Graph-Based Fades
|
||||
|
||||
Fading curves overlay graphically directly onto the highlighted waveform canvas region, enabling precise boundary attenuation adjustments:
|
||||
|
||||
```text
|
||||
Linear Fade-In Exponential Fade-Out
|
||||
+───────────────────────────+ +───────────────────────────+
|
||||
| /███████████████| |███████████\ |
|
||||
| / ███████████████| |███████████ \ |
|
||||
| / ███████████████| |███████████ \___ |
|
||||
| / ███████████████| |███████████ \______|
|
||||
+───────────────────────────+ +───────────────────────────+
|
||||
|<──────── Fade-In ────────>| |<─────── Fade-Out ────────>|
|
||||
|
||||
```
|
||||
|
||||
* **Fade-In:** Multiplies an ascending amplitude ramp from $0.0$ to $1.0$ at the starting index profile of the selection region. Users can toggle between **Linear** or **Exponential** curves to achieve a smoother, more psychoacoustically natural volume build-up.
|
||||
* **Fade-Out:** Multiplies a descending amplitude decay ramp from $1.0$ down to $0.0$ at the trailing boundary edge of the selection range.
|
||||
|
||||
---
|
||||
|
||||
## 4. Ruler Percentage Stretch Tool
|
||||
|
||||
A dedicated percentage metric scale control (`Ruler %`) sitting on the control toolbar dictates time-stretching and playback velocity parameters:
|
||||
|
||||
```text
|
||||
[ Speed Stretch %: |========o========| 100% (Native) ] -> Range: 50% (Half Speed) - 200% (Double Speed)
|
||||
|
||||
```
|
||||
|
||||
* **Interaction Mapping:** Users drag the percentage slider node or hold down the `Alt` key and drag the rightmost boundary edge of the clip along the horizontal axis to change this scale metric.
|
||||
* **Sync Formula:** Let $D$ map to the unscaled native duration value, and $D'$ map to the target modified duration footprint. The resulting structural playback speed ratio percentage ($S$) is given by:
|
||||
|
||||
$$S = \frac{D}{D'} \times 100\%$$
|
||||
|
||||
|
||||
* **UI Representation:** A bright yellow text metadata indicator (e.g., `Speed: 85.3%`) is rendered at the top-left section of the audio clip bounding boundary.
|
||||
|
||||
---
|
||||
|
||||
## 5. Ultra-Zoom & Zero-Crossing Alignment
|
||||
|
||||
To facilitate precision structural slicing at sample-level resolutions, the sub-tab canvas allows microscopic viewport expansion:
|
||||
|
||||
```text
|
||||
MICRO VIEWPORT ZOOM (ULTRA ZOOM-IN)
|
||||
+─────────────────────────────────────────────────────────────────+
|
||||
| Waveform renders discrete contiguous sample nodes explicitly |
|
||||
| ○ (Sample i) |
|
||||
| / \ |
|
||||
| ─────────────────/───\─────────────────────────────► 0V Axis |
|
||||
| \ ○ (Sample i+2) |
|
||||
| \ / |
|
||||
| \_○ (Sample i+1 - Zero-Crossing Point)|
|
||||
+─────────────────────────────────────────────────────────────────+
|
||||
|
||||
```
|
||||
|
||||
* **Upper Viewport Scaling Limit:** Allows zooming in up to an extreme lower threshold of $2000\text{ pixels/second}$. At this zoom metric, layout compilation transitions away from downsampled peak profiles (Peak Waveform) to render actual discrete **sample nodes** interconnected by fine lines.
|
||||
* **Zero-Line Snapping Logic:** When establishing selection boundaries, the tracking loop automatically snaps the horizontal selection cursor coordinate to the nearest available sample address exhibiting an algebraic phase inversion (sign change):
|
||||
|
||||
$$x[i] \cdot x[i+1] \le 0$$
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 6. Top Duration Timeline
|
||||
|
||||
Directly above the isolated sub-tab waveform canvas lane, a dedicated horizontal measuring ruler tracks clip timing data:
|
||||
|
||||
```text
|
||||
| 0:00.000 | 0:01.000 | 0:02.000 | 0:03.000 | 0:04.000 (Duration: 4.152s)
|
||||
+───────────────────────────────────────────────────────────────────────────────────────+
|
||||
| [==================== VÙNG QUÉT CHỌN (RANGE SELECTION) ====================] |
|
||||
+───────────────────────────────────────────────────────────────────────────────────────+
|
||||
|
||||
```
|
||||
|
||||
* **Total Duration Monitoring:** Renders the absolute, precise time extent of the isolated audio block in the right-hand corner of the timeline ruler layout (e.g., `Duration: 12.450s`).
|
||||
* **Duration Selection Drag:** Left-clicking and dragging horizontally inside this top duration bar defines a highlighted selection overlay window. This range indicator automatically projects down into the waveform lane underneath.
|
||||
|
||||
---
|
||||
|
||||
## 7. Bottom Transport Panel
|
||||
|
||||
A prominent master transport toolbar occupies the bottom row layout of the sub-tab layout to manage audio playback monitoring:
|
||||
|
||||
```text
|
||||
+───────────────────────────────────────────────────────────────────────────+
|
||||
| [Back to Start] [Play] [Pause] [Stop] | Loop Sequence: [X] |
|
||||
+───────────────────────────────────────────────────────────────────────────+
|
||||
|
||||
```
|
||||
|
||||
* **Back to Start:** Instantly updates the regional playhead time parameter back to the absolute starting point ($t = 0.0\text{ s}$).
|
||||
* **Play / Pause / Stop:** Drives regional audio engine playback loops restricted entirely to the data buffers allocated inside the current sub-tab workspace.
|
||||
* **Loop Toggle:** Toggles continuous cycle loops over the highlighted section or the whole clip.
|
||||
|
||||
---
|
||||
|
||||
## 8. Volume Automation Envelope (Pen Tool)
|
||||
|
||||
This advanced timeline automation layer allows audio designers to draw custom gain curves over the background waveform graphics.
|
||||
|
||||
```text
|
||||
VOLUME AUTOMATION ENVELOPE (PEN TOOL)
|
||||
+3 dB ──────────────────────────────────────────────────────────────
|
||||
\ Node 1 Node 3
|
||||
\ ○ ○
|
||||
0 dB ───\────/─\─────────────────────────────────────/─\─────────── (0 dB Unity Gain Axis)
|
||||
\ / \ / \
|
||||
\/ \ / \
|
||||
○ \_______________________________/ \________
|
||||
Node 2 Node 4
|
||||
-30 dB ──────────────────────────────────────────────────────────────
|
||||
|<─────────────────── Horizontal Axis (Time) ─────────────────────>|
|
||||
|
||||
```
|
||||
|
||||
### 8.1. Pen Tool Interaction Mechanics
|
||||
|
||||
* **Activation:** Clicking the designated Pen Tool icon in the control panel modifies the pointer device presentation into a drawing crosshair or pencil graphic.
|
||||
* **Envelope Initialization:** Activating the Pen Tool generates a solid horizontal neon green line representing $0\text{ dB}$ (Unity Gain) across the track workspace, acting as the baseline master axis.
|
||||
* **Drawing Automation Curves:**
|
||||
* Left-clicking anywhere along this line creates an adjustable anchor point (**Control Node**).
|
||||
* Dragging an initialized control node upward increases signal amplitude (up to a maximal ceiling boundary of $+3\text{ dB}$).
|
||||
* Dragging a control node downward reduces signal amplitude (down to a lower attenuation floor of $-30\text{ dB}$).
|
||||
* The graphics framework automatically updates straight vector paths between sequential nodes utilizing simple linear interpolation.
|
||||
|
||||
|
||||
|
||||
### 8.2. DSP Volume Envelope Math
|
||||
|
||||
Given two chronologically adjacent drawn points $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the targeted instantaneous decibel gain variable $V_{\text{dB}}(t)$ at an arbitrary time index $t$ ($t_1 \le t \le t_2$) matches the following linear equation:
|
||||
|
||||
|
||||
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
|
||||
|
||||
This decibel value must be translated into a standard linear gain scalar coefficient $G_{\text{linear}}(t)$ to multiply it into the core audio sample stream values:
|
||||
|
||||
|
||||
$$G_{\text{linear}}(t) = 10^{\frac{V_{\text{dB}}(t)}{20}}$$
|
||||
|
||||
$$x_{\text{automation}}[n] = x[n] \cdot G_{\text{linear}}\left( \frac{n}{\text{Sample Rate}} \right)$$
|
||||
|
||||
---
|
||||
|
||||
## 9. Python Porting Manual (Docker Server Platform)
|
||||
|
||||
When translating these graphical volume automation envelope features to a desktop PyQt6 interface or an asynchronous Celery Docker worker pipeline, the standard scientific function `numpy.interp` handles array vector scaling processing loops:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
|
||||
def apply_volume_automation_envelope(y: np.ndarray, sr: int, nodes: list) -> np.ndarray:
|
||||
"""
|
||||
Applies a user-drawn volume automation envelope onto an acoustic signal NumPy array.
|
||||
nodes: A list of point dictionaries, e.g., [{"time": 0.0, "db": 0.0}, {"time": 2.5, "db": -12.0}, ...]
|
||||
"""
|
||||
if not nodes:
|
||||
return y
|
||||
|
||||
# Sort envelope nodes chronologically by time axis
|
||||
nodes = sorted(nodes, key=lambda x: x["time"])
|
||||
|
||||
# 1. Map node variables into distinct coordinates arrays
|
||||
node_times = np.array([node["time"] for node in nodes])
|
||||
node_dbs = np.array([node["db"] for node in nodes])
|
||||
|
||||
# Hard-clamp boundary constraints matching the operational floor [-30.0dB, +3.0dB]
|
||||
node_dbs = np.clip(node_dbs, -30.0, 3.0)
|
||||
|
||||
# 2. Evaluate absolute timeline timestamps for every index position inside the signal array
|
||||
total_samples = len(y)
|
||||
sample_times = np.arange(total_samples) / sr
|
||||
|
||||
# 3. Linearly interpolate localized decibel thresholds across every single sample step
|
||||
interpolated_dbs = np.interp(sample_times, node_times, node_dbs, left=node_dbs[0], right=node_dbs[-1])
|
||||
|
||||
# 4. Map logarithmic values into standard linear gain scale arrays
|
||||
linear_gains = 10.0 ** (interpolated_dbs / 20.0)
|
||||
|
||||
# 5. Multiply the raw amplitude vector array by the linear gain modifier mask
|
||||
return y * linear_gains
|
||||
|
||||
```
|
||||
@@ -0,0 +1,283 @@
|
||||
# Technical Specification: Advanced Editing Toolset & Graph-Based Continuous Waveform Painting on Sub-Tab
|
||||
|
||||
This document defines the interactive layout design, the configuration of the toolbar button arrays, and the signal processing routines for compiling a Graph-based Continuous Waveform graph optimized for the microscopic viewports inside the isolated temporary document workspace (Sub-tab), referencing the structural paradigms of `image_5ec2e5.png` and `image_5ec363.png`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Target Selection Scope
|
||||
|
||||
The toolset within the Sub-tab environment supports two target operational boundaries:
|
||||
|
||||
* **Global Clip:** When no specific timeline selection highlighted mask is present, all active DSP effects apply uniformly across the entire length of the extracted Audio Clip.
|
||||
* **Selected Range:** When an explicit timeline segment $[T_{\text{start}}, T_{\text{end}}]$ is highlighted by the user, DSP routines calculate changes exclusively inside those boundaries. Splice junctions automatically compute crossfades to mitigate transient click/pop anomalies.
|
||||
|
||||
---
|
||||
|
||||
## 2. Ruler-Based Tools
|
||||
|
||||
These utilities display as intuitive, linear slider scales (Sliders/Rulers) embedded in the top toolbar row:
|
||||
|
||||
```text
|
||||
[ Normalize: |======o======| 0 dB ] [ Gain: |====o====| +3 dB ] [ Pitch: |==o==| -2 Semi ]
|
||||
|
||||
```
|
||||
|
||||
### 2.1. Peak Normalization
|
||||
|
||||
* **UI Layout:** A slide scale control allowing users to configure target amplitude thresholds variable from $-12\text{ dBFS}$ down to $0\text{ dBFS}$.
|
||||
* **DSP Math Algorithm:** Locate the maximum absolute peak amplitude value $A_{\text{max}}$ within the targeted area, then multiply all active samples by a static scalar gain multiplier $G$:
|
||||
|
||||
$$G = \frac{10^{\frac{\text{Target\_dB}}{20}}}{A_{\text{max}}}$$
|
||||
|
||||
|
||||
|
||||
### 2.2. Volume Up / Down (Quick Gain)
|
||||
|
||||
* **UI Layout:** A linear sliding ruler modulating the overall absolute gain structure of the focused segment.
|
||||
* **Operational Range:** Adjustable from $-\infty\text{ dB}$ (complete mute attenuation) up to $+12\text{ dB}$ of linear amplification.
|
||||
|
||||
### 2.3. Pitch Shifting
|
||||
|
||||
* **UI Layout:** A calibrated slider modifying the project's fundamental frequencies discrete in semitones or cents.
|
||||
* **Operational Range:** Boundaries map from $-12\text{ semitones}$ (one octave down) to $+12\text{ semitones}$ (one octave up).
|
||||
* **DSP Engine Routine:** Employs a spectral Phase Vocoder to shift frequencies without affecting the physical, real-time duration layout of the segment.
|
||||
|
||||
---
|
||||
|
||||
## 3. Graph-Based Fades
|
||||
|
||||
Fading curves overlay graphically directly onto the highlighted waveform canvas region, enabling precise boundary amplitude adjustments:
|
||||
|
||||
```text
|
||||
Linear Fade-In Exponential Fade-Out
|
||||
+───────────────────────────+ +───────────────────────────+
|
||||
| /███████████████| |███████████\ |
|
||||
| / ███████████████| |███████████ \ |
|
||||
| / ███████████████| |███████████ \___ |
|
||||
| / ███████████████| |███████████ \______|
|
||||
+───────────────────────────+ +───────────────────────────+
|
||||
|<──────── Fade-In ────────>| |<─────── Fade-Out ────────>|
|
||||
|
||||
```
|
||||
|
||||
* **Fade-In:** Multiplies an ascending amplitude ramp from $0.0$ to $1.0$ at the starting index profile of the selection region. Users can toggle seamlessly between **Linear** or **Exponential** curves to achieve a smoother, more psychoacoustically natural volume build-up.
|
||||
* **Fade-Out:** Multiplies a descending amplitude decay ramp from $1.0$ down to $0.0$ at the trailing boundary edge of the selection range.
|
||||
|
||||
---
|
||||
|
||||
## 4. Ruler Percentage Stretch Tool
|
||||
|
||||
A dedicated percentage metric scale control (`Ruler %`) sitting on the control toolbar dictates time-stretching and playback velocity parameters:
|
||||
|
||||
```text
|
||||
[ Speed Stretch %: |========o========| 100% (Native) ] -> Range: 50% - 200%
|
||||
|
||||
```
|
||||
|
||||
* **Interaction Mapping:** Users drag the percentage slider node or hold down the `Alt` key and drag the rightmost boundary edge of the clip along the horizontal axis to change this scale metric.
|
||||
* **Sync Formula:** Let $D$ map to the unscaled native duration value, and $D'$ map to the target modified duration footprint. The resulting structural playback speed ratio percentage ($S$) is given by:
|
||||
|
||||
$$S = \frac{D}{D'} \times 100\%$$
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 5. Continuous Graph-Based Waveform Painting & Microscopic Viewports
|
||||
|
||||
The waveform graph inside the Sub-tab is compiled as a unified, continuous line vector (Continuous Line Graph) that flows seamlessly along the timeline axis, mapping the literal physical phase displacements of the underlying audio signal.
|
||||
|
||||
### 5.1. Logarithmic Amplitude Axis Grid Layout
|
||||
|
||||
Following the professional paradigm established in `image_5ec2e5.png`, the waveform painting canvas is divided by a symmetrical layout grid reflecting both positive and negative polarity limits of the central horizontal axis:
|
||||
|
||||
```text
|
||||
+6.0 dB ───────────────────────────────────────────────────────────────────
|
||||
~ ~ ~ ~ ~ ~ ~ ~ (Sub-division Grid Line) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
|
||||
-6.0 dB ───────────────────────────────────────────────────────────────────
|
||||
\ / \ / \
|
||||
-Inf dB ─○───────────/───────────────○───────────/───────────────○───────── (Zero-Line Axis)
|
||||
\ / \ / \
|
||||
-6.0 dB ───────────────────────────────────────────────────────────────────
|
||||
+6.0 dB ───────────────────────────────────────────────────────────────────
|
||||
|
||||
```
|
||||
|
||||
* **Visual Bounding Thresholds:**
|
||||
* **Central Zero Axis (-Inf. dB):** Maps the absolute baseline $0\text{V}$ electrical reference (complete absence of audio signal / absolute silence).
|
||||
* **Symmetrical Decibel Grids:** Project accurate scale metrics tracking normalized peak levels (the inner $-6.0\text{ dB}$ sub-grid marks a $50.1\%$ amplitude ceiling, while the outermost physical frame boundary aligns to $+6.0\text{ dB}$ or $0\text{ dBFS}$).
|
||||
|
||||
|
||||
|
||||
### 5.2. Standard Workspace View vs. Ultra Zoom Viewport Scaling
|
||||
|
||||
The drawing engine dynamically hot-swaps its rendering calculations (Rendering Routine) depending on the active pixel compression metric $Z$ (pixels/second):
|
||||
|
||||
* **Standard View Mode ($Z < 500\text{ pixels/second}$):** The system deploys a structural peak compression layout algorithm (**Peak Waveform**—as referenced in `image_5ec2e5.png`). It connects the maximum absolute upper peak bounding indices (Max) with the lower minimum value ranges (Min) passing through a common pixel column into a unified vector line, generating an organic, aliases-free continuous waveform silhouette.
|
||||
* **Micro Viewport Zoom-In ($Z \ge 500\text{ pixels/second}$—as referenced in `image_5ec363.png`):** Once viewport stretching scales past this threshold, the framework transitions into a **Single Continuous Sine Polyline** loop. Chronologically sequential acoustic sample addresses ($x[i]$, $x[i+1]$) map as discrete vector coordinate indices bound together by thin lines (using sharp smooth polyline vectors or linear/cubic spline interpolation loops), charting pristine, individual sinusoidal phases explicitly.
|
||||
|
||||
### 5.3. Zero-Crossing Alignment within Ultra Zoom Viewports
|
||||
|
||||
When performing rapid cursor tracking edits (Scrub/Drag Selection), the alignment routine locks the selection boundary marker coordinates onto the nearest baseline sample offset exhibiting a complete algebraic phase conversion (sign inversion):
|
||||
|
||||
|
||||
$$x[i] \cdot x[i+1] \le 0$$
|
||||
|
||||
---
|
||||
|
||||
## 6. Top Duration Timeline
|
||||
|
||||
Directly above the isolated sub-tab waveform canvas lane, a dedicated horizontal measuring ruler tracks clip timing data:
|
||||
|
||||
```text
|
||||
| 0:00.000 | 0:01.000 | 0:02.000 | 0:03.000 | 0:04.000 (Duration: 4.152s)
|
||||
+───────────────────────────────────────────────────────────────────────────────────────+
|
||||
| [==================== VÙNG QUÉT CHỌN (RANGE SELECTION) ====================] |
|
||||
+───────────────────────────────────────────────────────────────────────────────────────+
|
||||
|
||||
```
|
||||
|
||||
* **Total Duration Monitoring:** Renders the absolute, precise time extent of the isolated audio block in the right-hand corner of the timeline ruler layout (e.g., `Duration: 12.450s`).
|
||||
* **Duration Selection Drag:** Left-clicking and dragging horizontally inside this top duration bar defines a highlighted selection overlay window. This range indicator automatically projects down into the waveform lane underneath.
|
||||
|
||||
---
|
||||
|
||||
## 7. Bottom Transport Panel & Master Tools
|
||||
|
||||
A comprehensive control framework containing expanded navigation buttons and deep session processing controls anchors the bottom row of the sub-tab environment, matching the layout structure in `image_5ec2e5.png`:
|
||||
|
||||
```text
|
||||
+─────────────────────────────────────────────────────────────────────────────────────────────+
|
||||
| [● Rec] [◀◀ Back] [▶ Play] [|| Pause] [■ Stop] | Rate: |====o====| 0.00 | Loop: [X] |
|
||||
|---------------------------------------------------------------------------------------------|
|
||||
| [Volume Pencil Tool] [AI Analysis Tool] | Active Asset: linh_ngua_powerup.wav |
|
||||
+─────────────────────────────────────────────────────────────────────────────────────────────+
|
||||
|
||||
```
|
||||
|
||||
### 7.1. Functional Mapping Matrix:
|
||||
|
||||
* **Record (● Red Indicator):** Drives live microphone capture sequences targeted straight into the isolated sub-tab data matrix.
|
||||
* **Back (◀◀ Rewind):** Resets the timeline playhead position index back to the absolute starting point ($t = 0.0\text{ s}$).
|
||||
* **Play / Pause / Stop:** Coordinates low-latency runtime audio execution tracking locked onto the sub-tab's RAM cache blocks.
|
||||
* **Rate Slider:** Adjusts the global monitoring playback pitch speed metrics in real time without overwriting source asset length (calibrated step ranges variable from `-1.00` scaling up to `+1.00`).
|
||||
* **Loop Toggle:** Toggles continuous cycle loops over the highlighted section or the whole clip.
|
||||
* **Volume Pencil Tool:** Engages the drawing framework to map point nodes for automated amplitude envelopes.
|
||||
* **AI Analysis Tool:** Instructs the dockerized engine to evaluate rhythmic transient markers and pitch tracking grids.
|
||||
|
||||
---
|
||||
|
||||
## 8. Volume Automation Envelope (Pen Tool)
|
||||
|
||||
This advanced timeline automation layer allows audio designers to draw custom gain curves over the background waveform graphics.
|
||||
|
||||
```text
|
||||
VOLUME AUTOMATION ENVELOPE (PEN TOOL)
|
||||
+3 dB ──────────────────────────────────────────────────────────────
|
||||
\ Node 1 Node 3
|
||||
\ ○ ○
|
||||
0 dB ───\────/─\─────────────────────────────────────/─\─────────── (0 dB Unity Gain Axis)
|
||||
\ / \ / \
|
||||
\/ \ / \
|
||||
○ \_______________________________/ \________
|
||||
Node 2 Node 4
|
||||
-30 dB ──────────────────────────────────────────────────────────────
|
||||
|<─────────────────── Horizontal Axis (Time) ─────────────────────>|
|
||||
|
||||
```
|
||||
|
||||
### 8.1. Pen Tool Interaction Mechanics
|
||||
|
||||
* **Activation:** Clicking the designated Pencil Tool icon in the control panel modifies the pointer device presentation into a pencil graphic.
|
||||
* **Envelope Initialization:** Activating the Pen Tool generates a solid horizontal neon green line representing $0\text{ dB}$ (Unity Gain) across the track workspace, acting as the baseline master axis.
|
||||
* **Drawing Automation Curves:**
|
||||
* Left-clicking anywhere along this line creates an adjustable anchor point (**Control Node**).
|
||||
* Dragging an initialized control node upward increases signal amplitude (up to a maximal ceiling boundary of $+3\text{ dB}$).
|
||||
* Dragging a control node downward reduces signal amplitude (down to a lower attenuation floor of $-30\text{ dB}$).
|
||||
* The graphics framework automatically updates straight vector paths between sequential nodes utilizing simple linear interpolation.
|
||||
|
||||
|
||||
|
||||
### 8.2. DSP Volume Envelope Math
|
||||
|
||||
Given two chronologically adjacent drawn points $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the targeted instantaneous decibel gain variable $V_{\text{dB}}(t)$ at an arbitrary time index $t$ ($t_1 \le t \le t_2$) matches the following linear equation:
|
||||
|
||||
|
||||
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
|
||||
|
||||
This decibel value must be translated into a standard linear gain scalar coefficient $G_{\text{linear}}(t)$ to multiply it into the core audio sample stream values:
|
||||
|
||||
|
||||
$$G_{\text{linear}}(t) = 10^{\frac{\text{V}_{\text{dB}}(t)}{20}}$$
|
||||
|
||||
$$x_{\text{automation}}[n] = x[n] \cdot G_{\text{linear}}\left( \frac{n}{\text{Sample Rate}} \right)$$
|
||||
|
||||
---
|
||||
|
||||
## 9. Porting Guidelines for Python Desktop Layouts (PyQt6 QPainter Context)
|
||||
|
||||
When translating the polyline vector engine and the symmetrical decibel gridding lines into a containerized desktop application using the native `QPainter` canvas inside PyQt6, leveraging a structured `QPainterPath` prevents rendering lag when mapping high-density signal segments:
|
||||
|
||||
```python
|
||||
# [PYTHON PORTING BLUEPRINT] - Continuous Polyline Waveform Rendering via QPainterPath
|
||||
from PyQt6.QtGui import QPainter, QPainterPath, QPen, QColor
|
||||
from PyQt6.QtCore import QPointF, Qt
|
||||
import numpy as np
|
||||
|
||||
def paint_continuous_waveform_path(painter: QPainter, rect_width: int, rect_height: int, y: np.ndarray, zoom_level: float):
|
||||
"""
|
||||
Renders a unified continuous single polyline path tracing absolute physical signal transitions.
|
||||
y: A 1D NumPy float32 array tracking raw sample amplitudes bounded within [-1.0, 1.0].
|
||||
zoom_level: The scale allocation mapping physical drawing pixels per second of audio data.
|
||||
"""
|
||||
if len(y) == 0:
|
||||
return
|
||||
|
||||
painter.setRenderHint(QPainter.RenderHint.Antialiasing, True)
|
||||
mid_y = rect_height / 2.0
|
||||
|
||||
# 1. Compile background Decibel reference grids (-6.0 dB, -Inf. dB, -6.0 dB)
|
||||
grid_pen = QPen(QColor(45, 45, 45), 1, Qt.PenStyle.DashLine)
|
||||
painter.setPen(grid_pen)
|
||||
|
||||
# A threshold of -6.0 dB maps approximately to an absolute scalar amplitude index of 0.501
|
||||
y_6db_top = mid_y - (0.501 * (rect_height * 0.42))
|
||||
y_6db_bottom = mid_y + (0.501 * (rect_height * 0.42))
|
||||
|
||||
painter.drawLine(0, int(y_6db_top), rect_width, int(y_6db_top))
|
||||
painter.drawLine(0, int(y_6db_bottom), rect_width, int(y_6db_bottom))
|
||||
|
||||
# Paint the absolute Zero-Line horizontal center axis (-Inf. dB)
|
||||
center_pen = QPen(QColor(60, 60, 60), 1, Qt.PenStyle.SolidLine)
|
||||
painter.setPen(center_pen)
|
||||
painter.drawLine(0, int(mid_y), rect_width, int(mid_y))
|
||||
|
||||
# 2. Initialize the Continuous Vector Polyline Route Layout Block
|
||||
wave_path = QPainterPath()
|
||||
wave_pen = QPen(QColor(100, 149, 237), 1.2, Qt.PenStyle.SolidLine) # Professional Cornflower Blue
|
||||
painter.setPen(wave_pen)
|
||||
|
||||
# Map raw buffer indexes into structural coordinate pixels
|
||||
start_point_set = False
|
||||
for x_pixel in range(rect_width):
|
||||
# Translate current canvas pixel offset back to timeline seconds metrics
|
||||
time_at_pixel = x_pixel / zoom_level
|
||||
|
||||
# Calculate target array element offset
|
||||
sample_index = int(time_at_pixel * 44100) # Assuming project sample rate baseline at 44.1kHz
|
||||
if sample_index >= len(y):
|
||||
break
|
||||
|
||||
amplitude = y[sample_index]
|
||||
y_pixel = mid_y + (amplitude * (rect_height * 0.42))
|
||||
|
||||
if not start_point_set:
|
||||
wave_path.moveTo(float(x_pixel), y_pixel)
|
||||
start_point_set = True
|
||||
else:
|
||||
wave_path.lineTo(float(x_pixel), y_pixel)
|
||||
|
||||
# Draw the continuous vector polyline overlay onto the viewport canvas
|
||||
painter.drawPath(wave_path)
|
||||
|
||||
```
|
||||
@@ -0,0 +1,322 @@
|
||||
# Technical Specification: Implementing Volume, Fades & Panning Envelope Arrays on Audio Signals
|
||||
|
||||
This document defines the mathematical models, data flow diagrams (Audio Node Graph), and execution source code required to apply interactive graphical curves (Volume Automation, Fades, and Panning Automation) into the real-time digital signal processing pipeline on the Frontend and offline file export rendering on the Dockerized Python Backend.
|
||||
|
||||
---
|
||||
|
||||
## 1. Multi-stage Audio Node Graph
|
||||
|
||||
To simultaneously compute all three graphical configurations over the audio stream without precipitating phase cancellation or signal latency anomalies, the environment builds an explicit downstream node connection graph:
|
||||
|
||||
```text
|
||||
┌─────────────────────────┐
|
||||
│ AudioBufferSourceNode │ --> Streams the native original raw buffer array
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ GainNode (Automation) │ --> Modulates Volume dynamically via multi-point automation arrays
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ StereoPannerNode │ --> Transposes the Stereo Image (L/R Balance Automation trajectory)
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ GainNode (Fades) │ --> Multiplies bounding Fade-In and Fade-Out curves
|
||||
└────────────┬────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ AudioContext.destination│ --> Routes processed signal to hardware device outputs (Speakers/Headphones)
|
||||
└─────────────────────────┘
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Mathematical Formulations for Modulators
|
||||
|
||||
### 2.1. Multi-Point Volume Automation Curves
|
||||
|
||||
The vertical coordinate axis $Y$ of the volume points plots decibel thresholds bounded from $-30\text{ dB}$ to $+3\text{ dB}$. Prior to applying multipliers onto the signal, the logarithmic values must be translated into a standard linear scalar gain coefficient $G_{\text{linear}}$:
|
||||
|
||||
$$G_{\text{linear}}(t) = 10^{\frac{V_{\text{dB}}(t)}{20}}$$
|
||||
|
||||
At an arbitrary timeline timestamp $t$ residing between two chronologically adjacent control nodes $P_1(t_1, V_1)$ and $P_2(t_2, V_2)$, the target volume attenuation value is computed via standard linear interpolation:
|
||||
|
||||
$$V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}$$
|
||||
|
||||
### 2.2. Constant-Power Stereo Panning
|
||||
|
||||
To ensure that when a user shifts the audio image toward the Left ($L$) or Right ($R$) perimeter channels, the cumulative output sound energy emitted by the drivers does not collapse (avoiding a volume drop at the absolute horizontal center axis—known as the *Center Dip* anomaly), the system implements the **Constant-Power Panning Law**.
|
||||
|
||||
Let $p(t) \in [-1.0, 1.0]$ map to the explicit panning index at timestamp $t$ (where $-1.0$ represents a hard-left channel displacement, $0.0$ marks absolute center, and $+1.0$ dictates a hard-right channel boundary).
|
||||
|
||||
Convert the raw linear panning factor $p(t)$ into a circular panning sweep angle coordinate $\theta(t) \in [0, \pi/2]$:
|
||||
|
||||
$$\theta(t) = \frac{p(t) + 1}{2} \cdot \frac{\pi}{2}$$
|
||||
|
||||
Calculate the independent amplitude scalar gains for the Left channel ($g_L$) and the Right channel ($g_R$) elements:
|
||||
|
||||
$$g_L(t) = \cos(\theta(t)), \quad g_R(t) = \sin(\theta(t))$$
|
||||
|
||||
*Mathematical Proof:* The total sound field energy remains perfectly preserved under all operational transformations because:
|
||||
|
||||
$$g_L(t)^2 + g_R(t)^2 = \cos^2(\theta(t)) + \sin^2(\theta(t)) = 1.0$$
|
||||
|
||||
### 2.3. Fade Curves (Fade-In & Fade-Out)
|
||||
|
||||
Fading shapes are driven by a trigonometric Cosine equation framework to build organic, smooth amplitude transitions at the structural boundary zones of the audio asset:
|
||||
|
||||
* **Fade-In Curve** (Across an introductory duration window of $L_{\text{fade}}$ seconds):
|
||||
|
||||
$$f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade}}} \right)}{2} \quad \text{for } 0 \le t < L_{\text{fade}}$$
|
||||
|
||||
* **Fade-Out Curve** (Across a trailing termination window of $L_{\text{fade}}$ seconds):
|
||||
|
||||
$$f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade}})}{L_{\text{fade}}} \right)}{2} \quad \text{for } T_{\text{max}} - L_{\text{fade}} \le t \le T_{\text{max}}$$
|
||||
|
||||
---
|
||||
|
||||
## 3. Client-Side Runtime Integration (Web Audio API - Live Playback Modulator)
|
||||
|
||||
This JavaScript module sets up the physical Web Audio node graphs and automates parameters directly matching the real-time audio thread clocks:
|
||||
|
||||
```javascript
|
||||
/**
|
||||
* Configures a real-time audio node processing graph with parameter automation.
|
||||
* @param {AudioContext} audioCtx - The active Web Audio runtime context instance.
|
||||
* @param {AudioBuffer} audioBuffer - Decoded original raw target audio source asset.
|
||||
* @param {number} startTime - Global time position index marking where playback initiates (seconds).
|
||||
* @param {Array} volumeNodes - Automation point layout maps: [{time: 0.5, db: -3.0}, ...].
|
||||
* @param {Array} panningNodes - Panning position layout maps: [{time: 1.2, pan: -0.5}, ...].
|
||||
* @param {object} fadeConfig - Bounding fade time constants: {fadeInLen: 0.5, fadeOutLen: 0.8}.
|
||||
*/
|
||||
function playTrackWithAutomation(audioCtx, audioBuffer, startTime, volumeNodes, panningNodes, fadeConfig) {
|
||||
// 1. Instantiate the Global Audio Source Buffer Node
|
||||
const sourceNode = audioCtx.createBufferSource();
|
||||
sourceNode.buffer = audioBuffer;
|
||||
|
||||
// 2. Instantiate the Gain Node managing Volume Automation tracking loops
|
||||
const volumeGainNode = audioCtx.createGain();
|
||||
|
||||
// Establish baseline default state variables at Unity Gain (0 dB)
|
||||
volumeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
|
||||
|
||||
// Map timeline automations for custom Volume Node trajectories
|
||||
if (volumeNodes && volumeNodes.length > 0) {
|
||||
// Purge legacy scheduled values to safely overwrite parameters
|
||||
volumeGainNode.gain.cancelScheduledValues(audioCtx.currentTime);
|
||||
|
||||
volumeNodes.forEach(node => {
|
||||
const timeOffset = startTime + node.time;
|
||||
const linearGain = Math.pow(10, node.db / 20); // Map logarithmic dB thresholds to linear multipliers
|
||||
volumeGainNode.gain.linearRampToValueAtTime(linearGain, audioCtx.currentTime + node.time);
|
||||
});
|
||||
}
|
||||
|
||||
// 3. Instantiate the StereoPannerNode for Panning Automation structures
|
||||
const pannerNode = audioCtx.createStereoPanner();
|
||||
pannerNode.pan.setValueAtTime(0.0, audioCtx.currentTime); // Standard initialization locked at Center
|
||||
|
||||
// Map timeline automations for Panning Node trajectories
|
||||
if (panningNodes && panningNodes.length > 0) {
|
||||
pannerNode.pan.cancelScheduledValues(audioCtx.currentTime);
|
||||
|
||||
panningNodes.forEach(node => {
|
||||
// Enforce rigid clipping bounds to keep panning factors inside [-1.0, 1.0]
|
||||
const clampedPan = Math.max(-1.0, Math.min(1.0, node.pan));
|
||||
pannerNode.pan.linearRampToValueAtTime(clampedPan, audioCtx.currentTime + node.time);
|
||||
});
|
||||
}
|
||||
|
||||
// 4. Instantiate the Gain Node dedicated to boundary Fades
|
||||
const fadeGainNode = audioCtx.createGain();
|
||||
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
|
||||
|
||||
const duration = audioBuffer.duration;
|
||||
|
||||
// Calculate and schedule introductory Fade-In values
|
||||
if (fadeConfig.fadeInLen > 0) {
|
||||
fadeGainNode.gain.setValueAtTime(0.0, audioCtx.currentTime);
|
||||
fadeGainNode.gain.linearRampToValueAtTime(1.0, audioCtx.currentTime + fadeConfig.fadeInLen);
|
||||
}
|
||||
|
||||
// Calculate and schedule terminating Fade-Out values
|
||||
if (fadeConfig.fadeOutLen > 0) {
|
||||
const fadeOutStart = duration - fadeConfig.fadeOutLen;
|
||||
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime + fadeOutStart);
|
||||
fadeGainNode.gain.linearRampToValueAtTime(0.0, audioCtx.currentTime + duration);
|
||||
}
|
||||
|
||||
// 5. Connect the physical downstream structural audio pipeline
|
||||
sourceNode.connect(volumeGainNode);
|
||||
volumeGainNode.connect(pannerNode);
|
||||
pannerNode.connect(fadeGainNode);
|
||||
fadeGainNode.connect(audioCtx.destination);
|
||||
|
||||
// 6. Drive hardware execution loops
|
||||
sourceNode.start(0);
|
||||
return { sourceNode, volumeGainNode, pannerNode, fadeGainNode };
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Server-Side Execution Engine (Dockerized Python Engine - NumPy Processing)
|
||||
|
||||
When a user triggers an *Apply* action or an offline *Export* script, the frontend dispatches serialized JSON configuration models down to the Python backend framework. The signal processing architecture uses high-efficiency vectorized loops inside NumPy to multiply envelope modulators straight onto raw multi-channel float data:
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
|
||||
class DSPAudioModulator:
|
||||
@staticmethod
|
||||
def apply_automation_and_panning(
|
||||
y_raw: np.ndarray,
|
||||
sr: int,
|
||||
volume_points: list, # [{"time": 0.5, "db": -6.0}, ...]
|
||||
panning_points: list, # [{"time": 1.0, "pan": -0.7}, ...]
|
||||
fade_in_sec: float = 0.0,
|
||||
fade_out_sec: float = 0.0
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Applies multi-point volume envelopes, constant-power panning, and trigonometric fades
|
||||
directly onto a 1D (Mono) or 2D (Stereo) acoustic NumPy signal array.
|
||||
|
||||
Input: y_raw maps to the raw sound array (Mono/Stereo matrix bounded inside [-1.0, 1.0]).
|
||||
Output: y_processed yields a 2D interleaved Stereo NumPy array (2, N) with baked modulations.
|
||||
"""
|
||||
total_samples = y_raw.shape[-1] if len(y_raw.shape) > 1 else len(y_raw)
|
||||
duration_sec = total_samples / sr
|
||||
|
||||
# 1. Guarantee Stereo geometry dimensions (2 discrete channels) for Panning operations
|
||||
if len(y_raw.shape) == 1:
|
||||
# For Mono arrays, clone sample metrics symmetrically to Left/Right matrices
|
||||
y_stereo = np.vstack((y_raw, y_raw))
|
||||
else:
|
||||
y_stereo = np.copy(y_raw)
|
||||
|
||||
# 2. Allocate Envelope Mask arrays matching total track samples limits
|
||||
volume_envelope = np.ones(total_samples, dtype=np.float32)
|
||||
pan_envelope = np.zeros(total_samples, dtype=np.float32) # Default initialization: Center (0.0)
|
||||
|
||||
# 3. Compile the Volume Envelope using linear interpolation bounds across nodes
|
||||
if volume_points and len(volume_points) > 0:
|
||||
# Enforce strict chronological sorting down the timeline axis
|
||||
points = sorted(volume_points, key=lambda x: x["time"])
|
||||
|
||||
# Pad introductory bounds if the initial point coordinate sits past t = 0.0s
|
||||
if points[0]["time"] > 0:
|
||||
first_gain = 10.0 ** (points[0]["db"] / 20.0)
|
||||
idx_end = int(points[0]["time"] * sr)
|
||||
volume_envelope[:idx_end] = first_gain
|
||||
|
||||
for i in range(len(points) - 1):
|
||||
p1, p2 = points[i], points[i+1]
|
||||
idx_start = int(p1["time"] * sr)
|
||||
idx_end = int(p2["time"] * sr)
|
||||
|
||||
gain_start = 10.0 ** (p1["db"] / 20.0)
|
||||
gain_end = 10.0 ** (p2["db"] / 20.0)
|
||||
|
||||
# Linearly interpolate vector increments between adjacent anchor positions
|
||||
volume_envelope[idx_start:idx_end] = np.linspace(gain_start, gain_end, idx_end - idx_start)
|
||||
|
||||
# Pad trailing bounds from the final milestone extending through end-of-file
|
||||
if points[-1]["time"] < duration_sec:
|
||||
last_gain = 10.0 ** (points[-1]["db"] / 20.0)
|
||||
idx_start = int(points[-1]["time"] * sr)
|
||||
volume_envelope[idx_start:] = last_gain
|
||||
|
||||
# 4. Compile the Panning Envelope using linear interpolation bounds across nodes
|
||||
if panning_points and len(panning_points) > 0:
|
||||
points = sorted(panning_points, key=lambda x: x["time"])
|
||||
|
||||
if points[0]["time"] > 0:
|
||||
pan_envelope[:int(points[0]["time"] * sr)] = points[0]["pan"]
|
||||
|
||||
for i in range(len(points) - 1):
|
||||
p1, p2 = points[i], points[i+1]
|
||||
idx_start = int(p1["time"] * sr)
|
||||
idx_end = int(p2["time"] * sr)
|
||||
pan_envelope[idx_start:idx_end] = np.linspace(p1["pan"], p2["pan"], idx_end - idx_start)
|
||||
|
||||
if points[-1]["time"] < duration_sec:
|
||||
pan_envelope[int(points[-1]["time"] * sr):] = points[-1]["pan"]
|
||||
|
||||
# 5. Apply Trigonometric Cosine Fade-In / Fade-Out functions onto the Volume Envelope mask
|
||||
if fade_in_sec > 0:
|
||||
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
|
||||
x_fade = np.linspace(0.0, np.pi, fade_in_samples)
|
||||
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
|
||||
volume_envelope[:fade_in_samples] *= cosine_ramp
|
||||
|
||||
if fade_out_sec > 0:
|
||||
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
|
||||
x_fade = np.linspace(0.0, np.pi, fade_out_samples)
|
||||
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
|
||||
volume_envelope[-fade_out_samples:] *= cosine_ramp
|
||||
|
||||
# 6. Bake Volume Envelope matrices onto the Left and Right discrete audio paths
|
||||
y_stereo[0, :] *= volume_envelope
|
||||
y_stereo[1, :] *= volume_envelope
|
||||
|
||||
# 7. Apply Constant-Power Stereo Panning allocations
|
||||
# Map panning metrics range [-1.0, 1.0] onto angular radians field array [0, pi/2]
|
||||
theta_envelope = ((pan_envelope + 1.0) / 2.0) * (np.pi / 2.0)
|
||||
|
||||
# Evaluate localized amplitude coefficients for physical channels split
|
||||
gain_left = np.cos(theta_envelope)
|
||||
gain_right = np.sin(theta_envelope)
|
||||
|
||||
# Multiply scaling factors directly across corresponding discrete matrices
|
||||
y_stereo[0, :] *= gain_left
|
||||
y_stereo[1, :] *= gain_right
|
||||
|
||||
return y_stereo
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Viewport Coordinate Mapping & Data Serialization Protocols
|
||||
|
||||
As users drag and adjust coordinate anchors over the visual drawing Canvas, mouse-event pixel coordinates are continuously calculated and mapped into absolute physical values to preserve data matching between frontend layouts and backend signal arrays:
|
||||
|
||||
```text
|
||||
[ GRAPH CANVAS VIEWPORT COORDINATES ] [ REAL-WORLD SYSTEM PHENOMENA VALUES ]
|
||||
x (pixel) ─────────────────────────────────────► Timeline position t (seconds) = x / zoom_level
|
||||
y (pixel) ─ (Volume: Center axis maps to 0dB) ─► db = ( (h - y) / h_half ) * range_db
|
||||
y (pixel) ─ (Panning: Center axis maps to 0) ──► pan = ( (h_half - y) / h_half ) -> Bounded [-1.0, 1.0]
|
||||
|
||||
```
|
||||
|
||||
### Serialized API Data Transfer Model (Standard JSON Package Syntax)
|
||||
|
||||
```json
|
||||
{
|
||||
"track_id": "1",
|
||||
"fades": {
|
||||
"fade_in_sec": 0.500,
|
||||
"fade_out_sec": 1.200
|
||||
},
|
||||
"volume_automation": [
|
||||
{ "time": 0.000, "db": 0.0 },
|
||||
{ "time": 1.450, "db": 3.0 },
|
||||
{ "time": 3.820, "db": -12.5 },
|
||||
{ "time": 6.000, "db": 0.0 }
|
||||
],
|
||||
"panning_automation": [
|
||||
{ "time": 0.000, "pan": 0.0 },
|
||||
{ "time": 2.100, "pan": -0.8 },
|
||||
{ "time": 4.500, "pan": 0.8 },
|
||||
{ "time": 6.000, "pan": 0.0 }
|
||||
]
|
||||
}
|
||||
|
||||
```
|
||||
@@ -0,0 +1,138 @@
|
||||
import numpy as np
|
||||
import scipy.signal as signal
|
||||
import librosa
|
||||
|
||||
class SubTabDSPEngine:
|
||||
@staticmethod
|
||||
def change_speed(y: np.ndarray, sr: int, speed_ratio: float, preserve_pitch: bool = True) -> np.ndarray:
|
||||
"""
|
||||
Alters the playback velocity (Time-Stretching) of a NumPy signal array.
|
||||
"""
|
||||
if speed_ratio == 1.0:
|
||||
return y
|
||||
|
||||
if preserve_pitch:
|
||||
return librosa.effects.time_stretch(y, rate=speed_ratio)
|
||||
else:
|
||||
num_samples_new = int(len(y) / speed_ratio)
|
||||
return signal.resample(y, num_samples_new)
|
||||
|
||||
@staticmethod
|
||||
def normalize(y: np.ndarray, target_db: float = 0.0) -> np.ndarray:
|
||||
"""
|
||||
Performs Peak Normalization on an array to scale it to the target decibel value.
|
||||
"""
|
||||
target_amplitude = 10.0 ** (target_db / 20.0)
|
||||
max_amplitude = np.max(np.abs(y))
|
||||
|
||||
if max_amplitude == 0:
|
||||
return y
|
||||
|
||||
gain = target_amplitude / max_amplitude
|
||||
return y * gain
|
||||
|
||||
@staticmethod
|
||||
def apply_volume_automation_envelope(y: np.ndarray, sr: int, nodes: list) -> np.ndarray:
|
||||
"""
|
||||
Applies a user-drawn volume automation envelope onto an acoustic signal NumPy array.
|
||||
nodes: A list of point dictionaries, e.g., [{"time": 0.0, "db": 0.0}, {"time": 2.5, "db": -12.0}, ...]
|
||||
"""
|
||||
if not nodes:
|
||||
return y
|
||||
|
||||
# Sort envelope nodes chronologically by time axis
|
||||
nodes = sorted(nodes, key=lambda x: x["time"])
|
||||
|
||||
# 1. Map node variables into distinct coordinates arrays
|
||||
node_times = np.array([node["time"] for node in nodes])
|
||||
node_dbs = np.array([node["db"] for node in nodes])
|
||||
|
||||
# Hard-clamp boundary constraints matching the operational floor [-30.0dB, +3.0dB]
|
||||
node_dbs = np.clip(node_dbs, -30.0, 3.0)
|
||||
|
||||
# 2. Evaluate absolute timeline timestamps for every index position inside the signal array
|
||||
total_samples = len(y)
|
||||
sample_times = np.arange(total_samples) / sr
|
||||
|
||||
# 3. Linearly interpolate localized decibel thresholds across every single sample step
|
||||
# Handle edge cases for interpolation: if sample_times is outside node_times range,
|
||||
# np.interp uses the first/last value of node_dbs.
|
||||
interpolated_dbs = np.interp(sample_times, node_times, node_dbs, left=node_dbs[0], right=node_dbs[-1])
|
||||
|
||||
# 4. Map logarithmic values into standard linear gain scale arrays
|
||||
linear_gains = 10.0 ** (interpolated_dbs / 20.0)
|
||||
|
||||
# 5. Multiply the raw amplitude vector array by the linear gain modifier mask
|
||||
return y * linear_gains
|
||||
|
||||
@staticmethod
|
||||
def pitch_shift(y: np.ndarray, sr: int, n_steps: float) -> np.ndarray:
|
||||
"""
|
||||
Shift the pitch of an audio signal by a specified number of semitones.
|
||||
|
||||
Args:
|
||||
y: Input audio signal
|
||||
sr: Sample rate
|
||||
n_steps: Number of semitones to shift (positive = higher pitch, negative = lower pitch)
|
||||
|
||||
Returns:
|
||||
Pitch-shifted audio signal
|
||||
"""
|
||||
if n_steps == 0:
|
||||
return y
|
||||
return librosa.effects.pitch_shift(y, sr=sr, n_steps=n_steps)
|
||||
|
||||
@staticmethod
|
||||
def merge_back_to_parent(
|
||||
parent_track_audio: np.ndarray,
|
||||
sr: int,
|
||||
edited_sub_audio: np.ndarray,
|
||||
start_seconds: float,
|
||||
original_duration_seconds: float
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Splices the modified audio segment from the Sub-tab back into the parent track array.
|
||||
Applies a 10ms micro-crossfade at the boundaries to eliminate pop/click noise.
|
||||
"""
|
||||
start_sample = int(start_seconds * sr)
|
||||
original_samples_len = int(original_duration_seconds * sr)
|
||||
edited_samples_len = len(edited_sub_audio)
|
||||
crossfade_samples = int(0.01 * sr) # 10ms crossfade window
|
||||
|
||||
# 1. Allocate the target output array dimension bounds
|
||||
new_total_len = len(parent_track_audio) - original_samples_len + edited_samples_len
|
||||
output_audio = np.zeros(new_total_len, dtype=np.float32)
|
||||
|
||||
# 2. Extract leading unedited block
|
||||
output_audio[:start_sample] = parent_track_audio[:start_sample]
|
||||
|
||||
# 3. Stitch the modified audio payload
|
||||
output_audio[start_sample:start_sample + edited_samples_len] = edited_sub_audio
|
||||
|
||||
# 4. Extract trailing unedited block
|
||||
post_start_original = start_sample + original_samples_len
|
||||
post_start_new = start_sample + edited_samples_len
|
||||
output_audio[post_start_new:] = parent_track_audio[post_start_original:]
|
||||
|
||||
# 5. Execute micro-crossfade across the initial splice junction
|
||||
if start_sample > crossfade_samples:
|
||||
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
|
||||
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
|
||||
|
||||
# Smooth 10ms interpolation overlay
|
||||
output_audio[start_sample : start_sample + crossfade_samples] = (
|
||||
edited_sub_audio[:crossfade_samples] * fade_in_ramp +
|
||||
parent_track_audio[start_sample : start_sample + crossfade_samples] * fade_out_ramp
|
||||
)
|
||||
|
||||
# 6. Execute micro-crossfade across the trailing splice junction
|
||||
if post_start_new + crossfade_samples < len(output_audio):
|
||||
fade_in_ramp = np.linspace(0.0, 1.0, crossfade_samples)
|
||||
fade_out_ramp = np.linspace(1.0, 0.0, crossfade_samples)
|
||||
|
||||
output_audio[post_start_new : post_start_new + crossfade_samples] = (
|
||||
parent_track_audio[post_start_original : post_start_original + crossfade_samples] * fade_in_ramp +
|
||||
edited_sub_audio[-crossfade_samples:] * fade_out_ramp
|
||||
)
|
||||
|
||||
return output_audio
|
||||
+2366
-612
File diff suppressed because it is too large
Load Diff
-1589
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,80 @@
|
||||
import os
|
||||
import sys
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from app.core.sub_tab_dsp import SubTabDSPEngine
|
||||
|
||||
def test_speed_ratio():
|
||||
sr = 44100
|
||||
y = np.sin(2 * np.pi * 440 * np.linspace(0, 1, sr, endpoint=False)).astype(np.float32)
|
||||
# Pitch-preserving speed change
|
||||
y_stretched = SubTabDSPEngine.change_speed(y, sr, 2.0, preserve_pitch=True)
|
||||
assert abs(len(y_stretched) - sr // 2) < 2000 # Librosa might have frame alignment differences
|
||||
|
||||
# Simple resampling speed change
|
||||
y_resampled = SubTabDSPEngine.change_speed(y, sr, 2.0, preserve_pitch=False)
|
||||
assert len(y_resampled) == sr // 2
|
||||
|
||||
def test_normalize():
|
||||
y = np.array([0.1, -0.5, 0.2, 0.4], dtype=np.float32)
|
||||
y_norm = SubTabDSPEngine.normalize(y, target_db=0.0)
|
||||
assert np.max(np.abs(y_norm)) == 1.0
|
||||
|
||||
def test_merge_back_to_parent():
|
||||
sr = 1000
|
||||
parent = np.ones(5000, dtype=np.float32)
|
||||
edited = np.zeros(2000, dtype=np.float32)
|
||||
|
||||
# Merge at t=1.0s (index 1000), original duration 1.5s (1500 samples)
|
||||
res = SubTabDSPEngine.merge_back_to_parent(
|
||||
parent_track_audio=parent,
|
||||
sr=sr,
|
||||
edited_sub_audio=edited,
|
||||
start_seconds=1.0,
|
||||
original_duration_seconds=1.5
|
||||
)
|
||||
|
||||
# Expected length: 5000 - 1500 + 2000 = 5500
|
||||
assert len(res) == 5500
|
||||
# Before 1.0s (1000 samples) should be mostly parent values (1.0)
|
||||
assert np.allclose(res[:900], 1.0)
|
||||
# Inside the edited range should be zero (except crossfades)
|
||||
assert np.allclose(res[1100:2900], 0.0)
|
||||
|
||||
def test_apply_volume_automation_envelope():
|
||||
sr = 44100
|
||||
duration = 2.0
|
||||
y = np.ones(int(sr * duration), dtype=np.float32) * 0.5 # Constant signal at -6dB
|
||||
|
||||
# Simple fade in from -inf to 0dB over 1 second
|
||||
nodes = [
|
||||
{"time": 0.0, "db": -60.0}, # Effectively -inf
|
||||
{"time": 1.0, "db": 0.0},
|
||||
{"time": 2.0, "db": 0.0},
|
||||
]
|
||||
|
||||
y_automated = SubTabDSPEngine.apply_volume_automation_envelope(y, sr, nodes)
|
||||
|
||||
y_automated = SubTabDSPEngine.apply_volume_automation_envelope(y, sr, nodes)
|
||||
|
||||
# Check start: should be 0.5 * 10**(-30/20) due to clipping
|
||||
expected_start_val = 0.5 * (10**(-30/20.0))
|
||||
assert np.isclose(y_automated[0], expected_start_val, atol=1e-5)
|
||||
|
||||
# Check at 0.5 seconds: interpolated to -15dB (halfway between -30dB and 0dB)
|
||||
# y * (10 ** (-15 / 20))
|
||||
expected_mid_val = 0.5 * (10**(-15/20.0))
|
||||
assert np.isclose(y_automated[int(0.5 * sr)], expected_mid_val, atol=1e-5)
|
||||
|
||||
# Check at 1.0 seconds: should be 0.5 * (10**(0/20)) = 0.5
|
||||
assert np.isclose(y_automated[int(1.0 * sr)], 0.5, atol=1e-5)
|
||||
|
||||
# Check at end: should be 0.5
|
||||
assert np.isclose(y_automated[-1], 0.5, atol=1e-5)
|
||||
|
||||
# Test with empty nodes
|
||||
y_no_nodes = SubTabDSPEngine.apply_volume_automation_envelope(y, sr, [])
|
||||
assert np.array_equal(y_no_nodes, y)
|
||||
|
||||
Reference in New Issue
Block a user