Compare commits
4 Commits
32fab9d366
...
3c77e98956
| Author | SHA1 | Date | |
|---|---|---|---|
| 3c77e98956 | |||
| 7d9267d175 | |||
| 283555c78e | |||
| 4ca90f15ca |
+167
@@ -0,0 +1,167 @@
|
|||||||
|
# Technical Specification: Mapping Matrix & Interactive Graphical Rendering Algorithms
|
||||||
|
|
||||||
|
This document defines the interactive real-time non-linear curves system showcased.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. UI Element Mapping Matrix
|
||||||
|
|
||||||
|
To upgrade the interface workflow from Image 1 to Image 2, a 1-to-1 mapping of graphical components is executed based on the structural breakdown below:
|
||||||
|
|
||||||
|
|
||||||
|
| --- | --- | --- |
|
||||||
|
| **Horizontal blue bar at the top** (Contains a node chain representing the default $0\text{ dB}$ volume level) | Multi-point peach-colored automation spine (**Automation Spline**) overlaying the waveform viewport area. | * **Double-click** anywhere along the spline to generate a new control node.<br>
|
||||||
|
|
||||||
|
<br>
|
||||||
|
|
||||||
|
<br>* **Click & Drag** a node vertically to scale Volume (Gain), or horizontally to adjust its chronological time position. |
|
||||||
|
| **"FI" text label** in the upper-left corner | Deep red arched **Fade-In Bezier Curve** smoothing the volume transition from $0\%$ up to $100\%$. | * **Click & Hold** the "FI" handle and drag rightward to increase the target Fade-In length ($L_{\text{fade\_in}}$). This action automatically projects a smooth curve overlay on top of the waveform graphic. |
|
||||||
|
| **"FO" text label** in the upper-right corner | Deep red arched **Fade-Out Bezier Curve** decaying the volume envelope from $100\%$ down to $0\%$ at the end of the clip boundary. | * **Click & Hold** the "FO" handle and drag leftward to increase the target Fade-Out length ($L_{\text{fade\_out}}$). The inverse curve automatically stretches or compresses based on the active dragging cursor coordinates. |
|
||||||
|
| **"VOL" button** in the lower-right corner | **Graphical Envelope Mode Switcher** (Toggles automation layer matrices). | * **Click** to hot-swap between multiple interactive graphs: Volume (VOL) (peach curve), Panning (PAN) (L/R Stereo Image automation trajectory), or FX Send grids. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Non-Linear Graphical Curve Rendering Algorithms (Image 2)
|
||||||
|
|
||||||
|
### 2.1. Multi-Point Volume Automation Curves (Smooth Monotone Spline)
|
||||||
|
|
||||||
|
To ensure the interpolating paths connecting the peach-colored nodes in Image 2 are curved smoothly without generating sharp angular peaks, the framework runs a **Monotone Cubic Hermite Spline** interpolation algorithm.
|
||||||
|
|
||||||
|
Given two chronologically consecutive control nodes $P_a(x_a, y_a)$ and $P_b(x_b, y_b)$, an arbitrary absolute timeline position $x$ is normalized into a relative horizontal index interval $t$:
|
||||||
|
|
||||||
|
|
||||||
|
$$t = \frac{x - x_a}{x_b - x_a} \quad (0 \le t \le 1)$$
|
||||||
|
|
||||||
|
The target interpolated amplitude value $y(x)$ at position $x$ is evaluated using the cubic polynomial equation:
|
||||||
|
|
||||||
|
|
||||||
|
$$y(x) = (2t^3 - 3t^2 + 1)y_a + (t^3 - 2t^2 + t)h \cdot m_a + (-2t^3 + 3t^2)y_b + (t^3 - t^2)h \cdot m_b$$
|
||||||
|
|
||||||
|
Where: $h = x_b - x_a$, and $m_a, m_b$ correspond to the localized slopes (tangents) computed from adjacent surrounding node coordinates. This constraint ensures strict monotonicity to eliminate graphical or mathematical overshoot anomalies.
|
||||||
|
|
||||||
|
### 2.2. Fade Curve Contours (Fade-In & Fade-Out)
|
||||||
|
|
||||||
|
The physical curvature profile of the two deep red envelopes in Image 2 is evaluated using a trigonometric Cosine S-Curve or a 3rd-order Cubic Bezier equation framework:
|
||||||
|
|
||||||
|
* **Trigonometric Cosine Fade-In Curve** (Across a duration bound of $L_{\text{fade\_in}}$ seconds):
|
||||||
|
|
||||||
|
$$f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade\_in}}} \right)}{2} \quad \left( 0 \le t \le L_{\text{fade\_in}} \right)$$
|
||||||
|
|
||||||
|
|
||||||
|
* **Trigonometric Cosine Fade-Out Curve** (Across a trailing termination window of $L_{\text{fade\_out}}$ seconds):
|
||||||
|
|
||||||
|
$$f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade\_out}})}{L_{\text{fade\_out}}} \right)}{2} \quad \left( T_{\text{max}} - L_{\text{fade\_out}} \le t \le T_{\text{max}} \right)$$
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Client-Side Runtime Integration (HTML5 Canvas Engine)
|
||||||
|
|
||||||
|
To make the static canvas layer from Image 1 respond fluidly to drag gestures like the interactive system in Image 2, the painting routine segregates graphic elements into distinct presentation layers, driven inside a low-latency `requestAnimationFrame` render loop:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
/**
|
||||||
|
* Renders non-linear Fade-In and Fade-Out curves over the Waveform canvas viewport.
|
||||||
|
* @param {CanvasRenderingContext2D} ctx - Target 2D rendering canvas context.
|
||||||
|
* @param {number} width - Total physical viewport tracking pixel width.
|
||||||
|
* @param {number} height - Total physical viewport tracking pixel height.
|
||||||
|
* @param {number} fadeInSec - Bounding target Fade-In duration in seconds.
|
||||||
|
* @param {number} fadeOutSec - Bounding target Fade-Out duration in seconds.
|
||||||
|
* @param {number} zoom - Current layout pixel compression scaling factor (pixels/second).
|
||||||
|
*/
|
||||||
|
function drawFadeCurves(ctx, width, height, fadeInSec, fadeOutSec, zoom) {
|
||||||
|
const fadeInWidth = fadeInSec * zoom;
|
||||||
|
const fadeOutWidth = fadeOutSec * zoom;
|
||||||
|
const midY = height / 2;
|
||||||
|
|
||||||
|
ctx.strokeStyle = '#800000'; // Professional dark deep red hue theme
|
||||||
|
ctx.lineWidth = 1.8;
|
||||||
|
|
||||||
|
// 1. Compile the non-linear Fade-In curve polyline
|
||||||
|
if (fadeInWidth > 0) {
|
||||||
|
ctx.beginPath();
|
||||||
|
for (let x = 0; x <= fadeInWidth; x++) {
|
||||||
|
const ratio = x / fadeInWidth;
|
||||||
|
// Apply trigonometric cosine to map curved vertical y-coordinates
|
||||||
|
const amp = (1 - Math.cos(Math.PI * ratio)) / 2;
|
||||||
|
const y = height - (amp * height); // Apply envelope tracking from bottom to top
|
||||||
|
if (x === 0) ctx.moveTo(x, height);
|
||||||
|
else ctx.lineTo(x, y);
|
||||||
|
}
|
||||||
|
ctx.stroke();
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Compile the non-linear Fade-Out curve polyline
|
||||||
|
if (fadeOutWidth > 0) {
|
||||||
|
ctx.beginPath();
|
||||||
|
const startX = width - fadeOutWidth;
|
||||||
|
for (let x = 0; x <= fadeOutWidth; x++) {
|
||||||
|
const ratio = x / fadeOutWidth;
|
||||||
|
const amp = (1 + Math.cos(Math.PI * ratio)) / 2;
|
||||||
|
const y = height - (amp * height);
|
||||||
|
if (x === 0) ctx.moveTo(startX + x, 0);
|
||||||
|
else ctx.lineTo(startX + x, y);
|
||||||
|
}
|
||||||
|
ctx.stroke();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Server-Side DSP Automation Processing (Dockerized Python Engine)
|
||||||
|
|
||||||
|
When an operator commits tracking edits via the Frontend client layer, the mapped coordinates are encoded as a serialized JSON package and transferred down to the FastAPI server gateway. The Python core layer runs performance-optimized, vectorized array loops inside NumPy to multiply envelope filters straight into the raw source data buffer matrices:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
class DSPAutomationProcessor:
|
||||||
|
@staticmethod
|
||||||
|
def apply_curves_to_samples(
|
||||||
|
y: np.ndarray,
|
||||||
|
sr: int,
|
||||||
|
fade_in_sec: float,
|
||||||
|
fade_out_sec: float,
|
||||||
|
automation_points: list # [{"time": 0.5, "db": -3.0}, ...]
|
||||||
|
) -> np.ndarray:
|
||||||
|
"""
|
||||||
|
Bakes multi-point Volume Automation splines and non-linear fade curves
|
||||||
|
directly onto a raw acoustic sample NumPy array.
|
||||||
|
"""
|
||||||
|
total_samples = len(y)
|
||||||
|
duration_sec = total_samples / sr
|
||||||
|
|
||||||
|
# 1. Initialize the baseline Gain Envelope at Unity Gain (1.0 or 0 dB)
|
||||||
|
gain_envelope = np.ones(total_samples, dtype=np.float32)
|
||||||
|
|
||||||
|
# 2. Evaluate Volume Automation scaling paths (Peach-colored nodes in Image 2)
|
||||||
|
if automation_points and len(automation_points) > 0:
|
||||||
|
points = sorted(automation_points, key=lambda x: x["time"])
|
||||||
|
xp = [p["time"] for p in points]
|
||||||
|
fp = [10.0 ** (p["db"] / 20.0) for p in points] # Map decibel factors to linear scalars
|
||||||
|
|
||||||
|
# Linearly interpolate point values quickly across the full timeline width
|
||||||
|
times = np.linspace(0, duration_sec, total_samples)
|
||||||
|
gain_envelope = np.interp(times, xp, fp)
|
||||||
|
|
||||||
|
# 3. Multiply the introductory Fade-In envelope (Cosine transition mask at Image 2 boundary)
|
||||||
|
if fade_in_sec > 0:
|
||||||
|
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
|
||||||
|
x_fade = np.linspace(0, np.pi, fade_in_samples)
|
||||||
|
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
|
||||||
|
gain_envelope[:fade_in_samples] *= cosine_ramp
|
||||||
|
|
||||||
|
# 4. Multiply the trailing Fade-Out envelope (Cosine decay mask at Image 2 boundary)
|
||||||
|
if fade_out_sec > 0:
|
||||||
|
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
|
||||||
|
x_fade = np.linspace(0, np.pi, fade_out_samples)
|
||||||
|
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
|
||||||
|
gain_envelope[-fade_out_samples:] *= cosine_ramp
|
||||||
|
|
||||||
|
# 5. Execute vectorized element-wise multiplication into raw audio values
|
||||||
|
return y * gain_envelope
|
||||||
|
|
||||||
|
```
|
||||||
@@ -136,3 +136,111 @@ class SubTabDSPEngine:
|
|||||||
)
|
)
|
||||||
|
|
||||||
return output_audio
|
return output_audio
|
||||||
|
|
||||||
|
|
||||||
|
class DSPAudioModulator:
|
||||||
|
@staticmethod
|
||||||
|
def apply_automation_and_panning(
|
||||||
|
y_raw: np.ndarray,
|
||||||
|
sr: int,
|
||||||
|
volume_points: list, # [{"time": 0.5, "db": -6.0}, ...]
|
||||||
|
panning_points: list, # [{"time": 1.0, "pan": -0.7}, ...]
|
||||||
|
fade_in_sec: float = 0.0,
|
||||||
|
fade_out_sec: float = 0.0
|
||||||
|
) -> np.ndarray:
|
||||||
|
"""
|
||||||
|
Applies multi-point volume envelopes, constant-power panning, and trigonometric fades
|
||||||
|
directly onto a 1D (Mono) or 2D (Stereo) acoustic NumPy signal array.
|
||||||
|
|
||||||
|
Input: y_raw maps to the raw sound array (Mono/Stereo matrix bounded inside [-1.0, 1.0]).
|
||||||
|
Output: y_processed yields a 2D interleaved Stereo NumPy array (2, N) with baked modulations.
|
||||||
|
"""
|
||||||
|
total_samples = y_raw.shape[-1] if len(y_raw.shape) > 1 else len(y_raw)
|
||||||
|
duration_sec = total_samples / sr
|
||||||
|
|
||||||
|
# 1. Guarantee Stereo geometry dimensions (2 discrete channels) for Panning operations
|
||||||
|
if len(y_raw.shape) == 1:
|
||||||
|
# For Mono arrays, clone sample metrics symmetrically to Left/Right matrices
|
||||||
|
y_stereo = np.vstack((y_raw, y_raw))
|
||||||
|
else:
|
||||||
|
y_stereo = np.copy(y_raw)
|
||||||
|
|
||||||
|
# 2. Allocate Envelope Mask arrays matching total track samples limits
|
||||||
|
volume_envelope = np.ones(total_samples, dtype=np.float32)
|
||||||
|
pan_envelope = np.zeros(total_samples, dtype=np.float32) # Default initialization: Center (0.0)
|
||||||
|
|
||||||
|
# 3. Compile the Volume Envelope using linear interpolation bounds across nodes
|
||||||
|
if volume_points and len(volume_points) > 0:
|
||||||
|
# Enforce strict chronological sorting down the timeline axis
|
||||||
|
points = sorted(volume_points, key=lambda x: x["time"])
|
||||||
|
|
||||||
|
# Pad introductory bounds if the initial point coordinate sits past t = 0.0s
|
||||||
|
if points[0]["time"] > 0:
|
||||||
|
first_gain = 10.0 ** (points[0]["db"] / 20.0)
|
||||||
|
idx_end = int(points[0]["time"] * sr)
|
||||||
|
volume_envelope[:idx_end] = first_gain
|
||||||
|
|
||||||
|
for i in range(len(points) - 1):
|
||||||
|
p1, p2 = points[i], points[i+1]
|
||||||
|
idx_start = int(p1["time"] * sr)
|
||||||
|
idx_end = int(p2["time"] * sr)
|
||||||
|
|
||||||
|
gain_start = 10.0 ** (p1["db"] / 20.0)
|
||||||
|
gain_end = 10.0 ** (p2["db"] / 20.0)
|
||||||
|
|
||||||
|
# Linearly interpolate vector increments between adjacent anchor positions
|
||||||
|
volume_envelope[idx_start:idx_end] = np.linspace(gain_start, gain_end, idx_end - idx_start)
|
||||||
|
|
||||||
|
# Pad trailing bounds from the final milestone extending through end-of-file
|
||||||
|
if points[-1]["time"] < duration_sec:
|
||||||
|
last_gain = 10.0 ** (points[-1]["db"] / 20.0)
|
||||||
|
idx_start = int(points[-1]["time"] * sr)
|
||||||
|
volume_envelope[idx_start:] = last_gain
|
||||||
|
|
||||||
|
# 4. Compile the Panning Envelope using linear interpolation bounds across nodes
|
||||||
|
if panning_points and len(panning_points) > 0:
|
||||||
|
points = sorted(panning_points, key=lambda x: x["time"])
|
||||||
|
|
||||||
|
if points[0]["time"] > 0:
|
||||||
|
pan_envelope[:int(points[0]["time"] * sr)] = points[0]["pan"]
|
||||||
|
|
||||||
|
for i in range(len(points) - 1):
|
||||||
|
p1, p2 = points[i], points[i+1]
|
||||||
|
idx_start = int(p1["time"] * sr)
|
||||||
|
idx_end = int(p2["time"] * sr)
|
||||||
|
pan_envelope[idx_start:idx_end] = np.linspace(p1["pan"], p2["pan"], idx_end - idx_start)
|
||||||
|
|
||||||
|
if points[-1]["time"] < duration_sec:
|
||||||
|
pan_envelope[int(points[-1]["time"] * sr):] = points[-1]["pan"]
|
||||||
|
|
||||||
|
# 5. Apply Trigonometric Cosine Fade-In / Fade-Out functions onto the Volume Envelope mask
|
||||||
|
if fade_in_sec > 0:
|
||||||
|
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
|
||||||
|
x_fade = np.linspace(0.0, np.pi, fade_in_samples)
|
||||||
|
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
|
||||||
|
volume_envelope[:fade_in_samples] *= cosine_ramp
|
||||||
|
|
||||||
|
if fade_out_sec > 0:
|
||||||
|
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
|
||||||
|
x_fade = np.linspace(0.0, np.pi, fade_out_samples)
|
||||||
|
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
|
||||||
|
volume_envelope[-fade_out_samples:] *= cosine_ramp
|
||||||
|
|
||||||
|
# 6. Bake Volume Envelope matrices onto the Left and Right discrete audio paths
|
||||||
|
y_stereo[0, :] *= volume_envelope
|
||||||
|
y_stereo[1, :] *= volume_envelope
|
||||||
|
|
||||||
|
# 7. Apply Constant-Power Stereo Panning allocations
|
||||||
|
# Map panning metrics range [-1.0, 1.0] onto angular radians field array [0, pi/2]
|
||||||
|
theta_envelope = ((pan_envelope + 1.0) / 2.0) * (np.pi / 2.0)
|
||||||
|
|
||||||
|
# Evaluate localized amplitude coefficients for physical channels split
|
||||||
|
gain_left = np.cos(theta_envelope)
|
||||||
|
gain_right = np.sin(theta_envelope)
|
||||||
|
|
||||||
|
# Multiply scaling factors directly across corresponding discrete matrices
|
||||||
|
y_stereo[0, :] *= gain_left
|
||||||
|
y_stereo[1, :] *= gain_right
|
||||||
|
|
||||||
|
return y_stereo
|
||||||
|
|
||||||
|
|||||||
+1319
-113
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user