15 KiB
Technical Specification: Implementing Volume, Fades & Panning Envelope Arrays on Audio Signals
This document defines the mathematical models, data flow diagrams (Audio Node Graph), and execution source code required to apply interactive graphical curves (Volume Automation, Fades, and Panning Automation) into the real-time digital signal processing pipeline on the Frontend and offline file export rendering on the Dockerized Python Backend.
1. Multi-stage Audio Node Graph
To simultaneously compute all three graphical configurations over the audio stream without precipitating phase cancellation or signal latency anomalies, the environment builds an explicit downstream node connection graph:
┌─────────────────────────┐
│ AudioBufferSourceNode │ --> Streams the native original raw buffer array
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ GainNode (Automation) │ --> Modulates Volume dynamically via multi-point automation arrays
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ StereoPannerNode │ --> Transposes the Stereo Image (L/R Balance Automation trajectory)
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ GainNode (Fades) │ --> Multiplies bounding Fade-In and Fade-Out curves
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ AudioContext.destination│ --> Routes processed signal to hardware device outputs (Speakers/Headphones)
└─────────────────────────┘
2. Mathematical Formulations for Modulators
2.1. Multi-Point Volume Automation Curves
The vertical coordinate axis Y of the volume points plots decibel thresholds bounded from -30\text{ dB} to +3\text{ dB}. Prior to applying multipliers onto the signal, the logarithmic values must be translated into a standard linear scalar gain coefficient G_{\text{linear}}:
G_{\text{linear}}(t) = 10^{\frac{V_{\text{dB}}(t)}{20}}
At an arbitrary timeline timestamp t residing between two chronologically adjacent control nodes P_1(t_1, V_1) and P_2(t_2, V_2), the target volume attenuation value is computed via standard linear interpolation:
V_{\text{dB}}(t) = V_1 + (t - t_1) \cdot \frac{V_2 - V_1}{t_2 - t_1}
2.2. Constant-Power Stereo Panning
To ensure that when a user shifts the audio image toward the Left (L) or Right (R) perimeter channels, the cumulative output sound energy emitted by the drivers does not collapse (avoiding a volume drop at the absolute horizontal center axis—known as the Center Dip anomaly), the system implements the Constant-Power Panning Law.
Let p(t) \in [-1.0, 1.0] map to the explicit panning index at timestamp t (where -1.0 represents a hard-left channel displacement, 0.0 marks absolute center, and +1.0 dictates a hard-right channel boundary).
Convert the raw linear panning factor p(t) into a circular panning sweep angle coordinate \theta(t) \in [0, \pi/2]:
\theta(t) = \frac{p(t) + 1}{2} \cdot \frac{\pi}{2}
Calculate the independent amplitude scalar gains for the Left channel (g_L) and the Right channel (g_R) elements:
g_L(t) = \cos(\theta(t)), \quad g_R(t) = \sin(\theta(t))
Mathematical Proof: The total sound field energy remains perfectly preserved under all operational transformations because:
g_L(t)^2 + g_R(t)^2 = \cos^2(\theta(t)) + \sin^2(\theta(t)) = 1.0
2.3. Fade Curves (Fade-In & Fade-Out)
Fading shapes are driven by a trigonometric Cosine equation framework to build organic, smooth amplitude transitions at the structural boundary zones of the audio asset:
- Fade-In Curve (Across an introductory duration window of
L_{\text{fade}}seconds):
f_{\text{in}}(t) = \frac{1 - \cos\left( \pi \cdot \frac{t}{L_{\text{fade}}} \right)}{2} \quad \text{for } 0 \le t < L_{\text{fade}}
- Fade-Out Curve (Across a trailing termination window of
L_{\text{fade}}seconds):
f_{\text{out}}(t) = \frac{1 + \cos\left( \pi \cdot \frac{t - (T_{\text{max}} - L_{\text{fade}})}{L_{\text{fade}}} \right)}{2} \quad \text{for } T_{\text{max}} - L_{\text{fade}} \le t \le T_{\text{max}}
3. Client-Side Runtime Integration (Web Audio API - Live Playback Modulator)
This JavaScript module sets up the physical Web Audio node graphs and automates parameters directly matching the real-time audio thread clocks:
/**
* Configures a real-time audio node processing graph with parameter automation.
* @param {AudioContext} audioCtx - The active Web Audio runtime context instance.
* @param {AudioBuffer} audioBuffer - Decoded original raw target audio source asset.
* @param {number} startTime - Global time position index marking where playback initiates (seconds).
* @param {Array} volumeNodes - Automation point layout maps: [{time: 0.5, db: -3.0}, ...].
* @param {Array} panningNodes - Panning position layout maps: [{time: 1.2, pan: -0.5}, ...].
* @param {object} fadeConfig - Bounding fade time constants: {fadeInLen: 0.5, fadeOutLen: 0.8}.
*/
function playTrackWithAutomation(audioCtx, audioBuffer, startTime, volumeNodes, panningNodes, fadeConfig) {
// 1. Instantiate the Global Audio Source Buffer Node
const sourceNode = audioCtx.createBufferSource();
sourceNode.buffer = audioBuffer;
// 2. Instantiate the Gain Node managing Volume Automation tracking loops
const volumeGainNode = audioCtx.createGain();
// Establish baseline default state variables at Unity Gain (0 dB)
volumeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
// Map timeline automations for custom Volume Node trajectories
if (volumeNodes && volumeNodes.length > 0) {
// Purge legacy scheduled values to safely overwrite parameters
volumeGainNode.gain.cancelScheduledValues(audioCtx.currentTime);
volumeNodes.forEach(node => {
const timeOffset = startTime + node.time;
const linearGain = Math.pow(10, node.db / 20); // Map logarithmic dB thresholds to linear multipliers
volumeGainNode.gain.linearRampToValueAtTime(linearGain, audioCtx.currentTime + node.time);
});
}
// 3. Instantiate the StereoPannerNode for Panning Automation structures
const pannerNode = audioCtx.createStereoPanner();
pannerNode.pan.setValueAtTime(0.0, audioCtx.currentTime); // Standard initialization locked at Center
// Map timeline automations for Panning Node trajectories
if (panningNodes && panningNodes.length > 0) {
pannerNode.pan.cancelScheduledValues(audioCtx.currentTime);
panningNodes.forEach(node => {
// Enforce rigid clipping bounds to keep panning factors inside [-1.0, 1.0]
const clampedPan = Math.max(-1.0, Math.min(1.0, node.pan));
pannerNode.pan.linearRampToValueAtTime(clampedPan, audioCtx.currentTime + node.time);
});
}
// 4. Instantiate the Gain Node dedicated to boundary Fades
const fadeGainNode = audioCtx.createGain();
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime);
const duration = audioBuffer.duration;
// Calculate and schedule introductory Fade-In values
if (fadeConfig.fadeInLen > 0) {
fadeGainNode.gain.setValueAtTime(0.0, audioCtx.currentTime);
fadeGainNode.gain.linearRampToValueAtTime(1.0, audioCtx.currentTime + fadeConfig.fadeInLen);
}
// Calculate and schedule terminating Fade-Out values
if (fadeConfig.fadeOutLen > 0) {
const fadeOutStart = duration - fadeConfig.fadeOutLen;
fadeGainNode.gain.setValueAtTime(1.0, audioCtx.currentTime + fadeOutStart);
fadeGainNode.gain.linearRampToValueAtTime(0.0, audioCtx.currentTime + duration);
}
// 5. Connect the physical downstream structural audio pipeline
sourceNode.connect(volumeGainNode);
volumeGainNode.connect(pannerNode);
pannerNode.connect(fadeGainNode);
fadeGainNode.connect(audioCtx.destination);
// 6. Drive hardware execution loops
sourceNode.start(0);
return { sourceNode, volumeGainNode, pannerNode, fadeGainNode };
}
4. Server-Side Execution Engine (Dockerized Python Engine - NumPy Processing)
When a user triggers an Apply action or an offline Export script, the frontend dispatches serialized JSON configuration models down to the Python backend framework. The signal processing architecture uses high-efficiency vectorized loops inside NumPy to multiply envelope modulators straight onto raw multi-channel float data:
import numpy as np
class DSPAudioModulator:
@staticmethod
def apply_automation_and_panning(
y_raw: np.ndarray,
sr: int,
volume_points: list, # [{"time": 0.5, "db": -6.0}, ...]
panning_points: list, # [{"time": 1.0, "pan": -0.7}, ...]
fade_in_sec: float = 0.0,
fade_out_sec: float = 0.0
) -> np.ndarray:
"""
Applies multi-point volume envelopes, constant-power panning, and trigonometric fades
directly onto a 1D (Mono) or 2D (Stereo) acoustic NumPy signal array.
Input: y_raw maps to the raw sound array (Mono/Stereo matrix bounded inside [-1.0, 1.0]).
Output: y_processed yields a 2D interleaved Stereo NumPy array (2, N) with baked modulations.
"""
total_samples = y_raw.shape[-1] if len(y_raw.shape) > 1 else len(y_raw)
duration_sec = total_samples / sr
# 1. Guarantee Stereo geometry dimensions (2 discrete channels) for Panning operations
if len(y_raw.shape) == 1:
# For Mono arrays, clone sample metrics symmetrically to Left/Right matrices
y_stereo = np.vstack((y_raw, y_raw))
else:
y_stereo = np.copy(y_raw)
# 2. Allocate Envelope Mask arrays matching total track samples limits
volume_envelope = np.ones(total_samples, dtype=np.float32)
pan_envelope = np.zeros(total_samples, dtype=np.float32) # Default initialization: Center (0.0)
# 3. Compile the Volume Envelope using linear interpolation bounds across nodes
if volume_points and len(volume_points) > 0:
# Enforce strict chronological sorting down the timeline axis
points = sorted(volume_points, key=lambda x: x["time"])
# Pad introductory bounds if the initial point coordinate sits past t = 0.0s
if points[0]["time"] > 0:
first_gain = 10.0 ** (points[0]["db"] / 20.0)
idx_end = int(points[0]["time"] * sr)
volume_envelope[:idx_end] = first_gain
for i in range(len(points) - 1):
p1, p2 = points[i], points[i+1]
idx_start = int(p1["time"] * sr)
idx_end = int(p2["time"] * sr)
gain_start = 10.0 ** (p1["db"] / 20.0)
gain_end = 10.0 ** (p2["db"] / 20.0)
# Linearly interpolate vector increments between adjacent anchor positions
volume_envelope[idx_start:idx_end] = np.linspace(gain_start, gain_end, idx_end - idx_start)
# Pad trailing bounds from the final milestone extending through end-of-file
if points[-1]["time"] < duration_sec:
last_gain = 10.0 ** (points[-1]["db"] / 20.0)
idx_start = int(points[-1]["time"] * sr)
volume_envelope[idx_start:] = last_gain
# 4. Compile the Panning Envelope using linear interpolation bounds across nodes
if panning_points and len(panning_points) > 0:
points = sorted(panning_points, key=lambda x: x["time"])
if points[0]["time"] > 0:
pan_envelope[:int(points[0]["time"] * sr)] = points[0]["pan"]
for i in range(len(points) - 1):
p1, p2 = points[i], points[i+1]
idx_start = int(p1["time"] * sr)
idx_end = int(p2["time"] * sr)
pan_envelope[idx_start:idx_end] = np.linspace(p1["pan"], p2["pan"], idx_end - idx_start)
if points[-1]["time"] < duration_sec:
pan_envelope[int(points[-1]["time"] * sr):] = points[-1]["pan"]
# 5. Apply Trigonometric Cosine Fade-In / Fade-Out functions onto the Volume Envelope mask
if fade_in_sec > 0:
fade_in_samples = min(total_samples, int(fade_in_sec * sr))
x_fade = np.linspace(0.0, np.pi, fade_in_samples)
cosine_ramp = (1.0 - np.cos(x_fade)) / 2.0
volume_envelope[:fade_in_samples] *= cosine_ramp
if fade_out_sec > 0:
fade_out_samples = min(total_samples, int(fade_out_sec * sr))
x_fade = np.linspace(0.0, np.pi, fade_out_samples)
cosine_ramp = (1.0 + np.cos(x_fade)) / 2.0
volume_envelope[-fade_out_samples:] *= cosine_ramp
# 6. Bake Volume Envelope matrices onto the Left and Right discrete audio paths
y_stereo[0, :] *= volume_envelope
y_stereo[1, :] *= volume_envelope
# 7. Apply Constant-Power Stereo Panning allocations
# Map panning metrics range [-1.0, 1.0] onto angular radians field array [0, pi/2]
theta_envelope = ((pan_envelope + 1.0) / 2.0) * (np.pi / 2.0)
# Evaluate localized amplitude coefficients for physical channels split
gain_left = np.cos(theta_envelope)
gain_right = np.sin(theta_envelope)
# Multiply scaling factors directly across corresponding discrete matrices
y_stereo[0, :] *= gain_left
y_stereo[1, :] *= gain_right
return y_stereo
5. Viewport Coordinate Mapping & Data Serialization Protocols
As users drag and adjust coordinate anchors over the visual drawing Canvas, mouse-event pixel coordinates are continuously calculated and mapped into absolute physical values to preserve data matching between frontend layouts and backend signal arrays:
[ GRAPH CANVAS VIEWPORT COORDINATES ] [ REAL-WORLD SYSTEM PHENOMENA VALUES ]
x (pixel) ─────────────────────────────────────► Timeline position t (seconds) = x / zoom_level
y (pixel) ─ (Volume: Center axis maps to 0dB) ─► db = ( (h - y) / h_half ) * range_db
y (pixel) ─ (Panning: Center axis maps to 0) ──► pan = ( (h_half - y) / h_half ) -> Bounded [-1.0, 1.0]
Serialized API Data Transfer Model (Standard JSON Package Syntax)
{
"track_id": "1",
"fades": {
"fade_in_sec": 0.500,
"fade_out_sec": 1.200
},
"volume_automation": [
{ "time": 0.000, "db": 0.0 },
{ "time": 1.450, "db": 3.0 },
{ "time": 3.820, "db": -12.5 },
{ "time": 6.000, "db": 0.0 }
],
"panning_automation": [
{ "time": 0.000, "pan": 0.0 },
{ "time": 2.100, "pan": -0.8 },
{ "time": 4.500, "pan": 0.8 },
{ "time": 6.000, "pan": 0.0 }
]
}