72 lines
5.7 KiB
Markdown
72 lines
5.7 KiB
Markdown
Here is the complete document converted into a clean, professionally formatted Markdown layout, with fully optimized math expressions and standardized structures:
|
|
|
|
# Analysis & Bug Fix Guide: Hybrid DSP Architecture & Shift+Click Selection Algorithms
|
|
|
|
This document clarifies the execution boundaries of real-time audio monitoring (Real-time Preview) between the workstation (Client) and the server (Docker Server). It exposes the root cause of the "Shift + Click" selection range failure and provides a direct solution on the client-side.
|
|
|
|
---
|
|
|
|
## 1. Technical Q&A (Zoom-In & Hybrid Model)
|
|
|
|
### 1.1. Is it necessary to process audio vectors directly on the Client machine like Reaper or Sound Forge?
|
|
|
|
* **Answer:** Absolutely necessary ($100\%$) for **Visual Rendering**.
|
|
* **Reason:** When zooming deeply to observe individual granular phase fluctuations (**Sample Nodes**), the browser must have direct access to the raw binary array (`Float32Array`) stored in the client's RAM.
|
|
* **The Pitfall of Server-side Rendering:** If a "server-side render and push image" approach is used, the system will suffer from image blurring and network latency ($100\text{ms} \rightarrow 500\text{ms}$) during high-speed zooming or scrubbing. Decoding the file once via the Web Audio API (`AudioContext.decodeAudioData()`) on the Frontend is the industry-standard DAW solution to unlock instantaneous vector rendering at $60\text{ FPS} \rightarrow 120\text{ FPS}$ directly inside the browser.
|
|
|
|
### 1.2. Can a hybrid web application match the performance of a native desktop application?
|
|
|
|
Yes, it can execute seamlessly provided there is a clean, structured separation of roles (**Symmetrical Hybrid Separation**):
|
|
|
|
* **Client (HTML5/Web Audio/WASM):** Handles low-latency user interface interactions. This includes reading sample arrays to paint waveforms, tracking the playhead line, defining selection ranges, and driving real-time preview monitoring filters using Web Audio Nodes or WebAssembly.
|
|
* **Server (Dockerized Python):** Executes heavy rendering blocks and exports studio-grade master files. This includes multi-track mixdowns, loading genuine VST3 plugin chains via a C++ core framework (e.g., `Pedalboard`), and processing complex AI models. When changes occur, the frontend simply dispatches a lightweight JSON configuration package (**Metadata**) back to the server for asynchronous rendering, bypassing audio streaming bottlenecks.
|
|
|
|
---
|
|
|
|
## 2. Root Causes of the "Shift + Click Selection" Defect
|
|
|
|
Many AI Code Agents fail or struggle when programming this interaction loop because of several fundamental flaws:
|
|
|
|
* **Audio Waveforms on Canvas Lack DOM Nodes:** Unlike standard HTML texts where double-clicking or `Shift + Click` selections can be tracked natively between text tags, audio waveforms are flattened onto a raw `<canvas>` element. Mouse clicks only return physical pixel coordinates ($X$). Agents frequently omit the coordinate translation logic needed to map pixels back into absolute timeline seconds:
|
|
|
|
$$T = \frac{X_{\text{pixel}} + \text{scrollLeft}}{\text{Zoom}}$$
|
|
|
|
|
|
* **Event Listener Collision:** In DAW workflows, the primary mouse-down trigger (`onMouseDown`) over a track lane handles multiple overlapping roles: updating playhead placement, dragging audio clips, dragging perimeters for time-stretching, and dragging to create selection windows. When a user executes a `Shift + Click` interaction, if default behaviors are not explicitly blocked via `e.preventDefault()` and `e.stopPropagation()`, the system misinterprets the gesture as a playhead reset or a clip drag event, instantly destroying the existing selection.
|
|
* **Missing Anchor Point Tracking:** For `Shift + Click` to scale a region properly, the application must persistently cache an **Anchor Point** variable in memory:
|
|
* **Click 1 (Initial Focus):** Sets the bounding anchor milestone (e.g., $T_{\text{start}}$).
|
|
* **Shift + Click 2 (Extension):** Locks the anchor milestone and assigns a new dynamic timestamp parameter ($T_{\text{end}}$) to the secondary click coordinate.
|
|
|
|
|
|
|
|
---
|
|
|
|
## 3. Shift + Click Interaction Selection Algorithm
|
|
|
|
This interaction sequence is implemented by intercepting the state of the modifier parameter `e.shiftKey` inside the click handler logic for both the track lanes (localized selection—Local) and the timeline ruler (global master selection—Global).
|
|
|
|
### 3.1. Mouse Event Control Logic Schema
|
|
|
|
```text
|
|
[ MOUSE PRESS EVENT ON CANVAS / RULER ]
|
|
│
|
|
┌───────────────┴───────────────┐
|
|
▼ (Is Shift Key Active?) ▼ (Shift Key Inactive)
|
|
[ SHIFT + CLICK LOGIC ] [ STANDARD CLICK LOGIC ]
|
|
- Lock the existing Anchor point - Instantiate a new Anchor = Click Time
|
|
- Map new Click Time = End Time - Prepare Drag state for new region draw
|
|
- Refresh selection overlay color - Update Playhead location
|
|
|
|
```
|
|
|
|
### 3.2. Implementation Blueprint
|
|
|
|
Update your `index.html` source script with the following event mapping rules:
|
|
|
|
#### **At the Waveform Lane Viewport:**
|
|
|
|
When a mouse press is detected, evaluate `e.shiftKey`. If `true`, lock the initial boundary position from the existing selection (`localSelLeft`) as the anchor point. If no selection is present, fallback to the current playhead position (`currentTime`). Then, assign the calculated timeline position of the new click event to override the secondary boundary marker (`localSelectionEnd`).
|
|
|
|
#### **At the Time Ruler Track:**
|
|
|
|
Mirror the exact same bounding logic block to manage the global system selection layer (`selectionStart` and `selectionEnd`), enabling users to stretch or compress the global transport loop constraints efficiently. |