132 lines
6.0 KiB
Markdown
132 lines
6.0 KiB
Markdown
# Technical Specification: Bug Fix for Clip Visual Stretching
|
|
|
|
This document analyzes the root cause and defines the waveform painting algorithm to fix the bug where a short audio clip is incorrectly stretched to fill the viewport when pasted onto a new track, referencing the real-world visual analysis in `image_f05ca7.png`.
|
|
|
|
---
|
|
|
|
## 1. Root Cause
|
|
|
|
Based on `image_f05ca7.png`, the error occurs because the track lane's canvas render logic utilizes the total viewport width ($W_{\text{viewport}}$) as the bounding milestone to distribute and draw the entire sample count of the buffer.
|
|
|
|
* **Bug Mechanism:** The system treats the clip's start point as $0$ and the clip's end point as the end of the screen, completely ignoring the clip's actual duration ($T_{\text{clip}}$) and starting time coordinates ($t_{\text{offset}}$) of the pasted segment.
|
|
* **Consequence:** The short clip is stretched with an incorrect display frequency, falling completely out of sync with the global Time Ruler at the top.
|
|
|
|
---
|
|
|
|
## 2. Technical Solution: Coordinate System Alignment
|
|
|
|
To display the audio clip at its correct duration and position, every clip on the timeline must be managed using two core attributes:
|
|
|
|
* **$t_{\text{offset}}$ (seconds):** The timeline insertion position where the clip starts (the position of the playhead at the moment of pasting).
|
|
* **$T_{\text{clip}}$ (seconds):** The actual duration of the sliced audio file ($T_{\text{clip}} = \text{samples} / \text{sample\_rate}$).
|
|
|
|
```text
|
|
Global Timeline
|
|
+───────────────────────────────────────────────────────────────────────────+
|
|
│ │
|
|
│ Track 01: [█████████████████████████████████████████████████████████] │
|
|
│ │
|
|
│ Track 02: [██████████████] <--- Render only this range │
|
|
│ ▲ ▲ │
|
|
│ │ │ │
|
|
│ t_offset t_offset + T_clip │
|
|
+───────────────────────────────────────────────────────────────────────────+
|
|
|
|
```
|
|
|
|
### 2.1. Pixel Mapping Formula
|
|
|
|
Let $Z$ be the current zoom level (the number of display pixels per second of audio). The initial rendering coordinate and the physical width of the clip on the Canvas must strictly follow these formulas:
|
|
|
|
* **Starting rendering coordinate ($X_{\text{start}}$):**
|
|
|
|
$$X_{\text{start}} = t_{\text{offset}} \times Z$$
|
|
|
|
|
|
* **Physical width of the waveform ($W_{\text{clip}}$):**
|
|
|
|
$$W_{\text{clip}} = T_{\text{clip}} \times Z$$
|
|
|
|
|
|
|
|
---
|
|
|
|
## 3. Safe Waveform Render Algorithm (Python & JS)
|
|
|
|
The Render Loop must exclusively calculate and draw peak amplitudes within the bound stretching from $X_{\text{start}}$ to $X_{\text{start}} + W_{\text{clip}}$. Any pixel region outside this interval must be painted with an empty background color (transparent or the track's dark background theme).
|
|
|
|
### 3.1. Pseudo-code
|
|
|
|
```python
|
|
def render_track_lane(canvas_width, zoom_level, track_clip):
|
|
# 1. Compute rendering boundaries based on synchronization formulas
|
|
x_start = track_clip.offset_seconds * zoom_level
|
|
w_clip = track_clip.duration_seconds * zoom_level
|
|
x_end = x_start + w_clip
|
|
|
|
# 2. Initialize empty background
|
|
initialize_background(0, canvas_width)
|
|
|
|
# 3. Scan the pixel array and only render within the active clip segment
|
|
for x in range(0, canvas_width):
|
|
if x < x_start or x > x_end:
|
|
# Paint empty background color for region with no data
|
|
draw_background_pixel(x)
|
|
else:
|
|
# Map current pixel x coordinate back to sample index in Buffer
|
|
time_in_clip = (x - x_start) / zoom_level
|
|
sample_index = int(time_in_clip * track_clip.sample_rate)
|
|
|
|
# Calculate amplitude peak and draw symmetrical vertical line
|
|
amplitude_peak = get_peak_amplitude(track_clip.buffer, sample_index)
|
|
draw_waveform_vertical_line(x, amplitude_peak)
|
|
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Guarding Array Boundaries on Python Docker Server
|
|
|
|
When a user triggers a cut/paste operation on the Frontend, the JSON data structure dispatched to the Python Server must explicitly specify the destination paste coordinates to prevent index calculation errors:
|
|
|
|
```json
|
|
{
|
|
"action": "paste_clip",
|
|
"source_clip": {
|
|
"clip_id": "clip_abc123",
|
|
"duration_seconds": 24.150,
|
|
"sample_rate": 44100
|
|
},
|
|
"destination": {
|
|
"track_id": "02",
|
|
"paste_at_seconds": 15.300
|
|
}
|
|
}
|
|
|
|
```
|
|
|
|
On the Python backend (utilizing `pydub` or `numpy`), the binary array insertion is executed at the exact time milestone by zero-padding the preceding segment to align perfectly:
|
|
|
|
```python
|
|
import numpy as np
|
|
|
|
def insert_clip_to_track_array(track_array: np.ndarray, sr: int, clip_array: np.ndarray, paste_sec: float) -> np.ndarray:
|
|
"""
|
|
Inserts clip_array into track_array at paste_sec without stretching the signal.
|
|
"""
|
|
paste_sample = int(paste_sec * sr)
|
|
clip_length = len(clip_array)
|
|
|
|
# Create a new array with a length covering the entire pasted segment
|
|
required_length = max(len(track_array), paste_sample + clip_length)
|
|
output_array = np.zeros(required_length, dtype=np.float32)
|
|
|
|
# Copy original track data over
|
|
output_array[0:len(track_array)] = track_array
|
|
|
|
# Overwrite the new clip at the precise real-time coordinate position
|
|
output_array[paste_sample:paste_sample + clip_length] = clip_array
|
|
|
|
return output_array
|
|
|
|
``` |