From b59e41656669d89b0ca8a6b4d29664f7c730a8c3 Mon Sep 17 00:00:00 2001 From: locphamtran Date: Sun, 16 Aug 2026 08:37:27 +0700 Subject: [PATCH] =?UTF-8?q?G2.5+G2.6:=20stress=20driver=20for=20persistenc?= =?UTF-8?q?e=20restart=20(N=20loop=20+=20kill/respawn)=20+=20verify=20?= =?UTF-8?q?=E2=80=94=20regression=20G1.5=205/5,=20G2.5=205+10=20PASS,=20G0?= =?UTF-8?q?.4=20probe=20PASS?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 20260815-tasks.md | 8 +- native_bridge/debug/test_g15_stress.py | 4 +- native_bridge/debug/test_g2_stress.py | 181 +++++++++++++++++++++++++ 3 files changed, 188 insertions(+), 5 deletions(-) create mode 100644 native_bridge/debug/test_g2_stress.py diff --git a/20260815-tasks.md b/20260815-tasks.md index 835d733..2e8fad9 100644 --- a/20260815-tasks.md +++ b/20260815-tasks.md @@ -1,7 +1,7 @@ # 20260815 — KẾ HOẠCH THỰC HIỆN G2 (PERSISTENCE RESTART) Nguồn: `big_tasks.md` — header **G2 — Persistence restart (2-3 tuần)**. -Trạng thái roadmap: G0 ✅, G1 ✅ (G1.1–G1.5), **G2 đang bắt đầu**. Branch: `standalone-shm-bridge`. +Trạng thái roadmap: G0 ✅, G1 ✅ (G1.1–G1.5), **G2 ✅ (G2.0–G2.6)**. Branch: `standalone-shm-bridge`. --- @@ -154,8 +154,8 @@ Lý do: G2.2 phụ thuộc G2.1 nhưng giá trị thấp hơn G2.3; làm G2.3 s | G2.2 | main.cpp: control type=5 DUMP_STATE (post saveStateJob len uiWorker ngay lap tuc). lib.rs: flush_bridge_state() — push_control(5,0,0,0) + sleep 300ms, goi dau kill_bridge() (cover restart_bridge_with_sample_rate + stall-restart) va truoc khi giet bridge trong window-close Destroyed | Build PASS (daw_vst_bridge.exe) + cargo check PASS (src-tauri) + deploy + verify live PASS: DUMP qua SHM -> log "DUMP_STATE - flushing state now", state file intact (7 inst), writeIndex advancing | Runtime Rust flush chua test duoc khi khong build app Tauri (cargo check xac nhan compile) | | ✅ | | G2.3 | main.cpp: startup restore block (doc SF_STATE_FILE, guard channel trong, post LOAD job tung channel qua uiWorker, assign + loadSerializedState preset / selectProgram bank-program, g_loadMutex serialize, restore transport PLAY + playhead neu state playing). Vst3Instrument: base64Decode + loadSerializedState (parse [4B BE compLen][comp][4B BE ctrlLen][ctrl], goi restoreState). INativeInstrument.h: virtual loadSerializedState no-op. lib.rs: spawn_bridge set env SF_STATE_FILE = %APPDATA%/SonicForgeDAW/state/bridge_state.json. | Build PASS (bridge + cargo check). Deploy PASS. Verify live PASS: LOAD ch5 + PLAY (playhead 5000) -> kill bridge -> respawn -> log "restoring 7 instruments" + "restored ch=0..6" (Nexus + EZkeys) + "restoring transport PLAY playhead=5000" + ch5 preset 3488B giu nguyen; state file playing=true playhead=5000 inst=7; writeIndex advancing (14859->15234). | - | - | ✅ (Win xong) | | G2.4 | main.cpp: LOAD case double-load guard — skip khi channel da co cung path (lower_plugin_path(ch) == lower_path_str(path)), giu instance + preset da restore khi JS reload sau respawn/bootstrap | Verify live PASS: LOAD lai Nexus ch5 sau respawn -> log "LOAD skip ch=5 (already loaded)", preset ch5 3488B giu nguyen, writeIndex advancing; 15 LOAD skip trong log (JS reload 7ch x2 + test) | Khong can sua JS: loadVstToBridge/loadTrackInstrumentToBridge khong co guard __bridgePlaying (da chay khi PLAY); chi openVstGuiRetry giu guard OPEN_GUI (G0.2, dung) | | ✅ | -| G2.5 | | | | | ⬜ | -| G2.6 | | | | | ⬜ | +| G2.5 | native_bridge/debug/test_g2_stress.py (mới, khung test_g15_stress.py: ctypes SHM, write_control, wait_log, bridge_pid, bad_since) + bước kill daw_vst_bridge giữa vòng + chờ watchdog respawn; verify restore + preset giữ + LOAD-skip guard (G2.4) + writeIndex advancing; test_g15_stress.py: LOAD ch5 chấp nhận "LOAD skip" (guard G2.4 đổi log) | Chạy 5 + 10 iterations PASS (tổng 20 lần kill/respawn): LOAD skip pre-kill/post-respawn OK, restore ch5/ch6 OK, watchdog "[tauri] bridge restarted" OK, bridge alive; preset Nexus giữ nguyên (exact), EZkeys sim>=0.87; writeIndex advancing (delta 283-376) | Assert preset ch6 byte-exact FAIL giả: EZkeys nhúng PlayHeadPos vào state riêng (đổi cả giữa 2 save thường, ratio 0.872) | ch5 so exact, ch6 so ratio>=0.7 + |len chênh|<=64 | ✅ (Win xong) | +| G2.6 | Chạy lại regression G1.5 (test_g15_stress.py) + G2.5 (5+10 iter) + probe G0.4 (bản cập nhật PID + guard-aware, OPEN_GUI full path) | G1.5 regression PASS 5/5 (LOAD/OPEN_GUI/PLAY/STOP/WM_CLOSE, writeIndex delta 307-551, bridge alive); G2.5 PASS 5+5+10 (20 lần kill/respawn, preset giữ, không crash); G0.4 probe PASS: LOAD skip -> OPEN_GUI hwnd -> WM_CLOSE -> capture 2628B -> reload -> restore 2628B -> OPEN_GUI lại attach OK, không restart/EXCEPTION/FAILED | Không có bug sản phẩm mới; 2 bug test-script: (1) probe G0.4 so sánh PID int vs list -> báo restart giả, (2) preset EZkeys byte-diff do PlayHeadPos runtime (đã phân tích ở G2.5) | Sửa test scripts (guard-aware wait, pid compare); không sửa code sản phẩm | ✅ (Win xong) | ### Nhật ký thực hiện - **G2.0** (2026-08-16, Windows): tao `native_bridge/include/StateStore.h` + `native_bridge/src/StateStore.cpp` (CRLF, JSON viet tay, atomic write .tmp→rename, escape \ " @@ -164,6 +164,8 @@ Lý do: G2.2 phụ thuộc G2.1 nhưng giá trị thấp hơn G2.3; làm G2.3 s - **G2.3** (2026-08-16, Windows): auto-reload sau respawn. - **G2.2** (2026-08-16, Windows): dump state truoc kill. main.cpp: them control type=5 DUMP_STATE sau case 4 — post saveStateJob len uiWorker ngay (khong cho debounce 300ms/2s), log "DUMP_STATE - flushing state now". lib.rs: them flush_bridge_state(app) — shm.push_control(5,0,0,0,"") roi sleep 300ms (du thoi gian UiThread ghi xong file); goi o dau kill_bridge() (cover ca restart_bridge_with_sample_rate va stall-restart) va trong on_window_event Destroyed truoc khi kill_process_tree(bridge). Build PASS (daw_vst_bridge.exe) + cargo check PASS (src-tauri). Deploy + verify live PASS: push DUMP qua SHM driver -> log "DUMP_STATE - flushing state now" xuat hien, state file intact (7 inst, playing=false), writeIndex advancing. Ghi chu: nhanh Rust flush chi cargo check (khong build app Tauri) — logic don gian, compile OK. Commit `G2.2: flush bridge state to disk before kill (control type=5 DUMP + Rust pre-kill)`. - **G2.4** (2026-08-16, Windows): go double-load khi JS reload sau respawn. main.cpp: them guard dau LOAD case (truoc setReloading) — neu channel da co cung path (so sanh lower_plugin_path(ch) vs lower_path_str(path)) thi log "LOAD skip ch=N (already loaded)" va return, khong assign lai (thay instance se mat preset da restore tu state file). Ly do: JS reconnect/bootstrap reload gui LOAD cung path sau respawn -> truoc day assign() thay instance -> mat preset. Guard PLAY cua G2.4 (cho restore chay khi PLAY) da co san o JS: loadVstToBridge/loadTrackInstrumentToBridge khong co guard __bridgePlaying, chi openVstGuiRetry giu guard OPEN_GUI (G0.2 — defer GUI khi PLAY van dung). Build PASS (daw_vst_bridge.exe) + deploy + verify live PASS: sau respawn, LOAD lai Nexus ch5 -> "LOAD skip ch=5 (already loaded: ...Nexus.vst3)", preset 3488B giu nguyen, writeIndex advancing; tong 15 LOAD skip (JS reload 7ch x2 + test ch5). Commit `G2.4: skip redundant LOAD of already-loaded instrument (keep restored preset)`. main.cpp: block restore truoc audio loop - doc state file, chi restore channel trong (guard instruments.get(ch)==nullptr), post LOAD job (assign + loadSerializedState / selectProgram) serialize bang g_loadMutex, neu state playing thi set playhead + g_transportPlaying=true + g_stateDirty. Vst3Instrument.cpp: base64Decode (b64val lookup) + loadSerializedState (parse blob, goi restoreState); Vst3Instrument.h + INativeInstrument.h: khai bao. lib.rs: spawn_bridge them env SF_STATE_FILE (fallback StateStore::defaultPath khi khong set). Build PASS (daw_vst_bridge.exe + cargo check). Deploy + verify live PASS: LOAD ch5 + PLAY playhead 5000 -> taskkill bridge -> respawn tu dong -> log restore 7 inst (Nexus ch0-5, EZkeys ch6) + restore transport PLAY playhead=5000 + ch5 preset 3488B giu nguyen; state file playing=true inst=7; writeIndex advancing. Commit `G2.3: auto-reload instruments from state file after bridge respawn`. +- **G2.5** (2026-08-16, Windows): tao native_bridge/debug/test_g2_stress.py (khung test_g15_stress.py). N vong: LOAD Nexus ch5 + EZkeys ch6 (real hoac skip deu OK) -> PLAY -> taskkill daw_vst_bridge -> cho watchdog "[tauri] bridge restarted" + "restored ch=5/6" -> LOAD lai -> verify "LOAD skip" (guard G2.4) + preset giu (ch5 exact, ch6 sim>=0.7 — EZkeys nhung PlayHeadPos vao state, ratio 0.872 giua save thong thuong) + writeIndex advancing. Chay 5 + 10 iterations: PASS (20 lan kill/respawn, khong crash, bridge alive). Kem sua test_g15_stress.py: LOAD ch5 chap nhan "LOAD skip" (guard G2.4 doi log). Commit `G2.5: stress driver for persistence restart (N loop + kill/respawn)`. +- **G2.6** (2026-08-16, Windows): verify tong. G1.5 regression PASS 5/5 (LOAD/OPEN_GUI/PLAY/STOP/WM_CLOSE, writeIndex delta 307-551, bridge alive). G2.5 PASS 5+5+10 (20 lan kill/respawn, preset giu, khong EXCEPTION/FAILED). Probe G0.4 (ban cap nhat: PID dong, LOAD guard-aware, OPEN_GUI full path): PASS — LOAD skip -> OPEN_GUI hwnd -> WM_CLOSE -> "state captured ch=5 component=2628B" -> reload -> "state restored ch=5 component=2628B" -> OPEN_GUI lai attach OK, khong restart. Bug tim duoc: khong co bug san pham; 2 bug test-script (probe PID int vs list -> bao restart gia; preset EZkeys byte-diff do PlayHeadPos runtime). Commit `G2.6: verify — regression G1.5 + stress G2.5 + probe G0.4 all PASS`. --- diff --git a/native_bridge/debug/test_g15_stress.py b/native_bridge/debug/test_g15_stress.py index a0a1f77..14afc70 100644 --- a/native_bridge/debug/test_g15_stress.py +++ b/native_bridge/debug/test_g15_stress.py @@ -92,9 +92,9 @@ fails = 0 for it in range(ITERS): print(f"--- iter {it} ---") since = len(log_lines()) - # LOAD Nexus ch5 + # LOAD Nexus ch5 (already loaded after G2.4 guard -> "LOAD skip" also OK) push_controls([(2, 0, 0, 5, NEXUS)]) - if not wait_log("instrument loaded ch=5 type=0", 30, since): + if not (wait_log("instrument loaded ch=5 type=0", 30, since) or wait_log("LOAD skip ch=5", 30, since)): print(f"iter{it} LOAD Nexus TIMEOUT"); fails += 1; continue # OPEN_GUI ch5 push_controls([(4, 0, 0, 5, NEXUS)]) diff --git a/native_bridge/debug/test_g2_stress.py b/native_bridge/debug/test_g2_stress.py new file mode 100644 index 0000000..b417fec --- /dev/null +++ b/native_bridge/debug/test_g2_stress.py @@ -0,0 +1,181 @@ +# G2.5 persistence-restart stress driver: N iterations of LOAD(Nexus ch5, EZkeys ch6)/PLAY +# -> kill daw_vst_bridge -> watchdog respawn -> verify: restore, preset preserved, +# LOAD-skip guard (G2.4), writeIndex advancing. Watch: no EXCEPTION/FAILED, bridge alive. +import ctypes, ctypes.wintypes as w, struct, subprocess, sys, os, time, json +from difflib import SequenceMatcher + +sys.stdout.reconfigure(encoding='utf-8', errors='replace') + +APP_PID = int(sys.argv[1]) +ITERS = int(sys.argv[2]) if len(sys.argv) > 2 else 3 +SHM_NAME = f"SonicForge_DAW_IPC_{APP_PID}".encode() +APP_LOG = os.path.join(os.environ.get('APPDATA', '.'), 'SonicForgeDAW', 'logs', 'bridge.log') +SPAWN_LOG = os.path.join(os.environ.get('APPDATA', '.'), 'SonicForgeDAW', 'logs', 'spawn.log') +STATE = os.path.join(os.environ.get('APPDATA', '.'), 'SonicForgeDAW', 'state', 'bridge_state.json') +NEXUS = r"C:\Program Files\Common Files\VST3\Nexus.vst3" +EZKEYS = r"C:\Program Files\Common Files\VST3\Toontrack\EZkeys 2.vst3" + +kernel32 = ctypes.WinDLL('kernel32', use_last_error=True) +kernel32.OpenFileMappingA.argtypes = [ctypes.wintypes.DWORD, w.BOOL, ctypes.c_char_p] +kernel32.OpenFileMappingA.restype = ctypes.c_void_p +kernel32.MapViewOfFile.argtypes = [ctypes.c_void_p, ctypes.wintypes.DWORD, ctypes.wintypes.DWORD, ctypes.wintypes.DWORD, ctypes.c_size_t] +kernel32.MapViewOfFile.restype = ctypes.c_void_p +hMap = kernel32.OpenFileMappingA(0xF001F, False, SHM_NAME) +if not hMap: + print(f"OpenFileMapping failed err={ctypes.get_last_error()}"); sys.exit(1) +ptr = kernel32.MapViewOfFile(hMap, 0xF001F, 0, 0, 0) +print(f"SHM mapped ({SHM_NAME}) iters={ITERS}") + +def w32(off, fmt, *args): + ctypes.memmove(ptr + off, struct.pack(fmt, *args), struct.calcsize(fmt)) +def rd32(off): + return ctypes.c_uint32.from_address(ptr + off).value + +def write_control(ctype, arg0, arg1, ch, path): + for i in range(8): + off = 2836 + i * 1040 + if rd32(off) == 0: + w32(off, 'IIII', ctype, arg0, arg1, ch) + if path: + b = path.encode('utf-8')[:1023] + b'\x00' + ctypes.memmove(ptr + off + 16, b, len(b)) + return i + raise RuntimeError("control queue full") + +def push_controls(events): + for i in range(8): + w32(2836 + i * 1040, 'I', 0) + for (ctype, arg0, arg1, ch, path) in events: + write_control(ctype, arg0, arg1, ch, path) + w32(11156, 'I', len(events)) + +def log_lines(path=APP_LOG): + try: + with open(path, 'r', encoding='utf-8', errors='replace') as f: + return f.readlines() + except FileNotFoundError: + return [] + +def wait_log(pattern, timeout, since, path=APP_LOG): + end = time.time() + timeout + while time.time() < end: + lines = log_lines(path) + for i in range(since, len(lines)): + if pattern in lines[i]: + return lines[i] + time.sleep(0.2) + return None + +def wait_load(ch, timeout, since): + # Either a real LOAD (plugin was different) or guard skip (same path already loaded). + return wait_log(f"instrument loaded ch={ch} type=0", timeout, since) or \ + wait_log(f"LOAD skip ch={ch}", timeout, since) + +def state(): + if not os.path.exists(STATE): return None + return json.load(open(STATE, encoding='utf-8')) + +def preset_of(ch, s=None): + s = s or state() + for e in (s or {}).get('instruments', []): + if e.get('channel') == ch: + return e.get('presetBase64', '') or '' + return '' + +def bridge_pid(): + out = subprocess.run(['powershell', '-Command', + '(Get-Process daw_vst_bridge -ErrorAction SilentlyContinue).Id'], + capture_output=True, text=True).stdout.strip().splitlines() + return [int(x) for x in out if x.strip().isdigit()] + +def kill_bridge(): + for pid in bridge_pid(): + subprocess.run(['taskkill', '/PID', str(pid), '/F'], + capture_output=True, text=True) + time.sleep(1) + return bridge_pid() + +def bad_since(since): + return [l for l in log_lines()[since:] if 'EXCEPTION' in l or 'FAILED' in l] + +fails = 0 +for it in range(ITERS): + print(f"--- iter {it} ---") + since = len(log_lines()) + # 1. Ensure both instruments loaded (already loaded -> LOAD skip guard is fine). + push_controls([(2, 0, 0, 5, NEXUS), (2, 0, 0, 6, EZKEYS)]) + m5 = wait_load(5, 20, since) + m6 = wait_load(6, 20, since) + ok = bool(m5) and bool(m6) + print(f"iter{it} LOAD pre-kill ch5={bool(m5)} ch6={bool(m6)} ({'skip' if 'skip' in (m5 or '') else 'real'} / {'skip' if 'skip' in (m6 or '') else 'real'})") + if not ok: fails += 1 + time.sleep(2) + base5 = preset_of(5); base6 = preset_of(6) + print(f"iter{it} baseline preset ch5={len(base5)}B ch6={len(base6)}B") + if not base5 or not base6: + print(f"iter{it} baseline preset missing — abort"); sys.exit(1) + # 2. PLAY -> writeIndex advancing. + wi0 = rd32(4) + push_controls([(3, 1, 0, 0, "")]) + if not wait_log("transport PLAY", 10, since): + print(f"iter{it} PLAY TIMEOUT"); fails += 1; continue + time.sleep(1.5) + wi1 = rd32(4) + ok = (wi1 - wi0) > 50 + print(f"iter{it} PLAY writeIndex {wi0}->{wi1} delta={wi1-wi0} {'OK' if ok else 'STALLED'}") + if not ok: fails += 1 + # 3. Kill bridge -> watchdog respawn. + ks = len(log_lines()) + dead = kill_bridge() + print(f"iter{it} killed bridge (remaining: {dead})") + if dead: + print(f"iter{it} bridge still alive after kill"); fails += 1; continue + r = wait_log("[tauri] bridge restarted", 30, ks, APP_LOG) + print(f"iter{it} watchdog restart log: {'OK' if r else 'TIMEOUT'}") + if not r: fails += 1; continue + r5 = wait_log("restored ch=5", 30, ks) + r6 = wait_log("restored ch=6", 30, ks) + print(f"iter{it} restore ch5={bool(r5)} ch6={bool(r6)}") + if not (r5 and r6): fails += 1 + # 4. LOAD-skip guard after respawn (explicit re-LOAD same path). + ls = len(log_lines()) + push_controls([(2, 0, 0, 5, NEXUS), (2, 0, 0, 6, EZKEYS)]) + m5b = wait_log("LOAD skip ch=5", 20, ls) + m6b = wait_log("LOAD skip ch=6", 20, ls) + print(f"iter{it} LOAD-skip post-respawn ch5={bool(m5b)} ch6={bool(m6b)}") + if not (m5b and m6b): fails += 1 + # 5. Preset preserved after respawn (+ redundant LOAD). + # ch5 Nexus: byte-identical (stable). ch6 EZkeys: plugin bakes playhead pos + # into its own state (changes between NORMAL saves too) -> similarity check. + time.sleep(3) + s2 = state() + p5 = preset_of(5, s2); p6 = preset_of(6, s2) + ok5 = p5 == base5 + ok6 = abs(len(p6) - len(base6)) <= 64 and SequenceMatcher(None, p6, base6).ratio() >= 0.7 + okp = ok5 and ok6 + print(f"iter{it} preset after respawn ch5={len(p5)}B(exact={ok5}) ch6={len(p6)}B(sim={round(SequenceMatcher(None, p6, base6).ratio(),3) if p6 else 0}) {'OK' if okp else 'CHANGED'}") + if not okp: fails += 1 + # 6. writeIndex advancing after respawn (PLAY re-sent in case restored playing=false). + wi2 = rd32(4) + push_controls([(3, 1, 0, 0, "")]) + time.sleep(2) + wi3 = rd32(4) + ok = (wi3 - wi2) > 50 + print(f"iter{it} post-respawn writeIndex {wi2}->{wi3} delta={wi3-wi2} {'OK' if ok else 'STALLED'}") + if not ok: fails += 1 + bad = bad_since(since) + if bad: + print(f"iter{it} EXCEPTION/FAILED: {bad[-2:]}"); fails += 1 + # stop transport at end of iter so next iter starts clean + push_controls([(3, 0, 0, 0, "")]) + time.sleep(1) + +# final watch: bridge alive + no new bad +since = len(log_lines()) +time.sleep(3) +bad = bad_since(since) +alive = bridge_pid() +print(f"bridge alive: {alive} (expect non-empty)") if alive else print("BRIDGE DEAD") +if not alive: fails += 1 +print(f"G2.5 RESULT: {'PASS' if fails == 0 else f'FAIL ({fails} issues)'}") +sys.exit(0 if fails == 0 else 1)