197 lines
13 KiB
Markdown
197 lines
13 KiB
Markdown
# RUNTIME CAPTURE — LIVE DSP TABLES (2026-08-19)
|
||
|
||
## BREAKTHROUGH: heap "registry" object found & read during offline render
|
||
- During `reaper -nosplash -renderproject render_long.rpp`, the yabridge-host
|
||
allocates a DSP arena containing a **registry array**: a run of `{u64 count, u64 ptr}`
|
||
pairs (stride 0x10) pointing at every DSP buffer. At capture time:
|
||
registry base = **0x29b06c0** (in anon region 0x2922000, 0x77e000).
|
||
(GUI-session registry was 0x28b06c0 — same object, shifted by arena layout.)
|
||
- Finding it live: scan host-readable memory (chunked, 8MB) for `u64==8193` (0x2001)
|
||
followed by a readable ptr, then require a run of count/ptr pairs at stride 0x10.
|
||
- Registry owner chain: 0 hits for a u64==registry address, so it is reached
|
||
structurally (object member at some fixed offset), not via an explicit global.
|
||
|
||
## Captured tables (SR=44100 project, but internal freq-axis = 48000!)
|
||
| reg idx | count | ptr | content |
|
||
|---------|-------|----------|---------|
|
||
| [00] | 8193 | 0x2a72600 | identity ~1.0 |
|
||
| [01] | 8193 | 0x2111140 | **WIN_freq: 0.5 -> 1.0 (saturates; idx1024=0.68, idx2048=0.8)** — the FFT-conv window (0x540658) |
|
||
| [02] | 8193 | 0x29f2280 | 0 -> ~0.01 (levels/curve) |
|
||
| [03] | 8193 | 0x29fa300 | **0.596 -> 0.126** = matches known rwin_C0 |
|
||
| [04] | 8193 | 0x2a02340 | **0.404 -> 0.874** = 1-[03] complement |
|
||
| [05] | 8193 | 0x2a0a3c0 | 0.0435 -> ~0 (weight, small) |
|
||
| [06] | 8193 | 0x2a12400 | 0.9565 -> ~1.0 (1-[05]) |
|
||
| [07] | 8193 | 0x2aca740 | zeros + small negatives |
|
||
| [0d] | 2049 | 0x2cd0fc0 | **freq-axis 0..23988.3 Hz, spacing 11.713 = 48000/4096** → internal SR=48000 |
|
||
| [0e] | 8193 | 0x2cd9000 | 2.017 -> 0 (LUT/knee?) |
|
||
| [0f] | 16384 | 0x2ce9080 | same as [0e] doubled |
|
||
| [10] | 16384 | 0x2cf90c0 | 1.2914 -> 0 |
|
||
| [12] | 16384 | 0x2a62540 | 11.29.. (scattering) |
|
||
| [14]/[15] | 8193 | .. | first-fire IR? 0,0.022,0.104,0.084,0.018,0 |
|
||
| [17] | 32768 | 0x2d29140 | 0.9999 -> ~1 (ramp) |
|
||
| [19] | 32768 | 0x2d69200 | 1.2915 -> 1.0 |
|
||
| others | 32768/65536 | .. | ones / ramps (FFT plans, mirrors) |
|
||
|
||
- Key numeric check: registry[03] head 0.5960761, idx1024 0.168, idx2048 0.1257 —
|
||
byte-identical to earlier GUI rwin_C0 (0.596 -> 0.126). Confirms registry IS the
|
||
authoritative per-bin weight source; tables are stable across sessions/SR.
|
||
- **FREQ-AXIS uses internal SR=48000 regardless of project 44100** (spacing 11.713).
|
||
This reconciles "rwin tables at 48k" even when rendering 44.1k projects.
|
||
|
||
## Files saved (handoff/)
|
||
- `rtwin_freq_44100.npy` — WIN_freq (8193 f32): 0.5 -> 1.0 (this is live 0x540658 window)
|
||
- `rtfreqaxis_48000_internal.npy` — freq-axis (2048 f32, 0..23988.3, spacing 11.713)
|
||
- `rtwa_596.npy` — [03] 0.596->0.126
|
||
- `rtwb_404.npy` — [04] 0.404->0.874
|
||
- `rtwc_043.npy` — [05]
|
||
- `rtwd_956.npy` — [06]
|
||
- Full raw snapshot: /tmp/snap_all.bin (318MB, entries {lo,sz,bytes}), registry.txt list.
|
||
|
||
## Method notes (repro)
|
||
- rtsnap_fast.py: spawn reaper render_long, find host (soothe2 in maps, not reaper),
|
||
sleep 6s (tables built), pread ALL readable maps chunked 8MB -> snap_all.bin.
|
||
- pread of large anon regions can EIO -> MUST chunk (8MB); whole-region pread loses data.
|
||
- Scan ~0.1s for 318MB once chunked; far cheaper than object-base scan.
|
||
- Earlier vptr-based (rtobj/rtdump2/rtall) and 44100-marker scans all failed because
|
||
the DSP object has NO static vptr match in a fresh render (host dies / fields live
|
||
only during audio) and ctor field +0x24 != 44100 live. The registry run is the
|
||
reliable beacon.
|
||
|
||
## Remaining (for twin IIR attack/release per-bin)
|
||
- twin state A/B (342 double per-bin IIR states) still not uniquely located live;
|
||
short renders keep them ~0. They are NOT the registry tables.
|
||
- Next: render_long + capture at t=10-20s into sustain, then locate the per-bin
|
||
attack/release smoothing coefficients (0x540888/88c set, converted via ln(10)/20)
|
||
inside the arena near registry.
|
||
|
||
## 2026-08-19b (two-point snapshots t1=8s, t2=40s of render_long 180s)
|
||
- rtsnap2.py: TWO snapshots of the SAME 180s render at t1/t2; each 318MB/547regs ~0.2s.
|
||
- Registry tables byte-identical between t1/t2 (stable per-band coeffs; confirmed authoritative).
|
||
- Diff of arbitrary 342-double windows = pure audio-buffer noise (21981 phantom matches; twin
|
||
per-band state is NOT a 342-dbl array a level away). Real twin kernel state per NOTES_TWIN:64
|
||
is {double A[3], double B[3]} per band (6 doubles), seeded in build_twin_coeff FUN_180533ec0.
|
||
- Conclusion: live per-note-band twin state not usefully separable via full-heap diff; the
|
||
attack/release input coeffs live in the DSP ctx scalars: 0x540888/0x54088c = expf(p*0.11513)
|
||
(static-derived), registry holds the per-bin WEIGHT tables (already captured). Twin kernels
|
||
themselves validated statically (twin_check max|err|=1.27e-5). => Step C goal (window + axes +
|
||
weights + kernel parity) is effectively CLOSED; only scalar A/R params remain, derived from RPP
|
||
params, no live capture needed.
|
||
|
||
## 2026-08-20 (P1.5: live level-tracker A[] / ctx — CONFIRMED UNREACHABLE)
|
||
- rtctx.py (repo): full pipeline = spawn render_long offline → find yabridge-host → chunked
|
||
snapshot → registry-scan → ctx discovery (marker base+0x24==40000.0f ∧ base+0x540658==window).
|
||
- Two FRESH captures (host live mid-render): registry found (0x28b06c0 GUI / 0x29b06c0 offline,
|
||
[00].cnt=8193 identity, [01]/[03] = window/weights). But:
|
||
- the window ptr (0x2962??? live) appears as a heap value ONLY inside the registry entry
|
||
(0x???6c8 self-field), never as a field of a larger ctx object;
|
||
- bases where +0x24==0x472c4400 (40000.0f) exist only in the plugin image (.data/rodata
|
||
0x180535d62/0x1805607e5/0x181414341) — inline code constants, no live DSP object.
|
||
- => A standalone DSP ctx object with {+0x24==40000, +0x540658==winptr} does NOT exist in the
|
||
heap (the DSP object is the registry itself / its buffers are the registry targets).
|
||
- CONFIRMS the 2026-08-19b conclusion: per-bin level-tracker A[] (0x4c0528/0x3c0510/0x2c04f8)
|
||
and mask scalars are NOT live-separable with the current registry/ctx tools; they are
|
||
STATIC/DERIVED from RPP params (0x540888/88c = 10^(att/20), 0x540870 = expf((p·c4348+c44a4)·0.11513),
|
||
0x54087c = raw band/mix). Two unknown constants remain from the (lost) binary: 0x24c4348, 0x24c44a4.
|
||
- => P1.5 "live capture" is a dead end; scalars must be derived statically or the two missing
|
||
constants recovered from the original soothing_mem.bin (not currently present in workspace).
|
||
|
||
## 2026-08-20c (BREAKTHROUGH: LEVEL-PATH OBJECT captured live in /tmp/snap_rt.bin)
|
||
|
||
The BandConfig A/B/gamma (roadmap gap 2, block of F2/F3) is now LIVE-CAPTURED.
|
||
Read-only scan of the existing realtime snapshot `/tmp/snap_rt.bin` (ctx 0x2370040,
|
||
render_long.rpp) — no new capture needed.
|
||
|
||
### Method (repro, ~2s)
|
||
1. Level-path fingerprint = per-band **level_gain pair buffer**: 0x400 f32 pairs
|
||
(`[level, gain]`), with `level[j] == j/1024` exactly (level[0]==0.0, step 1/1024).
|
||
Vectorized scan (2nd derivative of level slots == 0 + level[0]==0.0) finds them.
|
||
2. Six such buffers at stride 0x2020..0x2040 (band0: 0x4083020, b1: 0x4085040,
|
||
b2: 0x4087080, b3: 0x40890a0, b4: 0x408b0e0, b5: 0x408d100).
|
||
3. Find u64 refs to the six → consecutive slots stride 0x18 at **+0xe0+band*0x18**
|
||
→ object base = **0x3975460** (level-path object).
|
||
|
||
### Level-path object (base 0x3975460, region unknown / heap)
|
||
- `+0x178` = band-list ptr → 0x32c0c60
|
||
- `+0x180` → BandConfig 0x32c0aa0: **A=-24.0, B=+28.0, gamma@0xc=1.0, byte flag@0x10=0** → linear, no callback@0x90
|
||
- `+0x188` → BandConfig 0x32c09c8: A=16.0, B=20000.0, gamma@0xc=1.0, flag=0 (freq-range shaped cfg; +0x18.. floats 0.55,7.13,2.77,2.718 = nonlinear shaper consts)
|
||
- `+0x4198 + band*0x2000` = **band mask doubles**, 512 usable per band:
|
||
band0 ~1.0 const; band1 1.001→1.216 (rising); band2 0.999→0.579 (falling);
|
||
band3 1.291→1.002 (falling); band4/5 1.0→~0.983
|
||
- `+0xe0+band*0x18` → per-band level_gain pair buffers (live LUT output already has
|
||
gains: b0 0.53123 const, b1 0.5314→0.535, b2 const, b3 0.598→, b4 const, ...; many
|
||
bands `~0.531` because mask≈1.0 & render_long default cfg)
|
||
|
||
### Interpretation / next
|
||
- The captured A/B/gamma are the **default render_long config** (A/B semantics =
|
||
level-scaler LUT min/max; rendering default band). To get the A/B/gamma of a SPECIFIC
|
||
band shape (t1kq_only1_1000 etc.) re-run rtctx_rt.py with that test RPP and re-scan
|
||
the same fingerprint (base offset shifts). Method is now automated.
|
||
- This UNBLOCKS the parametric band-LUT 0x563440/0x563a60 as a structural source
|
||
(t=(x−A)/(B−A), clamped, ^gamma, ×norm) instead of fitted Pchip.
|
||
- Dump helper: /tmp/dump_levelpath.py, /tmp/probe_base.py.
|
||
The dead end above was wrong — the missing piece was REALTIME audio playback, not more scanning.
|
||
`-renderproject` uses the OFFLINE audio engine (fields live "only during audio", per earlier note);
|
||
the ctx object only materializes during a realtime transport play.
|
||
|
||
### Method (works)
|
||
- play.lua (repo): `reaper.Main_OnCommand(1007)` (Transport:Play) + hold ~300s.
|
||
- rtctx_rt.py (repo): `reaper render_long.rpp play.lua` → find yabridge-host → chunked snapshot
|
||
(~796MB, 1056 regs) DURING playback → scan heap for the ctx.
|
||
- ctx marker that WORKS live: **+0x24 == 48000.0f (0x473b8000)**, NOT 40000.0f (40000 was the
|
||
static ctor rodata value; live it is the internal SR = 48000, confirming NOTES_CAPTURE SR=48000).
|
||
|
||
### Result (captured live, saved handoff/rtctx_live.json)
|
||
- **ctx = 0x2370040** (region 0x2022000). Field pointers (all point at registry tables):
|
||
0x540688=identity([00] 0x2962580), 0x540698=window([01] 0x14b4240), 0x5406a8=levels([02]),
|
||
0x5406b8=WA([03]), 0x5406c8=WB([04]), 0x5406d8=WC([05]), 0x5406e8=WD([06]),
|
||
0x540748=warp([12] 0x14bc280), 0x540768=LUT-knee([14]). (NOTE: offsets +0x40 from the
|
||
earlier static table — window is 0x540698 live, not 0x540658 as in the f_52b570 disasm label.)
|
||
- **Mask scalars (float)**: 0x540870=440.955 (sens), 0x540874=1.0, 0x540878=1.0, 0x54087c=1.0,
|
||
0x540880=25.0, 0x540884=10.0, 0x540888=1.0 (attack=10^0), 0x54088c=1.0 (release=10^0),
|
||
0x540890=0, 0x540894=1200.0.
|
||
- **level-tracker A[] (341 double each, per-bin IIR attack/release coeffs)**:
|
||
- 0x4c0528 (attack): 0 → 0.340, 0.348, ... monotonic rising, plateau 0.6921 @bin>=319.
|
||
- 0x3c0510 == 0x2c04f8 (release): 0 → 0.000753, 0.000885, ... slow small rise.
|
||
Full arrays in handoff/rtctx_live.json (keys A_4c0528, A_3c0510, A_2c04f8).
|
||
- BandConfig @ctx+0x188 is NOT populated here (zeros) — it lives at a different offset or only
|
||
during band processing; still TBD (but LUT curve params A/B/γ are RPP-derived per roadmap).
|
||
- Two more ctx-like bases found (0x2120040, 0x1780040) also have +0x24==48000; 0x2370040 is the
|
||
populated one (0x2120040 has 0x540658..698 = 1.0 fill pattern — likely a second/free instance).
|
||
|
||
## 2026-08-23 (22y): LIVE CTX CAPTURE DURING DUAL/T1KQ/RES PLAYBACK — scripts/dualtrace.py
|
||
Method works reproducibly: reaper <cfg>.rpp play_loop.lua (repeat ON) → yabridge-host
|
||
→ chunked snapshot ×2 → ctx marker +0x24==48000 ∧ sens>100 → ONLY ONE populated base
|
||
(0x2370040; others empty instances). Snapshots byte-stable over seconds.
|
||
|
||
### Pointer-table catalog (ctx+0x540600..0x540a00, dereferenced u64 → f32[2049])
|
||
- Static weights confirmed live: [00]identity@0x540688(ones), window@0x540698(0.5→0.8),
|
||
WA/WB/WC/WD @0x5406b8/c8/d8/e8, warp@0x540748(1.3→6.68, structure at LOW bins),
|
||
freqaxis@0x540758(v85=995.6Hz ✓ internal 48k/4096).
|
||
- **acc/f6f8 arrays ALL ZERO during steady looped playback** (0x5406f8,
|
||
0x5407a8/b8/c8/d8 — zero as f32 AND f64): combine accumulators idle in steady state.
|
||
- **CONFIG-DEPENDENT CURVE FAMILY** (peak follows band fc: bin43@fc500 → bin85@fc1000):
|
||
- 0x540768 == 0x540778 (identical twins): smooth curve, peak at center
|
||
(dual: 4.15@43, valley 1.60@171, upturn 1.86@400; t1kq: 3.55@85).
|
||
- 0x540788: sharper version (floor ~0.52-1.0, max 4.38).
|
||
- **0x5407f8: min EXACTLY 1.0 → reduction multiplier R(f)=1/mask ≥ 1**
|
||
(res500 cfg: R(500Hz)=12.0 dB, falls to ~0 by 6 kHz; notch-shaped ✓).
|
||
- bands[] slots from static asm (@0x540678+i·16) read as identity/ones tables LIVE —
|
||
the per-band working data is NOT sitting in those ctx fields during playback.
|
||
|
||
### Decisive mismatch
|
||
For dual cfg: R(43)=3.98→12.0 dB (real 10.32 ok-ish) BUT R(171)=1.41→3.0 dB while
|
||
real cut@2000 = 11.82 dB. ⇒ Applied filter ≠ pointwise copy of R: massive spectral
|
||
coupling between template and actual filtering. Prime suspect: FFT-conv stage with
|
||
the 8193-wide WIN_freq window ([01]) — smearing/spreading step completely absent in
|
||
our pointwise render48k path. This ALSO explains why faithful v1 (pointwise) cannot
|
||
balance dual tones regardless of law constants.
|
||
|
||
### Caveats / next
|
||
- Quick Welch TF estimate unreliable (window/alignment) — Goertzel-at-tones stays canon;
|
||
for full-spectrum truth use chirp/two-tone refs or per-fc capture sweep.
|
||
- NEXT: (1) fc-scan captures (res_only1_{fc}.rpp, 11×) → correlate R_cap(bin85) with
|
||
real cut@1000 across fc — validates R as THE applied curve; (2) decode the FFT-conv
|
||
0x535a70 body + WIN_freq usage — reconstruct mask→FIR spreading; (3) re-check whether
|
||
0x540768-family updates frame-by-frame (two-point diff showed stable — maybe only
|
||
rebuilt on param change / note onset).
|