Matiq/soothe2-re
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
59619e4648618e18423736b3be6e9777542c19fa
soothe2-re/handoff/SESSION_HANDOFF.md
T

18 KiB
Raw Blame History

SESSION HANDOFF — soothe2 detector reverse → bit-exact render

Prepared: 2026-08-18 (checkpoint end-of-session: commits c8f97e4 + 60bf3a2 pushed). Start: READ THIS FIRST.

0. DECOMPILATION INVENTORY (2026-08-19 — what's decoded, where, and what's missing)

Goal: bit-exact parity is gated by EXACT tables/window/constants. The chunk-level model hits err ≤0.62 dB (dual) / ≤0.19 dB (al_*) — to go sample-exact we need precise values from the binary.

Static decomp assets IN REPO (use these, don't re-decompile):

  • ghidra-proj/soothe2.rep — full Ghidra project (vst3 at ImageBase 0x180000000).
  • decomp_funs.txt (312K lines, ~1640 functions), fun_map.txt (2285 addr→FUN), consts.txt (11602), decomp_dsp.txt / decomp_vtables.txt / decomp_candidates.txt / focus_decomp.txt. Generators: Dump*.java, ImportRtti*.java, ListFuns.java, SearchRefs.java (+ headless logs).
  • RTTI: rtti_dsp.json / rtti_full.json — class hierarchy SpectralProcessor<float,7,1>, Soothe2ModuleBase<float,1>, FilterGraph<float,6,0x400>, DigitalFilter<float,0xBA,1>, IIRFilterExtended<float,1>, AudioProcessingModule<float,1>.
  • handoff/nls_dasm/ — 120 hand-picked .dis (twin, iface_18052da00/dbc0, fft, generator, ctor).
  • dsp/ — working C++ transcription (twin, detect, freqpath, spectral, fft_stage) + harness; build/twin_check passes float-parity gate (§2).

KEY DSP ADDRESSES — decoded / not decoded:

DECODED (formula-level, notes at NOTES_TWIN.md / NOTES_LEVEL.md):

  • twin kernel FUN_180535880; generator FUN_180533ec0; caller FUN_180536300; dead sibling 180536f90.
  • level-weight formula FUN_180530d30 (0x540880/884, warp 0x5406a8=0.87·x/(1+x/7.942), w=0.1^(...)).
  • mask-apply entry FUN_180529fe0 (accumulator 0x5407c8 += w·res; mask *= warp; FIR *= 0x540658).
  • FFT-conv loop 0x52b550-0x52b8b5 (plan 0x540530, windows 0x540548/550/598, freq-axis 0x540698=offline const).
  • level-path map: 0x563440 (LUT curve +0x188, 6 band-slots, combine→+0x2198), 0x56e3e0 (twin-mask factory), 0x563ce0 (IIR level-tracker INIT only).

NOT DECODED / MISSING FROM decomp_funs.txt (critical):

  • FUN_180529fe0 body — mask/fir apply; only reachable via runtime + vtable slot 180529fe0 in fun_map.txt.
  • FUN_180563440 (LUT curve + gamma + combine), FUN_180563ce0 (IIR level-tracker UPDATE loop). Full disasms exist ONLY in /tmp/opencode/f_563440.dis, f_563ce0.dis, f529fe0.dis — /tmp is ephemeral; COPY INTO handoff/nls_dasm/ on next session start.
  • Window 0x540658 (single indirect write — statically invisible); live-captured copies saved as rwin_A0/A1/B0/C0.npy + r_freqaxis.npy (see NOTES_LEVEL.md §2026-08-18h3). Live data at SR=48000 (offline renders 44100 → renormalize warp x by actual Nyquist).
  • sens source: XML 12.0 → runtime sens_dB≈24.65 (host ×2) not found in dump; IAT\*0x181bab370 outside dump.

Bridge model STATUS (2026-08-19):

  • framed_render.py full STFT frame-render (N=2048, hop=512, sqrt-Hann, twin env tatt=11ms/trel=80ms): C(f)=G·LUT(xv)+W·warp(f)^A, G=1.0850, W=0.2819, A=1.1377 (joint dual+al_* refit).
  • Validation: dual q0.1/1/10 @500+2000 err ≤0.62 dB, envRmse@steady ≤0.78 dB; al_* lv3..24 err ≤0.19 dB.
  • LUT = slanted al_*-leg (0.366@xv=-0.50 → 0.636@xv=+0.55), replaces flat B.12 (~0.5). Bugfix: clip range must be [LY.min(), LY.max()] not [LY[0], LY[-1]].
  • Remaining structural residual: −0.6 dB systematic on dual 500Hz (q0.1/q1). Live evidence (avg per-bin gain H=|Y|/|X|): reference mask is FLAT ~-10.2dB across 100-540Hz regardless of Q — model produces res-shaped notch. Hypo tested: freq-smoothing of C(f) fails (kills 2000Hz). NEXT: scalar per-frame xv (broadband level, not per-bin am/res) — see /tmp/smoothtest.py (+ edit scalar=True).

1. Objective (unchanged since session 1)

Transcribe the decoded soothe2 detector ("twins" 0x180535880/0x180536f90, 2nd-order resonator) into C++ and reach bit-exact render parity with the Reaper reference wavs in /home/m/soothe-bt/*.wav (checking dB/magnitude parity first, then byte-diff).

Path A (bit-exact) chosen. Phase 0,1,2 done. Phase 3,4 advanced. Phase 5 (render diff) is next.

2. DECOMPILATION FINDINGS (2026-08-19 — FUN_180563440, FUN_180563ce0 decoded)

FUN_180563440 — LUT curve evaluation + band combine (222 lines disasm)

Structure: 3 phases per frame:

  1. 1024-bin LUT loop (0x400 iterations): for each bin k:
    • x = clamp(k * 0.0009775, 0, 1.0) = k/1024
    • Band config at +0x188: {A(+0x00), B(+0x04), threshold(+0x0c), flag(+0x10), callback(+0x50)}
    • Path 1 (callback exists): vtable call → dynamic LUT
    • Path 2 (flag=1, threshold≠1.0): power-law → centered = 2*x - 1, then sign(x) * 10^(log10(|x|) / threshold) — this is a compression curve controlled by sharpness/threshold
    • Path 3 (default): linear interpolation → (B - A) * x + A
    • Output: double-precision at +0x198, stride 8
  2. Twin-mask factory (FUN_18056e3e0): 6 bands × 1024 bins, stride 0x2000
  3. Combine loop: stereo (max 2 channels), 6 bands, 1 - sum(band_masks)

FUN_180563ce0 — IIR level-tracker INIT (163 lines disasm)

  • 341 bins (0x155 iterations), order-3 IIR (3 coefficients per bin)
  • Coefficient: 0.1 (0x3dcccccd = IEEE 754 float 0.1)
  • Initial state: [1.0, 0, 0, 0] and [-1.0, 0, 0, 0] (identity + zero)
  • Buffer layout: 3 × (16 bytes coeff) per bin, stored at rcx+0x28/+0x40/+0x58
  • Not the update loop — init only; UPDATE is elsewhere

FUN_180529fe0 — Coefficient setup (2051 instructions, in decomp_funs.txt)

  • vtable method on Soothe2Module<M,1>
  • Lock at +0x2404dc (atomic test-and-set)
  • PRNG state at +0x2404e0, LCG with offset 0x3cdca
  • 6-iteration coefficient generation from 0x5408b0 buffer (LCG-indexed)
  • Depth scaling: powf(normalized, depth) at +0x2c
  • Mask assembly: normalize by 0x1a0 (NFFT), × 0x540870 (level weight), × 0x54088c (sharpness), invert
  • Copy output via SIMD memcpy (thunk 0x181ba94b0)

FUN_18052e9b0 — SpectralProcessor main (3167 instructions, in decomp_funs.txt)

  • Same PRNG + coefficient setup as FUN_180529fe0
  • Band chain: FUN_18052f500 (interleave) → FUN_18052ee70 (per-bin gain) → FUN_18052d650 (setup) → FUN_18052d920 (window)
  • Buffer alloc: FUN_18052e190 for 0x540668, 0x540698, 0x5406a8, 0x5406b8-e8 (6 bands)
  • Depth scaling: powf(normalized, depth) with depth at +0x2c
  • Final mask: 1 - C (inversion)
  • Window application: FIR *= 0x540658 (the live-captured window table)

Constants verified from binary:

Constant Address Value Meaning
SCALE 0x24c3c54 0.000977517 1/1024 (bin→x)
ONE 0x24c3ea4 1.0 clamping max
TWO 0x24c41e0 2.0 centering (2*x-1)
NEG1 0x24c4680 -1.0 sign flip
HALF 0x24c3d8c 0.5 threshold
DEPTH_SCALE 0x24c4334 4.0 depth range
DB_CONV 0x24c43e0 8.6859 20/ln(10)
FLOOR 0x24c4704 -6.9078 ln(0.001)
IIR_COEFF embedded 0.1 attack/release per bin
IIR_ORDER embedded 3.0 IIR filter order
LCG_OFFSET embedded 0x3cdca PRNG state advance

Live-captured tables (re-verified):

  • rwin_A0.npy (0x1930100): 0.5→0.8, frequency window (NOT warp formula)
  • rwin_B0.npy (0x1938180): 0→3.899, power-law depth curve (exponent ~0.66)
  • rwin_C0.npy (0x19401c0): 0.596→0.126, level-dependent weight
  • r_freqaxis.npy: 0→23988.3 Hz, 11.71 Hz spacing (48000/4096)

Key insight: The warp formula is NOT a table — it's computed at runtime

The empirical 0.87*7.942*x/(7.942+x) is an approximation of a runtime computation in FUN_180563440 that evaluates the LUT curve parametrically. The actual LUT has TWO modes:

  • Linear (default): simple interpolation between A and B
  • Power-law (flag=1): sign(x) * 10^(log10(|x|) / C) — compression curve

3. PROOF OF STATE — run this first (5 min)

Everything below must reproduce. If twin_check fails, the transcription moved stale.

cd /home/m/re-tools/dsp && cmake --build build && ./build/twin_check
# expect:
#   max rel err = 1.268e-05, PASS = yes (float-parity)
#   max sweep rel err = 1.233e-05, SWEEP PASS = yes, exit 0

build/soothe2_dsp.so and build/twin_check are current. CMakeLists already has twin.cpp + twin_check executable.

3. TRANSCRIPTION STATE (summary — details in NOTES_TWIN.md)

  • dsp/twin.hpp + dsp/twin.cpp = twin kernel FUN_180535880:
    • build_twin_coeff (FUN_180533ec0 double pipeline → cvtpd2ps floats)
    • generator formula (confirmed): fVar1=sqrtf(param_5); w0=max(2.0,freq)·2π/fs_total; p=(sin(w0)·0.5)/Q; A=[1+dVar5, dVar2, 1−dVar5], B=[1+param_4, dVar2, 1−param_4], dVar2=cos(w0)·−2.0 (DAT_1824c46b0=−2.0).
    • cplx_div_exact (rcpps+Newton: ref=r0·(2−den·r0), NaN-guard cmpeqps+movmskps, numerator B·conj(A))
    • cplx_mul_exact (vfmaddsub213ps form re=fma(br,ar,−bi·ai), im=fma(br,ai,bi·ar))
    • twin_apply: conj(z1) mirroring 0x1800018b0, z2=z1·z1, out=2·B/A
  • OPEN phase question (byte-exactness): conjugate shaping — (-α+π/2) vs (-α) axis. Magnitudes match either way; final proof needed via phase of a single-bin run before claiming byte-exactness.

4. GRID + CALLER (Phase 3 — details in NOTES_TWIN.md §Phase 3)

  • Caller FUN_180536300 (decomp 6218, caller_536300.dis), scratch ≥52·N complex bytes: +0..8N A-accum, +8N..16N B-accum, +16N..24N z-powers, +24N..32N rotor-in (filled 1.0), +32N..36N phase ramp lVar1, +36N..44N rotor-out z1, +44N..52N lVar3(1.0)/lVar4(0.0)
  • 0x18052da00(lVar1, scale=2π/(os·sr), N) = ramp fill; 0x18052dbc0(param_2, lVar4, N) cplx-interleave → out slots get (cos,sin) = e^{+iθ}.
  • Grid law: os=*(int*)(cfg+0x240080)=4 (mov DWORD PTR [rbx+0x240080],0x4 @0x18052ce3e), sr=*(float*)(cfg+0x24)=44100.0 → generator fs_total=os·sr=176400; per-cell phase 2π/176400; cell k ↔ freq k/os Hz (0.25 Hz steps). Phase-1 FS=44100 was a valid normalization (res depends on f/fc).
  • twin always = FUN_180535880 (type-info compare equal); sibling FUN_180536f90 dead.
  • case8: numBands +0x814=1, +0x103=1; param_5=10^(sens_dB/20), gain=sqrtf(param_5)=10^(sens_dB/40) → fit gain=4.132 means stored sens_dB@+0x101 ≈ 24.65; freq=[band+0x804]=1000.0, Q=[band+0x80c].

5. LEVEL-WEIGHT FORMULA (Phase 4 — DECODED, NEGATIVE RESULT — NOTES_LEVEL.md §UPDATE 2026-08-18)

Exact formula from decomp 22330 (FUN_180530d30):

  • base=(2000/(sr·0.5))·iVar5/(bin+1), iVar5=NFFT/2+1
  • w8=powf(base,0.25)
  • v=[0x540880]·0.25·w8·fVar12 (fVar12=4.0 if offline flag [0x5408b8], else 1.0)
  • q=1/(1+v/([0x540880]·4)) (2nd pair 0x540884: q2=1/(1+v2/[0x540884]))
  • dVar1=(sr/[0x1a0])·[0x1ac]·0.001
  • w=0.1^(1/(max(q·v,floor)·dVar1)); bufs: 0x5406b8=w, 0x5406c8=1−w (and 0x5406d8/6e8 w/ 0x540884)
  • consts: c3d3c=0.25, c4334=4.0, c3d8c=0.5, c45b4=2000.0, c3f70=0.1 f32, c3e30=0.001 f64, c3f70=0.1
  • Numerically: w(500)=w(1000)=w(2000)≈0 (1e-20..1e-0.07), complements≈1 → but the weights are STILL APPLIED (as 1−w ≈ 1) by FUN_180529fe0 (0x5407c8[band] += 0x5406c8·res_upper + 0x5406e8·res_lower, thunk 003c40, kernel 0x1800752e0 = vfmadd213pd) — the form is weight·resonance, not dead code.
  • TILT analysis (2026-08-18, checkpoint NEGATIVE result): warp 0x5406a8 = 0.87·x/(1+x/K), K=exp(2.0723267)≈7.942, x=f/2000, applied by FUN_180529fe0 as mask *= 0x5406a8 (float, thunk 008700) BEFORE the FFT-conv. warp: 500→0.211, 1000→0.409, 2000→0.773 (lin ratio 2000/500=3.67). BUT warp ≠ empirical tilt (ratio 1.27): C(dB2000)=depth·warp(2000)·LUT_max = 0.864·0.772·0.667=0.445, yet red2000=15.2 dB needs C=0.826 ⇒ warp alone CANNOT produce deep-2000 (dual_b1q). Empirical tilt is the envelope of the BLEND term (0x5407c8 += weights·res + freq-axis 0x540698) + FFT-conv shaping, NOT warp — simple tilt→warp substitution in model_lut.py is INVALID. Nothing else decoded yet closes dual_b1q.
  • FUN_180563ce0 decoded (2026-08-18): INIT of per-bin IIR level-trackers (0x156=342 bins, order=3.0f 0x40400000, coeff A/B from 0x24c4780/4790, strided fill; third pass +0xffd0 fills 0x40e00000=7.0f). Not the update loop. Full disasm: /tmp/opencode/f_563ce0.dis.
  • Level-path map (2026-08-18, NOTES_LEVEL.md): 0x563440 (LUT curve +0x188, gamma, 6 band-slots, combine →+0x2198, LUT pairs +0x98 at 0x3ff pts) + 0x56e3e0 twin-mask factory (N=1024, DOUBLE, LUT-ramp input) + 0x563ce0 IIR init. 0x563440 NOT in decomp_funs.txt (search via /tmp/opencode/f_563440.dis + level_notes.md).
  • DIAGNOSTIC BRIDGE DONE (2026-08-18) — model_fir.py: реальная цепочка из FUN_180529fe0 воспроизводит 36 точек до rmse=0.236 dB БЕЗ эмпирического tilt: C(f) = g·LUT(log10(L0/res(f))) + w·warp(f)^a, g=1.221, w=0.358, a=3.143.
    • dual500-константа = res_band(500;fc=500)=0.117 Q-НЕЗАВИСИМ, не tilt;
    • dual2000-глубина = АДДИТИВНЫЙ терм w·warp^3.14 (на 1000 Гц вклад ≤0.021 — иначе t1k рушится; на 2000 = 0.149) — это аккумулятор 0x5407c8 += weights·res + mask (шаг 5), НЕ мультипликация warp·LUT (та проваливается >10 dB);
    • warp^~3.14 ≈ π — подозрение на кратный каскад (0x540698 freq-axis, ∏0x540688, двойной FFT);
    • остаток 0.7 dB при Q=0.1 = форма LUT-колена 0.574, не закрывается per-bin → нужен FFT-уровень.
  • FFT-CONV FULLY MAPPED (2026-08-18, NOTES_LEVEL.md §2026-08-18c, /tmp/opencode/f_52b570.dis): 0x535a70 = dispatcher to base-CRT FFT (0x140a10/0x140a70); 0x540548/0x540550/0x540598 = FFT-plan tables (plan 0x540530, built by 0x52dc30), NOT windows. Per-band loop 0x52b550-0x52b8b5: mask→fwdFFT→invFFT→fill low/high (xmm13/xmm9 consts)→inv→cplx-op→fwd→FIR *= 0x540658→fill→inv→ FIR[0]=1,FIR[1]=0→*=0x540888→copy out. freq-axis 0x540698: OFFLINE = constant scalar (NOT exp-formula — exp/rand = online dither). Window 0x540658: only ONE direct ref in dump (read at 0x52b771), written only indirectly — STATICALLY INVISIBLE → the only plausible source of warp^3.14/LUT-knee. Need runtime capture or full-pipeline empirical closure.

6. PHASE 5 PLAN (updated 2026-08-18 — bridge done, tilt replaced, FFT mined out)

Decision (user): bridge FIRST. DONE — bridge result above in §5. Status: step 2 DONE, step 3 PARTIAL (static analysis exhausted).

  1. (15 min) Proof-of-state above; confirm twin + mdl files complete; repo at 60bf3a2.
  2. Replace empirical tilt in model_lut.py with g·LUT(xv) + w·warp(f)^a — DONE, rmse 0.236 dB reproduced, committed c8f97e4. Optional refinement fit (2 LUT-knots) reaches 0.167 but is overfit.
  3. Close the 0.7-dB gap: FFT-conv loop mapped + freq-axis cleared (offline const) — DONE/PARTIAL, committed 60bf3a2. Remaining unknown = 0x540658 window content (static-invisible). Options: (a) runtime capture of 0x540658, (b) skip and close empirically in full-pipeline step 5.
  4. Runtime values: per-bin IIR level trackers (0x563ce0), 0x540880/884, [0x1a0]/[0x1ac], cell count N.
  5. Full pipeline + render diff vs /home/m/soothe-bt/*.wav: dB parity → byte parity.

NEXT SESSION START (recommendation): skip deeper FFT mining; go straight to step 5 full pipeline (mask-chain already in model_lut.py B.12 at 0.236; freq-axis const; FFT-conv reduces to a per-bin smoothing + 0x540658 shaping that can be fit/absorbed). Runtime capture of 0x540658 only if step 5 stalls on the Q=0.1 knee.

STEP 5 STARTED (2026-08-18) — dB-parity harness DONE (commit f96bb42)

  • /home/m/re-tools/render_parity.py: measures steady-state per-tone reduction DIRECTLY on the reference wavs (Goertzel tone-amp over late steady window; handles 16/24-bit, mono/stereo). NOTE: Goertzel must use the SEQUENTIAL update form (s2=s1;s1=s0;s0=v+cw*s1-s2) — the tuple-unpack form is numerically unstable (overflows ~1500 steps).
  • Validation: measured dual Q-scan == stored dataset (dual.wav is 4s mono 16-bit; renders 6s stereo 24-bit).
  • dB-PARITY RESULT (canonical B.12, 36 pts, real wavs): TOTAL rmse=0.268 dB (dual500 0.130, dual2000 0.387, t1kq 0.320, t1k 0.076). residual structure: dual2000 -0.80@Q0.1..+0.36@Q10 (LUT-knee), t1kq +0.36..+0.26 (level region), t1k ±0.1 (good). Refit (g,w,a)=(1.224,0.379,3.405)→0.264; +2 LUT-knots→0.203 (plateau).
  • CONCLUSION: B.12 form plateau ~0.20-0.27 dB vs real renders; residual is structural (0x540658 FFT shaping + level-curve region), matches plan expectation. Re-measured-on-wav numbers differ slightly from the stored model_lut.py dataset (t1kq ~+0.3 dB) — different steady-window method.
  • Open: full PIPELINE frame-render (STFT + per-frame mask + time constants) is the remaining big lift for byte-parity; dB-parity milestone achieved at 0.268 dB.

Candidates that may consume Phase 5 time: source of sens≈24.65 (host ×2 over XML 12.0 — not found in dump), IAT*0x181bab370 outside dump (non-blocking), final z¹ phase proof, the 0x563440 LUT curve (+0x188) values.

7. FILES LAYOUT

  • /home/m/re-tools/dsp/ — twin.{hpp,cpp}, twin_check.cpp, CMakeLists.txt, build/
  • /home/m/re-tools/model_lut.py — model B.12 (Q=xmlq, gain=10^(sens/20), depth 0.8639736175537109, C=g·LUT(xv)+w·warp^a, g=1.221/w=0.358/a=3.143) — 0.236 dB bridge model; PCHIP LUT nodes frozen
  • /home/m/re-tools/model_fir.py — bridge canonical source (rmse 0.236, committed b1066f3)
  • /home/m/re-tools/framed_render.py — full frame-render pilot (2026-08-19 params; modes dual|al)
  • /home/m/re-tools/rwin_A0/A1/B0/C0.npy, r_freqaxis.npy, rwin_warp.npy — live-window tables (48k)
  • /home/m/re-tools/rtsnap.py — live-process page snapshotter; snapshots /tmp/rt{A,B,C,D}.{raw,idx}
  • /home/m/re-tools/handoff/decode_rpp_full.py, handoff/rpp_allparams.py — RPP b64-XML full decoder (trim to len%4==0, <?xml@92, incl. nested processorStateData)
  • /home/m/re-tools/soothe_mem.bin — memory dump, offset=VA−0x180000000
  • /home/m/re-tools/decomp_funs.txt — decomp: FUN_180536300 (6218), FUN_180530d30 (22330), FUN_180533ec0 (123204), FUN_180529fe0 (missing body! vtable-only)
  • /home/m/re-tools/handoff/ — THIS DOC, NOTES_TWIN.md, NOTES_LEVEL.md, phase1/.py, nls_dasm/.dis
  • /home/m/soothe-bt/*.rpp (decoder /home/m/re-tools/dsp/...; also decode_rpp4.py referenced) + *.wav refs
Reference in New Issue View Git Blame Copy Permalink