soothe2-re/prd.md

prd.md — Project Requirements Document

1. Project Overview

Name: soothe2-re — bit-exact reverse engineering of oeksound soothe2 (VST3, Windows x64)

Goal: Reproduce the plugin's DSP core (level detector, mask computation, filter application) in C++17, byte-for-byte identical to the native binary.

Status: ~95% decompiled. Bridge 1.594 dB stable; structural 2.689 dB (L/R e343b0a, RT_VLAW=1 RT_SYN=1 RT_NOWARP=1 RT_NOIIR3=1 RT_IIR12=0, RT_CASC=0 gated). Historical best 0.341 (24mm14) not reachable on current HEAD. Target: <0.05 dB (bit-exact gate, requires chain calibration).

Golden rule: Every parameter must have a source (decomp address / live table). Empirical fits must be flagged EMPIRICAL.

---

2. Repository Structure

re-tools/
├── prd.md                    ← THIS FILE (project overview)
├── README.md                 ← TOTAL metric source (single source of truth)
├── AGENTS.md                 ← runbook: build, env flags, tooling hazard
├── BITEXACT_PLAN.md          ← path to byte-exact (3 steps, criteria)
├── roadmap.md                ← historical B-phase log (archived)
│
├── dsp/                      ← C++17 DSP reconstruction (THE CANON)
│   ├── framed_model.{cpp,hpp} ← MAIN: mask-apply chain (empirical bridge)
│   ├── render48k.cpp          ← 48k/4096 structural pipeline (resample→chain→resample)
│   ├── spectral.{cpp,hpp}     ← STFT/ISTFT processor + FIR builder
│   ├── fn529fe0.{cpp,hpp}     ← structural chain 9–19 (FUN_180529fe0)
│   ├── fnfaith.{cpp,hpp}      ← faithful detector cascade (BLOCKMAP transcription)
│   ├── fft{,_plan,_stage}.cpp/hpp ← FFT engine (canonical + RFFT bit-exact)
│   ├── twin.{cpp,hpp}         ← twin resonator (FUN_180535880, float-parity)
│   ├── rt_mask_tables.{hpp,.cpp} ← live IIR A/B coefficients (385K)
│   ├── rt_weights.{hpp,.cpp}  ← live kWarp/kBand768 tables (44K)
│   ├── fftconv.{cpp,hpp}      ← FFT convolution FIR application
│   ├── vlog.{cpp,hpp}         ← fast log2 approximation
│   ├── exp2.{cpp,_tables.cpp,hpp} ← fast exp2 approximation
│   ├── levelpath.{cpp,hpp}    ← level-curve path (xv = log10(am/res))
│   ├── freqpath.{cpp,hpp}     ← frequency-axis warp/freq-domain helpers
│   ├── leveltrack.{cpp,hpp,hpp} ← envelope follower (attack/release tables)
│   ├── log2_ln.{cpp,hpp}      ← log/ln utilities
│   ├── filter.{cpp,hpp}       ← detection kernel (legacy)
│   ├── detect.{cpp,hpp}       ← detector front-end (legacy)
│   ├── phase_table.{cpp,hpp}  ← phase table for FFT
│   ├── ms.hpp                 ← mid-side helpers
│   ├── params.hpp             ← parameter struct
│   ├── rotor_kernel.hpp       ← rotor transform kernel
│   ├── rt_div_tables.hpp      ← division tables
│   ├── soothe_constants.hpp   ← decoded constants
│   ├── cody_waite.hpp         ← Cody-Waite argument reduction
│   ├── twiddle_{builder,loader}.cpp/hpp ← FFT twiddle factors
│   ├── dsp_ctx.hpp            ← DSP context layout
│   ├── framed_test.cpp        ← CLI bridge renderer (44.1k)
│   ├── harness.cpp           ← legacy harness
│   ├── *_check.cpp            ← bit-exact unit test targets
│   └── CMakeLists.txt
│
├── scripts/
│   ├── corpus.py              ← bridge regression harness (62 cases, --compare)
│   ├── corpus_structural.py   ← structural chain harness (--vs-bridge)
│   ├── rendersnap2.py         ← RENDER_FILE stopper (reads from .rpp)
│   ├── campaign.py            ← parameter sweep cell (~8 min)
│   ├── cascade_sim.py         ← numpy step 9–19 simulator
│   ├── disasm_func.py         ← capstone disasm with RIP constants
│   ├── iat_name.py            ← runtime import resolution
│   ├── wine_{chain,stage,ptrace}_trace.py ← ptrace-based live trace
│   ├── dump_dispatch.py       ← table/state dumper
│   ├── probe_{states,mem}.py  ← live state/memory probes
│   ├── hunt2.py               ← ctx-instance hunter
│   ├── scan_{pairs,lutsub}.py ← memory diagnostics
│   ├── fit_vlaw_{params,by_group}.py ← VLAW law calibration
│   ├── lawfit22r.py           ← law fitting (VLAW α/β/c)
│   ├── mk_{dist,far,multi6}.py ← RPP generators
│   └── render_parity.py       ← model-vs-render comparison
│
├── handoff/
│   ├── NOTES_LEVEL.md         ← live journal (head: 24mm5+)
│   ├── NOTES_LEVEL_INDEX.md   ← journal index by date/topic
│   ├── BLOCKMAP_529fe0.md     ← FUN_180529fe0 method map (53K)
│   ├── NOTES_TWIN.md          ← twin reference
│   ├── NOTES_CAPTURE.md       ← live-capture protocol
│   ├── NEXT_PROMPT.md         ← entry point for new sessions
│   ├── SESSION_HANDOFF.md     ← handoff template
│   ├── archive/               ← historical NOTES_LEVEL_*.md, SESSION_HANDOFF_*.md
│   ├── nls_dasm/              ← 183 disassembly files (.dis, .bin)
│   ├── phase1/                ← phase-1 outputs
│   ├── rt*.npy                ← captured runtime tables (48k/44.1k)
│   └── *.py                   ← emit/extract/joint scripts
│
├── *.java                     ← Ghidra scripts (DumpFuns, ImportRtti…)
├── *.{bin,npz,npy,json,txt}   ← datasets, dumps, LUTs (mostly outside git)
└── soothe-bt/                 ← test corpus (~600 renders, outside git)

---

3. Build System

Generator: CMake 3.10+, C++17, GCC/Clang with -O3 -march=native.

# Key targets
soothe2_dsp          # shared library (all dsp/*.cpp)
framed_test          # CLI bridge renderer (44.1k)
render48k            # structural renderer (48k/48000)
twin_check           # float-parity unit test
tables_check         # live-table verification
fftconv_check        # FIR convolution check
vlog_check           # fast log2 check
leveltrack_check     # envelope follower check
levelpath_check      # level-curve path check
exp2_check           # fast exp2 check
fn529fe0_check       # structural chain check
soothe2_harness      # legacy harness

External deps: libsamplerate (render48k only), pthread.

Tooling hazard: CMake skips rebuild when source modified within same second. Protocol: touch source before build + verify binary mtime.

---

4. DSP Architecture

4.1 Signal Flow (current canon: structural render48k)

Host 44.1k → resample → 48k → [per-channel processing] → resample → 44.1k Host
                                              ↓
                                    ┌─────────────────────┐
                                    │  FramedDetector      │
                                    │  (per 4096 block)     │
                                    │                      │
                                    │  am[] ← envelope     │
                                    │  res[] ← twin resp   │
                                    │         ↓            │
                                    │  lvl_raw = am/res·sf │
                                    │         ↓            │
                                    │  [RT_VLAW=1]:        │
                                    │    cut = α·ln1p(L/β) │
                                    │         +c [+Δ]      │
                                    │    mask = 10^(-cut/20)│
                                    │         ↓            │
                                    │  warp: mask *=       │
                                    │    kBand·kWarp·res^rp│
                                    │         ↓            │
                                    │  IIR3 ×2 (bidir)     │
                                    │         ↓            │
                                    │  mask_out → multiply │
                                    │  spectrum[k] *= mask │
                                    └─────────────────────┘

4.2 Dual Render Path

Path	File	Grid	Use
Bridge	framed_model.cpp	44.1k/2048	Legacy, TOTAL 1.594 (canon)
Structural	render48k.cpp	48k/4096 L/R	2.689 (RT_CASC=0 gated, hist. 0.341)

Dual-solution env set: RT_VLAW=1 RT_SYN=1 RT_NOWARP=1 RT_NOIIR3=1 RT_IIR12=0 (chain RT_CASC=1 adds IIR4 double + FIR min-phase, gated)

4.3 Key Modules

Module	Responsibility
FramedDetector	Per-band, per-frame mask computation. Holds am_, res_, track_ state.
SpectralProcessor	STFT/ISTFT, OLA overlap-add, FIR application modes.
fn529fe0	Structural chain 9–19 (steps 9–19 of FUN_180529fe0): scale→IIR1→IIR2→blend→combine→warp→IIR3→dry/wet.
fnfaith	Faithful detector cascade transcription (BLOCKMAP).
twin	Twin resonator `
rt_mask_tables	Live IIR A/B coefficients (kIIR_A1/A2/A3, kIIR_B1/B2/B3, kRTAtt, kRTRel).
rt_weights	Live warp weights (kBand768, kWarp).
fft	Bit-exact RFFT (th1a90/th2180) with buf548/mask598 tables.
fftconv	FFT-based convolution for FIR application modes.
vlog/exp2	Fast polynomial approximations matching plugin精度.
leveltrack	Envelope follower with per-bin attack/release tables.

4.4 Detector Cascade (529c60)

complex_spectrum × twin_response → |z|
  → Haar smooth [0.25,0.5,0.25] × n_iters
  → peak = max(curve)
  → sin_peak = sin(param·30 − 90) · 0.115129 · peak
  → curve = max(curve, sin_peak)
  → w = -log10(pow(50, ratio·0.001) · ratio·0.001)
  → acc = acc·w + curve·(1-w)
  → bands_curve = acc

---

5. Environment Flags (experiment control)

Flag	Effect
RT_VLAW=1	Two-stage law: mask = 10^(−(α·ln1p(lvl/β)+c)/20)
RT_SYN=1	STFT without synthesis window (plugin's actual layer)
RT_WIN=0/1/2	Analysis window: sym-Hann / periodic / rect
RT_NOWARP=1	Skip warp modulation
RT_NOIIR3=1	Skip IIR3 ×2
RT_IIR12=0	Skip freq-domain IIR1/2 (critical with VLAW)
RT_DUMP_BIN=<f>	Dump tract binary (frame RT_DUMP_FRAME)
RT_VDBG=1	Print VLAW computations to stderr
RT_FAITHFUL=1	Use faithful chain (fnfaith.cpp)
RT_FIRCONV=1/3	FIR application mode (1=complex-mul, 3=1.019·mask^1.8345)
RT_ENV=live	Live envelope from kRTAtt/kRTRel tables
RT_KMAP=1	k-mapping correction (twin/am scaling)
RT_EQ=1	Pre-detector EQ bell
RT_LUT_OFF=1	Skip LUT transform
RT_LUT_A/B/G/M	LUT parameters (A=-24, B=28, gamma=1, mult=4.2)

---

6. Test Corpus & Metrics

Location: /home/m/soothe-bt/ (~600 renders, outside git).

Key sets:

Prefix	Content
tone1kq_*	Single tone, fc-scan, quiet (-18 dBFS)
tone1k_*	Single tone, fc-scan, loud (0 dBFS)
dual_b1q_*	Two tones (500+2000), q 0.1…10
al_*	Level sweep (fc=1000)
comb_*	4-band multiband
burst500_b1	Primary reference (burst 500 Hz)

Metric (Goertzel steady-state):

err_dB = 20·log1₀( ta(out,1000) / ta(ref,1000) )
# trimmed to last 75% of input, matched to plugin render length

Regression guards:

python3 scripts/corpus.py                                    # bridge + guard
python3 scripts/corpus_structural.py                         # structural chain
python3 scripts/corpus.py --compare scripts/baseline_bridge.json --tol 0.25
python3 scripts/corpus_structural.py --vs-bridge scripts/baseline_bridge.json

---

7. Current Status (2026-09-02, chain919)

Group	Bridge	Structural RT_CASC=0 (L/R)	Structural RT_CASC=1
t1kq (fc-scan)	0.226	0.356	0.717
t1k (loud)	1.801	5.437	5.631
al (level)	0.638	2.613	5.036
res	0.628	1.256	23.031
dual (q-sweep)	0.726	2.042	66.642
comb (4-band)	10.149	6.004	29.172
TOTAL	1.594	2.689	31.653

Historical best structural 0.341 (24mm14, f40f41e dual 0.193) not reachable on current HEAD (M/S→L/R + chain).

Decoded:

• Layer: STFT without synthesis window, per-bin mask multiply
• Law: mask = 10^(−(α·ln1p(lvl/β)+c)/20) with content-aware Δ
• FIR: exp(0.984·ln(raw)) + Hann + normalize → bit-exact RFFT to df0
• Chain 9–19: IIR4 double + FIR min-phase integrated, generate_iir4_coefs() (FUN_180533340) added, gated RT_CASC

Open gaps:

1. Chain 9–19 (priority #1): Integrated but uncalibrated (input format level vs cut, blocker ph*.npz/live-dump). Target: <0.05.
2. k-mapping (priority #2): twin/am scaling k(q≥2)=0.403.
3. Δ second-peak rule (priority #3): Content-dependent gain, requires live-dump.

---

8. Reference Documents

Doc	Content
README.md	TOTAL metric, reproduction steps
AGENTS.md	Build commands, env flags, hazard, corpus format
BITEXACT_PLAN.md	3-step plan to byte-exact
handoff/BLOCKMAP_529fe0.md	Method map for FUN_180529fe0
handoff/NOTES_LEVEL.md	Live working journal
handoff/NOTES_LEVEL_INDEX.md	Journal index
handoff/NOTES_TWIN.md	Twin resonator reference
handoff/NOTES_CAPTURE.md	Live-capture protocol
handoff/nls_dasm/	183 disassembly files

---

9. Reproduction & Development Protocol

1. Install soothe2 VST3 (Windows) under yabridge → dump_soothe.py → soothe_mem.bin
2. Ghidra headless: analyzeHeadless <proj> soothe_x64 -process soothe_mem.bin -noanalysis -postScript <X>.java
3. Renders: sweep.py → .rpp, reaper -renderproject → .wav
4. Analysis: render_parity.py / corpus.py
5. After C++ edit: touch source → cmake --build dsp/build --target framed_test → run corpus guards