prd.md: project requirements document. render48k L/R stereo baseline

- Add prd.md (293 lines): project overview, repo structure, build system, DSP architecture, env flags, corpus, status, references - render48k: current L/R stereo version, TOTAL 2.835 (requires parameter tuning vs canonical 0.341 VLAW dual-solution)
2026-09-02 17:42:51 +03:00
parent e9d6f2ca72
commit e343b0a0d0
2 changed files with 346 additions and 88 deletions
@@ -0,0 +1,293 @@
+# prd.md — Project Requirements Document
+
+## 1. Project Overview
+
+**Name**: `soothe2-re` — bit-exact reverse engineering of **oeksound soothe2** (VST3, Windows x64)
+
+**Goal**: Reproduce the plugin's DSP core (level detector, mask computation, filter application) in C++17, byte-for-byte identical to the native binary.
+
+**Status**: ~95% decompiled. Current best metric: **0.341 dB TOTAL** (structural 48k chain). Target: **<0.05 dB** (bit-exact gate).
+
+**Golden rule**: Every parameter must have a source (decomp address / live table). Empirical fits must be flagged `EMPIRICAL`.
+
+---
+
+## 2. Repository Structure
+
+```
+re-tools/
+├── prd.md                    ← THIS FILE (project overview)
+├── README.md                 ← TOTAL metric source (single source of truth)
+├── AGENTS.md                 ← runbook: build, env flags, tooling hazard
+├── BITEXACT_PLAN.md          ← path to byte-exact (3 steps, criteria)
+├── roadmap.md                ← historical B-phase log (archived)
+│
+├── dsp/                      ← C++17 DSP reconstruction (THE CANON)
+│   ├── framed_model.{cpp,hpp} ← MAIN: mask-apply chain (empirical bridge)
+│   ├── render48k.cpp          ← 48k/4096 structural pipeline (resample→chain→resample)
+│   ├── spectral.{cpp,hpp}     ← STFT/ISTFT processor + FIR builder
+│   ├── fn529fe0.{cpp,hpp}     ← structural chain 9–19 (FUN_180529fe0)
+│   ├── fnfaith.{cpp,hpp}      ← faithful detector cascade (BLOCKMAP transcription)
+│   ├── fft{,_plan,_stage}.cpp/hpp ← FFT engine (canonical + RFFT bit-exact)
+│   ├── twin.{cpp,hpp}         ← twin resonator (FUN_180535880, float-parity)
+│   ├── rt_mask_tables.{hpp,.cpp} ← live IIR A/B coefficients (385K)
+│   ├── rt_weights.{hpp,.cpp}  ← live kWarp/kBand768 tables (44K)
+│   ├── fftconv.{cpp,hpp}      ← FFT convolution FIR application
+│   ├── vlog.{cpp,hpp}         ← fast log2 approximation
+│   ├── exp2.{cpp,_tables.cpp,hpp} ← fast exp2 approximation
+│   ├── levelpath.{cpp,hpp}    ← level-curve path (xv = log10(am/res))
+│   ├── freqpath.{cpp,hpp}     ← frequency-axis warp/freq-domain helpers
+│   ├── leveltrack.{cpp,hpp,hpp} ← envelope follower (attack/release tables)
+│   ├── log2_ln.{cpp,hpp}      ← log/ln utilities
+│   ├── filter.{cpp,hpp}       ← detection kernel (legacy)
+│   ├── detect.{cpp,hpp}       ← detector front-end (legacy)
+│   ├── phase_table.{cpp,hpp}  ← phase table for FFT
+│   ├── ms.hpp                 ← mid-side helpers
+│   ├── params.hpp             ← parameter struct
+│   ├── rotor_kernel.hpp       ← rotor transform kernel
+│   ├── rt_div_tables.hpp      ← division tables
+│   ├── soothe_constants.hpp   ← decoded constants
+│   ├── cody_waite.hpp         ← Cody-Waite argument reduction
+│   ├── twiddle_{builder,loader}.cpp/hpp ← FFT twiddle factors
+│   ├── dsp_ctx.hpp            ← DSP context layout
+│   ├── framed_test.cpp        ← CLI bridge renderer (44.1k)
+│   ├── harness.cpp           ← legacy harness
+│   ├── *_check.cpp            ← bit-exact unit test targets
+│   └── CMakeLists.txt
+│
+├── scripts/
+│   ├── corpus.py              ← bridge regression harness (62 cases, --compare)
+│   ├── corpus_structural.py   ← structural chain harness (--vs-bridge)
+│   ├── rendersnap2.py         ← RENDER_FILE stopper (reads from .rpp)
+│   ├── campaign.py            ← parameter sweep cell (~8 min)
+│   ├── cascade_sim.py         ← numpy step 9–19 simulator
+│   ├── disasm_func.py         ← capstone disasm with RIP constants
+│   ├── iat_name.py            ← runtime import resolution
+│   ├── wine_{chain,stage,ptrace}_trace.py ← ptrace-based live trace
+│   ├── dump_dispatch.py       ← table/state dumper
+│   ├── probe_{states,mem}.py  ← live state/memory probes
+│   ├── hunt2.py               ← ctx-instance hunter
+│   ├── scan_{pairs,lutsub}.py ← memory diagnostics
+│   ├── fit_vlaw_{params,by_group}.py ← VLAW law calibration
+│   ├── lawfit22r.py           ← law fitting (VLAW α/β/c)
+│   ├── mk_{dist,far,multi6}.py ← RPP generators
+│   └── render_parity.py       ← model-vs-render comparison
+│
+├── handoff/
+│   ├── NOTES_LEVEL.md         ← live journal (head: 24mm5+)
+│   ├── NOTES_LEVEL_INDEX.md   ← journal index by date/topic
+│   ├── BLOCKMAP_529fe0.md     ← FUN_180529fe0 method map (53K)
+│   ├── NOTES_TWIN.md          ← twin reference
+│   ├── NOTES_CAPTURE.md       ← live-capture protocol
+│   ├── NEXT_PROMPT.md         ← entry point for new sessions
+│   ├── SESSION_HANDOFF.md     ← handoff template
+│   ├── archive/               ← historical NOTES_LEVEL_*.md, SESSION_HANDOFF_*.md
+│   ├── nls_dasm/              ← 183 disassembly files (.dis, .bin)
+│   ├── phase1/                ← phase-1 outputs
+│   ├── rt*.npy                ← captured runtime tables (48k/44.1k)
+│   └── *.py                   ← emit/extract/joint scripts
+│
+├── *.java                     ← Ghidra scripts (DumpFuns, ImportRtti…)
+├── *.{bin,npz,npy,json,txt}   ← datasets, dumps, LUTs (mostly outside git)
+└── soothe-bt/                 ← test corpus (~600 renders, outside git)
+```
+
+---
+
+## 3. Build System
+
+**Generator**: CMake 3.10+, C++17, GCC/Clang with `-O3 -march=native`.
+
+```cmake
+# Key targets
+soothe2_dsp          # shared library (all dsp/*.cpp)
+framed_test          # CLI bridge renderer (44.1k)
+render48k            # structural renderer (48k/48000)
+twin_check           # float-parity unit test
+tables_check         # live-table verification
+fftconv_check        # FIR convolution check
+vlog_check           # fast log2 check
+leveltrack_check     # envelope follower check
+levelpath_check      # level-curve path check
+exp2_check           # fast exp2 check
+fn529fe0_check       # structural chain check
+soothe2_harness      # legacy harness
+```
+
+**External deps**: `libsamplerate` (render48k only), `pthread`.
+
+**Tooling hazard**: CMake skips rebuild when source modified within same second. Protocol: `touch` source before build + verify binary mtime.
+
+---
+
+## 4. DSP Architecture
+
+### 4.1 Signal Flow (current canon: structural `render48k`)
+
+```
+Host 44.1k → resample → 48k → [per-channel processing] → resample → 44.1k Host
+                                              ↓
+                                    ┌─────────────────────┐
+                                    │  FramedDetector      │
+                                    │  (per 4096 block)     │
+                                    │                      │
+                                    │  am[] ← envelope     │
+                                    │  res[] ← twin resp   │
+                                    │         ↓            │
+                                    │  lvl_raw = am/res·sf │
+                                    │         ↓            │
+                                    │  [RT_VLAW=1]:        │
+                                    │    cut = α·ln1p(L/β) │
+                                    │         +c [+Δ]      │
+                                    │    mask = 10^(-cut/20)│
+                                    │         ↓            │
+                                    │  warp: mask *=       │
+                                    │    kBand·kWarp·res^rp│
+                                    │         ↓            │
+                                    │  IIR3 ×2 (bidir)     │
+                                    │         ↓            │
+                                    │  mask_out → multiply │
+                                    │  spectrum[k] *= mask │
+                                    └─────────────────────┘
+```
+
+### 4.2 Dual Render Path
+
+| Path | File | Grid | Use |
+|------|------|------|-----|
+| Bridge | `framed_model.cpp` | 44.1k/2048 | Legacy, TOTAL 1.594 |
+| Structural | `render48k.cpp` | 48k/4096 | Canon, TOTAL 0.341 |
+
+**Dual-solution env set**: `RT_VLAW=1 RT_SYN=1 RT_NOWARP=1 RT_NOIIR3=1 RT_IIR12=0`
+
+### 4.3 Key Modules
+
+| Module | Responsibility |
+|--------|---------------|
+| `FramedDetector` | Per-band, per-frame mask computation. Holds `am_`, `res_`, `track_` state. |
+| `SpectralProcessor` | STFT/ISTFT, OLA overlap-add, FIR application modes. |
+| `fn529fe0` | Structural chain 9–19 (steps 9–19 of FUN_180529fe0): scale→IIR1→IIR2→blend→combine→warp→IIR3→dry/wet. |
+| `fnfaith` | Faithful detector cascade transcription (BLOCKMAP). |
+| `twin` | Twin resonator `|2B/A|` — frequency response per band. |
+| `rt_mask_tables` | Live IIR A/B coefficients (kIIR_A1/A2/A3, kIIR_B1/B2/B3, kRTAtt, kRTRel). |
+| `rt_weights` | Live warp weights (kBand768, kWarp). |
+| `fft` | Bit-exact RFFT (th1a90/th2180) with `buf548`/`mask598` tables. |
+| `fftconv` | FFT-based convolution for FIR application modes. |
+| `vlog`/`exp2` | Fast polynomial approximations matching plugin精度. |
+| `leveltrack` | Envelope follower with per-bin attack/release tables. |
+
+### 4.4 Detector Cascade (529c60)
+
+```
+complex_spectrum × twin_response → |z|
+  → Haar smooth [0.25,0.5,0.25] × n_iters
+  → peak = max(curve)
+  → sin_peak = sin(param·30 − 90) · 0.115129 · peak
+  → curve = max(curve, sin_peak)
+  → w = -log10(pow(50, ratio·0.001) · ratio·0.001)
+  → acc = acc·w + curve·(1-w)
+  → bands_curve = acc
+```
+
+---
+
+## 5. Environment Flags (experiment control)
+
+| Flag | Effect |
+|------|--------|
+| `RT_VLAW=1` | Two-stage law: `mask = 10^(−(α·ln1p(lvl/β)+c)/20)` |
+| `RT_SYN=1` | STFT without synthesis window (plugin's actual layer) |
+| `RT_WIN=0/1/2` | Analysis window: sym-Hann / periodic / rect |
+| `RT_NOWARP=1` | Skip warp modulation |
+| `RT_NOIIR3=1` | Skip IIR3 ×2 |
+| `RT_IIR12=0` | Skip freq-domain IIR1/2 (critical with VLAW) |
+| `RT_DUMP_BIN=<f>` | Dump tract binary (frame `RT_DUMP_FRAME`) |
+| `RT_VDBG=1` | Print VLAW computations to stderr |
+| `RT_FAITHFUL=1` | Use faithful chain (`fnfaith.cpp`) |
+| `RT_FIRCONV=1/3` | FIR application mode (1=complex-mul, 3=`1.019·mask^1.8345`) |
+| `RT_ENV=live` | Live envelope from kRTAtt/kRTRel tables |
+| `RT_KMAP=1` | k-mapping correction (twin/am scaling) |
+| `RT_EQ=1` | Pre-detector EQ bell |
+| `RT_LUT_OFF=1` | Skip LUT transform |
+| `RT_LUT_A/B/G/M` | LUT parameters (A=-24, B=28, gamma=1, mult=4.2) |
+
+---
+
+## 6. Test Corpus & Metrics
+
+**Location**: `/home/m/soothe-bt/` (~600 renders, outside git).
+
+**Key sets**:
+
+| Prefix | Content |
+|--------|---------|
+| `tone1kq_*` | Single tone, fc-scan, quiet (-18 dBFS) |
+| `tone1k_*` | Single tone, fc-scan, loud (0 dBFS) |
+| `dual_b1q_*` | Two tones (500+2000), q 0.1…10 |
+| `al_*` | Level sweep (fc=1000) |
+| `comb_*` | 4-band multiband |
+| `burst500_b1` | Primary reference (burst 500 Hz) |
+
+**Metric** (Goertzel steady-state):
+```python
+err_dB = 20·log1₀( ta(out,1000) / ta(ref,1000) )
+# trimmed to last 75% of input, matched to plugin render length
+```
+
+**Regression guards**:
+```bash
+python3 scripts/corpus.py                                    # bridge + guard
+python3 scripts/corpus_structural.py                         # structural chain
+python3 scripts/corpus.py --compare scripts/baseline_bridge.json --tol 0.25
+python3 scripts/corpus_structural.py --vs-bridge scripts/baseline_bridge.json
+```
+
+---
+
+## 7. Current Status (2026-08-29, 24mm14)
+
+| Group | Bridge | Structural (VLAW) |
+|-------|--------|-------------------|
+| t1kq (fc-scan) | 0.226 | 0.426–0.852 |
+| t1k (loud) | 1.801 | 0.577–0.930 |
+| al (level) | 0.638 | 0.090–0.804 |
+| res | 0.628 | 0.395 |
+| dual (q-sweep) | 0.726 | **0.193** ✓ |
+| comb (4-band) | 10.149 | 2.678–4.335 |
+| **TOTAL** | **1.594** | **0.341** |
+
+**Decoded**:
+- Layer: STFT without synthesis window, per-bin mask multiply
+- Law: `mask = 10^(−(α·ln1p(lvl/β)+c)/20)` with content-aware Δ
+- FIR: `exp(0.984·ln(raw))` + Hann + normalize → bit-exact RFFT to df0
+
+**Open gaps**:
+1. **Chain 9–19** (priority #1): Dataflow decoded, bigkernel bodies known, I/O format unknown. Target: <0.05.
+2. **k-mapping** (priority #2): twin/am scaling `k(q≥2)=0.403`.
+3. **Δ second-peak rule** (priority #3): Content-dependent gain, requires live-dump.
+
+---
+
+## 8. Reference Documents
+
+| Doc | Content |
+|-----|---------|
+| `README.md` | TOTAL metric, reproduction steps |
+| `AGENTS.md` | Build commands, env flags, hazard, corpus format |
+| `BITEXACT_PLAN.md` | 3-step plan to byte-exact |
+| `handoff/BLOCKMAP_529fe0.md` | Method map for FUN_180529fe0 |
+| `handoff/NOTES_LEVEL.md` | Live working journal |
+| `handoff/NOTES_LEVEL_INDEX.md` | Journal index |
+| `handoff/NOTES_TWIN.md` | Twin resonator reference |
+| `handoff/NOTES_CAPTURE.md` | Live-capture protocol |
+| `handoff/nls_dasm/` | 183 disassembly files |
+
+---
+
+## 9. Reproduction & Development Protocol
+
+1. Install soothe2 VST3 (Windows) under yabridge → `dump_soothe.py` → `soothe_mem.bin`
+2. Ghidra headless: `analyzeHeadless <proj> soothe_x64 -process soothe_mem.bin -noanalysis -postScript <X>.java`
+3. Renders: `sweep.py` → `.rpp`, `reaper -renderproject` → `.wav`
+4. Analysis: `render_parity.py` / `corpus.py`
+5. After C++ edit: `touch` source → `cmake --build dsp/build --target framed_test` → run corpus guards