PROGRESS.md

Build progress / resume state

Updated 2026-06-16. Autonomous /loop building toward a verified end-to-end text→SFX.

FINAL STATE — whole pipeline on GPU, verified (supersedes stale notes below)

• All five networks run on CUDA: tokenizers (CPU, trivial) → MetaCLIP-text, T5, MM-DiT, oobleck VAE all on GPU. End-to-end ~29 s, spectral-corr 1.0000, wav rel-L2 1.4e-2 (DiT cuBLAS-13 TF32; perceptually identical).
• VAE on GPU: transpose-conv via the zero-stuff + conv1d identity (col2im_1d has no CUDA kernel; ggml_conv_transpose_1d's kernel is O(out·IC·OC·T_in) → hangs on ×2048). Kernel flipped + channel-swapped at conversion (convert_vae.py:to_conv1d_kernel, ggml ne=[K,IC,OC]). Zero-stuff avoids the ggml_pad ne1→gridDim.y≤65535 trap by padding the stride axis + permute/cont. VAE GPU parity rel-L2 6.8e-4 vs golden. See ARCHITECTURE.md §5.
• CUDA soft_max root-fixed environmentally: removed the Debian nvidia-cuda-dev/libcudart12 package that shadowed the 13.3 cudart (header/runtime ABI mismatch → garbage smpbo). Cleared all stale CUDA_*_LIBRARY CMake cache entries → re-resolve to /usr/local/cuda-13.3 → ldd libggml-cuda shows libcudart.so.13, test-backend-ops SOFT_MAX 212/212. My earlier ggml-cuda.cu patch was reverted — llama.cpp tree is clean. (So T5 + CLIP on GPU need no patch.)
• Memory: ts-generate loads each network in its own scope and frees before the next (all 4 weights + VAE intermediates OOM a 16 GB card otherwise). Peak VRAM = one stage, not the sum.
• Docs: top-level README.md (overview/usage) + ARCHITECTURE.md (as-built runtime). docs/ARCHITECTURE.md remains the model spec.

Done & verified

• M0 — env + weights + converters + build + harness.
  • venv: /media/ilintar/D_SSD/thinksound/.venv (py3.12, torch 2.12 cpu). Run reference scripts with PYTHONPATH=/tmp/ThinkSound and stub k_diffusion (sys.modules['k_diffusion']=types.ModuleType(...)) to dodge broken clip/pkg_resources transitive deps.
  • weights: /media/ilintar/D_SSD/thinksound/ckpts/{thinksound_light.ckpt,vae.ckpt}.
  • GGUF: gguf/vae-f32.gguf (78M decoder), gguf/dit-f32.gguf (1.277B). Converters in convert/.
  • C++ builds against prebuilt ggml at /devel/tools/llama.cpp/build (commit 911b67a60). cmake -S . -B build && cmake --build build -j4.
  • tools: ts-gguf-dump, ts-vae_decode.
• M1 — Oobleck VAE decoder in ggml. PARITY rel_L2 = 1e-6 vs PyTorch (ts-vae_decode on golden/golden_vae.gguf). Audible. src/vae_decoder.{h,cpp}.

In progress

• M2 — MM-DiT + rectified-flow sampler.
  • gguf/dit-f32.gguf ready.
  • convert/dump_dit_golden.py running in background (/media/ilintar/D_SSD/thinksound/golden_dit.log, ~10-15 min on CPU) → golden/golden_dit.gguf with {noise, clip_f, sync_f, text_f, t5_features, metaclip_global_text_features, flow0, latent, wav_ref}. Uses text-only path (video=learned empties, text/t5=seeded random).
  • NEXT: write src/mmdit.{h,cpp} (input projs, v2 timestep embed [freq 1024, max_period 1], FLUX interleaved RoPE on latent+clip, 7 JointBlocks [3-stream joint attn + adaLN + conv-SwiGLU-FFN], 14 fused blocks, FinalBlock, batch-CFG). Verify flow0 then full latent then decode→wav_ref.

Key technical learnings (reuse)

• /devel/tools/acestep.cpp is a ggml DiT-audio model with the same oobleck VAE — src/vae.h and src/dit.h are direct references for snake / conv / transpose-conv / DiT idioms.
• ggml_conv_1d forces F16 im2col (asserts F16 kernel). For f32 parity, build conv from explicit ggml_im2col(...,GGML_TYPE_F32) + mul_mat (see src/vae_decoder.cpp::conv1d).
• ggml_conv_transpose_1d asserts p0==0 (no padding) → unusable for oobleck. Use GEMM + ggml_col2im_1d: pack transpose weight to [IC, K*OC] (i=oc*K+k) in the converter (convert_vae.py::pack_convtranspose), then mul_mat(w, xᵀ)→col2im_1d(stride, OC, pad).
• ggml audio layout: [T, C] (ne[0]=time, ne[1]=channels). conv kernel ne [K,IC,OC]; data [T,IC].
• SnakeBeta: x + sin²(exp(α)·x)·exp(−β), α/β per-channel [C]→reshape [1,C].
• DiT ckpt prefix model.model.*; VAE pretransform.model.* (also in light ckpt) / vae.ckpt autoencoder.*. No EMA in light ckpt. mm_unchang conditioners are weightless.

Done since (all verified vs PyTorch)

• M2 sampler + M3: CFG + 24-step rectified-flow Euler + VAE decode. flow0 rel_L2 1.1e-3, latent 6.7e-4, wav 1.8e-3. (ts-dit_sample)
• M4/M5 encoders: T5-v1.1-xl (src/t5_encoder.cpp, gguf/t5-f32.gguf) rel_L2 2.2e-3; MetaCLIP-text (src/clip_text.cpp, gguf/metaclip-text-f32.gguf) per-token 1.6e-4, global 1.5e-4. (ts-encoders_test). KEY FIX: both HF encoders apply the tokenizer padding attention_mask (CLIP also causal); must replicate or only pooled@eos matches. Goldens: golden_text.gguf (caption "a dog barking", cot "...").

CRITICAL BUG FIXED (LayerNorm eps)

The DiT's affine-free nn.LayerNorm uses PyTorch default eps=1e-5, NOT 1e-6. Symptom: full text→audio diverged (latent 14%, wav 21%) while every component passed individually and the synthetic-feature DiT golden matched at 3e-4. Root cause: real text features contain padding tokens with ~0 variance; for those, layernorm (x-mean)/sqrt(var+eps) is eps-dominated, so 1e-6 vs 1e-5 (√10≈3.16×) corrupts every padding token → wrong text q/k/v → wrong joint attention. Synthetic random features have no padding → no near-zero-variance tokens → masked the bug. Fix in src/mmdit.cpp::layernorm (1e-5). After fix: e2e staged flow rel_L2 1e-5. Lesson: match norm eps exactly; test with REAL (padded) inputs, not just random. RMSNorm q/k kept at 1e-6 (not padding-sensitive).

CUDA soft_max bug FIXED — T5 + CLIP + DiT now all on GPU

Root cause (found via compute-sanitizer + instrumenting softmax.cu): ggml_cuda_info().smpbo was garbage (4294967297 = 0x100000001) because it's read from the cudaDeviceProp.sharedMemPerBlockOptin struct field, and ggml-cuda was compiled with CUDA 13.3 headers but links libcudart.so.12 → struct layout mismatch → that field read from the wrong offset (VRAM/cc sit at stable offsets, so only soft_max broke). Fix in ggml/src/ggml-cuda/ggml-cuda.cu (~line 290): read the limit via cudaDeviceGetAttribute(cudaDevAttrMaxSharedMemoryPerBlockOptin) (stable single-value ABI) instead of the struct field. test-backend-ops -o SOFT_MAX -b CUDA1 now 212/212 pass. Rebuild: cmake --build /devel/tools/llama.cpp/build --target ggml-cuda. (Real ggml fix — benefits all the user's CUDA usage; the deeper cause is a mismatched CUDA toolkit/runtime in their build.)

Result: T5, CLIP, DiT all run on GPU (T5/CLIP reverted to ts_backend_init()); full pipeline wav rel_L2 1.29e-3 (better than the old hybrid 1.86e-3 — T5 uses exact soft_max now, not CPU). Only the VAE stays on CPU (ggml_col2im_1d genuinely has no CUDA kernel — a separate gap, fixable via conv_transpose_1d(p0=0)+crop; VAE is one decode, negligible).

GPU (M7, earlier) — historical: DiT on CUDA, encoders+VAE on CPU

src/common/ts_backend.h picks the max-memory GPU (CUDA1 = RTX 5060 Ti 16GB); ts_model::load_backend streams weights into a backend buffer. Per-model: DiT → GPU, encoders+VAE → CPU (ts_backend_init(false); TS_FORCE_CPU=1 forces all-CPU; TS_BACKEND=CUDA0/CPU overrides).

This ggml build's CUDA backend gaps (commit 911b67a60), discovered the hard way:

• Standalone ggml_soft_max is BROKEN on CUDA (even [128,10] → "invalid argument"; llama.cpp's GPU path uses fused flash-attn so standalone soft_max is untested). → DiT attention rewritten to ggml_flash_attn_ext (k/v cast F16, set_prec(F32)), works great: flow rel_L2 2e-5, full pipeline wav rel_L2 1.86e-3.
• CUDA flash-attn mask requires mask->ne[2]==1 (broadcast over heads) + 16-byte-aligned strides → T5's per-head relative bias can't use it → T5 stays CPU.
• ggml_col2im_1d has no CUDA kernel → VAE transpose-conv stays CPU (or re-do as conv_transpose_1d(p0=0)+crop with an unpacked [K,OC,IC] weight).
• GGML device enum here: CUDA cards report type=2 (IGPU), not GPU(1) — select GPU||IGPU.
• CUDA binary ops need contiguous operands → chunk() must ggml_cont the adaLN views.

Path to full-GPU (future): CLIP→flash-attn with padded broadcast mask; VAE→conv_transpose_1d+crop; T5 likely stays CPU (per-head bias).

DONE — full standalone text→SFX (M6 complete)

• Tokenizers (src/tokenizer.cpp, gguf/{clip,t5}-tokenizer.gguf via convert/convert_tokenizers.py): CLIP byte-level BPE + T5 SentencePiece unigram (Viterbi). Bit-exact vs HF on the test caption/cot (ts-tok_test).
• ts-generate is now the standalone CLI: --caption/--cot raw text → tokenize → encode → DiT(GPU) → VAE → wav. Verified vs golden_e2e.gguf: latent rel_L2 8.3e-4, wav 1.86e-3. Generates novel prompts (e.g. "heavy rain on a metal roof").

Q8_0 quantization — VIABLE (assessed)

convert/quantize_gguf.py f32.gguf q8.gguf (or --quant q8_0 in the converters). Quantizes 2D linear weights to Q8_0; keeps convs (3D, fed to im2col), norms/biases (1D), embeddings & empties f32. No C++ change — ggml mul_mat auto-dispatches on weight type (works on CUDA + CPU).

• Size: DiT 4.8→3.7GB (23%; convs stay f32), T5 4.6→1.4GB (70%). Total weights ~11→6GB.
• Speed: 42.8s→31.3s (27% faster); peak RAM 8.9→5.6GB (37% less).
• Quality (vs PyTorch ref): waveform rel_L2 6.8% (phase-level, misleading) BUT spectral-corr 0.9987, envelope-corr 0.9995, log-spec-dist 0.318 vs 0.235 f32 → perceptually identical. Attribution: T5-Q8 4.6% wav (bigger), DiT-Q8 2.6%.
• To compress DiT convs too (~60% DiT saving): store conv weight pre-reshaped [K*IC, OC] and pass a shape-only kernel to im2col, OR a conv-friendly quant. Q4_K would be the next step to test.

BF16 (recommended near-lossless) + why T5 is CPU

T5-on-GPU is blocked by a real ggml bug, not my code: test-backend-ops -o SOFT_MAX -b CUDA1 (llama.cpp's own test) FAILS ("invalid argument") on commit 911b67a60 — standalone CUDA soft_max is broken. T5's per-head relative bias also can't use CUDA flash-attn (it requires mask->ne[2]==1). So T5 (and VAE col2im) stay CPU. The DiT only works on GPU because it uses mask-less flash-attn.

Conv weights are now quantizable: conv1d_f32 was restructured to mul_mat(weight, cols) (weight = first operand → can be bf16/quantized; im2col only needs its shape). Numerically identical for f32 (removes a transpose). quantize_gguf.py --type bf16 now covers convs too.

f32 vs bf16 vs Q8 (dog-barking, vs PyTorch reference):

	DiT	T5	spec-corr	env-corr	lsd	time
f32	4.8GB	4.6GB	1.0000	1.0000	0.235	39.2s
bf16 (full)	2.4GB	2.5GB	0.9999	1.0000	0.266	37.1s
Q8	3.7GB	1.4GB	0.9987	0.9995	0.318	37.1s

BF16 = perceptually identical (spec-corr 0.9999), true 50% DiT (convs incl). Q8 = smaller T5 but DiT partial (convs f32) + slightly more loss. Both fine. For max DiT-Q8 you'd pre-reshape conv weights to [K·IC, OC] (ne0 block-aligned) + feed im2col a shape-only tensor.

Optional future polish (not blocking)

• Full-GPU encoders/VAE (CLIP→flash-attn padded mask; VAE→conv_transpose_1d+crop; T5 likely stays CPU).
• Quantization (Q8/Q4 DiT+T5) for memory/speed; f16 weights to fit the 10GB card.
• Embed tokenizers + hparams into the model GGUFs for single-file distribution.
• CLIP regex pre-tokenization is simplified (whitespace+punct); exact for typical captions, may differ on unusual punctuation.

Dasheng-AudioGen — text→audio pipeline (VERIFIED FAITHFUL 2026-07-11; generation-path bugs fixed 2026-07-12)

• Architecture: Flan-T5-Large → content adapter (cross-attn + duration predictor) → LayerFusionDiT (U-Net flow-matching: 16 in + 1 mid + 16 out + final) → Vocos decoder (×2 upsampler + ConvNeXt + ISTFT) → 16 kHz mono WAV.
• C++ runtime: src/dasheng_adapter.{h,cpp}, src/dasheng_dit.{h,cpp}, src/dasheng_decoder.{h,cpp}, src/dasheng_tokenizer.{h,cpp}.
• Converters: convert/convert_dasheng.py (DiT+adapter), convert/convert_dasheng_decoder.py (Vocos+upsampler), convert/convert_dasheng_t5.py (Flan-T5-Large), convert/convert_dasheng_tok.py (T5 tokenizer).
• Golden dumper: convert/dump_dasheng_golden_v2.py produces gguf/golden_dasheng.gguf with real PyTorch reference tensors.
• Parity test: ts-test dasheng gguf/dasheng-dit.gguf gguf/golden_dasheng.gguf gguf/dasheng-decoder.gguf.

PARITY (rel_L2 vs PyTorch golden) — all pass (with the corrected golden that includes text_proj)

• adapter content 4.1e-5, local_duration 8.7e-5, global_duration 0, time_aligned 7.3e-6
• DiT flow (single step, t=1.0) 2.8e-5 (CPU) / 6.2e-4 (GPU)
• decoded audio 1.7e-3 (CPU) / 4.1e-3 (GPU) (waveform-level; perceptually identical, phase-domain)
• Earlier notes claimed time_aligned "reads ~1.4, a transposed layout artifact" — that was wrong; it was the missing text_proj (see generation-path bugs below). The golden dumper had omitted text_proj too, so the two errors cancelled and parity passed while real generation was garbage. Both fixed; time_aligned parity is now genuinely 7e-6.

Bugs fixed to reach parity (the networks had never actually been run before)

• DiT build_dit_block was rewritten to match LayerFusionDiTBlock: LayerNorm (not RMSNorm, eps 1e-5) incl. qk-norm; U-Net skip connections (out_blocks skip_norm/skip_linear, were absent); self-attn residual x + tanh(1-gate_msa)·attn(modulate(norm1(x))); norm2+norm_context on cross-attn; FFN x + (1-gate_mlp)·mlp(modulate(norm3(x))) with GEGLU exact ggml_gelu_erf (value=chunk0, gate=chunk1); adaLN bias; NEOX RoPE (rotate_half); timestep-embed freq 10000^(-k/half); final block silu(te)+LayerNorm; output transposed to time-major [T,1280]. Sampler = FlowMatchEuler sway sigmas, timestep=σ·1000, CFG uncond zeros both context & ta.
• Adapter: in_proj_weight q/k/v slice used wrong stride/offset (segfault); missing in_proj_bias and out_proj; LayerNorm eps (norm 1e-5, duration-predictor 1e-12); m_cache.local_dur/gdur were never assigned in build_adapter_cache.
• Decoder: ConvNeXt dwconv used reshape where a transpose was needed ([OW,C]→[C,OW]); LayerNorm not RMSNorm; exact GELU not SiLU; ISTFT head was missing exp(mag) + cos/sin→complex; missing ×2 ConvTranspose1d upsampler (convert_dasheng_decoder.py now pre-splits it into per-tap linears decoder.upsampler.w{k}); ISTFT hop 320 (istft_hop, not the mel hop_length=160); hann periodic; trim "same"-padding.
• cpu_istft: inverse-DFT sign was -, now +.
• Dangling ggml context (adapter/DiT/decoder): all three cache builders passed a local std::vector<uint8_t> buf as ggml_init mem_buffer → freed on return → cached graph dangled → nondeterministic mul_mat asserts / bad_alloc / segfault at compute. Now mem_buffer=nullptr (ggml owns it). The SAME bug existed in the ThinkSound MM-DiT (src/mmdit.cpp::build_forward_cache) — it stored m_cache.ctx but built it on a local buffer, so m_cache.fx dangled and grab() threw std::length_error (or the backend eval segfaulted) at compute time. It only "worked on CUDA" by luck (freed stack memory survived intact there); on this AMD box it crashed. Fixed the same way → ThinkSound generation now works on ROCm/Vulkan (verified end-to-end, CLI + server). This was the actual cause of the earlier "ThinkSound crash", NOT flash-attn.

Generation-path bugs (parity passed but real audio was garbage) — fixed 2026-07-12

The golden dumper used a simplified reference path: it fed T5 features straight in (bypassing the tokenizer) and omitted text_proj. So three bugs on the true text→audio path were invisible to the parity test. Root-caused by dumping the actual HF model's intermediates (tokens, ref_t5.npy, ref_content.npy) and diffing stage-by-stage against the C++.

• Missing text_proj (the big one): content_encoder.text_encoder.proj (Linear 1024→1024, gguf text_proj) must be applied to the T5 output before the content adapter. It was skipped → content ~1.38 off. Fixed in src/dasheng_adapter.cpp (content_in = text_proj(in_t5); the cross-attn query and the post-attn residual both use content_in, not the raw T5 features). dump_dasheng_golden_v2.py corrected to match, so the golden now exercises it.
• Tokenizer over-padding: ts_t5_tokenizer::encode always padded to 77 (ThinkSound's fixed length) and the whole padded sequence was fed to the DiT. Dasheng uses dynamic length (no pad). Added a bool pad param; the Dasheng path calls encode(..., pad=false). Ref token counts: "a dog barking" = 6, "<|caption|> a church bell ringing" = 15.
• Unk-penalty clobbering valid tokens: the SentencePiece unigram Viterbi used a hardcoded unk penalty best[i] - 10.0, which beat low-but-valid pieces like | (id 9175, score −11.6) and > (id 3155, score −10.6) → they came out as <unk> (id 2). Fixed: penalty = min(all_piece_scores) − 10.0, always below every real piece.
• After all three: reference tokens match exactly, ref_t5 1.4e-6, ref_content 8e-5, and generated-audio character matches the reference (dog barks: ~90% active → ~26% sparse; bell rings throughout).

GPU verified — ROCm/HIP + Vulkan (2026-07-12)

Both GPU backends run on the Radeon 8060S (gfx1151, RDNA 3.5), all stages pass:

backend	content	flow	audio
CPU (f32)	2.5e-5	2.8e-5	1.7e-3
ROCm/HIP (-DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151)	4.1e-5	6.2e-4	4.1e-3
Vulkan (-DGGML_VULKAN=ON)	5.7e-5	6.2e-4	4.2e-3

HIP uses f32 accumulation (near-exact); Vulkan is fp16 (still well within perceptual parity). End-to-end ts-dasheng_generate runs on both GPUs.

Vulkan path fixes (the whole Vulkan path had never compiled or run):

• cmake/spv_to_header.cmake: CMake regex has no {n} quantifier — ([0-9a-f]{2}) never matched, so the SPV byte array was empty; emit alignas(4) byte array + byte-count size.
• vulkan_istft.cpp: symbol name mismatch (*_comp_spv vs generated *_spv) + anonymous-namespace externs → now #includes the generated headers; pool_info name collision; enabled VK_EXT_shader_atomic_float; zero the scatter-add buffers before dispatch; glslc target bumped to vulkan1.2.
• vulkan_istft.comp: output is a reserved GLSL word (renamed out_audio); uint→int casts; inverse-DFT sign +; range-reduce the DFT angle (k*n) mod N (GPU trig lacks large-argument range reduction); atomicAdd for the overlapping-frame scatter.
• ISTFT bug root-caused & fixed: the GPU win_sum normalization (norm shader / win_sum atomic) was wrong → audio rel-L2 ~0.5. The window envelope is spectrum-independent, so it's now computed and applied on the CPU (trivial, exact) while the GPU does the DFT+scatter → audio 3.96e-3. The GPU ISTFT is the default on Vulkan builds (CPU fallback if init fails); vulkan_istft_norm.comp is now unused.

Server (ts-server)

Hosts both pipelines: POST /generate (ThinkSound) and POST /v1/dasheng/generate (Dasheng). Each pipeline is optional — the server loads whichever GGUFs are present, registers only the matching endpoints (startup log prints which), and exits only if neither loads. Verified all three cases over HTTP on ROCm: both-present → both endpoints; Dasheng-only → 404 on /generate; ThinkSound-only → 404 on the dasheng route; Dasheng generation returns a valid WAV (200). Loaded pipelines are warm — every model stays resident for the process lifetime (no per-request reload/free, unlike the CLIs), so peak memory is the sum of the loaded weights. A single mutex serializes generation across both endpoints. Both endpoints verified over HTTP on ROCm: /generate → 200 (2 s stereo 44.1 kHz) and /v1/dasheng/generate → 200 (2 s mono 16 kHz).

• DiT double-load fixed: the warm cache did adapter.load(dit_gguf) and dit.load(dit_gguf), loading the 8.7 GB file twice. ts_model::load_backend now takes an optional tensor-name filter; the adapter loads only non-dit.* (adapter.* + specials) and the DiT only dit.*, so the shared gguf is held once. Measured peak RSS for adapter+DiT+decoder dropped ~18 GB → 9.35 GB. Parity unchanged.

Generation-QUALITY bugs (audio noisy/loud even after the parity fixes) — fixed 2026-07-12

The single-step golden can't exercise the 25-step sampler, the dummy-ta path, or the duration/amplitude handling, so these survived it. Root-caused by running the reference model's own generate() (needs a torchaudio stub — the model never imports it but transformers gates on it) and adding two tools: ts-test dasheng_sampler (full 25-step flow-matching parity vs gguf/golden_sampler.gguf from convert/dump_dasheng_sampler.py — our latent 2.85e-3 vs the reference, decodes to peak 0.49) and ts-test dasheng_decode (decode a raw latent file in isolation).

• Latent length = PREDICTED duration, not requested. The reference sets latent_length = round((exp(global_duration_pred) − duration_offset=1.0) · latent_token_rate=25) (= 250 for "a dog barking", 10 s). We forced T = ts_dasheng_T(cli_dur) (9 s → 225). The model only produces in-distribution audio at its own predicted length — an arbitrary T noises/clips the content (T=225 latent decodes to peak 1.2; T=250 to 0.6). Fixed: ts_dasheng_T_from_global() (ts_utils.h); the pipeline runs the adapter, reads global_duration, derives T, then builds noise/ta/DiT/decoder at that T. --duration is now ignored (kept for API compat). For text→audio the duration-expanded time_aligned is discarded (all positions → dummy_ta_embed), so only the length matters from the duration path.
• REVERSAL — drop the <|caption|> tag (the earlier claim in this doc was wrong). generate() tokenizes the bare caption ("a dog barking" → 6 tokens); it does not call compose_prompt. <|caption|> is NOT a special token in this T5 tokenizer — it splits into literal chars [<unk> | cap tion | >] (13 tokens) and drives the reference output to near-silence (peak 0.016 vs 0.49 untagged). The user's "perfect" reference used the untagged prompt. compose_caption() now strips a leading tag instead of adding one. (The earlier "birds 141→781 Hz" claim was a mis-comparison.)
• No peak-normalization. The reference returns raw audio (peak ~0.3–0.7). Our WAV writer defaulted to normalize=true (x/max|x| → always peak 1.0), which is why every clip read peak 1.0 (this masked/confused the diagnosis for a while — the decoder output is already in range). The Dasheng pipeline now writes normalize=false. Cosmetic (same SNR) but matches the reference amplitude.
• Integer timestep. The reference backbone casts the timestep to torch.long before the sinusoidal embedding (dit.py), so the model always sees an integer. ts_dasheng_dit::sample now truncf(σ·1000). Minor (didn't move the noise floor) but faithful.

After all four: dog quiet-floor 0.00069 vs reference 0.00073, active 7.3% vs 7.1% — the "noisy background" is gone; generation is genuinely prompt-conditioned. Our sampler+decoder were proven faithful throughout: feeding the reference's exact noise/content through our sample()+decoder reproduces the reference audio; the earlier apparent "divergence" was a transposed reference-latent dump in a diagnostic script, not a real bug.

Notes / remaining

• Attention uses ggml_flash_attn_ext (via ggml_ops::attention, K/V cast to F16), same as the ThinkSound MM-DiT. Verified working on CPU/ROCm/Vulkan (flow parity: CPU 8e-5, ROCm 5.6e-4, Vulkan 6e-4; the f16 cast is slightly less precise than f32 soft_max but well within parity). NOTE: during bring-up the adapter/DiT crashed and I mis-attributed it to the CPU flash kernel and switched to a soft_max path — the real cause was the dangling ggml-context buffer (fixed above). Once that was fixed, flash works on every backend; the soft_max detour was reverted. (ggml_ops::masked_attention remains as a correct masked-softmax primitive but is unused.)
• Regenerated gguf/dasheng-decoder.gguf with the fixed converter (encoder tensors dropped, upsampler split, correct istft_hop). The DiT GGUF was already complete (all weights present).
• dasheng_tokenizer.cpp::encode() is a documented stub (audio→token not needed for text→audio).