Skip to content

perf(prover): device-resident rounds 2-4 and fused NTT for GPU continuations - #875

Open
ColoCarletti wants to merge 2 commits into
mainfrom
gpu-opt-round2-impl
Open

perf(prover): device-resident rounds 2-4 and fused NTT for GPU continuations#875
ColoCarletti wants to merge 2 commits into
mainfrom
gpu-opt-round2-impl

Conversation

@ColoCarletti

Copy link
Copy Markdown
Collaborator

Second GPU optimization round on top of #863: ethrex 10tx continuations on RTX 5090 go from 12.01s to 9.75s (-18.8%, 8 ABBA pairs, sd 1.3%). Proofs are unchanged and cross-verify against the previous prover.

  • R2 composition parts, their Merkle tree, R3 OOD evals and R4 openings now read the resident device buffer instead of round-tripping ~200MB per table through the host.
  • FRI keeps each layer's evaluations on device; the query phase gathers only the ~219 opened positions per layer.
  • The multiplicity subset tree of preprocessed tables stays resident (no per-epoch node download and host rebuild).
  • Kernels: first 8 NTT levels fused in shared memory on the row-major path, small Merkle levels built in one launch, batch-inverse total inverted on device, big trace uploads staged through pinned memory.
  • LogUp aux build no longer pays a host transpose the GPU path discards.

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

GPU Benchmark (ABBA) — ac3dee9e36 vs main (14 pairs)

RTX 5090 · Vast.ai datacenter @ $0.8013888888888888/hr · prover/cuda · ethrex 100tx continuations · drift-free A/B/B/A

=== ABBA paired result  (improvement: - = PR faster) ===
  pairs: 14   mean A (PR): 60.956s   mean B (base): 74.924s

  [parametric] paired-t   mean -18.64%   sd 0.72%   se 0.19%
               95% CI: [-19.06%, -18.22%]   (t df=13 = 2.16)
  [robust]     median -18.48%   Wilcoxon W+=0 W-=105  p(exact)=0.0001  (z=-3.26)

  --- server stability (this run; compare across servers) ---
  run-to-run jitter:    A CV 1.02%   B CV 0.62%        (lower = steadier)
  within-session drift: -0.15% over the run, 1st->2nd half -0.13%
    (jitter -> Tier-1 cached gate floor; drift -> whether the cached baseline can be trusted)

  VERDICT: REAL IMPROVEMENT - PR faster by ~18.64% (t-CI and Wilcoxon agree)

  raw pairs: /tmp/abba_run/pairs.csv

- = PR faster. Trust the verdict when paired-t and Wilcoxon agree.

@ColoCarletti
ColoCarletti force-pushed the gpu-opt-round2-impl branch from e9b01ad to e34b461 Compare July 31, 2026 15:50
Grid-stride the fused row-major NTT past gridDim.y (lde >= 2^24 silently
fell back to CPU), assert the device-only contract in the R2 composition
commit and preprocessed opening fallbacks, validate htod_via bounds, retain
FRI device evals only under device-only, and move the inverse fault-injection
hook so every batch-inverse entry is covered.
@ColoCarletti
ColoCarletti force-pushed the gpu-opt-round2-impl branch from e34b461 to ac3dee9 Compare July 31, 2026 15:56
@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant