Benchmark Methodology

This page documents the methodology behind the performance benchmarks reported in our 2026 Outlook blog post.

---

Hardware

---

Workload

---

FLUX.1-dev Benchmark Results

Iterations per second on RTX PRO 6000 Blackwell for megapixel inference:

Performance Gains

Range reflects end-to-end integration overhead: AdaLN memory ordering, VAE↔DiT precision boundaries, CFG batching.

---

s4 Op-Gen Kernel Performance

NVFP4 fusion patterns on RTX PRO 6000 Blackwell — sustained TFLOPS:

Summary Stats

All patterns hit 800+ TFLOPS sustained.

---

Kernel Fusion Analysis

Per-block latency measured via TensorRT Nsys profiling using NVFP4 bidirectional noncausal attention. Native Blackwell FP4 tensor core path confirmed:

---

Why Official Tools Miss This Path

TensorRT ModelOpt (~8 it/s)

• W4A8 weight-only quant → attention hits unfused Myelin block, graph breaks at precision boundaries
• Kernel: sm80_xmma_gemm_f16f16 (Ampere fallback, graph fragmentation at every boundary)

Nunchaku (~8 it/s)

• W4A4 + FP16 low-rank branch → SVD outlier absorption adds ~16% memory traffic
• Memory overhead from branch fusion (~57% naive, ~16% optimized)

VAE Decode

• BF16 everywhere—carrying 8× more bits than measured latent entropy requires
• "The only thing that's difficult is to forget precisely."

s4 myelin4 (~200 it/s)

• Block-scaled NVFP4 with forced tactic selection
• Bidirectional noncausal attention emitted as single fused op → Myelin selects sm120 tactics
• Precision matched to measured latent entropy—epilogue fusion at gauge boundaries
• Kernel: cutlass3x_sm120_bstensorop (Native Blackwell FP4, whole-graph fusion intact)

The moat isn't quantization math—it's knowing where to force tactics, what compositions preserve fusion, and matching precision to measured entropy at every boundary.

---

Full Generation Estimate

Kernel Benchmark (Isolated)

End-to-End Projected (with Integration Overhead)

---

What Drives The Range

Negative Factors

• AdaLN memory ordering — Modulation tensors computed per-step create serialization points
• VAE↔DiT boundary — Precision mismatch at FP4 DiT → FP16 VAE decode
• CFG batching — Doubles effective batch mid-graph, memory pressure

Positive Factors

• Coherent precision — Single FP4 precision across entire DiT eliminates cast overhead
• Graph-level fusion — Myelin sees full transformer block, not isolated ops

---

Caveats

1. High variance (100-300 it/s range) from thermal/power modulation at 600W sustained and end-to-end integration overhead

2. Synthetic weights — Kernel timing from synthetic weight-shared model; full 6GB unique weights may see additional memory bandwidth cost

3. Pattern requirements — NVFP4 bidirectional noncausal attention pattern required for Myelin to select optimal tactics

---

Related Reading

• 2026 Outlook Blog Post — Full context and charts
• Research Overview — Technical foundations
• Weyl .plan — Deep-dive technical articles