Six backends, one tape
cpu, cpu-jit, simd, vulkan, rocm, and cuda replay the exact same recorded tape three genuinely different ways — an interpreter, two ahead-of-time compilers, and a fused GPU kernel — with only a string changing in your code.
Every payoff on this site has run on .on("cpu-jit"). What actually
changes if that one string becomes "cpu", "simd", or a GPU backend
name — and why does replaying the exact same tape run at such wildly
different speeds depending on which string it is?
The whole story
cpu and cpu-jit run the identical tape with a genuinely different
execution strategy, not just different optimization flags. ScalarReplay's
own doc comment says it plainly: it "walks [the tape] node by node, forward
then backward, which costs an interpreter dispatch per node." cpu-jit's
KernelGenerator instead compiles that same tape, once, into a class file
with the JDK's Class-File API — "the forward and reverse sweeps become
straight-line bytecode with node indices and constants baked in — no
interpreter dispatch, no ops[i]/argA[i] array loads." Same math, same
tape, one fewer layer of indirection per node, replayed millions of times.
Three ways to replay without a GPU
cpu is the oracle every other engine is checked against — an
interpreter, nothing to configure, no compiler step. cpu-jit trades a
short compile (you've seen this: 1.3's build=31.2 ms) for a kernel with
no per-node dispatch at all. Real numbers, same 1,536-node Asian tape from
2.1 and 3.1:
| engine | precision | scenarios/s |
|---|---|---|
cpu | fp64 | 1.0×10⁶ |
cpu-jit | fp64 | 1.8×10⁶ |
simd | fp64 | 4.0×10⁶ |
simd goes further: instead of one scenario at a time, VectorReplayF64
packs several scenarios into one JDK Vector API instruction — one lane per
scenario, several lanes per instruction — which is why it needs an
explicit --add-modules jdk.incubator.vector the other two don't.
The simd engine's first version was slower than the plain scalar
cpu interpreter, and the reason is in VectorReplayF64's own doc
comment. With all fourteen opcodes and both sweeps written as one method,
C2's escape analysis couldn't prove a DoubleVector never escaped that
method, so it gave up and allocated every one of them on the heap instead
of keeping them in registers — measured at 88 bytes per node, 136 KB per
scenario. Splitting each opcode into its own small method let escape
analysis do its job, and the same arithmetic stopped allocating entirely.
The fix wasn't a smarter vector trick; it was making the methods small
enough for the JIT to see through them.
Three ways to talk to a GPU, briefly
vulkan, rocm, and cuda each compile the tape into a fused
forward-and-reverse kernel for their own device — a GLSL→SPIR-V compute
shader, HIP C++ via HIPRTC, and CUDA via NVRTC, respectively, each fp32 for
speed. All three gate themselves at runtime: no device or driver, and
.on(...) just reports that backend unavailable rather than failing
anything. This page stops at "what each one is called"; Module 4.3 is
where the differences between the three actually matter.
The README's own six-row matrix, all real numbers from one machine replaying the same tape:
| engine | precision | scenarios/s | 1e10-scenario run |
|---|---|---|---|
cpu | fp64 | 1.0×10⁶ | 2.7 h |
cpu-jit | fp64 | 1.8×10⁶ | 1.6 h |
simd | fp64 | 4.0×10⁶ | 42 min |
rocm | fp32 | 1.3×10⁷ | 13 min |
vulkan | fp32 | 1.6×10⁷ | 10 min |
cuda | fp32 | (no device on this box) | — |
Every row is the same recording, the same seed, the same price and Greeks to Monte-Carlo noise — only throughput moves, by more than an order of magnitude top to bottom.
Try it yourself
Nothing here needs a GPU to try: swap .on("cpu-jit") for .on("cpu") in
any example from Module 1 through 3 and rerun it. The price and every
Greek should match to the digits already printed on those pages — cpu is
the oracle, so agreement with it is the whole point, not a coincidence.
▶️ Run it
mvn -o -q install
mvn -o -q -pl nablatensor-examples exec:java \
-Dexec.mainClass=com.nablatensor.examples.AsianGreeksBackends \
-Dscenarios=2000000 -Dsteps=252
Prints one row per backend referential machine can actually run — cpu and
cpu-jit need nothing extra; add simd with MAVEN_OPTS="--add-modules jdk.incubator.vector" in front of the same command. A GPU row only
appears if referential machine has that vendor's driver; nothing breaks if it
doesn't.
⚠️ What this doesn't do
This page names all six backends and explains cpu/cpu-jit/simd
mechanically; it deliberately doesn't go past "compiles to a shader or a
device kernel" for vulkan/rocm/cuda — that comparison, and switching
between them on a real GPU, is Module 4.3. It also doesn't cover
checkpointing, which is what happens when a tape is too large to fit on a
device at all — Module 4.2.
What's next
→ Deeper: docs/validation.md
covers how all six backends are checked to agree — cpu-jit bit-exact
against cpu, the GPU backends to reduction/rounding order.
→ Next: Forget, then remember: checkpointing on a GPU
— what happens when a tape doesn't fit in memory at all.