CUDA Finally Speaks Rust: The GPU Kernel’s Memory-Safety Reckoning

Rust has spent the last decade conquering the systems layer of modern computing, from the Linux kernel to browser engines. NVIDIA’s own Nova driver is written in Rust. NVIDIA Dynamo, their distributed inference platform, sits on a Rust core. Inference engines like HuggingFace’s Grout and mistral.rs are increasingly built in the language.

But there was always a wall. A hard, immovable wall made of CUDA C++.

You could launch kernels from Rust, sure. But the kernel itself, the code that actually runs on the GPU’s thousands of cores, had to be written in another language. That meant maintaining dual codebases, crossing FFI boundaries, and accepting that the most performance-critical code in your stack lived outside Rust’s safety guarantees.

In September 2026, NVIDIA announced it’s tearing down that wall. CUDA Rust is here, with two distinct tracks for writing GPU kernels natively in Rust, compiled directly to PTX.

This isn’t a wrapper. It’s not a DSL that happens to look like Rust. It’s the real thing, and it has serious implications for how we architect GPU-accelerated systems.

The Two-Track Approach: SIMT vs. Tile

NVIDIA’s strategy mirrors the two programming models CUDA itself has evolved. But before diving into the technical weeds, let’s be clear about what’s actually new here.

cuda-oxide is a custom rustc codegen backend. When it encounters #[kernel] functions, it routes them through Rust’s MIR, the Pliron IR framework, and LLVM down to PTX, while handing everything else to the standard compiler backend. The result: you can write host and device code in the same file, build with one command, and skip the separate kernel crate entirely.

cutile-rs works at a higher level. Instead of describing what individual threads do, you express operations on data tiles. The compiler handles thread mapping, memory layout, and architecture-specific optimization. It JIT-compiles through CUDA Tile IR and runs on stable Rust with no custom LLVM.

The difference matters more than you might think. Here’s the same elementwise addition kernel, 1,024 floats, both tracks, complete programs:

#[cuda_module]
mod kernels {
    use super::*;

    #[kernel]
    #[launch_bounds(256)]
    #[launch_contract(domain = 1, block = (256, 1, 1))]
    pub fn vecadd(a: &[f32], b: &[f32], mut c: DisjointSlice<f32>) {
        let idx = thread::index_1d();
        let idx_raw = idx.get();
        if let Some(c_elem) = c.get_mut(idx) {
            *c_elem = a[idx_raw] + b[idx_raw];
        }
    }
}

#[cutile::module]
mod kernel {
    use cutile::core::*;

    #[cutile::entry()]
    fn add<const B: i32>(
        z: &mut Tensor<f32, { [B] }>,
        x: &Tensor<f32, { [-1] }>,
        y: &Tensor<f32, { [-1] }>,
    ) {
        let tx = load_tile_like(x, z);
        let ty = load_tile_like(y, z);
        z.store(tx + ty);
    }
}

What the Compiler Catches That C++ Can’t

Here’s where things get interesting. GPU race conditions are notoriously nasty. Thousands of threads hitting the same buffers in undefined order means these bugs rarely reproduce on demand, they pass your tests, ship to production, and explode under specific hardware or workload conditions.

CUDA Rust kills a whole class of these at compile time. Consider this: passing the SIMT kernel’s output buffer as one of its own inputs won’t compile:

module.vecadd(&stream, &prepared, &c_dev, &b_dev, &mut c_dev)?;

The compiler immediately rejects it: error[E0502]: cannot borrow c_dev as mutable because it is also borrowed as immutable.

The same aliasing mistake on the Tile side fails too: error[E0382]: use of moved value: z.

The safety mechanisms differ between tracks. cuda-oxide introduces DisjointSlice<f32>, a type that hands each thread exclusive access to its own element while standard shared slices handle reads. The type system prevents two threads from holding mutable references to the same memory.

cutile-rs extends Rust’s ownership model to tiles. Partitioning a mutable output tensor on the host gives each tile block one writable sub-tensor that no other block can overlap. The .partition([128]) call doesn’t just split data, it establishes exclusivity, fixes the launch geometry (1,024 / 128 = 8 tiles), and supplies the tile width as a compile-time constant.

The key architectural difference: cuda-oxide checks each launch call for safety, while cutile-rs’s ownership follows tensors across the launch boundary. NVIDIA argues the latter is the “stronger of the two claims”, and they’re right, ownership-based safety is inherently more compositional than per-call validation.

What You Trade Away

Of course, safety isn’t free. Tile gives you no shared memory or thread indexing to get wrong, the compiler owns both. A tile block is a single logical thread, so there are no threads for you to race.

That’s a feature. But it’s also what you lose. SIMT keeps the control you’d have in CUDA C++, and today, shared memory in cuda-oxide requires unsafe. NVIDIA acknowledges this is “active work”, shared memory is the bedrock of fast SIMT kernels, so closing that gap is critical.

There’s also the toolchain reality. cuda-oxide requires a pinned nightly toolchain, LLVM, clang, and CUDA 12.x+. The cargo oxide new vecadd_demo workflow works, but the first build compiles the entire codegen backend, so expect a long wait. cutile-rs is easier, stable Rust 1.89+, CUDA 13.3, no custom LLVM, published on crates.io.

Real-World Performance: The Numbers That Matter

The performance question is where skepticism is justified. GPU kernel performance has historically been about hand-tuning that C++ programmers spent decades perfecting. Can Rust match that?

The early data says: yes, surprisingly well. NVIDIA’s AI agent skill translated all 24 public TileGym operators from cuTile Python to cuTile Rust, and the results on an NVIDIA DGX B200 are compelling:

Across 347 paired configurations, the geometric mean speedup hits 0.995, essentially parity with cuTile Python. All 24 operators clear the 0.95 threshold, and about a third actually outperform their Python references. The largest wins come from element-wise and normalization kernels.

The performance researcher’s paper, Fearless Concurrency on the GPU, reports even more impressive results: roughly 7 TB/s on elementwise operations and 2 PFLOPS on GEMM on an NVIDIA B200, about 96% of cuBLAS performance in that specific test.

What This Means for Systems Architecture

The deeper implication here extends beyond programming convenience. We’re seeing the consolidation of the GPU computing stack into a single language with strong safety guarantees.

The Caveats You Should Know

Let’s be honest about the current state. NVIDIA calls cuda-oxide “early alpha” and cutile-rs “more advanced, but experimental.” Neither is production-ready. Coverage is incomplete, and APIs will move.

The debugging story is another gap. A recent survey from the Rust compiler team found only 46% of developers use a debugger for Rust work, with 81% saying logs and print statements are easier or faster. Poor value representation is the culprit, 74% report debuggers show values badly. GPU kernel debugging is traditionally harder than host debugging, so this gap will matter.

The crate naming controversy is minor but illustrative: NVIDIA named their project cuda-oxide, but a crate with that name already exists on crates.io (admittedly unmaintained). A forum commenter raised exactly this concern, noting it “seems like it will lead to needless confusion.” Fair point.

What You Can Do Today

If you want to experiment, the paths are relatively straightforward:

cargo +nightly-2026-04-03 install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide
cargo oxide new vecadd_demo
cd vecadd_demo
cargo oxide doctor
cargo oxide run

cargo new vecadd_demo
cd vecadd_demo
cargo add cutile

The Bottom Line

CUDA Rust is a bet, a long-term investment in making Rust the language of GPU computing. NVIDIA explicitly says they’re maturing it into 2027 and beyond.

The question now isn’t whether CUDA Rust will matter. It’s how quickly the ecosystem will catch up.