DE version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
77% Positive
Analyzed from 1176 words in the discussion.
Trending Topics
#rust#cuda#gpu#compile#projects#working#nvidia#language#more#cudarc
Discussion (20 Comments)Read Original on HackerNews
Rust-GPU: This is the org responsible for Rust CUDA, which was a historically non-working library I spent too much time trying to get working. Instead, Cudarc is a simple, "just-works" library with a responsive maintainer. For grpahics, WGPU and Vulkan bindings are good paths. To watch: Nvidia's official Cuda-Oxide, which uses Cudarc's style API for Host, and its own native Rust kernels. (Early/WIP stage on that). It is surprising to see "Rust GPU" there instead of WGPU, Nvidia/Cuda-Oxide, or Cudarc there, as it's the least viable of the set.
Embedded is a fantastic area for Rust to excel in, and it's already excellent thanks to the general language tooling, Cargo, and the Knurling tools Probe-RS and defmt. I'm not sold on the embedded working group's history and style. E.g. Embedded HAL is a poor fit for practical firmware and integrations. ESP on Rust was great but turned into a mess 1 - 1.5 years ago due to mismanagement, and letting a new maintainer rewrite large chunks of the HAL. I'm comfortable using Rust on STM32 and nRF, and have built working aerobatic quadcopter firmware in Rust, but have never seen eye-to-eye with the working group. In general, they trend towards complicated APIs which focus on Traits, safe abstractions, Async etc. My pref: Use Rust as a nice overall language, and write low-friction APIs, vs the abstractions.
I disconnected from the Embedded rust OSS scene; It's still my top choice for new hardware, and use it at work. It seemed like no one wanted to talk about projects and building things with rust; instead it was about using ownership, generics, safety, Async models etc to make clunky APIs.
1. Rust CUDA is over 5 years years old, was dead for 2(?) years, but was rebooted and works. It enables both rust on the GPU and controlling the GPU from the host. The host library (cust) predates cudarc. If it was started today it would just use cudarc for the host side (and indeed, you can use the device side with cudarc). Rust-cuda is based on nvvm, which was the supported layer in the past but Nvidia is moving away from.
2. Rust-gpu (vulkan) works but isn't 100% complete of course. The dimforge folks are using, there are some crypto folks using, and we have llms written in it. Rust-gpu is only the "running rust on GPU part", the host is left to wgpu or ash or whatever.
The Nvidia projects are great (we gave pre-release feedback on them), but they were not announced when the call for presentations went out. They were also released as experimental with only 2 people working on them so Nvidia was cautious about marketing and over-committing.
We are the only people trying to bring rust natively to the GPU rather than just making the GPU work with rust (which others like cubecl, wgpu, cuda-oxide handle well). This has some interesting considerations from a rust language and compiler standpoint, which is what was talked about at the conf.
We sponsored an unconf room and got as many folks from all the various GPU projects together, including nvidia. The big problems right now are a) everybody has different needs, b) there are very few people working in the space c) the entire rust project is generally indifferent to GPUs. I expect this to change in the next year or two.
There is also a Cuda-Tile rust lib which I have been meaning to try, but I haven't used tiles so far.
cuda-tile is great...I would use it for anything written from scratch as it is closer to how the hardware (and projects like triton) work. But there are tradeoffs as it requries your code to be structured a certain way.
Co-signed.
Really, if you're using a SaaSS LLM like Claude or Copilot or ChatGPT, which already sends all your code to someone else's computer to run on a beefy GPU, they should just send the Rust code to the same datacenter and send you back a binary.
Little shocked that Anthropic isn't offering this.
Rust compile speeds will matter less and less as hardware gets faster.
I’ve worked on some very large Rust projects. The incremental compile times are nowhere near the same order of magnitude of a bottleneck as an LLM turn.
(On my local machine, `cargo check` is roughly 2x faster than `cargo build`. It's still slower than hot-reloaded Python, but it's rarely my development bottleneck.)
Large Rust projects are broken up into modules. If you change one module you don’t have recompile the whole project.
Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
Have you tried Cerebras, Groq, Taalas, et al? It was a paradigm shift for me.
I was disagreeing with the concept that incremental compile times could be a bottleneck. LLMs are even better at doing large swaths of work at once and having it compile first or second try than a human.