I think implication being organizations with 40,000+ employees and even more consultants and contractors plus a lot of budget are also using LLMs to draft public facing content instead of paying for content writers or even just proof readers .
It points to friction rather than cost economics. Same reason we are always surprised why multi billion dollar product companies with millions of install base prefer electron instead of a native app.
Dude I am in slop fucking hell right now. There is still room for a human touch, without which the agents will lever us harder and faster into a world of incomprehensible garbage.
I totally concur. I'm almost lost for words at this stage. I need me some land to grow vegetables on and that's about it. Maybe some chickens. Every single day brings more despair (and not the prosperity we were promised).
I get the impression that Nvidia employees don't care too much - I started seeing fully AI-written "documentation" on some of their smaller projects more than a year ago (i.e., before it was even slightly a good idea).
Yeah lol it’s basically the only reliable way to know how things work. Pre-AI, I read documentation for libraries that I used almost every day.
And now with AI I’m using it to fact check Claude. And still reading it for myself to understand why other peoples code is written a certain way. It’s basically the most important thing to reference when coding.
Sure today Claude can just read the library code and tell you what a function does or how to do something. But it still won’t tell you why something is a certain way or won’t figure out specifically-designed usage patterns as reliably as the author telling you “this is an example of doing x”
when you start to internalize that these kinds of statements are an indication of how the average developer of the last 10-15 years operated the adoption rate of AI makes a lot more sense
I hadn't read the article and read this comment as though NVIDIA themselves were implying that this library was checked but not trusted by them since it was fully LLM generated.
It gets fun when someone uses an uncensored model to bypass a refusal, but they accidentally pick one that was trained for erotic writing and brings its particular talent to the documentation task.
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
Nobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
kernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc.
These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
> I strongly dislike CUDA. Once you have allowed that proprietary cr*p
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
It may be to bypass censorship, rather than self-censorship. Some platforms block or shadowban comments with curse words. Not sure about this platform.
Launching kernels manually is an error prone PITA which I believe is the principle reason for CUDA's popularity. Having the compiler give an error when you mess up is a huge benefit. But having the compiler allow you to express "I want to launch this kernel over a grid with these dimensions, with these arguments" as a single expression is where the vast majority of the value comes from.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
> Once you have allowed that proprietary cr*p into your C++ codebase
People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, use some sort of HAL if they are available, or you don't have a job. Even simpler.
> The CUDA runtime is a special case of one of the libraries provided by the CUDA Toolkit. The CUDA runtime provides both an API and some language extensions to handle common tasks such as allocating memory, copying data between GPUs and other GPUs or CPUs, and launching kernels. The API components of the CUDA runtime are referred to as the CUDA runtime API.
> The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually
No. CUDA allows you to write all the code in a single file, and uses a preprocessor to split it back out and pass it through separate compilers, one for host and one for device.
This true, but you can write the two separately if you want.
The disadvantages of writing them together are listed in the various parent posts. But some code authors really like the convenience of having the two in the same file.
I highly recommend Julia for (scientific) GPU programming but it would be nice if there was a larger community and/or funding behind the GPU side of things. It has very few core devs for what it is.
In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!
I think OP’s point is that the payoff in learning a new language has diminished in this AI era. You can call that lazy, I’d consider it smart to consider whether you could be doing other, better, things with your time.
I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.
Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
cudarc been great for me, because it's easy to look up existing examples and references, and it maps 1-to-1 with what I see. I'm already having a tough time with CUDA itself, a dialect of it makes a tad harder to rely on previous work.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.
what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.
Damn even Nvidia is putting out fully Claude-written articles.
Why "even Nvidia"?
They are fully behind using AI for basically everything.
What's next? "Damn, even McDonald's is putting out unhealthy food"
They are just better at hiding it or configuring Claude.
I have several skills that reformat text to remove AI-speak tells.
I put the "humanized" output through Pangram and it still comes out as 100% AI generated.
It points to friction rather than cost economics. Same reason we are always surprised why multi billion dollar product companies with millions of install base prefer electron instead of a native app.
The heck you talking about? How do you think we wrote software for the last 50 years?
And now with AI I’m using it to fact check Claude. And still reading it for myself to understand why other peoples code is written a certain way. It’s basically the most important thing to reference when coding.
Sure today Claude can just read the library code and tell you what a function does or how to do something. But it still won’t tell you why something is a certain way or won’t figure out specifically-designed usage patterns as reliably as the author telling you “this is an example of doing x”
I start with reading and exploring documentation first; with the codebase as a secondary tab.
When it’s not LLM generated, documentation is supposed to be easier to read and more insightful than code.
[1] https://github.com/huggingface/candle
The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.
Genuine question...why not just type "crap"? It's not even that much of a curse, but I've never really understood the point of self-censorship. If you don't want to curse then you could just use a non-curse word.
The having it all in a single file is mostly an artefact of the fact that it is C++, because C++ is single file at a time compilation. In D (which is multiple files in a single compiler invocation) with DCompute (which targets CUDA and OpenCL with upcoming support for Vulkan and Metal), you are required to write the kernels in a separate module, but you get all the benefits of the compiler complaining when you mess up _and_ the expressivity of "launch me this kernel".
People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy.
Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice.
Developing for embedded devices? So you read the manufacturers manual and implement based on the spec, use some sort of HAL if they are available, or you don't have a job. Even simpler.
From: https://docs.nvidia.com/cuda/cuda-programming-guide/01-intro...
Isn't that how CUDA code is normally written?
The disadvantages of writing them together are listed in the various parent posts. But some code authors really like the convenience of having the two in the same file.
And the Mojo standard library has been open source for over a year.
It’s all open source. Go check it out!
There were humans far superior than you for writting Rust before LLM, now there's a LLM. The only difference is price and time execution.
You get an awesome teacher (LLM) ready to answer all your questions about Rust.
And you still find excuses not to learn it ?
At some point, just realize you've been lazy to learn it and LLMs are just an excuse.
Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.
I haven't tried the tile API yet; looking forward to it.
The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.
Seems more ergonomic in general though, both approaches they share, compared to cudarc, and less build infrastructure and fiddling with environments, which is great.
Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).
Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.
This is more promising: https://github.com/Rust-GPU/rust-gpu/