Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
62% Positive
Analyzed from 1702 words in the discussion.
Trending Topics
#llms#llm#loop#lot#kernels#level#verify#sure#implementation#doing
Discussion Sentiment
Analyzed from 1702 words in the discussion.
Trending Topics
Discussion (37 Comments)Read Original on HackerNews
Personally, I believe that LLMs should be treated like an advanced version of Prolog or linear programming: you give the constraints, you have a way of verifying correctness, and you give it a clear goal. If the LLM can verify itself and course-correct you can basically leave it on autopilot
For anyone interested the ffmpeg is at https://github.com/poizan42/jellyfin-rpi-ffmpeg and a shim for using it with stock jellyfin at https://github.com/poizan42/jellyfin-rpi-ffmpeg-shim
https://github.com/qarl/arcade-js
Even the cheap LLMs are great in doing the awful crud work in the beginning: finding offsets, firmware update file structures, brute forcing checksums, etc.
It still produces a lot of crap in the later steps (understanding the implementation itself) but I'm happy doing this stuff myself then.
I've had success here by adding a phase called "grounding" that attempts to verify its "understanding" by creating tests that modify the running executable to ensure its made the right inference.
Is this variable really MARIO_X? Change it and see if Mario moves. Etc.
As an example in Donkey Kong - the system had trouble deciding if an array controlled barrels or fireballs. There was conflicting evidence.
After many trips through the loop - it realized it does BOTH, depending on which level you're on.
So the "understanding" grows with each iteration.
I then asked claude to compare the C#/.NET implementation in the library with the C++ version, and it quickly identified that the C# library was missing a couple of fairly cheap optimisations that were present in the C++ version.
If I can help get a PR merged, then it'll be by far the biggest impact of any work I've ever done.
I also compared the Rust version, it had this specific optimisation. The far more popular Tokio/Prost library did not.
Given appropriate guardrails, LLMs are impossibly fast at iterating to find root causes and specific performance bottlenecks.
Could you have manually profiled and compared the execution paths? Sure. Could you have translated the C++ optimizations to C#? Sure. But in such an obvious case, the LLM managed itself.
Watching claude and codex play winquake and age of empires, and debug support for Firefox 52 has been wild.
I wonder if there is extra effort put into this because they are useful for the researchers working on the models or just a sub-domain that language models are a great fit for and humans have trouble with?
Obviously it's still hard, but the point is that, by construction (cause like, NVIDIA literally releases primitives like this, and/or people like TK build slightly higher-level primitives over the base hardware primitives), if you learn the complicated language, you can get really good results, and on some level you "know" you're right by construction even before you go to the actual empirical tests (since you're operating over a higher-level "language", and not arbitrary byte accesses).
Honestly a lot of interfaces and frameworks you could argue are like that, so it's not really a point for GPU kernels relative to other things. But maybe a hint as to what I personally think is important in the AI era - finding the right cuts, the right high leverage abstractions, as otherwise AI is going to produce spaghetti nonsense.
Because pre-LLMs humans partially "autogenerated" kernels through hyperparameter search and in some sense eating the code complexity in return for performance, and thus built tools for the same automatic verifiability that is useful for LLMs.
In some other tasks, we never built the same level of automatic verifiability since the level of automation in creation being much lower meant it's not giving you as much of a marginal benefit. We prefer code readability and simplicity and such in say, web services, because, say, the database IO time is going to dominate. Here getting an LLM to write a cromulent C# web service is more difficult since it's not easy to automatically verify whether code is cromulent or not. So if you put up LLMs to it, you end up with slop (which works).
OTOH, in kernel design, you give it access to every perf counter, every observable possible and have it optimise all of them. And all are verifiable/hill-climbable
Which I believe was the word intended.
1) it's the only way to get something out of models (or people for that matter) that they don't know yet.
2) it's harder to do with an LLM than without. Not easier.
3) and when you fuck it up, half the time the LLM (or other ML technique) makes a fool out of you and you spent $1000 to find the quickest way to get a robot leg on the ground is just to crash it into the ground.
the code the LLM writes will be read and maintained and developed further by LLMs. so it doesn't really matter what it produces as long as all the tests are green and it achieves exactly what you want it to achieve.
IMO, LLMs will be a dead end to anything close to AGI because of this and hallucinations.
We're missing something in the mix, which I suspect is some kind of advanced JEPA model.
Humans do it ignorantly.
The LLMs will improve while average human IQ in the west dips closer and closer to the 80s on the global scale.
The existing models are surprisingly bad at it.