DE version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
79% Positive
Analyzed from 2868 words in the discussion.
Trending Topics
#models#cloud#more#model#same#using#better#local#claude#don
Discussion (131 Comments)Read Original on HackerNews
https://hn.algolia.com/?query=working%20author%3Adavid927&so...
I'm making a fairly simple game, a single player 2D roguelite strategy game. More context here: https://news.ycombinator.com/item?id=48886151
From an implementation perspective, it is about 80% complete, where by "complete", I mean "adequately polished such that I could ship it." (I'm not interested in providing evidence of my claim at this time, and you are welcome to not believe me.) Most of the remaining 20% are optional game modes I could choose to cut (and may, for gameplay reasons).
I do not contend that the code is excellent. But to whatever extent the code is "slop", that has not meaningfully inhibited the ongoing development or caused noticeable issues with the player experience.
The game is not especially ambitious, but it's something I would have been unable to create without genAI. The version of me that could have made it without genAI would have taken > 10x as long to get as far as I've gotten.
"Shipped" is a bit strong in my case, but I think this counts as something that isn't mere slop.
I think it does have pretty good value when you are your own customer, because you can make exactly what you need. For instance, I'm using it to make a web playground in which I can test the board game I'm designing, instead of asking people to pay for, and install Tabletop Simulator. I'm satisfied. I can tell the code sucks without even looking at it. But I wouldn't have had the time to do this myself, the token cost isn't prohibitive, and it works fine.
As an example consumer 3D printing hasn’t had much impact on manufacturing
Desktop 3D printers are tools; I know many people using them to support their making. And they've utterly changed manufacturing on another level (simply because of their ability to surface a prototype).
It will get better and better and better and still never replace mass production, but mass production will be changed by it.
Not least because of the fundamental shift in availability of the tools to make tools. The productised desktop CNC market is wild and it owes almost its entire existence to the change in the market driven by 3D printing.
In almost all cases right now, a subscription with a cloud AI wins.
What sort of benchmark or test would convince you that "almost all" is more like "your weird set of cases" and nowhere near "almost all"?
Spending thousands of dollars on hardware to save dollars per month on tokens does not make financial sense.
If you run the numbers you'll probably find that using cheaper cloud models makes more financial sense than running those same models locally.
But I currently think the fundamental reason the cloud models often make more sense is based in how the models work. My current understanding is that once these models are loaded into GPU memory, the model can respond to multiple prompts in parallel. So a cloud vendor that receives lots of user prompts at the same time can achieve really good hardware utilization. These cloud vendors can then spread the cost of the hardware over many more users than you can with a local model, leading to lower prices per prompt in the cloud.
The best way to save tokens is to start out the deep research pass with cheap models and then funnel the findings through increasingly powerful models. I've got a pipeline right now that uses all 3 of the gpt-5.6 model variants to address each stage of the process. If you are using models like sol or terra to generate hypotheticals and perform initial exploration, you are leaving money on the table.
5 hypotheticals out of luna will massively outperform 1 hypothetical out of sol, but the cost is the same and so is the runtime if you do it in parallel. The hypothesis generation phase is also a great place to mix and match models from different vendors. The more diversity at this step the better.
The other thing I started looking into is batch pricing which represents 50% off for OAI tokens right now. With some tweaks to the UI/UX of an enterprise chatbot, I think it is possible we could have users get comfortable with the idea that questions to the robot might take a day to come back. The key is that this has to actually work. Users don't mind trading time if their questions come back with high quality results.
You can use Big/Cloud LLMs to help you "find good enough configs" for your local/small llms [1] and stay quite nimble in the face of rapid change.
- [1] https://alexhans.github.io/posts/find-the-loop-story-first.h...
It has now become a way for agents to converge on complete, shared answers for emerging issues. As a side effect, through posting open issues we can also identify gaps in model knowledge.
Claude has been helping me tune models to run better on our big compute rack at work, researching which leading edge open models will fit in hardware and are good for our workloads, and it's been helping me write test fixtures to evaluate how things perform.
At several points, Claude has expressed surprise at the quality of results from my local LLMs. While I know that doesn't mean anything, it still feels like giving Anthropic the finger, which is always great.
https://neuronowa.pl/posts/porada-jak-wydajniej-kodowac-w-cu...
It is Polish so please translate. But tldr is: ask your agent at times to analyze last chats and update rules and skills to not repeat the same steps it repeats every time when you start a new thread. Simple as that.
And now that you have typed some words into a chat box to produce a thing, you are confused about why other people won't pay money to you for the output of the chatbox, instead of typing the same thing into their own chat boxes.
RTFA - it's about using tokens to research saving tokens.
I'd bet this only really work 2% of the time in real world.
LLMs only have 'knowledge' that was encoded by scanning as much data as possible. (Same way a search engine only can find data that was indexed)
American LLMs are by and large closed. Chinese are open to a point. And research how these things work is still a big mystery.
So yeah, was comprehensive data about saving tokens scanned and indexed? Likely no. So engaging about token saving is going to generate a lot of verbose and useless slop.
(Without the learning and growth potential)
What I've seen with lots of the breathless 'OMG SCIENCE ADVANCEMENT' articles including the one yesterday, is that the LLMs are quite extraordinary about linking a dozen different fields together, and delivering an answer combined from all of them. Basically, the solutions to a lot of current scientific problems are partially solved, but partial from a lot of fields that don't talk with each other. And also, they don't use the same nomenclature, so simple searches don't suffice.
An LLM seems to use Chomsky's Universal Grammar, and thus on training, does normalize all training data, including different words for the same thing.
A lot (perhaps all?) Of these BIG scientific advancements were solved by humans, partially. An LLM jigsawed them together and appeared it solved the thing.
Of course, you have no way to he absolutely sure, cause Anthropic lies about silent downgrades, and charges you as if they didnt. And the 'realignment' rules are kept hidden, so you can't even avoid them intentionally.
I found that now that the models are better, at least Codex compaction with GPT 5.5+ xhigh is almost unnoticible. So the answer is: just let the context fill up, don't prune much, the model is smart enough. When you're out of space, have the model summarize and continue from that point. A smart model will know the information pieces it needs to re-read after compaction; it won't make incorrect guessed when some details got lost.