ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
71% Positive
Analyzed from 4405 words in the discussion.
Trending Topics
#more#don#using#cost#same#code#going#llms#leaders#run
Discussion (119 Comments)Read Original on HackerNews
And I'm seeing almost no self-awareness from leaders. They are making decisions about things that they just don't understand. And are completely unworried about it. Just blindly following whatever the news cycle is about AI.
What he says he's consistently hearing from them mirrors what I saw at my own employer: they thought they had ROI metrics, but they actually only had usage metrics such as "lines of code committed" or "number of pull requests". The only way those could possibly work as an ROI measure is if your business charges customers by the line of code.
As I was saying, you're all fired.
Don’t play their game and call them leaders. They are management, bosses, executives.
> They are making decisions about things that they just don't understand. And are completely unworried about it.
Clowns, even.
> Just blindly following whatever the news cycle is about AI.
But followers might be most apt.
——
This is such a huge pet peeve of mine. Describing management goofs using their language that makes them sound all-so-brilliant. We constantly watch these people do the dumbest shit and then they go around describing themselves as “thought leaders” and “servant leaders”. When, really, most are just clowns with fragile egos.
And, while I’m rambling, they’ve tried to take away the fact we are workers by calling us individual contributors. Using language to attempt and hide the hierarchy and power dynamic at play. It just…bothers me so much.
And many of them still claim they are "risk takers", but have effectively insulated themselves from risk by socializing losses.
As a leader, pushing for rapid change cannot really be nuanced lest the push dissipates into the organization's entropy.
It's irrational to push for tokenmaxxing (literally "please increase our AI spending") and not expect that this is the result you are going to get. You won't get productivity increase, since that is not what you are pushing for - you will get token usage maximization (engineers running inane agentic tasks against your code base to increase usage, using company paid AI for their side projects, etc, etc).
Why be a normal guy that waits to see what happens and is measured and pragmatic when you can get attention basically through the whole cycle by being the earliest adopter, adopt it to the maxx, then also be the loudest big brain when the tide changes and be praised for "taking hard decisions" when you revert everything you said so far?
The fakemaxxing economy.
Understanding this was one of the most important things in my career.
It would only be laughable if they waited way too long to reverse course, but I don't think that's the case.
How much that makes it into enterprise pricing is TBD, since none of the hyper scalers are making money yet of selling AI inference.
Almost all businesses are ahead of the gun. For most of their use cases, AI is either not yet good enough on its own, or good enough but too expensive.
No one wants to get left behind, so everyone's trying to get onto it now, even though it's not ready for what most enterprises want to do with it.
It's easy for them to look at a small startup without billions of lines of legacy business logic debt and see them having success and wonder why they can't have just as much - or more - why they're bigger so they should have better and more success, right???
Wrong...
But when it gets ~99% cheaper for local inference over the next 4 years, at the same time the price per watt improve 4x -> a lot of those cases will start to pencil out.
The Chinese, since they lack computing hardware due to US export controls, are.
Do you mean the marginal cost by the producer, or the cost on the consumer? I can't see the price of electricity falling much, and the demand curve is apparently exponential if the hype is to be believed.
Computing has always been about how to wring out more efficiency. The ENIAC was 150,000 watts, with 3 phase 240 volt power, and cost about $500,000.
My day to day laptop (a year old) is 35 watts, with 1 phase 20 volt power, and cost $1,000, so that's 99.98% less power consumption, 99.8% cheaper, and it has about 10 orders of magnitude more computing power, all on a time span of 80 years.
And the technology already exists on the algorithmic front TODAY to lock in another 10x gain -> when, typically, algorithmic gains only account for ~30% of that drop and the other ~70% comes from better data (often synthetic) and knowledge distilation from frontier models.
Just look at DeepSeek's pricing...
Historic trends, every 18 months, performance for the same level of quality has gone down 90%.
See: https://www.reddit.com/r/LocalLLaMA/comments/1gpr2p4/llms_co...
And Chart 13 here: https://www.rdworldonline.com/ais-great-compression-20-chart...
And here: https://epoch.ai/data-insights/llm-inference-price-trends
Historically, algorithmic gains are only ~30% of the pie, but there's enough out there to get to 10x, with just what's available already. The other ~70% of the pie is better training data (often synthetic) and distilling frontier knowledge. There's no sign we are tapped out on that front.
Additionally, GRAM (from ~10 days ago) is likely to be a 5-10x on its own (if not substantially more for smaller models). It's unlikely within 4 years LeCun's JEPA ideas and similar ideas like GRAM applied to LLMs have ZERO impact. The preliminary results are absolutely astounding (5000x better reasoning - this is not peanuts).
Further, that's not even counting that cost per watt is still dropping ~2x every 2 years on its own on the hardware front.
If you look at the "cost" of inference. People think it's electricity - but it's currently almost ~80% hardware amortization. The memory shortage is not going to last, nor are Nvidia's ~80-90% margins.
The human brain is still 8-10 orders of magnitude more efficient than the best LLMs of today. With ~1/10th of global capex riding on AI, if you don't think they're going to knock of 2 orders of magnitude more, when it's this obvious and easy... I don't know what to tell you...
Sure, it might take 6 years instead of 4. My crystal ball isn't perfect.
People are willing to pay more for BETTER quality.
You obviously haven't seen DeepSeek v4 Pro's pricing if you think pricing only goes up...
Then buy $10 (or $2, if you're cheap, and they take PayPal) of DeepSeek credits.
Whilst you're at it spring for a Claude subscription too and GPT.
Switch models between Qwen, DeepSeek Flash, DeepSeek Pro, and you can meet 99% of your code generation needs.
Hop over to Opus 4.7 (or 4.8, but I haven't really used it yet) and GPT-5.5 when doing very complex architecture/design or troubleshooting something where DeepSeek Pro is getting stuck.
It is ridiculous how cheap this stuff is now. It's affordable at third world prices.
If it was so good I would expect to see 2005-2015 advancements yearly.
Meanwhile China is blowing past the world with real improvements in the real world- solar, EVs, etc. meanwhile people keep making their fancy sans serif websites about todo apps, faster than ever before. Useless.
I don’t disagree that AI is overhyped. But I think you are probably looking in the wrong place.
I think most software that is written isn’t really a product, at least not a public product. It’s an in-house tool or a one-off project needed to complete some larger task. People everywhere are always writing small programs that make their life or job just a bit easier (and explains why so many corporate projects are little more than an excel spreadsheet).
And there are a lot of people who have made custom software just for themselves with AI. Not a product, just a tool or project that finally made sense to build.
Very little about the American economy even makes sense for keeping the edge on LLMs beyond a few years. All the things I would think would be required: energy, research, construction capacity, labor costs -- it's pretty hard to deny who's on the upswing these days. China cranking out current generation microchips will be the last nail in the coffin.
I would agree that a lot of companies talking a big talk about using LLMs are failing to actually apply it in a sensible way to their business.
For example. Imagine that you are comparing two documents (let's assume diff doesn't exist). You could ask an AI to compare the differences from you or you could use AI to write a tool to do it. For whatever reason, people are starting to go with the former not realizing that now they basically have to pay to compare documents.
My answer: no, but it was able to help me find the website and social handles for every beat writer for every team, and generate a simple website where I can do a daily skim of teams/players and draw my own conclusions.
LLMs are a tool, not a panacea.
It just requires being willing to think instead of mashing prompts into a keyboard.
Normal people have never gone around automating their work. The most automation they do is dynamic tables on excel sheets.
I obviously know building a tool that can programmatically do something is a better solution, but I think that requires a fundamental shift in how people work. People need to be told by someone "this is how you should be using the AI" but right now they're simple told "use the AI".
I understand and agree with your point though.
With this AI is a fallback and not the default. Sounds like large companies have it backwards.
Here we have the opposite: In the land of the one-eyed, the blind are leading.
The blind in this case are all those executives and managers who don't understand much about AI's current potential and limitations, and so far have treated it like a magic button that will solve everything. The one-eyed are rank-and-file employees who maybe sort of know a little more about AI.
We are very close to the point where if Claude and ChatGPT APIs are down, companies cannot function. How is that introduced so quickly into so many critical places without taking that specific fact in consideration? What is the plan for all those companies whose workflows now depend heavily on a remote LLM whenever the services get cut? What if your company account gets banned?
In some ways it is worth than depending on a company for hosting, because even your debugging tools are based on AI. MCP is great to go through datadog, sentry, until your agent or the MCP server are down and you don't know how to look for the issue yourself because you do not actually understand how your systems work.
Between corporate FOMO and the rapidly decreasing costs of actually running LLM's I'm interested to see at which side of the spectrum these two meet
90%+ of corporate people are not programmers. 1 programmers can cause the same token damage with a bunch of concurrent agents as a couple thousand Karens in compliance asking a chatbot questions
It's much easier to deliver incremental AI ROI on the later even if it's hard to measure/quantify. A 1000 tokens might point this compliance person in the right direction on a key problem. Meanwhile 1000 tokens doesn't get you anything useful on coding
Only thing I can say AI was useful for, in a corporate environment, was learning a new coding language on the fly. Gives me a baseline to work off of and fix.
But I can learn without it, too. A nice tool, but not a need.
An ironic analogy sort of, once media started hiding behind a paywall, I just stopped reading them rather than paying. Same with LLMs - usable if cheap/free.
If LLMs are genuinely helpful or even decisive in a military engagement, you can expect any host country to commandeer whatever data centers they need, leaving commercial entities to bid up the prices on the leftover capacity.
Another risk is that data centers are a great target for cyber warfare.
It’s ideal if your business can leverage LLMs when they’re online but continue to operate profitably when they’re offline.
In other words, the news cycle is looking for an AI story that lands with readers, and that the example of Uber blowing through its AI budget and Microsoft discontinuing use of Claude internally are not good indicators.
I agree that those aren’t good indicators.
However, at some point we have to remember that CEOs and boards of directors are just regular morons who read the news the same way everyone else does.
At some point, if a lot of corporate leaders associate AI with mediocre results, high costs, and public backlash, they might just start saying “this juice isn’t worth the squeeze.”
https://news.ycombinator.com/item?id=48268871
https://news.ycombinator.com/item?id=48238896
https://news.ycombinator.com/item?id=48307098
I asked codex to take a look, and it:
- Grabbed the CI logs on its own to figure out what the CI error was
- Looked at my local setup
- Looked at the changes in sqlx from 0.8 to 0.9
And figured out that sqlx depends on an updated version of the “whoami” crate but doesn’t specify default features, which causes it to fall back on a stub implementation that makes the default user “anonymous”, which was failing to authenticate to the UNIX socket we use in our CI Postgres server. It patched the environment variable for our docker container to explicitly specify a username and the issue was fixed.
It would’ve taken me probably several hours to figure this out on my own. It took codex maybe 5 minutes.
Tell me again how LLM’s “don’t work”?
I'm not taking a shot, to be clear, we had a similar issue a few years ago and we made sure this wouldn't happen again, that's absolutely not a shot, nor do I think it's a character flaw to use AI, au contraire, this is a very good use. I'm just worried that because AI is so good at fixing minor issues caused by governance/organisation flaws, we will be stuck using it to fix those and be trapped in mediocrity (that's not an issue for me, mediocrity is where I work best, but I'm a bit sad for the great Devs I've worked with.)
It’s not in the changelog though, this is an update of a transitive dependency that inadvertently changed the default behavior. sqlx didn’t document this because they didn’t even know it changed.
Even if it was a documented change, our process caught it because it was caught by CI. The issue itself was only a result of how our CI was configured (we had a database url with a domain socket path that didn’t explicitly specify a username, and we inadvertently relied on the default of “the current user”, which the whoami crate now defaults to “anonymous”.) I don’t see an issue in our “team process” (whatever that means) at all.