ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
57% Positive
Analyzed from 3139 words in the discussion.
Trending Topics
#data#demand#more#models#centers#years#prices#those#center#token
Discussion (70 Comments)Read Original on HackerNews
I mean the physical hardware will be fine but the owners and investors should be worried then right
If the cutting edge OpenAI token prices are $80 per 1M token, and the open source tokens are $1 per 1M token, that's a huge gap of "this will never be able to make money under any scenario if the bubble bursts" that will catch a lot of these new datacenters. No one will run a datacenter that costs $5 per 1M token to sell at $1 per 1M token even if the debts are cleared.
That $5 per 1M token doesn't literally cost $5 per 1M token. It's more like they had to build a datacenter for $500M that can service 100T tokens over its lifetime. They did this by borrowing money on the capital markets, and now they have to pay interest to those bondholders, interest that they can recoup with their $80/1MT prices. But if it turns out they can't charge $80 and have to charge $1, they won't be able to make those interest payments. They enter bankruptcy, the court wipes the debt clean, and now they don't have to pay interest, only the actual operating costs, which may be more like 50c/1MT. The company gets recapitalized with the new owners being largely the bondholders, the existing equity holders get wiped out, and they can compete with the commodity producers now.
You have land taxes and or rent, building upkeep, staffing costs, electricity, water, hardware replacement costs.
And new build DCs have blown all these costs through the roof justifying the decision because the price of compute is so high. When the prices come crashing down, the expenses will remain fixed where they are now.
https://www.reuters.com/legal/litigation/openai-ipo-will-not...
They could potentially do another private bridge round, but for a company that was gearing up for the largest IPO in history a couple months ago, the reversal is a pretty bad sign. For investors that are looking for a fire sale, there's already smoke in the air.
What do you mean the "first" big? 1929? 2001? 2008?
Do you mean 1929 wasn't a big financial crisis and that, this time, we'll have the first "real" big financial crisis?
I'm confused.
National debt through the roof, inflation through the roof, PHD and research programs gutted, non-ai startups dead and unfunded for the last 4 years. These are just a few things that have been sacrificed on the altar of this bubble - there's far more I haven't recounted.
We're already in a widespread long term economic collapse, but the delusion just hasn't broken yet.
This tracks the evolution of how we use cloud computing and GPUs over the last 20 years. Cloud computing was originally just about saving companies from needing to maintain their own server racks, but has unlocked previously unserviced demand by allowing people to build an app or service and scale it to meet rapidly rising use without needing to invest a ton upfront in hardware. Suddenly a hobbyist could spin something up in their spare time that previously required many thousands of dollars of investment. Or when someone wants to run a single large scale computation, they can do it without needing to waste capital on maintaining idling servers to meet an occasional demand spike, like when a company I used to work at moved from running atmospheric calculations on a server in a closet to the cloud and were able to achieve double digit accuracy increases with the increased scale, while spending less overall on batch computing jobs.
GPUs, as the name implies, were created for graphics, and primarily for gaming graphics, but then more or less accidentally ended up enabling the present AI boom, which depends on a scale of computation that would have been impossible with older CPU architectures. Maybe someone at some point predicted this, but I think for the vast majority of people, it was extremely surprising that a niche gaming product would enable an industrial revolution level technological leap forward.
LLMs are just one way that increased compute scale unlocks seemingly magical results, but they are far from the only example and I have no doubt that there are many unknown examples remaining to be discovered yet.
New datacenter builds are so far along the curve of diminishing returns it's absurd. No one is going to want to pay to run a datacenter that costs 10x as much to run for the same compute.
And the problem is with all these "freed" resources, the entire pipeline will be affected. No one will want to buy any new silicon if they can buy a B200 for $1000. We could potentially see a decade or more of stagnation in the chip sector, or even significant regressions in capabilities as foundries are shut down due to lack of demand.
The impact of what is coming scares me to my core. I don't think we're going to bounce back from this any time soon.
A lot of investors may lose money as a result of the bubble bursting, but that does not mean the underlying asset will be forever worthless, just that it didn't provide sufficient returns sufficiently quickly to justify the upfront investment for the investors that funded it, at the time they made that investment. A different investor who could afford to ride out a period of reduced demand might have an entirely different experience.
Imagine you take out a five year loan to buy a truck to deliver packages, but then for the first two years you operate it, gas prices are elevated so you struggle to make the payments on your loan and end up not making as much money as you had hoped to originally, perhaps even to the point you need to declare bankruptcy and sell of the vehicle. But if not and then gas prices drop back down and demand shoots up for the last three years of the loan, you could then make up the difference. If the first two years drive you into bankruptcy, that is difficult for you, but amortized over the entire five year loan period, the truck may actually have been a profitable investment for someone who could have afforded to ride out the first two years. And just because you go bankrupt doesn't mean the truck stops being a valuable asset, it's just that you don't end up benefiting personally from that value because your timing was bad. From a macro perspective, the overall economy doesn't suffer, except to the extent that it might have been more efficient to invest the capital that went into procuring the truck elsewhere during those first two years. But only possibly and only on the margins, because the truck remains a profitable investment over the course of its entire lifetime.
Don't confuse the success or failure of individual investors or businesses with the success or failure of the overall economy. Current data center build outs premised on fanciful projections of demand for LLMs may end up not being profitable in the short term while still being profitable over the entire productive lifetime of the asset, if sufficient demand is found elsewhere or if demand for LLMs picks up later. Similarly, I anticipate at worst we will see chip prices plateau for a while if there is a pullback in LLM demand, but we won't see them fall and they will continue to rise over the longer term as more demand is generated elsewhere.
As one small example: we have barely begun to scratch the surface of what we can achieve with robotics. Think about the demand for video processing if you have tens of thousands of robots stocking shelves in supermarkets generating video all day long. On board processing will of course be the obvious primary demand for chips, which doesn't benefit data centers, but central processing of video to extract useful data from the entire fleet will generate demand for data centers. As will large scale training jobs. Now multiply that thinking across the entire scope of industries where robotics may be useful for replacing human labor, and you're talking about an extremely significant amount of valuable computational work.
Anyway, it's clear an AI data center has utility for crunching AI inference, and that there is and will continue to be demand for AI inference. The trillion dollar question is whether you can make money doing this, and so far the answer is no.
https://isaiprofitable.com/
something has to true to be surprising. your statement is false and nonsese
Gas turbines are all practically sold out till 2030, delivery time changed from 2-3 years to 5-7yrs.
Many announced data centers where destined to delay, independently from financial markets, due to the available infrastructure.
If the AI/data centers fomo dissapear, the effect on electrical production will take years in take effect, because capacity is already reserved and paid.
In oposition to software, energy infrastructure is not flexible: you can't speed up a turbine projected for 2032 neither cancel the order without a considerable cost.
https://pastebin.com/bBXbpDyu
Here's the chat template I use:
https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
Seriously, do you believe the garbage you wrote?
I am a software engineer, and using this software in my personal and professional work has lead me to a very different conclusion, but everyone is entitled to their opinion.
Also, Google and Facebook are still spending like drunken sailors. Nobody has stubbed their toe on hard limitations yet. So yes of course people will figure out how to optimize the cost of AI in their products. Just probably not this year.
Let's also not forget that even the open models are not getting smaller, they are getting larger. Of course, you can distill them down into something that will fit on smaller compute, but at the end of the day, the data centers of compute, still play a huge role.
There will be people who want to host things on-device. At some point, you could probably do most day-to-day tasks with a Siri-like agent, so you don't necessarily need it to be on a datacenter rack somewhere.
More complex tasks being run quickly opens up a choice: insanely beefy individual devices, on-prem hosting, or cloud hosting, whether that be some data center running FOSS models, or ones from people like Anthropic or OpenAI.
Beefy hardware for individual users? Not cost-effective. Could have people share that hardware by putting it in a data center. Do you want to operate that data center? For proven business cases, sure, why not? If you're still working out what your scale will be, maybe you ask the Googles, Amazons, or Microsofts of the world to rent you the hardware so you don't have wasted or too little capacity.
The real question is, how much value is there in a few companies that talk about how their eventual goal is to create AGI as opposed to just giving you enough intelligence to augment your current workers?
The answer is "probably not enough to justify more than one company having a valuation of over a trillion dollars, and that's generous".
This just doesn't pass the smell test.