The Hiddn Financial Bubble in AI Infrastructure [pdf]

And as models shrink in size yet go up in intelligence and performance, I'm finding ever more life in older hardware.

When I got my M1 Max in 2021, GPT-3 was about 1.5 years old and it was SOTA.

Yet, that machine is now able to run models that crush with gpt4, and even compete with o1 (SOTA from about 1.5 years ago.)

The idea that I could run something like that locally would have seemed absurd in 2021.

Yet, if somehow I'd had those local models in 2021 on the exact same hardware, I would have had, by far, the most powerful AI on the planet -- and that would have remained true for the next several years.

I'm also noticing that the ever-improving smaller models I can run on this machine are crossing the "good enough" threshold for ever more tasks by the month.

I just don't need a frontier model for every task.

I have an M4 Max 128 GB RAM now, but I still find plenty of tasks to delegate to the M1 Max machine.

I don't know how far this can go in the limit in terms of packing more intelligence into smaller models, but older hardware, if maintained well, seems like it's going to increase the value it can deliver in terms of "intelligence per watt-hour."

I'm CEO of an AMD neocloud. Confirming this is a myth.

https://x.com/HotAisle/status/2045181374030856300

Sourcery Show HN: https://news.ycombinator.com/item?id=47996426

The project is currently private, I'd love to have access to its source.