What Happens When the Cost of Intelligence Drops 100x

> But the costs didn't come down because they were able to add more lights and larger displays.

People are a gas; they expand the fill the space they're in. If you give someone a big house, they'll fill it with crap. If you make food cheap, they'll eat too much, even when it harms their health.

Making things cheaper usually just makes making them more prevalent. It's why computers are not faster than 20 years ago. They're merely more capable -- developers quickly and aggressively fill up (and overflow) all that added capability until you're back in the same place you used to be.

Jevon's Paradox says nothing about the size of the incremental demand though. Lighting is a great example. The US went from spending 15% of our electricity on lighting to ~4%. LEDs are maybe 5x as efficient, so that would be 3% without any change in usage. So we did see maybe a 30% increase in consumption as a result of lower prices, but that was nowhere near large enough to offset the efficiency gain in terms of overall consumption. Las Vegas may have, but that is a very small part of overall usage.

I anticipate we will see something similar with intelligence. There is probably headroom to consume 100x as much intelligence in R&D. But that isn't most of the economy. Will run of the mill service jobs increase their use of intelligence by enough to offset the effect of cheaper prices? I think that's the real question.

Definitely not going to argue against the Jevons paradox in general, it is observed in various cases.

The lightbulb thing seems different though? Or at least it is a specific subset. Lights in Las Vegas are sort of an advertisement, right? In the sense that having the brightest or most interesting (or whatever) lights draw attention to your show, casino, hotel, whatever. It’s kind of a zero sum game in that the different shops are competing for the finite attention of a more-or-less set number of tourist. I think part of the Jevons paradox is that society generally finds more useful applications of the newly cheap thing. If the thing’s only purpose is to compete better in a competition with a set prize (all of the tourists’ money), that’s constrained in some way.

Intelligence is weird though. I guess we could eventually hit the point where, I dunno, maybe there’s some information theoretic bound where we process all of our signals as cleverly as possible and aren’t bound by intelligence anymore. Obviously we’re nowhere near that. It would be a very alien environment.

> I think the same will happen with tokens. As the cost of tokens comes down, these models will just consume more tokens.

That sounds a race to the bottom for AI companies profits.

I do not think that LEDs are high profit margin items.

Tokens for sure because we still waiting for agents talking to agents as a default. Like your agent team and very long or 24/7 running agents.

But we never scaled intelligence like this. The industrieal revolution created for the people at that time quite a huge issue / it was disruptive.

What will hapen to us though?

Sounds like a variation on the induced demand principle: https://en.wikipedia.org/wiki/Induced_demand

Related:

- Parkinson's law: "Work expands to fill the available time." https://en.wikipedia.org/w/index.php?title=Parkinson%27s_Law

- Lewis–Mogridge position: "Traffic expands to meet the available road space." https://en.wikipedia.org/wiki/Lewis%E2%80%93Mogridge_positio...

And I pretty much just plain agree, this is exactly what will happen.

I don't think there's anything wrong with it (in isolation) either, though I do already find myself pointing out that we're misusing LLMs at work sometimes (most notably, a recent mini project could have been a jinja template - and it did become one thanks to me pushing back on this). Abundance is one thing, waste and misuse is another.

Turns out Grey Goo was just thermal paste

Douglas Adams captured this in "Dirk Gently's Holistic Detective Agency", a VCR watches TV for you and an Electric Monk believes things for you to save you the effort of doing it yourself.

Hasn't the computer already been spending time with your wife?

The next generation of bike computers will probably have links to an LLM cloud subscription service for real-time coaching and route planning.

Futurama did it first!

> they’re too slow to work with interactively for many use cases

This just demonstrates how much we already take for granted the LLMs that we have now. If you compare it to what we had before (hand the task off to a junior dev and wait for them to complete the work) then it doesn't seem slow at all.

Given the 750 tok/sec GPT 5.6 Sol Ultrafast (via Cerebras), the many-1000 tok/sec Chinese models, and the 15000 tok/sec Taalas HC1, I think we're well on the way towards seeing that solved too. Combine the two, and yeah, wild ride incoming.

What's especially bewildering to me is that translated back to raw bandwidth, even 15000 tok/sec is just like what, 75 KB/s? Extremely meager amounts of data, moving mountains.

It's already kinda funny seeing LLMs throw out effort estimates in wall time terms. It's always some "hours, days, weeks" tier thing, when in reality, it's gone and done in minutes.

Sensors are a huge challenge for robotics. We have very precise force-feedback on our joints, pressure and heat (temperature gradient) sensors all over our body, and our hands have a sensor density that allows us to count needle heads and detect the exact grip strength needed by feeling the micro-slippage of objects in our hands. Robots don't have that.

You can do backflips with pretty much just visual sensors for your environment, a good IMU for your spatial orientation, and some feedback on the position of a small number of really beefy joints and the force exerted on them. Folding laundry and opening doors is much more difficult, and trying to compensate with mostly vision requires going slow enough that things have time to move over appreciable distances before you take the next adjustment

Xiaomi robots do this in double digit seconds.

Still slow compared to humans, but Chinese robots will be as successful as Chinese EVs, phones and solar panels.

You must not have been paying attention to development with robots, there are many videos of robots moving really fast in "non controlled environments"

Wait til the robots get on Cerebras, it'll set your pants on fire.

Have LLMs improved at being able to process physics-based problems and environments? I remember that issue being discussed around generative gaming a while ago but I hadn't heard much about it recently.

Brings up the question of what the intelligence is used for. Humans exploited intelligence for competition. With each other to wipe out other Homo species, mate more and collect resources, with other animals to limit predator impact and gain food. Intelligence will be used offensively by corporations and their people to extract more from consumers (make pricing opaque, terms of service more complicated, etc.) and scams far more sophisticated. The "consumer" will need extra intelligence to fight all that off. There are only so many meals you can expertly produce, shirts to fold, itineraries to fun places you can execute, but there's a practical infinity of traps to set and avoid.

And why won’t the frontier models continue to become better? The open models are getting better but so are the frontier models. The frontier models might remain in a constant race to remain ahead

You're betting on getting getting ridiculously powerful chips to run on batteries in a tiny housing without cooling, while we can't even get enough RAM? It would be a terrible waste of resources. Now we already have TFLOPs wasting in our pockets and backpacks, then we'll have PFLOPs idling, because there's so much time between prompts. Much more efficient to batch it on a server.

I saw a 1B model yesterday that was fine tuned on Fable output. I found that hilarious, but it did actually make all the scores go up.

(Actually talking to it, it was about as coherent as you'd expect, i.e. 3/10)

The floor for "actually usable model" keeps dropping though. (Seems to be about 27B right now?)

Sooooo what does it matter if we do 99% of things a LLM can just solve as a 'advanced search engine'?

Btw. an advanced search engine is probably the worst comparision i have read so far.

A LLM is a latent space which is capable of a lot of things a search engine can't do. It can apply different type of patterns and flows onto data, it can combine these etc.

My 'advanced search engine' was just able to create a working PR for exactly what i wanted it to solve (fixing a bug) by analysing the bug, finding a valid solution then commiting the solution itself.

You're correct on the technology assessment.

But you forget all development today is just searching for a template, copy pasting, changing some small things. And a smart-ish search engine can do all that.

I think this viewpoint fails to understand what "intelligence" is. The idea must be that intelligence is some special thing that only humans have. So when machines couldn't do jack s... we said "it's the Turing test". When machines blew through the Turing test we said "that was just prediction..not really intelligence, that's different".

It's not different. The delusion humans have is that intelligence is special and magical. It's not. It's just nature's prediction machine. A very fancy version to be sure. But not qualitatively different .

All statements that "oh but it'll never be able to do that" will prove false.

How do we know inference is heavily subsidized? Are all these third party Chinese model providers subsidizing the true cost?

The other day I saw a benchmark of coding models that fit in 8GB VRAM - for reference some version of Mistral was added, normally requiring 32GB, but moving along at 4-5tok/sec when partially offloaded to CPU.

Surprisingly, some of the small models would not only give worse results, but also took longer than Mistral, because they were thinking so much.

That is an important detail which I was previously overlooking.

Whats your problem?

Even if its not intelligence, a LLM found a bug due to one error message, fixed it, created a PR and it solved it.

If an LLM is only able to do all of this after training on it and never achieving AGI, we already at the point were it is cheaper to teach one LLM one problem than teaching humans to do so.

AGI ≈ Ralph × Infinite persistence

Just like real life!

becomes rampant? Where have you been the past decades? Hell, centuries and millenia?

If we're going cynical, may as well go full throttle.

The animated Pareto front graph by time is truly inspired. Well done!

It blows my mind how fast models are getting better and this is the first article I’ve seen that shows just that and leaves almost no room for disagreement. Well done.

Thanks for the effort you put into this.

I wonder if it might drive the point even further if the graph scales were linear? Or maybe the progress has been so great that this would make the graphs unreadable?

https://xkcd.com/1162