Companies are scrambling to curtail soaring AI costs

I wonder how it's possible for a developer to assign profit. The article mentions Uber's $1500 limit per developer per month. At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour. Others don't use AIs very much. Should those not using so many tokens donate them to the crash dump people? And back to your point, how can we assign a profit to this? Customers love having their crash dumps analyzed quickly, but that's not the same as it being profitable.

It is just extremely difficult in most companies to draw a line between a specific feature and an amount of revenue. A lot of engineering has a stochastic type of impact.

(It’s often easier to see what a feature does to reduce costs though)

I don’t quite understand what you’re suggesting.

I looked at gptel and eca about a year and a bit ago… how far have they come, at the time they were more geared toward code completion and essentially a buffer for communications - but all while not having some kind of Ralph Loop, Skills, and auto-compaction.

Is this all built in these days so that you can Superpowers a prompt to the point where it will automagically chug along for hours until it’s satisfied it’s completed the end goal?

And, if you try it out and find that for your particular circumstances there's no level of spend on AI that's giving you positive ROI, you can just...not do it.

It seems like execs these days think that if they're not doing the One Thing Everyone's Doing, they're necessarily missing out, leaving potential revenue on the table, what have you. But the only reason to use AI in a business is if it makes you more money, so if you try it, and it's not doing that...I guess the question is, who are you going to believe, Sam Altman or your lying eyes?

Yes, according to this:

https://x.com/SemiAnalysis_/status/2064815044085318040?s=20

also, codex apparently uses fewer tokens to complete tasks, in part because their tokeniser is about 80% the length of that in tokens for the same string as that of the latest anthropic ones. This might be fable only though.

I have both the Claude and ChatGPT entry-level business plans, and I feel like I get more use out of Claude, and that's even with some Fable and Opus use.

Same, however managers are starting to ask questions, like 'what can we do to help remove bottlenecks' (they think sprinkle unlimited tokens -> revenue multiplies). In actual fact what happens is devs benefit.

It's like the company gave every senior dev a super eager brilliant fellow coder to do their work for them so they can 'do more important things' i.e. Netflix and sleep.

'Yes still working on this ticket' When codex/cc completed it flawlessly last week. The upside is absorbed by the devs knowing they could be soon surplus to requirements.

At the moment a lot of companies thinking of it like speculative capex - we can just stop doing it if we don’t get results, or we can easily optimize.

My concern is that with all the bundling that the model labs are doing the lock-in becomes harder than anticipated to unwind