Once Claude can measure something, it can make it faster

124

mmatthieu_bl about 3 hours ago 76 commentsRead Article on claude.dev

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

53% Positive

Analyzed from 2289 words in the discussion.

Discussion (76 Comments)Read Original on HackerNews

smy20011•about 2 hours ago

The way Claude did it is fight entropy with entropy.

"Add a static composer into the HTML" <- This seems like something can be done with SSR?

"For faster navigations, we kept the composer mounted between conversations" <- Your SPA should cache this between pages, why fetching it every time? Or you need better routing for your react components.

"cheap first-character check before the regex" <- Should we cache compiled Regex instead?

I think even 1.3 sec to load the front page is unacceptable. Something need to be reworked from basics (SSR, chunk-based rendering) to solve the problem. Focusing on invidual benchmarks may miss the opportunity.

crooked-v•10 minutes ago

Even Next.js, for all that people are unhappy about its quirks, complexity, and random undocumented behavior and bugs (God help you if you ever want to try and actually use parallel routes as documented), can do all of this SPA stuff out of the box. Use `<Suspense>` appropriately on data loading, and everything static (including e.g. purely input-output components like editors) will load once and be re-used forever, and your Suspense-wrapped items will show a loading placeholder and only re-load when you intentionally re-trigger the data loading (which you can push down all the way to the level of individual buttons if you want).

rustystump•about 2 hours ago

The amount of complexity added for the gains is depressing. I am confident a human and about 5 minutes with chrome debugger would yield better results with a fraction of the complexity at a fraction if the cost and in a fraction of the claude baby sitting time.

Reading this shows the authors have a profound lack of fundamental understanding on how to effectively optimize in the web domain.

This isnt claude being bad but how wild it is watch people from the cutting edge of ai brag about pretty mediocre gains.

adamddev1•about 1 hour ago

When people talk about AI being able to handle everything I keep wondering, have these people built anything complex, novel or serious? Just because people can see a website or a simple app improved, does that mean that all code can be handled by LLMs? It's like people are totally forgetting a whole category of careful, well-thought out programming for the critical parts.

whatisthiseven•about 1 hour ago

Yes, what you are seeing are amateur developers that barely understand the tools they are using either giving LLMs poor instructions, or totally taking whatever it says at face value, then not bothering to put in further effort.

If they just kept prompting it, or maybe used a different thinking level, it could have identified and solved this problem. Sometimes an engineer would look at a system and say "the current approach isn't delivering the desired engineering requirements. Maybe we need to rethink".

Either engineer or LLM could take that sentence and run with it. OP of the article clearly can't do either.

simonw•about 1 hour ago

I visited https://claude.ai/ over a mobile tethered connection from my laptop the other day and was pleasantly surprised at how quickly it loaded.

(That said, I just had a look in Firefox and it loads 20.78 MB of JavaScript (6.84 MB compressed) so I expect they could make it a bunch lighter if they kept trying.)

hungryhobbit•about 2 hours ago

How about you make Opus 5.5 actually work?

I had it try to prepare a code review for me. Not only did it refuse, it refused to even tell me what the prompt (written by another Claude!) was. Why?

When I had another model read the session (all of the "stupider" models handled it just fine) it explained that it had the word "reasoning" in it

That's the entirety of Anthropic's billions of dollars of research: any prompt with the word "reasoning" is trying to hack Claude to figure out how it reasons!

A model like that should never have gotten out of QA, let alone been released.

bitpush•about 2 hours ago

I understand the frustration but shows a lack of critical thinking. Esp when you start with 'How about ..'.

This blogpost is about frontend performance. It'll be akin to you commenting on a swift blogpost saying 'How about Airpods noise cancellation'. Sure both are Apple, but they are wildly different teams.

hungryhobbit•about 2 hours ago

This is a techie discussion forum.

In such a forum, it seems to me like it's fair game to point out that the company patting itself on the back about how great they are at programming (as evidenced in the article above about their 3x speed improvement) ...

... can't even make their latest model handle basic English without refusing to work.

jfidjcjwjcjwjd•about 1 hour ago

Not that I’m defending Anthropic but OP is right, you’re complaining about the taste of a pear on a blog post about roses with the excuse that you’re in a biology forum. Sure, they both come from the same family, and sure, they both fall under the purview of biology, but they’re not the same.

dolmen•about 1 hour ago

This issue is mentioned on the Opus 5.5 post [1] from Anthropic (no idea if it has been added after your rant):

  > Don’t ask it to show its reasoning in the reply
  >
  > What to do. Remove requests to reproduce its internal reasoning in the reply from your prompts and instructions.
  >
  > Why it matters on Opus 5.5. A request to reproduce its internal reasoning in the reply can be declined. It’s one of the flag categories.
  >
  > How. Ask Claude for what you need instead, for example, “Explain why you chose this approach in three sentences.”

[1]: https://claude.dev/blog/getting-the-most-out-of-opus-5-5/

vikramkr•about 2 hours ago

Probably it thinks you're doing some sort of system prompt exfiltration/distillation attack. Also what even is the workflow you're trying to have it do? It's doing code review but you're having it read some other AI models prompt/session history? Are you doing code review or like session history retrospectives?

railgunmerlin•about 2 hours ago

seems a bit weird to complain about the model issues in a post about the harness/sites?

post-it•about 2 hours ago

> When I had another model read the session (all of the "stupider" models handled it just fine) it explained that it had the word "reasoning" in it

Did it explain it did it hallucinate?

hungryhobbit•about 2 hours ago

This happened at the classifier level, there was no "Claude thought X about it (hallucinating or otherwise)": this was a glorified regex deciding Claude couldn't work on a prompt (a code review prep) because it contained a string ("reasoning") it didn't like.

It's more or less the same mistake we've seen Anthropic make repeatedly with it's brain-dead regex-based Fable/Mythos gates.

post-it•about 1 hour ago

> because it contained a string ("reasoning") it didn't like.

How do you know? How would the stupider model know?

frumplestlatz•about 2 hours ago

I’ve had the same thing occur five or six times over the past week; they seem to be attempting to prevent anything resembling chain of thought extraction.

Every single time it triggered, it was due to a prompt written by their own model in a dynamic workflow. The self-serving nanny oversight has to go.

The fact that they label model distillation as an “attack” is genuinely hilarious after they “distilled“ their models from all of our work, and continue to do so.

I believe AI is here to stay and an incredibly powerful tool, but these companies, and especially Dario and Altman, are the very last people I want to see in charge of it.

copperx•about 2 hours ago

> “distilled“ their models from all of our work

They distilled all digitized human knowledge and artifacts and they're now complaining about someone copying their outputs saying it's a "national security concern."

I'm not sure about how to classify that. Hilarious? Pathetic? Sad? Hypocritical? Hyperdramatic? All of the above?

pllbnk•about 2 hours ago

> $500k engineer: [X] feels slow. Make it faster.

> Claude: On it... Done.

> $500k: Can you make it faster still?

> Claude: On it...

hmokiguess•about 2 hours ago

You removed the load-bearing seams didn't you

minimaxir•about 3 hours ago

This writeup legit coincidentally matches the asking-agents-to-make-code-faster-but-with-constraints-to-stop-agents-from-breaking-things writeup I posted on Monday: https://news.ycombinator.com/item?id=49803085

Front-end UI optimization is slightly trickier than optimizing strict algorithms, but I found that prompts to the agents to build tooling to track visual regressions are more than sufficient. The main issue (at least with GPT models) is that you have to be very explicit about the use of padding/margins/negative space.

That said, for my front end projects from scratch, I'm staying away from front-end JS frameworks and seeing how far and fast I can get with just HTML/CSS/vanilla JS shenanigans now that agents can wield them effectively.

tombot•26 minutes ago

Perhaps they can hill climb a native app instead of the electron and react native nonsense we have today

vikramkr•about 2 hours ago

Less a post about performance and more about their Claude tag product. I guess it makes sense that there's not actually a ton of technical detail being moved into given probably opus is the only one that knows what all the dragons were lol - but cool workflow I guess

altern8•about 2 hours ago

I used Opus 5.5 today for the first time hoping the writing would be more bearable and it SUCKS.

Why can't they fix that

tarr11•about 2 hours ago

Opus 5.5 writing is much more concise than 5.0

m00x•about 2 hours ago

but Opus 5 is absolutely terrible. I'm still on 4.8 when I use Claude. I still think 5.6 Sol is the best available model outside of Astra/Fable.

altern8•about 2 hours ago

MAYBE, but the bar is so low... It's not that it's good in any way

whythismatters•about 2 hours ago

The juvenile nonchalance with which some Anthropic employees seem to be talking to their AI (wacky, sick, cook, ...) is truly bizarre.

unrented7977•7 minutes ago

That's literally just how kids talk these days. Are you really that unaware of modern slang?

ieie3366•about 2 hours ago

That’s just how people in their 20s casually communicate now. It’s mostly from tiktok

tantalor•20 minutes ago

They've never had a job at a normal company

kylecazar•42 minutes ago

"You know what we want. Let’s go"

montroser•about 2 hours ago

Okay, now fix the WYSIWYG markdown parsing in the chat input!

Paste in a stack trace, then try to put it in a code block. Add a newline above, then add the opening triple backticks, then arrow down and add closing triple backticks at the bottom. Opposite congrats -- you have ended up with raw triple backticks at the top, plain text stack trace, and your cursor in a brand new code block at the bottom starting where you tried to close.

Realize you want to go put code span backticks around some identifiers you wrote out earlier? Best make sure to insert them in the blessed left-to-right order, or else opposite congrats again -- you'll end up with a mix of raw backticks and code span treatment for the text between your identifiers.

If Claude can discover novel CRISPR enzymes, surely it can make a rich text markdown editor, no?

AlexErrant•about 2 hours ago

Here's a simpler bug:

Claude Cowork still can't persistently reference a local dir, broken ever since they moved to their cloud project system, breaking many non-technical people's workflows.

rancar2•about 2 hours ago

Since I didn’t love the technical approach here (once one takes the humans out of the loop, there’s more ambitious things that can be done), I do appreciate the process. It’s not until the last sentence when process inspiration is revealed: “Special thanks to Boris Cherny for encouraging us to be more ambitious.”

devin•about 1 hour ago

I dare you to try and get heavy CPU cache-level performance optimizations you might see in tried and true HFT code written this way.

geroge_kyaw•about 1 hour ago

Sorry! I don't see that much difference. I would be more impress if you can make TTFT and ITL faster.

khalic•about 2 hours ago

Love it, the kids rediscover plain HTML and optimisation.

RomanKornev•about 2 hours ago

The most important question is how much more unreadable the code became after all this "ratcheting the benchmark down". If you unroll a loop it will perform faster, but making changes to such unrolled code will be a mess. Will this make them ship slower overall? I'm sure at least half of it was just poorly written React code, but the other half?

It's the same problem as overfitting in model training. If you're not measuring something it will get sacrificed.

Or, perhaps the code quality literally doesn't matter anymore and we've reached "code quality escape velocity" where you can code as much slop as you want, the next generation of models will clean it up faster than the slop generates?

jackb4040•about 1 hour ago

Came here to post this. AI rationalists love thinking about paperclip maximizers, but don't seem to care when it turns their own codebase into paperclips. Or to rephrase, turns their whole engineering org into meat proxies, slowing down engineering productivity in the long term because understanding is drained out of the staff and flushed down the drain every time they close a Claude Code tab.

chaordCAD•about 2 hours ago

Great writeup really appreciate the detail on what actually worked vs. what didn't.

j45•about 1 hour ago

Faster is great, hopefully the quality remains.

binlog•about 2 hours ago

Step 1 - make a website that takes 3 seconds to load a blank page.

Step 2 - bring it down to 1 second and pat yourself on the back.

xnx•about 2 hours ago

Correct. A web page is fast by default. Everything that is added makes it slower.

joe_the_user•about 1 hour ago

Sure, but some things increase the delay much than others. More static text, for example, is pretty fast. Elaborate JS, not so much.

airstrike•about 1 hour ago

In Brazil they say you can make things better by first putting a goat in the living room. It's going to shit everywhere, chew, droll, and make a horrible mess.

Take the goat out and your life will improve right away!

schoen•about 1 hour ago

Also described as a traditional Yiddish folktale (a rabbi suggests responding to stress by moving a series of animals into the home, and then at the end they're removed and everything seems great by comparison!).

jfidjcjwjcjwjd•about 1 hour ago

60+ yo Brazilian here and I have never, in my entire life, heard this saying before. Sounds something a Brazuca would come up with after the 17th round of Kaiser though. Pretty on brand.

airstrike•17 minutes ago

https://issoeunaosabia.wordpress.com/2011/12/03/era-feliz-na...

Razengan•about 1 hour ago

A life without a goat sounds horrible

dude250711•about 1 hour ago

It can certainly measure your token spend rate.

techpression•about 1 hour ago

”With that approach, we merged more than three thousand changes…” Why are they writing this? That’s terrible marketing all around, it means they let it go so far, with so little care, that they needed 3000 changes to make it into just a mediocre website performance wise (sure, electron app too, but still).

datadrivenangel•about 2 hours ago

AI written slop. They need to upgrade to Opus 5.5 or switch to OpenAI for writing.

joe_the_user•12 minutes ago

My user experience was significantly worsened just by their referring to "user journeys". I mean, I definitely don't want to go on a journey with Claude or ChatGPT (admittedly my go-to). I want result quickly and be done.

owebmaster•about 2 hours ago

With a simple trick: make it very slow first

rvz•about 2 hours ago

Let's try this again if you want an instant 10x speed up:

Claude rewrite Claude Code from TypeScript into Rust. Make absolutely no mistakes.

cpursley•about 2 hours ago

Not sure why you're getting downvoted, the Rust based tui's absolutely smoke Claude Code.

minimaxir•about 2 hours ago

It's an overdone Reddit-esque comment that adds nothing to the discussion.

applfanboysbgon•about 2 hours ago

tl;dr if you make absolute dogshit software that takes 4.38 seconds to stabilize its first paint you can make really nice headline claims by "optimizing" it later

sashank_1509•about 2 hours ago

Unironically true, can someone question, what is Claude desktop app doing that needs 500k+ lines of code?

Agents complicate something that should be much smaller and simpler and then agents speed it up adding more complexity. I suppose functionally you may say this is fine but aesthetically it is hideous!

winstonp•about 2 hours ago

the codex desktop app bundles libreoffice's libraries. i wouldn't be surprised if claude does something similar.

bigwheels•about 2 hours ago

TempleOS had kernel, compiler, 2D and 3D graphics drivers + libs, and tons of games and applications and the entire thing weighed in at less than 100kloc.

AI coding agents of today definitely bloat things up [egregiously].

theshrike79•about 2 hours ago

On the other hand applications on TempleOS are expected/assumed to be perfect and have no bugs. It makes things a lot simpler =)

applfanboysbgon•about 2 hours ago

There is nothing even functionally fine about this. The me who has programmed for a 4mhz computer with 128kb of RAM is crying inside. It's a fucking trivial interface for writing and displaying text and sending HTTP requests. You could have that displayed ~instantly on an 80s home PC. Now we have home computers that can execute somewhere between billions and trillions of instructions per second and yet a task that should take <10ms takes 4500ms. Our industry has become an absolute embarrassment.

adamddev1•about 1 hour ago

Thank you. The apps created like this are absolutely an embarrassment. But to me the bigger horror is if people are going to be convinced to develop the critical libraries and infrastructure in the same way. Then we'll have bloat and rot in the deeper levels and everything will become exponentially worse and unreliable. It makes me so sad that people can't see this.

Daishiman•about 2 hours ago

The Claude CLI does a lot more....

Daishiman•about 2 hours ago

Most line-of-business software has a lot of optimization opportunities. Making software optimized takes up time that can be spent building features. The fact that you can just make things go fast without having to take time away from feature building is actually pretty awesome.

applfanboysbgon•about 2 hours ago

Not taking 5 seconds to load text is a feature. 10x more valuable than whatever other shitty feature you're thinking of piling onto your monstrosity. Like, actually tangible valuable to users and consequently your business; Google has already done the studies at scale that demonstrate how every 100ms of delay has an observable impact on usage statistics and user retention.

sonar_un•about 3 hours ago

This was a fantastic read. Lots of useful info in there for your own projects.