Agent swarms and the new model economics

188

jjlaneve about 15 hours ago 87 commentsRead Article on cursor.com

DE version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

62% Positive

Analyzed from 1799 words in the discussion.

Discussion (87 Comments)Read Original on HackerNews

anthonypasq•about 14 hours ago

Love to see these crazy kinds of experiments going on. Even if this doesn't 100% work or is prohibitively expensive for now, these are glimpses into the future in the same way people were talking about coding agents in 2023 when we just had tab complete.

htrp•about 14 hours ago

>The browser swarm from earlier this year peaked at roughly 1,000 commits per hour on Git. The new system peaks at around 1,000 commits per second.

>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.

Talk about inventing the universe to make a button.

jgalt212•about 13 hours ago

It's hard not read such quotes and immediately think of the Infinite Monkey Theorem.

https://en.wikipedia.org/wiki/Infinite_monkey_theorem

whatever1•about 12 hours ago

If at the end Shakespeare-level literature is produced, does it matter whether we arrived there by random keystrokes?

qjack•about 11 hours ago

The problem is the infinite other literature produced along the way. It doesn't work to throw more monkeys at the problem of verification, and checking for matches against existing Shakespeare plays is cheating.

gopalv•about 9 hours ago

> does it matter whether we arrived there by random keystrokes?

Douglas Adams said this beautifully in the Campaign for Real Time & the poet Lallalfa.

And I just found out h2g2.com is no longer online, so no reference.

w-ll•about 9 hours ago

Shakespeare-level literature was produced. We are the infinite monkeies, and one was shakespeare.

confidantlake•about 10 hours ago

Yes because it is akin to producing a universe of slop and trying to find the single atom of gold.

voidhorse•about 12 hours ago

Yes because the keystrokes aren't free and the world is finite. We need to pay these monkeys in bananas.

leoqa•about 10 hours ago

I mean why even use VCS at that point?

asim•25 minutes ago

Anything architectural like this is super interesting. We keep getting more powerful models sure. We're getting agents ok. But this coordination of many agents. This is the part that's really going to scale very fast and across many many machines in parallel. Very cool.

edg5000•about 2 hours ago

I initially thought that getting agents to work for longer and in large groups was the future, but I'm increasingly thinking that, at least for engineering, just one thread makes more sense. The agent pulls things into context as needed. One thing that I've been experimenting with is also letting the agent remove things from context, such as files. But just adding to the context and compacting when it's full seems like it might beat a lot of more advanced options. Because the model is good, it knows what to put in the summary; just enough for the model to be able to rebuild the context from that seed (e.g. pulling in relevant files/data into the context).

handfuloflight•about 14 hours ago

> To test that progress, we returned to a task the old swarm had struggled with: building SQLite from scratch, in Rust, from nothing but its documentation.

Isn't SQLite's source code in its training data?

Romario77•about 13 hours ago

not in Rust. might be different enough ...

kgeist•about 12 hours ago

Even if no Rust code for it was seen during training, an LLM can trivially transpile SQLite's C codebase to Rust on the fly. For example, I just asked ChatGPT to write John Carmack's famous Fast Inverse Square Root algorithm in Erlang, without searching online or thinking, and it transpiled it immediately (while also extracting the knowledge in the same step). SQLite's semantics/code are stored in the middle layers of an LLM, and the last layers are able to convert it into any representation, as conditioned by the prompt. Cursor's experiment is deeply flawed because they merely extracted the model's compressed, lossy knowledge of SQLite's codebase and then just ran a bunch of tests/fixing rounds to make up for the lossiness. The claim that the agents built it from scratch is false.

wrs•about 11 hours ago

Then you would expect the implementation to be structured the same as SQLite, and having glanced at the result, it looks like at least some things aren't. For example, it seems to use an operator-tree executor rather than SQLite's bytecode interpreter.

hiddendoom45•about 13 hours ago

There's Turso which is a sqlite rewrite in Rust with some additional features. It's open source so I'd imagine it would be in the training data.

shay_ker•about 12 hours ago

How do we know if these models weren’t trained on Turso’s rewrite of SQLite in Rust?

It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?

IanCal•about 11 hours ago

> What am I missing?

They’re testing the same models on the same task but different ways of organising the swarms and the new approach works better.

handfuloflight•about 12 hours ago

This is more about automated long horizon work.

adamtaylor_13•about 12 hours ago

Wouldn't that be in the authors' interest to disclose, if true?

whinvik•about 13 hours ago

I would have loved to see more of the harness engineering shared as code. Instead we are left with only the outcome.

I guess that makes sense since the harness is the product in the case of Cursor.

blmarket•about 11 hours ago

I don't think there is a lot test harness, given there is a golden implementation (sqlite3), with clear interface (SQL, and sqlite3 db files). Once you slop crappy implementation, then you can create one group of agents finding gap where slop doing wrong vs golden, and another group of agents filling the gap meeting the conformance.

I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.

I would not surprised they have 1+ millions of specs.

I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html

dakolli•about 13 hours ago

I call this junk "meta-agentic engineering", it reminds me of people who have the coolest nvim configs, spend hundreds of hours customizing it but ultimately get less work done than the guy with minimal workflows, if any at all.

I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.

svachalek•about 12 hours ago

Some of this stuff is ludicrous. I finally tried /loop last week and discovered every loop iteration passes the entire context history to the model. So pretty quickly you're running a full 1M context window, without cache, likely just to check if something is ready or needs to be done. It's miserably terrible engineering unless your entire and only goal is to burn tokens.

Wowfunhappy•about 12 hours ago

Wait, why isn't context cached with /loop?

timcobb•about 8 hours ago

> One reading is that it was more productive. Another is that most of those commits were busywork (thrash, contention, churn).

Kudos

smoyer•about 9 hours ago

This is almost a year behind Steve Yegge's first post on beads. Gas Town and Gas City provide orchestration for the swarm. So far I haven't seen a perfect implementation but this idea isn't new.

agustechbro•about 7 hours ago

Wow, people have been working hard for a long time in Turso: https://turso.tech/blog/introducing-limbo-a-complete-rewrite... to do the same cursor did in justo no time?

vessenes•about 11 hours ago

This is super fascinating, and I loved seeing testing of where exactly you need frontier intelligence -- looks like coordination / planning, but not coding right now -- the article's a bit of a tease, as we can't play with such a harness, or their new version control system or get a workable artifact out of it.

That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.

trevswarm•about 7 hours ago

Even at a much smaller scale of a few dozen parallel agents working I've found real benefit to a structured hierarchy of agents. Not just keeping the context clean and focused at the right level, but if you find that something has just gone off the rails, you can more easily rip out or even just fully delete a leg of work without damaging the original design. It's also nice when you find a design change is needed and you can communicate that to the top level and have it efficiently propagate down where it matters.

I like it so much I even recorded a video walking through the process!

https://www.youtube.com/watch?v=efUmcCiRoDU (I do walk through how I do this in our app DevSwarm here)

dymk•about 9 hours ago

This post describes ideas found in existing agent orchestrator systems since they've existed. You'll find the same "top level agent that breaks a problem down into spec / plan / implement / verify" pattern in get-shit-done or superpowers.

Is the original part the VCS that works at 1000 commits/second? Or spending a bunch of money comparing models?

pianopatrick•about 12 hours ago

I wonder what would happen if the agents did not have the spec. Like if you just said "build a simple SQL database that works as a single file", what would the agent swarm come up with.

mccoyb•about 14 hours ago

I find these blog posts (and the originals, with Anthropic's C compiler and Cursor's browser) somewhat funny, as if they have this enormous power to build ... but they can't build something unique or new. Like the software sucks, but look how powerful the process is (the models are indeed powerful).

And it's a bit of a shame: by virtue of their position (their embedding in the fabric of venture capitalism), it seems like they can only make a subset of things -- what they can make is dictated enormously by capital, as they are engines of capital.

Not sure the point I'm trying to make, I just find it amusing.

Perhaps the point is that it might be more worthwhile to give independent creators a billion dollars to play around with agent swarms if we want to keep diversity in the evolutionary algorithm that is the software industry high.

brap•about 14 hours ago

Because it's mostly a load of crap.

Most of these reusable, automated workflows sound really cool in theory, but in practice either they require a lot of customization (not really reusable) or a lot of babysitting (not really automated).

AI has improved my productivity significantly, I'm not a hater, but we also have to be honest about its (current) limitations. The whole "software factory" thing is just a pipe dream people sell to execs / investors / online courses.

maherbeg•about 14 hours ago

Yes, that's true, but in the post, they even mention that building the spec is the scarce resource.

  For that to work, the swarm has to actually follow the spec, which is what much of this post is about. We gave the swarm 835 pages of prose and it came back with a database. What was scarce in this experiment, and what we expect to be scarce in software engineering going forward, is the right description of intent.

inciampati•about 13 hours ago

but that's what we use the agent "swarm" to do .. it's also not scarce. what's scarce is imagination.

viccis•about 12 hours ago

We're rediscovering the "architect" job role a decade or so after switching to agile processes and staff engineers largely replaced it. God help us when they rediscover UML for agents.

chrisweekly•about 9 hours ago

I feel like the scarcity of (idealized) Product Managers is even more acute.

SpicyLemonZest•about 13 hours ago

I mean, how long are these pages? Assuming 40 or so lines per page, that would be 33400 lines of spec turned into 200,000 lines of Rust, or about 6 lines of code per line of spec. That's a very detailed spec, much more than any software specification I have ever written in my career. It's not obvious to me that writing this spec would meaningfully save work over incrementally building individual features, especially when you think about what it would take for human beings to meaningfully review whether those 835 pages contain the right intent.

FuckButtons•about 12 hours ago

not only that, but what all of this misses is the fact that the specification that they used was a product of writing the software, not the other way round. It would be much, much more difficult to first write a correct specification than to first write a correct program, because programs are formal systems that can be executed to observe their correctness. You can’t do that with a specification.

podgietaru•about 12 hours ago

Or to validate the accuracy and efficacy of the actual code produced.

arjie•about 13 hours ago

Well, the tool works well. I am grateful that Nature has seen fit that the same kind of person who can make it doesn’t also have the ability to identify what needs exist in the market because then all software would flow out of OpenAI and Anthropic and the rest of us would run about their legs and peep about. Whether product development will survive the next generation of models is still unknown.

dakolli•about 13 hours ago

bro it can't even make a button that doesn't look like it was designed in 2018, everything is ass. Product development isn't going anywhere.

viccis•about 12 hours ago

The bottleneck has always been on the product side for 99% of cases. For the 1% where it's engineering, LLMs don't help as much.

dougSF70•about 10 hours ago

Can a swarm of agents write the 835 page manual?

bofadeez•about 12 hours ago

The issue is that the only models that could be trusted to work autonomously cost more than a human employee

doctoboggan•about 11 hours ago

I feel bad for the engineers at Cursor who have to use Grok in these sorts of experiments.

alienbaby•about 8 hours ago

Tinfoil hat: what if the prices are artificially inflated deliberately to price out casual users being able to field large swarms of frontier model agents because it could be just too dangerous.

onion2k•about 5 hours ago

The fact that the swarm needs high quality docs to work from will keep most of the tech industry safe.