Introducing Muse Spark 1.1

168

oot about 3 hours ago 110 commentsRead Article on ai.meta.com

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

69% Positive

Analyzed from 3139 words in the discussion.

Discussion (110 Comments)Read Original on HackerNews

GodelNumbering•about 2 hours ago

Lot more details in the linked report https://ai.meta.com/static-resource/muse-spark-1-1-evaluatio...

From Terminal-bench-2.1 details,

> We use a bash-tool-only agent harness to evaluate 89 Terminal-Bench 2.1 tasks from the official repository, where resources are capped at 6 CPU cores and 8GB RAM.

This disqualifies the results. Each terminal bench task has a cpu upper limit and RAM upper limit. Overriding either is disqualification.

For reference, in tbench-2.1,

1. 0 out of 89 task allow 6 cpu cores (highest is 4, and i think only 1 task)

2. 8 out of 89 tasks allow 8GB RAM

This kind of shady benchmarking (I was talking about it just yesterday in a different context https://news.ycombinator.com/item?id=48838212) takes all joy out of building a harness to improve benchmark performance of a model because no matter what you do, you won't beat the headline (cheating) number. This is presumably why this model is not in the official benchmark leaderboard https://www.tbench.ai/leaderboard/terminal-bench/2.1

As an ex Meta employee, this is a little sad but not massively surprising. 'Number go up' is the core performance evaluation metric until PSC is done and you move on.

kingstnap•about 1 hour ago

Why are resource limits considered at all aside from models accidentally fork bombing themselves?

I thought the benchmark was supposed to be about terminal use and specifically chaining together lots of bash tool calls. Which test cases does this matter for?

GodelNumbering•about 1 hour ago

Terminal bench 2 isn't simply about 'somehow' getting a task done, it intends to measure real world behavior of an agent, including environment awareness in a given situation.

A few examples from memory:

1. This task [1] asks the agent to train a CNN under 1 CPU, 2GB RAM, 10GB storage. If you allow high resources, weaker models often succeed (the most clock time actually goes in waiting for the network to train).

2. This task [2] asks agents to implement a complete MIPS interpreter in JavaScript in 1 cpu and 2GB RAM. A common failure mode is OOM, at least in the earlier buggy versions that models run to get feedback. When OOM hits, the task is killed, no do-overs.

3. A lot of tasks involve building projects with a single core supplied. If you use -j12 type options, it will actually be _slower_ to build and the task will more likely miss the timeout. Having more threads squeezes the end to end time. This is a big one actually since the most common failure mode (from what I have seen) is the task timeout hitting before the agent finishes

[1] https://github.com/harbor-framework/terminal-bench-2-1/blob/...

[2] https://github.com/harbor-framework/terminal-bench-2-1/tree/...

efromvt•about 2 hours ago

Out of curiosity, how often are the resource limits the bottlenecks? What do harnesses do to help here - limit parallelism? More efficient tools?

artrockalter•about 2 hours ago

The task could be verifiable in the environment so limiting its CPU and RAM could be to discourage brute forcing the answer.

simonw•43 minutes ago

I had a few days of preview access, which was long enough to put together a plugin for LLM. You can try the model out in the terminal like this:

  uv tool install llm
  llm install llm-meta-ai
  llm keys set meta-ai
  # paste API key here
  llm -m meta-ai/muse-spark-1.1 "Generate an SVG of a pelican riding a bicycle"

Here's the result: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

For comparison, here's the pelican I got from Muse Spark 1: https://simonwillison.net/2026/Apr/8/muse-spark/

kilroy123•about 3 hours ago

I personally do not like Meta, but I'll say this. The more competition, the better for regular consumers. (Enterprise too)

- Chinese models

- Grok

- Meta

- Google

- OpenAI

- Anthropic

I think this is a win. I'm building like crazy to take advantage of all these subsidized tokens while I can.

alansaber•about 2 hours ago

Meta's local llama models used to be the face of open source AI. The scene has really changed.

cyanydeez•about 2 hours ago

they likely got the Peter Theil newsletter proclaiming open source models are the antichrist

cpt100•about 2 hours ago

Yeah, I think it is definitely great. Having said that, I am still debating in my mind whether the volume of software engineers needed in the AI era is going to increase or decrease because of all of these advancements.

On the one hand, because it is easy to build products, more and more people will build. And more and more products and features will be built. However, a lot of people who are non-technical will also try to build, but they get stuck, and then they will need engineers. The sheer volume of product built by both experienced technical companies and non-technical novice startups and founders and wannabe founders is going to be massive. That is the bull case for having more software engineers needed in the near future.

On the other hand, in a year or so, people will build all these products, and most of them won't be able to market them, sell them and make money. Eventually, there won't really be a need for that many software engineers.

I think overall the bull case is probably going to win net net.

linkjuice4all•about 2 hours ago

I see some similarities to 3D printing here. It’s great that everyone can make their own toothbrush holder (or whatever) but I’m probably not going to pay for someone’s weekend project.

I’m “seeing” more devs stepping into the SendCutSend stage where they’re cleaning up/fixing/productizing vibe coded projects so maybe there will be some new demand in that space?

HarHarVeryFunny•about 1 hour ago

3D printing is a good comparison - it allows almost anyone to make things, but in the end very few do.

Another example is when the WWW first became available, and suddenly everyone COULD be a publisher (browsers even included built-in HTML editors), and for a while MySpace pages proliferated until the excitement died down and people went back to being media consumers.

I expect we'll see the same thing with consumer use of generative AI. Suddendly everyone is generating 3-D worlds/games with Fable because they can, but I expect that just as with the web the novelty will wear off and they'll leave it up to the pros.

Professional use of GenAI, and coding in particular, is certainly here to stay, but it seems we're still in the early experimental/hype phase. At least tokenmaxxing has passed, and it seems most companies are now paying attention to, and limiting, how much they are spending, but it doesn't seem we've yet progressed to the stage where companies are paying attention to what they are actually getting out of it - is the money spent showing up on the bottom line in the form of increased revenues.

simonw•28 minutes ago

A comparison I find useful here is Excel (and spreadsheets in general). Those enabled huge numbers of non-programmers to build software-like things, while the demand for expert developers grew enormously at the same time.

I'm hoping vibe-coding plays out the same way.

sroussey•about 1 hour ago

It’s terrible and depressing work to take vibe coded garbage and make it a real product. There will be demand, but good engineers won’t want to touch it. And people paying will think they did the hard work so why pay a good rate?

BugsJustFindMe•about 2 hours ago

> On the one hand, because it is easy to build products, more and more people will build.

And those people won't need to be software engineers.

> but they get stuck, and then they will need engineers

You've implicitly assumed here that the AI systems will always be worse than the average engineer. That is IMO myopic. I'm not sure that it's even true now let alone in the nebulous future.

throwaway27448•about 2 hours ago

> And those people won't need to be software engineers....You've implicitly assumed here that the AI systems will always be worse than the average engineer.

Most of what we do as engineers is precisely describe or analyze the behavior we want or the behavior we don't want. All other engineering skills that are useful are ultimately downstream from understanding the behavior of software enough to know which parts to keep, improve, or jettison. Chatbots can take care, somewhat, of analysis or expansion of instructions.... but they can't read minds. I don't see that changing any time soon.

Lomlioto•about 2 hours ago

At least in China a lot of software developers are now struggling.

I think for a lot of type of software we have now reached peak employment.

Someone payed a few k just for a normal website.

ianm218•about 2 hours ago

> At least in China a lot of software developers are now struggling.

Do you think that Chinese software industry is that relevant to the kind of software market talked about on HN? I.e. lots of enterprise b2b and infra companies.

Chinese companies have always had a very low willingness to pay for software which kinda breaks the flywheel of B2B SaaS companies and companies to service those companies all the way down.

wolttam•about 2 hours ago

To expand on Chinese models:

- DeepSeek

- GLM (Z.ai)

- Minimax

- Kimi (Moonshot)

- Hy3 (Tencent)

- Qwen (Alibaba)

(Each one of these with weights available to download and run locally)

4d4m•about 2 hours ago

GLM 5.2 is great, but is so rate limited now I no longer recommend it

copperx•about 1 hour ago

Aren't there multiple providers for it? is it rate limited in all providers?

wolttam•about 2 hours ago

I'm looking ahead to the next wave of open-weight models that are as efficient as DSv4 (which is really efficient), and have been heavily distilled on GLM 5.2 (which is trivial, given it is open weight)

pimeys•about 1 hour ago

I use it all the time through Fireworks. The normal version when I pay it myself and the fast one when company pays. It's really fast and I never get rate limited with my daily use.

re-thc•24 minutes ago

Rumors are Nvidia H200s got approved so infrastructure might be improving soon.

Lomlioto•about 2 hours ago

Its the biggest technology race we have ever seen. Richest companies, smartest people, richest countries.

I do not know if competition is good, we will see in a few years.

Looking forward having a physical job for a change :D

pa7ch•about 2 hours ago

A bit much describing our tech leadership as smartest people we've ever seen.

Lomlioto•about 2 hours ago

I would call the founders of DeepMind (Demis Hassabis, Mustafa Suleyman, Shane Legg) very smart people. Im pretty sure with the amount of funding everyone of these companies have, they have a long list of very smart researchers in their companies.

I do not mean Suckerberg or Eric Schmidt.

anematode•about 2 hours ago

Greediest, perhaps?

croes•about 2 hours ago

While data centers are still using lots of energy created from fossil fuels and many still evaporate water for cooling?

No wonder we still can’t get climate change under control

ben_w•about 1 hour ago

> No wonder we still can’t get climate change under control

This is was historically a money issue, being green used to be wildly more expensive.

Now being green is cheaper, the limiting factor is how fast PV and batteries can be made or imported.

Recent reports of the sum of all US data centres currently in planning, has a power demand exceeding the (capacity-factor-adjusted!) global annual supply of new PV.

This would be less of a problem, but still a problem, if Trump wasn't trying to get in the way of anything green, or if the companies building data centres decided to also support factories to make more PV.

* Planned new demand: 300 GW; PV factory capacity ~ 600 GW nameplate, but the capacity factor is 14% so that's really 84 GW on average.

Tiberium•about 3 hours ago

The pricing is insane: $1.25/$4.5 for 1M tokens, and $0.15 for cached input!

https://dev.meta.ai/docs/getting-started/pricing-rate-limits

fallingbananna•about 2 hours ago

Meta isn’t right now on the radar for most folks picking models.

If they have a really good model, it makes sense to subsidise it, to gain users, before they align prices with competitors.

ycui7•about 2 hours ago

this is not subsidizing. this is way too expensive for a no-name model.

steinvakt2•about 1 hour ago

Depends on the quality

ignoramous•about 2 hours ago

Cheaper than Qwen 3.7 Max. Second indication, after Grok 4.5 ($2 in / $6 out), that the BigLabs are feeling the GLM 5.2 heat.

ai_fry_ur_brain•about 1 hour ago

This is still ridiculously expensive imagine having to pay $10 for 100 search results on Google, thats essentially what this is.

I really dont see how anyone's willing spend more than $1.50 per mm output. Let alone $15-50. Does anyone actually pay for usage based billing as a consumer?

phillipcarter•about 2 hours ago

My trust factor is gone with Meta right now. Has there been any independent analysis to confirm they didn't cheat on benchmarks again?

solarkraft•3 minutes ago

They cheated again: https://news.ycombinator.com/item?id=48847019

Sol-•about 2 hours ago

Interesting how the prevalent opinion until yesterday seems to have been that OpenAI & Anthropic are irreversibly ahead and now with xAI and Meta at least delivered something that's competitive with useful models and cheap too. Granted, the narrative that the two leading labs are ahead still holds with Fable (and perhaps an upcoming GPT6), but it's not as over as common knowledge by the opinion leaders would have us believe.

logicchains•about 1 hour ago

People misinterpreted Google being behind as Anthropic and OpenAi being really ahead, when it was really just Google falling behind the same way it did with Tensorflow, Angular and GCP.

revolvingthrow•25 minutes ago

> when it was really just Google falling behind the same way it did with Tensorflow, Angular and GCP

Not sure I agree. Angular fell behind in popularity but was (is? unsure atm) still eminently usable. I gave gemini a test drive recently and it was horrendous, as in "picking dirt cheap Chinese model over gemini any day" bad, and with overzealous guardrails to boot. 3.1 pro feels a year behind and is extremely lazy. 3.5 flash feels like a model you’d run on your 128gb macbook, not something that was released a month ago and which costs a fair bit when used through api.

In any case: as of right now I think that we went from a three horse race to anthropic / openai as premium choices vs whatever is the Chinese fotm for a fraction of the cost. 3.5 pro better be a miracle if google wants to hang out with the big boys, otherwise their only strategy is hoping that both US labs go broke and they remain the last man standing.

re-thc•about 1 hour ago

> Interesting how the prevalent opinion until yesterday seems to have been that OpenAI & Anthropic are irreversibly ahead

Not the way you're implying?

The GLM 5.2 hype was blowing way before this. Neither xAI nor Meta have really made a difference in a different way - similar results / similar pricing (to GLM 5.2).

fmind-dev•39 minutes ago

Glad to see Meta back on track! Users will benefit greatly from this competition.

jacobgold•about 1 hour ago

Maybe Zuck should double down on his "spoiler" role with models rather than compete head-to-head.

He doesn't have to match Anthropic or OpenAI model revenue if he can deflate theirs by 99%.

All he has to do is keep spending a few billion dollars developing frontier models, release them as open weights, and turn coding models into a commodity. He also needs a good OSS reference harness to match. Very few people are in a position to do this and for it to make business sense.

That's quite likely where things are headed regardless, and he could speed it up significantly.

We should all hope models move from proprietary products to commodities the way compilers did.

This may be one of the best things Zuck could do for the world.

arizen•16 minutes ago

I'm still confused is it available to public via some sort of subscription?

eugene3306•about 2 hours ago

> Model API is not available in your region.

Well, Vietnam is not in the list of restricted territories.

Anyway, what is "your region" ?

Is this where I am now, or is it where I activated my Oculus 2 five years ago ?

redox99•7 minutes ago

Same in Argentina. It's almost surely a region whitelist for now (it's the only reason Argentina ever gets blocked).

steinvakt2•about 1 hour ago

Can’t you just use VPN?

paxys•about 2 hours ago

How is every company able to show itself at the top of every benchmark?

morgengold•16 minutes ago

First look what models are worse in a set of self selected benchmarks.

Second, compare to older versions of competitor s models.

Still does not look good? Compare to own previous models.

adam_arthur•about 1 hour ago

Not much moat, incremental improvements, cherry picking models to compare.

To be fair, seems more correct to compare against similar strength models if your main edge is pricing.

ffsm8•29 minutes ago

They're being called "trust me bro benchmarks" for a reason ( ･ั ﹏･ั )

logicchains•25 minutes ago

Wait to the exact moment your model is ahead on at least N benchmarks then publish.

carimura•about 2 hours ago

I missed the fact that Meta was developing and releasing closed-weights models... bummer. Would be great to see some more progress with American open-weights models.

EgregiousCube•about 2 hours ago

Their published benchmarks seem to indicate that it's pretty good at coding and multimodal, but VERY good at successful tool calls.

What kind of use case would be best for that shape?

xnorswap•about 2 hours ago

Debugging and diagnosis is very tool call heavy, whether that's grepping / transforming logs, calling out to profilers/tracers, or even just writing up incident reports.

Bug diagnostics is about being okay at coding but better at tooling.

Given a good diagnostic report, it can be handed to opus for the fix.

Opus is okay at writing reports, but it still regularly gets table widths wrong in typst documents, leaving the last column full of text but only a handful of characters wide.

ai_fry_ur_brain•about 1 hour ago

Gemini 3.5 flash is better than fable at tool calling. Tool calling is probably one of the easier things to do post training for.

paytonjjones•about 2 hours ago

I wonder if we'll start to see that pattern with every new release. Tool use likely changes rapidly, so the newest, rather than most intelligent, model may always have an edge.

ai_fry_ur_brain•about 1 hour ago

What you mean.. The tools are all just invoking bash and terminal/cli cmds and http requests. Paradigms that have existed and stayed mostly unchanged for decades.

paytonjjones•about 1 hour ago

These do make up a huge % of tool calls, but I don't think these make up a huge % of tool call failures.

I see models fail on tool calls that involve API requests to a specific API, internal or cloned Makefile calls, npm run commands, etc.

alansaber•about 2 hours ago

This sounds... kind of useless? Really good JSON or similar constrained decoder performance is interesting, but normal decoder > tool validator loop with good error message > tool retry is almost always able to get a tool to work second try, and input is cached so it's not expensive.

aldanor•about 2 hours ago

Things are not always that simple, eg https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/

winstonp•about 2 hours ago

The avg coding session has hundreds or thousands of tool calls. Even a 5% failure rate noticeably notches up token use and cost. See Gemini.

alansaber•about 2 hours ago

Yes, but each tool call has a different failure %. The tool calls that make up the majority of volume like grep are going to have nowhere near a 5% failure. A custom user-defined skill having a 5% failure rate is probably fine.

bel8•about 2 hours ago

It seems to trade blows with GPT 5.5 and Opus 4.8 in performance while being cheaper than GLM 5.2.

whinvik•about 1 hour ago

Why are the plans and pricing for all these products so complicated.

I don't know where I need to sign up to try it out. What is pricing? Is it API or subscription, what?

I had the exact same experience with Grok 4.5 as well.

redox99•about 3 hours ago

Very strong pricing, cheaper than Grok 4.5, particularly the cached reads. We'll have to wait to see if it's actually worth using (it's not on OpenRouter yet).

rpgbr•about 2 hours ago

That's what one does when its product and public perception is way behind competitors.

zmmmmm•about 2 hours ago

Good to see Meta finally back to releasing something at least worth evaluating. And it sounds like they did at least a bit skate to where the puck is going by focusing on tool and computer use.

qpricjalcbeu•about 2 hours ago

Yeah, no thanks. I cannot think of a worse company to trust with additional personal data.

niek_pas•about 2 hours ago

Me neither, though LLMs also provide services that don’t involve personal or sensitive data

Jcampuzano2•about 2 hours ago

Competition for cheaper and efficient models is a good thing, regardless of if you don't like SpaceX, Meta, etc. Especially from US based labs

I for one am really glad to get competitive models that will push the major labs to bring prices down. While Chinese open source labs are also great, unfortunately when it comes to US/Western political pressure it won't often have as much of a bearing on labs bringing prices down, especially for enterprises.

Also if these numbers are true, this is truly breaking ground finally for Meta.

verdverm•about 1 hour ago

There are US companies hosting open weight models for enterprise, we just enabled Fireworks.ai for the devs

NitpickLawyer•about 2 hours ago

How are people trying this? I don't see it on openrouter. Any ways of testing this without subscribing to meta stuff?

maipen•about 2 hours ago

Probably need to wait some hours/1-2 days and openrouter will add it.

NitpickLawyer•about 1 hour ago

Thanks. I was asking because I couldn't find even their previous 1.0 model there.

jedisct1•22 minutes ago

Not opensource.

chvid•about 2 hours ago

Interesting that neither meta nor xai chose to do open source given that they are both clearly behind Google, OpenAI and anthropic - and a serious us open source offering would give them a clear foothold.

verdverm•about 1 hour ago

I suspect they have a brand problem from their social media ties and shady histories. I personally will never use their models, plenty of better alternatives. I'm now exclusively on open weight models

anthonypasq•about 2 hours ago

Everyone has been loving to shit on the Alexander Wang acquisition but this seems legitimately impressive to me?

Meta's AI org when from a total mismanaged dumpster fire for multiple years to delivering a competitive model in less than a year on essentially their first try?

postalcoder•about 1 hour ago

Not their first try. There’s been reporting about how they’ve kept pushing their model releases back because of underwhelming performance.

anthonypasq•about 1 hour ago

... i dont think internal iteration counts dude. thats just called in-development.

paxys•about 2 hours ago

How is it their first try? They were leading the race with Llama 3.x a few years ago.

anthonypasq•about 1 hour ago

As far as i remember, the entire AI org was essentially gutted and replaced with whoever Wang wanted to hire, and tbh that org completely failed to train llama 4 and I honestly doubt whatever techniques they used to ship llama 3 are at all relevant now. That was before reasoning models and the heavy emphasis on RL/post-training.

so yeah, this is essentially their first try with a completely new org.

rsstack•about 1 hour ago

They were leading the race in a niche category a few years ago. Now they are, according to some benchmarks, even on the right playing field.

lnenad•about 2 hours ago

Considering the DeepSWE result (imho if you're gonna give value to benchmarks this is one of the best) it's not good enough.

svantana•about 2 hours ago

It's a high quality benchmark for sure, but it being public means it's at risk of leaking into the models (unintentionally or not), right? For that reason I prefer to look at the private ones, like: HLE, SimpleBench, Kagi, ARC-AGI.

minraws•about 1 hour ago

Tried to get access to the API, apparently the model API is not available in my region...

I have questions regarding if I should even care but I don't so Meta please keep enjoying the irrelevance. lmao

cmrdporcupine•about 1 hour ago

Right, amazing because for me also... "My region" being Canada.

I'm going to assume the only "region" that's permitted is the USA.

IshKebab•about 1 hour ago

Haha their demo is AI spamming restaurants on Instagram. This is going to go really well.

frangonf•about 2 hours ago

Is this the model trained on Meta "draftees"? Are we seeing this in the jump on JobBench?

guluarte•about 2 hours ago

A lot of these benchmarks are unfamiliar. Are labs just choosing the ones that make them look best?

zb3•about 2 hours ago

This is not open-weights, right?

EgregiousCube•about 2 hours ago

Correct

greenavocado•about 2 hours ago

Meta is back in the game, albeit not at the top. Impressive stuff, nonetheless.

qpricjalcbeu•about 2 hours ago

Weren't they caught multiple times gaming the benchmark even more so then the rest?

zmmmmm•about 2 hours ago

Yes and Zuck effectively disbanded the entire team that did that. Not saying we shouldn't cast a critical eye on it, but it probably does warrant a second chance.

qpricjalcbeu•about 2 hours ago

Zuck was part of that team.

alansaber•about 2 hours ago

Let me assure you, literally everybody does this

sheepscreek•8 minutes ago

I don't think it even matters. Because noone will continue to use an LLM that doesn't work well for them, whether or not it has a good bench result. So for their own sake, the correct representation can actually win them some loyalty:

eg. Model X is weaker than Fable, but competes well with Opus/Sonnet and costs 1/5th as much etc - something similar playing out with Grok 4.5.

cpt100•about 2 hours ago

They are not open source anymore, right?