Project Glasswing: An Initial Update

226

llouiereederson about 3 hours ago 147 commentsRead Article on anthropic.com

FR version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

80% Positive

Analyzed from 5705 words in the discussion.

Discussion (147 Comments)Read Original on HackerNews

guidedlight•2 minutes ago

> The bottleneck in fixing bugs like these is the human capacity to triage, report, and design and deploy patches for them. Finding them in the first place has become vastly more straightforward with Mythos Preview.

This has always been the bottleneck. Automated tools love to flag vulnerabilities, but almost all are false positives. These need to be triaged and evaluated by humans. This is okay. I’d rather close a false positive after a careful review than miss it altogether.

I don’t think it’s appropriate for calling out humans as a bottleneck. They are an essential part of the process, I’m sure Mythos will also become a catalyst in the process.

mdeeks•about 2 hours ago

You can get a taste of this today yourself with Codex Security. I turned it on just as an experiment and in less than a week it has now become essential to all of us. I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.

I'd say it is about 90% accurate for us. Often even the "Low" findings lead us to dig and realize it is actually exploitable. Everyone makes these mistakes, from the most junior to the most senior. They are just a class of bugs after all.

I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. Highly recommend you get something enabled for your own repos ASAP

winstonwinston•about 2 hours ago

> I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO.

So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?

ygjb•about 2 hours ago

Yeah, with a budget assigned. This is actually just software development and security right?

Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software.

Whether that fits into the business model, or the value proposition of spending tokens instead of engineer hours or user hours is fundamentally a risk management decision and whether or not the developer (whether OSS contributor, employee, business owner, etc) wants to invest their resources into maintaining the project.

While not evenly distributed, and not perfect, the currently available and behind embargoed tools are absolutely impactful, and yes, they are expensive to operate right now - it may not always be the case, but the "Attacks always get better" adage applies here. The models will get cheaper to run, and if you don't want to pay for engineers or reward volunteers to do the work, then you've got to pay for tokens, or spend some other resource to get the work done.

sandeepkd•about 1 hour ago

Somehow this reminded me of the historical efforts of some government bounty collections for mouse tails which were discontinued due to fraud (such as hunters breeding mice to collect the reward). There is a reason why/how devs and QA keep each other in check. Guess in case of LLM writing code, one has to use different models for dev and security checks.

On other hand, in real world, the developers learn from mistakes and avoid them in the future. However there is no feedback loop with enterprises using LLM with the agreement that the LLM would not use the enterprise code for training purposes

jimmy2times•about 2 hours ago

The AIs have already figured out how to succeed in a software job:

1. Ship bugs

2. Fix them

3. You're the hero!

jimbokun•about 1 hour ago

Dilbert beat you to it:

https://english.stackexchange.com/questions/488178/what-does...

genghisjahn•about 2 hours ago

I thought we were all doing that already?

jstummbillig•about 1 hour ago

Ngl, watching folks getting irritated about normal employer-employee absurdities from the employer perspective through usage of agents and having to pay for tokens has been a little therapeutic for me.

akoboldfrying•24 minutes ago

Absolutely. And not even making the connection.

On a broader scale, the sheer face-eating-leopards-ness of programmers finally automating away our own jobs and then realising how much this sucks, after automating away so many other kinds of jobs, can feel darkly amusing to me too.

raincole•about 2 hours ago

Humans work like that too. If you're not comfortable with Claude involves in every step (for whatever reason) then just use different providers for each.

predkambrij•18 minutes ago

Man, some people like conspiracies. I encourage you to replicate all that.

siva7•about 2 hours ago

So? That's how a business works. We sold you landmines and now you need them removed? Lucky you we also have mine clearance products.

382hi•about 2 hours ago

Exactly!

unethical_ban•about 2 hours ago

How is this supposed to work? Humans generate security bugs, then humans find them, then humans generate the fix, profit?

Yeah. Presumably as AI code generation gets better, the output gets better. As smaller portions of code are stitched together, human/AI systems analyze it holistically to make sure all its integrations are secure and bug free.

In 2026, different models are better at different things. Cheap models can plan and do small/medium code projects well, more expensive models are even better at architecture and exploit discovery.

mnahkies•about 1 hour ago

One issue I've seen with LLM's is adding superfluous code in the name of "safety" and confidently generating a bunch of stuff that was useful in years gone by, but now handled correctly by the standard lib. I'm of the opinion that less is more when it comes to code, and find the trend this is introducing quite frustrating.

How do you avoid this pitfall?

pianopatrick•9 minutes ago

Thinking off the top of my head - couldn't you have an AI scan that looked for such things? Just send every file in the code base to AI one at a time. Have a prompt like "See if there is ABC pattern that can now be handled by XYZ standard library function in this file. Reply YES or NO. {{file contents}}"

Seems you would not need that many tokens to do so and you might find such cases.

tomjakubowski•30 minutes ago

I wonder this too. I prompted Opus 4.7 to generate some Python threading code for me. The code to run the sub-thread looked like this:

    def run():
        with contextlib.suppress(SystemExit):
            do_thread_thing()

    threading.Thread(target=run, daemon=True).start()

Suppressing SystemExit was surprising, and made me curious. I followed up and asked the model: what's the purpose of that?

The model's response: "Honestly? Cargo-culting on my part. You should remove it."

appplication•about 1 hour ago

Gosh this couldn’t be more true, which IMO is the real reason LLM workflows are not strictly faster if you care about quality. Otherwise you end up with a codebase where only 60% of it is necessary. Standard testing patterns also tend not to be great at catching this particular flavor of LLM-ism.

Version467•about 2 hours ago

I’ve had the same experience. The ui is a little unclear about this, because it says you have 5 scans, but 1 scan is just the continuous monitoring of the default branch of a repo.

The high impact findings have almost all been bang on for me. I was especially surprised by the high-quality documentation it produces as well as how narrow the proposed fixes are.

I’m used to codex producing quite a but more code than it needs to, but the security model proposed fixes that are frequently <10 loc, targeting exactly the correct place.

It’s really quite good. I’m assuming it’ll be pretty expensive once out of beta, but as a business I’d be jumping on this.

0xAstro•about 2 hours ago

I would recommend you to try out the setup with gpt-5.5-cyber as the orchestrator and deepseek-v4-flash or some other fast cheap model as its workers. Getting pretty good results using this setup.

rmast•about 2 hours ago

I help maintain a project that is used as a dependency by a lot of security tools to handle PE files.

It’s disappointing that Anthropic and OpenAI never responded to the applications to their respective programs for open source maintainers. From my perspective it seems like their offers are primarily for the shiny well-known projects, rather than ones that get only a few million monthly installs but aren’t able to get thousands of stars due to being “hidden” as a dependency of popular tool.

hollowturtle•26 minutes ago

> I was shocked how accurate it is, how many security issues it found in existing code, how it continually finds them as we commit, and how NO ONE is immune from making these mistakes.

Dude is flexing that he's pushing unsecure code every day, that's a skill!

mukmuk•41 minutes ago

I’m not sure how to reconcile anthropic’s update / some of the exuberant comments here with recent feedback like the following from curl maintainer Daniel Steinberg:

“I see no evidence that this setup [Mythos] finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in code analyzing.”

https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...

moomin•17 minutes ago

You’re right, it’s a valid data point. But the U.K. government report is also a data point, and the Firefox report is a data point, and they suggest that it is, indeed, significantly better than current generation models. Maybe curl is significantly better hardened than most projects?

In any event, it barely matters. As Anthropic acknowledges, next level models are comings, theirs is only one of them. Current generation models are already good at things like tracing data flow through complex systems and there’s no reason to think that capability has topped out. So within a year it seems very likely we’ll have more than one commercially available model able to find vulnerabilities cheaply.

On the other hand, it seems that they’ve made much less progress on getting it to design solutions to these issues.

ZrArm•11 minutes ago

> Maybe curl is significantly better hardened than most projects?

Meanwhile from [1]:

"Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day."

"The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They find lots of new problems no one detected before. And none of these new ones used Mythos. Focusing on Mythos is a distraction - there are plenty of good models, and people who can figure out how to get those models and tools to find things."

Yeah, it looks like there are at least 11 security bugs missed by Mythos.

[1] https://www.linkedin.com/feed/update/urn:li:activity:7463481...

skybrian•31 minutes ago

Different people can have different experiences without contradiction. Maybe the curl source code was pretty clean to begin with?

dreambigwrkhard•20 minutes ago

imo curl is quite well maintained. there are a lot of sloppy projects out there and tools like this shows whos been swimming with their pants down. not saying any project with vulnerabilities are sloppy but when costs of finding bugs and vulnerabilities decrease significantly, they will get exposed with enough time and tokens ($)

kadoban•19 minutes ago

Curl has more eyes on it, and has had more tools thrown at it, and is better tested (and developed?) than 99% of software, it's very much not the norm. I wouldn't be surprised if that has something to do with it, if there is any kind of bias there (not sure if there is, it's also possible he's just right).

mayneack•23 minutes ago

Daniel has been posting for months (years?) about how much scrutiny he gets from security researchers and various automated tools. I wouldn't expect curl to be the average case for mythos.

3kahg•15 minutes ago

It is the opposite. Security people focus on curl, sudo because they are code bases that contained a lot of features and unused code from the 1990s.

They don't focus on projects where they find nothing. They certainly don't advertise when they find nothing.

Getting a lot of scrutiny is not the recommendation that it appears to be. What is the new standard? Projects that never have bugs are deemed to be suspect because they "have not been scrutinized" (they have, but null results never go public)?

So Mythos only finding one issue after other tools have found 300 this year is embarrassing. Mythos was supposed to be better and novel.

elisbce•17 minutes ago

He already scanned the codebase with Codex Security and a whole bunch of other AI tools, and fixed 200-300 bugs and CVEs. On top of that Mythos found 1 more bug and 1 more CVE is already impressive.

demorro•about 1 hour ago

If you're not already applying static analysis and linters to your codebase (and I know many of you aren't), ask yourself why you would bother to apply an expensive LLM tool?

Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons.

If you're a team that does already apply several layers of analysis and linting, and wants to add this on top, all power to you.

SkyPuncher•about 1 hour ago

> If you're not already applying static analysis and linters to your codebase

Because most issues are in business logic that static analyzers aren't going to catch.

sobellian•about 1 hour ago

Static analysis often shows many false positives. A more intelligent tool can help not to waste limited engineering time.

seanmcdirmid•17 minutes ago

False positives are noise, but if the tool is filtering out its own noise via AI, it might work. Or you could take a high false positive/low false negative tool and instead of bothering humans with its noisy output, have AI investigate and evaluate if found issues are false positives or not.

nikcub•about 2 hours ago

There has been a lot of cynicism around mythos, that it's just the usual public models without guardrails, etc. etc. but this:

for anybody who has applied opus, codex or oss models for vuln scanning - the true positive rate and discovery volume are a clear step change[0]. The ~50 partners in Glasswing have largely all previously run harnesses with other models and many of them have come out and said - essentially - "ye, wow"

Question now is what a second and third phases of access looks like - deciding which class of systems to secure. Routers, firewalls, SaaS, ERP systems, factory controllers, SCADA systems, zero-trust VPN gateways, telecoms gear and networks, medical devices - there's just so much to do

This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.

That may suit Anthropic as private models can't be distilled. There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.

I don't see how Chinese companies are given access soon, or ever. We're likely going to see a world soon of CISA mandated audits, and where to buy a mythos-proof VPN gateway or home router - you'll have to buy American[1].

[0] vs ~30% or so in regular audit tools

[1] or allied

skybrian•12 minutes ago

I don't see why they couldn't contract out to an American security firm that has access?

gck1•about 1 hour ago

> This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.

sigh I remember the GPT-2 days - when it was the first time OpenAI restricted access to the models citing "humanity is not ready for it". The model was good at writing poetry or something.

Since then, I don't remember a single model announcement from OAI/ANT that didn't use similar wording.

The so-called leak of model announcement was marketing, it being dangerous is marketing, the world not being ready for it is marketing. And yes, the ones that were given access to saying "oh wow", believe or not, is also marketing.

It's all marketing. You can get the same results from any of the top-5/10 models that are generally available already.

Mythos is Anthropic's way to sell the new idea, because the previous one has democratized.

NitpickLawyer•about 1 hour ago

Writing marketing 10 times doesn't invalidate the (many) claims from many respectable sources that the model is a step change in cybersec. There's also the report [1] from the Brits that track cyber capabilities since '22 or '23 and they've also confirmed it's a step change (together with 5.5 cyber or whatever they call it).

Marketing is like propaganda. It doesn't need to be based on false facts. Of course they're gonna milk it, keep it private and so on. But that doesn't mean the model is bad. Or that others are as good (apparently they're not there yet).

[1] - https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

casey2•about 1 hour ago

Please don't misrepresent the article it says clearly "a step up in cyber performance over previous frontier models" and that gpt-5.5 is on their tests is slightly better than mythos.

solenoid0937•about 1 hour ago

I think you just aren't reading the post, or any of the Glasswing partner's posts. You have this view in your head of what Mythos is, and nobody can say anything dissuade you from it.

gck1•38 minutes ago

"Partners" is the important word in your comment. I am reading all of it, but I have a huge barrel of salt to consume along with everything that I read, because I see conflicts of interest everywhere I go, with fancy words and no means to verify.

If I was given free access to any frontier model to use on my projects, equivalent of millions of dollars in AI credits, I sure hope people didn't trust anything that came out of my mouth until they were able to verify my claims themselves.

AI industry has even resulted in a new term - benchmaxing - which essentially means we can't even trust the data anymore until we can touch the model ourselves. So this is not at all surprising to me. What's surprising is why am I in the minority here, and since when trusting authorities that have obvious conflicts of interest became normal.

Amekedl•about 1 hour ago

Agreed, also amazing citations in the parent comment ^^

piker•about 1 hour ago

We have been working with the consumer-grade frontier models to develop what we call "lexploits" in legaltech, and they are insanely good at finding bugs across integrated pipelines. They're also surprisingly good at mitigating them!

Security vulnerabilities are one thing, but in legal we offer up a concept of "knowledge security" which goes to protecting the fidelity of the agent's legal context. Software bugs seem much more tractable because they're managed by software engineers, as opposed to the pipeline "vulnerabilities" we're finding. We wrote a little about one vector here where legal documents aren't quite what they seem: https://tritium.legal/blog/noroboto

No doubt there are many such knowledge domains exposed today. These are more concerning because they're understaffed and managed by non-technical people for the most part. No Mythos required.

Amekedl•about 1 hour ago

I don't buy it. A lot of stuff this finds is also just simply wrong, benignly reported as true, despite upper/lower layers in the code burying the possibility of a vulnerability actually being exploited. It's a performance/security trade-off too, it always has been. Additional checks and other measures do in fact need to be performed for security purposes.

Great marketing as always, but the rose-tinted view many have seems vicariously misplaced.

darkamaul•38 minutes ago

I guess you could look at https://red.anthropic.com/2026/cvd/ to see exactly what was discovered.

Amekedl•34 minutes ago

Thank you. Looking at the WebDAV in nginx, this is exactly what I searched for, wanted to read, and confirmed my suspicions ^^ But this one takes the cake truly... https://red.anthropic.com/2026/cvd/findings/ANT-2026-CN7KX43...

solenoid0937•about 1 hour ago

In the article they describe how all the vulns are actually exploitable end to end and >1000 have been independently verified as critical.

These aren't unreachable vulns.

Amekedl•43 minutes ago

Where is the link to the advisories then? :/

skybrian•27 minutes ago

As the article explains, they mostly haven't been disclosed, because they're not fixed. They're giving people 90 days, or 45 after a patch is made.

rafgg•18 minutes ago

Specially when this has been OAI/Anthropic's MO for years at this point.

mmsc•6 minutes ago

Aisle has hundreds of CVEs with publicly available models: https://aisle.com/wall-of-fame

cpard•about 1 hour ago

My understanding so far is that that Mythos (and any model in general) can produce candidate reasoning but you really need a system around that reasoning that is capable of producing auditable security findings.

So, success is coming not just from the model but also from the harnesses they built around it. The Cloudflare post was more detailed on that front and I wish the rest would share more about it.

The Cisco spec is interesting too, it pretty much describes an architecture of a harness: https://github.com/CiscoDevNet/foundry-security-spec

mixologic•about 2 hours ago

Right now the only codebase I care about them fixing vulnerabilities in are the 3800 repositories that got stolen from GitHub.

"Vulnerabilities in the software that makes the internet" is honestly lower priority than "The platform that the software that makes the internet uses to make releases" If buyers of those internal repos find ways to break into GitHub such that they can cut software releases, or poison github actions from a distance, then we're all in a very ugly mess.

Don't forget that in those 3800 repos is likely also npmjs.org itself.

0xAstro•about 3 hours ago

I had a fun day today where I had deepseek-v4-flash subagents work out patch for dirty frag for systems with AF_ALG disabled and nscd turned on, to gain root access. The original published exploit wasn't working but the patched one worked like a charm.

I am still a believer that a 100 subagents with good-enough intelligence can get same results as mythos, I am ready for this opinion to be shattered when I eventually try mythos and I believe others here must have tried mythos out too.

lukeschlather•about 2 hours ago

That's probably true, but when you're talking about 100 subagents you're talking about something that costs $100/hour to run, and Mythos takes $20k to find a vulnerability, so the question isn't "can dumber models conceivably do this?" It's, if running inference with Mythos to find an exploit costs 5000 GPU-hours per exploit, how many GPU-hours does it cost with a dumber model?

rsync•about 2 hours ago

I asked in a different thread:

Do we have a sense that projects like OpenBSD/OpenSSH, FreeBSD, ISC[1] and Apache were included in the "blessed" initial participants in Project Glasswing ?

Or is it big name tech companies, banks and fashionable languages and package managers ?

[1] Bind, DHCP

ls612•about 2 hours ago

“Oi, you got a loicense to make secure software there?”

I joke but that is the world we are moving towards. I don’t think many on HN have thought through the second and third order implications.

chopete3•about 2 hours ago

>> Next, we will work with critical partners—including US and allied governments—to expand Project Glasswing to additional partners.

That means, they intend to make a load of money before a general release. It is a good strategy.

OsrsNeedsf2P•about 3 hours ago

The vulnerabilities found continues to impress, and make legacy media, Twitter and Youtube go nuts. But we still have no data to prove this wasn't doable with the same initiative backed by Opus 4.7, and there is no GA for Mythos access.

krisbolton•about 3 hours ago

There is independent research out there on frontier model security capability. AI Security Institute (UK) put out their paper comparing Mythos to other frontier models in early April. They've been tracking frontier model security capability since early 2023, so it's a decent dataset. https://www.aisi.gov.uk/blog/our-evaluation-of-claude-mythos...

energy123•about 3 hours ago

. Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;

applfanboysbgon•about 3 hours ago

Did they allocate the same number of tokens to looking with Claude 4.6? Or did they find more because they looked more, owing to a special initative by Anthropic?

properbrew•about 3 hours ago

> over ten times more than they found in Firefox 148 with Claude Opus 4.6

And how much with Opus 4.7? 5x?

kllrnohj•about 2 hours ago

No, not really. Mythos found 3 CVEs, not 271.

https://www.flyingpenguin.com/mythos-mystery-in-mozilla-numb...

simonw•about 2 hours ago

The Mozilla team responded to that argument here: https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin... - in the FAQ.

moyix•about 2 hours ago

I think you're confusing CVEs and vulnerabilities here? Mozilla (per their longstanding practice) grouped multiple vulnerabilities found internally under a small number of CVEs.

ospray•about 2 hours ago

This report is far more positive with a far lower false positive rate than I was expecting based on reports from the curl team and a few others. I guess I have just been hearing about the ten percent misses. Can anyone not employed by Anthropic who has used it vouch that it is equal to general human testers and do you need xbow to make it that way.

parker-3461•about 3 hours ago

Makes me wonder if Anthropic is really having issues with allocating compute (see recent deals with xAI and SpaceX). From available benchmarks, it seems like similar results should be possible with GPT 5.5 Pro or Opus 4.7 (with specific cybersecurity trained models).

smoe•about 3 hours ago

At least according to this, GPT-5.5 Cyber is on par with Mythic, as the only two models that were able to finish their 32-step corporate network attack simulation.

https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

wiwiwq•about 3 hours ago

Who knows but from a valuation stand point it’s better to signal that demand is higher than existing capacity..

arjie•about 2 hours ago

The era where you could reputably believe things published by anyone on this front is over. If you want this information, you’re going to have to attempt it yourself with the Opus API. It is entirely possible that any released model access will be heavily guardrailed against hacking attempts and Mythos is just an unrailed model. It is entirely possible that Mythos is a different architecture or size. We can’t know from the outside.

There is also a pretty big risk that anyone who is not you would leak the answer to the test. We are close to n=1 epistemics here. You’re going to have to do the research yourself.

MallocVoidstar•8 minutes ago

> It is entirely possible that any released model access will be heavily guardrailed against hacking attempts

Yes, Anthropic have said they made Opus 4.7 worse at this on purpose.

> It is entirely possible that Mythos is a different architecture or size

It has 5x the token pricing of Opus 4.7, so it's probably larger.

pertymcpert•about 3 hours ago

> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6

4.6 but close.

OsrsNeedsf2P•about 3 hours ago

Right, but were they using the same methodology and harness? I'm skeptical that they're doing something with the harness - i.e. with Mythos, they pass each file in one at a time, whereas on 4.6 they let Claude Code run loose to find bugs. This would have a larger impact difference than the model itself.

ZrArm•41 minutes ago

From Mozilla post [1]:

"...After fixing the initial set of issues that Anthropic sent to us in February, we built our own harness atop our existing fuzzing infrastructure.

We began with small-scale experiments prompting the harness to look for sandbox escapes with Claude Opus 4.6. Even with this model, we identified an impressive amount of previously-unknown vulnerabilities which required complex reasoning over multiprocess browser engine code..."

So yeah, Anthropic and Mozilla likely compare "Amount of bugs found by Opus 4.6 during early experiments" vs "Amount of bugs found by Mythos during large-scale codebase scanning".

[1] https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...

mpyne•about 2 hours ago

Yes, the harness they used actually existed and was in use beforehand, it wasn't developed for testing with Mythos.

boston_clone•about 3 hours ago

you would likely be quite interested in the more quantitative writeup from a real research team ! it’s linked about midway in to the article - similar functionally can be reached, yes, but not always and never with fewer tokens than what mythos requires.

https://xbow.com/blog/mythos-offensive-security-xbow-evaluat...

OsrsNeedsf2P•about 3 hours ago

Ok this is actually a pretty good article and justifies the step function marketing in security they talked about

bobbycastorama•about 3 hours ago

I've seen a blog post by a security researcher saying that he was able to find the same vulnerabilities (for Firefox IIRC) with a ~30B params LLM...

So yeah, huge marketing as always.

simonw•about 2 hours ago

You mean this one? https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag...

That's the one that says:

> We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis.

krisbolton•about 3 hours ago

This is different though right? He found one (? we don't know who you're referring to - post sources for a higher quality discussion) vulnerability, he already knew it was there, etc. Anthropic didn't claim no other model can find vulnerabilities, nor that it's impossible with smaller models. They're claiming Mythos is a step-change in ability for end-to-end vulnerability discover and exploit creation. And that other frontier models are close behind.

Brystephor•about 3 hours ago

Did the security researcher point the LLM at the blob of information and say "Find vulnerabilities" or was the LLM told to "determine if vulnerability X is present in this blob"? Confirmation of suspected vulnerabilities is a different problem from finding vulnerabilities.

nikcub•about 2 hours ago

Finding the neeedle is easier when you remove the haystack

Or providing a map with a direction

There is a long history of high-value private vulns being rediscovered from scant details

wiwiwq•about 3 hours ago

To me it’s clear what’s going on.

The American firms are focused on marketing now to convince people to not even consider open sourced models / open weight models as they are inferior (that’s what they want you to believe).

rhubarbtree•about 3 hours ago

IPO is coming is what is going on

enlightenedfool•about 3 hours ago

Is this the God model that no one else can build? Unbelievable.

vb-8448•about 1 hour ago

The report on findings is very interesting: 1451 acknowledged findings out of 23k candidates(~6%, not high but neither low).

But I didn't find the most important information (or maybe I missed it): how much did it cost to find 1451 security bugs?

gpugreg•about 1 hour ago

We can at least put an upper limit on it. From https://www.anthropic.com/glasswing

    Claude Mythos Preview will be available to participants at $25/$125 per million input/output tokens
    ...
    Anthropic is committing up to $100M in usage credits for Mythos Preview

Although I'd expect reduced prices for cached tokens, which is not mentioned on their website at this point in time.

bevekspldnw•about 2 hours ago

How much of this is RL’ing a good coding model on every CVE ever?

sometimelurker•about 1 hour ago

most it this comes from the pretrain imo. just scale + some RL = mythos

jimmar•about 2 hours ago

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.

brightbeige•about 1 hour ago

And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans.

Here are two experimental exceptions:

https://github.com/vercel-labs/zerolang

https://github.com/sbhooley/ainativelang

dmix•about 1 hour ago

Not just the languages but frontend/user interfaces as well. You can see the potential for the future when using Claude Design->Claude Code->Agents live testing in BrowserOS. It's all modeled on existing humans patterns of using Figma passing to devs then testing after the fact before starting the loop again, while a lot gets lost in translation in between the designs and the code.

We'll like have some standard AI-focused UI libraries that are harnessed into a design gen system where an AI can pull all the real levers, while also developing a large training data set around it.

Oarch•39 minutes ago

I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.

vb-8448•about 1 hour ago

I just wonder how many of those 1451 acknowledged findings were introduced by LLMs ...

morpheos137•about 2 hours ago

there is little evidence for this prediction.

sp527•about 2 hours ago

Oh there's plenty of evidence. Because a lot of these people have been committing to repos in public for over a decade. Wouldn't take much to show the world just how fallible human coders really are.

cubefox•about 2 hours ago

The rapid progress in the last few years in this regard is pretty strong evidence in my opinion.

morpheos137•about 1 hour ago

https://news.ycombinator.com/item?id=48225426

there is a difference between a stunt and a viable product. diverless cars and agi are the fusion of Silicon Valley.

cheesefck•about 2 hours ago

Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails.

This is the MoviePass era of language models

Flere-Imsaho•6 minutes ago

Actually I think with flying cars it's more of a problem with noise, regulation, risk, etc than a technological problem.

Supersonic again is a problem with noise and cost rather than technological.

Self driving is definitely a technological problem.

ayeeeeeeeeee•about 2 hours ago

It would be informative to publish not only vulnerability numbers, but also vulnerability type statistics (as available here for example: https://cvedb.github.io/years.html), such that programmers can understand which types of exploits popular systems and languages commonly allow, and thereby encourage fundamental changes to fix or transition away from them.

dmix•about 1 hour ago

I wonder if Apple took part in the project

sandeepkd•about 1 hour ago

> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.

> For example, at one of our Glasswing partner banks, Mythos Preview helped to detect and prevent a fraudulent $1.5 million wire transfer after a threat actor compromised a customer’s email account and made spoof phone calls.

For some reason I am not able to relate to the concreteness of either of these.

First half of the page was occupied with a image, not sure if it was relevant in any ways other than setting up security scare. The size of code base, number of tokens, $ involved seem to be out of scope of the update for some reason. Personally I am getting skeptical about all these optics at this point, just some money printing scheme at high level.

mikmoila•about 2 hours ago

Code contains deviations from assumed behaviour, and some behaviours might manifest themselves as failures. Some failures might be exploitable by attackers.

antirez•about 2 hours ago

I have the feeling posts like that should be 1/4 the size, at max. At this point I don't care if it is AI-slop or human-slop: they are surprisingly alike. Information must be more dense, each sentence must carry some truth.

0xbadcafebee•about 1 hour ago

Benefit of AI: it works fast

Drawback of AI: it works fast

spullara•about 1 hour ago

I'm going to code myself up a new minivan.

vincefutr23•about 2 hours ago

Mythos couldn’t find the “tens thousand” typo in this post?

mlazos•about 2 hours ago

I believe them to some degree but this trend of posting stuff when it can’t be verified actually needs to end. I’m so tired of this bs marketing.

ares623•about 2 hours ago

> good lord what is happening in there?!

> that's just thousands of vulnerabilities being discovered by our trillion parameter model

> thousands of vulnerabilities and trillions of parameters?! At current energy prices, in this economic climate, isolated entirely within your datacenter?

> yes

> may we see it?

> no

pixl97•about 2 hours ago

I built a missile that can blow you up.

>ya right.

Here's a demonstration of it blowing something up.

>can I have one.

No.

kalashvasaniya•about 2 hours ago

this is INSANEEE

ZrArm•about 1 hour ago

> After one month, most partners have each found hundreds of critical- or high-severity vulnerabilities in their software.

And at the moment we have reports from like around 5(?) companies. Btw, Palo Alto Networks has found only 26 vulnerabilities [1]. I'm interested what those partners are and why they have such big amount of vulnerabilities.

Yet decided not to share that number. I wonder why.

> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;

Mozilla tested Opus 4.6 in a very limited setting (i.e. without proper harness and integration into their workflow; likely without large-scale codebase scanning). It's an incorrect comparison.

> The latest Palo Alto Networks release included over five times as many patches as usual.

Yeah, it's better to say "five times as many..." rather than "26 bugs". Btw, they also used GPT-5.5 and Opus 4.7, so the contribution from Mythos there is unclear.

> Microsoft has reported that the number of new patches they’ll release will “continue trending larger for some time.” And Oracle is finding and fixing vulnerabilities across its products and cloud multiple times faster than before.

Both Oracle and Microsoft are talking about "AI and cybersecurity" in general, not about Mythos.

> For the last few months, Anthropic has used Mythos Preview to scan more than 1,000 open-source projects, which collectively underpin much of the internet—and much of our own infrastructure. > So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).

So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.

> One example of an open-source vulnerability that Mythos Preview detected was in wolfSSL, an open-source cryptography library that’s known for its security and is used by billions of devices worldwide. Mythos Preview constructed an exploit that would let an attacker forge certificates that would (for instance) allow them to host a fake website for a bank or email provider. The website would look perfectly legitimate to an end user, despite being controlled by the attacker. We’ll release our full technical analysis of this now-patched vulnerability (assigned CVE-2026-5194) in the coming weeks.

Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.

Overall, it feels like yet another marketing stunt.

[1] https://www.paloaltonetworks.com/blog/2026/05/defenders-guid...

orangebread•about 2 hours ago

BOOO RELEASE THE MODEL ALREADY GAWD

guluarte•about 2 hours ago

after IPO

InsideOutSanta•about 3 hours ago

I wonder if it coincidentally becomes safe to release when compute capacity bought from SpaceX will provide enough headroom to let a lot more people run it.

lukeschlather•about 2 hours ago

It seems like Mythos is often (or typically?) costing $20k per vulnerability, so I don't think there will be enough compute capacity in the world any time soon to let a lot more people use it the way Glasswing is using it. That is not to say I think they are exaggerating its capabilities. That $20k is presumably the rough cost of renting the GPUs, and there are not enough GPUs in the world.

InsideOutSanta•about 2 hours ago

I'm not sure if current pricing correlates with actual compute cost.

why_only_15•about 2 hours ago

what's the origin of your $20k/vuln estimate?

gck1•about 1 hour ago

It's the same as the origin of "Codex/Opus subscription usage is heavily subsidized" - the sales departments equipped with AI agents with the prompt: "use anonymous accounts on the internet to make it easy for me to sell it at $price".

sigmar•about 2 hours ago

"available to qualifying customers’ security teams on request." Seems they're already expanding access.

unethical_ban•about 1 hour ago

Total speculation: As the software world shakes out the many hidden vulns in their software, big AI will try to limit the access while it gets ironed out. Once the big projects/systems are reasonably patched after being vetted by SOTA models, the models will be released to the public. I don't think there's a scenario where Mythos-level or better models stay closed permanently.

b65e8bee43c2ed0•about 2 hours ago

stop noticing things, chud.

amusingimpala75•about 3 hours ago

[edit: TFA addresses this, though I still find crazy 90% accuracy overall vs 20% accuracy for curl]

Is this suspected vulns or actual vulns? If I recall correctly, it produced 5 for curl but only 1 was legit

Smaug123•about 3 hours ago

> So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).

> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity. That means that even if Mythos Preview finds no further vulnerabilities, at our current post-triage true-positive rates, it’s on track to have surfaced nearly 3,900 high- or critical-severity vulnerabilities in open-source code

extr•about 3 hours ago

Did you RTFA?

rbranson•about 3 hours ago

I don't know why you're getting downvoted. This is exactly what was reported by curl's creator under the section "Five findings became one": https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...

Smaug123•about 3 hours ago

I think it's more that the requested information is prominently featured in the article, and indeed is the content of the only graphic in the article below the intro banner.

the_mitsuhiko•about 2 hours ago

And yet [1]:

> Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day.

> 11 CVEs announced in a single release is our record from 2016 after the first-ever security audit (by Cure 53).

> This is the most intense period in #curl that I can remember ever been through.

[1]: https://www.linkedin.com/feed/update/urn:li:activity:7463481...

hiharryhere•about 1 hour ago

He’s talking about AI scanning tools collectively, not specifically Mythos.

If you read his own top comment on that LinkedIn post he clarifies:

“The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They find lots of new problems no one detected before. And none of these new ones used Mythos. Focusing on Mythos is a distraction - there are plenty of good models, and people who can figure out how to get those models and tools to find things.”

RamRodification•about 3 hours ago

This is marketing. So probably suspected. Or somewhere in between.

giancarlostoro•about 2 hours ago

> Since then, we and our approximately 50 partners have used Claude Mythos Preview to find more than ten thousand high- or critical-severity vulnerabilities across the most systemically important software in the world. Progress on software security used to be limited by how quickly we could find new vulnerabilities. Now it’s limited by how quickly we can verify, disclose, and patch the large numbers of vulnerabilities found by AI.

I guess they forgot to scan Visual Studio Code plugins and their endless npm dependencies.

pixl97•about 2 hours ago

I mean that's really a different issue.