Shai-Hulud Themed Malware Found in the PyTorch Lightning AI Training Library

People are ramming tons of code into places without ever looking at it, it would follow that supply chain attacks would increase thusly.

No need to invoke frequency illusion when every moderate HN lurker already stopped counting. https://socket.dev/blog gives a good impression, but a dedicated article would be nice. Maybe recurring once or twice a year.

If you're interested in synchronicity and frequency illusion, Sergei v. Chekanov wrote a book that sounds interesting https://jwork.org/designed-world/

Have you ever experienced coincidences that cannot be logically explained? This book helps the readers understand the meaning of synchronicity, or remarkable coincidences in people's lives. This work not only explains the mystery of synchronicity, originally introduced by Carl Jung, but it also shows how to make simple calculations to estimate the chances that coincidences are not due to mere randomness.

> Looking back ten years to `left-pad`, are there more successful attacks now than ever? I would suspect so, and surely the value of a successful attack has also increased, so are we actually getting better as a broad community at detecting them before package release?

The value has increased, and that is what drives all these attacks. Cryptocurrencies are to blame in particular because they not just provided a way for money laundering the proceeds but also a juicy target in itself.

And what is stolen with today's malware? Cloud credentials. Either to use for illicit mining, which is on the decline, or to run extortion campaigns, which is made possible by cryptocurrencies. All too often it's North Korea or Iran running these campaigns.

> Looking back ten years to `left-pad`, are there more successful attacks now than ever?

I can't vouch for the number of attacks, but, and since we are talking about Python, nothing substantially changed since the time of `left-pad`. The same bad things that enabled supply chain attacks in Python ten years ago are in place today. However, it looks like there are more projects and they are more interconnected than before, so, it's likely that there are either more supply chain attacks, or that they are more damaging, or both.

Here's my anecdotal experience with Python's packaging tools. For a while, I was maintaining a package to parse libconfuse configuration language. It started as a Python 2.7 project, but at the time there was already some version of Python 3 available, so, it was written in a way that was supposed to be future-proof.

I didn't need to change the code of the project in the last ten or so years, but roughly once a year something would break in the setup.py. Usually, because PyPA decided to remove a thing that didn't bother anyone.

When Python 3.13 came out, as clockwork, setup.py broke. I rolled up my sleeves and removed the dependency on setuptools, instead, I wrote some Python code that generated a wheel from the project's sources. I didn't look up the specification of the RECORD file in dist-info directory, and assumed that sha256().hexdigest() will generate the checksums in the desired format. And that's how I shipped my packages...

Some time later, the company added an AI reviewer to the company's repo and it discovered that instead of hexdigest() the checksums have to be base64-encoded and then padding removed...

Now, to the punchline: nobody cared. The incorrectly generated packages installed perfectly fine without warnings. Nobody checks the checksums.

More so: nobody checks that during `pip install` or the more fancy `uv pip install` the packages aren't built locally (i.e. nobody cares that package installation will result in arbitrary code execution). It's not just common, it's almost universal to run `pip install` on production machines as a means of deploying a Python program. How do I know this? -- The company I work for ships its Python client as a... source package. Not intentionally. We are just lazy. But nobody cares.

The problem with this is now you are solely responsible for managing all of the changes, all of the variation of life. Chrome changed the shape of this API, you are responsible for finding it and updating it. Morocco changed when their daylight savings took effect, now you need to update your date/time handling code. There are a lot of these things that we take for granted because our libraries handle it for us, and with no dependencies you have to do all the work. Not a big deal for making a double-pendulum simulator for your daughter to play with that will stop mattering next week, but is a concern for a company which is trying to build something that can run indefinitely into the future.

I am torn because I like rust over go, and rust is better from an LLM perspective. But the dependency philosophy on rust is basically a security blackhole whereas go is much better.

well surely Opus would never introduce vulnerabilities into the code so that sounds like the solution.

And of course, you will go over every line of code that Opus produces with the same scrutiny we expect of open source maintainers, right? Right?

I'm going to go publish some MIT-licensed remote access code and get that into Opus's training data.

I think in the relatively near future we’re going to start seeing sophisticated supply chain attacks into language model training data.

It should be feasible to design vulnerabilities which look benign individually in training data, but when composed together in the agent plane & executed in a chain introduce an exploit.

There’s nothing technical really stopping that from existing right now. It’s just that nobody has put the effort in yet.

Your LLM isn't a dependency?

The repository names all look like two terms/words from dune (harkonen, mentat, ornithoptor, etc.) followed by a number. This would indicate that the account (possibly GitHub auth/actions token) has been compromised and then used to create the repository.

Why can't GitHub get on the case and just block any repo where the README matches the regex? I thought they'd have learned their lesson the last time it happened.

This malware isn't even trying. Then again it's Microsoft so they're not even trying either.

what's this all about?

> If I understand correctly, the source code for building Python is not fetched directly from python.org. Not so sure how secure is that.

python-build-standalone fetches CPython sources directly from python.org[1]. I don't even know where else we would get them from!

[1]: https://github.com/astral-sh/python-build-standalone/blob/a2...

i mean... uv is already a binary you run on your computer to manage python binaries, packages (and any binaries with those), systemwide tools etc; how much does it change whether they build the python binaries or someone else?

What filter?

You say you rely on CC to suggest software to install from the internet, and then you install it.

I haven't heard anyone suggest CC or any LLM as a "filter" for "is this package safe right now", and it seems like a very bad heuristic to me, not only, but also for the reason you gave.

By "the worst possible filter" do you mean "hitting enter when claude tells you to"?

There are many people in that ecosystem, who are not primarily software engineers. Some just learned some coding along the way. Some are mathematicians. Some are devs who are AI drunk or something. Some have the mindset of "code doesn't matter any longer, if it works it works". For many proper dependency management is just a chore, that they don't want to care about. These things come together in various ML projects, even though ML projects should be amongst the projects most focused on reproducibility.

For those using uv: https://docs.astral.sh/uv/reference/settings/#exclude-newer

Andy from Lightning here. The malicious code was not submitted to the main repo at Github. It appears our PyPi credentials were leaked and compromised packages were published directly there for versions 2.6.2 and 2.6.3

Oh shit I had assumed PyTorch Lightning was affiliated with PyTorch. Not a great name for an unaffiliated third party thing.

Yep. Lighting is a blatant, shameless rip off of Jeremy Howard's FastAI library BTW.

Yes

Nixpkgs uses the GitHub source, not the PyPI dist, for lightning; unclear to me from the advisory whether this should also be considered compromised.

It's always crypto. A client got some AWS credentials stolen and without anyone checking the account, the hacker managed to spin up big EC2 instances across many regions. The bill after a month as I recall was around 100K. Since the activity was clearly fraudulent the bill was forgiven eventually. So remember to lock down your AWS keys permissions...

When I was a kid, we've been told to be cautious with third party dependencies, that code can do anything and it's a risk to evaluate.

With the new generation of yolo NPM scripters, they simply don't evaluate the risks. They will even fight back telling you that it's the way of doing things.

In reality, it's the warning we learnt back then, that's the result of be mindlessly importing third dependencies without thinking.

In other words, the risks were always there, the new "modern way", let's put it that way, doesn't put the effort anymore.

Yeah, pin the malware :p

Shai-Hulud dug my 100 ft trench. Should be OSHA compliant right?

this is fine, we are definitely a perfectly normal industry that knows what it is doing