Cloudflare responded to the "Copy Fail" Linux vulnerability

mmobeigi about 2 hours ago 43 commentsRead Article on blog.cloudflare.com

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

33% Positive

Analyzed from 638 words in the discussion.

Discussion (43 Comments)Read Original on HackerNews

john_strinlai•about 2 hours ago

this is a techincal dive into how cloudflare responded, not a confirmation that they responded

for whatever reason, unknown to me, hn automatically strips "how" from the start of titles. i cant remember ever seeing a title where this was an improvement.

trollbridge•about 2 hours ago

Starting a title with “How” is standard clickbait.

Goronmon•about 1 hour ago

If we are taking that attitude why not go all the way?

Titles are standard clickbait.

sammy2255•about 1 hour ago

Any Cloudflare employees reading this, your network map has a few PoPs missing from it https://www.cloudflare.com/network/ notably, Perth (PER) Australia. Hobart (HBA) Australia. Wellington (WLG), New Zealand. Christchurch (CHC), New Zealand. Nausori (SUV), Fiji.

srcreigh•about 1 hour ago

It’s fascinating that already had a system which could identify the exploit at runtime. How can I learn more about that?

skinfaxi•about 2 hours ago

Would love to learn more about their internal behavioural detection program.

> One of the first things our security team did was confirm that our existing endpoint detection would catch this exploit. Our servers run behavioral detection that continuously monitors process execution patterns. It doesn't rely on knowing about specific vulnerabilities; it watches for anomalous behavior across the fleet.

CGamesPlay•about 2 hours ago

Would certainly be interesting to learn more about. A simple check: allowlist of known "processes that run as root". Any new process shows up, something happened.

jeffbee•about 2 hours ago

Based on what? Proc title?

parliament32•about 2 hours ago

It's curious they're just "monitoring" rather than preventing.

In a serious environment you'd run IPE with dm-verity/fs-verity to ensure binaries are whitelisted and integrity-checked at every execution.

CGamesPlay•about 2 hours ago

Proc title is very easily forged (without root even). Obviously a real privileged process could modify the kernel and do whatever it wants, but if I were trying to detect this I would start with /proc/$id/exe.

mkj•42 minutes ago

If they're already running a custom Linux kernel build, why did they have AF_ALG enabled? Seems the perfect situation to limit features to only those actually being used.

computerfriend•22 minutes ago

In the article they explain that some of their services use it.

PunchyHamster•about 1 hour ago

for us it was

* Get list of modules from Puppet's facts, confirm module isn't used anywhere (it wasn't) * `install algif_aead /bin/false` in /etc/modprobe.d/disable-algif.conf * Run a check using exploit code to check it is no longer working

I imagine CF runs more stuff that could use it I guess but apparently it's not often used API

cube00•about 1 hour ago

> At the time of the "Copy Fail" disclosure, the majority of our infrastructure was running the 6.12 LTS version

That could be as low as 50.1%, I wish they'd provide an actual percentage.

jmclnx•about 1 hour ago

> Linux kernel build based on the community's Long-Term Support (LTS)

CopyFail only highlights why Companies want LTS. If there was a supported kernel built prior to 2017, most large companies would still be on that version, avoiding this issue all-together.

The corporate mindset is usually "never upgrade unless there is new hardware needed or critical software failure". All CopyFail did was reinforce that mindset.

I wonder if CopyFail will cause enterprises put pressure on the Linux Foundation to maintain a "ultra LTS" were it is supported for 20 years ?

PunchyHamster•about 1 hour ago

> CopyFail only highlights why Companies want LTS. If there was a supported kernel built prior to 2017, most large companies would still be on that version, avoiding this issue all-together.

Sadly not really how it works for say Red Hat. They routinely backport features while keeping whatever "stable" number on kernel. We even had displeasure of them backporting a bug... same bug to 2 different RHEL versions

tempest_•10 minutes ago

The longer you wait the more painful the switch will eventually be.

dboreham•about 1 hour ago

The "Hunting for Exploitation" section is unclear to me: "The exploit leaves a distinctive trace in kernel logs when it runs." Hmm. Wouldn't a system with a compromised kernel also log exactly what the attacker wanted logged?

cube00•about 1 hour ago

I guess the hope is the kernel has been able to successfully transmit that log message to the immutable central logging infra before it gets compromised.

Although given the tendency for end point logging agents to run on buffers to reduce their network chattiness I do wonder if a fast acting exploit could dump that buffer before it manages to be transmitted.

I don't think any of the agents are complex enough to immediately transmit permission elevation log messages over the regular background noise.

rithdmc•about 1 hour ago

The attack itself creates the logs, which - reading between the lines - are shipped to a central log server. A compromised server might not send any new indicators to the logs, but existing logs moved off device would still be available.

I'd like to know what those distinctive traces are, which is also missing :(

PunchyHamster•about 1 hour ago

Your exploit would have to get root and kill/exploit the logging daemon near instantly, else the log will already be sent to remote before you can change it locally