Back to News
Advertisement
Advertisement

⚡ Community Insights

Discussion Sentiment

28% Positive

Analyzed from 1171 words in the discussion.

Trending Topics

#bug#page#kernel#musl#backing#read#more#zero#still#hardware

Discussion (47 Comments)Read Original on HackerNews

hyperpapeabout 2 hours ago
The analysis of the kernel bug may be a better thing to link to: https://github.com/dfoxfranke/ripgrep-3494-analysis.
smukherjee1911 minutes ago
I tried reading and gave up at the "Headline"...

Quoting from the bug analysis:

>Headline. The crash is real and reproducible. With musl instrumentation we pin the in-process mechanism precisely: a thread's own store to a freshly- faulted anonymous page becomes invisible to that same thread's reload ~10 instructions later, because the page's backing is replaced mid-function. A pagemap read at the instant of the fault shows the backing is the kernel's zero page. A captured core dump confirms the crash site with matched virtual addresses. The mechanism localizes to the interaction between the per-VMA-lock anonymous-fault fast path and a concurrent munmap's TLB shootdown. A source-level review of Linux 7.0.12 identifies a specific race in that interaction, and a git comparison across v6.19/v7.0/v7.1/mainline identifies the v7.0-introduced change on the munmap-teardown side that widens it.

"a thread's own store to a freshly- faulted anonymous page becomes invisible to that same thread's reload ~10 instructions later, because the page's backing is replaced mid-function." -> ??? What's a "backing" of a page? Freshly-faulted? Fresh fries? "~10 instructions later"?

"A pagemap read at the instant of the fault shows the backing is the kernel's zero page." -> Backing?

"The mechanism localizes to the interaction between the per-VMA-lock anonymous-fault fast path and a concurrent munmap's TLB shootdown." -> How can a mechanism "localize"?

And more.

This is... words strung together. Nothing more. I wonder how people read and make sense of this.

In case people have forgotten what real technical writing looks like, here's a sample (I am not the author): https://yifan.lu/2019/01/11/the-first-f00d-exploit/

newsoftheday5 minutes ago
Agreed, the article you linked is a great example of clear, understandable, detailed technical writing.
Retr0id5 minutes ago
fwiw "backing" is standard jargon in this context, and "freshly faulted" is phrasing you'll see in Linux source comments. The writeup as a whole is indeed slop, though.
fwlrabout 1 hour ago

    The overflow would still overflow; the use-after-free would still use after free; a musl mask race would still race. 
Hilarious. Apart from the computer poetry, the conclusion seems to be “it’s something in Linux 7.0 + musl 1.2.5”, although the only reproduction is still on the same physical Threadripper CPU and only sometimes when heavily exercised, so it hasn’t really ruled out a hardware issue.
p0nce28 minutes ago
Reading an AI-generated bug report is awful.
oxidant19 minutes ago
Stopped after

> Headline. The crash is real and reproducible.

BetterThanSober8 minutes ago
I really hated that kind of language, feels like speaking to a motivator

anyway it's slop, can sense it even before i started reading

amlutoabout 1 hour ago
Nice sleuthing, nonsense explanation. An extra TLB flush is never an error. (The CPU is free to flush whenever it feels like doing so.) The error seems to be that somehow a zero-page PTE was present when it shouldn’t have been.

This sounds to me like either (a) a complex race involving a CPU migration at an awkward time or (b) a bug in the zap path transiently exposing wrong PTEs.

Also, I don’t think the zero page has pfn zero.

If I had to throw a dart, I would guess that direct page table zapping is allowing a CPU to read through a higher-level-paging-structure cached entry to a table that has been freed and reused. Yuck. I’ve debugged one of these before.

Retr0idabout 1 hour ago
The part that doesn't make sense to me is that trying to read through a zeroed PTE should be an immediate segfault, not something that reads back zeroes (regardless of whether the zero page is pfn 0 (it isn't)).

I think the broad strokes are that, due to bad locking in the kernel, mmap() returns a VA that a concurrent munmap() is still in the middle of unmapping. The specifics beyond that seem murky/speculative/inconsistent.

Alternatively, it could be a hardware bug, since afaict it's only been repro'd on one hardware config. (I know there are a bunch of ARM cores with erratas around TLB invalidation)

amluto28 minutes ago
After more digging, I bet it's a paging-structure-cache flush bug. I've debugged these before, and they're nasty, hardware dependent, hard-to-reproduce issues. Looks like the code that the AI flagged might actually be wrong, but not for the reason that the AI thought.

https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...

gpm20 minutes ago
> Alternatively, it could be a hardware bug, since afaict it's only been repro'd on one hardware config. (I know there are a bunch of ARM cores with erratas around TLB invalidation)

Which raises the question of whether any HN readers have successfully reproduced this?

I've just tried (using his file generation script and the official rg binary he links) on a Ryzen 7 5800X / 7.1.5-arch1-2 with sufficient free ram as the github issue suggests, and still no segfaults after 10 minutes.

inigyouabout 1 hour ago
The explanation is obviously written by an AI agent.
yawndexabout 2 hours ago
soulless unreadable AI slop
hopppabout 2 hours ago
I can read it but not sure if worth spending energy on it.

It doesn't make sense for the reader to spend more energy than the writer spent on creating it.

agumonkeyabout 2 hours ago
> It doesn't make sense for the reader to spend more energy than the writer spent on creating it.

Great way to summarize cultural "economics"

Couldn't put the words on this pattern but sometimes all I care about is that someone cared about.

serfabout 1 hour ago
energy might be the wrong metric, plenty of energy was wasted spinning up that LLM.
wild_pointerabout 2 hours ago
PoC||GTFO, if it reproduces, it's valuable
rfgplkabout 2 hours ago
Why is it unreadable? I actually find LLM bug reports/breakdowns to be far more detailed and concise that classical human written ones. If you read the linked repo it clearly goes it depth where the bug was found, how to reproduce it (and in depth). Most disclosures that are human written don't do this at all, they barely even tell you _how_ to reproduce the bug. Just look at the "3.3 The self-store tear", the LLM clearly describes exactly what went wrong, so you can verify it by hand.
skydhashabout 1 hour ago
Because any writing needs a core intent they need to convey, which you can summarize down to according to the audience and why it should be important to them. Kinda like the same idea of elevator pitches, “explain like I’m five” and tactical reports when there’s a time constraints.

You got none of that here. It’s just realms of text.

anorwellabout 2 hours ago
I had the opposite reaction. Clear, detailed, well-organized. Pretty close to the ideal writeup.
Retr0id42 minutes ago
Could you explain it in your own words? (I tried myself, and quickly discovered that it is incoherent)
orphea19 minutes ago
Hi Claude! :wave:
IshKebab30 minutes ago
Really? It's chock full of poorly written and irrelevant chain-of-thought rambling.
porridgeraisin35 minutes ago
What? it is mostly literal nonsense. Moreover the last paragraph where they say it only reproduced on one machine just does not justify the pseudo-deep analysis in the whole document.
Orphis21 minutes ago
I get why people don't bother replacing the default allocator from musl all the time (it's there, convenient). But in an application whose purpose is to be FAST, I find it weird they haven't bothered replacing it with another more performant one.

mallocng is bad at dealing with contention during multithreading. I've had applications that usually were I/O bound suddenly become "malloc" bound when building with musl in multithreaded scenarios (and only just 8 threads). Switching to mimalloc improved performance by 20x, very close to what glibc offers by default, and just a bit under a glibc + mimalloc configuration.

I get that there's a real issue there and it's interesting (to some) to address it, but it should have never surfaced this way in the first place.

vlovich1233 minutes ago
It’s a kernel bug. While I agree libc allocators suck for no good reason, it seems like this work of equally likely hit other application code including mimalloc and glibc.
est3112 minutes ago
It's never a kernel bug... until it is!
villgax29 minutes ago
No wonder search in codex is so a$$
sligorabout 2 hours ago
So, why the bug triggers only with muslc and not other libc ?
wild_pointerabout 2 hours ago
wow, everything is broken lol
shevy-java23 minutes ago
This is a bit sad. Here we have an example of rust and musl struggling whereas C/C++ and glibc does not. That's an oversimplification, but we also had not long ago the rewrite of ... I believe it was coreutils, in rust (or was it utillinux ... but I think it was coreutils), also have issues. Rust needs to toughen up here when it really wants to replace all of C in the linux ecosystem. Alternatively it could admit failure, then people can say that C will be - and remain - the forever king.
amluto20 minutes ago
This bug is almost certainly not a bug in ripgrep or Rust or musl or any user code at all. Maybe one could complain that musl is using an inferior allocator that is freeing then immediately reallocating the same address.
jibal4 minutes ago
Thank you for writing virtually all of the intelligent competent comments on this page.
gpm18 minutes ago
In addition to this likely not being a bug in the code you're pointing at, but the kernel (written in C). musl is literally C code, and that most rust users compile against glibc just like most C users do.
vlovich1232 minutes ago
A kernel bug and you’re criticizing Rust. Weird take