We found a division by zero bug in FFmpeg with a vibecoded fuzzer

Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.

> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal.

No big deal? It’s not like it’s free… tokens cost money.

I don't care if you call it an over-engineered looping machine or what, there are concrete benefits to using LLMs for this. They work faster than developing your own looping algorithm and more often produce useful results than not.

I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.

But what's your expectation of the net?

> but can't you just mark all "/" as potential divide by zero errors?

If you’re accepting large false positives rates: yes.

If you want users to take your warnings serious: no.

(Nitpick: you certainly don’t want to flag _all_ of them. Divisions by non-zero constants definitely should be excluded, for example (integer division by -1 can lead to overflow, but that would be a different warning))

Fuzzers find inputs, not just "potential" errors that aren't triggerable.

What are you suggesting and how would it be different than how SIGFPE already works?

I mean there could be a guard clause? But yeah, seems like this could be statically evaluated like how some IDEs see a null check and don’t complain about nullability within the same scope.

For stuff like niche value optimization sure. For practical arithmetic code, nah. Like with this bug, all that changed is that garbage data in gives the user an error that they tried to process garbage data. Adding a new type doesn't make the code better, it just moves the error around. And you really don't want an infix division operator to fail to type check if the right hand side isn't a nonzero type, do you?

Every day, we stray closer to Haskell. Dare I say it: good!

What we need are refinement types, where there’s a base type and a predicate. F* has this:

     val (/) : int -> (divisor:int { divisor <> 0 }) -> int

And also cannot be INT_MIN, otherwise -1 / INT_MIN is undefined behaviour(!) in C and C++.

It would be more flexible for a compiler to reuse the range analysis logic used in optimizations for statically verifiable divide by zeros. That way you could extend it to other things like statically verifiable overflows.

Lots of projects run their own git or forgejo or similar. I run my own private forge, and it has a higher uptime than GitHub. (A shockingly low bar, tbh)

It’s surprisingly simple to setup, and the hardware requirements are pretty small for a private or small forge, as it’s usually a relatively small number of users/repos/etc.

> It is interesting that FFmpeg has its own Git server. Maybe we should move there too?

Git is a DVCS. I know many people only ever used Git through Github and forgot what the 'D' in DVCS means but whether or not they remember what the 'D' stands for, running your own Git server is trivial. Especially in this day and age of LLMs were you can just ask: "Clone this repo and convert it to base Git repo and serve it on the LAN PLZ KTHX".

The result is going to be more stable than Github and, arguably, more secure too.

...they did.

Those sneaky LLMs going 7 years into the past and committing as a human:

https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/8eda3c7f91e1a5b...

You're not reading it right. The bug was found using a vibecoded fuzzer.