DskDitto: Ultra-fast, parallel duplicate-file detector

Wow. Wasn’t expecting to see this here… I am the author…

imo fclones is the one to beat, https://github.com/pkolaczk/fclones#benchmarks

nice that there are actually some ok interfaces here. with fclones, i tend to generate a file of candidates then do a little review to make super certain everything is square / as expected. it's very built for intermediary files a core pattern, which is very unix, very convenient. but it did take me a little while to settle on this, and it felt like i wasn't being offered a ton of options for management out of box. looks like some real attempts to be more user friendly here.

Interested to know what algorithm you use for --fuzzy. Presumably not any of the worst-case-quadratic diff algorithms... MinHash on n-grams?