Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

DE version is available. Content is displayed in original English for accuracy.

Discussion (2 Comments)Read Original on HackerNews

stephantul•about 1 hour ago

The (relevant) segue into tires elevates this post so much. I don’t understand why, but it does

vintagedave•about 1 hour ago

Maybe that it's an example of the same pattern in a real-world, non-digital area? It makes it feel more grounded.