Back to News
Advertisement
kkilliandunne1 about 13 hours ago 5 commentsRead Article on laugh.so

RU version is available. Content is displayed in original English for accuracy.

What if you could measure humor?

Well we've trained a model on our own dataset of ~50k human ratings to detect what jokes people find funniest. We know it's part objective, part subjective component. Subjective is out of our depth for now haha

The main results: Fable 5 is funniest - beating the average model 67% of the time, with GPT 4o last at 17%.

Other findings: - The models never refused to try, even with dark prompts - Thinking longer has a slight benefit - Absurdness correlates negatively with joke quality

Some methodology notes: - We benchmarked our model against the human majority and it agreed 72% of the time in a blind sample test. - We had 51 US adults rate the jokes, each blind to the models, with joke order randomized, and quality checked for attention and speed. - To rate some yourself visit https://pair.laugh.so

The full benchmark here:

https://laugh.so/benchmark

Am taking requests if there's more research you want to see! Cheers

Advertisement

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 116 words in the discussion.

Trending Topics

#humor#llm#interesting#models#though#measuring#imagine#correctly#assess#ability

Discussion (5 Comments)Read Original on HackerNews

maxzhdevabout 8 hours ago
I can’t imagine how to correctly assess the ability to humor, because this is a very subjective and relative phenomenon. The work ahead is serious
dlcarrierabout 12 hours ago
Can you also measure how often the LLM response makes people laugh? Sometimes the responses that aren't attempting a joke are the funniest, and I'd be more interested in stats of which LLM succeed in that metric.
killiandunne1about 12 hours ago
Interesting point - thoughts on how to do this? Honestly most models are not-to-kinda funny so I'd be surprised if there were many lol moments. Oral delivery is something I think is v interesting though
rafaeptaabout 13 hours ago
measuring humor might be halfway to measuring taste. congrats on this... really original contribution. wonder if you're planning to evolve the benchmark to incorporate a multi-language dimension. would love to see how Mistral and models built outside the US would perform.
killiandunne1about 13 hours ago
Nice a good way to 10x inference costs... worth seeing though ahaha