Detecting LLM-Generated Texts with “Classical” Machine Learning

uuneven9434 about 3 hours ago 41 commentsRead Article on blog.lyc8503.net

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

53% Positive

Analyzed from 1695 words in the discussion.

Discussion (41 Comments)Read Original on HackerNews

docheinestages•about 1 hour ago

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.

calebh•34 minutes ago

The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/

For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.

visarga•29 minutes ago

That human might have used AI. You can never know. Hand fixed AI output, human just polished the corners? Light rewording of a full text written by hand, because the author is not confident in their writing? Actual human text, but after researching with AI?

thatjoeoverthr•6 minutes ago

Sufficiently advanced AI use is probably fine. The slop everyone complains about has certain tells specifically due to some combination of the following:

- The author is conducting some kind of hustle.

- The author doesn't bother editing.

- The author lacks the taste and awareness enough to see it looks.

- The author thinks you, the reader, lack taste and awareness.

- The author is using it as a kind of smoke bomb to get rid of you.

In such cases, nothing is done about the LLM's distinctive "voice". It dominates the text and it's easy to detect. It stands as a signifier of the above, even if it's otherwise not intrinsically a problem to use AI.

akersten•about 2 hours ago

Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.

Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.

driverdan•about 1 hour ago

There are two problems, false positives and changing the LLM's pattern.

It's really easy to have a false positive and false positives can be very harmful if the person using the detector isn't aware of that risk.

It's also very easy to change the pattern of LLM output. You can provide basic prompting that will significantly change the structure of the output. For example, having it utilize the Wikipedia article on signs of AI writing and avoid everything it describes. https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing

Retric•about 1 hour ago

Signal is easier to detect with more data to work with.

Largely AI generated books are a vastly different situation than a one paragraph homework assignment. But multiple rounds of homework assignments would change the accuracy.

stymaar•about 2 hours ago

> but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.

Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do).

But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. And I don't think that's going to change anytime soon, unless their incentives change.

(We can say exactly the same thing about man-made stuff optimized for a specific purpose, like stock photography, clickbait titles or industrial food: they aren't stereotypical because their creator lacks the skill to make them otherwise, they are like that because that's what works best).

BoredomIsFun•about 1 hour ago

> especially base ones

Did you actually try them? I did.They generated even more "slopey" text than instruction-tuned ones.

AnthonyMouse•35 minutes ago

> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either.

There are two problems with this.

The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".

And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.

stymaar•13 minutes ago

> The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".

The thing is, humans are significantly worse at maximizing numerical goals than computers.

> And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.

Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.

OneManyNone•30 minutes ago

All of that may be true, but pangram currently has a false positive rate of about 1 in 10000, and this has been tested by feeding in thousands of texts written before 2020.

That may not last if AI companies start trying to build models that fool it, but for the time being at least, modern models do have strong tells.

ravenstine•about 1 hour ago

They're also designed to not offend anybody, so their output tends to be very bland even compared to the most milquetoast of human beings. I was only surprised once when ChatGPT responded with an enthusiastic "hell yes" seemingly organically, but 99.9% of the time these AI services clearly are instructed and trained to provide flavorless word vomit. I don't think there's a technical reason why an LLM couldn't produce totally convincing output, but internet grifters don't need to go through that trouble. It's like how most phone, email, and social media scams come off as completely transparent to most of us, but that's the whole point; we're not the target audience of the scams. Readers looking for substance, nuance, and real opinions aren't going to notice if something with written by an LLM – unless there are some cliche punctuation tells.

empath75•about 2 hours ago

It does mean that this will have a drift problem if it's just trained on the idiosyncrasies of model fine tuning. That's fine! But it is something to be aware of.

zmjone2992•about 1 hour ago

i think one thing overlooked by this perspective is that many of a detectors adversaries are not that sophisticated. so despite this i think it is a useful thing to try to do. particularly when people are trying to do fraud which will often having to use abliterated models and generally trying to be as economical in their efforts

cyanydeez•about 1 hour ago

Sure it is; we do it all the time, and then we modify each other's etc, etc; english we speak today was spoke yesterday waspake the same in yesteryears; we have no trouble dating english or other languages to a time.

A better argument is people themselves are just too influenced by reading that they'll sound like LLMs in a couple of years.

jgalt212•about 2 hours ago

It depends on how much text. For example, chardet often falls down on short strings, but 1K characters it nails it.

Krssst•about 2 hours ago

The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ?

If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.

(the article was a good read, thanks!)

xiaoyu2006•about 2 hours ago

I assume different models will have different distribution, so it has to be kept updated?

Krssst•about 2 hours ago

The article mentions that AI texts are often caught by multiple models, so hopefully text from newer LLMs could still be caught without updating the model?

40four•about 1 hour ago

I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solve this problem.

arjie•25 minutes ago

Neat. I will implement something like this for myself. I just need to reduce the spam a little. Imperfection is okay for a social network context like HN.

gleenn•about 2 hours ago

I think the fundamental problem is that training current SOTA AI models is very expensive. If a simple "classical" model can detect them, presumably at much lower algorithmic cost, then why wouldn't the model trainers use these same tools to feed back into their models to improve them at low cost to make them better? It's an arms race. Any cheap pattern can and presumably will be used to retrain if it becomes and effective way to catch AI.

arjie•41 minutes ago

It’s simply not a priority. The labs can do many things. Making text non-LLM is not really that useful. Analogous to Facebook not picking up the obvious $20 bill in front of them. It’s because they’ve got $100 bills at their feet they’re picking up.

Retric•about 1 hour ago

It’s an arms race where the AI companies are at an extreme disadvantage due to relative training costs.

teeray•about 2 hours ago

The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.

rayval•25 minutes ago

As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.

If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.

EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823

dmurvihill•42 minutes ago

It depends on the application. Dissertation? Hell naw. Blog post? Absolutely, run it through that thing.

unfocso•about 2 hours ago

I had done the same for classifying and generating bookmarks of thousands of datasheets, along with a very naive yolo-based classificator (to detect pages made out of diagrams and pictures mostly).

Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema

aberoham•about 2 hours ago

I wonder about this technique vs simple SVM classifiers: https://x.com/rosmine/status/2056406399471558872?s=20

janalsncm•about 1 hour ago

This article is about training a classifier to detect synthetic text.

The link you sent is for generating text which attempts to defeat those classifiers.

richard_chase•40 minutes ago

Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.

therealdrag0•30 minutes ago

I wouldn’t go that far… but it can be kinda like Wikipedia, clean and readable.

XiphiasX•about 2 hours ago

Anything too “clever” and “snappy” = instaLLM

hasteg•about 1 hour ago

This is also how I pretty much filter LLM generated text in my head.

metalman•about 1 hour ago

there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow for the rest I do my own research and verification thank you very much

cyanydeez•about 2 hours ago

today, sure.

Tomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.

Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.