A third of Perplexity's citations don't contain the number they're cited for

jjakobgreenfeld about 4 hours ago 31 commentsRead Article on hausresearch.com

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

47% Positive

Analyzed from 937 words in the discussion.

Discussion (31 Comments)Read Original on HackerNews

wrs•about 2 hours ago

Same is true of Google search “summaries” where quite often I click on the link provided and it doesn’t support the statement Google made.

BTW, this post would be more convincing if it wasn’t written in Claude voice itself!

ok123456•38 minutes ago

Google putting AI search summaries is the worst of all worlds.

You need cheap, fast inference to put on a search results page and finish near the deadline (~3 seconds). Cheap, fast inference is more likely to get stuff wrong. No one is happy.

eitally•38 minutes ago

This has become a common pattern in my Claude & Gemini usage. Always require citations, then check those citations to validate they actually contain the information/data the LLM's output claims. Claude, in particular, seems to make massive logic jumps and trust tertiary data sources way more than it should.

chicken-stew•about 2 hours ago

There’s a fun variation in W-Europe that google needs to spend some time on:

Northern Belgium and the Netherlands have web content in the same language. But google uses the content in one lump. Problem is when you search for employment/fiscal/legal/… you constantly get content that applies to the wrong nationality.

ozlikethewizard•10 minutes ago

How do you think us brits feel?

taeric•about 1 hour ago

This is true of a ton of online discourse. Worse, when the headline of a claim doesn't even match the article it is fronting. I've seen more than a few articles that basically contradict the headline, but end in a "despite all evidence, we think it is correct to say X."

darth_avocado•about 1 hour ago

I don’t know if Claude performs similarly from a percentage standpoint, but if you’re using it for search (online or personal docs or wikis), it often also just makes things up.

When you point it out, it’ll do the “ohh you’re absolutely right!” bs. Marketing material and management that believes the material wants to pretend that AI agents are junior employees, but forget that junior employees get fired for doing something like this.

croes•about 1 hour ago

It was true even before AI summaries.

The search result showed a paragraph containing the search word, the website did not

Gecko4072•about 3 hours ago

I noticed this personally. Saw a citation with a preview for source A, which I knew was reliable. Checked, and it referenced a Reddit article and various other less reliable sources. Was a direct citation too that actually wasn't.

abdullahkhalids•about 2 hours ago

If you are building your own harness that does correct citations, is the correct thing to give AI access to some deterministic tool that allows it to actually copy paste parts of documents its reading (with links), rather than stochastic reproduction that they do by default?

rahimnathwani•44 minutes ago

I did something similar for structured text extraction. I added markers throughout each source document and then, for each piece of info I wanted, I asked the LLM to provide two separate fields:

  xyz
  xyz_citation

The latter was just the node number. So then my code could extract the exact snippet, instead of trusting the LLM to quote something verbatim.

clickety_clack•about 1 hour ago

That’s what I did when I built my stuff. I have deterministic content with AI commentary, where it seems most people are doing this crazy thing of sending data through the model. I can’t understand it.

realsarm•about 1 hour ago

You can just look at nouswise or nblm. They do have such harness behind.

betree•about 2 hours ago

100% my experience with this service. The intent is good, but it seems they're still in the "fake it until you make it" stage.

automatic6131•about 2 hours ago

>The intent is good, but it seems they're still in the "fake it until you make it" stage

Please notice the internal contradictions here.

hek2sch•about 1 hour ago

To be honest perplexity does nothing to make sure it's answer are correct let alone the citations. They just look plausible. For anything little bit serious I use nouswise or nblm that sometimes abstain instead of making things up.

realsarm•about 1 hour ago

Yes this paper even tries their enterprise case. Perplexity performs worse. https://arxiv.org/html/2604.17843v1

vikramkr•about 1 hour ago

> the intent is good

What is that supposed to mean? They're trying to be an llm search engine that's not some radical new concept

gamblor956•about 2 hours ago

This is why lawyers have been getting in trouble using AI to review case law or (worse) to generate documents.

It creates citations and references that look close enough to be plausible but are just made up of thin air. CA passed a law explicitly requiring lawyers to review AI-generated documents that is now before the governor for signing (previously, lawyers were ethically expected to review documents submitted to the court or provided to clients but that doesn't have the same level of force as an explicit requirement).

ModernMech•about 2 hours ago

Even worse than lawyers is the government doing it: https://newrepublic.com/post/215001/judge-rfk-jr-hhs-fake-ai...

motbus3•about 2 hours ago

I felt that myself. And I have the same problem with gemini

wopwops•about 1 hour ago

My favorite is hallucinated slop with sources that 404.

mandolingual•44 minutes ago

"The unit above is the citation, not the claim." The RPM of the slop ouroboros ever rises.

asmodeuslucifer•about 2 hours ago

That's the one that powers Truth Social

Trump Media and Technology Group announced that it partnered with Perplexity to test and integrate an AI search feature, referred to as Truth Social AI or Truth Search AI, directly into the Truth Social platform.

(I just use the free account from truth+ to waste their money)

how do you reverse a linked list in python

Answers Sources Use either an iterative pointer-reversal approach or a recursive approach. The standard iterative version is the most common and runs in (O(n)) time with (O(1)) extra space:

class ListNode: def __init__(self, val=0, next=None): self.val = val self.next = next

def reverse_list(head): prev = None curr = head

    while curr:
        nxt = curr.next
        curr.next = prev
        prev = curr
        curr = nxt

    return prev

If you already have a Python list, reversing it is simpler with slicing: items[::-1], but that is not a linked list reversal.

cmiles8•about 2 hours ago

This plus AI just citing AI slop. Theres a real downward spiral unfolding with the quality of information available on the internet.

gsumk•about 1 hour ago

That’s what I feel too. From the references in their PDF, a few links are returning 404s, while others don’t even seem relevant to the so-called “research.” It is pure AI slop. https://hausresearch.com/data/perplexity-citation-audit/perp...

tempfile•about 3 hours ago

BS machine produces BS; in other news water is wet and sky is blue.

Spivak•about 2 hours ago

What's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.

delichon•about 2 hours ago

On the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.

tempfile•about 2 hours ago

> you can get the bullshit machine to produce correct externally validate citations

How?

pessimizer•1 minute ago

I assume by forcing them to validate their citations with a deterministic tool.

MarkusQ•23 minutes ago

By training it on a whole bunch of examples with valid external citations, to the point where it's able to hallucinate something that's valid. Of course, it won't always work, which is the point of TFA.