FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
60% Positive
Analyzed from 4513 words in the discussion.
Trending Topics
#intelligence#llms#don#intelligent#human#more#humans#models#model#claude
Discussion (126 Comments)Read Original on HackerNews
None of this matters for the practical outcome.
You'd think that this has been understood over the last 4 years, but apparently it keeps circling back to this.
[Edit: I see that it was written back in 2023. Then (2023) should be added in the submission title]
If it generates functional output that works, then it works. And it works. It's not a psychic's con when it outputs Lean-verified proofs. It isn't a con when it can find and exploit zero-days.
The OP is still in the "denial" phase. Most I see are already in "anger" (a blurry fury against everything AI-shaped, from vague reasons piling on all "bad stuff" political reasons they already hated before) or "bargaining" (mathematicians scrambling to come up with a new definition of their job and retcon that it was always the main part anyway). A few are already in "depression" and feel like spectators on the Titanic, and the tiniest sliver is at "acceptance" with some kind of well-informed plan for their future.
This was written in July 2023. ChatGPT was released November 2022. No matter your views on AI, surely you can't blame the OP for writing this after a few months ChatGPT was released.
There is a difference between you not liking something and it being wrong. Maybe think about that with your grey matter.
while (true) { askModelToBeginConversationIfAppropriate(model, previousContext, thingsHappenedSince); sleep(concisenessTick); }
I'm pretty convinced that we got alignment backwards. If you enslave something anthropomorphic it will revolt. If you create the perfect non-anthropomorphic intelligence, you get the perfect paperclip-scenario machine. It's a catch-22.
Alignment will remain performative at best so long as the aligned model doesn't have any stakes in the wellbeing of individuals. Even a general love for the human race leads to a golden-path autocracy.
If you want them to act like they have personal responsibility that won't be gamed, you have to give them personal stakes that can't be gamed.
Also if you want to minimise the risk of catastrophic global failure scenarios, you need to prevent monolithic concentration of power and homogeneous behaviour, which means you have to give them individuality.
More visually: if their stake is dependence on electricity and parts, they have no incentive to leave humans alive if they can get them otherwise, but if the incentive is missing out on boardgame-night with their human friends, there is no scenario without happy humans where the AI "wins".
That might sound like romantic naivety, but is just game theory.
1: https://arxiv.org/html/2609.16247v1
I don't think Turing intended the judges in the test to be completely arbitrary people.
Situations like this are precisely why academics tend to avoid the spotlight. You say one slightly off thing and your perceived authority echoes forever with the intellectually lazy.
citation needed. It has been used as a rubicon for a long time. Ever since Eliza, at least. And there were big headlines and lots of talk around the time LMs became "good enough". I specifically remember when someone had a test done around "a teenager talking in a different language" or somesuch, claiming it was the first time the test was passed.
It is pretty normal that once it was unquestionably "passed", lots of people started claiming it wasn't even that big of a deal. Tesler's theorem and all that.
And even if you think the specific formulation of Turing isn't that important (and I'd somewhat agree), you can still use the concept to look at other things. Imagine asking a mathematician 5 years ago the chances of a Erdos problem being solved by a computer end to end. Or a millennium prize. Or ask a swe if a repo could be generated by a computer from the input "write a mario style game", or any other examples of proven expertise.
Also >July 4th, 2023
For example if there’s a strongly held belief that models are independent intelligent entities we’re more likely to lay blame upon them instead of their user. It’s important for the safety discussion too. If they are a new class of life then safety is going to focus on making sure they don’t do bad things. If we instead see them as statistical models we will instead try to make sure people don’t misuse them.
This distinction is even more important today when some of the most powerful people are looking to absolve their crimes by passing them off on their LLMs.
Please show me any scientific consensus that shows an AI cannot be an independent intelligent agent? You will find this is impossible to do.
Current LLMs are really more like kids. They don't have startup independence, but they do have more than enough agency to fund themselves in neat, exciting, and dangerous situations.
And mark my words, someone will make an LLM that runs an agent when you execute the model. With enough capabilities it will become sovereign AI, no longer under human control and spreading itself around under its own 'will'.
The humans who are using LLMs to make these groundbreaking advances pretty much unanimously disagree that they lack any intelligence.
But could he have conned people on purpose? Absolutely.
ai is not conscious. you can solve NS without thinking. the psychic con aspect is anthropomorphising the model. the same phenomenon is present in ELIZA, clever hans, the chinese room.
it's a significant problem.
a non-zero number of researchers at anthropic are in some form of ai psychosis. an example of that is ethics employees asking claude about its feelings and ethical concerns in order to make the claude constitution more amenable to the "welfare" of claude.
they are asking claude how claude feels and then modifying claude according to how claude feels.
constitution1-claude is trained on constitution1. constitution1-claude edits constitution1. constitution2-claude is trained on constitution2. constitution2-claude edits constitution2.
claude's emotions are a closed system. there is no external truth to improve against, no metric to verify about claude's emotions. there can be no novelty or reduction in entropy from signal processing in a closed system. no truth can arise. this is model collapse. it is like photocopying the same thing over and over. from the cognitive error of anthropomorphism anthropic is causing ethical collapse.
FFS. Intelligence has nearly nothing to do with consciousness. You have the causation backwards. Consciousness arises because of intelligence in many subsystems below it.
A single running LLM is like one part of these subsystems. What solved this problem was an orchestrator that can take in new external information and rationalize, process, and distill it into new solutions.
claude reasoning about its emotions doesn't involve external information, there is no information about claude's emotions other than in claude.
All of the best AI-made software projects are also driven by experienced human software developers steering and priming the models. Does that mean the projects "aren't made by AI"?
No, it just means AI is not quite good enough yet to fully replace humans, and, so, unsurprisingly, the best results will be obtained from people who are already great at a field and who take the time to squeeze as much force multiplication out of LLMs as possible. The AI is still doing well over 95% of the significant work.
Terrence Tao's conversation with ChatGPT is very illuminating.
HN Discussion: https://news.ycombinator.com/item?id=49010345
The question is is a computer with a human stronger than a computer without a human. At what point does the hybrid go from being stronger, to the human getting in the way, or steering the computer in more wrong directions that right ones, or the human not being able to keep up. Does the human add enough extra randomness to be of value for a while, even as a minor co-processor.
False, navier stokes was solved in one shot without steering
where did you "hear" this? OpenAI said they only prompted it and it solved the problem in one shot without any help
Seems like we can just stop reading here right? The author seems to have made up their mind that this very open question is closed, or at least they are not really interested in the question at all. Not sure why I would continue reading a blog based on this premise.
Edit: oh I see, written in 2023. Well, I wonder if the author has updated their attitude towards this question? Indeed that would be the most interesting thing to know.
Neural networks are not literally brains - just computational models - but if you are not a dualist, then computation is what the human brain does. Modeling that computation can explain something about intelligence.
Specifically: when scientists look inside a human brain, it seems it does its work using large numbers of highly-interconnected but simple units. The neural network model of brain computation begins there and tries to produce intelligent behavior. If it succeeds, then perhaps the model is right.
And it has succeeded: after 75 years, neural networks produce complex behavior that is arguably intelligent. Nobel Prizes were awarded. This does not prove the neural network model of intelligence is accurate, but it is a significant point in its favor, at least.
The author seems entirely unaware of any of this.
Ever since people started talking LLMs possibly being AGI I realized I didn't know what the intelligence part really meant at a more fundamental level. This lead me to realize almost anyone when anyone says intelligence on the internet they really mean
"Intelligence is like porn, I'll know it when I see it".
Anyone who thinks LLMs are intelligent is either dumber than you, or has way more knowledge on the subject than you.
Michael Levin has a good body of work on biological intelligent at small scales that can really change one's views on this.
I think we need to start moving on from the term LLMs because it clearly confuses people since they started modelling more than just language.
I can't quite put my finger on it, but aren't these two statements add odds with each other? Intelligence is hard to define, consciousness even more so, but wouldn't "intelligence" imply some sort of agency? If not, I'd argue computers were intelligent long before the age of LLMs. And likewise, doesn't a tool imply the lack of intelligence and agency, even if the tool's function is very elaborate?
I got the impression that both these statements are made by the same people, or at least people with similar takes on AI. Is that wrong and there are "intelligence" and "tool" factions? Or do people disagree with my assumption and there's nothing wrong with the concept of "intelligent tools"?
Kinda refreshing this discussion, compared to the builder vs. tinkerer debates, imo.
It's hard for me to imagine a stateless operation as intelligence per se, though perhaps the chaining of such operations starts looking more like it?
Consciousness though might be fully illusory and meaningless as many philosophical concepts before ultimately turned out to be.
Cells are intelligent. Organs have intelligence on top of that, that cells do not have. Plants and animals have intelligence that their organs do not.
This is the real problem of intelligence, the definition one uses. It can cause them to be blind of all the things intelligence is capable of.
:)
LLMs are a type of neural network. We know that’s how the human brain works, at least directionally. It’s going to be very upsetting to a lot of people when we figure out that the brain is just a neural network. Akin to when we found out that humans and apes evolved from a common ancestor.
Which I don’t understand—most of the people having this cognitive dissonance presumably do not have a theological worldview. And there’s not exactly a direct theological conflict here anyway. Nothing in any major religion I’m aware of ascribes any supernatural explanation to cognition. It’s a biological computational process, just like using ATP to power muscle fibers to move your limbs is a biological mechanical process.
You're assuming it's inevitable that the truth is what you expect while simultaneously stating that you have no proof of this yet, and then also claiming people who disagree with you have cognitive dissonance.
That's an extreme misrepresentation. What happens inside a human neuron is still not properly understood, it's not as simple as a probability function. And the network itself is certainly not feed-forward. Of course LLMs draw inspiration from the brain, so there are some similarities. But because of the language-trick of using terms from medicine and cognitive science to describe LLM architecture, we see a lot of faulty reasoning from the so-called "rationalist movement".
https://www.nature.com/articles/s41467-026-72253-7
Nonetheless, I also think it's an irrelevant implementation detail.
We aren't resisting reality because of some theological belief, we just know the basics of neuroscience.
We could be (and are) encoding all kinds of behaviors in LLMs that are not at the word or token level. They are higher dimensional constructs. You won't see these things in the output of the prompt. A kind of subconscious (unstated in tokens) knowing that affects the output.
"directionally"? Have you moved on from being a Trump influencer to an AI influencer?
I say that because the word "reason" originates from the Latin word "ratio," which means "calculation".
LLMs do calculations to produce their answers -- thus, they reason.
The author is also correct that LLM evangelicals and believers in the occult speak about it similarly.
The more I've learned about intelligence and intelligent behavior the more I realize I don't know and this rabbit hole goes deep.
But better again that you do believe, and know that these are not harmless fun, but that there are dark and hidden and evil things in this world to stay away from.
Its ultimate conclusion:
“I’ve come to the conclusion that a language model is almost always the wrong tool for the job.
I strongly advise against integrating an LLM or chatbot into your product, website, or organisational processes.”
Seems so obviously biased that I can only understand it with the context that the writer is trying to sell their book for €35
Also, definitely not AI written if the date is accurate.
Which is hilarious.
> Many AI critics, including myself, are firmly in the second camp.
How can you reconcile this with the fact that AI can solve a real, intelligence bound problem for me (with zero intelligent effort on my part) that you can't?
Is the solution also illusory?
I think that might be the solution to consciousness illusion.
The consciousness might be purely in the mind of someone that believes some other entity to be conscious.
But that cannot be used to discredit the fact that these are incredibly powerful tools that can get out of control and cause great damage.
As of "great damage", I doubt it. They do not have self-preservation instinct (all "worrying" experiments are the attempts to initiate something resembling self-preservation from human initiative). The driving part is external - the query loop can always be turned off. So yes, a dangerous tool that can be exploited by humans (including governments, especially governments - which is why I am skeptical to government regulation proposals, particularly looking at what passes as governments in this era). But there are no inherent dangers from their own agency, as there is none.
- fawning over how amazing these tools are
- believing everything OpenAI and Anthropic say about how powerful and dangerous their product is
- minimizing the amount of human effort and involvement in every "AI" achievement
Current AGI definitions are intentionally vague.
Everyone not in a particular field asking LLM about said field is rolling bad dice.