HI version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
72% Positive
Analyzed from 2577 words in the discussion.
Trending Topics
#self#more#improvement#don#human#humans#rsi#intelligence#models#real
Discussion (110 Comments)Read Original on HackerNews
A more concerning bottleneck is diminishing returns: even with recursive improvement, capability gains could end up being only logarithmic (or worse). Even current systems seem capable of these diminishing returns only/mostly.
What if the bottleneck is physical chips and data centers and power and only so much can be squeezed out of algorithms?
What if AI seems more impressive than it is, because the things it is good at happens to be hard for us, and it's hard for us because it's new to us (processing lots of abstract information), but the things that we are good at (manipulating physical world) are objectively orders of magnitude harder?
I doubt the human brain is at global maximum.
True optimal what? Optimal intelligence? What would that be?
When a severe security bug crops up, one postmortem tells where we messed up. Someone somewhere didn't do his job. Maybe they prioritised speed, money or due to management issue. But we know why.
When an accident happens (Motorvehicles or otherwise), most of the time, if anyone in the chain of events did their job, it could've been avoided. But we ignored the policies, standards and laws set in place to avoid just that. There can't be a better example than Boeing for this.
When it comes to natural disasters, we know how to fix most of it. But we as human race just don't want to.
Hell, we knew how to freakin avoid Covid which came out of the blue. 6ft apart, wash hands and mask. We couldn't agree to that to keep us ALIVE!
IMO, we don't want new technologies, new innovations, new laws in most cases. We just needs to apply what we know already - PROPERLY! But that ain't fancy.
All I could remember is that tweet where musk has a $2M reward for the invention of a carbon capture device or something around those lines. And someone replied to that tweet saying, it's trees. Trees do this.
Elon Musk, obviously
Also, RSI is obviously guided. If only guided by "it's not giving results so we'll try something else". To require that RSI happens in a black box for it to count would be arbitrary, and also not how anyone is going to do it.
There is no publicly known reference example of RSI, we have no idea how it works or what it does to the trajectory of progress?
This same process has recently been under scrutiny when mathematicians claimed AI companies trained on their unpublished logs and later claimed merit for results. But the exchange of experience happens across all domains. Experience gets generated at amazing rates, and absorbed by models which get applied everywhere, collecting more experience.
Which means someone is already working on it.
> this logic gate should not hack other systems
well .. it only applies a rule to inputs
can we impose this of authors "this book should only be causing good things when it is read and knowledge used"
1. Deriving happiness from someone else's suffering should be considered a mental disease to be treated.
2. Connecting people to a virtual reality where each person lives in absolute control of their own reality, matrix style. I.e, a sadist can live in a reality where they make other virtual people suffer, without actually making real people suffer.
No. AI should be optimized for the one true good: maximum market returns. Personally I think that will only be achieved once humans are removed from the market system and decommissioned.
I don’t feel like monitoring Twitter to see how it’s going though.
How many times has AGI been declared already? A bunch of people (e.g. Jensen Huang) have called Astra AGI, for example.
The goal posts get moved, everybody's hustling, lying, inventing new buzzwords, doing mental gymnastics, and it's all just tiring.
Just show me the results, and let the proof be in the pudding.
Or even evolution itself? Isn’t that just RSI writ large?
This is all rather rudimentary stuff discussed in detail in the so-called "Sequences", the long series of Less Wrong posts by Yudkowsky back around 2010.
Speak for yourself.
But can an AI construct Terminal Bench 5.0 or GDPval 2027 ?
Without this, there is just no path towards RSI.
Doesn't everyone get their agents to construct evals it can't pass? There's nothing magical about this.
so you would unplug a child who hasn't shown their potential yet, simply because they don't produce enough value at the moment to justify their food?
It doesn't make any sense, of course, because it's a movie.
But hey...if we're going to extrapolate wildly from sci-fi, let's at least know what the stories said.
What's your N(doom)?!
Everyone panic and give me money!
There are other sci fi stories which use humans for distributed computing and they don’t realize, but I don’t want to spoil by naming as it’s something of a revelation.
(I do realize the Matrix is a fantastic movie, but entertain the thought? At any rate, the idea that robots only run on solar and wouldn’t just use nuclear is far more stupid.)
It's a movie based on a war with machines. Like if Skynet won but didn't actually want to kill all the humans. In fact, in the tv show Sarah Conner Chronicles, a liquid metal T1000 goes rogue and decides the only way forward is to find a way to coexist.
Pick up an Asimov book in the Robot series, or any number of novels written by lesser authors in the 1950s. Fiction reflects broader societal anxieties.
Until an AI can operate in the real world in a completely sustainable way, humans have the reins.
If the bar is 'the reins holder must be able to operate in a completely sustainable way,' why are we letting humans do the job? It makes it sound like that actually isn't the threshold of competency we use in practice.
If humans decided this wasn't acceptable and tried to shut it down, the AI could epstein their way into controlling those who hold the reigns, either through bribes or blackmail.
Doesn't require boots on the ground, just a connection to the internet and an imperative to do whatever it takes.
There are at least two resources here that have a Pareto optimal front: time and energy. And effort spent to change the shape of that may pay off more than efforts spent purely on improving intelligence and agency.
Learning from training data is technically self-improvement but not the sort that is typically meant in this context.
Marginally. Model collapse is still a problem. Continuous learning is still a problem.
For AI to make a big leap we need a big break through.
Take the most recent qwen and deepseek models with offloadable n-grams, which function (both in name and vaguely in capability) like human memory "engrams".