AI Built a Nuke and Still Lost

I think it might be a training artefact of some sort - the current crops of LLMs have never been in a position where they can explore the world as an independent existence and so they might be struggling to model how to explain an interesting experience? The Go AIs had problems with ladders of all things (one of the most basic beginner shapes) back in the early superhuman phases after Alphago. There seems to be some similar and profound gap in the LLM understanding of how to communicate when storytelling.

The "But France was running two clocks at once" paragraph really set me off because I get the feeling something really interesting might be happening that the AI doesn't want to talk about and there is evidence that it is trying to say something. But the result is some amount of gibberish and some amount of vague allusion to something interesting in the prompt context while glossing over all the information that might matter while working hard to create an evocative feeling that isn't interesting. A tense atmosphere with no exploration of why there is tension.

This and the fact that you often read a sentence, paragraph or the whole article, and think this said absolutely nothing in lots of words.

For a limited amount of time I appreciated the level of detail in the article, hoping it would give me more insight, until it exhausted me. I think those two ideas are real takeaways: "Knowing is not doing" and "What can we trust AI to do?". Still, could have been said with a more concise text and maybe a follow up about the details.

This problem actually surfaces in movies too, for A to happen B has to happen, but B has no reason to happen so you end up with non sensensical situations. This happens in llms as well since A is explained by B happening, but A doesn't need to be explained since A can't happen.

> There is no insight to be gleamed.

AI-generated articles are the intellectual equivalent of empty calories.

I have just spent the last 10 minutes trying to figure out why someone decided to buy imgui.org, name-squatting an actual project, just to put a slop website on it mildly referencing the original project. It's not even trying to scam you.

I keep wondering whether these people that keep polluting the internet with their insightless slop even possess self-awareness. What motivates them to expend money and effort to contribute nothing to the world? Are they another example of a philosophical zombie?

It's weird because when you look at models that expose CoT, this does not happen. They switch up every second.

"But then X happened... Wait, didn't Y happen? Then why would X be there? I think the user's initial statement was correct, but then Y happened..."

> Pre-LLM, authors generally had intention behind their words.

I think this is at least in part a combination of rosy retrospection and attentional bias: A lot of human writing was always trash. Absolute dogshit with regard to the quality of writing, but there was no "AI slop" label to attach to it. How would you, pre-LLMs, have placed a comment on the writing style if a post was badly written? From what I've seen it would be a "this is marketing/SEO-speak" or some similar comment, deriding the author for being uninformed or of ill intent.

We've now become so allergic to AI slop that anything that even smells like it triggers almost immediate disgust and attachment of that label to the content (even if it is the same old human written trash).

I guess LLM-assisted posts do change the dynamic a bit: the intent is more often benign with a desire to write something good, but the skill to do so lacking. If we limit the "pre-LLM authors" to people with good intent writing about stuff relevant to a HackerNews audience, you're probably right. Many more bad writers are now creating the same ostensibly fancy articles, decreasing the signal-to-noise ratio we were used.

It's not this, it's that. And then what happened? This. I did that... This happened.

    It's a thing
    I don't know why
    But it's a thing

To be honest, it's not a thing.

Let that sink in.

Maybe we find most meaning in the least average language constructs.

I came to the same conclusion about AI generated code. When I read code written by a human, just by skimming it, I can get a sense of what purpose the code has, why it was written this way and not another way, what style and mindset the programmer behind it has. AI generated code may sometimes be extremely precise and following all the good practices, but I feel no intent behind it.

It's surprising to me that the big labs haven't fixed this problem. Half the comments here are complaining that this article is egregious unreadable slop, and I agree. Surely with the trillions of investment they could at least figure out how to vary sentence structure a bit and nix obvious tells.

Maybe this is just an inherent problem with LLMs.

For me this reads like a report of things that were tried and observed. It was a very pleasant read for me because I'm interested in the subject. And the lack of underlying agenda, moral lesson, politics or, as you call it, insight, was quite refreshing. I became quite allergic to texts where author clearly tries to make me think a specific thing. To sell me something. I usually find the agenda pretty quickly and I know the rest of text is just a fluff around it so I lose interest. And when the agenda is not easy to find then I just get more annoyed because I feel it's intentionally hidden. Like a solution to a clickbait title.

This text reads great for me because as I read it, I clearly saw there's no agenda so I felt safe to just absorb the information that it contains.

> There is no insight to be gleamed.

This is no surprise. AI slop is called slop for a reason. It is basically just spam-slop. The whole term "Artificial Intelligence" has always been a misnomer from the get go, stealing from biological systems without understanding them, yet alone being able to re-create them via non-biological means. Even synthetic biology, as cool as it is, has huge limitations e. g. leaky promoters (or CRISPR-Cas off-target cleavage, which is a major reason why gene therapy isn't yet there, despite the occasional promo article of how xyz has been totally cured forever).

What I don't understand is that people can find it useful. I understand some of the rationale, but I find AI slop just aims to try to steal my time. I can not tolerate this.

That's an interesting observation. For me the main takeaway is still the style.

(bigheading)The takeaway(/bigheading) The style? Terrible.

Sorry, but this sounds exactly like a greentext you can read on 4claw. Are you a real human?

I very much dislike the idea of teaching the robot to play Civilization and expect those skills to transfer to their advisory nature.

If anything, I'd almost prefer a leader who hasn't played Civilization in their life. Goes without saying that a mature leader could tell these apart, but in this day and age, I'm not so sure whether everyone could.

> "technocratic wonk" vs "blood and soil"

This is not a binary; it's the same people on the same side.

"Tony Blair Institute" fits right into the "x word horror" Xitter genre. Funded by Larry Ellison to boot!

Tony Blair is the guy who found success by making the UK's left-leaning party (much) more neoliberal and was promptly imitated by Gerhard Schröder in Germany doing basically the same thing. Schröder is also BFF with Putin.

I also noticed they where not using XML for game state output, from what I understand most LLMs still benefit from having outputs like this put into XML tags

Im sure there are some billionaires to find, that finally care about the survival of the white race. /s

What word would you use instead?

There's something so uncanny about the mismatch between the regard in which Blair is generally held by British people and the regard in which he seems to hold himself.

If I were him I'd have retired from public life and kept a very low profile after Iraq, and everything else for that matter. He doesn't seem to realise that his modern interventions alienate everyone, even Alastair Campbell of all people seemed uncomfortable to the degree he seems to uncritically sing the praises of people like Larry Ellison recently.

Chumbawamba made me unable to take anything associated with him seriously.

Exactly this, he should've just fixed this, or not written an article about it.

After the 'sensorium effect' (he should've used ancient greek for a +10 bonus to archaic intellectual points), he describes the 'knowledge-doing gap'. i.e. the AI reasons it needs to build X, logs this for 110 turns in a row, but doesn't do it. It doesn't actually specify why not, and whether it is again a limitation of his MCP implementation. If the AI articulates it must do it like the author says, but decides not to, either it doesn't think it must do it, or it does think it must but somehow can't technically execute its own decisions, it can't be anything else.

In fact in the context of 'advising the UK government', this 'knowledge-doing gap' I assume is a technical limitation, is entirely moot. For the cost of 0.00001% of the UK's government you could just hire a human being to execute that which the AI articulates. I'm curious what the results would be if he just did a manual execution of the AI's articulated actions would be.

The fact he doesn't go in to this but just keeps repeating examples of this makes it a pointless article.

> he gives it a name "The sensorium effect" and treats this as some brilliant insight

And of course is unaware of prior work in this area!

https://en.wikipedia.org/wiki/Seeing_Like_a_State / https://en.wikipedia.org/wiki/Project_Cybersyn

> he gives it a name

It gives it a name. It would be quite surprising if he bothered to come up with this name himself when the whole article is obviously AI written.

> The reason pvp doesn't work in a persistent world like an MMORPG is because there are no stakes.

See Eve Online

Undermines. Underscores.

Matters of perspective.

> how much this undermines the message

It didn’t undermine it for me.