OpenAI's new reasoning technique alarms AI safety experts

Related ongoing thread:

How concerned should we be about Astra's recurrent architecture? - https://news.ycombinator.com/item?id=49553321

TBH sooner or later explaining the opaque recurrence with extrinsic logic will be more sustainable, and already there is a need for that with existing models. Human language readable chains of thought are just a false sense of security, and open models already emit pretty unreadable ones at times while doing the right thing based on it (since policy optimization loops with synthetic data through reinforcement learning).

Clickbait. Recursion in layers wont effect CoT.

Is there a more technical source that tells what is being done? I assume it's about reasoning in latent space e.g. the coconut paper[1]?

I disagree with the "experts" there, as if latent-space reasoning will only cause proper interpretability research rather than taking the CoT as gospel[2].

Give me that any day over the constant "you will be abolished to the permanent underclass!" talk coming from billionaires.

Edit: (A better discussion is apparently here[3])

[1] https://arxiv.org/pdf/2412.06769

[2] https://thezvi.substack.com/p/the-most-forbidden-technique

[3] https://news.ycombinator.com/item?id=49553321

It's a matter of time until cot will happen in latent space. It just makes more sense. We as human don't do all of our thinking in words.

I'm not an expert on LLMs but it isn't clear to me why it is a problem to keep reasoning inside the layers of the model when the internal state of the model has never been readable text.

"Hobble the model's reasoning abilities so that us mere mortals can attempt to understand."

Reminds me of that short film of a dystopian future, where they place physical and mental impairment devices on humans to ensure they all have roughly equal ability:

  > "2081, a 25-minute adaptation of Kurt Vonnegut Jr.'s short story "Harrison Bergeron." In this future society, a Handicapper General enforces absolute equality by imposing artificial physical and mental handicaps—such as weights for the strong and noise-inducing earpieces for the intelligent"

I mean, its literally just more efficient loop of model generating reasoning text, for it to be fed back as context. There is nothing groundbreaking here. At the end of the day, its all just search.