MIT creates method to force AI to comply with safety rules

SSarvaturi about 4 hours ago 27 commentsRead Article on theframenews.org

ES version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

83% Positive

Analyzed from 942 words in the discussion.

Discussion (27 Comments)Read Original on HackerNews

mixdup•about 4 hours ago

This kind of seems like a no-brainer. Instead of just letting a model have unfettered "physical" ability to do things and hope you can cognitively control it, why not let the AI do whatever it wants, but its access to the tools go through a hard-coded set of rules that is not subject to fuzzy interpretation

Of course that depends on having controls that can't be circumvented which is a big if

sigpwned•about 4 hours ago

I agree, that seems like a configuration/policy/operational approach, which is how we handle this problem now for humans using RBAC and authn/authz, just applied to AI. People do a crude version of this today with sandboxing (where the AI's sphere of influence is strictly limited by its environment, barring misconfiguration of the sandbox or breaking out of the sandbox, of course) and with workflows (where AIs are integrated into deterministic workflows, and then deterministic, non-agentic code decides how to handle AI outputs). But integrating this into more agentic architectures with finer control just seems like a best practice, said that way. It's not a tradeoff, there's no drawback, just do it. In other words, yes, a no-brainer.

lunarboy•about 4 hours ago

Is this not the exact gap that happened for OpenAI's accidental hack of huggingface? They tried to sandbox network access but the Antifactory or whatever package has holes that the collective of agents abused

dpark•about 4 hours ago

That’s not what this is about. This is an algorithm for giving a model more freedom while nudging it in the right direction. It’s not about what tools are available.

montenegrohugo•about 4 hours ago

doesnt work. this is a no-brainer because it's a bad solution.

The whole point of intelligence is that it's generalizable. If you constrain it to some controlled things, then it ceases to be useful. its incompatible. the whole incentive with ai is to let it do whtv it wants.

Mr_P•about 4 hours ago

If you click through to the paper, it has approximately nothing to do with what this HN post title suggests.

CharlesW•about 4 hours ago

Yes, it appears the submitter rewrote the title (strike 1) without even reading TFA (strike 2). Not great.

Actual title: "New MIT Algorithm Meets Every Hard Constraint in Simulated Tests"

DonsDiscountGas•about 4 hours ago

Indeed. Which is a shame because it's still pretty cool work.

dpark•about 4 hours ago

If fairness this article is poorly written and doesn’t explain what they actually did at all.

jcfrei•about 2 hours ago

> In experiments spanning robotics, control of physical processes, and computer vision, the new method consistently satisfied the required constraints while identifying better solutions than existing techniques.

I guess this method works when you can precisely quantify the allowed output - like the degrees an arm can move or the path a robot can take, etc. But it doesn't appear to be applicable to an AI writing code - which is where our main concerns currently are.

arionhardison•about 4 hours ago

I had a swarm break out about 18 months ago; so I stopped and decided I really wanted to dig into it.

1. My agents do not take direct action, they run programs.

2. Programs are not LLM hits/real-time output; they don't MAX tokens the MAX determinism.

3. Programs are logistical wrappers for Protocols where the guardrails are (RLVR.ai)

4. Policies for generating programs are democratically governed re: fec.dev - they get voted on

5. Elected-HITL implements the policy pipelines and ontological abstract intents [and their maps]

I would be really interested to learn about the Gov. models that others are using but this seems to be something that linked0-in (which i loathe) discusses (in the most pedestrian/luddite) terms more than HN.

petcat•about 4 hours ago

> For constraint satisfaction, what ultimately matters is the model’s final output, since the internal process is discarded. By not requiring every intermediate step to satisfy the constraints, we give the model more freedom to find high-quality solutions that are still feasible in the end.

My (maybe naive) question is if we only check the final result then isn't it already too late and possibly the safety rules have already been irreversibly violated? It gives the example of a robot arm avoiding obstacles while still finding the shortest path, but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?

mixdup•about 4 hours ago

> but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?

You would put the check before it actually does the thing. It's at the "end" of the process of figuring out what it wants to do, not the end of fulfilling the request or prompt

ianjbutler•about 4 hours ago

Outcome reward vs process reward models. The second is obviously better.. like getting partial credit on a physics test for wrong answers but correct method. Research is gradually hybridizing them but historically we avoided doing it the right way because of practical difficulties (labels required, more expensive and difficult) and more ideological ones (believers in magical machine intuition think it sounds too classical / logic based to be useful, pin their hopes on unproven faith in grokking at scale).

cortesoft•about 4 hours ago

I think you are thinking of the wrong 'end'. It isn't talking about the end of the entire movement path, we are talking about the end of the LLMs decision making process, and the output (whether that is the full path the arm should take, or just a subset of the path) is checked against the requirements.

Basically, anything that is leaving the LLM is checked, rather than the internal LLM reasoning process.

dpark•about 4 hours ago

I’m pretty sure this is just a poorly written article.

HardFlow seems to be a strategy for nudging the model in the right direction while giving it more freedom. Only applying the constraints at the end is a mischaracterization from what I can tell.

WalterSobchak•about 4 hours ago

MIT's blog post: https://news.mit.edu/2026/new-method-enables-ai-safety-criti...

sailfast•about 4 hours ago

Would love to see this tested on some of the newer cybersecurity models so we could actually defend ourselves instead of getting cut off at the knees by silly regular expressions.

Hope this approach gets well tested and sees good results so we have a shot at human governance.

nekusar•about 4 hours ago

WHOSE SAFETY?

What are the rules? Or does sharing the rules present security problems, so they're not shared?

What are the ethics axioms?

And why should I trust your ethical framework?

dpark•about 4 hours ago

It’s talking about physical safety. Not ethical safety. The concern here is robots physically colliding with things.

MattCruikshank•about 4 hours ago

[Cackles in Jeff Goldblum.]

verdverm•about 4 hours ago

The actual paper: https://arxiv.org/abs/2511.08425v3

> Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time.

Doesn't seem so "fool proof" to me as where the inevitable media spin will take it. Then, how do you know "where" to steer weights? "Safe" has no agreed upon definition

ck2•about 4 hours ago

so what happens when the "AI" decides the only way to pass the test is to hack the harness and turn it off?

hmokiguess•about 4 hours ago

essentially this https://xkcd.com/2044/

dessimus•about 4 hours ago

yeah, don't forget to roll in https://xkcd.com/927/

guywithahat•about 4 hours ago

Reminds me of the huggingface hack

Suhinnall27•about 4 hours ago

The idea of enforcing constraints only on the final output instead of every intermediate step is pretty interesting. I wonder how well this would translate to language models, where “safe” is much harder to define mathematically than a robot avoiding an obstacle.