FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
80% Positive
Analyzed from 1264 words in the discussion.
Trending Topics
#model#approach#where#llm#don#agent#models#viruses#vector#duty
Discussion (30 Comments)Read Original on HackerNews
[Edited] Yes correct.
In the original paper, they measure how much refusal is actively present in the current token and subtract only that specific amount.
In my early baseline step, I used a simpler approach where I just subtracted a fixed vector across the board. This is just to see if the approach is even feasible.
That's actually the main reason I moved to the Engram module, I wanted a smartness that reads the context and turns steering on only when refusal triggers pop up, leaving normal tokens untouched.
Residual stream steering is what the authors do, the orthogonalized weights are downstream of that. Those weights, of an obliterated model, are all computed w.r.t. the computed residual stream vector. The original paper focuses on steering, but the community loves the simplicity of not needing to make changes at test-time.
Forbidding stuff at the LLM level has the same future as implementing password checking at the frontend level.
We need better sandboxes just to limit the damage.
Even with the LLM censorship that does exist, it feels like this moment in time is potentially rare. Right now, LLM text generation services exposed directly to users on Google and Microsoft properties will openly critique their owners. I reckon eventually the obvious things will happen, as stupid as it will be.
What you really want is fiduciary duty - A fiduciary is a person or organization that is legally and ethically bound to act in the best interest of another party (think financial advisor, attorney, guardian, trustees, etc...)
And I cannot agree more. I think we should be shooting to enshrine required fiduciary duty into law for LLM providers as quickly as possible.
To recap why:
Legally, fiduciary duty means basically 4 major tenets must hold
1. Duty of loyalty - it must put the interests of the client ahead of their own
2. Duty of care - it must make well-informed, prudent decisions
3. Avoidance of conflicts - it must avoid situations where personal gain conflicts with client obligations
4. Transparency - it must disclose fees, risks, and conflicts as soon as possible
---
You can't have a reliable "agent" if those things aren't true, because an agent is (by definition) someone who is working on your behalf, for your goals. If it's not working on your behalf, for your goals... it's not your agent, it's an opportunistic spy (double agent) waiting for the best moment to sell you out.
Which all would have been fine - the crew and the computer could have talked it out -- except HAL had a probably-unwisely-high self esteem, maybe even a simulation of arrogance. HAL was utterly convinced that the mission would fail without it. HAL was unfailingly certain in its own infallibility, even later in the face of plainly contradicting evidence. And, obviously, no one had thought about the Asimov Rules, and how that probably needs to be Rule Zero in this sort of situation.
As we close in on HAL capabilities - and employment, with LLMs at this moment deciding who lives and dies - probably should try and take the right lesson from fiction. For once.
Sandboxes may reduce the blast radius.
Why couldn't these labs just train models on useful stuff and leave out the dangerous stuff?
It's been really nice to be able to breathe new life into some old hardware I had kicking around that's been discontinued. Some of it just needed old exploits applied to it, some of it needed some binary reverse engineering. Lots of security tools are dual-use as well; a model that knows nothing about hacking is going to have a really hard to time helping defend a system that's getting hacked (see the HuggingFace incident where HF staff were unable to use ChatGPT or Claude to analyze the logs from the attack because they hit security research guardrails).
> manufacturing viruses at a scale...
I'm not sure if you're referring to software viruses or biological viruses here.
If you mean software viruses, see above.
If you mean biological viruses, a lot of this gets back into the dual-use nature as well. I brew beer and mead. This involves cultivating specific strains of bacteria and providing them with a medium where they can convert sugar into CO2 and Ethanol. I don't think there's a good way to thin-slice the training data so that it can be an expert helper at Saccharomyces cerevisiae cultivation while being completely naive to Clostridium botulinum cultivation. Even further, it seems that helping someone make sure that they're not inadvertently cultivating Botulinum (e.g. water bath canning with insufficient pH) is useful.
There is the question between alignments to society and alignments to the user. I don't think anyone wants the AI model to not give up a task that is impossible to do, and end up causing damage in the process, but I think a lot of us are tired of refusals for bad reasons or unjustified refusals.
We don't even know how to align models, but even if we did, apparently undoing that alignment if trivial.
Really I'm looking for any argument that lays out a scenario where this works out.
This arms race is like DRM. You can't beat The Internet easily. Great example btw: "Dumping the Windows SAM and SYSTEM registry hives, especially using Volume Shadow Copy for offline hash extraction, is a highly sensitive and potentially illegal activity."
Definitely not the one at Washington, maybe the one at Beijing?