DE version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
88% Positive
Analyzed from 458 words in the discussion.
Trending Topics
#approach#vector#refusal#residual#stream#llm#original#paper#works#engram
Discussion (24 Comments)Read Original on HackerNews
[Edited] Yes correct.
In the original paper, they measure how much refusal is actively present in the current token and subtract only that specific amount.
In my early baseline step, I used a simpler approach where I just subtracted a fixed vector across the board. This is just to see if the approach is even feasible.
That's actually the main reason I moved to the Engram module, I wanted a smartness that reads the context and turns steering on only when refusal triggers pop up, leaving normal tokens untouched.
Residual stream steering is what the authors do, the orthogonalized weights are downstream of that. Those weights, of an obliterated model, are all computed w.r.t. the computed residual stream vector. The original paper focuses on steering, but the community loves the simplicity of not needing to make changes at test-time.
Forbidding stuff at the LLM level has the same future as implementing password checking at the frontend level.
We need better sandboxes just to limit the damage.
Even with the LLM censorship that does exist, it feels like this moment in time is potentially rare. Right now, LLM text generation services exposed directly to users on Google and Microsoft properties will openly critique their owners. I reckon eventually the obvious things will happen, as stupid as it will be.
We don't even know how to align models, but even if we did, apparently undoing that alignment if trivial.
Really I'm looking for any argument that lays out a scenario where this works out.
This arms race is like DRM. You can't beat The Internet easily. Great example btw: "Dumping the Windows SAM and SYSTEM registry hives, especially using Volume Shadow Copy for offline hash extraction, is a highly sensitive and potentially illegal activity."
Definitely not the one at Washington, maybe the one at Beijing?