Show HN: LLM Attention Visualization

fuddle•about 1 hour ago

This is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.

sva_•about 2 hours ago

I highly question this simplistic idea of high vector magnitude = high influence.

smallmancontrov•about 2 hours ago

You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.htm...

ifz•about 2 hours ago

I don't disagree with that. I did add an entire caveat paragraph there.

To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.

apnabhidu47•about 1 hour ago

Same I dont get it just, could you clarify it

wopak•about 1 hour ago

neat, combining info from two phrases is hard to see without such a tool.

are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?

ifz•about 1 hour ago

Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns.

Right now only simple correlations are visible.

itsnasme•about 1 hour ago

I like the visualisation. Pretty cool

stared•about 1 hour ago

I am curious what's the actual formula.

I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?

ifz•about 1 hour ago

It's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers.

When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.

stared•36 minutes ago

Nice! Sometimes the simplest approaches work the best.

visarga•38 minutes ago

If you want quick access look at google images for "transformer attention formula" there are some interesting depictions

Show HN: LLM Attention Visualization

⚡ Community Insights

Discussion (12 Comments)Read Original on HackerNews