A Mathematical Framework for Transformer Circuits (2021)

BBluestein about 3 hours ago 4 commentsRead Article on transformer-circuits.pub

ZH version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

50% Positive

Analyzed from 63 words in the discussion.

Discussion (4 Comments)Read Original on HackerNews

oofbey•2 minutes ago

I’ve tried reading this many times. But it’s sooooo long. Is it worth it?

myself248•about 2 hours ago

Nothing about B-H curves or magnetization current, must be that other kind of transformer, in that other kind of circuit.

amelius•about 2 hours ago

> A previous project, the Distill Circuits thread, has attempted to reverse engineer vision models, but so far there hasn’t been a comparable project for transformers or language models.

How successful was that?

Bluestein•about 2 hours ago

The entire thing went down:

- https://distill.pub/2021/distill-hiatus/

... and, here we are.-