A Mathematical Framework for Transformer Circuits (2021)

I’ve tried reading this many times. But it’s sooooo long. Is it worth it?

Nothing about B-H curves or magnetization current, must be that other kind of transformer, in that other kind of circuit.

> A previous project, the Distill Circuits thread, has attempted to reverse engineer vision models, but so far there hasn’t been a comparable project for transformers or language models.

How successful was that?