Getting 50 GB/S Back from the Apple Neural Engine

This website hijacked my back button during a simple page load. You should fix that, it’s not an acceptable way to behave.

Nice investigation.

It is always surprising to me when a nice round number like 1MiB results in the “bad performance” configuration (although it happens).

Are you sure erratum is the right word in this context? I usually see it used to describe the notice that a document has an error in it.

Just checking here- this systemverilog is a hypothetical telling of what you think is going on? Or do you have the actual source of the RTL?

RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.

Ummm, wow. That's really bad.