Show HN: A tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo)

Sure, you can go ASIC and go even faster, but the thing around GPU is that they scale well for both training and inference, and the technical floor is low.

The level to get into FPGA design is insanely high, you've got to read timing diagrams, you need to know combinatorial and sequential logics and good sense of boolean algebra, you need to have an asynchronous signal based mindset which is vastly different from CPU/GPU, you need to know netlist and you need to endure the time it takes for the EDA to finish generating it. Yosys is still years behind Xilinx

There is a reason GPUs are called accelerators; it sacrifices and does not try to really specialize on one particular thing, except high parallel dataflow and branch-free calculation. Otherwise we will all be using DSPs

Any article, even the really good ones on HN, while they get positive comments, for whatever reason, always get a lot of negative ones, too...

That is, the negative comments are absolutely unavoidable, even for people accomplishing great things!

I personally think that what you've done is brilliant, absolutely brilliant!

I can't wait to see more in this space...

Brilliant, absolutely brilliant!