High-Bandwidth Flash offers efficient storage for model weights

So like.. conceptually kind of like memory mapped files on fast flash persistent storage, IIUC? Or maybe it's more like GPU-managed demand paging, caching and DMA? That could get you the capacity and better I/O characteristics.

I'm curious about how the unifying architecture is going to evolve between CPU/GPU having direct access to a singular pool of memory/storage also.

I also keep wondering when memristor technology might enter the ring, because as I understand it, it would be like moving compute into the memory, which would potentially remove the need to move the data in and out of storage as much also.

It feels like computing hardware infrastructure is fundamentally evolving.

Necessity being the mother of all invention. I also thought the compute-in-memory approach was interesting re: https://mythic.ai

I've been wondering when we'd get around to having the equivalent of "memory that runs at very GDDR6 speeds for reading, but is much slower for writing", which is exactly what you need when working with an AI model. Versus current HBM which has the same speeds writing as reading.

Good use case for fast read, slow write.

stick enough floppy's in parallel and you could do the same thing

I have fond memories of Flash 5 in early 2000s. At the time I knew BASIC and Pascal. I could figure out how to draw things, how to animate them using keyframes and motion tweening, and how to make things interactive with ActionScript, all by poking around and reading the help manual (F1) on our underpowered offline PC.