High-Bandwidth Flash offers efficient storage for model weights

So like.. conceptually kind of like memory mapped files on fast flash persistent storage, IIUC? Or maybe it's more like GPU-managed demand paging, caching and DMA? That could get you the capacity and better I/O characteristics.

I'm curious about how the unifying architecture is going to evolve between CPU/GPU having direct access to a singular pool of memory/storage also.

I also keep wondering when memristor technology might enter the ring, because as I understand it, it would be like moving compute into the memory, which would potentially remove the need to move the data in and out of storage as much also.

It feels like computing hardware infrastructure is fundamentally evolving.

I've been wondering when we'd get around to having the equivalent of "memory that runs at very GDDR6 speeds for reading, but is much slower for writing", which is exactly what you need when working with an AI model. Versus current HBM which has the same speeds writing as reading.

Necessity being the mother of all invention. I also thought the compute-in-memory approach was interesting re: https://mythic.ai

stick enough floppy's in parallel and you could do the same thing