I didn't see cost. If cost is similar to flash drives then this could be massive. Every GPU ships with a lot of ram still but for AI inference and games you have 10TB of this stuff to stuff all your textures into and all your model weights in. For fine tuning models this would also be great if using lora or similar.

So like.. conceptually kind of like memory mapped files on fast flash persistent storage, IIUC? Or maybe it's more like GPU-managed demand paging, caching and DMA? That could get you the capacity and better I/O characteristics.

I'm curious about how the unifying architecture is going to evolve between CPU/GPU having direct access to a singular pool of memory/storage also.

I also keep wondering when memristor technology might enter the ring, because as I understand it, it would be like moving compute into the memory, which would potentially remove the need to move the data in and out of storage as much also.

It feels like computing hardware infrastructure is fundamentally evolving.

Intel really missed out, when they discontinued the Optane line, right before the RAMpocalypse.

Necessity being the mother of all invention. I also thought the compute-in-memory approach was interesting re: https://mythic.ai

I've been wondering when we'd get around to having the equivalent of "memory that runs at very GDDR6 speeds for reading, but is much slower for writing", which is exactly what you need when working with an AI model. Versus current HBM which has the same speeds writing as reading.

stick enough floppy's in parallel and you could do the same thing

Good use case for fast read, slow write.

I have fond memories of Flash 5 in early 2000s. At the time I knew BASIC and Pascal. I could figure out how to draw things, how to animate them using keyframes and motion tweening, and how to make things interactive with ActionScript, all by poking around and reading the help manual (F1) on our underpowered offline PC.