High-bandwidth flash progress and future

https://blocksandfiles.com/2026/01/19/a-window-into-hbf-progress/

Comments

jauntywundrkindJan 25, 2026, 12:30 AM
The potential here with High-Bandwidth Flash is super cool. Effectively trying to go from 8 or a dozen flash channels to having a hundred or hundreds of channels would be amazing:

> The KAIST professor discussed an HBF unit having a capacity of 512 GB and a 1.638 TBps bandwidth.

One weird thing about this would be that it's still NAND flash and NAND flash still has limited read/write cycles, often measured in the thousands (Drive-Writes-a-Day across 5 years). If you can load a model & just keep querying it, that's not a problem. Maybe it's small enough to not be so bad, but my gut is that writing context here too might present difficulty.

digiownJan 25, 2026, 12:44 AM
I assume the use case is that you are an inference provider, and you put a bunch of models you might want to serve in the HBF to be able to quickly swap them in and out on demand.
jauntywundrkindJan 25, 2026, 1:12 AM
I think the hope is to run directly off of HBF directly, to eventually replace RAM with it entirely. 1.5TB/s is a pretty solid number! It's not going to be easy, it doesn't just drop in and replace (vastly bigger latency) but HBF replacing HBM for gobs of bandwidth is the intent, I believe.

Kioxia & Nvidia are already talking about 100M IOps SSD's directly attached to GPUs. This is less about running hte model & more about offboarding context for future use, but Nvidia is pushing KV cache to ssd. And using BlueField-4 which has PCIe on it to attach SSDs, process there. https://blocksandfiles.com/2025/09/15/kioxia-100-million-iop... https://blocksandfiles.com/2026/01/06/nvidia-standardizes-gp... https://developer.nvidia.com/blog/introducing-nvidia-bluefie...

We've already deepseek running straight off NVMe, weights runnig there. Slowly, but this maybe could scale. https://www.reddit.com/r/LocalLLaMA/comments/1idseqb/deepsee...

Kioxia for example has AiSAQ, which works in a couple places such as Milvus; not 100% clear but me exactly what's going on there, but it's trying to push work to the NVMe. And with NVMe 2.1 having computational storage, I expect we see more pushing work to the SSD.

These aren't directly the same thing as HBF. A lot is caching, but also, I tend to think there is an aspiration of trying to move some work out of ram, not merely to be able to load into ram faster.

ameliusJan 25, 2026, 2:42 PM
Flash has limited write cycles. The faster you write, the faster it wears out. How do you overcome that?
nutjob2Jan 25, 2026, 6:34 PM
The other thing having many channels gives you is the ability to have much larger drive sizes, which fixes that problem.
ameliusJan 25, 2026, 7:42 PM
Yeah but where is your advantage now?
goinghjukJan 25, 2026, 8:37 AM
they will probably use a simpler more direct protocol than NVMe
chmod775Jan 25, 2026, 9:52 AM
Can't wait to have another cache layer I have to think about.
jiggawattsJan 25, 2026, 2:24 AM
Now I understand why NVMe flash drive prices have rocketed up to triple the normal in the last few months! The AI hyperscalers aren't just sucking up the wafer runs for memory, they're also monopolising the wafers for SSDs.
soulofmischiefJan 25, 2026, 6:32 AM
Sam Altman bought 40% of the world's supply of DRAM in an underhanded, secret deal with two large manufacturers. It will take years for supply to recover.

The best part is the wafers are being bought with no plans to use them; just to keep them in storage so that competition cannot easily access RAM. Supervillain shit, should have been the last straw for PG to publicly denounce Sam and for OpenAI to be sued by the US government for anticompetitive practices. All this does is harm the consumer. Of course that ks never going to happen.

utxutxitxiJan 25, 2026, 7:06 AM
He didn't actually buy it, nor does he have the money to. He just "committed" to buying it at a later date to disrupt the supply chain for his competitors. It's scams all the way down.
nutjob2Jan 25, 2026, 6:32 PM
And he doesn't even have to follow through if he doesn't need it, he can probably sell it into the super tight market and profit that way.

I guess the memory companies loved the deal because they knew prices would skyrocket. I wonder if there are rules against cornering the market that apply here.

All in all, it stinks.

soulofmischiefJan 25, 2026, 6:54 PM
No, he made deals with each manufacturer in secret. That way neither of them realized how much the deal would affect global supply. It was underhanded all around.
soulofmischiefJan 25, 2026, 7:16 AM
Yeah, good clarification. But the deal is made nonetheless, for the time being we have to expect it to be carried out and act accordingly.
goinghjukJan 25, 2026, 8:39 AM
china also buys raw materials (metals, ...) with no intention of using them immediately

but they do because they prefer holding commodities to us dollars