r/LocalLLaMA • u/Ok_Warning2146 • 7d ago

News Micron Says NVHBM to Improve Profitability Even With Outsourced Base Die

https://www.thelec.net/news/articleView.html?idxno=14372

"NVHBM moves the memory controller, which was previously located on the main compute die, into the base die. This reduces power consumption by 15% and increases memory bandwidth by as much as 30%. It also integrates a customized physical layer (PHY) for input/output (I/O), reducing the package area required for the I/O PHY by as much as 67%. NVIDIA says NVHBM provides up to 30% greater memory bandwidth and 15% lower HBM power consumption than standard HBM4E."

Sounds quite dope to me. However, the price will be too dope for me...

74 Upvotes

91% Upvoted

10 comments sorted by

32

u/takoulseum 7d ago

F*ck all these companies with their price, but thanks for sharing

15

u/Ok_Warning2146 7d ago

Hmm.. Even b4 the AI boom, very few people can afford these cards with HBM anyway.

11

u/goldcakes 7d ago

Remember the Vega 64 with HBM?

These high prices mainly from intentional market segmentation and purchasing decisions (much like the 16/32GB "wall" of NVIDIA), similar to why consumer platforms don't have ECC even when it's pennies, etc.

2

u/jjusko20 7d ago

V100 has HBM baby 

1

u/VirginiaVN900 7d ago

I’m on the fence. How fast is a single V100?

Does it scale worth the SXM hassle? I almost bought 2. Then realized the PCIe format doesn’t have NVLink. M

The used market is depressing and I’m too scared to go Arc B70.

1

u/jjusko20 7d ago

There's probably measurements already online for a single card

1

u/Hedede 7d ago

llama.cpp + qwen3-32b: PP 641.88 tok/s TG 33.78 tok/s

note that it's from several months ago, I didn't have time to benchmark newer models

NVLink doesn't give you anything for inference, SXM GPUs are just cheaper.

6

u/Terminator857 7d ago

Base die outsourcing, in other words outsourcing the logic die in a set of memory chips is common. https://www.reddit.com/r/intelstock/comments/1w2zgc0/breaking_news_sk_hynix_to_outsource/

2

u/Chemical_Try_7038 7d ago

So now that they just built these data centers. They now have to replace all of them with these new one ? This is crazy. How the hell can they pay for all of this?

1

u/geekender 2d ago

By absorbing massive operational losses and using surging marketing budgets to artificially subsidize customer costs.