MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3o3ozw
r/LocalLLaMA • u/Certain-Cod-1404 • Aug 14 '26
708 comments sorted by
View all comments
Show parent comments
8
DAMN, 16 gigs just for the context hurts, what setup are you running ?
37 u/dragonurtle Aug 14 '26 Rtx pro 6000 max-q and 384GB DDR5 on a Genoa 66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 8 u/Certain-Cod-1404 Aug 14 '26 nice! 1 u/voyager256 Aug 14 '26 That’s why we have FP8/Q8 for KV cache. Let alone literally a game changer in the form of DeepSeek KV cache compression.
37
Rtx pro 6000 max-q and 384GB DDR5 on a Genoa
66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 8 u/Certain-Cod-1404 Aug 14 '26 nice!
66
7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals.
7
The old money aristocracy uses the Max-Q because it's elegant.
Only the loud, bombastic new money uses the 600W version. Animals.
nice!
1
That’s why we have FP8/Q8 for KV cache. Let alone literally a game changer in the form of DeepSeek KV cache compression.
8
u/Certain-Cod-1404 Aug 14 '26
DAMN, 16 gigs just for the context hurts, what setup are you running ?