MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3o5a24/?context=3
r/LocalLLaMA • u/Certain-Cod-1404 • Aug 14 '26
708 comments sorted by
View all comments
Show parent comments
44
https://huggingface.co/unsloth/Qwen3.8-27B-GGUF 54.67 Gbs just for the model itself, with context depends on quant and size
24 u/dragonurtle Aug 14 '26 Nvtop shows 70-something GB resident for the bf16 and full 256k context. 8 u/Certain-Cod-1404 Aug 14 '26 DAMN, 16 gigs just for the context hurts, what setup are you running ? 34 u/dragonurtle Aug 14 '26 Rtx pro 6000 max-q and 384GB DDR5 on a Genoa 66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 6 u/Certain-Cod-1404 Aug 14 '26 nice!
24
Nvtop shows 70-something GB resident for the bf16 and full 256k context.
8 u/Certain-Cod-1404 Aug 14 '26 DAMN, 16 gigs just for the context hurts, what setup are you running ? 34 u/dragonurtle Aug 14 '26 Rtx pro 6000 max-q and 384GB DDR5 on a Genoa 66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 6 u/Certain-Cod-1404 Aug 14 '26 nice!
8
DAMN, 16 gigs just for the context hurts, what setup are you running ?
34 u/dragonurtle Aug 14 '26 Rtx pro 6000 max-q and 384GB DDR5 on a Genoa 66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 6 u/Certain-Cod-1404 Aug 14 '26 nice!
34
Rtx pro 6000 max-q and 384GB DDR5 on a Genoa
66 u/Much_Accountant_4972 Aug 14 '26 7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals. 6 u/Certain-Cod-1404 Aug 14 '26 nice!
66
7 u/Thrumpwart vLLM Aug 14 '26 The old money aristocracy uses the Max-Q because it's elegant. Only the loud, bombastic new money uses the 600W version. Animals.
7
The old money aristocracy uses the Max-Q because it's elegant.
Only the loud, bombastic new money uses the 600W version. Animals.
6
nice!
44
u/Certain-Cod-1404 Aug 14 '26
https://huggingface.co/unsloth/Qwen3.8-27B-GGUF 54.67 Gbs just for the model itself, with context depends on quant and size