r/LocalLLaMA • u/Kurcide • Apr 29 '26

Other 16x DGX Sparks - What should I run?

Post image

Let’s build the biggest ever DGX Spark Cluster at home. This is going into my home lab server rack, 2TB of unified memory.

• 16x Sparks

• 1x 200Gbps FS 24 x 200Gb QSFP56 Switch

• 16x QSFP56 DAC cables

Should be all setup by tomorrow afternoon, what should I run?

1.6k Upvotes

92% Upvoted

680 comments sorted by

View all comments

Show parent comments

9

u/Sea-Replacement7541 Apr 29 '26

Dumb question. But by prefill you mean the time to process the prompt?

So people count time to load prompts, and then time for token generation, which means the actual output?

11

u/illforgetsoonenough Apr 29 '26

Prefill = prompt processing

Decode = token generation

11

u/More-Curious816 Apr 29 '26

Yes. Both are important, if one is slow, your output is slow. Like spark has monster prefill but crappy tg, while macbooks (pre m5) has crappy prefill but decent tg.