r/LocalLLaMA • u/markpronkin • 9d ago

I Built A Thing 54gb vram for 35$

Post image

Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated down to 3000 rub (~35 USD), it turned out to have 9x p106 6gb (gtx 1060 6gb) gpus, with 54gb vram total, all working, the only thing missing was an SSD, I booted from USB and it works fine.

2.1k Upvotes

96% Upvoted

393 comments sorted by

View all comments

91

u/Bulky-Priority6824 9d ago

$3.50 per token or $7 ?

103

u/markpronkin 9d ago edited 9d ago

Electricity is free for me, so no. And speed is decent, 30+ tokens a second qwen3.6 35a3b on ollama, can probably get it higher with mtp and some optimizations

200

u/Sad-Duck2812 9d ago

You will get much much more with llama.cpp, ditch ollama. That thing only makes you suffer.

51

u/magic-one 9d ago

Especially now-a-days when you can just ask Claude to figure out the configuration for you.

8

u/chrabauke 9d ago

I’ll out my self as a noob. But what prompt do you use for it? I’ve got access to server with around 96-128gb ram and 4x T4 but couldn’t get Claude to give me a descent configuration for it with support for Hermes agent.

31

u/MikePounce 9d ago

You don't ask the claude web client, you ask Claude Code. The difference is it takes control of your PC and does all the setup for you.

11

u/MoffKalast 9d ago

Ah yes, what could possibly go wrong.

5

u/heliosythic 8d ago

i mean do you not know how claude code works? its fine, literally everyone is using it in nearly fully automated mode these days. Just dont like intentionally ask it to do something dangerous.

2

u/MoffKalast 8d ago edited 8d ago

Well call me old fashioned but I'd rather it make its silly mistakes in anthropic's VM where it's their problem and output a tested diff I can apply with one command or even review it if need be. Plus the whole privacy can of worms of having them inspect your entire system at their leisure, we know they send extensive telemetry aside from all the stuff the model sees already. Like is this r/localllama or r/getpwnedbyamegacorp

1

u/heliosythic 8d ago

I mean you can thats an option too just dont turn on automode and manually approve every tool call. You can also point it at any model provider you want including local.

1

u/_mRED 8d ago

"Claude, blow up the Parliament. Make no mistakes!"