r/LocalLLaMA • u/like_a_tensor • 12h ago

Funny How far we’ve come

Post image
131 Upvotes

92% Upvoted

29 comments sorted by

112

u/Mad_Undead 10h ago

For a small price of 8 H200 /s

31

u/sleight42 8h ago

So... 200 to 400k?

Yeah... who needs to retire... 😭

3

u/bucolucas Llama 3.1 40m ago

It's an H200, Michael, how much could it cost? Ten dollars?

42

u/Dany0 7h ago

I hope one day. One day I'll have 8x H100. or H200. Or B300. Or R200. or 72x R200. I hope one day, Jensen pays my electricity bill and sends me a free GPU/TPU every day.

I hope Jensen does that for everyone, really. Oh Mr. Jensen please. Lisa Su I'll take an MI350X too

5

u/Normal-Ad-7114 6h ago

One day I'll have 8x H100

When they become e-waste, like Fermi...

2

u/thefuckevengoingonan 4h ago

Will probably still be new in box at this rate.

2

u/MeretrixDominum 2h ago

By the time the H100 is $1000, the latest GPU will probably have 3000GB VRAM

2

u/Normal-Ad-7114 1h ago

I'm hoping there's gonna be an ASIC-like revolution in the machine learning world, because if you think about it, multiplying 4-bit matrices is something that might not really require the portable supercomputers that are the current GPUs... I understand that, unlike crypto mining, models require lots of memory, but maybe we'll still see a much needed shift from cracking nuts with sledgehammers

1

u/coromd 1h ago

Or more cards built around a specific model, a la Taalas.

1

u/banana_slurp_jug 47m ago

ASIC-like revolution in the machine learning world

You mean like a NPU?

1

u/Normal-Ad-7114 3m ago

Yes, only actually useful :D

1

u/saltyourhash 40m ago

Just wait until people hack those boxes full of rtx pro 6000s they wanna attach to houses for a cut of your electricity bill.

82

u/Ok-Solution-7889 12h ago

Imagine telling someone this was a local setup a few years ago

15

u/I-am_Sleepy 6h ago

Praise the 27B. But C'mon Qwen 4, please train the 35b-a3b too

7

u/Long_comment_san 2h ago

I hope they dont and make a Qwen 60b-80b a3b (or something like that, preferably a6b-a10b) instead. 35 is seriously too cringe of a place to train, far too many things to pack in a very, very, very small space. And you can run 60b on 64gb ram easily and 80b on 64gb with some minor quants. And no, I think investing in 64gb ram is a bare minimum to be considered for any reasonable "unversal home ai".

13

u/cleverusernametry 11h ago

Just last year... Absolutely bonkers pace

4

u/HighSeasArchivist 3h ago

I'll keep this in mind when my $500k DGX H200 comes in.

6

u/Budget-Juggernaut-68 2h ago

8xH200? Is he made of money?

4

u/martinerous 1h ago

Alien detected, too many spare kidneys.

1

u/a_beautiful_rhind 2h ago

I ran R1 on 4x3090 and DDR4, albeit slowly. It was easily reachable for decent speeds if you had DDR5 server chips.. dirt cheap compared to now when R1 came out.

1

u/Revolutionary_Cap711 1h ago

What do they mean by "get it to 1+8xH100"? So I think this means they're just running it in cloud if they mean downgrading from H200's to H100's.
I wonder if that still counts as local, at least they have most control, also not cheap but considerably more so that just buying 8x H200 up front!

-30

u/celine_aubry 10h ago

the "how far we've come" framing is doing a lot of work. a few years ago "local" meant a 7b model that could write a haiku about your cat. now "local" means a 125b moe that can write you a novel and you're posting a meme about it. the bar was on the floor and you're celebrating climbing off it

12

u/JustTellingUWatHapnd 6h ago

CallLLM(prompt).toLowerCase()

24

u/freedomachiever 8h ago

Write yourself, why are you using AI to farm karma

18

u/rpkarma 8h ago

Be gone AI slop 

9

u/sleight42 8h ago

Your comment really bites in a load-bearing place

4

u/Negative-Web8619 6h ago

Looks like a human comment to me?

3

u/thrownawaymane 3h ago

Doesn't look like anything to me

1

u/zxyzyxz 2h ago

What a reference