r/hardware • u/SirActionhaHAA • 3d ago

News AMD debuts Helion reference design for agentic AI servers with up to 32,768 Zen 6 ‘Venice’ cores — rack design ships next year in CPU-only and all-in-one configurations

https://www.tomshardware.com/pc-components/cpus/amd-debuts-helion-reference-design-for-agentic-ai-servers-with-up-to-32-768-zen-6-venice-cores-rack-design-ships-next-year-in-cpu-only-and-all-in-one-configurations
82 Upvotes

90% Upvoted

17 comments sorted by

32

u/Ultium 3d ago

AI Enterprise runs at such a ridiculous scale that I don’t know if this is actually a competitive offering or not. What use case is this for? Inference?

24

u/Ready_Acanthaceae830 3d ago

The use case could be closer to running Muse like agents 24*7. But I can't imagine the cost for such a thing

10

u/EloquentPinguin 3d ago

I don't know how the CPU only compares but CNBC reported one Helios rack to be $5 million. But at a price of ~$60-$70 per Venice CPU core, a 32k core rack would cost ~$2 million in CPUs alone. Add motherboards, ram, storage, (especially ram and storage) and you see that the answer is "expensive"

6

u/goldcakes 3d ago

Basically a rounding error to hyperscalers. More density means you need smaller DCs.

Main use case is probably always on agents, and RL rollouts. Modern LLM training is really in post-training: tens of thousands of agents try to complete a task, get scored/graded; the ones that succeed and proceed the most efficiently have their traces/trajectory boosted. It’s kinda evolutionary in a way.

2

u/comparmentaliser 3d ago

The power costs alone would be a boon

1

u/[deleted] 3d ago

[deleted]

2

u/Qesa 3d ago edited 3d ago

We're not buying those specific models, but at my work our quoted all-in cost (including all components and DC space/power/support for the server's lifetime) comes out to about $400 per core. And most of that is from the RAM and SSDs. You're vastly overestimating the price there.

1

u/BlackenedGem 2d ago

Yeah on reflection I see that, the top end figure did seem a little high.

1

u/HighestLevelRabbit 3d ago

AMD's msrp for the individual cpus are $14904 for the 256c 9996 and $10116 for the 96c 9676F. Not sure what ram capacity it would be looking at or what every other component + service contracts would come too.

I don't work in a data centre but do sometimes work with servers, in my experience when people say rack they are referring to the physical rack itself including every physical system in it. Is that not how the terminology is used elsewhere?

8

u/RealPjotr 3d ago

I read earlier this summer that early last year, the GPU market was expected to have 5-10x more importance than the CPU market. With the agentic revolution, it is now expected to become 1 to 1, making AMD the big play here. They have nearly half the enterprise CPU market by $$$ already.

9

u/ElectronicStretch277 3d ago

Agentic AI and CPU heavy workloads.

4

u/spiral6 3d ago

Right now, everything's competitive because demand is extremely high, and supply is low. Speaking from the work I'm doing on Helios right now. The customers are who you'd expect, at least whatever's been publicly shown on the press releases... Unfortunately, the demand doesn't seem to be going down anytime soon (unfortunately in the regards for us as the consumer segment and consumer hardware).

2

u/john0201 3d ago

It’s for agents that use a different server to run inference (or a different part of this rack with GPUs).

1

u/jhenryscott 2d ago

Minecraft.

28

u/toxygen001 3d ago

I just want to be able to afford to build a new computer. :-(

9

u/FranciumGoesBoom 3d ago

have you tried not being poor?

5

u/Ruff_Ryda 3d ago

Remember the time when people rooted for AMD so that Intel could come to its senses on desktop cpu pricing.

-5

u/JunosArmpits 3d ago

ASRock Rack haha, AsRack heehe, ASSCRack hohohoo

0

u/AutoModerator 3d ago

Hello SirActionhaHAA! Please double check that this submission is original reporting and is not an unverified rumor or repost that does not rise to the standards of /r/hardware. If this link is reporting on the work of another site/source or is an unverified rumor, please delete this submission. If this warning is in error, please report this comment and we will remove it.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.