AMD 7nm EPYC "Rome" CPUs in Upcoming Finnish Supercomputer, 200,000 Cores Total

[Space Lynx]

well i think they are dumb for not waiting and going all in with AMD 7nm... why wasiting money on power hungry cascade lake is beyond me.

 

[R-T-B]

Both, but it really is essential these machines run within a certain power envelope or the cost of ownership will skyrocket. There is a balance to be struck, if you project a supercomputer to run X years you might as well just add more racks instead of dealing with all the disadvantages of high clocks/perf per 1U. Extra surface area is extremely cheap anyway considering where these centers are built.

Another aspect is probably also the fact that bottlenecking occurs, storage and RAM are much more important in this space. No point oversaturating anything. And on top of all that, there are limitations to what can be fitted under the IHS, before you straight burn a hole in your server + yield issues. High clocking many cores are progressively harder to make, its the whole reason Epyc and TR are so amazing, they cut that yield risk / the number of dies.

Makes sense. I'm a desktop guy, so I'll fully admit I've never built or budgeted a super computer...

 

[m4dn355]

well i think they are dumb for not waiting and going all in with AMD 7nm... why wasiting money on power hungry cascade lake is beyond me.

Maybe they want to internally test price/perf/power equation of intel vs AMD vs nVidia

 

[renz496]

They most surely are not.

These supercomputers tend to cost as much in power bills over a few years as the cost of building them in the 1st place: it's why server chips have much lower clocks VS regular desktops because it helps tremendously with the power bills.

Yes power consumption and efficiency is one important factor when building a supercomputer but that is not the reason why the CPU or GPU in such system have much lower clock than regular consumer part. They have much lower clock to ensure stability on 24/7 full load operation. Our PC at home did not under go such stress the whole time. That's why to ensure stability in overclocking we often run stuff like Prime95 for a few hours or even the entire day.

well i think they are dumb for not waiting and going all in with AMD 7nm... why wasiting money on power hungry cascade lake is beyond me.

Professional client are not like regular consumer that can change their hardware provider anytime they like. Before they go all in with AMD they still need to evaluate AMD based system in small scale first. Just look at Hawaii S9150. When it comes to raw power and effiiency it simply kick nvidia GK210 ass. Nvidia need two GK210 just so they can beat a single Hawaii in DP. But a few years later how many supercomputer in the top ten even employ S9150?

 

[HTC]

Yes power consumption and efficiency is one important factor when building a supercomputer but that is not the reason why the CPU or GPU in such system have much lower clock than regular consumer part. They have much lower clock to ensure stability on 24/7 full load operation. Our PC at home did not under go such stress the whole time. That's why to ensure stability in overclocking we often run stuff like Prime95 for a few hours or even the entire day.

I should have said "it's one of the reasons" instead of "it's why".

Question for you: GPUs in supercomputers also run @ lower speeds than regular consumer parts? I know about it in CPUs but not GPUs.

 

[renz496]

Yes

I should have said "it's one of the reasons" instead of "it's why".

Question for you: GPUs in supercomputers also run @ lower speeds than regular consumer parts? I know about it in CPUs but not GPUs.

Yes. Not just supercomputer any professional grade hardware like quadro have much lower clock than geforce.

 

[Space Lynx]

Yes power consumption and efficiency is one important factor when building a supercomputer but that is not the reason why the CPU or GPU in such system have much lower clock than regular consumer part. They have much lower clock to ensure stability on 24/7 full load operation. Our PC at home did not under go such stress the whole time. That's why to ensure stability in overclocking we often run stuff like Prime95 for a few hours or even the entire day.


Professional client are not like regular consumer that can change their hardware provider anytime they like. Before they go all in with AMD they still need to evaluate AMD based system in small scale first. Just look at Hawaii S9150. When it comes to raw power and effiiency it simply kick nvidia GK210 ass. Nvidia need two GK210 just so they can beat a single Hawaii in DP. But a few years later how many supercomputer in the top ten even employ S9150?

Finland has 5 million people. This two phase plan is all in for that country... your point is moot.

 

[renz496]

So? It doesn't mean they just can blindly commit themselves to certain hardware for their need without proper evaluation.

 

[Space Lynx]

So? It doesn't mean they just can blindly commit themselves to certain hardware for their need without proper evaluation.

I mean the software they will be using they should know already if it runs better on AMD or Intel... they are not the only ones building a "supercomputer". Surely, they could have called around and been like hey does this modeling work well for you on EPYC or XEON better? ..... ugh nevermind, I don't feel like typing anymore

 

[RichF]

Before they go all in with AMD they still need to evaluate AMD based system in small scale first. Just look at Hawaii S9150. When it comes to raw power and effiiency it simply kick nvidia GK210 ass. Nvidia need two GK210 just so they can beat a single Hawaii in DP. But a few years later how many supercomputer in the top ten even employ S9150?

I was surprised to see multiple Piledriver systems in lists of top supercomputers in recent times. They must have gotten quite a good bulk price from AMD. Also, if the work is GPU-centric, the CPU is less important.

 

[Vya Domus]

well i think they are dumb for not waiting and going all in with AMD 7nm... why wasiting money on power hungry cascade lake is beyond me.

Time, these things are conceived at least a year prior to their deployment. That being said back then they had no idea about 7nm Epyc, this was likely a last minute thing.

 

[Vayra86]

Makes sense. I'm a desktop guy, so I'll fully admit I've never built or budgeted a super computer...

Most of what I know comes from reading articles of people who visited these sites Not building these either

 

[HTC]

Professional client are not like regular consumer that can change their hardware provider anytime they like. Before they go all in with AMD they still need to evaluate AMD based system in small scale first. Just look at Hawaii S9150. When it comes to raw power and effiiency it simply kick nvidia GK210 ass. Nvidia need two GK210 just so they can beat a single Hawaii in DP. But a few years later how many supercomputer in the top ten even employ S9150?

I'd argue there's another reason: certain loads favor a manufacturer over the other and, by using both, they have "the best of both worlds".

OTOH, i wouldn't be surprised if this stems from a previous contract with Intel forcing them to have @ least a certain amount of processors every generation. No: i don't have a source for this but, given Intel's track record in "the shady deal department", i wouldn't be that farfetched if current Finnish contract (from previous supercomputers) forced them to go this route.

 

[stimpy88]

excuse my ignorance but what the f-ck does one do with such a machine.. assuming its all being put together for some practical purpose..

trog

To play a nice game of Thermonuclear War.

 

[DeathtoGnomes]

To play a nice game of Thermonuclear War.

The only winning move is not to play.

 

[lemonadesoda]

I think performance per watt is a significant consideration, as well as cooling the entire configuration. Just think of the difference that even 5W more TDP per socket would add up to in a 3125 socket system. That’s 15000+ more watts of cooling required.

Finland is so cold, they only need to open the windows.

 

[m4dn355]

Finland is so cold, they only need to open the windows.

Impossible since they are using Linux.

 

[jabbadap]

Ironic that LLNL is installing 170 * 2 socket 24-core Epyc processors with 2 MI25 cards each totaling only 8,160 cores and 340 cards for 4.2 PFLOPS. They can install double that number of cards (four each) to bump it up to 8.4 PFLOPS. CSC approach is...weird.

Not really, precision is not the same mind you, fp32 vs fp64.

 

[FordGT90Concept]

Peak capacity is always given in binary32 (IEEE 754) performance. Yes, their motivation for going big on CPUs and small on GPUs may be because they're intending to do more logic or binary64 computations on it rather than brute force SIMD binary32 operations which GPUs can do with relatively few transistors. Then again, the OP sounds like the machine is a cobble job--jack of all trades and master of none. LLNL has a variety of machines suitable for all kinds of tasks. This might be the only one CSC has which means it has to be flexible.

 

[John Naylor]

How long before we'll be seeing posts claiming not enough cores to run today's AAA games. ?

 

[hat]

How long before we'll be seeing posts claiming not enough cores to run today's AAA games. ?

Well, we already had the now over 10 year old Crysis joke...

 

[FreedomEclipse]

All cores lead to Rome?

 

[Tartaros]

Well, we already had the now over 10 year old Crysis joke...

I really advocate for minecraft loading time jokes. Is the second most ludicrous shit I ever seen on gaming, first being you can recreate the events of the movie Your Name on Fallout 76 (less groping your own breasts if you change from man to woman. That would be sick).