• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

AMD Collaborates with US DOE to Deliver the Frontier Supercomputer

AleksandarK

News Editor
Staff member
Joined
Aug 19, 2017
Messages
2,662 (0.99/day)
The U.S. Department of Energy today announced a contract with Cray Inc. to build the Frontier supercomputer at Oak Ridge National Laboratory, which is anticipated to debut in 2021 as the world's most powerful computer with a performance of greater than 1.5 exaflops.

Scheduled for delivery in 2021, Frontier will accelerate innovation in science and technology and maintain U.S. leadership in high-performance computing and artificial intelligence. The total contract award is valued at more than $600 million for the system and technology development. The system will be based on Cray's new Shasta architecture and Slingshot interconnect and will feature high-performance AMD EPYC CPU and AMD Radeon Instinct GPU technology.



By solving calculations up to 50 times faster than today's top supercomputers-exceeding a quintillion, or 10^18, calculations per second-Frontier will enable researchers to deliver breakthroughs in scientific discovery, energy assurance, economic competitiveness, and national security. As a second-generation AI system-following the world-leading Summit system deployed at ORNL in 2018-Frontier will provide new capabilities for deep learning, machine learning and data analytics for applications ranging from manufacturing to human health.

"Frontier's record-breaking performance will ensure our country's ability to lead the world in science that improves the lives and economic prosperity of all Americans and the entire world," said U.S. Secretary of Energy Rick Perry. "Frontier will accelerate innovation in AI by giving American researchers world-class data and computing resources to ensure the next great inventions are made in the United States."

Since 2005, Oak Ridge National Laboratory has deployed Jaguar, Titan, and Summit, each the world's fastest computer in its time. The combination of traditional processors with graphics processing units to accelerate the performance of leadership-class scientific supercomputers is an approach pioneered by ORNL and its partners and successfully demonstrated through ORNL's No.1 ranked Titan and Summit supercomputers.

"ORNL's vision is to sustain the nation's preeminence in science and technology by developing and deploying leadership computing for research and innovation at an unprecedented scale," said ORNL Director Thomas Zacharia. "Frontier follows the well-established computing path charted by ORNL and its partners that will provide the research community with an exascale system ready for science on day one."

Researchers with DOE's Exascale Computing Project are developing exascale scientific applications today on ORNL's 200-petaflop Summit system and will seamlessly transition their scientific applications to Frontier in 2021. In addition, the lab's Center for Accelerated Application Readiness is now accepting proposals from scientists to prepare their codes to run on Frontier.



Researchers will harness Frontier's powerful architecture to advance science in such applications as systems biology, materials science, energy production, additive manufacturing and health data science. Visit the Frontier website to learn more about what researchers plan to accomplish in these and other scientific fields.

Frontier will offer best-in-class traditional scientific modeling and simulation capabilities while also leading the world in artificial intelligence and data analytics. Closely integrating artificial intelligence with data analytics and modeling and simulation will drastically reduce the time to discovery by automatically recognizing patterns in data and guiding simulations beyond the limits of traditional approaches.

"We are honored to be part of this historic moment as we embark on supporting extreme-scale scientific endeavors to deliver the next U.S. exascale supercomputer to the Department of Energy and ORNL," said Peter Ungaro, president and CEO of Cray. "Frontier will incorporate foundational new technologies from Cray and AMD that will enable the new exascale era-characterized by data-intensive workloads and the convergence of modeling, simulation, analytics, and AI for scientific discovery, engineering and digital transformation."

Frontier will incorporate several novel technologies co-designed specifically to deliver a balanced scientific capability for the user community. The system will be composed of more than 100 Cray Shasta cabinets with high density compute blades powered by HPC and AI- optimized AMD EPYC processors and Radeon Instinct GPU accelerators purpose-built for the needs of exascale computing. The new accelerator-centric compute blades will support a 4:1 GPU to CPU ratio with high speed AMD Infinity Fabric links and coherent memory between them within the node. Each node will have one Cray Slingshot interconnect network port for every GPU with streamlined communication between the GPUs and network to enable optimal performance for high-performance computing and AI workloads at exascale.

To make this performance seamless to consume by developers, Cray and AMD are co-designing and developing enhanced GPU programming tools optimized for performance, productivity and portability. This will include new capabilities in the Cray Programming Environment and AMD's ROCm open compute platform that will be integrated together into the Cray Shasta software stack for Frontier.

"AMD is proud to be working with Cray, Oak Ridge National Laboratory and the Department of Energy to push the boundaries of high performance computing with Frontier," said Lisa Su, AMD president and CEO. "Today's announcement represents the power of collaboration between private industry and public research institutions to deliver groundbreaking innovations that scientists can use to solve some of the world's biggest problems."

Frontier leverages a decade of exascale technology investments by DOE. The contract award includes technology development funding, a center of excellence, several early-delivery systems, the main Frontier system, and multi-year systems support. The Frontier system is expected to be delivered in 2021, and acceptance is anticipated in 2022.

Frontier will be part of the Oak Ridge Leadership Computing Facility, a DOE Office of Science User Facility. ORNL is managed by UT-Battelle for DOE's Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE's Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit DOE's webiste.

View at TechPowerUp Main Site
 
Joined
Apr 21, 2010
Messages
5,731 (1.07/day)
Location
West Midlands. UK.
System Name Ryzen Reynolds
Processor Ryzen 1600 - 4.0Ghz 1.415v - SMT disabled
Motherboard mATX Asrock AB350m AM4
Cooling Raijintek Leto Pro
Memory Vulcan T-Force 16GB DDR4 3000 16.18.18 @3200Mhz 14.17.17
Video Card(s) Sapphire Nitro+ 4GB RX 580 - 1450/2000 BIOS mod 8-)
Storage Seagate B'cuda 1TB/Sandisk 128GB SSD
Display(s) Acer ED242QR 75hz Freesync
Case Corsair Carbide Series SPEC-01
Audio Device(s) Onboard
Power Supply Corsair VS 550w
Mouse Zalman ZM-M401R
Keyboard Razor Lycosa
Software Windows 10 x64
Benchmark Scores https://www.3dmark.com/spy/6220813
What an advert for epyc and radeon instinct! massive contract, government contract, no doubt others will follow suit as I think the uptake of epyc hasn't been as swift as amd probably would have liked though if its good enough for the doe I suspect other government agencies, education institutes etc will begin to look at epyc as a viable xeon alternative.
 
Joined
Sep 27, 2014
Messages
550 (0.15/day)
US DOE spreads the "wealth" around :)
AMD, nVidia, Intel , PowerPC... my tax money at work.

31MW of power. Talking about "global warming"? :laugh:

Also AMD is only saying that the GPUs are “based on the Radeon Instinct family” and have “yet to be announced."
 
Last edited by a moderator:
Joined
Aug 20, 2007
Messages
21,556 (3.40/day)
System Name Pioneer
Processor Ryzen R9 9950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage Intel 5800X Optane 800GB boot, +2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64 / Windows 11 Enterprise IoT 2024
PS: 31MW of power. Talking about "global warming"

It'd be chump change for a meaningful understanding of what's going on up in the atmosphere so the world can stop bickering and make a decision... I mean honestly compared to the worlds heat output in a day, you are aware how little this is, right?

US DOE spreads the "wealth" around :)
AMD, nVidia, Intel , PowerPC... my tax money at work.

China's been kicking our ass here, bad. It started with Loongson and spiraled out of control since then... We need advances here or one day we'll be on the tail end of basic important technology like encryption. You DO NOT want that.

Far be it for me to SUPPORT Trump, but I think him boosting funding to projects like this is something he accidentally got right.
 
Last edited by a moderator:
Joined
Sep 27, 2014
Messages
550 (0.15/day)
I was sarcastic.

As a EE, I would love to be in the design team for the support building and utilities - power, HVAC, water... We are used to see something like max 30kW per cabinet :)
And I would rather have the tax money spent here in US than in rebuilding failed countries.
 
Joined
Sep 17, 2014
Messages
22,708 (6.05/day)
Location
The Washing Machine
System Name Tiny the White Yeti
Processor 7800X3D
Motherboard MSI MAG Mortar b650m wifi
Cooling CPU: Thermalright Peerless Assassin / Case: Phanteks T30-120 x3
Memory 32GB Corsair Vengeance 30CL6000
Video Card(s) ASRock RX7900XT Phantom Gaming
Storage Lexar NM790 4TB + Samsung 850 EVO 1TB + Samsung 980 1TB + Crucial BX100 250GB
Display(s) Gigabyte G34QWC (3440x1440)
Case Lian Li A3 mATX White
Audio Device(s) Harman Kardon AVR137 + 2.1
Power Supply EVGA Supernova G2 750W
Mouse Steelseries Aerox 5
Keyboard Lenovo Thinkpad Trackpoint II
VR HMD HD 420 - Green Edition ;)
Software W11 IoT Enterprise LTSC
Benchmark Scores Over 9000
This really is a confirmation AMD is back in the CPU game. A huge win for them in mindshare.

Well played.
 
Joined
Jan 8, 2017
Messages
9,517 (3.27/day)
System Name Good enough
Processor AMD Ryzen R9 7900 - Alphacool Eisblock XPX Aurora Edge
Motherboard ASRock B650 Pro RS
Cooling 2x 360mm NexXxoS ST30 X-Flow, 1x 360mm NexXxoS ST30, 1x 240mm NexXxoS ST30
Memory 32GB - FURY Beast RGB 5600 Mhz
Video Card(s) Sapphire RX 7900 XT - Alphacool Eisblock Aurora
Storage 1x Kingston KC3000 1TB 1x Kingston A2000 1TB, 1x Samsung 850 EVO 250GB , 1x Samsung 860 EVO 500GB
Display(s) LG UltraGear 32GN650-B + 4K Samsung TV
Case Phanteks NV7
Power Supply GPS-750C
Oh, no, selling a mirage!

You know well enough any dedicated GPU is yet to see the light of day from Intel. There's no marriage here mate.
 
Joined
Sep 15, 2007
Messages
3,946 (0.63/day)
Location
Police/Nanny State of America
Processor OCed 5800X3D
Motherboard Asucks C6H
Cooling Air
Memory 32GB
Video Card(s) OCed 6800XT
Storage NVMees
Display(s) 32" Dull curved 1440
Case Freebie glass idk
Audio Device(s) Sennheiser
Power Supply Don't even remember
US DOE spreads the "wealth" around :)
AMD, nVidia, Intel , PowerPC... my tax money at work.

31MW of power. Talking about "global warming"? :laugh:

Also AMD is only saying that the GPUs are “based on the Radeon Instinct family” and have “yet to be announced."

Will the AMD fans cry now like they cried when the Intel announcement was made on a similar note? Oh, no, selling a mirage! This is collusion, evil Intel AMD at work!

You're a bit special aren't ya? Intel has no product, period. AMD currently has one, another releasing first quarter of next year and then the next one in the pipe. What does intel have? Marketing lies lol.
 
Joined
Aug 20, 2007
Messages
21,556 (3.40/day)
System Name Pioneer
Processor Ryzen R9 9950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage Intel 5800X Optane 800GB boot, +2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64 / Windows 11 Enterprise IoT 2024
I was sarcastic.

As a EE, I would love to be in the design team for the support building and utilities - power, HVAC, water... We are used to see something like max 30kW per cabinet :)
And I would rather have the tax money spent here in US than in rebuilding failed countries.

I agree with all that, short of the "failed countries" bit. I'd like to believe all countries can one day succeed.

You're a bit special aren't ya? Intel has no product, period. AMD currently has one, another releasing first quarter of next year and then the next one in the pipe. What does intel have? Marketing lies lol.

Xeon... is a product? They do have those, say what you will about them.
 
Joined
Oct 17, 2011
Messages
857 (0.18/day)
Location
Oregon
System Name Red 101
Processor 9th Gen Intel Core i9-9900k
Motherboard EVGA Z370 Classified
Cooling Custom Primochill and Heatkiller water cooling loop
Memory 16GB of Gskill 3200Mhz CL14
Video Card(s) EVGA GeForce GTX 1080 FTW2 with Heatkiller block @2114Mhz
Storage 4- Samsung Evo 250GB, 1- Pro 512GB and 1-512GB M.2
Display(s) LG 38" UW
Case In Win 101 customized a lot and painted red
Audio Device(s) Razer Kraken 7.1 Chroma
Power Supply EVGA 850w G2
Mouse Razer DeathAdderv2
Keyboard Razer Ornata Chroma
Software Win10Pro and games
Benchmark Scores NA
going to need a nuclear power plant to run all those AMD chips :p
 
Joined
Jun 28, 2016
Messages
3,595 (1.16/day)
I'm not surprised by the EPYC part, but using Radeon Instinct is a slight concern.

Maybe Cray will help AMD write a proper API. Or port CUDA...
 
Joined
Aug 20, 2007
Messages
21,556 (3.40/day)
System Name Pioneer
Processor Ryzen R9 9950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage Intel 5800X Optane 800GB boot, +2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64 / Windows 11 Enterprise IoT 2024
Maybe Cray will help AMD write a proper API. Or port CUDA...

I have a feeling they are using OpenCL, which AMD has excellent support for.
 
Joined
Dec 12, 2016
Messages
1,959 (0.67/day)
I agree with all that, short of the "failed countries" bit. I'd like to believe all countries can one day succeed.



Xeon... is a product? They do have those, say what you will about them.

He's talking about Intel Xe (unreleased) vs. Instinct/Epyc series which have been released.

I'm not surprised by the EPYC part, but using Radeon Instinct is a slight concern.

Maybe Cray will help AMD write a proper API. Or port CUDA...

From the Anandtech article,

"And as the principle processor provider, AMD will also be taking on a lot of the responsibility for developing the software stack as well, with the company working with Cray to develop an enhanced version of their ROCm environment to best extract performance from the massive cluster of CPUs and GPUs. "
 
Joined
Jun 28, 2016
Messages
3,595 (1.16/day)
I have a feeling they are using OpenCL, which AMD has excellent support for.
Who's "they"? :)
These supercomputers are used by researchers. You have a project, you apply for access and they decide whether you're worthy or not. ;-)

It seems like going for Nvidia GPUs would be more flexible. Also, majority of their clusters use Nvidia GPUs already.
Suddenly ORNL ordered 2 supercomputers with GPUs made by Intel and AMD. It's slightly surprising - that's all.
 
Joined
Sep 17, 2014
Messages
22,708 (6.05/day)
Location
The Washing Machine
System Name Tiny the White Yeti
Processor 7800X3D
Motherboard MSI MAG Mortar b650m wifi
Cooling CPU: Thermalright Peerless Assassin / Case: Phanteks T30-120 x3
Memory 32GB Corsair Vengeance 30CL6000
Video Card(s) ASRock RX7900XT Phantom Gaming
Storage Lexar NM790 4TB + Samsung 850 EVO 1TB + Samsung 980 1TB + Crucial BX100 250GB
Display(s) Gigabyte G34QWC (3440x1440)
Case Lian Li A3 mATX White
Audio Device(s) Harman Kardon AVR137 + 2.1
Power Supply EVGA Supernova G2 750W
Mouse Steelseries Aerox 5
Keyboard Lenovo Thinkpad Trackpoint II
VR HMD HD 420 - Green Edition ;)
Software W11 IoT Enterprise LTSC
Benchmark Scores Over 9000
Who's "they"? :)
These supercomputers are used by researchers. You have a project, you apply for access and they decide whether you're worthy or not. ;-)

It seems like going for Nvidia GPUs would be more flexible. Also, majority of their clusters use Nvidia GPUs already.
Suddenly ORNL ordered 2 supercomputers with GPUs made by Intel and AMD. It's slightly surprising - that's all.

Might be a versatility move. They now have two different setups with I reckon the AMD being noticeably cheaper but perhaps equally good at getting a job done. Also, new Nvidia GPUs carry different hardware that may or may not be useful for the objectives they have in mind; and I'm not sure they still had the possibility for major requests of Pascal GPUs for example.
 
Joined
Jun 28, 2016
Messages
3,595 (1.16/day)
Might be a versatility move. They now have two different setups with I reckon the AMD being noticeably cheaper but perhaps equally good at getting a job done.
Well, the reality is that Nvidia cluster can run CUDA and this can't. That covers the "versatility" issue.
Whether this is cheaper or not - I have no idea. Maybe they simply wanted a customized GPU, in which case AMD is an easier partner.
Also, new Nvidia GPUs carry different hardware that may or may not be useful for the objectives they have in mind; and I'm not sure they still had the possibility for major requests of Pascal GPUs for example.
Once again: this is an all-round cluster, not built for a particular task. So the "additional hardware" is a plus. Especially when it's made for machine learning (it's quite popular, really :p).
Anyway, both V100 and P100 are still offered by Nvidia. I'm not sure about K80 - maybe it's limited to existing clients.
 

phill

Moderator
Staff member
Joined
Jun 8, 2011
Messages
17,005 (3.43/day)
Location
Somerset, UK
System Name Not so complete or overkill - There are others!! Just no room to put! :D
Processor Ryzen Threadripper 3970X
Motherboard Asus Zenith 2 Extreme Alpha
Cooling Lots!! Dual GTX 560 rads with D5 pumps for each rad. One rad for each component
Memory Viper Steel 4 x 16GB DDR4 3600MHz not sure on the timings... Probably still at 2667!! :(
Video Card(s) Asus Strix 3090 with front and rear active full cover water blocks
Storage I'm bound to forget something here - 250GB OS, 2 x 1TB NVME, 2 x 1TB SSD, 4TB SSD, 2 x 8TB HD etc...
Display(s) 3 x Dell 27" S2721DGFA @ 7680 x 1440P @ 144Hz or 165Hz - working on it!!
Case The big Thermaltake that looks like a Case Mods
Audio Device(s) Onboard
Power Supply EVGA 1600W T2
Mouse Corsair thingy
Keyboard Razer something or other....
VR HMD No headset yet
Software Windows 11 OS... Not a fan!!
Benchmark Scores I've actually never benched it!! Too busy with WCG and FAH and not gaming! :( :( Not OC'd it!! :(
I love seeing posts and news about AMD winning over contracts like this, as many have said, AMD are back in the game and so rightfully too :)

AMD, hats off to you sir/maam :)
 
Joined
Aug 20, 2007
Messages
21,556 (3.40/day)
System Name Pioneer
Processor Ryzen R9 9950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage Intel 5800X Optane 800GB boot, +2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64 / Windows 11 Enterprise IoT 2024
I doubt they will use open solutions.

A lot of mainframe/supercomputer projects depend on open source so it would surprise me if they didn't.

Who's "they"?

Unless you want to code to metal, you use the framework provided.

So effectively everyone.

Well, the reality is that Nvidia cluster can run CUDA and this can't. That covers the "versatility" issue.

There is also the other versatility, you know, the one of being able to operate on more platforms and with more software (open vs closed drivers).


It's slightly surprising - that's all.

I don't disagree. That's why I suspect a strange software stack that needs an open source driver for some part of the solution. I don't see another justification.
 
Last edited:
Joined
Aug 20, 2007
Messages
21,556 (3.40/day)
System Name Pioneer
Processor Ryzen R9 9950X
Motherboard GIGABYTE Aorus Elite X670 AX
Cooling Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory 64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s) XFX RX 7900 XTX Speedster Merc 310
Storage Intel 5800X Optane 800GB boot, +2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s) 55" LG 55" B9 OLED 4K Display
Case Thermaltake Core X31
Audio Device(s) TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply FSP Hydro Ti Pro 850W
Mouse Logitech G305 Lightspeed Wireless
Keyboard WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software Gentoo Linux x64 / Windows 11 Enterprise IoT 2024
Joined
Jun 28, 2016
Messages
3,595 (1.16/day)
There is also the other versatility, you know, the one of being able to operate on more platforms and with more software (open vs closed drivers).
I know you're advocating open source a lot, but this argument makes no sense.
If you're moving to a platform with different API, you have to rewrite everything. It doesn't matter if it's open or closed.

Who already has access to existing Nvidia clusters will likely stay there (especially for AI-related computing). New users will be moved to Frontier.

People have been using CUDA for a decade. It's the de facto standard.
Sure, I'd rather have something market wide in case Thanos snaps fingers and we're unlucky enough to lose the whole Nvidia team. But this standard should be CUDA. It's excellent. And everyone already uses it.
AMD and Intel should simply pay Nvidia and port it instead of wasting money on developing alternatives.

Anyway, we're going to see one more exa cluster announcement in USA (for LLNL). One went to Intel, one to AMD. Maybe that was the idea: provide 3 different architectures.
 
Top