Affordable HEDT, is there such a thing?

[Rorre]

Ok so you want to stay with standard ATX and Intel. Sadly, there are not a lot of options anymore for that combination. I'm in very much your situation. I want to upgrade from my X299 system and other than Threadripper, there just aren't any great options.

Hey Lex, I just outlined the options above in my discussions with Solaris17, beyond Threadripper 5900wx and TR 3900x which both are affordable and provide the necessary increase in PCIe lanes, they do not have AVX512 instruction set support that I require. The Xeon W-3300 IceLake platform has the increased lanes with Gen4 support, and AVX512 instruction set support. The motheroard and Xeon CPU can be had on eBay for less than the TR series.

 

[lexluthermiester]

The Xeon W-3300 IceLake platform

I looked at IceLake. The clock speeds seem a limiting factor. Can't debate the AVX512 though so if that is a desired feature then yeah, certain Intel Xeons would be the way to go.

 

[Wirko]

However it was the requirement for overclockable ECC RDIMM DDR5 ram that broke me....$1000 for 128GB?!!

Overclockable? It's the CPU that sets the memory data rate, not the memory itself. But 5 sure is much faster than 4 at 5600 MT/s vs 3200 or so. How much does memory bandwidth matter for CFD?

Also, those are Canadian dollars, right?

 

[Rorre]

I looked at IceLake. The clock speeds seem a limiting factor. Can't debate the AVX512 though so if that is a desired feature then yeah, certain Intel Xeons would be the way to go.

Affordability, increased PCIe gen4.0 lanes, and AVX512, in that order. With regards to clock speeds, I do wish to point out that the W-3335 (16c,32t) has a somewhat impressive base clock of 3.4 GHz an all core Turbo of 3.7 GHz with single core Turbo of 4.0. Pretty competitive with TR. but what really matters is the IPC. Number of cores is not important for my workloads, however I do agree the higher core counts do clock markedly lower.
My projects have migrated to more GPU compute.

Overclockable? It's the CPU that sets the memory data rate, not the memory itself. But 5 sure is much faster than 4 at 5600 MT/s vs 3200 or so. How much does memory bandwidth matter for CFD?

Also, those are Canadian dollars, right?

Overclockable to the extent that the W790 motherboards and Sapphire Rapids allow for overclocking and to use XMP profiles. Previous ECC DDR4 could not utilize XMP profiles.

Affordability, increased PCIe gen4.0 lanes, and AVX512, in that order. With regards to clock speeds, I do wish to point out that the W-3335 (16c,32t) has a somewhat impressive base clock of 3.4 GHz am all core Turbo of 3.7 GHz with single core Turbo of 4.0. Pretty competitive with TR. but what really matters is the IPC. Number of cores is not important for my workloads, however I do agree the higher core counts do clock markedly lower.
My projects have migrated to more GPU compute.


Overclockable to the extent that the W790 motherboards and Sapphire Rapids allow for overclocking and to use XMP profiles. Previous ECC DDR4 could not utilize XMP profiles.

And that's just it, I don't need overclocking capability.

 

[lexluthermiester]

Affordability, increased PCIe gen4.0 lanes, and AVX512, in that order. With regards to clock speeds, I do wish to point out that the W-3335 (16c,32t) has a somewhat impressive base clock of 3.4 GHz am all core Turbo of 3.7 GHz with single core Turbo of 4.0. Pretty competitive with TR. but what really matters is the IPC. Number of cores is not important for my workloads, however I do agree the higher core counts do clock markedly lower.

Fair enough. However, that 16core model is only marginally better than what you have now. But as you said PCIe lanes for GPGPUs are more important for you.

 

[Rorre]

Overclockable? It's the CPU that sets the memory data rate, not the memory itself. But 5 sure is much faster than 4 at 5600 MT/s vs 3200 or so. How much does memory bandwidth matter for CFD?

Also, those are Canadian dollars, right?

Memory bandwidth is extremely important for CFD, 8 channels would ne most helpful. Thus is a good article addressing best requirements.

General recommendations for CFD hardware [WIP] -- CFD Online Discussion Forums

*WORK IN PROGRESS* The goal of this thread is to cover most of the basics that come up in almost every CFD hardware recommendation topic. There will

www.cfd-online.com

Actually thats incorrect now. 621 was the intermediary socket. With some manufacturers designating "WC" instead of just "C"; Passed that workstation chipsets are designated "W" with server chipsets designated "C". Additionally, server chipsets keep the nomeclature such as "C741" whereas workstation chipsets adhere (for now) to desktop naming conventions IE "W790". Of course given there slower release cadence they will reflect whatever is current at time of release.

Just something to keep in mind when you come across a good deal since they changed the naming convention. For example some supermicro C741 can be had used cheap, but they will not boot your $1000 ES xeon.



Snap it up before its gone!

There are a few Xeon W-3300 series CPU's on eBay competitively priced. As well the corresponding SuperMicro and AsRock motherboards.

 

[Wirko]

Yeah, so any upgrade based on DDR4 is very sub-optimal, but with Ice Lake W at least you have the possibility to upgrade again in a couple years. That's if/when many-core models become cheaper on the used market. Server variants (Gold etc) would also work in the same motherboards, right?

 

[unwind-protect]

Yeah, so any upgrade based on DDR4 is very sub-optimal, but with Ice Lake W at least you have the possibility to upgrade again in a couple years. That's if/when many-core models become cheaper on the used market. Server variants (Gold etc) would also work in the same motherboards, right?

The speed difference between DDR4 and DDR5 was very small.

And DDR4, at least ECC registered is like half the price. For the typical usage pattern (with lots of RAM) that translates to substantial overall savings.

 

[lexluthermiester]

Yeah, so any upgrade based on DDR4 is very sub-optimal

Not at all. Quad(or more) channel DDR4 provides more than enough memory bandwidth not to be a bottleneck. I have X299 and the memory is not remotely the limiting factor.

 

[venturi]

You have some choices for the AVX 512:
You could build a dual intel 8280L, DDR4, on a Asus Sage c621e mobo. Affordable, plenty of 8280 on ebay, even new, and the cpus cost very little, the Asus mobo is 600 on amazon

you could do dual 7773x Epyc on MZ72-HB0, DDR4

you could do dual 9684x Epyc on MZ73-LM0 DDR5

All these mobos allow multi GPU,while the cheaper asus board, whild older design, offers some really nifty Vrock raid option.
All boards offer at least 4 of the u.2 and some u.3 connectivity
Two choices are on DDR4.

all these builds will fit on a a smaller build, no need for anything rack sized.
Get a decent PSU, at least Corsair Ax1600i or Hela 2050R, both PSUs work on 120 and 240

OS choices are Ubuntu or MS Server 2022 data center or 2025 preview (you can auto update tp 2025)
You can do bare metal or VM, kubernetes, token, etc etc.

Soma savviness, and you can go a long way with either choices, and you can build any of these into a relatively small footprint, and low noise.

J

 

[Rorre]

You have some choices for the AVX 512:
You could build a dual intel 8280L, DDR4, on a Asus Sage c621e mobo. Affordable, plenty of 8280 on ebay, even new, and the cpus cost very little, the Asus mobo is 600 on amazon

you could do dual 7773x Epyc on MZ72-HB0, DDR4

you could do dual 9684x Epyc on MZ73-LM0 DDR5

All these mobos allow multi GPU,while the cheaper asus board, whild older design, offers some really nifty Vrock raid option.
All boards offer at least 4 of the u.2 and some u.3 connectivity
Two choices are on DDR4.

all these builds will fit on a a smaller build, no need for anything rack sized.
Get a decent PSU, at least Corsair Ax1600i or Hela 2050R, both PSUs work on 120 and 240

OS choices are Ubuntu or MS Server 2022 data center or 2025 preview (you can auto update tp 2025)
You can do bare metal or VM, kubernetes, token, etc etc.

Soma savviness, and you can go a long way with either choices, and you can build any of these into a relatively small footprint, and low noise.

J

The Asus C621e Socket 3647 motherboard in single or dual CPU is a nice option. The Xeon W-3200 series and the Xeon Scalable are most affordable, especially in low core count. However, they only provide for PCIe Gen3 lanes, it may disadvantage the the PCIe 4.0 GPUs that I intend to install.
Multi CPU in openFoam or any number of Computational Fluid Dynamics software can be an advantage, but if you balance properly the 8 channel memory with the number of cores available you can achieve very good compute even in single CPU systems. High bandwidth to memory and PCIe 4.0 bandwidth to the data is most important. Low Core Count CPUs usually have higher clock speeds, and that is preferred; high core count not so much.
For not much more, I think Socket 4189 of the C621a motherboards might provide a system of increased relevance for my intended use.

Yeah, so any upgrade based on DDR4 is very sub-optimal, but with Ice Lake W at least you have the possibility to upgrade again in a couple years. That's if/when many-core models become cheaper on the used market. Server variants (Gold etc) would also work in the same motherboards, right?

High core count is a detriment to CFD compute due to usually, the lower clock speed of HCC. The 8 channel memory with higher CPU clock speeds allow for faster transfers of memory to the CPU. PCIe 4.0 bandwidth is also good for speedy transfer between GPU compute, and data transfers also to the storage RAID.

The speed difference between DDR4 and DDR5 was very small.

And DDR4, at least ECC registered is like half the price. For the typical usage pattern (with lots of RAM) that translates to substantial overall savings.

And ECC DDR5 is not cheap. From what've seen 256GB of ECC DDR5 is about 1000 to 1200 USD.

 

[lexluthermiester]

And ECC DDR5 is not cheap. From what've seen 256GB of ECC DDR5 is about 1000 to 1200 USD.

And that's the slower models, IF you can find a good deal.

 

[eidairaman1]

Ok now that I have current info on AMD HEDT, TR with ddr 5 is on Skt sTR5, Epyc derivative is on SP6. At least AMD doesn't have 2 separate TR sockets like with ddr4.