# Xeon or 3900x for home server



## Papahyooie (Jul 16, 2020)

I'm in an upgrade conundrum.

I have a home server that I do/would like to use for home server things. Main purposes are, in order of priority:

NAS
Plex server/torrent server (CPU encoding if I can get it, but GPU if I can't, already have GPU for this. May use docker, maybe not.)
Various docker toys
Game servers
Web Server
Virtualization (probably use unraid, one 24/7 windows VM, one 24/7 linux VM, and whatever else I may decide I want to spin up/down as I please. Maybe play around with a virtualized gaming rig like Linus or something)
Twitch streaming machine (CPU encoding if I can get it, but GPU is ok too, already have GPUs for it)

Target the above, with 48 gigs of ram (will want more later, but this is to start)
Options:

1. I have a broadwell-ep mobo and 16gb of DDR4. So for the money I've got, I can buy an engineering sample E5-2699 v4, 22c/44t processor, 32 gigs of DDR4 and throw the extra on some hard drives (not important at the moment, but nice.)


2. For the same money as above, I could buy a 3900x, mobo, and 32 gigs of ram. Sell above mobo, maybe buy some storage/more ram/a sandwich/whatever.

3. ???

Pros/Cons:


1 Pros:
CORES! Super lotta cores to spin up VMs as needed, lots of breathing room to assign cores as needed.
Already have mobo with ten Sata ports for lots of drives
quad channel memory
Although older, CPU isn't ancient. Decent single threaded performance.
CORES!

1 Cons:
Older processor
12 cores on option 2 beat total passmark of even the 22 cores by quite a margin
Engineering sample (can't afford a real one lol) so used/no warranty, etc

2 Pros:
Beefy 3900x kills total cpu grunt of Xeon.
Newer/energy efficient/warranty

2 cons:
Not as many cores to spread around, so virtualization will be way less
Not as many cores, so CPU encoding for Plex/streaming is out of the picture most likely, as I'd need dedicated cores for that.

I'm really torn on this. I can't decide whether it's more important to me to have a shed load of cores to assign as I please, or have the overall massive CPU power of the Ryzen with less cores. 

Help me push a button. I can't decide!


----------



## basco (Jul 17, 2020)

if it runs 24\7 option 2

and i did not have good experience with es chips with no warranty and a 980x es that lost all but one core after 1 week.
you want a stable setup and es chips are not for this-m2c


----------



## TheLostSwede (Jul 17, 2020)

What OS are you going to run? I think AMD might be a bit behind still in terms of Linux support, although I guess it helps that Linus Torvalds got himself an AMD rig recently...


----------



## silentbogo (Jul 17, 2020)

Forget about ES, just get a retail Xeon. Price difference is minimal or non-existent for certain models. 
Or simply look around ebay or equivalent for something like a used HP Z440/640 workstation, or Dell T7810. Those are getting cheaper by the day.
If your priority is cores (e.g. running as many VMs as smoothly as possible),  then this definitely beats Ryzen, since you can get a whole workstation for the price of CPU alone.
Also, both support NVME out of the box (just need a PCIe to M.2 adapter). You can spend your leftover budget on fancy things like 10G NICs, decent switch, HDDs/SSDs etc etc etc.
Another big benefit  is reliability. If you plan to run it 24/7 - that hardware is made for it. I'm still a bit skeptical about running consumer Ryzen boards reliably in a server.


----------



## Papahyooie (Jul 17, 2020)

basco said:


> if it runs 24\7 option 2
> 
> and i did not have good experience with es chips with no warranty and a 980x es that lost all but one core after 1 week.
> you want a stable setup and es chips are not for this-m2c



Good to hear some experience with ES chips. I've heard good things and bad things. I'd be buying from Ebay, so if it's DOA ebay would protect me. But if it dies down the line, I'd be real mad lol.



TheLostSwede said:


> What OS are you going to run? I think AMD might be a bit behind still in terms of Linux support, although I guess it helps that Linus Torvalds got himself an AMD rig recently...



Probably unraid as a hypervisor just cause it's so easy. Under that, windows, ubuntu, probably random other things. I haven't heard of any problems with AMD processors on linux. What specifically?



silentbogo said:


> Forget about ES, just get a retail Xeon. Price difference is minimal or non-existent for certain models.
> Or simply look around ebay or equivalent for something like a used HP Z440/640 workstation, or Dell T7810. Those are getting cheaper by the day.
> If your priority is cores (e.g. running as many VMs as smoothly as possible),  then this definitely beats Ryzen, since you can get a whole workstation for the price of CPU alone.
> Also, both support NVME out of the box (just need a PCIe to M.2 adapter). You can spend your leftover budget on fancy things like 10G NICs, decent switch, HDDs/SSDs etc etc etc.
> Another big benefit  is reliability. If you plan to run it 24/7 - that hardware is made for it. I'm still a bit skeptical about running consumer Ryzen boards reliably in a server.



A retail 2699 goes for over twice my budget (~650 USD) on Ebay. The retail xeons that fit in my budget are 10-12 cores, which would defeat the purpose of going with the xeon, as the Ryzen is well over twice as powerful. Looking at workstations, those with high core counts go again, for twice my budget. I'm def not going for a xeon with much lesser core counts than the 2699, as that would defeat the purpose of going with the xeon (more cores.) Plus, I already have a mobo for a xeon. Are you seeing deals that I'm not seeing? Please share if you are lol.
As for reliability, I've been running an AMD FX-8320 for years 24/7 in a consumer asus board as my server. Not too worried about it.


----------



## TheLostSwede (Jul 17, 2020)

Papahyooie said:


> Probably unraid as a hypervisor just cause it's so easy. Under that, windows, ubuntu, probably random other things. I haven't heard of any problems with AMD processors on linux. What specifically?


Mainly driver related things, as it's still "early" days for the platform in a Linux sense. It was a while since a looked into it and things might well have improved.


----------



## Papahyooie (Jul 17, 2020)

TheLostSwede said:


> Mainly driver related things, as it's still "early" days for the platform in a Linux sense. It was a while since a looked into it and things might well have improved.


Ah I see. Well, everybody seems to be using Ryzen processors for unraid virtualization servers, so it must at least meet the basic functionality. Any other linux distros running under the hypervisor won't need any hardware pass-through like GPUs, so I'm not too worried about that.


----------



## TheLostSwede (Jul 17, 2020)

Papahyooie said:


> Ah I see. Well, everybody seems to be using Ryzen processors for unraid virtualization servers, so it must at least meet the basic functionality. Any other linux distros running under the hypervisor won't need any hardware pass-through like GPUs, so I'm not too worried about that.


From a quick search, it seems like some of the peripheral interfaces can still have some issues, but it doesn't seem like there are any major issues at least.


----------



## BoboOOZ (Jul 17, 2020)

Papahyooie said:


> 2 Pros:
> Beefy 3900x kills total cpu grunt of Xeon.


It doubles it, basically. I really don't think there are so many scenarios where you will be happier with the Xeon.

If really in doubt, wait a few more months for Ryzen 4000 to launch and grab a cheaper 3950X.


----------



## silentbogo (Jul 17, 2020)

Papahyooie said:


> A retail 2699 goes for over twice my budget (~650 USD) on Ebay. The retail xeons that fit in my budget are 10-12 cores, which would defeat the purpose of going with the xeon


No one forbids you from going dual-socket. 
Cheapest option is Dell T7810 or T7910 combined with pair of 2660v3's. Just don't buy a pre-built to your spec, get a barebone or single-CPU variant, and then add a pair of 2660s or higher to get the most out of it. Sometimes you can score a decent T7810 barebone for less than $300, incl. both heatsinks and PSU. 2660s go for ~$250/pair or something like that. That's 20C/40T in a box. If you don't mind lackluster general performance, you can even go as low as matching that Ryzen core/thread count by going with cheapest-of-the-cheap E5-2620v3 (which is currently THE cheapest 6-core CPU, even comparing to a good-ole x5650). Bought me a test chip for an equivalent of $18 just a couple of months ago (and it's a retail version, not QS/ES crap).



Papahyooie said:


> Ah I see. Well, everybody seems to be using Ryzen processors for unraid virtualization servers, so it must at least meet the basic functionality.


Ryzen is more than fine for that. Scheduling issues on Linux(and windows, for that matter) have been resolved a long time ago. Didn't hear about any other serious issues either. Everything should work OOB.


----------



## thesmokingman (Jul 17, 2020)

The 3900x, hands down. Not only is is better and more efficient, it's cores do much more even with 10 less cores.


----------



## Papahyooie (Jul 17, 2020)

silentbogo said:


> No one forbids you from going dual-socket.
> Cheapest option is Dell T7810 or T7910 combined with pair of 2660v3's. Just don't buy a pre-built to your spec, get a barebone or single-CPU variant, and then add a pair of 2660s or higher to get the most out of it. Sometimes you can score a decent T7810 barebone for less than $300, incl. both heatsinks and PSU. 2660s go for ~$250/pair or something like that. That's 20C/40T in a box. If you don't mind lackluster general performance, you can even go as low as matching that Ryzen core/thread count by going with cheapest-of-the-cheap E5-2620v3 (which is currently THE cheapest 6-core CPU, even comparing to a good-ole x5650). Bought me a test chip for an equivalent of $18 just a couple of months ago (and it's a retail version, not QS/ES crap).
> 
> 
> Ryzen is more than fine for that. Scheduling issues on Linux(and windows, for that matter) have been resolved a long time ago. Didn't hear about any other serious issues either. Everything should work OOB.



Dual 2660's is def an option. I didn't realize they went for as cheap as they do. Matching the ryzen's core count with the xeons def isn't an option, as then I'd just buy the Ryzen which way faster lol.


----------



## thesmokingman (Jul 17, 2020)

Yea, and you have a path to Ryzen 3... or discounted 3950x in the future.


----------



## Papahyooie (Jul 17, 2020)

thesmokingman said:


> Yea, and you have a path to Ryzen 3... or discounted 3950x in the future.


I realize how much more powerful the ryzen is. The issue is the lower core number (which, if I'm running VMs under a hypervisor, need cores assigned to them.) That's the very issue that's plaguing me.


----------



## thesmokingman (Jul 17, 2020)

Papahyooie said:


> I realize how much more powerful the ryzen is. The issue is the lower core number (which, if I'm running VMs under a hypervisor, need cores assigned to them.) That's the very issue that's plaguing me.



Is it the number of cores or how much work they do that actually counts?


----------



## Solaris17 (Jul 17, 2020)

I would never use a consumer CPU for virt, or buy into the platform. Seems like the thread is going in circles though because it looks like your mind is already made up.


----------



## Papahyooie (Jul 17, 2020)

thesmokingman said:


> Is it the number of cores or how much work they do that actually counts?


Lol that's the conundrum. Since one of my goals is to run VMs, which need sequestered cores assigned to them, the lower core count on the ryzen reduces headroom to run multiple VMs. BUT, on the other hand, the ryzen does more work per core.

That's the exact question I'm trying to decide here, lol.



Solaris17 said:


> I would never use a consumer CPU for virt, or buy into the platform. Seems like the thread is going in circles though because it looks like your mind is already made up.


I'm not sure where you got that idea. My mind is far from made up. @silentbogo provided a solution that sidesteps the engineering sample issues and has nearly as many cores as the 2699, and I'm seriously considering that path vs the ryzen. Why wouldn't you go with a consumer processor for virt?


----------



## dragontamer5788 (Jul 17, 2020)

This seems odd to me that you're comparing a 12-core Ryzen to a 22-core Xeon. Shouldn't you be comparing against a 24-core Threadripper or 24-core EPYC instead?

----------

Hmmm... I guess the 22-core Xeon is far older and will be on used hardware. I'd personally want newer systems. I guess buying used (and using older equipment) is going to be cheaper in a sense. The E5-2699 v4 is rated at 145W TDP, and is made on an older node (higher watts == more power / air conditioning to run your server).

I'm biased towards newer equipment in general. I'd say buy whatever's newer, and in this case, that's the Ryzen. I would rerun the numbers against a modern 2x Xeon Silver system, as well as EPYC and Threadripper systems.


----------



## ERazer (Jul 17, 2020)

I have r5 2600 eventually will pop-in my r9 3900x, been running 2600 for about x6 months now and no issues with VM's so far (osx, ubuntu, and win10)

got 6 dockers, GPU for Plex encoding, still have lotta juice for what i use it for, you should be good with 3900x.


----------



## silentbogo (Jul 17, 2020)

Papahyooie said:


> Matching the ryzen's core count with the xeons def isn't an option, as then I'd just buy the Ryzen which way faster lol.


Then Ryzen it is ))) Just make sure you get a board with decent VRM, if you go B550 route.


----------



## Solaris17 (Jul 17, 2020)

Papahyooie said:


> I'm not sure where you got that idea. My mind is far from made up. @silentbogo provided a solution that sidesteps the engineering sample issues and has nearly as many cores as the 2699, and I'm seriously considering that path vs the ryzen. Why wouldn't you go with a consumer processor for virt?



A lot. So much. Id maybe do it if I was learning but not for production. Consumer chips and boards have the ability to enable virt but that doesnt make them servers.

Core count is also a misconception. You might not be understanding it incorrectly because vCPUs are not CPUs. They are slices of CPU time, the host CPU scheduler will run the tasks on whatever cores it can or wants. Thats why its possible to over provision CPU cores. 

As for CPU counts and correlation if your primary reason is transcoding then maybe its important, but VMs will eat tons of RAM. For the most part you will run out of memory resources before you run out of processing resources.

IF you are thinking about running Plex in a VM as well thats going to be tough, some hypervisors might let you pass through a GPU for encoding but I doubt you could do it on a consumer board because the PCI-E pass through virt extensions dont generally exist on consumer gear.

You would likely need to run plex on bare metal


----------



## Papahyooie (Jul 17, 2020)

dragontamer5788 said:


> This seems odd to me that you're comparing a 12-core Ryzen to a 22-core Xeon. Shouldn't you be comparing against a 24-core Threadripper or 24-core EPYC instead?
> 
> ----------
> 
> ...



Well yea, if I could buy a brand new threadripper, I would. You're completely ignoring budget lol. The idea is that I can get a high core count xeon for the same price as I'd get a ryzen machine. 



Solaris17 said:


> A lot. So much. Id maybe do it if I was learning but not for production. Consumer chips and boards have the ability to enable virt but that doesnt make them servers.
> 
> Core count is also a misconception. You might not be understanding it incorrectly because vCPUs are not CPUs. They are slices of CPU time, the host CPU scheduler will run the tasks on whatever cores it can or wants. Thats why its possible to over provision CPU cores.
> 
> ...



In the title: "Home Server." This is not a "production" environment. 
Most hypervisors that I'm aware of require (or at least support) dedicated CPU cores. Yea, if I was running VMs under windows, it'd use virtual CPUs and the scheduler would manage that. If unraid, vmware, etc can use virtual CPU's that's fine, but the whole point of using a hypervisor in the first place is to have near-bare-metal performance, so I probably wouldn't do that anyway even if I had the option. 
I've already confirmed that docker plex can use hardware transcoding under unraid at least, on the board I'm planning on using if I go the ryzen route.


----------



## ERazer (Jul 17, 2020)

Solaris17 said:


> IF you are thinking about running Plex in a VM as well thats going to be tough, some hypervisors might let you pass through a GPU for encoding but I doubt you could do it on a consumer board because the PCI-E pass through virt extensions dont generally exist on consumer gear.



Imma chip in, if he pick unraid definitely run plex thru docker and pass thru gpu for encoding.

with recent unraid updates they made it really really easy to pass thru pcie devices even with consumer boards


----------



## Solaris17 (Jul 17, 2020)

ERazer said:


> Imma chip in, if he pick unraid definitely run plex thru docker and pass thru gpu for encoding.
> 
> with recent unraid updates they made it really really easy to pass thru pcie devices even with consumer boards



I saw docker mentioned in the OP as well. If its going to be done this is by far the easiest option.


----------



## dragontamer5788 (Jul 17, 2020)

Papahyooie said:


> Well yea, if I could buy a brand new threadripper, I would. You're completely ignoring budget lol. The idea is that I can get a high core count xeon for the same price as I'd get a ryzen machine.



Hmmm... https://www.ebay.com/itm/Intel-Xeon...-LGA-2011-3-2-2GHz-22-Core-SR2J0/351996572963

What prices are you seeing for E5 2699 v4? I did a quick check on Ebay and it seems to be $1000 USD or so. That really seems close to a modern Xeon-Silver (or even dual-Xeon Silvers), or a Threadripper / EPYC build rather than a Ryzen build to me.

A lot of this comes down to price. So... what prices are you basing your opinion off of?


----------



## Papahyooie (Jul 17, 2020)

dragontamer5788 said:


> Hmmm... https://www.ebay.com/itm/Intel-Xeon...-LGA-2011-3-2-2GHz-22-Core-SR2J0/351996572963
> 
> What prices are you seeing for E5 2699 v4? I did a quick check on Ebay and it seems to be $1000 USD or so. That really seems close to a modern Xeon-Silver (or even dual-Xeon Silvers), or a Threadripper / EPYC build rather than a Ryzen build to me.
> 
> A lot of this comes down to price. So... what prices are you basing your opinion off of?


As I said, engineering samples of 2699 go for far less. Options 1 & 2 in my OP are basically the same price.


----------



## dragontamer5788 (Jul 17, 2020)

Papahyooie said:


> As I said, engineering samples of 2699 go for far less. Options 1 & 2 in my OP are basically the same price.



I'd go with the Ryzen. But there's some stuff to discuss still...

1. Do you care about ECC RAM? This plays to Xeon's benefit of course.
2. Do you have a BMC on that Xeon motherboard? Remote-access to a server-like system is extremely useful, and would also play to the Xeon's advantage.

Remote-KVM probably can be hacked onto the Ryzen system with the right hardware. BMC / Remote KVM features of the Xeon might be locked out behind a license (really depends on who manufactured your server-motherboard). But... if you have any confidence in the Xeon-BMC working, I'd lean the Xeon if only for the ease of maintenance associated with IPMI and whatnot.


----------



## Papahyooie (Jul 17, 2020)

dragontamer5788 said:


> I'd go with the Ryzen. But there's some stuff to discuss still...
> 
> 1. Do you care about ECC RAM? This plays to Xeon's benefit of course.
> 2. Do you have a BMC on that Xeon motherboard? Remote-access to a server-like system is extremely useful, and would also play to the Xeon's advantage.
> ...



ECC ram isn't a factor. No need for it. 
There is no BMC on the mobo that I have, it's a consumer Broadwell EP mobo. Though it would be nice, it's not really necessary.  I will have local access to the machine pretty much all the time. The current one usually runs headless, but there's a monitor/kbm sitting right beside it if needed. Not really a factor. When the machine is live, I use remote desktop. Not sure if the workstations that were suggested earlier have one, but maybe. I think I'm pretty much ruling out the Engineering sample route at this point, just due to the nature of such a part being possibly unreliable. Now the contenders are the workstation with dual 10-core xeons (which actually seem to beat the single 22 core xeon by a slight edge, and are surprisingly within budget) and the ryzen. So the mobo I'm using now is pretty much out of the equation anyway.


----------



## Cybrnook2002 (Jul 17, 2020)

Super happy with my setup. I was running a handful of 2011v3 system for unraid for some time, but ultimately stopped since it was high power draw for low per core clock speeds. 1920x (Bought it when it was $200) has been great in the AsRock X399D8A-2T. I am on a AsRock beta bios (1.37) and everything is working that I want. Bifurcation works as expected on a per PCIe slot basis, and GPU passthrough works fine for my 2060. Prior to this I was running an AsRock X470D4U2-2T (still have it) with an 3600X (still have it), and that also worked fine. However I found that I wanted more PCie lanes, and being in a 2U chassis, I really couldn't go over the 3600X without a fancy cooling solution (think Dynatron L3). I had a 3700X in there, but it was just cooking, so if you are looking at a 3900X, make sure you can cool it.


----------



## Papahyooie (Jul 17, 2020)

Cybrnook2002 said:


> Super happy with my setup. I was running a handful of 2011v3 system for unraid for some time, but ultimately stopped since it was high power draw for low per core clock speeds. 1920x (Bought it when it was $200) has been great in the AsRock X399D8A-2T. I am on a AsRock beta bios (1.37) and everything is working that I want. Bifurcation works as expected on a per PCIe slot basis, and GPU passthrough works fine for my 2060. Prior to this I was running an AsRock X470D4U2-2T (still have it) with an 3600X (still have it), and that also worked fine. However I found that I wanted more PCie lanes, and being in a 2U chassis, I really couldn't go over the 3600X without a fancy cooling solution (think Dynatron L3). I had a 3700X in there, but it was just cooking, so if you are looking at a 3900X, make sure you can cool it.



I looked into early threadripper, but since the 3900x beats it in total power, plus the fact that the processor (if I went with one with a core advantage over the 3900x) plus the mobo will likely be out of my budget, I decided against it. Not TOO awfully worried about PCI-E lanes, as I won't be doing more than MAYBE two gpu's, and then only if I need them for separate VMs, which I don't think I will. The machine will be on an open-air case in a closet, so cooling shouldn't be awful. The FX-8320 and FX-8370 before that are fine, so I don't expect the 3900X would be much worse if any.

Thanks for the perspective from someone that has done both!


----------



## Cybrnook2002 (Jul 17, 2020)

If you are in the market for the AsRock rack board (X470D4U2-2T) let me know, has dual intel 10Gbe lan, and a couple PCIe slots. Might work out better for you, plus has onboard BMC.


----------



## Papahyooie (Jul 17, 2020)

Cybrnook2002 said:


> If you are in the market for the AsRock rack board (X470D4U2-2T) let me know, has dual intel 10Gbe lan, and a couple PCIe slots. Might work out better for you, plus has onboard BMC.


PM'd


----------



## ERazer (Jul 17, 2020)

Cybrnook2002 said:


> If you are in the market for the AsRock rack board (X470D4U2-2T) let me know, has dual intel 10Gbe lan, and a couple PCIe slots. Might work out better for you, plus has onboard BMC.



I been checking that board how is it? I read some issues heat sink compatibility.

on that note we should start unRaid club , it gets so underrated.


----------



## Cybrnook2002 (Jul 18, 2020)

ERazer said:


> I been checking that board how is it? I read some issues heat sink compatibility.
> 
> on that note we should start unRaid club , it gets so underrated.


The only issue I had was that the mosfet heatsink next to the CPU normally slants towards the CPU socket. It hovers over some caps, but doesn't actually make contact. The easy fix was to just unscrew the heatsink, rotate it 180 and screw it back down. Heatsink and thermal pads land right back on the mosfets where they belong, and the slant now runs towards the PCIe slot, which has miles of clearance. I put some photos and all over in the level1techs forums discussion about that board.


----------



## silentbogo (Jul 18, 2020)

dragontamer5788 said:


> 1. Do you care about ECC RAM? This plays to Xeon's benefit of course.
> 2. Do you have a BMC on that Xeon motherboard? Remote-access to a server-like system is extremely useful, and would also play to the Xeon's advantage.


1. Pretty much all AMD CPUs since FX support ECC UDIMM. Even mainstream intel chips do. RDIMMs is a whole other story
2. DASH or AMT will do just fine for a hobbyist. Even some Xeon workstations don't have a dedicated BMC chip and get by just fine with software solutions


----------



## Papahyooie (Jul 18, 2020)

silentbogo said:


> 1. Pretty much all AMD CPUs since FX support ECC UDIMM. Even mainstream intel chips do. RDIMMs is a whole other story
> 2. DASH or AMT will do just fine for a hobbyist. Even some Xeon workstations don't have a dedicated BMC chip and get by just fine with software solutions


I've been reading about dash and AMT, but from what I'm reading it isn't available on normal Ryzen processors. Is there something I'm missing?


----------



## silentbogo (Jul 18, 2020)

Papahyooie said:


> . Is there something I'm missing?


At least according to AMD you need any Ryzen Pro and a compatible board. Realistically CPU shouldn't matter, just the board.
AsRock and GB had a few models with DASH


----------



## Papahyooie (Jul 18, 2020)

silentbogo said:


> At least according to AMD you need any Ryzen Pro and a compatible board. Realistically CPU shouldn't matter, just the board.
> AsRock and GB had a few models with DASH


I see. Thanks for the info!


----------

