# 1950X Goes full blown 128GB RAM!



## xkm1948 (May 21, 2018)

So the PI I have been working with tasked me to add more RAM to the Threadripper Genome Workstation we built back in February.

The RAM arrived today. So popped them right in and BAM works right off the bat. Gotta say Threadripper 128GB memory tuning felt a lot easier than my X99. With XMP they run at 2666 rated speed just fine for 1.2V. Bumping the voltage to 1.35V would allow me to run these Corsair LPX kits at 2933.  However since what they do is pretty intolerant of ANY error we decided to run the RAM only at rated 2666.

While I was at it I replaced the stock NZXT X62 thermal paste with Kryonaut. Used TPU's own memory testing app for a quick 4 hours RAM stability test and everything runs just fine. Finally I had a bit fun OCing the 1950X, pushing it to 4.05GHz with 1.4V Vcore. Not too bad.

The workstation is already back to work assembling another genome. Gotta say the TR series is really awesome. Potentially we may be building several more TR based bioinformatics station once Threadripper 2 comes out later this year.


----------



## _UV_ (May 21, 2018)

Maybe used real server from data center will be same price and performance and more reliable for scientific calculations? It will be louder, but... just my opinion.


----------



## xkm1948 (May 21, 2018)

_UV_ said:


> Maybe used real server from data center will be same price and performance and more reliable for scientific calculations? It will be louder, but... just my opinion.



It would be difficult to justify buying used equipment in Academia. 

Passing used equipment between labs in the same department is fine. Buying used parts is not.


----------



## _UV_ (May 21, 2018)

How about "academic prices" from HP or Dell, if they exist, probably not, but you may try next time.


----------



## xkm1948 (May 21, 2018)

_UV_ said:


> How about "academic prices" from HP or Dell, if they exist, probably not, but you may try next time.



Yeah no. For a 8 core server Dell wanna charge us $5000. Hell no.

We did all the homework, Threadripper is the best bang for the buck.


----------



## Caring1 (May 21, 2018)

The +4 on your 24 pin power connector isn't pushed in all the way, it should be put in first or coupled correctly with the 20 pin when placed in.
Also EEC ram could be used in that board.


----------



## xkm1948 (May 22, 2018)

Caring1 said:


> The +4 on your 24 pin power connector isn't pushed in all the way, it should be put in first or coupled correctly with the 20 pin when placed in.
> Also EEC ram could be used in that board.




Good eye. Just fixed it.


----------



## Boatvan (May 22, 2018)

I assume you work in higher ed because we couldn't even dream of a system like this to play with on our poverty k12 budget 
Seriously though that's really cool. So it is for research purposes (you mentioned genome work)? I won't pry for details but I love seeing hardware of this caliber being used for a real project instead of just for enthusiast fun.


----------



## qubit (May 22, 2018)

How much did that RAM cost you?! 

Great machine and awesome use as a genome sequencer. That's for fighting genetic diseases, I take it?


----------



## xkm1948 (May 22, 2018)

Caring1 said:


> The +4 on your 24 pin power connector isn't pushed in all the way, it should be put in first or coupled correctly with the 20 pin when placed in.
> Also EEC ram could be used in that board.



ECC unbuffered RAM costs quite a lot more. For now these will serve fine.



Boatvan said:


> I assume you work in higher ed because we couldn't even dream of a system like this to play with on our poverty k12 budget
> Seriously though that's really cool. So it is for research purposes (you mentioned genome work)? I won't pry for details but I love seeing hardware of this caliber being used for a real project instead of just for enthusiast fun.



Not prying at all! Check out this thread here. This workstation is used for sequencing and assembly of genome: bacteria, plants, animals and etc.

https://www.techpowerup.com/forums/threads/we-need-moar-cores-for-science.240729/




qubit said:


> How much did that RAM cost you?!
> 
> Great machine and awesome use as a genome sequencer. That's for fighting genetic diseases, I take it?



lol I didn't pay for it so I don't know. I just installed the RAM and did some tweaking.
Yes. Studying of different organism's genome. Not just diseases though. It can be used for selective genetic engineering as well.


----------



## hat (May 23, 2018)

@xkm1948 is studying genetic engineering. He also has a picture of a cat in his avatar, and in his signature. Genetic engineering is nothing to be taken lightly, and these facts lead me to assume either he is trying to become a cat, or he already is a cat now, altered by genetic engineering to at least (and maybe beyond) human levels of intelligence and ability. Or maybe he wishes to increase the power of other cats, thereby enabling them to take over the world?


----------



## er557 (May 23, 2018)

xkm1948 said:


> Yeah no. For a 8 core server Dell wanna charge us $5000. Hell no.
> 
> We did all the homework, Threadripper is the best bang for the buck.




Actually the best BANG FOR A BUCK are haswell-ep QS qualification stepping xeons, not any latest tier hardware. But those are hard to come by and not suitable for people not looking for used procs or non oem hardware. But having the turbo frequency of said xeons unlocked and tweaked, they have more productivity than current tier hardware, but again, it's not for everyone.



hat said:


> @xkm1948 is studying genetic engineering. He also has a picture of a cat in his avatar, and in his signature. Genetic engineering is nothing to be taken lightly, and these facts lead me to assume either he is trying to become a cat, or he already is a cat now, altered by genetic engineering to at least (and maybe beyond) human levels of intelligence and ability. Or maybe he wishes to increase the power of other cats, thereby enabling them to take over the world?



AAnd you got a like from remixedcat,  a mixed variety of android and feline


----------



## ChristTheGreat (May 23, 2018)

Nice rig!

Too bad for dell charging you 5000$ for an 8 core server, we used to have a R730 with 2x E5-2690 V3 + 256gb of ram for around 20k (CAD).

But TR FTW!!!!


----------



## Bones (May 24, 2018)

hat said:


> @xkm1948 is studying genetic engineering. He also has a picture of a cat in his avatar, and in his signature. Genetic engineering is nothing to be taken lightly, and these facts lead me to assume either he is trying to become a cat, or he already is a cat now, altered by genetic engineering to at least (and maybe beyond) human levels of intelligence and ability. Or maybe he wishes to increase the power of other cats, thereby enabling them to take over the world?



Could be trying to do something anime-related with it...... I dunno. 

However with all that RAM he's set to do whatever is possible with it.


----------



## er557 (May 24, 2018)

Usually after 64gb or so it's the ram speed and performance that matters more than it's capacity, and with a fast nvme drive can certainly afford spilling over some of it to page file, If the task is large enough


----------



## DeathtoGnomes (May 24, 2018)

No RGB? So disappointing.


----------



## Caring1 (May 24, 2018)

DeathtoGnomes said:


> No RGB? So disappointing.


Those fans, and NZXT CPU cooler light up


----------



## HTC (May 24, 2018)

er557 said:


> Actually the best BANG FOR A BUCK are *haswell-ep QS qualification stepping xeons, not any latest tier hardware*. But those are hard to come by and not suitable for people not looking for used procs or non oem hardware. But having the turbo frequency of said xeons unlocked and tweaked, they have more productivity than current tier hardware, but again, it's not for everyone.



1st, let me just say i have zero idea about the prices for these types of CPUs.

With that disclaimer out of the way, let me tell you why i decided to reply to your post in this thread: meltdown / spectre.

Before these security issues were known, you may have been probably right but now it all depends on the type of workload, and i dunno what type of workload is OP's intended workload.

Meltdown is Intel only while spectre affects pretty much all manufacturers, which means even OP's rig can be affected severely by this.


----------



## R0H1T (May 24, 2018)

er557 said:


> Actually the best BANG FOR A BUCK are haswell-ep QS qualification stepping xeons, not any latest tier hardware. But those are hard to come by and not suitable for people not looking for used procs or non oem hardware. But having the turbo frequency of said xeons unlocked and tweaked, they have more productivity than current tier hardware, but again, it's not for everyone.
> 
> 
> 
> AAnd you got a like from remixedcat,  a mixed variety of android and feline


I don't think QS chips are allowed in "critical" systems. Also TR has better IPC than Haswell & lower power consumption, can use more RAM as well IIRC.


----------



## PerfectWave (May 24, 2018)

Fu ck so sexy!


----------



## er557 (May 24, 2018)

QS chips are exactly oem chips, except they are not marked that way, once booted with microcode they are IDENTICAL in function and features to oem, except way cheaper.
About meltdown/spectre , iirc haswell-ep were not affected, but if they are affected, I installed the ms microcode update and the system is mitigated with ZERO performance penalty.
TR can use more ram? more than 768 GB of xeon? I don't think so. And how many pcie lanes does it have? dual xeons have 80 pcie 3 lanes. And with c states and c1e/speedstep in idle the xeons hardly use any power at all.     What about ram bandwidth then? dual xeon have octal channel ddr4 @100gb/s speed, while TR has only quad


----------



## xkm1948 (May 24, 2018)

No getting used/old system like Haswell-EP is impossible simply due to lack of warranty. All new equipment must have at least 2 years of manufacture warranty, hence why everything must be new components.


----------



## er557 (May 24, 2018)

Of course, I figured as much, although haswell-ep xeons are still available today brand new, albeit cost an arm and a leg. I probably built my system saving 100K $.....


----------



## chaosmassive (May 24, 2018)

3% utilization of memory?
it bizarre and surreal at same time


----------



## er557 (May 24, 2018)

Yeah, 128gb a bit overkill if not on a busy web server


----------



## Aquinus (May 24, 2018)

er557 said:


> Yeah, 128gb a bit overkill if not on a busy web server


...or if you're doing genomic data processing like the OP? His machine isn't loaded but, I'm sure he has workloads that will even take 128GB of memory to its knees.


----------



## er557 (May 24, 2018)

Oh, and 1950x is 180w, how is it more power efficient than 120w xeon/ 18 cores? About max ram , TR can pull 1TB theoretically, while xeon does 768gb, but TR does not support dual cpu, so the ram bandwidth of haswell-ep is better


----------



## dj-electric (May 24, 2018)

I would kill for this amount of RAM
*Juicy*.


----------



## er557 (May 24, 2018)

And you might have to, with today's prices. I simply re-used my kits from x99 and combined them-  4x4gb  +  4x8gb


----------



## xkm1948 (May 24, 2018)

Aquinus said:


> ...or if you're doing genomic data processing like the OP? His machine isn't loaded but, I'm sure he has workloads that will even take 128GB of memory to its knees.




Yep. If situation allows we would love to go load reduced unbuffered ECC to achieve 256GB and etc. 


Here is a RAM requirement chart for genome assembly, using a regular 4 core 8 thread Intel CPU.






Source: https://github.com/fenderglass/Flye/blob/flye/docs/USAGE.md


For comparison, assembly of a E.coli genome on my 6950X@4.2GHz takes about 10min. On the 1950X@4GHz it took about 6 mins.


----------



## Vya Domus (May 24, 2018)

er557 said:


> Oh, and 1950x is 180w, how is it more power efficient than 120w xeon/ 18 cores?



Clock speed and performance.

I am under the impression you simply don't know what power efficiency is. Does the Xeon use less power ? Yes. Is it faster ? Did you factor that in ?

There is clear and distinct difference between power consumption and performance per watt. Performance per watt is the metric that tells you what is more power efficient *not* absolute power consumption not correlated to anything.


----------



## er557 (May 24, 2018)

OK, clock speed and performance, here we go. xeon v3 has 3.8ghz vs. 4.0 ghz on the TR, once turbo patched, the xeon does 3000 cinebench on average, the 2696v3 does more, about same as TR, while the latter sits @180w peak. Dont forget that these 5~% percent differences of performance per watt are on cpus released four years apart. I was talking about bang for buck, the TR alone is 1K$, while two xeons QS can be had for 400$ each, providing almost double the memory bandwidth, all while using lower clocked ram. Octal channel 2133mhz is 100gb/s, TR does about 60gb/s @2666mhz, not to mention the L3 caches speeds, and the combined 80 pcie lanes vs. 64 on the TR. BTW, meltdown /spectre mitigated my cpus have zero performance drop, AMD has yet to patch all the vulnerabilities discovered.

BTW, once paired in tandem the dual xeons trounce both TR and i9-7980xe even overclocked


----------



## Vya Domus (May 24, 2018)

A 4 Ghz 1950X gets around 3400 in Cinebench , you didn't even bother to look that up. And also that Xeon at 3.8 Ghz is never going to stay at that 120W TDP figure , that's the TDP for when the chip runs at a measly 2.0 Ghz. It will probably draw around 200W while still being slower. Power efficiency is better on TR overclocked or not dual socket or not  , with 100 memory channels or not. Case closed.



er557 said:


> I was talking about bang for buck



No you didn't , you are just changing the subject after proven wrong , give it up.


----------



## phanbuey (May 24, 2018)

i just dated myself thinking... man... who still has a radeon 1950x? and i thought it already had 256 MB!?!



need more coffee


----------



## er557 (May 24, 2018)

before coffee you first need to wake up... This is a thread about cpus...

On topic, who cares about power efficiency? what is this, a toyota mini? Call me up when TR gets 5020 cinebench, and that's running complete stock speeds, 100mhz bclk, and no overclocking options whatsoever in bios, my vcore sits at less than 1.0v. You cant compare full fledged server hardware to a consumer chip, even if it's released four years later. And how's the temps on TR when running 4.2ghz? server grade xeons rarely pass 65c. Even in gaming, intel has the better optimization...


----------



## Durvelle27 (May 24, 2018)

Can I have the rig 

Man I’d be giddy to own something like this


----------



## Vya Domus (May 24, 2018)

So...



er557 said:


> Oh, and 1950x is 180w, how is it more power efficient than 120w xeon/ 18 cores?



Then I explained how it is in fact more power efficient , after that you went on about a completely different subject saying that you were referring to bang for buck while you clearly didn't and in fact you tried to argue about it in the very comment you posted.

And now you're saying this :



er557 said:


> who cares about power efficiency?



It's hilarious how you are constantly changing the subject and posting irrelevant information. Stop capping this thread with your dual socket , 768GB octa channel and whatever nonsense you repeated yourself like three times. No one cares,  the guy said this is what he needed and that's all there is to it , he doesn't need to justify anything.


----------



## xkm1948 (May 24, 2018)

In an ideal world where money grows on petri dishes i would buy all thr server grade stuff. Unfortunately i cannot. These are tax payer dollars and we will always buy the best cheapest option.


----------



## er557 (May 24, 2018)

Vya Domus said:


> So...
> Then I explained how it is in fact more power efficient , after that you went on about a completely different subject saying that you were referring to bang for buck while you clearly didn't and in fact you tried to argue about it in the very comment you posted.
> 
> And now you're saying this :...
> It's hilarious how you are constantly changing the subject and posting irrelevant information. Stop capping this thread with your dual socket , 768GB octa channel and whatever nonsense you repeated yourself like three times. No one cares,  the guy said this is what he needed and that's all there is to it , he doesn't need to justify anything.




Look, dude, i never attacked you personally, you didnt need to stoop lower and punch under the belt. This is a hardware forum and we discussed cpus, did i ever say OP needs to justify anything? if no one cares about this topic, why do we post? what do they care about? slandering?
I'm done with this, each user needs to enjoy what his system does for him, otherwise the benchmarks tell the truth.


----------



## Norton (May 24, 2018)

Please take any arguments to PM rather than crapping in the OP's thread. Only public warning- points to follow if you decide to continue


----------

