# Database Server Build



## Kreij (May 10, 2012)

I need to build a new server for our ERP database system
I good with all of the parts I want but I'm pondering the storage.

The current server uses less than 20GB on the OS drive and less than 40GB on the database drive. Both are RAID1 pairs. DB only has about 10-12 concurrent users/processes running at any given time.

I was pondering going to SSDs for the OS drive and 7200s for the database, but I'm wondering if I might see better performace if I leave the OS on 7200s and put the database on SSDs. The database probably has a 60/40 read/write ratio when you look at usage.

Looking at using a 6-core Xeon with 24GB of RAM.

Thoughts?


----------



## Kreij (May 10, 2012)

Here's what I'm planning ...

Antec 1200 V3 Full Tower case
Corsair Pro AX650 Gold
Still choosing mobo ...
Intel Xeon E5645 6-Core 2.4GHz
Corsair CAFA50 Cooler
G.Skilll Ripjaw DDR3 1333 (24GB ... 6x4GB)
Mushkin Enhanced Chronos 120GB SSD (x2 in RAID1 )
Velociraptor 150GB 10000 HDD (x2 in RAID1 )
Asus EAH5450
LG DVD Burner
AS Ceramique 2


----------



## T4C Fantasy (May 10, 2012)

Kreij said:


> Here's what I'm planning ...
> 
> Antec 1200 V3 Full Tower case
> Corsair Pro AX650 Gold
> ...



if its just the database then that is overkill... unless you plan on using it as a daily pc while running the database lol...., i host a game server with database, a website, with database on a pentium 4 ht 3.4, 2gb ddr1 ram and 2 PATA 500gb hdds xD.


----------



## Frick (May 10, 2012)

T4C Fantasy said:


> if its just the database then that is overkill... unless you plan on using it as a daily pc while running the database lol...., i host a game server with database, a website, with database on a pentium 4 ht 3.4, 2gb ddr1 ram and 2 PATA 500gb hdds xD.



That would depend on the datadase. They can be quite heavy to run, if they're big and there's a lot of operations going on.


----------



## T4C Fantasy (May 10, 2012)

Frick said:


> That would depend on the datadase. They can be quite heavy to run, if they're big and there's a lot of operations going on.



thats absolutely true, however it varies completely if the database holds 50000000 accounts you may need more 

if it holds 25000....


----------



## T4C Fantasy (May 10, 2012)

Kreij said:


> I need to build a new server for our ERP database system
> I good with all of the parts I want but I'm pondering the storage.
> 
> The current server uses less than 20GB on the OS drive and less than 40GB on the database drive. Both are RAID1 pairs. DB only has about 10-12 concurrent users/processes running at any given time.
> ...



we would need more info on exacty what this database  is going to do,  what is the expected user load etc

and what database software you using, Oracle? SQL? Microsoft SQL? Microsoft Access?


----------



## theeldest (May 10, 2012)

Kreij said:


> I need to build a new server for our ERP database system
> I good with all of the parts I want but I'm pondering the storage.
> 
> The current server uses less than 20GB on the OS drive and less than 40GB on the database drive. Both are RAID1 pairs. DB only has about 10-12 concurrent users/processes running at any given time.
> ...



You're really not going to need SSDs for the OS on a server. Once the OS and applications are loaded into memory very little would be read/written on the OS drives. And I can't imagine that you're going to be rebooting this server more than a couple times a year.

SSDs are great in the desktop space because desktop users load many different applications and the loading of apps to memory is what's faster. You won't be doing this. Do standard enterprise class 7200 rpm drives for reliability in RAID1 for the OS.

For the database I'd go 4 to 6 drives in RAID10. 10k drives if possible (or even 15k but those price points start to hurt).

You can use SSDs for the database but you may run into a problem where garbage collection never gets a chance to run depending on the hours this server gets used. If it's business hours then a couple 120GB or 240GB SSDs in RAID1 for a 40GB database will *probably* maintain performance. If you did 60GB drives you'd probably drop to pretty low levels of performance (though, still higher than even 4x 15k drives in RAID10).

On second thought, SSDs for the database--even in the worst case scenario--are probably the best bang / dollar.

*What's your total budget looking like and how 'mission critical' is this server?*

If this is actually important to your business you should really consider a solution with redundant power supplies. Even if you keep a spare PSU on hand, the downtime from doing a swap can be quite long.


----------



## Kreij (May 10, 2012)

A) It's an Oracle database with (as I stated) has about 10-12 concurrent users/processes.
B) The system IS overkill for the database load. Way overkill. That's makes no difference.
C) I don't want other options (like RAID10, etc.). I know what my options are, I just wanted to know about putting the database files on SSDs.

So ...
Is anyone running a database on SSDs?


----------



## theeldest (May 10, 2012)

Here's a pretty fantastic review of a non-SLC SSD in an enterprise usage scenario.
http://www.storagereview.com/intel_ssd_520_enterprise_review

When they thrash a 240GB Intel 520 drive in the database test they manage about 3100 IOPS (67% read, 33% writes; small block random data).

These are 'steady-state' numbers. ie, worst case scenario if the drive can't keep up with garbage collection.

*To give you a point of comparison:*
As you read this, I'm testing a couple 12-disk arrays. (12x 300GB 15k drives in RAID 10). Each provides about 450 IOPS using the exact same test that StorageReview ran.


The SSDs will give you best performance and for your use I very highly doubt you'll have wear issues in the next 3-5 years.


----------



## v12dock (May 10, 2012)

I would just pick something up from dell


----------



## Disparia (May 10, 2012)

v12dock said:


> I would just pick something up from dell



 There's the door ==>


----------



## Kreij (May 10, 2012)

theeldest said:


> The SSDs will give you best performance and for your use I very highly doubt you'll have wear issues in the next 3-5 years.



That was my concern, but after reading quite a bit on it, it doesn't seem to be that big of an issue. With 120GB SSDs at $100 they are a cheap replacement given the speed increase over a 7200 drive.



> There's the door ==>



LOL ... I've nothing against Dell, but when you start customizing things their prices go through the roof in a hurry. I tried to configure my file server on the Dell site and without even getting everything I wanted it was over $6000.
I built it the way I wanted (and added some stuff too) for about $2800


----------



## theeldest (May 10, 2012)

Kreij said:


> That was my concern, but after reading quite a bit on it, it doesn't seem to be that big of an issue. With 120GB SSDs at $100 they are a cheap replacement given the speed increase over a 7200 drive.



Regarding degradation: here's another good review: http://www.storagereview.com/ssd_performance_review_270tb_written

They wrote 270 TERABYTES to an SSD and compared performance before and after.

If you use 15 workers accessing a database 40 hours a week at a sustained rate of 8kB transfers at 100 IOPS and 50/50 read/write we get a rate of 824 GB of data written per week. At that rate it would take you 335 weeks (or 6.4 YEARS) to write that much data to a drive.

And your usage scenario has fewer users, fewer IOPS, and less writes. I wouldn't worry about drive degradation if I were you.


----------



## yogurt_21 (May 10, 2012)

Kreij said:


> That was my concern, but after reading quite a bit on it, it doesn't seem to be that big of an issue. With 120GB SSDs at $100 they are a cheap replacement given the speed increase over a 7200 drive.
> 
> 
> 
> ...



when buying a server from dell, never use the online configuration. Call them, get a rep and tell them what you need/want then ask them for the best price they can give you and emphasize how this will influence your companies future purchases. (also if you're a 5 man crew, say it's 50, etc)

even still for single server applications it doesn't make all that much sense to go with a prebuilt. This is especially true if you already have some stuff laying about.

SSD's should be ok here, especially since you're planning raid 1, just make sure the ssd you select is raid freindly.


----------



## Kreij (May 10, 2012)

I used to buy all Dell's for the shop. Very reliable machines and never had any problems with tech support or warranty issues.

That being said ... it's not as much fun getting a pre-built and just plugging it in as compared to getting a box full of parts and some thermal compound. 

I ordered everything so we'll see what happens. I figure of the SSDs don't play well I'll just get a couple more Raptors to replace them, and put one SSD in the boss's computer and the other in mine.


----------



## Mindweaver (May 10, 2012)

My database is around 70gbs. I have 4x 150 10k drives in RAID 5 _(read 340, write 245)_ on a 3ware 9650SE with battery. My user base on that server is *1*15x users. I've been debating this as well. I bought a RevoDrive 3 x2 PCIe 240gb _(read 1500, write 1225)_ drive for testing.. And I can say WOW!  I write a lot of queries and some that take 5 to 10 mins.. only take around 2-3.. hehehe I would never use this drive as a production drive.. So my question is what is the best Hardware RAID Controller? But to answer your question Kreij.. Yes I see a difference.  For the money I'll be moving over to SSD's, but not sure which controller to use.. 

EDIT: I put 15.. I meant 115... hehehe


----------



## theeldest (May 10, 2012)

Mindweaver said:


> My database is around 70gbs. I have 4x 150 10k drives in RAID 5 _(read 340, write 245)_ on a 3ware 9650SE with battery. My user base on that server is *1*15x users. I've been debating this as well. I bought a RevoDrive 3 x2 PCIe 240gb _(read 1500, write 1225)_ drive for testing.. And I can say WOW!  I write a lot of queries and some that take 5 to 10 mins.. only take around 2-3.. hehehe I would never use this drive as a production drive.. So my question is what is the best Hardware RAID Controller? But to answer your question Kreij.. Yes I see a difference.  For the money I'll be moving over to SSD's, but not sure which controller to use..
> 
> EDIT: I put 15.. I meant 115... hehehe



LSI, Intel, and Areca make some of the best controllers.

I know Albuquerque is running 6x OCZ SSDs on a Highpoint 2720 SGL (though that's RAID0).


----------



## Mindweaver (May 10, 2012)

theeldest said:


> LSI, Intel, and Areca make some of the best controllers.
> 
> I know Albuquerque is running 6x OCZ SSDs on a Highpoint 2720 SGL (though that's RAID0).



Yea my 3ware 9650SE is LSI. 3ware was bought out by LSI. It has been a great card. I'm looking more from a user that has one in a redundant array and can share what model they are using.  Oh and it needs to be a SATAIII card.. Don't really care about a SATAII card.


----------



## Kreij (May 11, 2012)

I ordered a HighPoint RR 640 4xSATA3 card. I'll let you know if it's any good.


----------



## Aquinus (May 11, 2012)

If you do decide to go with SSDs, make sure to go with the enterprise grade SSDs, they're incredibly over-provisioned to help out with reliability if you have the budget to go that route. Keep in mind though that databases can do a lot of small updates which could degrade a sdd pretty quickly, but then again I've never tried to run a production database off of an SSD or SSD RAID.

I don't see this benefitting you if queries to your database are already fast, though because more often than not CPU horse power when joining or doing a seq scan or index scan is what is really going to be your bottleneck and in that case you really would want to opt for fewer cores and higher clocks. The only time I've found 15k RPM SAS drives to improve DB performance over 7200k SATA is on a database copy from template in PostgreSQL, but not on normal queries (except really heavy queries, like 10+ joins on large tables.)

Is SAS an option?


----------



## Kreij (May 11, 2012)

My general thinking ...
The database is in general, "sluggish". It's on an old machine and with each Oracle update it's been progressively getting slower (so more CPU horsepower) and heavy queries can take a long time (so SSDs for fast querying).
There are not many concurrent users, but some of the tables have grown quite large over time, and quite a bit if the querying does heavy joining.
I don't want to purge the information as they want a complete history for reporting certain things.

SAS is an option, but the geek in me wants to see how the SSDs perform on a production database. 
If they work well, I could replace them every year if I want to make sure there is no speed degredation. $200 for a pair od SSDs to hold the DB is nothing, and the old SSDs (that probably would still have a ton of life left) could then be migrated to my gaming rig at home.  lol


----------



## theeldest (May 11, 2012)

Kreij said:


> If they work well, I could replace them every year if I want to make sure there is no speed degredation. $200 for a pair od SSDs to hold the DB is nothing, and the old SSDs (that probably would still have a ton of life left) could then be migrated to my gaming rig at home.  lol



I think this point is pretty key. SSDs have dropped significantly in price over the past year to the point where we can realistically plan on replacing them once a year (or even more) in the enterprise space and the cost is almost negligible.

It does help that most enterprise apps that require very fast disks are usually not space constrained. IE: databases are small compared to the performance they need and what most of us have on our home systems.

Out of curiosity, how big is the company? 10 - 100 employees, 100 - 500, 500 - 1000, 1000+?

Just trying to get an idea of what an "IT Budget" would look like for you and how $200 fits in the bigger picture.


----------



## Kreij (May 11, 2012)

Small company. Maybe 50 employees, but <10 concurrent DB licenses.
I don't have a budget. I just beg for money when I need to do something. lol

When I first started there, they were still running DOS machines on a thinnet coax network.
I completely rewired the shop with cat5E and put in gigabit switches. This is obvious overkill for what they need, but an over-engineered network is a lot less problematic in the long run from an administration standpoint.

For instance, "Is the network bottlenecking?". No, not a single node pushes over 1% bandwidth usage through the switches ... ever ... even the servers. This leads to me only having to make sure the machines are capable of handling their application load(s). So I over-engineer the servers and don't have to worry about that either. Then I sit back and happily enjoy writing code while the network hums along.


----------



## Mindweaver (May 11, 2012)

Kreij said:


> I ordered a HighPoint RR 640 4xSATA3 card. I'll let you know if it's any good.



I'm looking at that card too! Looks really good for $95.99 @ the Egg. I'm also, looking at the Intel RAID Controller RT3WB080. You can buy a BBU _(Battery Backup Unit)_ for it. I'm looking forward to seeing how that HighPoint performs.


----------



## Disparia (May 11, 2012)

Kreij said:


> LOL ... I've nothing against Dell, but when you start customizing things their prices go through the roof in a hurry. I tried to configure my file server on the Dell site and without even getting everything I wanted it was over $6000.
> I built it the way I wanted (and added some stuff too) for about $2800



Yup, every now and then I buy boxes from Dell Outlet when it's the better option... this is not one of those times! $2800 buys one banging box (relative to your needs). Since you've bought your parts already, we can all go back to what we wanted to do earlier in the thread - fantasize about builds 

I'll take the minimalist approach.

Xeon E3-1275 3.4Ghz
Intel DBS1200KP
8GB x 2 1333 ECC
OCZ VeloDrive 300GB (enterprise version of the Revo, allegedly).
Lian-Li PC-Q07.

Leaves about $400 for PSU, cooler, optical if desired.

Has less RAM, less cores, but sports kicked-in-the-groin storage performance. Also, IT WOULD BE SO FRICKIN' CUTE!

Came close to buying that board for one of my own projects, unfortunately it only features 4 SATA ports instead of 6. May have a use for it at a later time though...


----------



## Aquinus (May 11, 2012)

Anandtech has a review on RAID and different drives in RAID, including SATA, SAS, and SSD drives. I really think SAS is the better option. The SSD really only helps on random reads and when you're joining on large tables you're going to be doing very large sequential reads. Also keep in mind TRIM doesn't get passed on SSD raids so you're write speed will suffer over time and on databases that are update/insert/delete heavy you may run into issues. So the heavier the read bandwidth, I would go SSD, the heavier the write, go SAS.

http://www.anandtech.com/show/2739/10


----------



## theeldest (May 12, 2012)

Aquinus said:


> Anandtech has a review on RAID and different drives in RAID, including SATA, SAS, and SSD drives. I really think SAS is the better option. The SSD really only helps on random reads and when you're joining on large tables you're going to be doing very large sequential reads. Also keep in mind TRIM doesn't get passed on SSD raids so you're write speed will suffer over time and on databases that are update/insert/delete heavy you may run into issues. So the heavier the read bandwidth, I would go SSD, the heavier the write, go SAS.
> 
> http://www.anandtech.com/show/2739/10



Hey Aquinus,

Did you check out the review I posted by StorageReview?

They post numbers for an Intel 520 in a 'steady state' environment. IE, worst case scenario when Trim and Garbage Collection don't have time to run.

The numbers that a single SSD is able to provide are quite a bit higher than SAS drives. I ran the exact same tests on a 12 disk array of 15k SAS drives and the array was slaughtered by the SSD.

The SSD is going to give better performanc--hands down--even if Trim and Garbage Collection don't have a chance to run in the background.


Also,
On the review you linked, the very next page gives "Real World Database Results". It shows that a single SSD as the data disk provides 66% higher performance than 8 SAS drives.

... Did you even read the article _you _posted?


----------



## Aquinus (May 12, 2012)

theeldest said:


> Also,
> On the review you linked, the very next page gives "Real World Database Results". It shows that a single SSD as the data disk provides 66% higher performance than 8 SAS drives.
> 
> ... Did you even read the article you posted?



Real world databases aren't going to be doing a ton of random reads and writes. When you're using large tables and joining on large sets of data you're looking at sequential reads more than anything else. Also my concern isn't with the speed of the SAS drives vs SSD, my concern is the reliability of the SSD which becomes a concern in write-heavy scenarios.

Yes, I did read it and it does say SSD is faster for many things (sequential reads/writes are another story), but in mission critical applications you don't want to worry about drives failing and I suspect that a SAS drive will last longer than an SSD while providing more storage.

Kreij: If you went with SSDs, were you planning on going with SLC or some incredibly over-provisioned MLC?


----------



## theeldest (May 12, 2012)

Aquinus said:


> Also my concern isn't with the speed of the SAS drives vs SSD, my concern is the reliability of the SSD which becomes a concern in write-heavy scenarios.



Check this review I posted earlier.



theeldest said:


> Regarding degradation: here's another good review: http://www.storagereview.com/ssd_performance_review_270tb_written
> 
> They wrote 270 TERABYTES to an SSD and compared performance before and after.



Also, a single Intel 520 will give sequential writes around 300 - 500 MB/s . You'll need 4 - 12 SAS drives to give the same performance depending on RAID type. That's a significant difference in cost.

It just makes more sense to use SSDs and replace as necessary. The cost will be lower, they've proven to be reliable under heavy write conditions, and the performance will be higher, even in sequential loads.


And finally, this 'Planned Obsolescence' gives Kreij a little more job security ;-)


----------



## Kreij (May 12, 2012)

Aquinus said:


> Kreij: If you went with SSDs, were you planning on going with SLC or some incredibly over-provisioned MLC?



Neither. The Mushkins Enhanced Chronos' were only $102 for 120GB, so I'm winging it.
I just glanced at the specs and thought, "let's see how it goes."
If it doesn't work out I'll just pick up another pair of 150GB raptors and use those.



theeldest said:


> And finally, this 'Planned Obsolescence' gives Kreij a little more job security ;-)



My title is IT Manager. It's a little misleading as I am the whole IT department. 
I don't worry about job security, never have, never will. There is always work for experienced network people, even moreso if you can write code.


----------



## suraswami (May 12, 2012)

If the database is set right rather designed right with proper indexes then you can even run it in a VM.

I use SQL Server and been a while since touched a Oracle DB server.  my lowest user (10 to 15) base reporting server with nightly updates is running a 1U IBM with a xeon 2.33 ghz quad and 12gb ram and 2 x 1TB WD black enterprise 7200 rpm in RAID 1 (yes OS and everything on one set, bad but that is what I had as budget and option).  The DB size is about 200GB and reporting tables are in the million row range, nightly or monthly about 80 to 90 million rows gets scrubbed.  I designed the tables and indexes in the most optimum way and never the reports seem slowing down even with couple of graphs and charts to render using SSRS.

So I would imagine the 7200 drives would give decent performance if DB is configured right, but the SSDs would do better if it can handle garbage and TRIM properly.

my 2 cents.


----------



## Kreij (May 12, 2012)

I agree on the importance of a good DB design.
In this case, though, it's a 3rd part app and I have no control over how the tables, views, packages, etc. are/were written.


----------



## Mindweaver (May 12, 2012)

Kreij said:


> Neither. The Mushkins Enhanced Chronos' were only $102 for 120GB, so I'm winging it.
> I just glanced at the specs and thought, "let's see how it goes."
> If it doesn't work out I'll just pick up another pair of 150GB raptors and use those.


I've been eyeing those Mushkins.. and for 102 for 120gb the pot just got sweeter... hehehe I rma'ed a few 150gb raptor a few months ago. The new ones I got back are totally redesigned. They are 2.5 notebook drives incased in a 3.5 heatsink.  they look cool and so far they have performed well... But if you ask me I would get 2x of those SSD's and put them in RAID 1 or get 3x and go RAID 1e. I'm using 4x of those raptors in RAID 5 w/ a Hardware RAID controller and read and write is still lower than the SSD's in RAID 1. 



Kreij said:


> My title is *IT Manager*. It's a little misleading as I am the whole IT department.
> I don't worry about job security, never have, never will. There is always work for experienced network people, even moreso if you can write code.



You're in the same boat as me... lol about 6 years ago I had about 15 people that I managed... but I left that company.. tired of traveling.. leaving Monday and not coming home until Friday.  So, now I manage a few different smaller companies, but get to come home every night!  The traveling was fun when I was younger.. hehehe  Suck getting old..lol


----------



## Kreij (May 12, 2012)

I was pondering going with 4x 60GB SSDs and putting them in RAID0+1, but I really don't think I need the speed boost.

I know what you mean MW. I work to live, I don't live to work ... and there is a lot to be said about coming back to your own home after work.


----------



## Mindweaver (May 12, 2012)

Kreij said:


> I was pondering going with 4x 60GB SSDs and putting them in RAID0+1, but I really don't think I need the speed boost.



Yea RAID0+1 would be nice. I would be hard pressed to get the 60gb over the 120gb seeing how the 60gb's are still around 80 bucks and the 120gb's just drop to around 100 bucks. Double the space for quarter of the price. Also, I noticed the 640 RAID controller card you are looking at* 1x to 3x* ports are rated 6gb and the *4x* is rated at 5gb. You shouldn't see a big difference, I just wanted to make sure you seen that buddy. 



Kreij said:


> I know what you mean MW. *I work to live, I don't live to work ... and there is a lot to be said about coming back to your own home after work.*



You said it Brotha!


----------



## yogurt_21 (May 14, 2012)

Kreij said:


> I agree on the importance of a good DB design.
> In this case, though,* it's a 3rd part app and I have no control over how the tables, views, packages, etc. are/were written*.



hence the larger than needed hardware specs I'd imagine. Somthing we've been having to do at my company since I've been here, throw bigger hardware at the issue since the programming isn't getting any better. lol


----------

