# ESXi 4.0 and Ubuntu LTS VM Raw Drive Mapping/Software Raid



## bpgt64 (May 3, 2010)

Ok, so as the title explains.  My overall goal is map 4 1.5TB Drives to a Ubuntu LTS 10.04 VM, and create a software based raid 5 array and host that to the network inside an ESXi VM.  Any idea if this is possible?


----------



## Easy Rhino (May 3, 2010)

bpgt64 said:


> Ok, so as the title explains.  My overall goal is map 4 1.5TB Drives to a Ubuntu LTS 10.04 VM, and create a software based raid 5 array and host that to the network inside an ESXi VM.  Any idea if this is possible?



you will be able to map those drives to an ubuntu system. from there when you setup your VM software just reference that mount point. i suggest using command line to mount those drives, not the GUI. i am pretty sure there is open source software for software based raid 5 but i have never set that up before. look into cloud software which is what you want to do essentially.


----------



## bpgt64 (May 3, 2010)

Thinking of checking out Open Filer, instead of Ubuntu...

The main thing is making the drives available for Raw Device Mapping.  We'll see how that goes tomorrow...


----------



## Easy Rhino (May 3, 2010)

bpgt64 said:


> Thinking of checking out Open Filer, instead of Ubuntu...



there are pre-made operating systems for what you are looking to do. it would probably be easier!


----------



## bpgt64 (May 4, 2010)

http://www.youtube.com/watch?v=wddY7qCn-ig

For background music...hopefully it works!!!


----------



## Easy Rhino (May 4, 2010)

which version of vmware are you using?


----------



## bpgt64 (May 4, 2010)

ESXi v4.0 Update 1


----------



## Easy Rhino (May 4, 2010)

bpgt64 said:


> ESXi v4.0 Update 1



hahaa. as i posted it i just remembered what you wanted to do. vmware server is also free and is a bit more powerful. you may want to give it a try once you get comfortable with your setup.


----------



## bpgt64 (May 4, 2010)

I dunno, that sounds alot like ESXi, but it runs over a generic OS...where as ESXi is a stripped down version of ESX that is remotely controlled by vSphere.  I am not sure what advantage could be gained from trying VMware Server over ESXi?  ESXi 4.0 is also free, with its limitations..of course.


----------



## Disparia (May 4, 2010)

I don't really think of one as more powerful or not, they're just aimed at different situations.

Server: computer -> host OS -> Server -> VMs.

ESXi: computer -> ESXi -> VMs.

If you, for example, wanted to use ICHxR or any other RAID controller not supported under ESXi, you could set up the array, install the host OS, then install VMware Server and create VMs.

But if this way is working for you, then by all means go ESXi. It's "closer" to the hardware, so performance is a little better and there's no issues when rebooting a host OS for updates and whatnot.


----------



## bpgt64 (May 4, 2010)

Jizzler said:


> I don't really think of one as more powerful or not, they're just aimed at different situations.
> 
> Server: computer -> host OS -> Server -> VMs.
> 
> ...



Great points, certainly something to think about.  I could host the soft raid in Win Server 08 or RHEL.  It would allow me to make the raid normally, and then just host it over the v Switch to the other drives.  Which would be fast enough.  I figure since I have this in place, if I ever need to make a switch, I probably would switch to Vmware Server, but since this is in place....yea.


----------



## Easy Rhino (May 4, 2010)

bpgt64 said:


> Great points, certainly something to think about.  I could host the soft raid in Win Server 08 or RHEL.  It would allow me to make the raid normally, and then just host it over the v Switch to the other drives.  Which would be fast enough.  I figure since I have this in place, if I ever need to make a switch, I probably would switch to Vmware Server, but since this is in place....yea.



ahh, that makes a lot more sense now. do you guys know how much vsphere costs?


----------



## bpgt64 (May 4, 2010)

It's free, it's the ESXi licence that's limited with key.  They limit you to a single CPU, and a certain amount of ram.  Which is no biggie for home testing, but limits you in commercial application.  Just do what I did and sign up at VMware.com, and pick your brand of fancy, ESX/ESXi/ESX server..They give you a free licence to it and Vsphere, but again its limited.


----------



## Easy Rhino (May 4, 2010)

bpgt64 said:


> It's free, it's the ESXi licence that's limited with key.  They limit you to a single CPU, and a certain amount of ram.  Which is no biggie for home testing, but limits you in commercial application.  Just do what I did and sign up at VMware.com, and pick your brand of fancy, ESX/ESXi/ESX server..They give you a free licence to it and Vsphere, but again its limited.



i have been a member with vmware for several years now. i have never tried anything other than vmware server because of how well it runs with ubuntu and that it does not cost me a dime and is not limited. there is a pay version of vsphere which i am curious to know how much it costs.


----------



## Disparia (May 4, 2010)

I was looking at $20K+ to license three 2P servers and with all the 'goodies' a few months ago at my old job.

Wasn't ready to drop that kind of cash and besides, as I was at an SMB, I wanted to wait until desktop solutions matured to really utilize the benefits.


----------



## Easy Rhino (May 4, 2010)

Jizzler said:


> I was looking at $20K+ to license three 2P servers and with all the 'goodies' a few months ago at my old job.
> 
> Wasn't ready to drop that kind of cash and besides, as I was at an SMB, I wanted to wait until desktop solutions matured to really utilize the benefits.



yea, it seems most of the benefit of using that stuff comes when you are a multinational enterprise.


----------



## suraswami (May 4, 2010)

I am working on a similar project at home.  I am using the free VMWare 2.02.  Last night I installed W2K8 Storage Server as a VM and configured it as my iSCSI Target to be used as storage on all VMs.  Once you set it up I think all you need is to use the ip of the Storage Server, Folder, login info and you can use as regular windows storage.  Or you can use Microsoft Initiator software and run on any 2K3 or 2K8 VMs to connect to the iSCSI targets.

After struggling for last 4 days I finally made progress with this setup.  I will see if I can open a separate thread to post my findings.

I initially went with Openfiler in a VM route.  I finished setting up the iSCSI targets but couldn't make a proper connection to those targets.  I will try out this route one more time tonight and see if I can make it work.  This is the simplest and less overhead route.


----------



## Disparia (May 4, 2010)

Easy Rhino said:


> yea, it seems most of the benefit of using that stuff comes when you are a multinational enterprise.



Yup. It's a little more work using Server/ESXi (you do all the backups, transfers of VM's to other machines) but free is free and I didn't have to manage dozens of servers, just a handful.

The idea was to find out how much it would cost to only have three servers in the building and no desktops (all virtualized) with auto backup and failover for about 50 users. Would have been hella sweet, but there was no way we could swing the licensing and PCoIP monitor costs at that time.


----------



## bpgt64 (May 5, 2010)

Well the shit hit the fan, I am going back in with Window's Server 08...the drive finished fine, but the permissions for it...won't work at all.  Meaning Samba doesn't work at all.  Meaning...yea.


----------



## Easy Rhino (May 5, 2010)

bpgt64 said:


> Well the shit hit the fan, I am going back in with Window's Server 08...the drive finished fine, but the permissions for it...won't work at all.  Meaning Samba doesn't work at all.  Meaning...yea.



what samba permissions are you not figuring out? it is pretty easy once you get a hang of it.


----------



## bpgt64 (May 5, 2010)

Sorry if I didn't clarify, the drive isn't acting properly, it won't let me alter any of the permissions on the drive, there's something inherently wrong with the drive(probably because of the way I am trying to configure it).  But trying to access the drive outside of the VM box, isn't working at all.  I have spent too much time on this for the results I am getting.  And thus going back to 08.

I have setup shares in with Samba before, and never had anything like this happen..


----------



## bpgt64 (May 5, 2010)

My boss is convinced that we got that far, he can fix the issue with permissions...so it looks like I am going to rebuild the array tomorrow....*holds shotgun to foot*


----------



## Easy Rhino (May 5, 2010)

bpgt64 said:


> My boss is convinced that we got that far, he can fix the issue with permissions...so it looks like I am going to rebuild the array tomorrow....*holds shotgun to foot*



 yea you can figure it out.


----------



## bpgt64 (May 5, 2010)

That and just to show off, he setup exactly what I was trying to do in his VM environment at work in 20 minutes with much smaller disks...So it does work...just yea.


----------



## Easy Rhino (May 5, 2010)

bpgt64 said:


> That and just to show off, he setup exactly what I was trying to do in his VM environment at work in 20 minutes with much smaller disks...So it does work...just yea.



well good to know it will work. why does your boss want to go with ubuntu over server 2008? i don't mind server 2008 for small businesses but it gets really bogged down after awhile.


----------



## bpgt64 (May 5, 2010)

Well, it's not so much that he prefers ubuntu verses RHEL, or CentOS.  It's just Ubuntu is a good baby step for me into the Linux world.  We don't deal with ALOT of linux environments, but I am studying for my RHEL CE, and I am cutting my teeth on Linux with it.  Also, mainly because of transfer rates.  When you host a share on Window's I barely get 70-90mb/s on a gigabit network sometimes less.  If I host a share on Ubuntu, I get 90-110mb/s pushing the theoretical limit of the transfer.  Which is important to me when syncing two large media shares daily.  15GB files, transfer in seconds, not minutes.  And when you host PS3 Media Center files and use "Loss less" quality setting it saturates that pipe with streaming blu-ray level media, you need that 100mb/s...I guess is what it comes down to.


----------



## Easy Rhino (May 5, 2010)

bpgt64 said:


> Well, it's not so much that he prefers ubuntu verses RHEL, or CentOS.  It's just Ubuntu is a good baby step for me into the Linux world.  We don't deal with ALOT of linux environments, but I am studying for my RHEL CE, and I am cutting my teeth on Linux with it.  Also, mainly because of transfer rates.  When you host a share on Window's I barely get 70-90mb/s on a gigabit network sometimes less.  If I host a share on Ubuntu, I get 90-110mb/s pushing the theoretical limit of the transfer.  Which is important to me when syncing two large media shares daily.  15GB files, transfer in seconds, not minutes.  And when you host PS3 Media Center files and use "Loss less" quality setting it saturates that pipe with streaming blu-ray level media, you need that 100mb/s...I guess is what it comes down to.



those are all very good reasons and hope other people thinking about linux will read what you just wrote and take it into consideration. 

red hat is a tad more involved, but most definitely worth it for the enterprise support. good luck on taking the test. are you in a class at a school or is it online or what?


----------



## bpgt64 (May 5, 2010)

I actually just graduated from Georgia Tech, with a degree in Management, cert in Information Technology Management(but I have worked in Window's Shops doing hardware roll outs and general tech support while in college).  I just got into the industry working with a small Software as a Service firm.  I am just kind of prepping for the test at the moment..going to probably take it in a year or two.


----------



## bpgt64 (May 6, 2010)

It's gonna be a good day...


----------



## suraswami (May 6, 2010)

bpgt64 said:


> It's gonna be a good day...
> 
> http://img.techpowerup.org/100506/Untitled.png



Could you also please share with us on how you managed to configure the share?


----------



## Easy Rhino (May 6, 2010)

bpgt64 said:


> It's gonna be a good day...
> 
> http://img.techpowerup.org/100506/Untitled.png



a good day because you are going to watch cloverfield or because you got the shares working?


----------



## bpgt64 (May 6, 2010)

It's going to be a good day, because I have been slamming my head against this issue for 2 weeks and I am so close to being finished....I'll have a more thurough update later today...busy busy busy....moving into a new place in a month or so, have to finish signing the paperwork today!


----------



## bpgt64 (May 10, 2010)

So here's a run down of how it works.  I am going to make some assumptions, that if your viewing this you've installed ESXi before, or atleast know what it is.  Raw Device mapping doesn't work, unless you have a Raid card recognized by ESXi.  So...when I installed ESXi, I only installed a VM store on my main 2TB drive.  The other four HDD's were configured in IDE mode in Bios with software raid off.  I then went into the Management of ESXi, using Vsphere and made myself 4 new virtual disks.  And then added them to VM I wanted to form the raid.  A few things about this...First of all you will have to change the block size to 2TB/8MB in order to provision a single drive above 256GBs.  Once inside the Ubuntu VM, I configured the drives in a Linux Software raid using 10.04 LTS's HDD management tool.  My boss then showed me how to change the ownership of the raid to everyone, which apparently took some work..

I am still trying to iron out the kinks, the post I made above is legit, it has been pushing that kind of write speed.  But sometimes SMBD works overtime, and sometimes it hardly pulls any CPU usage and I get perfect transfer rates.  So there's still some bugs to be worked out. 

For some reason, Samba has to work overtime in some transfers, and I get really crappy transfer times.  But other times Samba doesn't do anything, and I pull 90mb/s or higher..  I know it's not a limitation of the Drive I am reading the data from...because I have been intentionally using a Western Digital Vrap 6gbps, and it hauls arse on read speed.  There doesn't seem to be reason for samba to have to work overtime in some cases and hardley at all in others.  But when it works overtime(pulling about 2 cores at 100% of 8)...I get crappy read/writes(under 3mb/s)...but there's no rhyme or reason to it...so I am just gonna keep working on it till I figure it out.


----------

