beinguseless

This is partly a continuation of a previous post which I described my Esxi 3.5u5 I/O problems. Since that post I’ve searched google and read in the vmware community forums for any known performance issues with Raid 1E in a four drive configuration compared to Raid 10, checked if there was a known problem with the controller card itself I’m using (LSI SAS3442E), and also proper disk alignment with virtualization which is something I’ll probably revisit once other matters are taken care of.

A point I didn’t mention in my previous post is that when I first purchased the raid card I checked the firmware and updated it as people should. This was around the first of the year and was version 1.26 of the firmware and version 6.24 of the bios, LSI Logic packages them together for ease of upgrade for the user. Good job to LSI on that one. A couple days ago I went back to their site to check for any updates and I happily saw there was, firmware v1.28.02. I’m not sure of what the new bios version is since I didn’t write it down from the screen after flashing and it’s not readily available from LSI’s support site, but since as I mentioned packaged together it’s in with the firmware download. Anyhow, v1.28.02 came out May 5th and scrapping bottom as to what to do I wasted no time in downloading it and prepared for the update.

I created a standard dos boot disk from windows on one floppy and put the firmware files on another. I don’t have a floppy drive in the server but I do use a handy usb based floppy drive. I plugged it in, updated the boot options in the motherboard bios and it was up and running. It did error a couple times on not finding the ‘command.com’ file on the firmware disk but I negated that by copying the one on the dos boot disk to the c:\ drive that gets created in memory. Then when the error came up I could leave the firmware disk in the drive and reference the file on c:\ drive. A couple adapter/chip selections and confirmations later and the new firmware is loaded.

Upon the server booting I find the nearest file on my windows home server virtual machine that’s somewhat sizable to copy to my local machine just to see if there is any noticeable gain. I find a 1GB file and see the MB/sec in the windows transfer window be about 25-35MB/sec. A smile instantly comes to my face. Checking the network utilization that’s hovering around 45% on a gigE adapter. About half way through the file the speed drops to a piddly 2-3MB/sec. I think it has something to do with my local machine since the hard drive light was still ticking pretty steadily. I cancel the transfer and move onto something a little more scientific.

I booted up a separate vm I use for software testing, which is an XP SP3 x32 build set to use an LSI Logic controller type within the vm settings. I re-run the same tests (again referencing my previous post) and for ease of reference below is the before results, taken from the previous post, and after results which are currently the latest.

Before

After

The first thing the jumps out at me is the ‘Max Throughput 100% Read’ test with how much it dipped. The others are about equal and since I did only run these once (I don’t have time to average out 3 5min tests on each) it could be within the margin of error for these. I do know one thing, and so far within the real world test of transferring files the difference is as clear as day in the direction of being a great improvement.

I’ll give it a little more time and general usage before I fully make up my mind on this but I am very hopeful. I think the next change would be upgrading to ESXi vSphere 4 (aka ESXi 4.0), I think the LSI Logic drivers have used have been updated.

About a year ago I wanted to setup a VMWare ESX 3.0 server to test out their new (at the time) release, unfortunately, I didn’t have the funds needed to setup a true production environment with it. I was able to find a fantastic alternative that is great to learn off of at a much more cost effective price point. It seems that the LSI Logic driver included is compatible with scsi and sata controllers, which is great news for us small folks wanting to check out this virtualization environment.

I won’t get into the details of a step by step setup installation but will jump ahead to the post installation changes needed. I’m probably going to miss or incorrectly state a technical term here or there but if there are any corrections needed or questions let me know.

The hardware (least the parts that matter for this writeup) I ended up using is as follows:
Motherboard: Tyan Transport GX28 (B2881)
Controller: LSI MegaRaid
Hard Drives: Seagate 320GB SATA 2

Keep in mind that the installation of ESX can be on any drive, including IDE, the datastores are what need to be on a supported device such as SAN, iSCSI, etc, or in this case a budget SATA setup. After the initial installation is complete you’ll need to check on and modify a file. It took some rummaging around at the time and after some good old trial and error along with a couple installations I was able to narrow it down to the following steps.

#cat /etc/modules.conf

alias eth0 e100
alias eth1 tg3
alias eth2 tg3
alias scsi_hostadapter megaraid2
alias usb-controller usb-ohci

From the above lines I think I did this to either check that “alias scsi_hostadapter megaraid2″ was there or I added it in.

#lspci | grep LSI

020:0e.0 RAID bus controller: LSI Logic / Symbios Logic: Unknown device 0409 (rev 0a)

This is to find out what the device number is for the controller, in this case it is “0409″.

#cat /etc/vmware/vmware-devices.map | grep LSI

vendor ,0x1000,Symbios,LSI Logic / Symbios Logic
device 0x1000,0x0050,scsi,LSI1064,mptscsi_2xx.o
device 0x1000,0x0054,scsi,LSI1068,mptscsi_2xx.o
device 0x1000,0x0056,scsi,LSI1064E,mptscsi_2xx.o
device 0x1000,0x0058,scsi,LSI1068E,mptscsi_2xx.o
device 0x1000,0x005a,scsi,LSI1066E,mptscsi_2xx.o
device 0x1000,0x005c,scsi,LSI1064A,mptscsi_2xx.o
device 0x1000,0x005e,scsi,LSI1066,mptscsi_2xx.o
device 0x1000,0x0060,scsi,LSI1078,mptscsi_2xx.o
device 0x1000,0x0407,scsi,LSI Logic MegaRAID,megaraid2.o
device 0x1000,0x0408,scsi,LSI Logic MegaRAID,megaraid2.o
device 0x1000,0x0411,scsi,LSI Logic MegaRAID SAS1064R,megaraid_sas.o
device 0x1000,0x1960,scsi,LSI Logic MegaRAID,megaraid2.o
device 0x1000,0x9010,scsi,LSI Logic MegaRAID ,megaraid2.o
device 0x1000,0x9060,scsi,LSI Logic MegaRAID ,megaraid2.o

The spacing in there is exactly how it was returned, without the word wrapping. I’m generally pretty organized with files and code so seeing this I needed to keep myself from fixing it :) Take note of the device line with 0×0408 in it, this will be changed to 0×0409 which we found out from the previous command.

From: device 0x1000,0x0408,scsi,LSI Logic MegaRAID,megaraid2.o
To: device 0x1000,0x0409,scsi,LSI Logic MegaRAID,megaraid2.o


Now that the updated the vmware-devices.map file to see the controller card we’ll need to update ESX and reboot. I’m not fully sure if each of these are needed, I would think that at least the first is, but I have done in before for safe practice to make sure the system is up to date.

#esxcfg-boot -p (reloads PCI data)
#esxcfg-boot -i (reloads initrd information)
#esxcfg-boot -b (sets up boot information)
#reboot

Upon rebooting and logging into the VMware Virtual Infrastructure Client you should be able to access that datastore and begin to create virtual machines. Watch out with keeping snapshots around too long, I talked about this some in a previous post. I now also recall watching the various services starting on boot and that it would begin to fail on a particular one until this was fixed. I didn’t write that down but I’ll try and find out what it is and add it here.