Techtips: Optiplex 745 vs. VMWare ESXi 3.5 Update 3

Optiplex 745 vs. VMWare ESXi 3.5 Update 3

Ok – story of the day is me trying to put ESXi on an Optiplex 745 that I have at work (need to setup a few test machines for various reasons and I’m dying to tinker around with the latest release of ESXi anyway..). So.. I got a nice new Seagate 1 TB drive so I would have some extra space for VMs and a 16 GB Kingston memory stick just in case. My system’s got 4 GB of RAM, and while I know that ESX doesn’t really support SATA that well – I’ve done it before and it normally works fine.

First thing after you get your copy of ESXi burnt to a CD/DVD, stick it in and boot up – you’ll get something like this: “Unable to find a supported device to write the VMware ESX Server 3i 3.5.0 image to”. Strongly reaks of bad support for the motherboard/chipset right?? Google it up and there is plenty of discussion about other losers trying to put an enterprise class virtual solution on to their desktop PC lol. There are several rather convoluted ways to get around the bad support in this version of ESXi for SATA (specifically for the onboard stuff on the Dell Optiplex 745) – mainly involving installing or even booting it from a USB key. A good resource for determining the level of support is going here – it lists the models, controllers, motherboards and how many other geeks have tried successfully or unsuccessfully to get it up and going. I used a combination of methods that I think will satisfy someone that JUST WANTS TO GET THE DAMN THING WORKING (QUICKLY).

Things you will need:

- Dell Optiplex 745
- VMware ESX Server 3i 3.5.0 ISO image
- USB stick with > 300 MB free
- HD with Windows, Linux or a Linux live CD for tinkering with stuff
- the SATA HD you want to install ESX on
- unopened bottle of kentuck straight bourbon

Making the USB bootable installer:
(follow this guide to putting syslinux on to make it bootable or read the coles notes below)

- format your USB stick as FAT32
- download syslinux here: http://www.kernel.org/pub/linux/utils/boot/syslinux/
- uncompress syslinux somewhere
- On Windows run: $syslinuxdirectory$\win32\syslinux.exe -m -a Driveletter: (make sure you put the right one in…)
- On Linux run: syslinux /dev/sdX (sdX being your stick)
- You are now ready for the next step

Getting the ESXi Installer Ready:

- use Winrar or whatever app you want to uncompress the VMware ESX Server 3i 3.5.0 ISO
- copy all of the files from the ISO on to your USB stick that you are working with
- rename the file isolinux.cfg to syslinux.cfg
- Download the following replacement for oem.tgz – the original post is here
- Replace the oem.tgz with the new copy that will recognize your board

Installing ESXi:

- plug the HD you want into install ESXi on to SATA 4. 745’s have 0, 1, 4 and 5 on the board (tested with 0 and 1 and does not work…)
- plug your USB key in
- boot with F12 and select your USB device
- You should now be able to install ESXi on to the drive you want
- It will prompt you to reboot

Crossroads:

When you boot up with your SATA 4 disk you will get a panic message stating it can’t find the boot HD…. !

The long and shorts of this is that if you look at the ISO inside install.tgz under \usr\lib\vmware\installer, you will see a file called
VMware-VMvisor-big-3.5.0_Update_3-123629.i386.dd.bz2 or something like that. This is what is installed on the actual machine and it contains another oem.tgz that is dropped onto the system during installation. So while the installer works, the not-so-good version of this file is what gets installed and is what causes it not to recognize your SATA drive on boot. Nice!! You can verify this by using WinImage or any Linux utility that will look at disk images (dd files) and checking the Hypervisor1 partition.

Here is where you reach a crossroads… If you’ve googled and got a bit stumped or gone down the rabbit hole than I can relate to you. There are several options:

1. Uncompressing the iso, install.tgz and then the VMVisor bz2 and replace the oem.tgz – then pack it all back up neat and tidy in the install.tgz and replace it
2. Boot ESX off a USB key using this article since it will all be in memory anyhow and the VMs will be on your SATA drive.
3. Give up and then drink some bourbon and forget about wasting your life doing geeky shit like installing ESXi on a crappy desktop
4. Follow the next easy easy step and then keep rolling

My next step:

- Shake off the frustration and have a drink, no problems here
- Boot up your Windows HD, Linux HD or Linux live CD (or plop that ESX drive into a system running any OS)
- Mount the ESX drive
- Drop the modified oem.tgz into the Hypervisor1 partition (overwriting what is there)
- Boot up your ESX box on your Optiplex 745

This skips all the mucking around with the installation, saves booting the server from USB, it saves trying to SSH into the ESX server to do it – skips all the headaches that you could go through when all you need to do is replace a simple file and it can even be done in Windows explorer. I’m sure it wasn’t that simple for the first person who created the new sample.map but thats neither here nor there.

Drink and review:

- have a drink
- by following this you’ve hopefully saved yourself a ton of time and can start messing with your ESXi box

Thanks and good luck out there!