Ubuntu as your hypervisor

Ubuntu is a free server operating system that is easy to maintain and build on.  I’m a big fan; and recently, I’ve been using it to run our development environments at the office.  If, like us, you’re looking to build a low-cost environment for non-production work, here’s an article that may be a useful start.

Note that many production environments run on KVM – the setup I describe below would need some tweaking, especially from the hardware side, before it would be ready for that…  And while I do talk about production versus staging considerations throughout the article, there are some fundamental aspects that I do not talk about and which should be in the very least touched upon for production — such as setting up your environment with redundant compute nodes and redundant gigabit switches that are separate from your LAN switches, enabling of jumbo frames, disabling of multicasting, use of an iSCSI SAN with snapshotting and replication capability, not to mention your hardware’s scalability — please do therefore be mindful of the fact that, while this article is sufficient for development and testing, you should consider it as incomplete for a production environment.  Ye be warned.

 

Getting Ubuntu up and running
  • To get started, you’ll need Ubuntu Server. If you’re planning to use the server for production, download the latest LTS; otherwise, you can just get the latest version.
  • You’ll need the hardware to run the hypervisor, of course. Make sure that the machine that you use has ample CPU’s (64-bit processors with hypervising instructions), memory, and storage space (RAID-1 15K SAS should be sufficient for a production environment as long are you’ve got a SAN for storage; if your physical host is also supposed to be storing the VM’s, I assume this is a test or dev environment — I’d recommend at least two SATA drives in RAID-0). If you have spare hardware, I would definitely recommend setting up one machine as an iSCSI or NFS SAN instead.
  • Run the Ubuntu Server installation on your compute node. I won’t walk you through the installation, as this is not a KB article on setting up Ubuntu. However, do make sure you at least consider the following:
    • You may wish to set up your storage partitions as LVM so that you can add disks later (that is, if you’re using your compute node as a storage device as well)
    • When prompted for the services to install, you should at least set up openSSH and VM Server services.
  • Once your system is installed, you may wish to set up your public-key authentication. You can find information on how to do this in putty here: http://www.ualberta.ca/CNS/RESEARCH/LinuxClusters/pka-putty.html
  • Make sure that you have all the necessary libraries: apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils openssh-server virt-manager convirt
  • Set up your putty server profile (skip for Linux users):
    • Specify the host address
    • Specify the auto-login username in Connections > Data
    • Enable X11 forwarding in Connections > SSH > X11
    • Specify the private key file to be used in Connections > Auth (if applicable)
    • Make sure you have Xming installed. It needs to be running when you run putty.
  • If you’re using linux, when you connect via SSH be sure to specify the X11 parameters and public key parameters like so:ssh <host> -X -i <private key file>
    Private keys can be generated using ssh-keygen as described here:
    https://help.ubuntu.com/community/SSH/OpenSSH/Keys
  • At this point, your host should be ready to be used. You can create the VM in two ways:
    • run virt-manager and create the machine using the GUI
    • run ubuntu-vm-build with syntax like this:sudo ubuntu-vm-builder kvm hardy –addpkg vim –mem 256 –libvirt qemu:///system
Tools
The tools below are for Windows clients only — they are not needed for linux as the functionality is built-in:
Putty — Windows SSH client with some nifty advantages: you can create server profiles, set them to use public key authentication, enable X11 forwarding and TCP tunneling, all from a GUI.
X-ming — Windows X server that allows you to run Linux graphic applications remotely over SSH. Used with putty, you can run apps such as ghex or gedit from your Windows machine.
The tools  below are for managing the hypervisor. They are Linux applications, which is why you need the above tools if you’re running Windows
virt-manager — GUI interface for creating, starting, stoping or moving VM’s.
virsh — Command-line equivalent of virt-manager. Practical when you just want to start or shutdown a VM.
Useful commands:
virsh list –all → lists all machines running on the host
virsh start <machine name> → starts the machine
virsh shutdown <machine name> → attempts to gracefully shut down a VM
virsh suspend <machine name> → pauses the VM
virsh destroy <machine name> → forces the VM off
virsh can be used to migrate machines live from one host to another. Use this syntax:
virsh migrate –live <name of the machine> qemu+ssh://<destination physical host name>/system
convirt — similar to virt-manager, this GUI tool purportedly allows you to drag & drop VM’s from one server to another. Still under evaluation.
URLs:
Next Steps
Here are a few next steps that you may wish to consider for enhancing your hypervised environment:

Set up NFS4 shares, so that you can share VM’s and migrate them from one compute node to another:  https://help.ubuntu.com/community/NFSv4Howto
Set up a bridge so that your VM’s can use the LAN: https://help.ubuntu.com/8.04/serverguide/C/libvirt.html

NOTE FOR LINUX MACHINES: when cloning a linux machine, don’t forget to delete /etc/udev/rules.d/70-persistent-net.rules: http://muffinresearch.co.uk/archives/2008/07/13/vmware-siocsifaddr-no-such-device-eth0-after-cloning/

>Extracting install files uploaded to Kace

>I’ve been working on Kace more and more recently, and I have come to realise that once you upload a binary file for a managed installation, you can’t download it again… At least, not easily. The following is one possible way for you to extract your binary back out of your Kace K1000 box — practical if you’ve lost or deleted your original file and do not wish to lose your work!

In order to proceed, you need to know a little about XML and how files work. You’re going to be working with a hex editor; if you’re not comfortable with that, you may wish to reconsider undertaking this little manipulation.

First, log into your Kace admin console over the web, then go to Settings > Security Settings. Scroll down to the Samba section and enable file sharing by ticking on the corresponding checkbox, and setting the admin password. Next, go to Settings > Resources > Export K1000 resources. Select your managed install package and under Actions, click on Export to Samba Share. This will effectively export your entire managed installation package to the \k1000clientdrop share.

Kace saves the configuration and binaries in a format that is relatively easy to read — a compressed XML file. It is saved as a file of extension .KPKG; if you rename the file to .ZIP, you can extract the underlying XML file to a location where you can work on it.

As mentioned before, you’ll need a hex editor in order to proceed. When working with Windows  I’ve used Olly even if it’s not really intended as a hex editor. If you’re a Linux buff, ghex is a great little tool, very simple and straightforward. For my experimentation, I went with HxD, which is free and is very much like ghex in terms of its simplicity.

Open up the XML and locate the beginning of your file. This is relatively simple if you’re used to working with raw files; if you’re not, you may find that this site might help you. I suspect that most of your binaries, like mine, will be self-extracting files — in other words, executables — in which case, the file header that you’re looking for is ‘4D 5A’ (that’s “MZ” in ASCII). If you truncate your XML file just before that, you should be good to go!

>”Inventory fun” follow-up… AKA Kace’s built-in service tag and warranty report

>Back in September, I wrote a script to grab Dell machines’ warranty information based on their service tags, which I had retrieved from Kaseya or LANSweeper. A pretty nifty trick, or so I thought…

Since then, I’ve been introduced to Kace — a suite of Dell tools for inventory management, scripting, software and patch deployment, and ghosting. Think of it as a solution that offers the functionality of your FOG server, LANSweeper and Kaseya.

The Kace solution is actually divided in two parts: one component handles inventory, application and update deployment, and scripting, while the other component handles ghosting and driver management. A “component”, in this context, can be a piece of hardware (a physical server that you connect to your LAN – the O/S is a custom Unix distro) or simply a virtual machine (a VMWare application which can run on your existing ESX or ESXi box). The config is rather light — the only thing these devices need is storage space. For those of you that are looking for a free / SOHO-level solution for all your sysadmin  needs: stick to OSCInventory, Zabbix, and FOG… These things have a price tag.  However, though not free, these are well worth it in my opinion.

I digress. I’ve been porting a lot of my existing stuff to Kace recently; this week, I’m working on the inventory report from September. It turns out that my work is pretty much done: perhaps unsurprisingly, Kace already has a report for extracting machine names, service tags and expiry dates. They have two boiler-plate reports, dubbed “Dell Warranty Expired” and “Dell Warranty Expires in the next 60 days”, which dishes out all the information you may need in HTML, CSV or TXT format.

In point of fact, I needed something a bit more customized; I don’t actually need the full warranty information but rather the date the warranty expires. This is because with our clients, the machines are amortized from the moment we get them to the moment they’re no longer covered under warranty.

The nice thing about Kace is that you can take a boiler-plate report, duplicate it and change the SQL request directly, like with LANSweeper. This makes reporting a cinch. Here’s my final report for all machines on my campus:

SELECT DISTINCT M.NAME AS MACHINE_NAME,M.CS_MODEL AS MODEL, DA.SERVICE_TAG, DA.SHIP_DATE, M.USER_LOGGED AS LAST_LOGGED_IN_USER,
DW.END_DATE AS EXPIRATION_DATE
FROM KBSYS.DELL_WARRANTY DW
LEFT JOIN KBSYS.DELL_ASSET DA ON (DW.SERVICE_TAG = DA.SERVICE_TAG)
LEFT JOIN MACHINE M ON (M.BIOS_SERIAL_NUMBER = DA.SERVICE_TAG OR M.BIOS_SERIAL_NUMBER = DA.PARENT_SERVICE_TAG)
WHERE M.CS_MANUFACTURER LIKE ‘%dell%’
AND M.BIOS_SERIAL_NUMBER!=”
AND DA.DISABLED != 1
AND DW.END_DATE = (SELECT MAX(END_DATE) FROM KBSYS.DELL_WARRANTY DW2 WHERE DW2.SERVICE_TAG=DW.SERVICE_TAG AND DW2.SERVICE_LEVEL_CODE=DW.SERVICE_LEVEL_CODE);

It gives you a nice simple list with the machine name, model, service tag, shipment date, warranty expiry date, and the user that last logged on to the system.  Cool eh? Or maybe I’m just easily impressed. Regardless, it saves me time… Yay!