It's been a while since I've had to set up a virtualisation server, but today I needed to configure a brand-new install of CentOS 6 to act as a virtual host. The hardware doesn't have virtualisation support (an old G3) so I had to use Xen so that paravirtualisation was available (not currently supported by KVM). Oops, not so easy now that Xen isn't included by default, Red Hat having opted to use KVM instead.
Despite that, getting things set up isn't that hard, although not nearly as easy as it was when you could just
yum install xen
This documentation details the steps you'll need to follow.
So let's start with a few basic pre-requisites;
yum install sanlock presto libblkid
As helpful as it is for security, it's going to interfere like nobodies business! Either disable it or make permissive
# Change SELINUX=enabled to SELINUX=disabled
This won't be effective until we reboot, but let's get a few bits done first!
Bridge the network
The package should already be installed, but just to be safe, run
yum install bridge-utils
Next we need to configure the bridge
cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-br0
Then we edit the configuration file we just created, so that we get something like
Next we need to edit the original file and add a line at the end, so
#add a line at the bottom
So now that's set up, we can move on and proceed with the install.
Note: I had a few issues with the kernel we'll be downloading in this step, and eventually resorted to compiling and installing the upstream kernel myself. Steps for that are below.
Use wget to install the aussie repo (I'm sure there must be a closer one!). If you don't have wget start with a yum install wget.
yum install http://au1.mirror.crc.id.au/repo/kernel-xen-release-6-3.noarch.rpm
We now need to go off on a slight tangent from the official walkthroughs now, as attempting to install at this point will tell us that qemu-img isn't available. So, we need to install from repoforge as follows
yum install http://pkgs.repoforge.org/qemu/qemu-img-0.15.0-1.el6.rfx.i686.rpm
Then we can proceed to run
yum install kernel-xen xen
To download and build the bits we actually want! Once that's complete there are a few more steps to go through before we're quite ready to reboot.
First we need to reconfigure grub to boot the Xen hypervisor before the kernel
title CentOS (22.214.171.124-2.el6xen.i686)
kernel /vmlinuz-126.96.36.199-2.el6xen.i686 ro root=/dev/mapper/vg_virty-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkerne$
So that kernel becomes module. We also need to add a line to load the xen hypervisor so should end up with something like
title CentOS (188.8.131.52-2.el6xen.i686)
kernel /xen.gz dom0_mem=2048M cpufreq=xen dom0_max_vcpus=2 dom0_vcpus_pin
module /vmlinuz-184.108.40.206-2.el6xen.i686 ro root=/dev/mapper/vg_virty-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkerne$
You'll notice dom0_max_vcpus=2. As the G3 I was building on has 2 cores, I set this to 2. Adjust as necessary for your system (the same goes for dom_0_mem=2048M)
Double check that grub default is set to 0 and then save and close the grub file.
We now need to install libvirt, unfortunately there's a few extra steps as the repo builds no longer include support for Xen, so we need to do the following
yum install libvirt python-virtinst
to install the files from the repo.
Now we'll install a few pre-requisites so that we can rebuild the src rpm's with Xen support
yum groupinstall 'Development Tools'
yum install python-devel xen-devel libxml2-devel xhtml1-dtds readline-devel ncurses-devel libtasn1-devel gnutls-devel augeas libudev-devel libpciaccess-devel yajl-devel sanlock-devel libpcap-devel libnl-devel avahi-devel libselinux-devel cyrus-sasl-devel parted-devel device-mapper-devel numactl-devel libcap-ng-devel netcf-devel libcurl-devel audit-libs-devel systemtap-sdt-devel
The next step is to download a source RPM for libvirt, but first we need to know which version we're running
rpm -qa | grep libvirt
We need version 0.9.10. So, browse to http://vault.centos.org/6.3/os/Source/SPackages/ and you'll find that there's a package available there.
rpm -i libvirt-09.10-21.el6.src.rpm
Now that the RPM is installed we need to do some patching (ignore warnings about user mockbuild not existing).
wget http://pasik.reaktio.net/xen/patches/libvirt-spec-rhel6-enable-xen.patch cd /root/rpmbuild/SPECS
cp -a libvirt.spec libvirt.spec.old
patch -p0 < ~/srcbuild/libvirt-spect-rhel6-enable-xen.patch
We're now ready to build the new RPM,
rpmbuild -bb libvirt.spec
Once this completes you should see a few lines beginning with 'wrote', this will include the path to the new RPM's, probably /root/rpmbuild/RPMS/i686.
rpm -Uvh --force libvirt*
Finally! That's everything installed, we can now reboot
When the server has restarted, you should find that running uname -r reveals a Xen kernel in use. If not, see Troubleshooting below.
Let's start by checking that Xen is actually running
----------- Name ID Mem VCPUs State Time(s)
Domain-0 0 941 4 r----- 68.1
If instead you receive an error stating that logfiles could not be opened, try and write to any file on the filesystem. If you find that the system is read-only then it means something has gone wrong. There could be a range of causes, but begin by checking for hardware issues (especially disk) as this is the primary cause (dmesg and the syslog are your friends). If that doesn't seem to be the problem, see Read-Only filesystem on Dom0 below
Now we just need to build a VM!
As we configured libvirt we'll switch to using that, first lets check it's working OK
Id Name State
0 Domain-0 running
Creating a virtual machine
Libvirt appears to be working, so let's create a LVM based virtual machine using paravirtualisation.
virt-install --prompt --network bridge=br0 --virt-type=xen
Follow the prompts, for the install URL I used http://mirror.ellogroup.com/centos/6.3/os/i386/ but lookup a mirror close to you here (browse to the version you want, then find the folder os/ followed by the relevant architecture). If you receive an error about not being able to retrieve a specific file (in my case vmlinux) just use a different mirror.
See Install Troubleshooting if needed
Install CentOS 6.3
Once the installer starts, you'll be asked whether you want to use the text installer or start a VNC session so you can use the graphical installer. Make the choice that suits you (I went for text out of convenience).
You're on your own here, follow the CentOS documentation if you struggle with the install!
Access the Virtual Host