Setting up an Isolated Driver Domain in Xen

Disclaimer : This article assumes that you have some experience in compiling kernels and are familiar with installing and using Xen.

Xen has a very useful feature of Driver Domains. This post will not talk about the theory, but instead focus on setting up a Driver Domain.
Information about driver domains can be found on the Xen-wiki .

So, as mentioned on the wiki page, we basically need 3 things to setup a driver domain :
1. PCI Passthrough
2. Backend Driver
3. Hotplug scripts

One of the easiest ways to create a Driver Domain is the boot the Dom0 kernel as a DomU.
This article talks about setting up a driver domain for a PCI device. The process described below does not apply for creating a driver domain for block devices.

There are some config options that have to be taken care of.

1. Backend Driver :

The back-end driver for the device should be compiled. In this post, we will be creating a driver domain for a PCI device. Hence the PCI backend driver should be present in the DomU that is to be used as the driver domain.
Since we will be using the same kernel for Dom0 and DomU, we compile the PCI backend driver in the kernel configuration. "pciback" is the PCI backend driver. It can be compiled either as a module or built into the kernel.
I have compiled pciback into the kernel. This makes the process of passthrough extremely easy.




2. PCI Passthrough :
PCI passthrough is a feature that allows DomU to gain direct access to the physical device.
The first thing that has to be done is to compile the Dom0 kernel with PCI Passthrough enabled. This step is optional.

Compile and install the Dom0 kernel.

The first step for acheiving PCI passthrough is to find the PCI ID of the device.
The PCI ID can be found using "lspci". Once the PCI ID has been found out, follow these steps :


If pciback is built into the kernel, then add the following options to the Dom0 kernel command line, in the bootloader's config file :

pciback.permissive pciback.hide=(03:00.0)

Here, 03:00.0 is the PCI ID of the device. ( A NIC on my machine). This hides the device from Dom0. If the device is not hidden from Dom0, then Dom0 initialises the PCI device...which we do not want.

The next step is to modify the DomU's configuration file. Add the following line to expose the PCI device to the DomU that is to e used as the driver domain.

pci = [ '03:00.0' ].

Now, start the DomU. Remember to use the same Dom0 kernel for this DomU.

Once the DomU is booted, issue "lspci" from within the DomU. The PCI device should appear.

If lscpi shows the PCI device, then it has been successfully pass-throughed !

3. Hotplug scripts :

Now, to use the DomU containing the PCI device as a driver domain, we need hotplug scripts. Hotplug scripts allow other DomUs' frontend drivers to attach themselves with the backend.

The easiest way to get the hotplug scripts in the Driver Domain is to get the xen source code and do a "make install-tools" within the Driver Domain.
This should install the required hotplug scripts in the Driver Domain.

This completes the process of setting up a driver domain ! :-)

For a NIC, run the network-bridge script from /etc/xen/scripts to bring up Xen's virtual bridge in the Driver Domain.

Now, to let the other DomUs know about the driver domain, edit the configuration file of the DomU.

vif = [ 'mac=some_mac_id, backend=id_of_the_driver_domain' ]

the "backend" option identifies the Driver Domain for the DomU.
Now boot the other DomUs.
It can be verified that the vifs from the DomUs are connected to the bridge in the Driver domain,
run

$brctl show

in the driver domain.