Use the Qemu guest agent with Memory Overcommitment Manager

Qemu now has an official guest agent. Programs on a KVM host can now talk to guests using a virtual hardware channel (either virtio-serial or an emulated ISA serial port). Using this mechanism, it is now possible to reliably power down a guest, read and write files, and perform filesystem snapshotting. More information about the guest agent is here.

As of today, Memory Overcommitment Manager (MOM) can use the qemu guest agent for guest statistics collection. This has several key advantages:

  • No need for host->guest network connectivity
  • No need to map guest names to guest IP addresses
  • Fast and secure hardware channel
  • Uses a standard guest agent

Configuring MOM to use the qemu guest agent involves a few simple steps on the host and guest:

Host setup

1. Update MOM

Download the latest version of MOM from github and install it on your host.

2. Reconfigure MOM

  • Remove any existing name-to-ip setting from mom.conf
  • Remove GuestNetworkDaemon from the list of guest collectors (if present)
  • Add GuestQemuAgent to the list of guest collectors

2. Modify your guest domain.xml files to create the communication channels:

Place the following stanza into the devices section of each domain configuration file:

<serial type='unix'>
  <source mode='bind' path='/var/lib/libvirt/qemu/va-$NAME-isa.sock'/>
  <target port='1'/>
<channel type='unix'>
  <source mode='bind' path='/var/lib/libvirt/qemu/va-$NAME-virtio.sock'/>
  <target type='virtio' name='org.qemu.guest_agent.0'/>
  <address type='virtio-serial' controller='0' bus='0' port='1'/>

Replace $NAME with the guest’s name. Redefine/restart the domains to ensure the changes take effect. This will require a VM restart. Note: In this example we create both an ISA-serial channel and a virtio-serial channel. The MOM Collector will automatically connect to whichever of these channels is active. Always specifying both allows you to have a standardized libvirt XML configuration regardless of underlying guest support.

Guest setup

1. Build the guest agent binary on your guest

The guest agent is supported in qemu versions v0.15.0-rc0-413-g957f1f9 or later. Download and build the qemu source on your guest and find the binary ‘qemu-ga’.

2. Configure the guest agent to start automatically

The agent can be started on a guest that supports virtio-serial with no arguments. Follow your distribution’s recommendations to start the agent automatically at boot. If you were previously using the mom-guestd agent, you can use the same process for autostarting qemu-ga.

Testing it out

Assuming the above has gone ok, when you start up MOM on the host and start some guests, the mom log should show messages like “GuestMonitor-$NAME is ready” and your statistics should appear as usual.

Posted in KVM, libvirt, MOM, qemu | 37 Comments

What is the IP address of my guest?

When using libvirt to create virtual machines, I find that networking works pretty well right out of the box. My VMs are connected to one another via a virtual subnet and have outbound access to the internet. But there is one itch that needs scratching. When I am ready to SSH into a VM, what hostname/IP address should I use? That answer is attained only after connecting to the VM with VNC, logging in, and looking at ‘ifconfig’ output. Sure, you could configure your VMs with static IPs, but that’s not a very elegant solution either.

The ideal solution to this problem is to be able to do: ssh <vm name> and have it “just work”. In this article I will show you how to use a libvirt virtual network to do just that.

Libvirt Virtual Networks

Libvirt provides many useful networking functions to domains through the concept of a virtual network. Among the services that can be provided are addressing via DHCP and local DNS, LAN connectivity via Network Address Translation (NAT), and network boot services (TFTP and BOOTP). When you configure your domain with
<interface type='network'> and <source network='default'> you are using the ‘default’ virtual network which provides DHCP and NAT. By creating our own virtual network, we can customize how these services are provided and produce the results we want.

Create your own libvirt network

The first step is to define a new virtual network. To do this, we will create an XML document to describe what we want. Details on the format of this document can be found here.

Let’s begin:

  <bridge name="virbr1" />
  <ip address="" netmask="">
      <range start="" end="" />
      <host mac='de:ad:be:ef:00:01' name='vm-1' ip='' />
      <host mac='de:ad:be:ef:00:02' name='vm-2' ip='' />

On line 2, we define the name of the network. The name is how we select this network in the domain configuration. On line 3, we define the bridge device name that will be used to build the virtual network. Be sure to choose a name that is not already taken (virbr0 is reserved for the ‘default’ network). On line 4, we enable NAT so our VMs can access the internet. The <ip> tag on line 5 is where things get interesting. This is where we configure addressing. Line 5 defines the gateway (or router) for the virtual network. Here we reserve a block of 254 possible addresses all beginning with ‘192.168.123.’. This should be enough but make sure this subnet does not conflict with your existing network configuration. On the next line, the <dhcp> tag enables DHCP services and gives us the opportunity to customize address and hostname assignment. The <range> tag tells the DHCP server which addresses it is allowed to assign to virtual machines.

Immediately following that, are explicit per-host address assignments. You will want one of these lines for each of your VMs. The DHCP server will identify your VMs by mac address. Therefore, make sure that the network config and domain config files agree about these mac addresses. The name tag will create a named entry in the local DNS server for this VM which is very handy. I recommend using your VM name here. ‘.localdomain’ must be appended to produce a fully-qualified domain name. Since these IP addresses are local to the host, use localdomain. Finally, the ip allows you to reserve a specific IP address for the VM.

Once you have made your network config file, save it to /etc/libvirt/qemu/networks and create a symlink in /etc/libvirt/qemu/networks/autostart to have it started automatically. Start up the network with the following:

virsh net-define /etc/libvirt/qemu/networks/mynetwork.xml
virsh net-start mynetwork

If you typed your network xml carefully you should see a message indicating that your network was started successfully.

Modify your libvirt domains

With the custom virtual network created, the last step is to modify your domains to use it. This should be a simple process if your domains were previously using the ‘default’ network. In that case, change:

<source network='default'>
<source network='mynetwork'>

in your domain configuration files. Remember to make sure that the mac address in your network xml matches the domain xml. After making changes, shut down and restart your virtual machines. If you decided to change the mac address in your domain xml, you may need to reconfigure networking inside the VM.

At this point, your efforts should bear fruit. From your host, you should be able to map vm name to ip address automatically (ie. ssh vm-1 or host vm-1). Enjoy!

Posted in libvirt | 2 Comments

Enable python bindings for hello

This is the eighth and final post in the series “HOWTO: Add libvirt support for a qemu command”

One more to go. This time we will enable our command in the libvirt python bindings. These bindings are auto-generated but (due to the virHelloParams structure, we must hand code an implementation. While not that much code, this part can be tricky because the API generator doesn’t seem to be documented. Also, you will need to learn how to work with python objects in C.

diff --git a/python/ b/python/
index 569f4a8..5b830bc 100755
--- a/python/
+++ b/python/
@@ -166,7 +166,6 @@ def enum(type, name, value):
 functions_failed = []
 functions_skipped = [
-    "virDomainHello",
 skipped_modules = {
@@ -181,7 +180,6 @@ skipped_types = {
      'virConnectDomainEventIOErrorCallback': "No function types in python",
      'virConnectDomainEventGraphicsCallback': "No function types in python",
      'virEventAddHandleFunc': "No function types in python",
-     'virHelloParamsPtr': "Not implemented yet",

First, undo the compile fixes we made at the start of this patch series. Removing these lines tells the generator not to skip our function anymore.

@@ -257,6 +255,7 @@ py_types = {
     'const virDomainSnapshotPtr':  ('O', "virDomainSnapshot", "virDomainSnapshotPtr", "virDomainSnapshotPtr"),
     'virDomainSnapshot *':  ('O', "virDomainSnapshot", "virDomainSnapshotPtr", "virDomainSnapshotPtr"),
     'const virDomainSnapshot *':  ('O', "virDomainSnapshot", "virDomainSnapshotPtr", "virDomainSnapshotPtr"),
+    'virHelloParamsPtr': ('O', "virHelloParams", "virHelloParamsPtr", "virHelloParamsPtr"),
 py_return_types = {

Tell the generator about the virHelloParamsPtr argument type. We will treat this C structure as a python object (actually a dictionary — read on).

@@ -343,6 +342,7 @@ skip_impl = (
+    'virDomainHello',

This tells the generator that we will supply a hand-coded implementation of virDomainHello.

diff --git a/python/libvirt-override-api.xml b/python/libvirt-override-api.xml
index 54deeb5..4d1ea7d 100644
--- a/python/libvirt-override-api.xml
+++ b/python/libvirt-override-api.xml
@@ -308,5 +308,11 @@
       <arg name='flags' type='unsigned int' info='flags, curently unused'/>
       <return type='int' info="0 on success, -1 on error"/>
+    <function name='virDomainHello' file='python'>
+      <info>Say hallo to a domain</info>
+      <arg name='domain' type='virDomainPtr' info='pointer to the domain'/>
+      <arg name='params' type='virHelloParamsPtr' info='A dictionary of parameters' />
+      <return type='str *' info='A Greeting from the domain' />
+    </function>

Manually document the API

diff --git a/python/libvirt-override.c b/python/libvirt-override.c
index 4a9b432..20df116 100644
--- a/python/libvirt-override.c
+++ b/python/libvirt-override.c
@@ -2350,6 +2350,42 @@ libvirt_virDomainGetJobInfo(PyObject *self ATTRIBUTE_UNUSED, PyObject *args) {
+static PyObject *
+libvirt_virDomainHello(PyObject *self ATTRIBUTE_UNUSED,
+                       PyObject *args) {
+    virDomainPtr domain;
+    struct _virHelloParams params;
+    PyObject *pyobj_domain, *pyobj_params, *pyobj;
+    char *c_retval;
+    PyObject *ret;
+    if (!PyArg_ParseTuple(args, (char *)"OO:virDomainHello",
+                          &pyobj_domain, &pyobj_params))
+        return(NULL);
+    domain = (virDomainPtr) PyvirDomain_Get(pyobj_domain);
+    pyobj = PyDict_GetItemString(pyobj_params, "exclaim");
+    if (!pyobj)
+        params.exclaim = 0;
+    else
+        params.exclaim = (int) PyInt_AS_LONG(pyobj);
+    pyobj = PyDict_GetItemString(pyobj_params, "lang");
+    if (!pyobj)
+        params.lang = NULL;
+    else
+        params.lang = PyString_AsString(pyobj);
+    c_retval = virDomainHello(domain, &params);
+    if (c_retval == NULL)
+        return VIR_PY_NONE;
+    ret = PyString_FromString(c_retval);
+    free(c_retval);
+    return ret;
  * Helper functions to avoid importing modules
@@ -3585,6 +3621,7 @@ static PyMethodDef libvirtMethods[] = {
     {(char *) "virDomainGetJobInfo", libvirt_virDomainGetJobInfo, METH_VARARGS, NULL},
     {(char *) "virDomainSnapshotListNames", libvirt_virDomainSnapshotListNames, METH_VARARGS, NULL},
     {(char *) "virDomainRevertToSnapshot", libvirt_virDomainRevertToSnapshot, METH_VARARGS, NULL},
+    {(char *) "virDomainHello", libvirt_virDomainHello, METH_VARARGS, NULL},
     {NULL, NULL, 0, NULL}

Here lies the meat of today’s exercise. Above we supply a python-to-c wrapper function. According to our oft-repeated formula we: unpack arguments, make the C API call, and package the return value. The python representation of virHelloParams is a dictionary containing the keys lang and exclaim. Both keys are optional. If supplied, lang is a string and exclaim is a boolean.

Well that’s it! You’re done with “Hello World” for libvirt. Thanks for following my tutorial. I welcome your comments on anything (the tutorial itself but especially the process for adding new functionality to libvirt). If my opinion hasn’t come through in the tutorial, I think there is room for simplification 🙂

Posted in Uncategorized | Leave a comment

Create hello virsh command

This is the seventh post in the series “HOWTO: Add libvirt support for a qemu command”

Phew, we are getting near the end! In contrast to the remote support, this part will seem easy. We have only one file to change. In that file, we define the command and some metadata. Then we provide an API wrapper function that parses virsh command line arguments, makes the API call, and outputs the result to the user. Find the patch here.

diff --git a/tools/virsh.c b/tools/virsh.c
index 50ca50f..254a2bd 100644
--- a/tools/virsh.c
+++ b/tools/virsh.c
@@ -3987,6 +3987,59 @@ done:
+ * "hello" command
+ */
+static const vshCmdInfo info_domhello[] = {
+    {"help", N_("say hello to a domain")},
+    {"desc", N_("Qemu demonstration command -- does nothing useful .")},
+    {NULL, NULL}
+static const vshCmdOptDef opts_domhello[] = {
+    {"domain", VSH_OT_DATA, VSH_OFLAG_REQ, gettext_noop("domain name, id or uuid")},
+    {"exclaim", VSH_OT_BOOL, 0, N_("Get a more excited greeting") },
+    {"lang", VSH_OT_DATA, VSH_OFLAG_NONE, N_("Select desired language (english or spanish)")},
+    { NULL, 0, 0, NULL },

First create two structures that define the virsh command. info_domhello defines some help text that will get displayed by the help command. opts_domhello defines the virsh options for our command. Each entry sets: the option name, the type, is whether the parameter is optional, and the help text.

+static int
+cmdDomHello(vshControl *ctl, const vshCmd *cmd)
+    virDomainPtr dom;
+    const char *name;
+    struct _virHelloParams params;
+    int exclaim = 0;
+    const char *lang;
+    char *response;
+    if (!vshConnectionUsability(ctl, ctl->conn))
+        return FALSE;
+    if (!(dom = vshCommandOptDomain(ctl, cmd, &name)))
+        return FALSE;
+    if (vshCommandOptBool(cmd, "exclaim"))
+        exclaim = 1;
+    if (vshCommandOptString(cmd, "lang", &lang) < 0)
+        lang = NULL;
+    params.lang = (char *) lang;
+    params.exclaim = exclaim;
+    response = virDomainHello(dom, &params);
+    if (response) {
+        vshPrint(ctl, "%s", response);
+        VIR_FREE(response);
+    } else {
+        virDomainFree(dom);
+        return FALSE;
+    }
+    virDomainFree(dom);
+    return TRUE;
  * "net-autostart" command
 static const vshCmdInfo info_network_autostart[] = {
@@ -10504,6 +10557,7 @@ static const vshCmdDef domMonitoringCmds[] = {
     {"dommemstat", cmdDomMemStat, opts_dommemstat, info_dommemstat},
     {"domstate", cmdDomstate, opts_domstate, info_domstate},
     {"list", cmdList, opts_list, info_list},
+    {"domhello", cmdDomHello, opts_domhello, info_domhello},

By now, for structure of this function should be looking very familiar. It starts by testing the connection and domain for validity. Next, the exclaim and lang arguments are extracted if present and the params structure is built. Next, the API call is made. Either the response or error message is displayed and cleanup is done.

Up next… Enable python bindings for helloThe last one!

Posted in Uncategorized | Leave a comment

Add hello remote transport support

This is the sixth post in the series “HOWTO: Add libvirt support for a qemu command”

One great thing about libvirt is its ability to work with remote hosts just as easily as the local system on which it is installed. Of course this feature doesn’t come for free. Every new libvirt API requires an XDR implementation to transmit function calls and their results over the wire. Note: virsh always uses the remote driver to execute commands so this functionality must be in place for virsh to work even for local operations. I would regard this part to be the most difficult task when adding a new API to libvirt. Unless you are familiar with XDR, there is quite a bit going on and not much documentation to explain it.

This patch provides an XDR protocol specification which is used to generate several XDR-specific stubs. Additionally, a client and server implementation are provided. See the unadultered patch here.

    hello: Add hello remote transport support
    * src/remote/remote_protocol.x: provide defines for the new entry point
    * src/remote/remote_driver.c daemon/remote.c: implement the client and
      server side
        daemon/remote_dispatch_args.h daemon/remote_dispatch_prototypes.h
        daemon/remote_dispatch_ret.h daemon/remote_dispatch_table.h
        src/remote/remote_protocol.c src/remote/remote_protocol.h

When adding remote support for an API it is only necessary to change three files. The others are generated automatically from remote_protocol.x during the build process. When submitting patches, include the changes to the generated files. I will make one additional point regarding XDR generation. I have found it helpful to manually regenerate the files whenever I modify remote_protocol.x. To do this, execute the following command from the top of the libvirt source tree:

make -C src rpcgen

Onward and upward!

diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x
index 7310689..54346cd 100644
--- a/src/remote/remote_protocol.x
+++ b/src/remote/remote_protocol.x
@@ -1926,6 +1926,20 @@ struct remote_domain_open_console_args {
     unsigned int flags;
+struct remote_domain_hello_params {
+    remote_string lang;
+    int exclaim;
+struct remote_domain_hello_args {
+    remote_nonnull_domain domain;
+    remote_domain_hello_params params;
+struct remote_domain_hello_ret {
+    remote_string response;
 /*----- Protocol. -----*/
 /* Define the program number, protocol version and procedure numbers here. */
@@ -2159,7 +2173,8 @@ enum remote_procedure {
      * Notice how the entries are grouped in sets of 10 ?

The first step when adding remote support should be to define the wire protocol for your API. You must define the format of the arguments and return value (including explicit format definitions for any structures that are to be passed or returned). For virDomainHello, the arguments include a domain and a structure. The return is a string that can be NULL. The format of this file is C-like but not strict C. There should be enough examples in the file to figure out how to express the various C types.

After defining the types, allocate the next available RPC number for this API. The RPC number is used to figure out if both the client and server support a specific function.

diff --git a/daemon/remote.c b/daemon/remote.c
index d49e0d8..9be77ad 100644
--- a/daemon/remote.c
+++ b/daemon/remote.c
@@ -6995,6 +6995,51 @@ remoteDispatchDomainEventsDeregisterAny (struct qemud_server *server ATTRIBUTE_U
+static int
+remoteDispatchDomainHello (struct qemud_server *server ATTRIBUTE_UNUSED,
+                           struct qemud_client *client ATTRIBUTE_UNUSED,
+                           virConnectPtr conn,
+                           remote_message_header *hdr ATTRIBUTE_UNUSED,
+                           remote_error *rerr,
+                           remote_domain_hello_args *args,
+                           remote_domain_hello_ret *ret)
+    virDomainPtr dom;
+    struct _virHelloParams params;
+    char **response, *ret_str;
+    int rc = 0;
+    dom = get_nonnull_domain (conn, args->domain);
+    if (dom == NULL) {
+        remoteDispatchConnError(rerr, conn);
+        return -1;
+    }
+    /* XDR passes lang as a pointer to a pointer ... */
+    params.lang = args->params.lang == NULL ? NULL : *args->params.lang;
+    params.exclaim = (args->params.exclaim != 0);        
+    /* ... and the same thing for the return string */
+    if (VIR_ALLOC(response) < 0) {
+        remoteDispatchOOMError(rerr);
+        virDomainFree (dom);
+        return -1;
+    }
+    ret_str = virDomainHello(dom, &params);
+    if (!ret_str) {
+        remoteDispatchConnError(rerr, conn);
+        VIR_FREE(response);
+        rc = -1;
+    } else {
+        *response = ret_str;
+        ret->response = response;
+    }
+    virDomainFree (dom);
+    return rc;
 static int
 remoteDispatchNwfilterLookupByName (struct qemud_server *server ATTRIBUTE_UNUSED,

The function above implements the server side of the remote function call. This code runs within libvirtd on the “remote” host and calls down into the qemu driver on behalf of a client that may be running on another host. The pointer arguments args and ret contain encoded data representing the arguments to virDomainHello and a place to store the return value. Regardless of the API being called, this function returns an integer to indicate success or failure at the RPC level.

The first step is to convert args->dom into a valid local domain pointer. Next we build a local virHelloParams structure from the rest of the arguments. Something I found very confusing (mostly due to lack of documentation) is that XDR passes strings as char** pointers. That explains the strange way of accessing args.params->lang. Likewise to return a string we must allocate response a char** pointer. Once this is done we can make the actual API call on behalf of the client. If successful, the returned string is packed into response and then packed into the XDR return parameter. Finally, the domain pointer must be freed.

diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index 87a68a6..75c655a 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -9632,6 +9632,33 @@ done:
+static char *
+remoteDomainHello(virDomainPtr dom, virHelloParamsPtr params)
+    char *rv = NULL;
+    remote_domain_hello_args args;
+    remote_domain_hello_ret ret;
+    struct private_data *priv = dom->conn->privateData;
+    remoteDriverLock(priv);
+    make_nonnull_domain (&args.domain, dom);
+    args.params.lang = params->lang == NULL ? NULL : (char **) &params->lang;
+    args.params.exclaim = params->exclaim;
+    memset(&ret, 0, sizeof ret);
+    if (call (dom->conn, priv, 0, REMOTE_PROC_DOMAIN_HELLO,
+              (xdrproc_t) xdr_remote_domain_hello_args, (char *) &args,
+              (xdrproc_t) xdr_remote_domain_hello_ret, (char *) &ret) == -1)
+        goto done;
+    /* Caller frees this. */
+    rv = *ret.response == NULL ? NULL : *ret.response;
+    remoteDriverUnlock(priv);
+    return rv;
@@ -11202,7 +11229,7 @@ static virDriver remote_driver = {
     remoteDomainSnapshotDelete, /* domainSnapshotDelete */
     remoteQemuDomainMonitorCommand, /* qemuDomainMonitorCommand */
     remoteDomainOpenConsole, /* domainOpenConsole */
-    NULL, /* domainHello */
+    remoteDomainHello, /* domainHello */
 static virNetworkDriver network_driver = {

Here is the implementation of the client side which is organized as a driver (just like qemu, xen, etc). I followed the example set by other functions in the remote driver to handle driver locking. With that taken care of, the structures for XDR arguments and return value are prepared. The domain pointer is encoded using a helper function. As stated previously, XDR requires character pointers to be passed by reference. Next we make the actual RPC call. REMOTE_PROC_DOMAIN_HELLO refers to the RPC number that we specified in the protocol definition. The prepared args and ret structures are passed along wih their types. After the call, we unpack the returned string, clean up and then return our result string. By convention, we own the reference on the returned string so we do not need to duplicate it. The caller will free it.

diff --git a/daemon/remote_dispatch_args.h b/daemon/remote_dispatch_args.h
index 15fa1a0..cb6091a 100644
--- a/daemon/remote_dispatch_args.h
+++ b/daemon/remote_dispatch_args.h
@@ -176,3 +176,4 @@
     remote_domain_set_blkio_parameters_args val_remote_domain_set_blkio_parameters_args;
     remote_domain_get_blkio_parameters_args val_remote_domain_get_blkio_parameters_args;
     remote_domain_migrate_set_max_speed_args val_remote_domain_migrate_set_max_speed_args;
+    remote_domain_hello_args val_remote_domain_hello_args;
diff --git a/daemon/remote_dispatch_prototypes.h b/daemon/remote_dispatch_prototypes.h
index 3fcf87c..5f2264f 100644
--- a/daemon/remote_dispatch_prototypes.h
+++ b/daemon/remote_dispatch_prototypes.h
@@ -338,6 +338,14 @@ static int remoteDispatchDomainHasManagedSaveImage(
     remote_error *err,
     remote_domain_has_managed_save_image_args *args,
     remote_domain_has_managed_save_image_ret *ret);
+static int remoteDispatchDomainHello(
+    struct qemud_server *server,
+    struct qemud_client *client,
+    virConnectPtr conn,
+    remote_message_header *hdr,
+    remote_error *err,
+    remote_domain_hello_args *args,
+    remote_domain_hello_ret *ret);
 static int remoteDispatchDomainInterfaceStats(
     struct qemud_server *server,
     struct qemud_client *client,
diff --git a/daemon/remote_dispatch_ret.h b/daemon/remote_dispatch_ret.h
index 114e832..cdc9c89 100644
--- a/daemon/remote_dispatch_ret.h
+++ b/daemon/remote_dispatch_ret.h
@@ -140,3 +140,4 @@
     remote_domain_is_updated_ret val_remote_domain_is_updated_ret;
     remote_get_sysinfo_ret val_remote_get_sysinfo_ret;
     remote_domain_get_blkio_parameters_ret val_remote_domain_get_blkio_parameters_ret;
+    remote_domain_hello_ret val_remote_domain_hello_ret;
diff --git a/daemon/remote_dispatch_table.h b/daemon/remote_dispatch_table.h
index c5f6653..fb2c074 100644
--- a/daemon/remote_dispatch_table.h
+++ b/daemon/remote_dispatch_table.h
@@ -1042,3 +1042,8 @@
     .args_filter = (xdrproc_t) xdr_remote_domain_migrate_set_max_speed_args,
     .ret_filter = (xdrproc_t) xdr_void,
+{   /* DomainHello => 208 */
+    .fn = (dispatch_fn) remoteDispatchDomainHello,
+    .args_filter = (xdrproc_t) xdr_remote_domain_hello_args,
+    .ret_filter = (xdrproc_t) xdr_remote_domain_hello_ret,

diff --git a/src/remote/remote_protocol.c b/src/remote/remote_protocol.c
index 7ecea9d..ba31d50 100644
--- a/src/remote/remote_protocol.c
+++ b/src/remote/remote_protocol.c
@@ -3872,6 +3872,37 @@ xdr_remote_domain_open_console_args (XDR *xdrs, remote_domain_open_console_args
+xdr_remote_domain_hello_params (XDR *xdrs, remote_domain_hello_params *objp)
+         if (!xdr_remote_string (xdrs, &objp->lang))
+                 return FALSE;
+         if (!xdr_int (xdrs, &objp->exclaim))
+                 return FALSE;
+        return TRUE;
+xdr_remote_domain_hello_args (XDR *xdrs, remote_domain_hello_args *objp)
+         if (!xdr_remote_nonnull_domain (xdrs, &objp->domain))
+                 return FALSE;
+         if (!xdr_remote_domain_hello_params (xdrs, &objp->params))
+                 return FALSE;
+        return TRUE;
+xdr_remote_domain_hello_ret (XDR *xdrs, remote_domain_hello_ret *objp)
+         if (!xdr_remote_string (xdrs, &objp->response))
+                 return FALSE;
+        return TRUE;
 xdr_remote_procedure (XDR *xdrs, remote_procedure *objp)
diff --git a/src/remote/remote_protocol.h b/src/remote/remote_protocol.h
index 87de0da..318fa34 100644
--- a/src/remote/remote_protocol.h
+++ b/src/remote/remote_protocol.h
@@ -2184,6 +2184,23 @@ struct remote_domain_open_console_args {
         u_int flags;
 typedef struct remote_domain_open_console_args remote_domain_open_console_args;
+struct remote_domain_hello_params {
+        remote_string lang;
+        int exclaim;
+typedef struct remote_domain_hello_params remote_domain_hello_params;
+struct remote_domain_hello_args {
+        remote_nonnull_domain domain;
+        remote_domain_hello_params params;
+typedef struct remote_domain_hello_args remote_domain_hello_args;
+struct remote_domain_hello_ret {
+        remote_string response;
+typedef struct remote_domain_hello_ret remote_domain_hello_ret;
 #define REMOTE_PROGRAM 0x20008086
@@ -2395,6 +2412,7 @@ enum remote_procedure {
 typedef enum remote_procedure remote_procedure;
@@ -2776,6 +2794,9 @@ extern  bool_t xdr_remote_domain_snapshot_current_ret (XDR *, remote_domain_snap
 extern  bool_t xdr_remote_domain_revert_to_snapshot_args (XDR *, remote_domain_revert_to_snapshot_args*);
 extern  bool_t xdr_remote_domain_snapshot_delete_args (XDR *, remote_domain_snapshot_delete_args*);
 extern  bool_t xdr_remote_domain_open_console_args (XDR *, remote_domain_open_console_args*);
+extern  bool_t xdr_remote_domain_hello_params (XDR *, remote_domain_hello_params*);
+extern  bool_t xdr_remote_domain_hello_args (XDR *, remote_domain_hello_args*);
+extern  bool_t xdr_remote_domain_hello_ret (XDR *, remote_domain_hello_ret*);
 extern  bool_t xdr_remote_procedure (XDR *, remote_procedure*);
 extern  bool_t xdr_remote_message_type (XDR *, remote_message_type*);
 extern  bool_t xdr_remote_message_status (XDR *, remote_message_status*);
@@ -3131,6 +3152,9 @@ extern bool_t xdr_remote_domain_snapshot_current_ret ();
 extern bool_t xdr_remote_domain_revert_to_snapshot_args ();
 extern bool_t xdr_remote_domain_snapshot_delete_args ();
 extern bool_t xdr_remote_domain_open_console_args ();
+extern bool_t xdr_remote_domain_hello_params ();
+extern bool_t xdr_remote_domain_hello_args ();
+extern bool_t xdr_remote_domain_hello_ret ();
 extern bool_t xdr_remote_procedure ();
 extern bool_t xdr_remote_message_type ();
 extern bool_t xdr_remote_message_status ();

The rest of this is auto-generated by rpcgen. Therefore, I don’t see a need to discuss it any further.

Up next… Create hello virsh command

Posted in Uncategorized | Leave a comment

Implement the qemu version of hello

This is the fifth post in the series “HOWTO: Add libvirt support for a qemu command”

Now begins the fun part where we can actually interact with qemu to implement the command. In this patch we first create a virDrvDomainHello function in the qemu driver. This function locates the domain we are targetting, issues a monitor command, and returns the result. We provide support for issuing the command using the qmp and hmp monitors.

Find the patch here.

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 40bb4c0..de6c719 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -7079,6 +7079,50 @@ cleanup:
     return ret;
+static char *
+qemuDomainHello(virDomainPtr dom,
+                virHelloParamsPtr params)
+    struct qemud_driver *driver = dom->conn->privateData;
+    virDomainObjPtr vm = NULL;
+    char *greeting = NULL;
+    qemuDriverLock(driver);
+    vm = virDomainFindByUUID(&driver->domains, dom->uuid);
+    qemuDriverUnlock(driver);
+    if (!vm) {
+        char uuidstr[VIR_UUID_STRING_BUFLEN];
+        virUUIDFormat(dom->uuid, uuidstr);
+        qemuReportError(VIR_ERR_NO_DOMAIN,
+                        _("no domain with matching uuid '%s'"), uuidstr);
+        goto cleanup;
+    }
+    if (qemuDomainObjBeginJob(vm) < 0)
+        goto cleanup;
+    if (virDomainObjIsActive(vm)) {
+        qemuDomainObjPrivatePtr priv = vm->privateData;
+        qemuDomainObjEnterMonitor(vm);
+        greeting = qemuMonitorHello(priv->mon, params);
+        if (!greeting)
+            qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+                            _("monitor command failed"));
+        qemuDomainObjExitMonitor(vm);
+    } else {
+        qemuReportError(VIR_ERR_OPERATION_INVALID,
+                        "%s", _("domain is not running"));
+    }
+    if (qemuDomainObjEndJob(vm) == 0)
+        vm = NULL;
+    if (vm)
+        virDomainObjUnlock(vm);
+    return greeting;
 static virDriver qemuDriver = {
@@ -7191,7 +7235,7 @@ static virDriver qemuDriver = {
     qemuDomainSnapshotDelete, /* domainSnapshotDelete */
     qemuDomainMonitorCommand, /* qemuDomainMonitorCommand */
     qemuDomainOpenConsole, /* domainOpenConsole */
-    NULL, /* domainHello */
+    qemuDomainHello, /* domainHello */

Above is the top-level implementation for the qemu driver. The first thing we do is find the vm object corresponding to the domain passed in by performing a UUID lookup. Be careful to follow driver locking rules. Just follow the many existing examples in order to get this right. In order to issue commands to a qemu monitor, you must enter a job context. This is another form of locking to make sure that commands are executed synchronously. Again, look to other commands for examples of correct usage. Once in job context, we call into the monitor code to perform the monitor command. Let’s look at that part next.

diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c
index 800f744..46b89ae 100644
--- a/src/qemu/qemu_monitor.c
+++ b/src/qemu/qemu_monitor.c
@@ -2189,6 +2189,24 @@ int qemuMonitorDeleteSnapshot(qemuMonitorPtr mon, const char *name)
     return ret;
+char *qemuMonitorHello(qemuMonitorPtr mon, virHelloParams *params) {
+    char *ret;
+    VIR_DEBUG("mon=%p, params=%p",mon,params);
+    if (!mon) {
+        qemuReportError(VIR_ERR_INVALID_ARG, "%s",
+                        _("monitor must not be NULL"));
+        return NULL;
+    }
+    if (mon->json)
+        ret = qemuMonitorJSONHello(mon, params);
+    else
+        ret = qemuMonitorTextHello(mon, params);
+    return ret;
 int qemuMonitorArbitraryCommand(qemuMonitorPtr mon,
                                 const char *cmd,
                                 char **reply,
diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h
index c90219b..c1ecd10 100644
--- a/src/qemu/qemu_monitor.h
+++ b/src/qemu/qemu_monitor.h
@@ -418,6 +418,8 @@ int qemuMonitorCreateSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorLoadSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorDeleteSnapshot(qemuMonitorPtr mon, const char *name);
+char *qemuMonitorHello(qemuMonitorPtr mon, virHelloParams *params);
 int qemuMonitorArbitraryCommand(qemuMonitorPtr mon,
                                 const char *cmd,
                                 char **reply,

The function qemuMonitorHello is (yet another) wrapper around actual functionality. Libvirt supports the qmp and hmp monitor protocols. This function selects the appropriate function depending on the protocol being used by the active monitor.

diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index 6bd03d6..5f18cfa 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -2470,6 +2470,49 @@ cleanup:
     return ret;
+char *qemuMonitorJSONHello(qemuMonitorPtr mon, virHelloParams *params) {
+    const char *response;
+    char *ret = NULL;
+    virJSONValuePtr cmd;
+    virJSONValuePtr reply = NULL;
+    virJSONValuePtr obj;
+    if (params->lang == NULL)
+        cmd = qemuMonitorJSONMakeCommand("hello",
+                                         "b:exclaim", params->exclaim != 0,
+                                         NULL);
+    else
+        cmd = qemuMonitorJSONMakeCommand("hello",
+                                         "b:exclaim", params->exclaim != 0,
+                                         "s:lang", params->lang,
+                                         NULL);
+    if (!cmd)
+        return NULL;
+    if (qemuMonitorJSONCommand(mon, cmd, &reply) < 0 ||
+        qemuMonitorJSONCheckError(cmd, reply) < 0)
+        goto cleanup;
+    if (!(obj = virJSONValueObjectGet(reply, "return")) ||
+        obj->type != VIR_JSON_TYPE_OBJECT)
+        goto cleanup;
+    response = virJSONValueObjectGetString(obj, "response");
+    if (response == NULL) {
+        qemuReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+                        _("malformed response to hello command"));
+        goto cleanup;
+    }
+    /* Return a copy because we will be freeing reply below */
+    ret = strdup(response);
+    virJSONValueFree(cmd);
+    virJSONValueFree(reply);
+    return ret;
 int qemuMonitorJSONArbitraryCommand(qemuMonitorPtr mon,
                                     const char *cmd_str,
                                     char **reply_str,
diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h
index 086f0e1..e0cf5bd 100644
--- a/src/qemu/qemu_monitor_json.h
+++ b/src/qemu/qemu_monitor_json.h
@@ -199,6 +199,8 @@ int qemuMonitorJSONCreateSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorJSONLoadSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorJSONDeleteSnapshot(qemuMonitorPtr mon, const char *name);
+char *qemuMonitorJSONHello(qemuMonitorPtr mon, virHelloParams *params);
 int qemuMonitorJSONArbitraryCommand(qemuMonitorPtr mon,
                                     const char *cmd_str,
                                     char **reply_str,

Next, implement the JSON version of the hello command (for qmp monitors). This file contains very useful functions for manipulating JSON objects. qemuMonitorJSONMakeCommand is used to build the proper JSON command string. Next, we send the command, receive the response, and check for errors. We then use the virJSONValueObjectGet family of functions to extract the returned string from the response. Be careful when returning pointers and be sure to think about how the API is supposed to work. We want to return a new string to the caller and transfer ownership of that string to the caller. In order to do this we must make a copy (because the JSON response is going to be freed next. The string we return will eventually be freed by the caller when it is no longer needed.

diff --git a/src/qemu/qemu_monitor_text.c b/src/qemu/qemu_monitor_text.c
index 75b2995..452c43f 100644
--- a/src/qemu/qemu_monitor_text.c
+++ b/src/qemu/qemu_monitor_text.c
@@ -2607,6 +2607,36 @@ cleanup:
     return ret;
+char *qemuMonitorTextHello(qemuMonitorPtr mon, virHelloParams *params)
+    char *cmd;
+    char *reply = NULL;
+    int rc;
+    if (params->exclaim != 0)
+        rc = virAsprintf(&cmd, "hello -e %s",
+                         params->lang ? params->lang : "");
+    else
+        rc = virAsprintf(&cmd, "hello %s",
+                         params->lang ? params->lang : "");
+    if (rc == -1) {
+        virReportOOMError();
+        return NULL;
+    }
+    if (qemuMonitorHMPCommand(mon, cmd, &reply)) {
+        qemuReportError(VIR_ERR_OPERATION_FAILED,
+                         _("failed to perform command '%s'"),
+                         cmd);
+        VIR_FREE(reply);
+    }
+    VIR_FREE(cmd);
+    /* We own the reference to reply, return it and let the caller free it */
+    return reply;
 int qemuMonitorTextArbitraryCommand(qemuMonitorPtr mon, const char *cmd,
                                     char **reply)
diff --git a/src/qemu/qemu_monitor_text.h b/src/qemu/qemu_monitor_text.h
index 0838a2b..6c5f4f2 100644
--- a/src/qemu/qemu_monitor_text.h
+++ b/src/qemu/qemu_monitor_text.h
@@ -195,6 +195,8 @@ int qemuMonitorTextCreateSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorTextLoadSnapshot(qemuMonitorPtr mon, const char *name);
 int qemuMonitorTextDeleteSnapshot(qemuMonitorPtr mon, const char *name);
+char *qemuMonitorTextHello(qemuMonitorPtr mon, virHelloParams *params);
 int qemuMonitorTextArbitraryCommand(qemuMonitorPtr mon, const char *cmd,
                                     char **reply);

Last but not least, we have the hmp implementation of the command. Hopefully some day we will only need to program one monitor protocol but for now there are two. First, build a command string based on the parameters. Then execute the command and check for errors. If successful, reply will contain the hmp reply. No further parsing of this string is required so we just return it. Why don’t we duplicate the string as we did for the JSON monitor implementation? In this case, qemuMonitorHMPCommand allocates the reply string and transfers control of the reference to its caller. Thus, it is safe to return the pointer directly and let the caller free it.

Up next… Add hello remote transport support

Posted in Uncategorized | Leave a comment

Add hello to the public libvirt API

This is the fourth post in the series “HOWTO: Add libvirt support for a qemu command”

Although we have defined a top-level libvirt API, as yet there is no entry point implemented. In this patch, we create the entry point which does some basic sanity checking and dispatches the call down to a driver to perform the real work. Make this entry point visible by updating the version script. Find the patch here.

diff --git a/src/libvirt.c b/src/libvirt.c
index e46c18b..774ab3c 100644
--- a/src/libvirt.c
+++ b/src/libvirt.c
@@ -13355,3 +13355,46 @@ error:
     return -1;
+ * virDomainHello:
+ * @dom: pointer to the domain object
+ * @params: pointer to a virHelloParams object
+ *
+ * Returns: A pleasant greeting.
+ */
+char *
+virDomainHello(virDomainPtr dom,
+               virHelloParamsPtr params)
+    virConnectPtr conn;
+    char *ret = NULL;
+    VIR_DOMAIN_DEBUG(dom, "params=%p", params);
+    if (params == NULL) {
+        virLibDomainError (VIR_ERR_INVALID_ARG, __FUNCTION__);
+        goto error;
+    }
+    virResetLastError();
+    if (!VIR_IS_CONNECTED_DOMAIN (dom)) {
+        virLibDomainError (VIR_ERR_INVALID_DOMAIN, __FUNCTION__);
+        goto error;
+    }
+    conn = dom->conn;
+    if (conn->driver->domainHello) {
+        ret = conn->driver->domainHello (dom, params);
+        if (ret == NULL) {
+            goto error;
+        }
+        return ret;
+    } else {
+        virLibDomainError (VIR_ERR_NO_SUPPORT, __FUNCTION__);
+    }
+    virDispatchError(dom->conn);
+    return NULL;

Shown above is the entry point. If you look at src/libvirt.c you will notice a lot of functions that look similar to this one. This is mostly boiler-plate code that: checks for obvious problems with parameters (like NULL pointers where they are not allowed), checks for a valid connection and valid domain, and dispatches the call down to the driver associated with this connection. If the driver does not support the operation a suitable error is returned.

diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms
index 5caab4c..379ee6c 100644
--- a/src/libvirt_public.syms
+++ b/src/libvirt_public.syms
@@ -434,4 +434,9 @@ LIBVIRT_0.9.0 {
 } LIBVIRT_0.8.8;
+LIBVIRT_0.8.7 {
+    global:
+        virDomainHello;
+} LIBVIRT_0.8.6;
 # .... define new API here using predicted next version number ....

All public symbols in libvirt must be declared in the version script in the section corresponding to the release in which the symbol first became available.

Up next… Implement the qemu version of hello

Posted in Uncategorized | 1 Comment