Add hello remote transport support

This is the sixth post in the series “HOWTO: Add libvirt support for a qemu command”

One great thing about libvirt is its ability to work with remote hosts just as easily as the local system on which it is installed. Of course this feature doesn’t come for free. Every new libvirt API requires an XDR implementation to transmit function calls and their results over the wire. Note: virsh always uses the remote driver to execute commands so this functionality must be in place for virsh to work even for local operations. I would regard this part to be the most difficult task when adding a new API to libvirt. Unless you are familiar with XDR, there is quite a bit going on and not much documentation to explain it.

This patch provides an XDR protocol specification which is used to generate several XDR-specific stubs. Additionally, a client and server implementation are provided. See the unadultered patch here.

    hello: Add hello remote transport support
    
    * src/remote/remote_protocol.x: provide defines for the new entry point
    * src/remote/remote_driver.c daemon/remote.c: implement the client and
      server side
    * GENERATED STUBS:
        daemon/remote_dispatch_args.h daemon/remote_dispatch_prototypes.h
        daemon/remote_dispatch_ret.h daemon/remote_dispatch_table.h
        src/remote/remote_protocol.c src/remote/remote_protocol.h

When adding remote support for an API it is only necessary to change three files. The others are generated automatically from remote_protocol.x during the build process. When submitting patches, include the changes to the generated files. I will make one additional point regarding XDR generation. I have found it helpful to manually regenerate the files whenever I modify remote_protocol.x. To do this, execute the following command from the top of the libvirt source tree:

make -C src rpcgen

Onward and upward!

diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x
index 7310689..54346cd 100644
--- a/src/remote/remote_protocol.x
+++ b/src/remote/remote_protocol.x
@@ -1926,6 +1926,20 @@ struct remote_domain_open_console_args {
     unsigned int flags;
 };
 
+struct remote_domain_hello_params {
+    remote_string lang;
+    int exclaim;
+};
+
+struct remote_domain_hello_args {
+    remote_nonnull_domain domain;
+    remote_domain_hello_params params;
+};
+
+struct remote_domain_hello_ret {
+    remote_string response;
+};
+
 /*----- Protocol. -----*/
 
 /* Define the program number, protocol version and procedure numbers here. */
@@ -2159,7 +2173,8 @@ enum remote_procedure {
     REMOTE_PROC_DOMAIN_SET_MEMORY_FLAGS = 204,
     REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205,
     REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206,
-    REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207
+    REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207,
+    REMOTE_PROC_DOMAIN_HELLO = 208
 
     /*
      * Notice how the entries are grouped in sets of 10 ?

The first step when adding remote support should be to define the wire protocol for your API. You must define the format of the arguments and return value (including explicit format definitions for any structures that are to be passed or returned). For virDomainHello, the arguments include a domain and a structure. The return is a string that can be NULL. The format of this file is C-like but not strict C. There should be enough examples in the file to figure out how to express the various C types.

After defining the types, allocate the next available RPC number for this API. The RPC number is used to figure out if both the client and server support a specific function.

diff --git a/daemon/remote.c b/daemon/remote.c
index d49e0d8..9be77ad 100644
--- a/daemon/remote.c
+++ b/daemon/remote.c
@@ -6995,6 +6995,51 @@ remoteDispatchDomainEventsDeregisterAny (struct qemud_server *server ATTRIBUTE_U
 }
 
 
+static int
+remoteDispatchDomainHello (struct qemud_server *server ATTRIBUTE_UNUSED,
+                           struct qemud_client *client ATTRIBUTE_UNUSED,
+                           virConnectPtr conn,
+                           remote_message_header *hdr ATTRIBUTE_UNUSED,
+                           remote_error *rerr,
+                           remote_domain_hello_args *args,
+                           remote_domain_hello_ret *ret)
+{
+    virDomainPtr dom;
+    struct _virHelloParams params;
+    char **response, *ret_str;
+    int rc = 0;
+
+    dom = get_nonnull_domain (conn, args->domain);
+    if (dom == NULL) {
+        remoteDispatchConnError(rerr, conn);
+        return -1;
+    }
+
+    /* XDR passes lang as a pointer to a pointer ... */
+    params.lang = args->params.lang == NULL ? NULL : *args->params.lang;
+    params.exclaim = (args->params.exclaim != 0);        
+
+    /* ... and the same thing for the return string */
+    if (VIR_ALLOC(response) < 0) {
+        remoteDispatchOOMError(rerr);
+        virDomainFree (dom);
+        return -1;
+    }
+
+    ret_str = virDomainHello(dom, &params);
+    if (!ret_str) {
+        remoteDispatchConnError(rerr, conn);
+        VIR_FREE(response);
+        rc = -1;
+    } else {
+        *response = ret_str;
+        ret->response = response;
+    }
+    
+    virDomainFree (dom);
+    return rc;
+}
+
 
 static int
 remoteDispatchNwfilterLookupByName (struct qemud_server *server ATTRIBUTE_UNUSED,

The function above implements the server side of the remote function call. This code runs within libvirtd on the “remote” host and calls down into the qemu driver on behalf of a client that may be running on another host. The pointer arguments args and ret contain encoded data representing the arguments to virDomainHello and a place to store the return value. Regardless of the API being called, this function returns an integer to indicate success or failure at the RPC level.

The first step is to convert args->dom into a valid local domain pointer. Next we build a local virHelloParams structure from the rest of the arguments. Something I found very confusing (mostly due to lack of documentation) is that XDR passes strings as char** pointers. That explains the strange way of accessing args.params->lang. Likewise to return a string we must allocate response a char** pointer. Once this is done we can make the actual API call on behalf of the client. If successful, the returned string is packed into response and then packed into the XDR return parameter. Finally, the domain pointer must be freed.

diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index 87a68a6..75c655a 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -9632,6 +9632,33 @@ done:
 
 }
 
+static char *
+remoteDomainHello(virDomainPtr dom, virHelloParamsPtr params)
+{
+    char *rv = NULL;
+    remote_domain_hello_args args;
+    remote_domain_hello_ret ret;
+    struct private_data *priv = dom->conn->privateData;
+    
+    remoteDriverLock(priv);
+    
+    make_nonnull_domain (&args.domain, dom);
+    args.params.lang = params->lang == NULL ? NULL : (char **) &params->lang;
+    args.params.exclaim = params->exclaim;
+    memset(&ret, 0, sizeof ret);
+    if (call (dom->conn, priv, 0, REMOTE_PROC_DOMAIN_HELLO,
+              (xdrproc_t) xdr_remote_domain_hello_args, (char *) &args,
+              (xdrproc_t) xdr_remote_domain_hello_ret, (char *) &ret) == -1)
+        goto done;
+        
+    /* Caller frees this. */
+    rv = *ret.response == NULL ? NULL : *ret.response;
+    
+done:
+    remoteDriverUnlock(priv);
+    return rv;
+}
+
 
 /*----------------------------------------------------------------------*/
 
@@ -11202,7 +11229,7 @@ static virDriver remote_driver = {
     remoteDomainSnapshotDelete, /* domainSnapshotDelete */
     remoteQemuDomainMonitorCommand, /* qemuDomainMonitorCommand */
     remoteDomainOpenConsole, /* domainOpenConsole */
-    NULL, /* domainHello */
+    remoteDomainHello, /* domainHello */
 };
 
 static virNetworkDriver network_driver = {

Here is the implementation of the client side which is organized as a driver (just like qemu, xen, etc). I followed the example set by other functions in the remote driver to handle driver locking. With that taken care of, the structures for XDR arguments and return value are prepared. The domain pointer is encoded using a helper function. As stated previously, XDR requires character pointers to be passed by reference. Next we make the actual RPC call. REMOTE_PROC_DOMAIN_HELLO refers to the RPC number that we specified in the protocol definition. The prepared args and ret structures are passed along wih their types. After the call, we unpack the returned string, clean up and then return our result string. By convention, we own the reference on the returned string so we do not need to duplicate it. The caller will free it.

diff --git a/daemon/remote_dispatch_args.h b/daemon/remote_dispatch_args.h
index 15fa1a0..cb6091a 100644
--- a/daemon/remote_dispatch_args.h
+++ b/daemon/remote_dispatch_args.h
@@ -176,3 +176,4 @@
     remote_domain_set_blkio_parameters_args val_remote_domain_set_blkio_parameters_args;
     remote_domain_get_blkio_parameters_args val_remote_domain_get_blkio_parameters_args;
     remote_domain_migrate_set_max_speed_args val_remote_domain_migrate_set_max_speed_args;
+    remote_domain_hello_args val_remote_domain_hello_args;
diff --git a/daemon/remote_dispatch_prototypes.h b/daemon/remote_dispatch_prototypes.h
index 3fcf87c..5f2264f 100644
--- a/daemon/remote_dispatch_prototypes.h
+++ b/daemon/remote_dispatch_prototypes.h
@@ -338,6 +338,14 @@ static int remoteDispatchDomainHasManagedSaveImage(
     remote_error *err,
     remote_domain_has_managed_save_image_args *args,
     remote_domain_has_managed_save_image_ret *ret);
+static int remoteDispatchDomainHello(
+    struct qemud_server *server,
+    struct qemud_client *client,
+    virConnectPtr conn,
+    remote_message_header *hdr,
+    remote_error *err,
+    remote_domain_hello_args *args,
+    remote_domain_hello_ret *ret);
 static int remoteDispatchDomainInterfaceStats(
     struct qemud_server *server,
     struct qemud_client *client,
diff --git a/daemon/remote_dispatch_ret.h b/daemon/remote_dispatch_ret.h
index 114e832..cdc9c89 100644
--- a/daemon/remote_dispatch_ret.h
+++ b/daemon/remote_dispatch_ret.h
@@ -140,3 +140,4 @@
     remote_domain_is_updated_ret val_remote_domain_is_updated_ret;
     remote_get_sysinfo_ret val_remote_get_sysinfo_ret;
     remote_domain_get_blkio_parameters_ret val_remote_domain_get_blkio_parameters_ret;
+    remote_domain_hello_ret val_remote_domain_hello_ret;
diff --git a/daemon/remote_dispatch_table.h b/daemon/remote_dispatch_table.h
index c5f6653..fb2c074 100644
--- a/daemon/remote_dispatch_table.h
+++ b/daemon/remote_dispatch_table.h
@@ -1042,3 +1042,8 @@
     .args_filter = (xdrproc_t) xdr_remote_domain_migrate_set_max_speed_args,
     .ret_filter = (xdrproc_t) xdr_void,
 },
+{   /* DomainHello => 208 */
+    .fn = (dispatch_fn) remoteDispatchDomainHello,
+    .args_filter = (xdrproc_t) xdr_remote_domain_hello_args,
+    .ret_filter = (xdrproc_t) xdr_remote_domain_hello_ret,
+},

diff --git a/src/remote/remote_protocol.c b/src/remote/remote_protocol.c
index 7ecea9d..ba31d50 100644
--- a/src/remote/remote_protocol.c
+++ b/src/remote/remote_protocol.c
@@ -3872,6 +3872,37 @@ xdr_remote_domain_open_console_args (XDR *xdrs, remote_domain_open_console_args
 }
 
 bool_t
+xdr_remote_domain_hello_params (XDR *xdrs, remote_domain_hello_params *objp)
+{
+
+         if (!xdr_remote_string (xdrs, &objp->lang))
+                 return FALSE;
+         if (!xdr_int (xdrs, &objp->exclaim))
+                 return FALSE;
+        return TRUE;
+}
+
+bool_t
+xdr_remote_domain_hello_args (XDR *xdrs, remote_domain_hello_args *objp)
+{
+
+         if (!xdr_remote_nonnull_domain (xdrs, &objp->domain))
+                 return FALSE;
+         if (!xdr_remote_domain_hello_params (xdrs, &objp->params))
+                 return FALSE;
+        return TRUE;
+}
+
+bool_t
+xdr_remote_domain_hello_ret (XDR *xdrs, remote_domain_hello_ret *objp)
+{
+
+         if (!xdr_remote_string (xdrs, &objp->response))
+                 return FALSE;
+        return TRUE;
+}
+
+bool_t
 xdr_remote_procedure (XDR *xdrs, remote_procedure *objp)
 {
 
diff --git a/src/remote/remote_protocol.h b/src/remote/remote_protocol.h
index 87de0da..318fa34 100644
--- a/src/remote/remote_protocol.h
+++ b/src/remote/remote_protocol.h
@@ -2184,6 +2184,23 @@ struct remote_domain_open_console_args {
         u_int flags;
 };
 typedef struct remote_domain_open_console_args remote_domain_open_console_args;
+
+struct remote_domain_hello_params {
+        remote_string lang;
+        int exclaim;
+};
+typedef struct remote_domain_hello_params remote_domain_hello_params;
+
+struct remote_domain_hello_args {
+        remote_nonnull_domain domain;
+        remote_domain_hello_params params;
+};
+typedef struct remote_domain_hello_args remote_domain_hello_args;
+
+struct remote_domain_hello_ret {
+        remote_string response;
+};
+typedef struct remote_domain_hello_ret remote_domain_hello_ret;
 #define REMOTE_PROGRAM 0x20008086
 #define REMOTE_PROTOCOL_VERSION 1
 
@@ -2395,6 +2412,7 @@ enum remote_procedure {
         REMOTE_PROC_DOMAIN_SET_BLKIO_PARAMETERS = 205,
         REMOTE_PROC_DOMAIN_GET_BLKIO_PARAMETERS = 206,
         REMOTE_PROC_DOMAIN_MIGRATE_SET_MAX_SPEED = 207,
+        REMOTE_PROC_DOMAIN_HELLO = 208,
 };
 typedef enum remote_procedure remote_procedure;
 
@@ -2776,6 +2794,9 @@ extern  bool_t xdr_remote_domain_snapshot_current_ret (XDR *, remote_domain_snap
 extern  bool_t xdr_remote_domain_revert_to_snapshot_args (XDR *, remote_domain_revert_to_snapshot_args*);
 extern  bool_t xdr_remote_domain_snapshot_delete_args (XDR *, remote_domain_snapshot_delete_args*);
 extern  bool_t xdr_remote_domain_open_console_args (XDR *, remote_domain_open_console_args*);
+extern  bool_t xdr_remote_domain_hello_params (XDR *, remote_domain_hello_params*);
+extern  bool_t xdr_remote_domain_hello_args (XDR *, remote_domain_hello_args*);
+extern  bool_t xdr_remote_domain_hello_ret (XDR *, remote_domain_hello_ret*);
 extern  bool_t xdr_remote_procedure (XDR *, remote_procedure*);
 extern  bool_t xdr_remote_message_type (XDR *, remote_message_type*);
 extern  bool_t xdr_remote_message_status (XDR *, remote_message_status*);
@@ -3131,6 +3152,9 @@ extern bool_t xdr_remote_domain_snapshot_current_ret ();
 extern bool_t xdr_remote_domain_revert_to_snapshot_args ();
 extern bool_t xdr_remote_domain_snapshot_delete_args ();
 extern bool_t xdr_remote_domain_open_console_args ();
+extern bool_t xdr_remote_domain_hello_params ();
+extern bool_t xdr_remote_domain_hello_args ();
+extern bool_t xdr_remote_domain_hello_ret ();
 extern bool_t xdr_remote_procedure ();
 extern bool_t xdr_remote_message_type ();
 extern bool_t xdr_remote_message_status ();

The rest of this is auto-generated by rpcgen. Therefore, I don’t see a need to discuss it any further.

Up next… Create hello virsh command

Advertisements

About aglitke

I am a software engineer working on Linux, open source software, and virtualization. I am proud to work at Red Hat on oVirt and Red Hat Virtualization with a focus on software defined storage. Other notable projects I have been involved in include: The Linux ppc64 architecture, Linux kernel crash dumps (kdump), Linux huge pages and libhugetlbfs, qemu, libvirt, and the Memory Overcommitment Manager.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s