Testing D-Bus clients with libglib-testing

I’ve always found it a bit of a pain to write unit tests for D-Bus client libraries, where you’re testing that your code calls methods on a D-Bus service appropriately and, in particular, correctly handles a variety of return values and errors. Writing unit tests like this traditionally involves writing a mock D-Bus service for them to talk to, which validates the input it receives and provides appropriate responses. That often goes most of the way towards reimplementing the entirety of the real D-Bus service.

Part of the difficulty of testing D-Bus clients like this is synchronising the state of the mock D-Bus service with the test code, and part of the difficulty is the fact that you have to write mock service code for each D-Bus method before you can test it — which is a lot of investment in writing code before you can even start writing your unit tests themselves.

As an experiment in finding a better way of doing this kind of testing, I’ve written GtDBusQueue in libglib-testing, and I think it might be ready for some wider use. Thanks a lot to Endless for allowing me to work on such projects! I’ve used it in a couple of projects now, particularly in libmalcontent (which handles implementing parental controls policy on the desktop, and needs to talk to the accountsservice D-Bus service).

GtDBusQueue basically implements a queue for D-Bus messages received from your D-Bus client code. Each D-Bus message is typically a method call: your unit test can inspect the queue, and will typically pop messages off the front of the queue to assert they match a certain method call, and then send a reply to that call.

A key feature of GtDBusQueue is that it operates as a queue of D-Bus messages, rather than as a collection of D-Bus object proxies (typically GDBusObjectProxy), which means that it can be used to handle method calls to arbitrary D-Bus object paths without having to implement a new proxy class for each of them.

Message matching is typically implemented using gt_dbus_queue_assert_pop_message() (though other methods are available which give you finer-grained control over message matching and removal from the queue). It blocks until the queue is not empty, pops the first message off the front, asserts that its D-Bus object path, interface name and method name are as expected, and then returns the method call parameters to your unit test code using the same syntax as g_variant_get(). Your unit test code can then check the values of those parameters how it pleases.

If your D-Bus client code is asynchronous, GtDBusQueue can be used inline in your unit test. Your client code will start a method call asynchronously, then the test code will pop the method call off the GtDBusQueue, check it and reply, and then your client code will asynchronously finish its method call and handle the results. You can see an example of this in the test_app_filter_bus_get_error_disabled() test in libmalcontent which, in a single function, tests that the mct_manager_get_app_filter_async() API can correctly handle a D-Bus InvalidArgs error returned by the second D-Bus call it makes.

If your D-Bus client code is synchronous, GtDBusQueue needs to run in a thread using gt_dbus_queue_set_server_func(), since otherwise it would block your D-Bus client code. The unit test and the server thread take turns at blocking on pushing messages onto the queue or popping them off. You can see an example of this (which also works for asynchronous client code, testing both the synchronous and asynchronous code paths in a single test) in the GtDBusQueue usage example in its documentation.

That’s a brief introduction to GtDBusQueue; hopefully it’s given you a bit of an idea about where it’s appropriate and how it can be used. There’s documentation in the source code (including some usage examples), and a load of usage examples in libmalcontent. Feedback, questions and improvements are always welcome!

g_get_os_info() and GLib 2.63.1

GLib 2.63.1 has been released. The final new API to mention in this mini-series of blog posts about what’s in 2.63.1 is g_get_os_info().

g_get_os_info() is a way to get identifying information about the OS. On Linux, this is gathered from /etc/os-release. On other OSs, it’s gathered using platform-specific APIs (on other Unixes, this means falling back to uname()).

It was written by Robert Ancell, with contributions from Ruslan Izhbulatov, Ting-Wei Lan and Simon McVittie; and it came out of proposals from Robert at GUADEC.

To use it, pass it a key, like G_OS_INFO_KEY_PRETTY_NAME, and it’ll return a newly-allocated string with the corresponding value for the current OS, or NULL if there was none. Different OSs support different sets of keys, and the amount of support and set of keys may change over time.

An example:

g_autofree gchar *os_name = g_get_os_info (G_OS_INFO_KEY_PRETTY_NAME);
g_print ("OS: %s\n", os_name);
/* Prints “OS: Fedora 30 (Workstation Edition)” for me */

g_array_steal() and g_ptr_array_steal() in GLib 2.63.1

Another set of new APIs in the upcoming GLib 2.63.1 release allow you to steal all the contents of a GArray, GPtrArray or GByteArray, and continue using the array container to add more contents to in future.

This is work by Paolo Bonzini and Emmanuel Fleury, and will be available in the soon-to-be-released 2.63.1 release.

Here’s a quick example using GPtrArray — usage is similar in GArray and GByteArray:

g_autoptr(GPtrArray) chunk_buffer = g_ptr_array_new_with_free_func (g_bytes_unref);

/* Some part of your application appends a number of chunks to the pointer array. */
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("hello", 5));
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("world", 5));

…

/* Periodically, the chunks need to be sent as an array-and-length to some other part of the program. */
GBytes **chunks;
gsize n_chunks;

chunks = g_ptr_array_steal (chunk_buffer, &n_chunks);
for (gsize i = 0; i < n_chunks; i++)
  {
    /* Do something with each chunk here, and then free them, since g_ptr_array_steal() transfers ownership of all the elements and the array to the caller. */
    …

    g_bytes_unref (chunks[i]);
  }

g_free (chunks);

/* After calling g_ptr_array_steal(), the pointer array can be reused for the next set of chunks. */
g_assert (chunk_buffer->len == 0);

A diary program for GNOME: Almanah Diary

tl;dr: I’m giving up maintaining Almanah as it no longer scratches my itch — it’s yours if you want it, but maybe it should die in favour of more modern apps like Lifeograph or Red Notebook.

Almanah Diary is a project I started many years ago for maintaining a personal diary, with encryption and tracking of the events you did that day (from your calendar). It’s been neglected for a long time — given that I no longer use it, I have no incentive to maintain it and improve it, and have only held on to maintainership for so long out of a feeling of duty.

However, having me listed as a maintainer might have been giving people the false impression that it was actually maintained. So I’m removing myself as maintainer, having just made the 0.12.0 release. Álvaro Peña is also listed as a maintainer, but hasn’t been active for over a year. The project is technically now his, but if someone else turns up wanting to work on it, I am happy to add them as a co-maintainer, especially if they are going to revitalise things.

Other diary programs exist, and while I haven’t tried them, Lifeograph and Red Notebook look like they could be much more featureful and maintained than Almanah. Perhaps it’s time for them to be blessed as GNOME apps?

That said, if you want to take over maintenance of Almanah, please be my guest. It’s been ported to Meson, but needs some UI updates and needs flatpacking and putting on flathub.

If you’re interested, please get in touch, or send a merge request through.

Maintainer wanted: gnome-directory-thumbnailer

tl;dr: I’m giving up maintaining gnome-directory-thumbnailer as it no longer scratches my itch — it’s yours if you want it.

gnome-directory-thumbnailer is a little project I started a while ago for creating thumbnails for directories (rather than showing a plain directory icon for them in the file manager). Times change, I no longer have the itch that it was developed to scratch, and so I’m giving up maintenance of it after making the 0.1.11 release.

If you use it, or think you should use it, please care about it and take over maintenance. It needs porting to Meson, and some changes to handle working in the brave new world of sandboxed thumbnailers. It’s not a complicated code base to maintain.

If you’re interested, please get in touch, or send a merge request through.

GTimeVal deprecation in GLib 2.61.2

tl;dr: GTimeVal and g_get_current_time() are not year-2038-safe and have been deprecated; use GDateTime and g_get_real_time() instead.

One of the latest changes in GLib (released in 2.61.2) is the deprecation of GTimeVal, g_get_current_time(), and a number of other time functions. This is because we can’t guarantee they’re wide enough on all platforms to be year-2038-safe.

Instead, you should use GDateTime or, if you just need to store epoch time, guint64. They are year-2038-safe — and with that, GLib should be entirely year-2038-safe.

GTimeVal is used in a number of places, and widespread (but simple) changes will need to be made to stop using it. You will likely have already seen some deprecation warnings popping up to inform you of this, if you use any C-based and GLib-based libraries.

If you can’t allocate time to fixing these deprecation warnings yet, you can silence them by explicitly stating your minimum and maximum supported versions of GLib. If your minimum supported version of GLib is older than 2.62, you won’t see deprecation warnings for GTimeVal (since it was deprecated in 2.62, and your code is claiming to need to support older GLib versions than that).

You can do that by setting the following in meson.build:

common_c_args = [
  '-DGLIB_VERSION_MIN_REQUIRED=GLIB_VERSION_2_44',
  '-DGLIB_VERSION_MAX_ALLOWED=GLIB_VERSION_2_60',
]

and then using common_c_args in the c_args field of every executable and library you define in Meson. Your minimum required GLib version should match the version you list in your dependency('glib-2.0').

So,

GTimeVal tv;
g_get_current_time (&amp;tv);
g_message ("Current time: %lu", tv.tv_sec);

becomes

gint64 tv;
tv = g_get_real_time ();
g_message ("Current time: %" G_GINT64_FORMAT, tv / G_USEC_PER_SEC);

And

GTimeVal tv;
if (!g_time_val_from_iso8601 ("2019-08-24T10:47:05Z", &amp;tv))
  g_error ("Conversion failed");

becomes

g_autoptr(GDateTime) dt = g_date_time_new_from_iso8601 ("2019-08-24T10:47:05Z", NULL);
if (dt == NULL)
  {
    g_error ("Conversion failed");
    return;
  }

gint64 time_val = g_date_time_to_unix (dt);
/* or (simpler) use the #GDateTime directly */

Finally

GTimeVal tv = some timeval;
g_autofree gchar *tv_in_iso8601 = g_time_val_to_iso8601 (&amp;tv);
g_message ("In ISO 8601 format: %s", tv_in_iso8601);

becomes the following, using the new g_date_time_format_iso8601() function:

g_autoptr(GDateTime) dt = some GDateTime;
g_autofree gchar *dt_in_iso8601 = g_date_time_format_iso8601 (dt);
g_message ("In ISO 8601 format: %s", dt_in_iso8601);

g_assert_finalize_object() in GLib 2.61.2

One more API in this mini-series! g_assert_finalize_object(), which is available in GLib 2.61.2, which was released today.

This one’s useful when writing tests (and only when writing tests). It’s been put together by Simon McVittie to implement the common pattern needed in tests, where you want to unref a GObject and assert that you just dropped the final reference to the object — i.e., check that no references to the object have been leaked in the test.

Use it in place of g_object_unref(). If G_DISABLE_ASSERT is defined, it will actually just be a call to g_object_unref().

Here’s an example usage of it, straight out of the GLib unit test for it:

static void
test_assert_finalize_object (void)
{
  GObject *obj = g_object_new (G_TYPE_OBJECT, NULL);

  /* do some things with the obj here */

  g_assert_finalize_object (obj);
}

g_array_binary_search in GLib 2.61.2

The final API so far in this mini-series on new APIs in the GLib 2.62 series is g_array_binary_search(), put together by Emmanuel Fleury and based on code by Christian Hergert. It’s due to be released in 2.61.2 soon. But first, a reminder about GLib version numbering.

Like the rest of GNOME’s official module set, GLib follows an odd/even versioning scheme, where every odd minor version number, like 2.61.x, is an unstable release building up to an even minor version number, like 2.62.x, which is stable. APIs may be added in unstable releases. They may be modified or even removed (if they haven’t been in a stable release yet). So all of the APIs I’ve blogged about recently still have a chance to be tweaked or dropped if people find problems with them. So if you see a problem or think that one of these APIs would be awkward to use in some way, please say, sooner rather than later! They need fixing before they’re in a stable release.

Back to today’s API, g_array_binary_search(). As its name suggests, this does a binary search on an array (which it requires is already sorted). You can use it like this:

static gint
compare_guint64 (gconstpointer a,
                 gconstpointer b)
{
  guint64 uint64_a = *((guint64 *) a);
  guint64 uint64_b = *((guint64 *) b);

  if (uint64_a < uint64_b)
    return -1;
  else if (uint64_a > uint64_b)
    return 1;
  else
    return 0;
}

g_autoptr(GArray) my_array = g_array_new (FALSE, TRUE, sizeof (guint64));

for (guint i = 0; i < 100; i++)
  {
    guint64 random_uint64 = ( (guint64) g_random_int () << 32) | g_random_int ();
    g_array_append_val (my_array, random_uint64);
  }

g_array_sort (my_array, compare_guint64);

/* Is ‘1234’ in the array? If so, where? */
const guint64 search_uint64 = 1234;
guint search_index;
if (g_array_binary_search (my_array, &amp;search_uint64, compare_guint64, &amp;search_index))
  g_message ("Found ‘1234’ at index %u", search_index);
else
  g_message ("Didn’t find ‘1234’");

As all computer science algorithms courses will tell you, a binary search is faster than a linear search, so you should use this in preference to iterating over an array to find an element in it, where possible.

(That’s not entirely true: the overheads of accounting for the binary search bounds, and the slowness of scattered memory loads from the array in a binary search vs sequential access in a linear search, will probably make it slower than a linear search for small arrays. But both will be fast, and if you need to care about that level of performance, you should be using a custom data structure rather than GArray.)

Array copying and extending in GLib 2.61.2

A slightly more in-depth post in the mini-series this time, about various new functions which Emmanuel Fleury has landed in GLib 2.61.2 (which is due to be released soon), based on some old but not-quite-finished patches from others.

There’s g_ptr_array_copy() and g_array_copy(); and also g_ptr_array_extend() and g_ptr_array_extend_and_steal().

g_ptr_array_copy() and g_array_copy() are obvious functions and it’s not clear why they haven’t been added before. They allow you to copy a GPtrArray or a GArray, including its contents.
When copying a GPtrArray, you pass in a GCopyFunc to copy each element (for example, by increasing its reference count). If the GCopyFunc is NULL, the element is copied by value.

For example,

g_autoptr(GPtrArray) object_array = g_ptr_array_new_with_free_func (g_object_unref);

for (gsize i = 0; i < 10; i++)
  g_ptr_array_add (object_array, create_new_object (i));

object_array_copy = g_ptr_array_copy (object_array, g_object_ref, NULL);
/* object_array and object_array_copy now contain copies of the same elements, but
 * modifying one array will not modify the other */

The g_ptr_array_extend() functions are used to join one array onto the end of another. This means you can turn the following code to join the GObject elements of array2 onto the end of array1 and ref them all:

for (gsize i = 0; i < array2->len; i++)
  g_ptr_array_add (array1, g_object_ref (g_ptr_array_index (array2, i)));

into

g_ptr_array_extend (array1, array2, g_object_ref, NULL);

If you no longer need array2, you can go further and use g_ptr_array_extend_and_steal() to avoid copying each element. This might be particularly beneficial when using string arrays, where each copy (a g_strdup()) is more expensive. So the following code:

g_autoptr(GPtrArray) array1 = g_ptr_array_new_with_free_func (g_free);
for (guint i = 0; i < 10; i++)
  g_ptr_array_add (array1, g_strdup_printf ("array1 %u", i));

g_autoptr(GPtrArray) array2 = g_ptr_array_new_with_free_func (g_free);
for (guint i = 100; i < 110; i++)
  g_ptr_array_add (array2, g_strdup_printf ("array2 %u", i));

for (gsize i = 0; i < array2->len; i++)
  g_ptr_array_add (array1, g_strdup (g_ptr_array_index (array2, i)));

would become:

g_autoptr(GPtrArray) array1 = g_ptr_array_new_with_free_func (g_free);
for (guint i = 0; i < 10; i++)
  g_ptr_array_add (array1, g_strdup_printf ("array1 %u", i));

g_autoptr(GPtrArray) array2 = g_ptr_array_new_with_free_func (g_free);
for (guint i = 100; i < 110; i++)
  g_ptr_array_add (array2, g_strdup_printf ("array2 %u", i));

g_ptr_array_extend_and_steal (array1, g_steal_pointer (&amp;array2));
/* array2 has now been destroyed */