Tag Archives: documentation

g_array_steal() and g_ptr_array_steal() in GLib 2.63.1

Another set of new APIs in the upcoming GLib 2.63.1 release allow you to steal all the contents of a GArray, GPtrArray or GByteArray, and continue using the array container to add more contents to in future.

This is work by Paolo Bonzini and Emmanuel Fleury, and will be available in the soon-to-be-released 2.63.1 release.

Here’s a quick example using GPtrArray — usage is similar in GArray and GByteArray:

g_autoptr(GPtrArray) chunk_buffer = g_ptr_array_new_with_free_func (g_bytes_unref);

/* Some part of your application appends a number of chunks to the pointer array. */
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("hello", 5));
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("world", 5));

…

/* Periodically, the chunks need to be sent as an array-and-length to some other part of the program. */
GBytes **chunks;
gsize n_chunks;

chunks = g_ptr_array_steal (chunk_buffer, &n_chunks);
for (gsize i = 0; i < n_chunks; i++)
  {
    /* Do something with each chunk here, and then free them, since g_ptr_array_steal() transfers ownership of all the elements and the array to the caller. */
    …

    g_bytes_unref (chunks[i]);
  }

g_free (chunks);

/* After calling g_ptr_array_steal(), the pointer array can be reused for the next set of chunks. */
g_assert (chunk_buffer->len == 0);

gtk-doc knows your package version

tl;dr: gtk-doc will now bind autoconf PACKAGE_* variables for use in your documentation.

For various modules which use gtk-doc, it’s a bit of a rite of passage to copy some build machinery from somewhere to generate a version.xml file which contains your package version, so that you can include it in your generated documentation (“Documenting version X of package Y”).

This is a bit of a pain. It doesn’t have to happen any longer.

gtk-doc master (to become 1.25) now generates a xml/gtkdocentities.ent file containing a few PACKAGE_* variables as generated by autoconf. So, you can now get rid of your version.xml machinery, and change your top-level documentation XML file’s DOCTYPE to the following:

<?xml version="1.0"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
   "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"
[
 <!ENTITY % local.common.attrib "xmlns:xi CDATA #FIXED 'http://www.w3.org/2003/XInclude'">
 <!ENTITY % gtkdocentities SYSTEM "xml/gtkdocentities.ent">
 %gtkdocentities;
]>

You can now use &package_string;, &package_version;, &package_url;, etc. in your documentation. See gtk-doc.make for a full list.

Thanks to Stefan for getting this feature in!

D-Bus API design guidelines

D-Bus has just gained some upstream guidelines on how to write APIs which best use its features and follow its conventions. Hopefully they draw together all the best practices from the various articles which have been published about writing D-Bus APIs in the past. If anybody has suggestions, criticism or feedback about them, or ideas for extra guidelines to include, please get in touch!

GNOME programming guidelines: the rise of gnome-devel-docs

Codethink logo

Before I start, an addendum to my last post about the DX hackfest: I wish to thank Codethink for sponsoring dinner one night of the event. I forgot to include that in my original post, sorry! Thanks again to all the companies who let employees attend: Endless, Codethink, Canonical and Red Hat.

tl;dr: Check out the new GNOME Programming Guidelines and file bugs in Bugzilla. Though it looks like a cronjob may be taking the docs offline occasionally. This will be fixed.

Now, to some of the results of the hackfest. In the last week or so, I’ve been working on expanding the GNOME programming guidelines, upstreaming various bits of documentation which Collabora have been writing for a customer who is using the GNOME stack in a large project. The guidelines were originally written in the early days of GNOME by Federico, Miguel and Morten; Federico updated them in 2013, and now they’ve been expanded again.

It looks like these guidelines can fill one of the gaps we currently have in documentation, where we need to recommend best practices and give tutorial-style examples, but gtk-doc–generated API manuals are not the right place. For example, the new guidelines include recommendations for making libraries parallel-installable (based off Havoc’s original article, with permission); or recommendations for choosing where to store data (in GSettings, a serialised GVariant store, or a full-on GOM/SQLite database?). The guidelines are intended to be useful to all developers, although inherently need to target newer developers more, so may simplify a few things.

I’ve still got some ideas for things to add. For example, I need to rework some of my blog posts about GMainContext into an article, since we should be documenting before blogging. Other ideas are very welcome, as is criticism, feedback and improvements: please file a bug against gnome-devel-docs. Thanks to the documentation team for their help and reviews!

(const gchar*) vs. (gchar*) and other memory management stories

This article now been upstreamed to the GNOME Programming Guidelines, where future updates will be made to it.

Memory management in C is often seen as a minefield, and while it’s not as simple as in more modern languages, if a few conventions are followed it doesn’t have to be hard at all. Here are some pointers about heap memory management in C using GObject and the usual GNOME types and coding conventions, including GObject introspection. A basic knowledge of C is assumed, but no more. Stack memory allocation and other obscure patterns aren’t covered, since they’re much less commonly used in a typical GObject application.

The most important rule when thinking about memory management is: which code has ownership of this memory?

Ownership can be defined as the responsibility to free or deallocate a piece of memory. If code which does own some memory doesn’t deallocate that memory, that’s a memory leak. If code which doesn’t own some memory deallocates it (in addition to the owner doing so), that’s a double-free. Both are bad.

For example, if a function returns a newly allocated string and doesn’t retain a pointer to it, ownership of the string is transferred to the caller. Note that when thinking about ownership transfer, malloc()/free() and reference counting are treated the same: in the former case, a newly allocated piece of heap memory is being transferred; in the latter, a newly incremented reference.

gchar *transfer_full_function (void) {
	return g_strdup ("Newly allocated string.");
}

/* Ownership is transferred here. */
gchar *my_string = transfer_full_function ();

Conversely, if the function does retain a pointer (e.g. inside an object), ownership is not transferred to the caller.

const gchar *transfer_none_function (void) {
	return "Static string.";
}

const gchar *transfer_none_method (MyObject *self) {
	gchar *new_string = g_strdup ("Newly allocated string.");
	/* Ownership is retained by the object here. */
	self->priv->saved_string = new_string;
	return new_string;
}

/* In both of these cases, ownership is not transferred. */
const gchar *my_string1 = transfer_none_function ();
const gchar *my_string2 = transfer_none_method (some_object);

In all of these examples, you may notice the return type of the functions reflects whether they transfer ownership of their return values. const return types indicate no transfer, and non-const return types indicate some transfer.

While this is useful, it’s by no means completely clear. For example, what if a function returns a non-const GList*? Is ownership of the list elements transferred, or just the list? What if the function’s author forgot to make the return type const, and actually there’s no transfer? This is where documentation comments are useful.

There’s a convention in GNOME documentation comments to specify the function which should be used to free a returned value. If such a function is mentioned, ownership of the returned memory is transferred. If no function is mentioned, ownership probably isn’t transferred, but it’s hard to be sure. That’s why it’s good to always be explicit when writing documentation comments.

/**
 * my_object_get_some_string:
 * @self: a #MyObject
 *
 * Gets the value of the #MyObject:some-string property.
 *
 * Return value: (transfer none): some string, owned by the
 * object and must not be freed
 */
const gchar *my_object_get_some_string (MyObject *self) {
	return self->priv->some_string;
}

/**
 * my_object_build_result:
 * @self: a #MyObject
 *
 * Builds a result string which probably represents something
 * meaningful.
 *
 * Return value: (transfer full): a newly allocated result
 * string; free with g_free()
 */
gchar *my_object_build_result (MyObject *self) {
	return g_strdup_printf ("%s %s",
	                        self->priv->some_string,
	                        self->priv->some_other_string);
}

/**
 * my_object_dup_controller:
 * @self: a #MyObject
 *
 * Gets the value of the #MyObject:controller property,
 * incrementing the controller's reference count.
 *
 * Return value: (transfer full): the object's
 * controller; unref with g_object_unref()
 */
MyController *my_object_dup_controller (MyObject *self) {
	return g_object_ref (self->priv->controller);
}

When GObject introspection was introduced, these kinds of documentation comments were formalised as introspection annotations: (transfer full), (transfer container) or (transfer none), as documented on wiki.gnome.org. These allow the runtimes of language bindings to manage memory correctly. Since a C programmer is essentially doing the same job as a language runtime when writing C code, the information provided by transfer annotations is sufficient for perfect memory management. Unfortunately, not all parameters and return types of all functions have these annotations added. The examples above do, however.

Finally, a few libraries use a function naming convention. Functions named *_get_* do not transfer ownership, whereas functions named *_dup_* (for ‘duplicate’) do transfer ownership. This can be seen in the examples above, or with json_node_get_array() vs. json_node_dup_array(). Be aware, though, that only a few libraries use this convention. Other libraries use *_get_* for both functions which do and do not transfer ownership of their results. Other code, such as that generated by gdbus-codegen, uses a different and incompatible convention: *_get_* methods signify full transfer, and *_peek_* methods signify no transfer. For example, goa_object_get_manager() vs. goa_object_peek_manager(). For this reason, going by function naming conventions only works within libraries, not between different libraries.

Memory management of parameters is analogous to return values: look at whether the parameter is const and whether there are any introspection annotations for it.

In summary, here are a set of guidelines one can follow to determine whether ownership of a return value is transferred, and hence whether the caller needs to free it:

  1. If the type has an introspection (transfer) annotation, look at that.
  2. Otherwise, if the type is const, there is no transfer.
  3. Otherwise, if the function documentation explicitly specifies the return value must be freed, there is full or container transfer.
  4. Otherwise, if the function is named *_dup_*, there is full or container transfer.
  5. Otherwise, if the function is named *_peek_*, there is no transfer.
  6. Otherwise, you need to look at the function’s code to determine whether it intends ownership to be transferred. Then file a bug against the documentation for that function, and ask for an introspection annotation to be added.

Some common pitfalls:

  • If you’re using an explicit typecast (e.g. casting a (const gchar*) return value to (gchar*)), it’s likely something’s wrong.
  • Generally, return values which are not transferred (such as (const gchar*)) are freed when the owning object is destroyed — so if you need such a value to persist, you must copy it or increase its reference count. You then have ownership of the copy or new reference, and are responsible for freeing it (not the original).

How can one check for incorrect memory handling? Use Valgrind. It will detect leaks and double-frees, and is simple to use:

valgrind --tool=memcheck --leak-check=full my-program-name

Or, if running your program from the source directory, use the following to avoid running leak checking on the libtool helper scripts:

libtool --mode=execute valgrind --tool=memcheck --leak-check=full ./my-program-name

Valgrind lists each memory problem it detects, along with a short backtrace (if you’ve compiled your program with debug symbols), allowing the cause of the memory error to be pinpointed and fixed!

Totem manual

In time for GNOME 2.26, the Totem help manual has been given a much-needed revamp. It's now completely up-to-date, and the first manual to reach "final" status with the new documentation status tracking features. I must apologise to all the translators whose work I trampled on by repeatedly changing things just after they updated the translations. That won't happen any more; as the documentation's now at "final" status, there shouldn't be any more changes to it for the 2.26 release. Translate away!

My thanks to Shaun McCance for stubbornly continuing to put all his effort into documentation work despite dwindling numbers of documenters, and for speedily reviewing the updated Totem manual.

Building PDFs of documentation

Still working on that coursework project, I'm now building PDFs of the three pieces of documentation the application itself requires: the API reference, the source code listings and the user manual.

Using gtk-doc to build the API reference makes it quite easy to add something in the Makefile to build a PDF from the DocBook:

docs/reference/Makefile.am

I've put this at the bottom of the Makefile.am, and it's all working nicely, apart from the fact that distcheck (legitimately) complains about me redefining all-local. I can't find any other way to make it build the PDF while also building the normal docs with make docs, so my question here is: how do I make this implementation cleaner?

If anybody wants to use it, they'll need to install dblatex.

The next piece of documentation is the source code listing. This is completely custom, using source-highlight to make LaTeX files for each source file, and then pdflatex to build them into a PDF.

docs/source/Makefile.am

I'm quite pleased with the way this works, although I'm sure there will be better solutions out there. source-highlight deposits many LaTeX files in the build directory, which are then included in the PDF build by dint of being automatically concatenated together and included by a surrounding LaTeX file, project-name-source.tex.

docs/source/two-hundred-club-source.tex

Finally, the user manual also needs building into a PDF. This again uses dblatex, although it's a little more complex, as a PDF has to be built for each language directory in the help directory.

docs/help/Makefile.am

Hopefully some of that will be helpful to somebody in a similar situation, and I'd welcome any hints or pointers as to how to improve the implementations, as my automake powers are about as strong as a damp tissue.