Notes on Vala and automake

Here are some brief notes about getting automake’s Vala support1 to play nicely with you when writing a library, mostly for my own future benefit.

Firstly, use target-specific *_VALAFLAGS variables (rather than AM_VALAFLAGS) to specify the flags to pass to the valac compiler. Doing this enables a couple of dist and clean features in automake which wouldn’t otherwise be enabled. It’s also means you can specify multiple targets in a single Makefile.am, which is good for non-recursive automake.

When building a library, explicitly specify --vapi, --header, --internal-vapi, --internal-header and --gir (as appropriate) in the *_VALAFLAGS variable directly, rather than including them through another variable. automake looks for these flags in order to build dist and clean rules for the generated VAPI, C header and GIR files.

automake will automatically delete .stamp, VAPI and generated C header and source files on maintainer-clean, but not as part of any other clean target — this is because automake is designed to ensure generated C files are included in the distribution tarball. Failing to specify --header, --vapi (etc.) directly in *_VALAFLAGS will cause the appropriate maintainer-clean rules to not be generated.

automake knows nothing about .deps files (which accompany .vapi files), and will not generate dist rules for them; that must be done manually using:

vapidir = $(datadir)/vala/vapi
dist_vapi_DATA = my-library.deps

Source: lang_vala_finish_target() in the automake-1.13 script. Read it: it’s quite clear and informative.


  1. Note: That documentation seems to be a bit out of date regarding VALAFLAGS

Clang plugin for GLib and GNOME

This past week, I’ve been working on a Clang plugin for GLib and GNOME, with the aim of improving static analysis of GLib-based C projects by integrating GIR metadata and assumptions about common GLib and GObject coding practices. The idea is that if running Clang for static analysis on a project, the plugin will suppress common GLib-based false positives and emit warnings for common problems which Clang wouldn’t previously have known about. Using it should be as simple as enabling scan-build, with the appropriate plugin, in your JHBuild configuration file.

I’m really excited about this project, which I’m working on as R&D at Collabora. It’s still early days, and the project has a huge scope, but so far I have a Clang plugin with two major features. It can:

  • Load GIR metadata1 and use it to add nonnull attributes based on the presence (or absence) of (allow-none) annotations.
  • Detect g_return_if_fail(), assert(), g_return_val_if_fail() (etc.) preconditions in function bodies and add nonnull attributes if the assertions are non-NULL checks (or are GObject type checks which also assert non-nullability).

Additionally, the plugin can use GIR metadata to add other attributes:

  • Add deprecated attributes based on Deprecated: gtk-doc tags.
  • Add warn_unused_result attributes to functions which have a (transfer container) or (transfer full) return type.
  • Add malloc attributes to functions which are marked as (constructor).
  • Constify the return types of (transfer none) functions which return a pointer type.

By modifying Clang’s abstract syntax tree (AST) for a file while it’s being scanned, the plugin can take advantage of the existing static analysis for NULL propagation to function calls with nonnull attributes, plus its analysis for variable constness, use of deprecated functions, etc.

So far, all the code is in two ‘annotaters’, which modify the AST but don’t emit any new warnings of their own. The next step is to implement some ‘checkers’, which examine (but don’t modify) the AST and emit warnings for common problems. Some of the problems checked for will be similar to those handled above (e.g. warning about a missing deprecated attribute if a function has a Deprecated: gtk-doc tag), but others can be completely new, such as checking that if a function has a GError parameter then it also has a g_return_if_fail(error == NULL || *error == NULL) precondition (or something along those lines).

I’ve got several big ideas for features to add once this initial work is complete, but the next thing to work on is making the plugin effortless to use (or, realistically, nobody will use it). Currently, the plugin is injected by a wrapper script around Clang, e.g.:

cd /path/to/glib/source
GNOME_CLANG_GIRS="GLib-2.0 Gio-2.0" gnome-clang -cc1 -analyze … /files/to/analyse

This injects the -load $PREFIX/lib64/clang/3.5/libclang-gnome.so -add-plugin gnome arguments which load and enable the plugin. By specifying GIR namespaces and versions in GNOME_CLANG_GIRS, those GIRs will be loaded and their metadata used by the plugin.

This can be configured in your ~/.jhbuildrc with:

static_analyzer = True
static_analyzer_template = 'scan-build --use-analyzer=$JHBUILD_PREFIX/bin/gnome-clang -v -o %(outputdir)s/%(module)s'
os.environ['GNOME_CLANG_GIRS'] = 'GLib-2.0 Gio-2.0 GnomeDesktop-3.0'

Please note that it’s alpha-quality code at the moment, and may introduce program bugs if used during compilation, so should only be used with Clang’s -analyze option. Furthermore, it still produces some false positives, so please be cautious about submitting bug reports to upstream projects ‘fixing’ problems detected by static analysis.

The code so far is here: http://cgit.collabora.com/git/gnome-clang.git/ and there’s a primitive website here: http://people.collabora.com/~pwith/gnome-clang/. If you’ve got any ideas, bug reports, patches, or just want to criticise my appalling C++, please e-mail me: philip {at} tecnocode.co(.)uk.


  1. Actually, it loads the metadata from GIR typelib files, rather than using the source code annotations themselves, so there is a degree of data loss incurred due to the limitations of the typelib binary format. 

libfolks: BlueZ backend for phone address book access

libfolks now has a backend for accessing address book data from Bluetooth-enabled phones, making use of BlueZ 5. This has been a long time in development, with various developers from Collabora contributing to it (thanks to Matthieu Bouron, Gustavo Padovan, Arun Raghavan and Jeremy Whiting!).

It works using the Bluetooth Phone Book Access Profile (PBAP), which allows a computer to download a phone’s address book as an array of vCards. libfolks will automatically do this for any phone which is paired with the computer (and which isn’t blocked), establishing a connection with any (paired, non-blocked) phone in range.

We’ve only been able to test this with a limited number of phones, so further real-world testing and feedback would be greatly appreciated. To do so, grab the latest version of libfolks from git (https://git.gnome.org/browse/folks/log/), build it and run folks-inspect (making sure that the ‘bluez’ backend is not disabled in ~/.local/share/folks/backends.ini, if you’ve created and customised that file).

As is the design of libfolks, this is not intended as a solution for synchronising contacts between your phone and your computer, but it does provide a handy way of accessing contacts on your phone from GNOME Shell or Contacts, or from other consumers of libfolks data.

Note that the backend only supports BlueZ 5 and explicitly doesn’t support BlueZ 4.

Statically linking using pkg-config and JHBuild

tl;dr: Export PKG_CONFIG="pkg-config --static" or add it to your .jhbuildrc file.

Statically linking libraries which use libtool (specifically, the LT_INIT autoconf macro) is easy: you pass --enable-static --disable-shared to configure, and out pop static .a libraries instead of dynamic .so libraries.

However, things get more complex once you start to consider dependencies of your library. For example, if I’m statically linking libfoo against libbar, and libbar uses libwidgit internally (i.e. consumes the libwidgit symbols but does not expose any of them as part of libbar’s public interfaces), libfoo needs to know about libwidgit and needs to list it in the libfoo linker command. This is because UNIX archives (statically linked .a files) do not list their dependencies (whereas dynamically linked .so files do (on sensible UNIX platforms, including Linux)). Effectively the transitive closure of link dependencies needs to be resolved at link time, rather than at run time.

How can we cope with this when configuring libfoo? By using pkg-config --static instead of normal pkg-config. This can be achieved by setting the PKG_CONFIG environment variable to pkg-config --static or by adding the following lines to .jhbuildrc (which also set the libtool options mentioned above):

autogenargs = '--disable-shared --enable-static'
os.environ['PKG_CONFIG'] = 'pkg-config --static'

pkg-config’s --static option changes the program’s output to include the Libs.private and (recursively the) Requires.private variables. For example:

$ pkg-config --libs gobject-2.0
-L/opt/gnome3/build/lib64 -lgobject-2.0 -lglib-2.0
$
$ pkg-config --static --libs gobject-2.0
-pthread -L/opt/gnome3/build/lib64 -lgobject-2.0 -lffi -lglib-2.0

Here, libffi and pthread are both private dependencies of GObject: they’re used internally in GObject, but don’t appear in its external interface. When statically linking against GObject, though, their symbols must be resolved and hence they must appear in the linker command.

For more about the differences between pkg-config’s Libs and Libs.private variables, see the excellent guide to pkg-config. For more information about static linking with pkg-config, see the Autotools Mythbuster.

(There are probably errors or omissions in this post, since I’m nowhere near an expert on linking. Please comment with any corrections! Also, this is not meant to suggest that statically linking libraries is a good idea in general — there are only a few situations where it’s sensible. But for those situations, hopefully this post makes things a little easier.)

Colourful GCC output

For a long time, I’ve used colorgcc as a way to highlight warnings and errors in GCC output. It’s worked wonderfully, but today Travis pointed out that GCC has grown an option to enable colourised output: -fdiagnostics-color. This will be available in GCC 4.9, but the patch seems to have been backported to Fedora 19 so that more people can join in the fun.

This means I can throw away colorgcc from my .jhbuildrc:

os.environ['CC'] = '/usr/bin/color-gcc'

and enable -fdiagnostics-color:

os.environ['CFLAGS'] += ' -fdiagnostics-color=auto'

Even better, adding the following snippet to configure.ac in a standard automake project will automatically enable colourised compiler output if the compiler (is GCC and) supports it:

AS_COMPILER_FLAG([-fdiagnostics-color=auto],
                 [AM_CFLAGS="$AM_CFLAGS -fdiagnostics-color=auto"])

(In actual fact, I used ERROR_CFLAGS for this in folks, to keep unrelated CFLAGS separated, but that’s an implementation detail of folks’ build system.)

Writing a GNOME thumbnailer

How does GNOME generate thumbnails for files? It uses a collection of programs called thumbnailers, each one generating thumbnails for a specific set of content-types of files. For example, totem-video-thumbnailer generates thumbnails for video files using GStreamer; evince-thumbnailer generates thumbnails for PDFs and other document files.

To generate a thumbnail, an appropriate thumbnailer program is selected then executed, passing it the path of the file to thumbnail, plus a path to write the thumbnail image to. If thumbnailing succeeds, the thumbnailer should have written the image to disk before terminating; but if thumbnailing fails, no image should be written, and the thumbnailer should return a non-zero exit status.

Thumbnailers are chosen by examining a series of .thumbnailer files in $PREFIX/share/thumbnailers. Each is in a simple key-file format:

[Thumbnailer Entry]
TryExec=evince-thumbnailer
Exec=evince-thumbnailer -s %s %u %o
MimeType=application/pdf;application/x-bzpdf;application/x-gzpdf;

The .thumbnailer format supports three keys:

  • TryExec: Optional. The name of the the thumbnailer program in the current $PATH. This allows the calling code to quickly check whether the thumbnailer exists before trying to execute it.
  • Exec: Required. The command to execute the thumbnailer. It supports a few different parameters which are replaced before calling the thumbnailer: %u is the URI of the file being thumbnailed; %i is its filename; %o is the filename of the image file to be written to; %s is the maximum desired size of the thumbnail image (in pixels); and %% is a literal percent character.
  • MimeType: Required. A semicolon-separated list of MIME types which the thumbnailer supports generating thumbnails for.

So in the evince-thumbnailer file above, the command passes the requested thumbnail size, then the input file’s URI, then the path for the output image file to evince-thumbnailer.

The code to examine and call a thumbnailer is in gnome-common’s GnomeDesktopThumbnailFactory class, which handles looking up the right thumbnailer script, building and executing the command for it, and loading the resulting thumbnail image into a GdkPixbuf.

Thumbnail caching is supported by GnomeDesktopThumbnailFactory. When calling a thumbnailer, the path passed for the output image file is in ~/.cache/thumbnails/$SIZE/. The cached image file is given a (probably) unique filename, generated by hashing the original file’s URI, so the thumbnail can be looked up in future. gnome-desktop supports two sizes of thumbnails: ‘normal’ and ‘large’. Normal thumbnails are up to 128×128 pixels, whereas large thumbnails are up to 256×256 pixels. Thumbnails which are larger than this are scaled down before being cached, and non-square thumbnails are scaled so their largest dimension is at most 128 or 256 pixels.

gnome-desktop also handles failed thumbnails. If a thumbnailer can’t generate a thumbnail for a file (e.g. because the file is corrupt or because the right video codecs aren’t available), it returns a non-zero exit status. gnome-desktop then writes an entry to ~/.cache/thumbnails/fail/gnome-thumbnail-factory/ which is named after the hash of the input file URI (just like a successful cached thumbnail). For future queries for thumbnails for that file, GnomeDesktopThumbnailFactory can immediately return an error after looking up the fail entry.

If a file changes content, GnomeDesktopThumbnailFactory knows to generate a new thumbnail because each cached image has associated metadata (stored as PNG tEXt keys) storing the full URI of the thumbnailed file (to check for hash collisions) and its last modification time at the point of thumbnailing. If the stored modification time doesn’t match the file’s current one, a new thumbnail is generated.

I don’t care. Show me the code.

As an example of this, I present gnome-directory-thumbnailer, which generates thumbnails for directories (which is possible with a few small fixes to gnome-desktop). I wrote this as part of a project for my new employer, Collabora. It generates a thumbnail for a directory by heuristically choosing its ‘most interesting’ file (or subdirectory) and taking the thumbnail from that. It works fairly well for generic directories, but there’s great scope for improving the heuristics to cater for directories which match different templates (e.g. a directory of audio files for an album; a directory containing an automake project; a mounted DVD directory; etc.).

Nautilus showing some album directories.

Nautilus showing some album directories.

The code is in gnome-directory-thumbnailer on git.gnome.org and it’s in the process of getting a gnome-directory-thumbnailer product on bugzilla.gnome.org, so please file bugs there. Note that you’ll need the latest release (3.10.1) of gnome-desktop for it to work correctly in Nautilus.

If you want to write your own thumbnailer, a good place to start is with the gnome-thumbnailer-skeleton by Bastien Nocera. It provides the generic wrapper code for command line parsing and output handling which would otherwise get rewritten for each new thumbnailer.

There are actually other ways of generating thumbnails, such as the Tumbler thumbnailing system which is accessed via D-Bus and implements the proposed thumbnailer specification which is used by Rygel. As far as I’m aware, Nautilus doesn’t use D-Bus thumbnailers.

Using libgee from C to access libfolks

If you’re interfacing with Vala code from C, it’s possible that you’ll have to make calls to libgee, a Vala data structure library. libgee is written in Vala, but its C API is perfectly usable once you get your head around the idioms.

A common idiom is that of the GeeIterator, which is the recommended way to iterate through sets, maps and lists (which all implement the GeeIterable interface). For a loop, you create a new GeeIterator, then call its get and next methods to retrieve the object for the current iteration and advance to the next iteration, respectively.

For example, to list all the phone numbers in a FolksIndividual:

GeeSet *phone_numbers;
GeeIterator *iter;
FolksPhoneDetails *details;

details = FOLKS_PHONE_DETAILS (my_individual);
phone_numbers = folks_phone_details_get_phone_numbers (details);
iter = gee_iterable_iterator (GEE_ITERABLE (phone_numbers));

while (gee_iterator_next (iter)) {
	FolksPhoneFieldDetails *details;
	gchar *number;

	/* Get the current iteration’s element.
	 * Note that ownership is transferred. */
	details = gee_iterator_get (iter);
	number = folks_phone_field_details_get_normalised (details);

	g_message ("Phone number: %s", number);

	g_free (number);
	g_object_unref (details);  /* important! */
}

g_object_unref (iter);  /* important! */

One key thing to note here is that the return values from gee_iterator_get() and gee_iterable_iterator() are transfer-full and need to be unreffed. Failing to unref them is a common mistake when using libgee from C, and results in leaked objects — and was the cause of several memory leaks recently fixed in Empathy. When using libgee from Vala this isn’t a problem, since the memory management is handled automatically.

This has been a public service announcement.

Travelling in Honduras

Having just spent 7 weeks in Honduras, working and travelling, I thought it might be useful to document some of the things I’ve learned about the country.

In my experience, Honduras has three distinct areas: the north (Caribbean) side where English is spoken moderately commonly, and many people go to learn to dive (especially on the Bay Islands); the three major cities (Tegucigalpa, San Pedro Sula and La Ceiba) where tourism infrastructure is practically non-existent, pollution is high and crime is high; and the rest of the countryside, which is pleasant, safe (for the most part) and has little tourism infrastructure.

Lightning on the cloudy mountains around Tegucigalpa at night.I spent 5 weeks in the capital (Tegucigalpa) and saw the bad side of Honduras. It is not a safe city to walk around, even in the richer areas, so you have to drive everywhere. I hadn’t previously realised how much I value being able to walk the streets and go where I please without probability of being robbed at gunpoint. Despite its security problems, Tegus was a reasonable-ish place to be. It has all the amenities of a big city — clubbing, for example, there is just as grim as in England, except with more Latin music. There is nothing for tourists to see, and they should only use it as a transport hub (since it has one of the country’s two major airports, and is a major bus terminal) or as a base to explore the much-nicer Valle de Ángeles, La Tigra national park, and Comayagua nearby.

Tegucigalpa demonstrates the two major problems which Honduras suffers from: a huge imbalance in the distribution of wealth, and endemic corruption in government and the police force. These have knock-on effects on everything else. The rich live in enclosed neighbourhoods with high walls, barbed wire, electric fences, and armed private security. They drive luxury 4×4s and have lives which would be considered privileged in Europe. The poor live in shacks which line the roads and are stacked up the hillsides. There is a huge amount of gang crime: it seems that most of the drugs passing from South America to the USA come through Honduras. This gang crime contributes hugely to Honduras’ murder rate, making it officially one of the most dangerous countries in the world — although if you’re not involved in a gang, you’re as safe as in other Central American countries. A lot of this crime goes unpunished, both because the police are unable to tackle the gangs, and also because they’re corrupt. It is the army which enforces the law here.

One of the many building-piers on Utila.The Bay Islands are completely different. I spent a week on Utila, and it was a safe, friendly place, quite different from Tegucigalpa. Its summer climate is hot (30?) and humid (up to 80% relative humidity, with very little wind), which I found unbearable at times. Still, it is easy to get to, has excellent diving opportunities, and caters well for someone who wants to party every night. It’s quite drug-friendly.

The rest of the country is mostly small villages and farmland. There isn’t much tourism infrastructure, but travel is easy and cheap on ‘collectivo’ and chicken buses (old, reappropriated US school buses; less gaudily decorated than in Guatemala). Most villages don’t have much to offer other than colonial architecture and old churches (and a lot of auto repair shops). A few are more popular: Gracias, Lempira; and Copán Ruinas.

Gracias, Lempira in the sun.

Gracias has more quaint architecture than most places, and is next to Celaque national park, home of Honduras’ tallest mountain (Cerro Las Minas). Worth a visit if you like cloud forests, bird watching or hiking in general. Summitting Cerro Las Minas normally takes two days, but an experienced (and fit — it’s a steep mountain) hiker can do it in 7 hours.

A stone face in the ruins at Copán.Copán Ruinas is the gateway to the country’s Mayan ruins, which are definitely worth a visit. Very well preserved hieroglyphics, and lots of wildlife (including a Macaw breeding programme). If you visit Copán Ruinas, seeing the ruins will take half a day, and there is little else to do in the town unless you plan in advance with a local tour company to do a trek or horse ride. I didn’t, so I don’t know what they’re like.

Infrastructure is what Honduras lacks. Its roads are full of potholes (leading to its infamous crazy driving), there are frequent (roughly once a week) power outages and internet outages. None of the tap water is potable and drainage often runs straight onto the street (which in turn expedites damage to the roads). Building and maintaining infrastructure in such a mountainous country is tricky, but it’s necessary.

With the benefit of hindsight, there are a few things I would do differently if I came to Honduras again. Quite a few of them are the standard recommendations for travelling in Central/South America anyway:

  • If you can afford it, hire a car or motorbike (the latter are incredibly popular here). Having your own transport gives you a lot of freedom and safety. Get a vehicle suited to off-road driving: even if you don’t end up on a dirt track (which is quite possible if one of the few major roads is closed due to landslides), there are nasty speed bumps throughout the country which chew up cars with low ground clearance.
  • If you can’t, use collectivo buses, which are safe and, while the routes aren’t listed anywhere online or on paper, generally follow major roads. Any local will know about them. They run very frequently.
  • Find out which areas of a city are considered safe, and which aren’t. If you stick to the safe areas and use common sense, everything will be fine. If you go to an unsafe area, you’re likely to be robbed or kidnapped. For example, even the locals in Tegus don’t venture to Comayaguela. The hotels aren’t always in the safe areas, especially in Tegucigalpa — definitely avoid anything in the Comayaguela area.
  • Don’t plan to do much hiking. It’s not a particularly popular pastime here, so while there are many ‘national parks’, they mostly exist to protect the cloud forests. There are some nice trails in some (for example, La Tigra and Celaque), but don’t expect too much. If you’re interested in birds, however, Honduras is supposed to be pretty good; there are hotels dedicated to bird watching around La Ceiba.
  • If you go to the Bay Islands to dive, have a contingency plan in case of illness (which would prevent you from diving). When I was there, several people I knew got colds and ended up doing absolutely nothing for a week until they got better.
  • Don’t try and plan a tight itinerary in advance, as unexpected transport problems might throw it off. Internet access is easy to come by in Honduras (especially 3G), so make things up as you go along. Factor in plenty of time for making travel connections, as few of the bus times are synchronised.
  • Aside from touristy places (like Copán Ruinas), Honduras doesn’t do European-style hostels. You’re going to have to look at hotels, but there are cheap ones.
  • Similarly, don’t expect to send any postcards. In my experience, they’re hard to find, and harder to send. Post offices are like gold dust.
  • Don’t trust the websites describing most bus routes in Honduras (apart from the huge companies like Hedman Alas). Even medium-sized Honduran companies don’t do websites well, or keep them updated. Phone the bus company up instead, or ask a local.

A summer with XOs in Honduras

One of the main streets in Valle de Ángeles, a village near Tegucigalpa, the capital.

One of the main streets in Valle de Ángeles, a village near Tegucigalpa, the capital.

I’ve been lucky this summer to spend 7 weeks in Honduras, working on the OLPC deployment for primary school kids here. I’ve been training the local team (part of Educatrachos, a government- and IADB-funded education project in Honduras): teaching them Python, how to create activities for Sugar, and some Unix server administration magic.

The deployment itself is impressive. Over the past two years, they’ve delivered 40?000 laptops to 400 schools. A lot of the work has been in providing infrastructure (power and internet): this is tricky given Honduras’ hugely irregular terrain. A lot of the schools are using satellite internet, which is inherently affected by the weather. Despite these obstacles, the infrastructure has been in place for a while and is working nicely.

Showing a hint about which cells to sum in Pascal’s triangle.

Showing a hint about which cells to sum in Pascal’s triangle.

The focus is now moving towards producing and updating educational resources for the laptops. That means creating new Sugar activities and refreshing and redeploying existing resource collections. This is what the bulk of my time has been spent on: training the team here in how to create activities and collections, starting from basic Python and working upwards. It’s been tricky (because of my poor Spanish if nothing else) but the team have tackled the learning with enthusiasm. I hope to see new Sugar activities on the Educatrachos gitorious page soon! So far, we’ve produced one new Sugar activity: a Pascal’s triangle game. We’ve also published the training materials I used for teaching Python and Sugar. They’re available in English and Spanish, although the Spanish translation is pretty patchy (my fault!).

What remains to be seen is how the project will evolve after the change of government from the upcoming Honduran elections.

Tomorrow is my final day in Tegucigalpa: on Friday I leave for two weeks of time off, travelling around the country to end a fun summer of travelling to celebrate graduating from university after four years.

Thanks to Raúl Segales, Walter Bender, Daniel Drake and Martin Abente for answering my silly questions as I dived into Sugar!

(const gchar*) vs. (gchar*) and other memory management stories

Memory management in C is often seen as a minefield, and while it’s not as simple as in more modern languages, if a few conventions are followed it doesn’t have to be hard at all. Here are some pointers about heap memory management in C using GObject and the usual GNOME types and coding conventions, including GObject introspection. A basic knowledge of C is assumed, but no more. Stack memory allocation and other obscure patterns aren’t covered, since they’re much less commonly used in a typical GObject application.

The most important rule when thinking about memory management is: which code has ownership of this memory?

Ownership can be defined as the responsibility to free or deallocate a piece of memory. If code which does own some memory doesn’t deallocate that memory, that’s a memory leak. If code which doesn’t own some memory deallocates it (in addition to the owner doing so), that’s a double-free. Both are bad.

For example, if a function returns a newly allocated string and doesn’t retain a pointer to it, ownership of the string is transferred to the caller. Note that when thinking about ownership transfer, malloc()/free() and reference counting are treated the same: in the former case, a newly allocated piece of heap memory is being transferred; in the latter, a newly incremented reference.

gchar *transfer_full_function (void) {
	return g_strdup ("Newly allocated string.");
}

/* Ownership is transferred here. */
gchar *my_string = transfer_full_function ();

Conversely, if the function does retain a pointer (e.g. inside an object), ownership is not transferred to the caller.

const gchar *transfer_none_function (void) {
	return "Static string.";
}

const gchar *transfer_none_method (MyObject *self) {
	gchar *new_string = g_strdup ("Newly allocated string.");
	/* Ownership is retained by the object here. */
	self->priv->saved_string = new_string;
	return new_string;
}

/* In both of these cases, ownership is not transferred. */
const gchar *my_string1 = transfer_none_function ();
const gchar *my_string2 = transfer_none_method (some_object);

In all of these examples, you may notice the return type of the functions reflects whether they transfer ownership of their return values. const return types indicate no transfer, and non-const return types indicate some transfer.

While this is useful, it’s by no means completely clear. For example, what if a function returns a non-const GList*? Is ownership of the list elements transferred, or just the list? What if the function’s author forgot to make the return type const, and actually there’s no transfer? This is where documentation comments are useful.

There’s a convention in GNOME documentation comments to specify the function which should be used to free a returned value. If such a function is mentioned, ownership of the returned memory is transferred. If no function is mentioned, ownership probably isn’t transferred, but it’s hard to be sure. That’s why it’s good to always be explicit when writing documentation comments.

/**
 * my_object_get_some_string:
 * @self: a #MyObject
 *
 * Gets the value of the #MyObject:some-string property.
 *
 * Return value: (transfer none): some string, owned by the
 * object and must not be freed
 */
const gchar *my_object_get_some_string (MyObject *self) {
	return self->priv->some_string;
}

/**
 * my_object_build_result:
 * @self: a #MyObject
 *
 * Builds a result string which probably represents something
 * meaningful.
 *
 * Return value: (transfer full): a newly allocated result
 * string; free with g_free()
 */
gchar *my_object_build_result (MyObject *self) {
	return g_strdup_printf ("%s %s",
	                        self->priv->some_string,
	                        self->priv->some_other_string);
}

/**
 * my_object_dup_controller:
 * @self: a #MyObject
 *
 * Gets the value of the #MyObject:controller property,
 * incrementing the controller's reference count.
 *
 * Return value: (transfer full): the object's
 * controller; unref with g_object_unref()
 */
MyController *my_object_dup_controller (MyObject *self) {
	return g_object_ref (self->priv->controller);
}

When GObject introspection was introduced, these kinds of documentation comments were formalised as introspection annotations: (transfer full), (transfer container) or (transfer none), as documented on wiki.gnome.org. These allow the runtimes of language bindings to manage memory correctly. Since a C programmer is essentially doing the same job as a language runtime when writing C code, the information provided by transfer annotations is sufficient for perfect memory management. Unfortunately, not all parameters and return types of all functions have these annotations added. The examples above do, however.

Finally, a few libraries use a function naming convention. Functions named *_get_* do not transfer ownership, whereas functions named *_dup_* (for ‘duplicate’) do transfer ownership. This can be seen in the examples above, or with json_node_get_array() vs. json_node_dup_array(). Be aware, though, that only a few libraries use this convention. Other libraries use *_get_* for both functions which do and do not transfer ownership of their results. Other code, such as that generated by gdbus-codegen, uses a different and incompatible convention: *_get_* methods signify full transfer, and *_peek_* methods signify no transfer. For example, goa_object_get_manager() vs. goa_object_peek_manager(). For this reason, going by function naming conventions only works within libraries, not between different libraries.

Memory management of parameters is analogous to return values: look at whether the parameter is const and whether there are any introspection annotations for it.

In summary, here are a set of guidelines one can follow to determine whether ownership of a return value is transferred, and hence whether the caller needs to free it:

  1. If the type has an introspection (transfer) annotation, look at that.
  2. Otherwise, if the type is const, there is no transfer.
  3. Otherwise, if the function documentation explicitly specifies the return value must be freed, there is full or container transfer.
  4. Otherwise, if the function is named *_dup_*, there is full or container transfer.
  5. Otherwise, if the function is named *_peek_*, there is no transfer.
  6. Otherwise, you need to look at the function’s code to determine whether it intends ownership to be transferred. Then file a bug against the documentation for that function, and ask for an introspection annotation to be added.

Some common pitfalls:

  • If you’re using an explicit typecast (e.g. casting a (const gchar*) return value to (gchar*)), it’s likely something’s wrong.
  • Generally, return values which are not transferred (such as (const gchar*)) are freed when the owning object is destroyed — so if you need such a value to persist, you must copy it or increase its reference count. You then have ownership of the copy or new reference, and are responsible for freeing it (not the original).

How can one check for incorrect memory handling? Use Valgrind. It will detect leaks and double-frees, and is simple to use:

valgrind --tool=memcheck --leak-check=full my-program-name

Or, if running your program from the source directory, use the following to avoid running leak checking on the libtool helper scripts:

libtool --mode=execute valgrind --tool=memcheck --leak-check=full ./my-program-name

Valgrind lists each memory problem it detects, along with a short backtrace (if you’ve compiled your program with debug symbols), allowing the cause of the memory error to be pinpointed and fixed!