Monthly Archives: December 2012

Single-thread synchronisation in Vala

Update: You might also be interested in the follow-up article on rate limiting of asynchronous calls in Vala.

In libfolks, there are some asynchronous prepare() methods, which may be called multiple times, but must only change program state when called for the first time — subsequent calls must do nothing. As the prepare() method is asynchronous, and yields on other asynchronous methods, control may be yielded to the main loop part-way through running. At this point, another call to prepare() could happen from the same thread; and without adequate synchronisation the second call would proceed to change program state a second time. This is bad. (Throughout this post I’m only considering calls from a single thread; multi-threaded concurrency is completely different, and can rely on locking for safety.)

What are the possible solutions?

  • Locking.
  • Condition variables.
  • Boolean “prepare() in progress” flag.

Locking and condition variables can’t be used because the prepare() calls may come from any thread. Vala uses recursive locks, so calls from the same thread wouldn’t get blocked at all. If non-recursive locks were used, calls from the same thread would cause a deadlock.

A boolean “prepare() in progress” flag is what folks used to use. This was set to true as soon as the first asynchronous call began, so it prevented subsequent calls from changing program state erroneously, but also meant that subsequent calls returned immediately. Returning from a prepare() call typically implies that preparation has completed — but with this implementation that wasn’t correct for anything except the first call to prepare(). How can a call know that it’s not the first call? It can’t. This solution is unsatisfactory.

The solution which has just been implemented in folks is to use a list of waiting continuations for all calls subsequent to the first. The first call will execute all these continuations once it’s finished. This solution feels quite elegant to me, but suggestions from improvement are more than welcome.

A code example and explanation:

public class Logger : GLib.Object
  private static Logger _logger;
  private static DelegateWrapper[] _prepare_waiters = null;

  public async void prepare () throws GLib.Error
      if (Logger._logger == null && Logger._prepare_waiters == null)
          /* If this is the first call to prepare(), start some async calls. We
           * then yield to the main thread. Any subsequent calls to prepare()
           * will have their continuations added to the _prepare_waiters list,
           * and will be signalled once the first call returns. */
          Logger._prepare_waiters = new DelegateWrapper[0];

          /* Insert asynchronous initial preparation code here. */
          Logger._logger = yield new Logger ();

          /* Wake up any waiters. */
          foreach (unowned DelegateWrapper wrapper in Logger._prepare_waiters)
              wrapper.cb ();

          Logger._prepare_waiters = null;
      else if (Logger._logger == null && Logger._prepare_waiters != null)
          /* Yield until the first ongoing prepare() call finishes. */
          var wrapper = new DelegateWrapper ();
          wrapper.cb = prepare.callback;
          Logger._prepare_waiters += (owned) wrapper;

      /* Put shared code here which is executed on every call to prepare().
       * For example, it could check for failure in the results of the
       * initial call and throw an error as appropriate. */
      if (Logger._logger == null)
          /* Throw an error or something. */

/* See: */
private class DelegateWrapper
  public SourceFunc cb;

(For the full code to this example, see folks’ source code.) The first call to prepare() is meant to asynchronously set the static Logger._logger variable. Subsequent calls must not re-initialise this variable, but should block until the variable has been set.

How does this work? The first call to prepare() finds Logger._logger is null and Logger._prepare_waiters is also null, so it falls into the initialisation block. It sets Logger._prepare_waiters to be an empty array (which is distinct from null), then proceeds to initialise Logger._logger asynchronously. Since the initialisation is asynchronous, control returns to the main loop, and a second call to prepare() could be made. Assume it is. The second call finds Logger._logger is still null, but finds that Logger._prepare_waiters is non-null, and so falls into the second branch of the if-statement. This saves its continuation into the array of waiters, and then yields control to the main loop again, indefinitely delaying the second asynchronous call from finishing. The first call returns from its asynchronous initialisation code, and proceeds to “Wake up any waiters”. For each of the saved waiters, it calls the waiter’s continuation (in an idle callback), which resumes execution just after the yield statement. Consequently, each waiter executes the shared code at the bottom of prepare(), then its prepare() call finishes and any asynchronous callbacks are executed in normal GLib fashion. The first call then proceeds to the shared code at the bottom of prepare(). Finally, its asynchronous callback is executed, and everything’s finished.

This all hinges on the fact that Vala supports “delegates with targets” (or continuations): the prepare.callback in the code above. This is a copy of the runtime state of the prepare() call, complete enough that it can later be called (using wrapper.cb()) to continue execution as if it never stopped at the yield. In C terms, a delegate is a function pointer; a delegate with a target is a function pointer with a struct full of the values of local variables at the time the delegate was stored. This is similar to the this pointer in C++, but includes more local state.

Why is DelegateWrapper needed? Why can’t the array just contain the continuations directly? Good question. I’d like to know the answer. Vala throws an error message if one tries to define an array of delegates with targets (“Delegates with target are not supported as array element type”). Other methods of encapsulating the continuations result in just the delegate being stored, and the target (local state) being lost, which is not what we want. I got the tip to use a DelegateWrapper class from Jim Nelson in this mailing list post (thanks!).

So that’s it: one way to synchronise multiple concurrent asynchronous method calls in Vala.

Adventures with an ATmega

As part of some university coursework, I’ve recently been playing around with an Atmel 8-bit microcontroller in a quest to build a data logger for home brewing. The aim is to log various sensor readings over the course of primary fermentation of a batch of beer to see how it’s progressing without having to disturb the brew.

While working with hardware is fun, it has the downside that you’re working with hardware. Things break as much as they do in software, but are harder to debug. A nice solution to this is to use a simulator to test the software, rather than using the hardware itself. I’ve been using the simavr simulator. It’s got all the right features, but more importantly its code is easy to understand and extend. In fact, most of the simulator is written as a library which one can build a simulated circuit board around.

I’ve built simulations of several of the components used in the brewing logger now, including an SD card and a flash memory. The code isn’t brilliant, but they’re working well enough to get my firmware booting. Hopefully they might be useful to other people too — hence the code is on gitorious. The SD card simulator is for a generic SDv2 card. The flash memory is for an ST M25P16 (but should support any of the M25P family, with a little work). The serial LCD is for a custom LCD daughter board provided by my university, so that’s probably not so useful. The RHT03 temperature/humidity sensor is general purpose; as is the DS3231 real-time clock.

Put together, and with a little extra work, these will allow closed loop testing of the microcontroller firmware — without ever having to go near the hardware itself. Bliss. (This is all under the misguided assumption that the simulations are sound and complete, but they’ll get there eventually.)

As an example of how easy simavr makes this, take a look at the code needed to instantiate and hook up the humidity sensor simulation:

/* Create the sensor. */
rht03_t rht03;
rht03_init (avr, &rht03);

/* Connect its bidirectional data pin to pin D3 on the microcontroller. */
avr_connect_irq (avr_io_getirq (avr, AVR_IOCTL_IOPORT_GETIRQ ('D'), 3), rht03.irq + IRQ_RHT03_DATA);
avr_connect_irq (rht03.irq + IRQ_RHT03_DATA, avr_io_getirq (avr, AVR_IOCTL_IOPORT_GETIRQ ('D'), 3));