Tag Archives: autotools

Running GitLab CI on autotools projects

Inspired by the talk at FOSDEM, I’ve just enabled GitLab’s continuous integration (CI) for building make distcheck for Walbottle, and it was delightfully easy. The results are on Walbottle’s GitLab page.

Steps

  1. Create a ci branch to contain the mess you’ll make while iterating over the correct compile steps.
  2. Create and push a .gitlab-ci.yml file containing build rules similar to the following:
    image: debian:unstable
    
    before_script:
      - apt update -qq
      - apt install -y -qq build-essential autoconf automake pkg-config libtool m4 autoconf-archive gtk-doc-tools libxml2-utils gobject-introspection libgirepository1.0-dev libglib2.0-dev libjson-glib-dev
    
    stages:
      - build
    
    # FIXME: Re-enable valgrind once running the tests under it doesn’t take forever (it causes timeouts).
    # Re-add valgrind to apt-install line above
    build-distcheck:
      stage: build
      script:
        - mkdir build
        - cd build
        - ../autogen.sh --disable-valgrind
        - make V=1 VERBOSE=1
        - DISTCHECK_CONFIGURE_FLAGS=--disable-valgrind make distcheck V=1 VERBOSE=1
    
      # The files which are to be made available in GitLab
      artifacts:
        paths:
          - build/*
  3. Iterate a few times until you get all the dependencies right.
  4. Fix any problems you find (because this might well find problems with your dependency declaration in configure.ac, or other distcheck problems in your project).
  5. Merge ci to master and profit from CI results on every branch and master commit.

Looking at the .gitlab-ci.yml file

For information on the overall layout of the YAML file, and the phases available, you’re best off looking at the comprehensive GitLab documentation. Here are some notes about the autotools-and-C–specific bits of it:

  • The image is a Docker image; I picked a Debian one from the Docker hub.
  • Package installation seems to need to be done in the before_script phase, or the packages can’t be found (which is presumably a protection against rogue build systems).
  • I chose to build distcheck in my build rule because that runs the build, runs the tests, and tries various srcdir ? builddir configurations. You can add other build targets (like build-distcheck to try other build setups).
  • Pass V=1 VERBOSE=1 to get verbose build and test log output in your CI build logs, otherwise you will struggle to work out what is causing any failures.
  • Note that configure flags passed to ./configure are not automatically passed in again when ./configure is run as part of distcheck — so use DISTCHECK_CONFIGURE_FLAGS for that. Ideally, your project will be less fragile than mine, and hence not need any of this.
  • Export the whole build directory as an artifact on success, so you can look at any of the build objects, or the generated tarball, or documentation. You could limit this (for example, to just the tarball) if you’re sure you’ll never need the rest of it.

gnome-common deprecation, round 2

tl;dr: gnome-common is deprecated, but will be hanging around for a while. If you care about modernising your build system, migrate to autoconf-archive macros.

This GNOME release cycle (3.18), we plan to do the last ever release of gnome-common. A lot of its macros for deprecated technologies (scrollkeeper?!) have been removed, and the remainder of its macros have found better replacements in autoconf-archive, where they can be used by everyone, not just GNOME.

We plan to make one last release, and people are welcome to depend on it for as long as they like. However, if you want new hotness, port to the autoconf-archive versions of the macros; but please do it in your own time. There will be no flag day port away from gnome-common.

Note that, for example, porting to AX_COMPILER_FLAGS is valuable, but will probably require fixing a number of new compiler warnings in your code due to increased warning flags. We hope this will make your code better in the long run.

There’s a migration guide here: https://wiki.gnome.org/Projects/GnomeCommon/Migration.

We’ve tried to make the transition as easy and smooth as possible, but there will inevitably be hiccups. Please let me know about anything which breaks or doesn’t make sense, or discuss it on the desktop development list thread. First person to complain about -Wswitch-enum gets a prize.

For developers

When building from a tarball of a module which uses the new macros, you will no longer need gnome-common installed. (Although you may not have needed it before.)

When building from git, you will need m4-common or autoconf-archive installed.

JHBuild bootstrap installs m4-common automatically, as does gnome-continuous; so you don’t need to worry about that.

For packagers

In the 3.14.0 release, gnome-common installed some early versions of the autoconf-archive macros which conflicted with what autoconf-archive itself installs. It now has a --[with|without]-autoconf-archive configure option to control this. We suggest that all packagers pass --with-autoconf-archive if (and only if) autoconf-archive is packaged on the distribution. See bug #747920.

m4-common must not be packaged. See its README. m4-common is essentially a caching subset of autoconf-archive.

For continuous integrators

Modules which use the new AX_COMPILER_FLAGS macro gain a new standard --disable-Werror configure flag, which should be used in CI systems (and any other system where spurious compiler warnings should not cause total failure of a build) to disable -Werror. The idea here is that -Werror is enabled by default when building from git, and disabled by default when building from release tarballs and in buildbots.

For further discussion

See the thread on the desktop development mailing list.

Long live gnome-common? Macro deprecation

We’ve worked out the details, and have new recommendations for porting to autoconf-archive, including justifications and a migration guide. The information below is outdated.

gnome-common is shrinking, as we’ve decided to push as much of it as possible upstream. We have too many layers in our build systems, and adding an arbitrary dependency on gnome-common to pull in some macros once at configure time is not helpful — there are many cases where someone new has tried to build a module and failed with some weird autotools error about an undefined macro, purely because they didn’t have gnome-common installed.

So, for starters:

What does this mean for you, a module maintainer? Nothing, if you don’t want it to. gnome-common now contains copies of the autoconf-archive macros, and has compatibility wrappers for them.

In the long term, you should consider porting your build system to use the new, upstreamed macros. That means, for each macro:

  1. Downloading the macro to the m4/ directory in your project and adding it to git.
  2. Adding the macro to EXTRA_DIST in Makefile.am.
  3. Ensuring you have ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} in your top-level Makefile.am.
  4. Updating the macro invocation in configure.ac; just copy out the shim from gnome-common.m4 and tidy everything up.

Here’s an example change for GNOME_CODE_COVERAGE ? AX_CODE_COVERAGE in libgdata.

It seems from the comments that there’s more discussion to be had about the best way to implement this. So hold off on these changes for the moment!

This is the beginning of a (probably) long road to deprecating a lot of gnome-common. Macros like GNOME_COMPILE_WARNINGS and GNOME_COMMON_INIT are next in the firing line — they do nothing GNOME-specific, and should be part of a wider set of reusable free software build macros, like the autoconf-archive. gnome-common’s support for legacy documentation systems (DocBook, anyone?) is also getting deprecated next cycle.

Comments? Get in touch with me or David King (amigadave). This work is the (long overdue) result of a bit of discussion we had at the Berlin DX hackfest in May.