Tag Archives: automake

Long live gnome-common? Macro deprecation

gnome-common is shrinking, as we’ve decided to push as much of it as possible upstream. We have too many layers in our build systems, and adding an arbitrary dependency on gnome-common to pull in some macros once at configure time is not helpful — there are many cases where someone new has tried to build a module and failed with some weird autotools error about an undefined macro, purely because they didn’t have gnome-common installed.

So, for starters:

What does this mean for you, a module maintainer? Nothing, if you don’t want it to. gnome-common now contains copies of the autoconf-archive macros, and has compatibility wrappers for them.

In the long term, you should consider porting your build system to use the new, upstreamed macros. That means, for each macro:

  1. Downloading the macro to the m4/ directory in your project and adding it to git.
  2. Adding the macro to EXTRA_DIST in Makefile.am.
  3. Ensuring you have ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} in your top-level Makefile.am.
  4. Updating the macro invocation in configure.ac; just copy out the shim from gnome-common.m4 and tidy everything up.

Here’s an example change for GNOME_CODE_COVERAGE ? AX_CODE_COVERAGE in libgdata.

It seems from the comments that there’s more discussion to be had about the best way to implement this. So hold off on these changes for the moment!

This is the beginning of a (probably) long road to deprecating a lot of gnome-common. Macros like GNOME_COMPILE_WARNINGS and GNOME_COMMON_INIT are next in the firing line — they do nothing GNOME-specific, and should be part of a wider set of reusable free software build macros, like the autoconf-archive. gnome-common’s support for legacy documentation systems (DocBook, anyone?) is also getting deprecated next cycle.

Comments? Get in touch with me or David King (amigadave). This work is the (long overdue) result of a bit of discussion we had at the Berlin DX hackfest in May.

Notes on Vala and automake

Here are some brief notes about getting automake’s Vala support1 to play nicely with you when writing a library, mostly for my own future benefit.

Firstly, use target-specific *_VALAFLAGS variables (rather than AM_VALAFLAGS) to specify the flags to pass to the valac compiler. Doing this enables a couple of dist and clean features in automake which wouldn’t otherwise be enabled. It’s also means you can specify multiple targets in a single Makefile.am, which is good for non-recursive automake.

When building a library, explicitly specify --vapi, --header, --internal-vapi, --internal-header and --gir (as appropriate) in the *_VALAFLAGS variable directly, rather than including them through another variable. automake looks for these flags in order to build dist and clean rules for the generated VAPI, C header and GIR files.

automake will automatically delete .stamp, VAPI and generated C header and source files on maintainer-clean, but not as part of any other clean target — this is because automake is designed to ensure generated C files are included in the distribution tarball. Failing to specify --header, --vapi (etc.) directly in *_VALAFLAGS will cause the appropriate maintainer-clean rules to not be generated.

automake knows nothing about .deps files (which accompany .vapi files), and will not generate dist rules for them; that must be done manually using:

vapidir = $(datadir)/vala/vapi
dist_vapi_DATA = my-library.deps

Source: lang_vala_finish_target() in the automake-1.13 script. Read it: it’s quite clear and informative.


  1. Note: That documentation seems to be a bit out of date regarding VALAFLAGS

Colourful GCC output

For a long time, I’ve used colorgcc as a way to highlight warnings and errors in GCC output. It’s worked wonderfully, but today Travis pointed out that GCC has grown an option to enable colourised output: -fdiagnostics-color. This will be available in GCC 4.9, but the patch seems to have been backported to Fedora 19 so that more people can join in the fun.

This means I can throw away colorgcc from my .jhbuildrc:

os.environ['CC'] = '/usr/bin/color-gcc'

and enable -fdiagnostics-color:

os.environ['CFLAGS'] += ' -fdiagnostics-color=auto'

Even better, adding the following snippet to configure.ac in a standard automake project will automatically enable colourised compiler output if the compiler (is GCC and) supports it:

AS_COMPILER_FLAG([-fdiagnostics-color=auto],
                 [AM_CFLAGS="$AM_CFLAGS -fdiagnostics-color=auto"])

(In actual fact, I used ERROR_CFLAGS for this in folks, to keep unrelated CFLAGS separated, but that’s an implementation detail of folks’ build system.)

The last GUADEC post?

GUADEC's over (and has been so for a while) and was great. The talk videos are up, and still I haven't blogged about the conference. Naughty me.

The conference was great: it was good to meet up with friends (old and new) again, get a bit of hacking done and hear lots of discussion about the future of GNOME. My thanks to the GNOME Foundation and my summer employer, Collabora, for sponsoring my accommodation and travel. Thank you to the local team for organising a brilliant GUADEC in a nice city, and here's to a successful GCDS 2011 in Berlin!

In other news, I've finally got fed up with forgetting to add new files in my project to either the project-wide header file or the documentation index, so I wrote some `make check` rules which will check for my braindeadness:

This one goes in the Makefile.am in the directory which builds your public header (such as gtk/gtk.h), and assumes a list of all the headers you're going to install is in public_headers, and that your main header's path is in main_header.

check-local: check-headers
check-headers:
	@any_missing=0; for header_file in $(public_headers); do \
		if test "x\(header_file" != "x$(main_header)"; then \
			if ! grep "#include <\)header_file>" $(main_header) >/dev/null; then \
				echo "$(main_header) doesn't appear to include \"\(header_file\""; \
				any_missing=1; \
			fi; \
		fi; \
	done; exit "\)any_missing"

This one goes in the Makefile.am in your gtk-doc directory (e.g. docs/reference), and only assumes the existence of DOC_MAIN_SGML_FILE, which needs to be defined for gtk-doc anyway.

check-local: check-xml-includes
check-xml-includes:
	@any_missing=0; find $(srcdir) -name "*.xml" | while read x; do \
		xml_file="\({x#./}"; \
		if test "x\)xml_file" != "x$(DOC_MAIN_SGML_FILE)"; then \
			if ! grep "\"\(xml_file\"" $(DOC_MAIN_SGML_FILE) >/dev/null; then \
				echo "$(DOC_MAIN_SGML_FILE) doesn't appear to include \"\)xml_file\""; \
				any_missing=1; \
			fi; \
		fi; \
	done; exit "$$any_missing"

I've filed bgo#627920 about adding the second rule to gtk-doc itself. I'm not sure the first rule is general enough to be put anywhere common.

Non-recursive automake

A while back, I was toying with the idea of adding gcov support to libgdata, to give metrics on the code coverage of the test suite. I played around with adding it, but my attempts were thwarted by the fact that I couldn't easily get a list of all C files which were to be compiled, in all source directories, to use in the top-level Makefile.

I recently found a little time to work on this some more and, as usually happens, it ballooned into something bigger, and I ended up redoing the entire build system for libgdata. It now uses non-recursive automake, which makes things a lot simpler in many respects. The diffstat was favourable ("21 files changed, 641 insertions(+), 833 deletions(-)"), and it has made the autogen/clean/build cycle about 20 seconds faster (on average, from some very rough tests). Accomplishing this was largely possible due to the (fairly) recent literature on the subject from Murray Cumming and Daniel Elstner, which proved rather useful; so many thanks to them.

As a result of all this, I now know that only just over 50% of libgdata's code was actually being exercised before by the test suite. I've now pushed this up to 60%, and uncovered a few buglets in the process. I dread to think what other problems lie in the remaining 40%; there's certainly lots of work left to be done.

Anyway, since all the snippets of automake magic I could find were subtly incompatible with non-recursive automake, I ended up with a slightly different one (based on gobject-introspection's gcov support). Hopefully it's useful to someone, though it could probably do with some tidying up:

if GCOV_ENABLED
gcov-report.txt: gcov-clean all check
	$(AM_V_GEN)(rm -f $@; \
	echo -e "Test coverage for libgdata:\n" >> $@; \
	total_covered=0; total_actual=0; \
	for file in $(filter %.c,$(gdata_libgdata_la_SOURCES)); do \
	  file2=\({file##*/}; \
	  cd $(abs_srcdir)/\){file%/*}; \
	  gcov -o `find -newer \(file2 -name "*-\){file2/.c/.gcda}" -print0 | sed -e 's/\.gcda/\.o/'` \(file2 > /dev/null; \
	  if test -f \)file2.gcov; then \
	    actual=`grep -v '        -:' \(file2.gcov  | wc -l`; \
	    uncovered=`grep '#####:' \)file2.gcov | wc -l`; \
	    covered=\(((actual - uncovered)); \
	    total_covered=\)((total_covered + covered)); \
	    total_actual=\(((total_actual + actual)); \
	    echo -e "\)file:\t\(covered / \)actual\t(\((((\)covered * 100) / \(actual))%)"; \
	  fi \
	done >> $@; \
	cd $(abs_srcdir); \
	echo -e "\nSource lines: \)total_actual\nCovered statements: \(total_covered\nTotal coverage: \)(((\(total_covered * 100) / \)total_actual))%" >> $@)

gcov: gcov-report.txt
	@cat gcov-report.txt

clean: gcov-clean
gcov-clean:
	@find . -name "*.gcda" -o -name "*.gcov" -delete

MAINTAINERCLEANFILES += gcov-report.txt

.PHONY: gcov gcov-clean gcov-report.txt
else
gcov:
	@echo "Need to reconfigure with --enable-gcov"
endif

It will output a code coverage report of the test suite (`make check`) when `make gcov` is called, and will also save it as "gcov-report.txt" in the root source directory.

Auto-generated class hierarchy diagrams

Unfortunately still working on that coursework, I've just added support in the API documentation makefile to automatically generate a class hierarchy diagram using the GObject hierarchy file created by gtk-doc and marshalling it to GraphViz. It basically analyses the depth of each line in the hierarchy file and creates relationships in the dot file as appropriate. It doesn't sound complex, but it took a heck of a lot of fiddling to get automake happy with my syntax.

graph-build.stamp:
	echo "digraph class_hierarchy" > $(DOC_MODULE).dot
	echo "{" >> $(DOC_MODULE).dot
	(IFS=$$'\n'; \
	for current_line in `cat $(DOC_MODULE).hierarchy`; do \
		depth_colours=( red green blue yellow ); \
		depth=`echo $$current_line | grep -o "  " | wc -l | sed s/\ //g`; \
		echo "$$current_line [shape=box, color=$${depth_colours[$$depth]}]" >> $(DOC_MODULE).dot; \
		last_line[$$depth]=$$current_line; \
		if [ $$depth -gt 0 ]; then \
			echo "$${last_line[`expr $$depth - 1`]} -> $$current_line" >> $(DOC_MODULE).dot; \
		fi; \
	done)
	echo "}" >> $(DOC_MODULE).dot
	dot -Tpng $(DOC_MODULE).dot > $(DOC_MODULE).png
	touch graph-build.stamp

Building PDFs of documentation

Still working on that coursework project, I'm now building PDFs of the three pieces of documentation the application itself requires: the API reference, the source code listings and the user manual.

Using gtk-doc to build the API reference makes it quite easy to add something in the Makefile to build a PDF from the DocBook:

docs/reference/Makefile.am

I've put this at the bottom of the Makefile.am, and it's all working nicely, apart from the fact that distcheck (legitimately) complains about me redefining all-local. I can't find any other way to make it build the PDF while also building the normal docs with make docs, so my question here is: how do I make this implementation cleaner?

If anybody wants to use it, they'll need to install dblatex.

The next piece of documentation is the source code listing. This is completely custom, using source-highlight to make LaTeX files for each source file, and then pdflatex to build them into a PDF.

docs/source/Makefile.am

I'm quite pleased with the way this works, although I'm sure there will be better solutions out there. source-highlight deposits many LaTeX files in the build directory, which are then included in the PDF build by dint of being automatically concatenated together and included by a surrounding LaTeX file, project-name-source.tex.

docs/source/two-hundred-club-source.tex

Finally, the user manual also needs building into a PDF. This again uses dblatex, although it's a little more complex, as a PDF has to be built for each language directory in the help directory.

docs/help/Makefile.am

Hopefully some of that will be helpful to somebody in a similar situation, and I'd welcome any hints or pointers as to how to improve the implementations, as my automake powers are about as strong as a damp tissue.