Category Archives: GNOME

Posts about GNOME and programming for it.

GUADEC 2023

Leave a Reply

I attended GUADEC 2023 this year in-person in Rīga. It was nice to be able to see more people in person than last year’s mini-GUADEC in Berlin, and nice to be able to travel overland/oversea to Rīga, avoiding flights and the associated huge carbon emissions. Thank you to the GNOME Foundation and my employer, Endless, for covering the travel.
And a big thank you to the local event organising team, the AV team, the volunteers and the Foundation staff for making it all happen.

The quality of the talks this year was really high. I don’t think there was a single talk slot I skipped. As a result, I didn’t get much hacking done! But there were good hallway conversations and catch ups.

I gave two talks, one on some simple improvements people can make to their apps to reduce internet data use and power use when doing so would be beneficial to the user (when on a metered network or in power-saver mode).
The aim was to remind people how easy it is to do this, and provide some examples of how different apps present these states/events in the UI, since the best way to do that can differ between apps.

You can find the slides to that talk here (notes here), and a video of it is on YouTube.

The talk has resulted in three GNOME initiatives. Each corresponds to a different state your app can pay attention to — some apps should pay attention to all of them, some apps only some of them. Please take 10 minutes to
check your app against the initiatives and make updates if necessary. I’m in the gnome-hackers and gnome-circle Matrix rooms if you have any questions about them.

My second talk was an overview of work I’ve been doing on-and-off over the past couple of years (based on work by others) to allow apps to save and restore their state when the user logs out or shuts down the computer. The idea is that the user can restore
the set of apps they were using, in the state and configuration they were left in, when next starting the computer or logging in. It’s a big project, and there’s a long way to go, but it felt like the right time to present what
we’ve got so far, let more people know how the project is structured, and get feedback from toolkit developers and app authors about the whole idea.

You can find the slides to that talk here (notes here), and a video of it is also on YouTube.

Thankfully there was 5 minutes for questions at the end of the talk, and people used them well to raise some good points and ask some good questions. I’m still in the process of factoring all that feedback into the plan, but
should hopefully have an update to give on the project in a future blog post soon.

Interesting talks I attended included Peter Hutterer’s talk about sending and capturing input in Wayland, which I think distilled his blogposts about the topic very clearly.
Allan Day’s talk about communication, which was an excellent summary of a rather long list of books on the subject, and it was very clearly presented. I feel like I could do with a cheatsheet list of his recommendations to stick next to my computer sometimes.
Evan Welsh, Philip Chimento, Nasah Kuma and Sonny Piers’s talks about JavaScript, TypeScript and the latest changes in GJS. These provided a good update on a lot that has been happening in GJS, a useful overview of TypeScript, and were really clearly presented.
Jussi Pakkanen’s talk about circles, or was it about settings? It was a good example of a lightning talk which draws you in then spins you around.

In the hallway track, I had some interesting chats with Bartłomiej about the infrastructure of ODRS, which is used to provide ratings/review data to gnome-software.
I also had some positive conversation with Cassidy about plans for GUADEC 2024 in Denver.
And at the event dinner, a really energising chat with Scott about canyoning, hiking, skiing and caving in the US and the UK.

Carbon emissions analysis of GUADEC 2022

Leave a Reply

I’ve just finished estimating the carbon emissions from GUADEC 2022, and have a few interesting highlights from the report. The report is based on data collected from the streaming servers, registration data, and the post-conference survey.

  • Having an online component to the conference increased the audience by a factor of 10: there were around 120 in-person attendees in Mexico, but there were an average of 1300 people using Big Blue Button.
  • The carbon emissions from providing the remote infrastructure were around 2.8tCO2e, or about 2kgCO2e per remote attendee.
  • Having a remote attendance party in Berlin allowed around one tenth of the attendees to attend with a factor of 10 lower transport emissions than those who attended in-person in Mexico. The average transport emissions for those who went to Berlin were 88kgCO2e, whereas they were around 1tCO2e for those going to Mexico.
  • For context, the annual emissions per person can be at most 2.3tCO2e by 2030 in order to limit global warming to 1.5C by the end of the century. That covers all food, travel, heating, purchases, etc. So travel to Mexico was 40% of the average attendee’s annual target.
  • Half of the in-person attendees travelled from within Mexico, which will have skewed the mean transport emissions downwards. The distribution is more likely bimodal between around 50kgCO2e for locals and more like 2-3tCO2e for those coming from outside Mexico.
  • As I wrote at the time, the remote attendance party was fun to attend and felt like it worked as a way to attend the conference (there were no A/V problems, we had some nice local socials, etc.). I would do it again.
  • The post-conference survey had a low response rate of about 19% of registered attendees, which made some of this analysis hard. Please always fill in the post-conference survey! It doesn’t take long, and aside from any of this analysis, it helps the organisers plan conferences better in future.
  • Many thanks to Kristi, Caroline and Bartłomiej for collecting the data needed for this analysis.
  • If anyone spots problems in the analysis please say! This is not the kind of thing I practice doing very often and I’ve had to make a number of assumptions.

Other conferences

A post-conference survey has been done for other big GNOME events, such as GNOME.Asia and LAS. Would anyone be interested in doing a similar analysis for those events? Perhaps we can get a semi-automated pipeline in place for it.

Getting from the UK to Riga for GUADEC 2023

I’ve just booked travel for getting to Riga, Latvia for GUADEC 2023, and I thought I’d quickly write up my travel plans in case it saves someone else some time in planning travel.

I am not flying, because planes are too polluting. Instead, I am taking the train to Lübeck in Germany, then an overnight ferry to Liepāja, and then a bus the following morning to Riga. It’s a bit slower, but means it’s a bit easier to get some hacking done, stretch my legs and move around, and not fund fossil fuel companies as much. I’ll get enough stopover time in Köln, Hamburg and Lübeck to quickly look round, and a night in Liepāja to see it.

Overall the travel time is just over 2 days, with half of that spent on trains, and half on a ferry. By comparison, a flight is about 7 hours (5 hours flying, 2 hours faffing in airports) plus travel time to the airport.

The carbon emissions (140kgCO2e return) are roughly a quarter of those from flying (520kgCO2e), and interestingly a significant part of those emissions (46kgCO2e) is the 3 hour bus journey to get from Liepāja to Riga, though that’s quite sensitive to the occupancy level of the bus.

The financial cost (£800 return) is about two times that of flying (£380), though I have not factored in the costs of getting to/from airports and have not fully explored the hidden fees for baggage and other essentials so the ratio might be a little lower. This is quite upsetting. A disproportionate part of the cost (£178 return) is the Eurostar, because it’s oversubscribed and I missed the early ticket releases due to waiting for grant approval. Perhaps I should not wait next time.

The journey

On 2023-07-22:

  • Eurostar from London to Brussels-Midi, departing 07:04
  • Train from Brussels-Midi to Lübeck-Travemünde Skandinavienkai, departing 10:25 (ICE 15, ICE 200, RE 11428, RB 11528)
  • Nice walk from there to the ferry terminal for half an hour
  • Overnight ferry from Lübeck/Travemünde to Liepāja, departing 23:30

On 2023-07-23: On the ferry all day, then stay overnight in Liepāja

On 2023-07-24: Bus from Liepāja to Riga, departing early morning

Alternatives

I strongly looked at taking the train from Hamburg to Stockholm, and then the ferry from there to Ventspils. Unfortunately, it has limited capacity and there is track maintenance planned for around my travel dates, so I could not get suitable tickets. It would have made the timings a little more convenient overall, for about the same overall carbon emissions and cost.

Join me

If anybody else is going overland from the UK or far-western Europe, this is hopefully a sensible route for you to take, and it would be lovely if you wanted to join me. I will be arriving 2 days early for GUADEC (as we’re having an Endless OS Foundation meetup), but if you wanted to do the same journey 1 or 2 days later then it shouldn’t differ significantly. In any case, I can put you in touch with others making this journey if you want.

Tip for debugging refcounting issues: change ref calls to copy

Over the last couple of days I’ve been looking at a refcounting issue in GLib’s D-Bus implementation.

As with many things in GLib, the allocations in this code use refcounting, rather than new/free, to manage object lifecycles. This makes it quite hard to debug lifecycle issues, because the location of a bug (a ref or unref call which isn’t correct) can be quite far removed (in time and code) from where the effects of that bug become visible. This is because the effects of refcounting problems only become visible when an object’s refcount reaches zero, or when the program ends and its refcount still hasn’t reached zero.

While debugging this code, I tried an approach I haven’t before: changing some of the ref calls on the buggy object to be copy calls instead. (Specifically, changing g_object_ref() to g_dbus_message_copy().) That split up the lifecycle of the object into smaller pieces, narrowing down the sets of ref/unref calls which could be buggy. Ultimately, this allowed me to find some bugs in the code, and hopefully those are the bugs causing the refcounting issue. Since the issue is intermittent, it’s a bit hard to be sure.

This debugging approach was possible in this case because the object I was debugging is immutable, so passing around copies of it doesn’t affect the behaviour of other bits of code vs passing around the original. Hence this approach is only applicable in some situations. But it’s another good reason why using immutable objects is quite helpful when writing code, and it’s certainly an approach I’m going to be using again when I can.

Mini-GUADEC 2022 Berlin: retrospective

I’m really pleased with how the mini-GUADEC in Berlin turned out. We had a really productive conference, with various collaborations speeding up progress on Files, display colour handling, Shell, adaptive apps, and a host of other things. We watched and gave talks, and that seemed to work really well. The conference ran from 15:00 to 22:00 in Berlin time, and breaks in the schedule matched when people got hungry in Berlin, so I’d say the timings worked nicely. It helped to be in a city where things are open late.

c-base provided us with a cool inside space, a nice outdoor seating area next to the river, reliable internet, quality A/V equipment and support for using it, and a big beamer for watching talks. They also had a bar open later in the day, and there were several food options nearby.

At least from our end, GUADEC 2022 managed to be an effective hybrid conference. Hopefully people in Guadalajara had a similarly good experience?

Tobias and I spent a productive half a day working through a load of UI papercuts in GNOME Software, closing a number of open issues, including some where we’d failed to make progress for months. The benefits of in-person discussion!

Sadly despite organising the mini-GUADEC, Sonny couldn’t join us due to catching COVID. So far it looks like others avoided getting ill.

Club Mate at c-base.
View of the river Spree and Jannowitzbrücke station from the outdoor area behind c-base.

Travel

Allan wrote up how he got to Berlin, for general reference and posterity, so I should do the same.

I took the train from north-west England to London one evening and stayed the night with friends in London. This would normally have worked fine, but that was the second-hottest day of the heatwave, and the UK’s rails aren’t designed for air temperatures above 30°C. So the train was 2.5 hours delayed. Thankfully I had time in the plan to accommodate this.

The following morning, I took the 09:01 Eurostar to Brussels, and then an ICE at 14:25 to Berlin (via Köln). This worked well — rails on the continent are designed for higher temperatures than those in the UK.

The journey was the same in reverse, leaving Berlin at 08:34 in time for a 18:52 Eurostar. It should have been possible to then get the last train from London to the north-west of England on the same day, but in the end I changed plans and visited friends near London for the weekend.

I took 2 litres of water with me each way, and grabbed some food beforehand and at Köln, rather than trying to get food on the train. This worked well.

Within Berlin, I used a single 9EUR monatskarte for all travel. This is an amazing policy by the German government, and subjectively it seemed like it was being widely used. It would be interesting to see how it has affected car usage vs public transport usage over several months.

Eurostar train waiting on the platform at London St Pancras.

Climate

Overall, I estimate the return train trip to Berlin emitted 52kgCO2e, compared to 2610kgCO2e from flying Manchester to Guadalajara (via Houston). That’s an impact 50× lower. 52kgCO2e is about the same emissions as 2 weeks of a vegetarian diet; 2610kgCO2e is about the same as an entire year of eating a meat-heavy diet.

(Train emissions calculated one-way as 14.8kgCO2e to London, 4.3kgCO2e to Brussels, 6.5kgCO2e to Berlin.)

Tobias gave an impactful talk on climate action, and one of his key points was that significant change can now only happen as a result of government policy changes. Individual consumer choices can’t easily bring about the systemic change needed to prevent new oil and coal extraction, trigger modal shift in transport use, or rethink land allocation to provide sufficient food while allowing rewilding.

That’s very true. One of the exceptions, though, is flying: the choices each of the ~20 people at mini-GUADEC made resulted in not emitting up to 50 tonnes of CO2e in flights. That’s because flights each have a significant emissions cost, and are largely avoidable. (Doing emissions calculations for counterfactuals is a slippery business, but hopefully the 50 tonne figure is illustrative even if it can’t be precise.)

So it’s pretty excellent that the GNOME community supports satellite conferences, and I strongly hope this is something which we can continue to do for our big conferences in future.

Tourism

After the conference, I had a few days in Berlin. On the recommendation of Zeeshan, I spent a day in the Berlin technical museum, and another day exploring several of the palaces at Potsdam.

It’s easy to spend an entire day at the technical museum. One of the train sheds was closed while I was there, which is a shame, but at least that freed up a few hours which I could spend looking at the printing and the jewellery making exhibits.

One of the nice things about the technical museum is that their displays of old machinery are largely functional: they regularly run demonstrations of entire paper making processes or linotype printing using the original machinery. In most other technical museums I’ve been to, the functioning equipment is limited to a steam engine or two and everything else is a static display.

The palaces in Potsdam were impressive, and look like a maintenance nightmare. In particular, the Grotto Hall in the Neues Palais was one of the most fantastical rooms I’ve ever seen. It’s quite a ridiculous display of wealth from the 18th century. The whole of Sanssouci Park made another nice day out, though taking a picnic would have been a good idea.

Printing exhibition in the technical museum, with a linotype machine.
Jewellery making preparation area in the technical museum. Various jewellery machines.
Steam train 012 082-4 in the technical museum, dating from 1940. It’s a class 01.10.
Neues Palais in Potsdam. A two-winged building with 2 storeys and 16 bays, and a copper roof dome.
Grotto Hall in the Neues Palais. A 240m2 hall decorated with stones, shells, marble and quartz.
Close up of some of the stone/quartz decoration in the Grotto Hall. Bands of rough stones alternating with marble bands up a column.

Thanks!

Thanks again to everyone who organised GUADEC in Guadalajara, Sonny and Tobias for organising the mini-GUADEC, the people at c-base for hosting us and providing A/V support, and the GNOME Foundation for sponsoring several of us to go to mini-GUADEC.

Sponsored by GNOME Foundation

Looking at project resource use and CI pipelines in GitLab

While at GUADEC I finished a small script which uses the GitLab API to estimate the resource use of a project on GitLab. It looks at the CI pipeline job durations and artifact storage for the project and its forks over a given period, and totals things.

You might want to run it on your project!

It gives output something like the following:

Between 2022-06-23 00:00:00+00:00 and 2022-07-23 00:00:00+00:00, GNOME/glib and its 20 forks used:

  • 4592 CI jobs, totalling 17125 minutes (duration minimum 0.0, median 2.3, maximum 65.0)
  • Total energy use: 32.54kWh
  • Total artifact storage: 4426 MB (minimum 0.0, median 0.2, maximum 20.9)

This is useful for giving a rough look at the CI resources used by a project, which could be useful for noticing low-hanging fruit for speeding things up or reducing resource waste.

What can I do with this information?

If total pipeline durations are long, either reduce the number of pipeline jobs or speed them up. Speeding them up almost always has no downsides. Reducing the number of jobs is a tradeoff between convenience of development and resource usage. Two ideas for reducing the number of jobs are to make some jobs manual-only, if they are very unlikely to find problems. Or run them on a schedule rather than on every commit, if it’s OK for them to catch problems up to a week after they’re introduced.

If total artifact storage use is high, store fewer artifacts, or expire them after a week (or so). They are likely not so useful after that point anyway.

If artifacts are being used to cache build dependencies, then consider moving those dependencies into a pre-built container image instead. It may be cached better between CI runners.

This script is rubbish, how do I improve it?

Merge requests welcome on https://gitlab.gnome.org/pwithnall/gitlab-stats, or perhaps you’d like to integrate it into cauldron.io so that the data could be visualised over time? The same query code should work for all GitLab instances, not just GNOME’s.

How does it work?

It queries the GitLab API in a few ways, and then applies a very simple model to the results.

It can take a while to run when querying for large projects or for periods of over a couple of weeks, as it needs to make a REST request for each CI job individually.

Mini-GUADEC 2022 in Berlin

GUADEC 2022 has been happening in person for the first time in two years, in Guadalajara. Twenty of us in Europe met up in Berlin for a mini-GUADEC, to attend the main conference remotely. There have been several talks given from here using the nice A/V setup in c-base, who are hosting us.

I gave my talk this afternoon, on the threading rework which is ongoing in gnome-software. The slides are here, the notes are here (source is here), and the recording should be available soon on the GUADEC YouTube channel.

As part of the question and answer session afterwards, it was suggested that it might be helpful to write a blog post about strategies for making async code in C more readable. I’ll try and write something about that soon.

Thanks to the GUADEC organising team for hosting the conference and integrating remote participation so well, to Sonny and Tobias for organising the Berlin mini-GUADEC, and to the c-base technical people for setting things up nicely here.

State persistence for apps and sessions: Endless Orange Week

The second part of my project in Endless Orange Week was to look at state persistence for apps and sessions. At its core, this means providing

  • some way for apps to save and restore their state (such as which part of the UI you’re looking at, what’s selected, and unsaved content);
  • code in the session manager to save and restore the state of all applications, and a list of which applications are running, when shutting down/restarting/logging out/logging in/starting up.

Those two bullet points hide a lot of complexity, and it’s not surprising that I didn’t get particularly far in this project! It requires coordinated changes in a lot of components: GLib, GTK, gnome-session and applications themselves.

A lot of these changes have been prototyped or worked on before, by various people, but nothing has yet come together. In fact, gnome-session used to fully support restoring apps to a certain degree — before it was ported away from XSMP, it used to support saving the set of apps when closing a session, and re-starting those apps when starting the session again. It did not support restoring the state of each app, though, just the fact that it was running.

Breaking down the problem

Updating and building on the proposals and branches from others so far, we end up with:

  • Changes to GLib to add a ‘restart data’ property to GApplication, which allows the application to expose its current state to the session manager (to be saved), and which is initialised with the restored state from the session manager on startup. These build heavily on changes proposed by Bastien Nocera, but my tweaks to them are still pretty exploratory and not ready for review.
  • Code in GTK to support serialising the widget tree. This implements saving the state of the UI, and was prototyped by Matthias Clasen. My additions (also not yet ready for review) tie it in to the gnome-session API. Further work (but not very much of it) would have to be done to tie Matthias’ proposals into the final shape of the GLib API.
  • Preparatory cleanups of old code in gnome-session (this one is ready to review and hopefully merge!).
  • Work to re-support session restore in gnome-session. This is mostly ready, but needs tidying up and testing (which is likely to be a significant amount of work). It ties in with systemd transient scope work which Benjamin Berg and Iain Lane have been working on.

The final two pieces of the puzzle above took most of my week, and included a lot of time spent learning the architecture of gnome-session, and working out a bit of a protocol for apps to register for session restore and, in particular, for gnome-session to be able to re-launch apps the right way when restoring a session. That’s something which deserves a blog post of its own at some point in the future, once I’m sure I’ve got everything straight in my head.

In summary, there’s not been much progress in terms of creating merge requests. But the week was, I think, well spent: I’ve learned a lot about the shape of the problem, and feel like I have a better idea of how to split it up into chunks which it might be possible to land over time to get the feature working. Many thanks to Endless for giving me the opportunity to do so.

Certainly, I think this project is too big to easily do in a single GNOME release. There’s too much coordination required between different projects with different cadences and development resources.

The next step will be to land the preparatory gnome-session cleanups, and to discuss and land the GLib API so that people can start building with that.

Runtime control of debug output: Endless Orange Week

Recently at Endless we had a week of focused working on projects which are not our day-to-day work. It was called ‘Endless Orange Week’, and everyone was encouraged to explore a topic of their choosing.

I chose to look at two projects, both of which included a D-Bus/API component. My thinking was that review of the new interfaces on each project might take a while, so it would make sense to have two projects running in parallel so I could switch between them when blocked.

I’m going to blog about the two projects separately, to avoid one mega-long post.

The first project was to add a D-Bus debug interface for applications. This would allow debug output from an application to be turned on and off at runtime, rather than just being set with a command line argument or environment variable when the application is first started.

This would allow users and developers to get debug output from long-running applications without having to restart them, as quite often restarting a process will destroy the state you were hoping to debug.

What I came up with is GDebugController, which is awaiting review in GLib now. It’s an interface, currently implemented by GDebugControllerDBus. When instantiated, GDebugControllerDBus creates a new D-Bus object and interface for controlling the debug output from the application. It hooks into the standard g_debug() message functions, and can be hooked into a custom log writer function if your application uses one of those.

It essentially exists to expose one D-Bus property and allow that to be hooked in to your log writer. It has to be a bit more complex than that, though, as it needs to be able to handle authorisation: checking that the D-Bus peer who’s requesting to enable debug output on your application is actually allowed to do so. For services in particular, this is important, as allowing any peer to enable debug output could count as a privilege escalation. They might be able to slow your process down due to the volume of debug output it produces; fill the disk up; or look at the outputted logs and see private information. GDebugControllerDBus has an authorize signal to support this, and it works quite similarly to the GDBusInterfaceSkeleton::g-authorize-method signal.

Using it in an application

Firstly, you need to wait for it to be reviewed and land in GLib. The API might change during review.

Once it’s landed, assuming nothing changes, you just need to create an instance of GDebugControllerDBus. It will create the D-Bus object and hook it all up. When peers change the debug level in your application, the default handler will call g_log_set_debug_enabled() which will change the behaviour of GLib’s default log writer function.

If you have a custom log writer function, you will need to change it to check g_log_get_debug_enabled() and output debug messages if it’s true.

Using it in a service

Using it in a service will typically involve hooking up authorisation. I’ve implemented support for it in libgsystemservice, so that it will be enabled for any user of libgsystemservice after version 0.2.0.

To use polkit for authorisation, set the GssService:debug-controller-action-id property to the ID of the polkit action you want to use for authorising enabling/disabling debug mode. libgsystemservice will handle the polkit checks using that. Here’s an example.

If that property is not set, a default policy will be used, where debug requests will be accepted unconditionally if your service is running on the session bus, and rejected unconditionally if it’s running on the system bus. The thinking is that there’s no security boundary on the session bus (all peers are equally trusted), whereas there are a lot of security boundaries on the system bus so libgsystemservice is best to fail closed and force you to write your own security policy.

That’s it! Reviews and feedback welcome. Many thanks to Endless for running this week and actively encouraging everyone to make use of it.

Add metadata to your app to say what inputs and display sizes it supports

The appstream specification, used for appdata files for apps on Linux, supports specifying what input devices and display sizes an app requires or supports. GNOME Software 41 will hopefully be able to use that information to show whether an app supports your computer. Currently, though, almost no apps include this metadata in their appdata.xml file.

Please consider taking 5 minutes to add the information to the appdata.xml files you care about. Thanks!

If your app supports (and is tested with) touch devices, plus keyboard and mouse, add:

<recommends>
  <control>keyboard</control>
  <control>pointing</control>
  <control>touch</control>
</recommends>

If your app is only tested against keyboard and mouse, add:

<requires>
  <control>keyboard</control>
  <control>pointing</control>
</requires>

If it supports gamepads, add:

<recommends>
  <control>gamepad</control>
</recommends>

If your app is only tested on desktop screens (the majority of cases):

<requires>
  <display_length compare="ge">medium</display_length>
</requires>

If your app is adaptive and works on mobile device screens through to desktops, add:

<requires>
  <display_length compare="ge">small</display_length>
</requires>

Or, if you’ve developed your app to work at a specific size (mostly relevant for mobile devices), you can specify that explicitly:

<requires>
  <display_length compare="ge">360</display_length>
</requires>

Note that there may be updates to the definition of display_length in appstream in future for small display sizes (phones), so this might change slightly.

Another example is what I’ve added for Hitori, which supports touch and mouse input (but not keyboard input) and which works on small and large screens.

See the full specification for more unusual situations and additional examples.