Free Mercurial hosting for open source projects on GitLab fork Heptapod

The Trosnoth source code has been kept in a Mercurial repository on BitBucket since not long after we migrated from Subversion many years ago. But BitBucket no longer supports Mercurial repositories: all existing Mercurial repositories on BitBucket will be shut down on 1 June.

So the Trosnoth source repository will soon be migrating to Heptapod. Heptapod is a ‘friendly fork’ of GitLab community edition, built with the purpose of bringing Mercurial support to GitLab. In January, Octobus and Clever Cloud announced that they were offering free Heptapod hosting to open source projects (within certain parameters). They have also said that they plan to offer commercial hosting for non-open-source projects soon.

Why not self-host the mercurial repository?

If all we wanted was somewhere to host the Trosnoth source code, we could easily have set up hosting on our own server, and linked it to a trac instance. In fact, that’s what we originally did before moving to BitBucket. The motivation for moving to BitBucket in the first place was to make public collaboration easier. Platforms like GitLab, GitHub, and BitBucket make the Trosnoth project easy to discover, and provide straightforward tools for contributing to it. In contrast, self-hosting the repository only provides the bare essentials for collaboration.

(These platforms also provide added bonuses like easy automated builds and testsTrosnoth used BitBucket Pipelines to run our test suite. But collaboration is the main draw-card for us.)

Why not migrate to Git?

When BitBucket announced they were shutting down Mercurial repositories, we seriously considered migrating the Trosnoth repository to Git. After all, Git has become the far more popular choice for distributed version control. This option could still be on the table if things change in the future, but for now, we’ve chosen to stick with Mercurial. Here are a few of our reasons:

It’s less effort. The easiest option is not always the best option. But in an open source project which nobody’s paid to work on, effort spent migrating to Git is effort that isn’t being spent improving and developing Trosnoth itself.

Mercurial is easy for newcomers. If you’re a software developer these days, the odds are that you’ll have to learn Git at some stage. But when we use Trosnoth as an example project for high school students, none of them have ever used version control software before, and some of them may never use it again. I want to give them a positive experience of version control, and Mercurial is very friendly to newcomers. (From what I’ve seen, Mercurial is sometimes confusing for those who’ve come from Git, but that’s mostly because ‘branch’ and ‘pull’ don’t mean what you expect them to mean.)

Converting from Mercurial to Git is lossy. You don’t lose a lot in the conversion. And it’s true, you can generally live without what you’ve lost. And people who really buy into Git’s philosophy tell me that most of what you lose in the conversion (e.g., branch name metadata on every changeset) is clutter that you don’t want to keep anyway. But I disagree. The main difference I’ve seen between Mercurial and Git is that Mercurial aims to record project history, while Git aims to record patches. This might seem like the same thing, but imagine this: you’ve spent months working on a rewrite to a major part of your project and in the end you decide to scrap your work. Maybe the original need is no longer as pressing, or perhaps you realise that your rewrite doesn’t add as much as you’d first hoped. So you get rid of your branch and move back to the main line. If you’re using Mercurial, the history of your ill-fated rewrite is still preserved, and you can go back and view it any time you like. If you’re using Git, all your changes are lost unless you take a deliberate action to preserve them (e.g., create a tag). I prefer Mercurial’s default behaviour, because I never know when I might want to reuse something I did in an old branch. Differences like this cause challenges when converting from Mercurial to Git: I have to make a conscious decision what to do with each closed branch during the conversion, and if I have closed heads within a branch things get even worse. And this brings us back to the first reason to stick with Mercurial: it’s less effort.

Sticking with Mercurial is good for the internet. I admit that this reason is a philosophical one, and wouldn’t hold much weight if all the practical pressures said we needed to convert to Git. But the truth is, it’s better for everyone when there are multiple choices available. Chrome gets better because it has to compete with Safari, Firefox and Edge. And over their histories, Git and Mercurial have both borrowed features from one another. If Mercurial dies out, Git will be the worse for it. So we have very few misgivings about sticking with the underdog here. Mercurial may not be the popular choice, but it has very few other disadvantages, and we commend Octobus for starting the Heptapod project and helping keep Mercurial alive.

Posted in long | Tagged , , , , , , | Comments Off on Free Mercurial hosting for open source projects on GitLab fork Heptapod

Fixing grub error about unsigned kernel in Ubuntu

Just a quick note in case someone else runs into the same issue as me: I have a machine running Ubuntu 18.04 (LTS), and in order to get some of the hardware working, I used ukuu to upgrade the kernel from the Ubuntu versions (4.15.0.xx.xx) to an upstream version (4.18.20).

At first this caused me no problems, but after a while, I started to run into the following message when trying to update packages:

Cannot upgrade Secure Boot enforcement policy due to unsigned kernels

Your system has UEFI Secure Boot enabled in firmware, and the following kernels present on your system are unsigned:

4.18.20-041820-generic

These kernels cannot be verified under Secure Boot. To ensure your system remains bootable, GRUB will not be upgraded on your disk until these kernels are removed or replaced with signed kernels.

What seems to be happening is that newer versions of grub enforce more secure booting policies, which is a good thing. The problem is that the upstream kernels aren’t signed with Ubuntu’s trusted keys.

When I googled for this problem the suggestions seemed to be disabling UEFI secure booting in the BIOS, or removing the kernels. Removing the kernels would break some of my hardware, and I didn’t want to disable the secure boot setting because I sometimes boot the machine to Windows and I really don’t want any malware inserting itself secretly into the Windows boot process (I don’t want this on Linux either, but any malware that has root permissions on my Linux boot has already won).

After I realised what I needed to actually google for, I found this Ubuntu blog article which explains how to create a machine owner key (MOK), set up Ubuntu to trust it, and sign things with it. I followed the steps described starting with “Certificates in shim” to generate and enrol a certificate. (You may be able to use update-secureboot-policy directly to create and enrol the certificate, but I didn’t discover that until I got to the end of the blog article, so I haven’t tried that way.)

It’s worth noting as you read that article that you do not want to include “1.3.6.1.4.1.2312.16.1.2” in the extendedKeyUsage of your certificate or you’ll only be able to sign kernel modules and not kernel themselves.

I then used sbsign as described in the article to sign /boot/vmlinuz-4.18.20-041820-generic. I moved the original to a different directory for safekeeping and replaced it with the signed version. (Keeping both versions side by side in /boot/ is a bad idea because grub will see both and still complain about having an unsigned one there.)

I then did two final steps:

sudo dpkg-reconfigure grub-pc

This causes grub to rebuild its menus. This is probably not strictly necessary because I didn’t actually add any new kernels to /boot/, but I thought it was worth running to make sure there weren’t any errors.

sudo dpkg --configure grub-efi-amd64-signed

This fixes the installation of the package that was broken. If all goes according to plan, it should no longer show an error.

Posted in midlength | Tagged , , , , | Comments Off on Fixing grub error about unsigned kernel in Ubuntu

Trosnoth Development Priority: Simple Server Deployment

Earlier this year, a volunteer on Übertweak tried to use Trosnoth on camp, but couldn’t work out how to run and configure a Trosnoth server. In the current version of Trosnoth, the server is not shipped in any easy-to-use bundle; instead, you must run the Trosnoth server from the source code. This effectively means that you need an expert present to run a Trosnoth server. If the person on camp with the motivation to run Trosnoth does not have the required expertise, it doesn’t work.

I consider this to be a major failure in Trosnoth. Trosnoth was created many years ago for use on Übertweak. If it can’t be easily used on camp, this is a problem.

Over the weekend, we released Trosnoth 1.13.0. This was a huge milestone, incorporating many months of work and completely overhauling some Trosnoth internals.

In January, I listed some upcoming Trosnoth development priorities, and said that I hadn’t yet decided on the importance of each priority. That has now changed. Now that version 1.13.0 is released, all other development priorities are on hold until we’ve fixed this shortcoming. The aim of the next release is to make it really easy to install, configure and run your own Trosnoth server, so that people can play Trosnoth at LANs or on camps without any specific technical knowledge.

Posted in short | Tagged , , | Comments Off on Trosnoth Development Priority: Simple Server Deployment

Trosnoth in 2018 and Beyond

The past year has been a good year for Trosnoth. Over this time, we have:

  • migrated Trosnoth to Python 3
  • made several UI improvements, including adding indicator lights around orbs, showing the short names of friendly players on the minimap, and adding a respawn gauge to all ghosts
  • adjusted the costs of items based on observed usage
  • added a way for players to select their preferred scenario for the next match, along with a bunch of new scenarios including Elephant King, Orb Chase and Cat Among Pigeons
  • updated the server to keep track of some statistics across multiple matches in a tournament
  • made huge improvements to the momentum-based physics that was released in late 2017: we overhauled the collision detection to use separating axis theorem which eliminates the possibility of falling through walls; we also made improvements to how grappling hook works (these are yet to be released)
  • as part of the grappling hook changes, given players the ability to hold on to roof surfaces, and shoot while holding onto walls and roof
  • made bots feel a bit more fair—they have a short simulated reaction time, but they’re still a challenge to defeat
  • fixed a bunch of bugs
  • made other minor changes and improvements in response to player feedback
  • made fair progress towards bots using the updated momentum-based physics

Trosnoth League

In 2018 a small number of players formed a Trosnoth League. Once a month we scheduled time to play Trosnoth competitively online. The internet causes some frustrations that aren’t experienced on a LAN (chiefly of the, ‘I hit you on my screen!’ variety). But our experiences showed us that internet Trosnoth is still a lot of fun!

One of the fun things we did during Trosnoth League was to play Human vs. Machines matches. These matches were often very tense matches. Over the course of the year, 8 HvM matches were won by the humans, compared to 6 won by machines.

Trosnoth League also provided regular opportunity for player feedback—we often played with an alpha or beta Trosnoth version and solicited feedback. This resulted in us fixing a number of minor annoyances for players, and adding a number of low-effort medium-payoff features to the game. For instance, we added a visual indicator for ninjas to see how close they can get to enemy orbs before being seen, and we made it possible to remap the right mouse button (previously only keyboard keys could be remapped).

If you’re interested in joining in Trosnoth League in 2019, please fill out this survey, and also send an email to the trosnoth-players Google group to ask us how to join in.

Project Participation

In 2017, three people contributed code to Trosnoth. In 2018, there was only me. This 67% drop isn’t a concern in itself, but it does serve to illustrate how the project operates at the moment: I’m the main code contributor, and other contributors offer a bit of code every so often as they’re motivated an able. Maybe this is ok for a working game like Trosnoth, but I’d still love more regular contributors. We’ve actually never been great at attracting new coders to Trosnoth. Almost all of the occasional contributors were contributors 8 to 10 years ago when the project was still getting off the ground. If you’re a coder and you’re interested in helping out with Trosnoth, please get in touch! I’ll be as helpful and encouraging as I can!

The previous paragraph may sound like doom and gloom, but having no new coders doesn’t mean I’m the only one involved in the project. Trosnoth League has been a great source of ideas and suggestions, and has also resulted in one or two people contributing to future artwork and design of Trosnoth, which I find exciting! And there are still 5–10 people who help me decide on project priorities when I do spend time on Trosnoth.

I always welcome new contributors to Trosnoth, but right at the moment we don’t have any regular contributors with particular expertise in game graphics, and it would also be really cool to have someone who could put together video tutorials or a Let’s Play. Please get in touch if you’d like to be involved! You can connect with us on the trosnoth-dev Google Group.

Trosnoth repository history over the past 5 years (number of commits)

Releases and Artist in Residence Grant

2018 saw three Trosnoth releases in January, April and May. This means that there has been no new release in over six months. The man thing that’s been holding up the release of Trosnoth 1.13.0 is the bot pathfinding: I’d like the bots and humans to be using the same physics, but that can’t happen until the pathfinding is working. I keep thinking that I’ve almost finished updating the pathfinding code only to find that there’s more to do.

One huge motivator to work on the bot pathfinding was when I was named as the Pygame Artist in Residence in October. During October I wrote a bunch of Trosnoth-related blog posts, but I also spent significant time working on bot pathfinding. (I did a time-lapse video of my work too!)

The Artist in Residence program also came with a small grant of money. I spent some of it obtaining a Windows code signing certificate and related hardware, so future releases of Trosnoth should actually be signed!

What Next: Server Deployment, New Levels, Chrome, and Synchronisation

Once I finally finish the bot path-finding, there are a few different ideas that I’d like to work on. I haven’t yet prioritised these, so I can’t tell you which one is coming soon. But here are four things somewhere in the pipeline at present:

  1. Simpler server deployment: I want to make it really easy to install, configure and run your own Trosnoth server, so that people can play Trosnoth at LANs even without specific technical knowledge.
  2. New levels: And not just new levels, but a restructure of how levels work so that there are coherent rooms such as a control centre, cargo bay, living quarters, recreation areas, and so on.
  3. HTML and CSS UI elements: I’d like to experiment with using Chromium Embedded Framework for some Trosnoth UI elements. This wouldn’t be used to render the game itself, but for menus and some parts of the heads-up display. One potential advantage of this approach is that it allows people who know HTML and CSS to tweak the UI without forcing them to learn how Python and pygame work. This also might simplify things if we ever move to using Panda3D for rendering: the HTML/CSS parts wouldn’t need to be rewritten to make the shift.
  4. Speed/smoothness/synchronisation improvements: At the moment Trosnoth has a very workable process for synchronising the game state between the server and clients. But I’d like to generalise what Trosnoth does into a separate library. This will allow for more automated testing of the synchronisation code, and more consistency and robustness. This will also make it easier to reason about the code when making improvements to how the game handles internet latency smoothly.

With the current level of time I spend on Trosnoth, I don’t expect to complete more than one (or at most two) of these four things in 2019, but that gives you an idea of where things are heading at the moment.


Posted in long | Tagged , , , , , | Comments Off on Trosnoth in 2018 and Beyond

Timelapse Trosnoth Development Video

Sometimes an artist in residence program has a closing exhibition. In October 2018 I was pygame.org’s virtual artist in residence. As a kind of closing exhibition for this artist residency, I’ve put together a time-lapse video of most of my Trosnoth development during October.

If you want to actually be able to read any of the text on the screen you’ll probably need to watch it in 1080p.

Enjoy!

 

Posted in short | Tagged , , , , , , , , | Comments Off on Timelapse Trosnoth Development Video