Reminder: in October 2018, I’m pygame.org’s artist in residence. You can ask me questions about Trosnoth, support me in my Trosnoth development, join in online alpha testing, and follow this blog for articles about Trosnoth and what I’m working on.
What’s under the bonnet?
In my previous article, I wrote about what makes Trosnoth Trosnoth. But what powers Trosnoth, and why did we choose those building blocks? In this article I’ll try to answer that question.
A python.
1. Python
Trosnoth is a pure Python project. We do use third-party libraries that include binary components. But all of the source code of Trosnoth itself is in Python1. The Python programming language is widely used in scientific computing, education, web services and even embedded computing. And according to the 2018 Stack Overflow survey it’s the 4th most popular programming language2 after JavaScript, Java and shell script. But why choose it for a game?
In years gone by, people used to ask me, ‘Isn’t Python too slow for a game?’ Those were the days when people thought real games should be written in C or C++. But these days many games are written in interpreted languages like Java, C#, or JavaScript. So choosing Python no longer seems quite as weird. Even so, I think it’s worth pointing out that Python was never too slow to run Trosnoth. Thanks to pygame, all the graphics calculations are done by SDL. CPython3 has always been more than capable of keeping up with everything else we need to do, so we never needed to seriously consider speeding things up with PyPy or Cython.
There are three main reasons why Python was an appealing option when we started the Trosnoth project. Firstly, it’s easy to learn. We wanted to make it easy for new programmers to contribute to Trosnoth—or even to learn programming by hacking Trosnoth. Any language that was difficult to learn would make it harder from Trosnoth to be helpful for new programmers.
Secondly, Python allows rapid development. In my experience good developers can do things with Python to the same level of quality in much less time than with many other languages. (Though I do think the difference between languages by this measure has lessened somewhat over the years with the availability of free IDEs containing high quality tools.) This advantage is of great help to an open source project like Trosnoth because developers are usually writing code in their spare time. A developer doesn’t want to spend 50% of their time writing boilerplate code when they only have limited time to spend on Trosnoth.
Thirdly, Python makes it easy to write maintainable code. This is a benefit to any software project, but particularly one where developers are strapped for time. Hunting bugs is much harder when it takes several hours to understand what the code in question was trying to do in the first place.
2. Twisted
Trosnoth runs on a Twisted main loop. That means that it uses a funny thing called cooperative multitasking. I’ve explained this in two different conference talks in the past but many developers are at least somewhat familiar with the concept from JavaScript: you start a task running (say, an AJAX request), then you provide a callback function that should be run when the task completes. In modern JavaScript, you can accomplish this by defining an async function and calling await on the task/promise. This is all cooperative multitasking because it’s the programmer, not the operating system, that decides when one task should pass control to another.
When compared with shared-memory thread-based multitasking, cooperative multitasking eliminates a whole class of synchronisation bugs. Since Trosnoth aims to be welcoming to new programmers this seemed like a good idea. Back in the days of Python 2, you needed a third-party library in order to do cooperative multitasking. Twisted was one of the prominent options, and one of our original contributors had lots of experience using Twisted. So it was an obvious choice to make.
Some twisted things.
These days, Python has built-in primitives for cooperative multitasking: async def, await, async for etc.. It also has asyncio in the standard library. If we’d started the Trosnoth project today, perhaps we would not have chosen to use Twisted. Having said that, Twisted also provides a whole raft of network protocols straight out of the box, which have proven useful at times. For instance, we can run a network-bound interactive Python REPL in a Trosnoth process using Twisted’s manhole module.
3. Pygame
Trosnoth uses pygame for its graphics and audio. When we started working on Trosnoth, pygame and pyglet were the two main contenders for 2D Python audiovisual libraries. Pyglet was relatively young and had only a small community around it. Pygame was mature and had a large community. We chose the safe option and went with pygame. Even though development on pygame stalled from 2009 until late 2016, it continued to serve our needs.
During recent years we’ve toyed with the idea of moving Trosnoth to 2.5D graphics. That is, using a 3D game engine but keeping the gameplay all in a single 2D plane. To this end, we experimented with Panda3D. We even had a working (feature-incomplete) prototype that used the same network protocol and backend code but used Panda3D for the front-end. But sadly, when we updated our codebase to Python 3, the prototype broke. The development version of Panda3D, version 1.10, does support Python 3, but the latest stable release at time of writing does not. Until Panda3D 1.10 is released, any experimenting on this front has stalled.
4. Django (server-only)
The Trosnoth server has a web interface, mostly for displaying statistics from past games. When we first set up the web interface we just used twisted.web. (Yes of course Twisted comes with HTTP support built in!) But twisted.web is very rudimentary and the more information we wanted to display on the interface the more it made sense to move to a more complete web framework.
Django is a widely used and mature web framework so it was an obvious candidate: we wanted to be friendly to new programmers, and Django has lots of documentation and a supportive community. There was just one hassle: some elements of the web interface could interact with a running game. This meant that if we used Django we’d need to establish some communication between the web service and the game server. Or… we could run Django inside a Twisted main loop. This last idea sounds like it would never work. But I read a blog post explaining that actually it works pretty well, if you write code that plays nicely. Doing things this way actually made the transition to Django fairly simple. But talking about it does earn me looks of horrified admiration from certain other Python developers. (These days we actually run the web server in a separate process from the main game server, but there’s still some Twisted-y things going on in the web server.)
And that’s how Trosnoth works
…well that’s what Trosnoth’s built on anyway. There are some other very interesting technical details in how Trosnoth itself works: like where we hide network lag and how bots navigate the map. But those are stories for another day.
Keep an eye out for more articles about Trosnoth here on this blog throughout October.
Notes
1 Yes, we also have HTML, CSS and JavaScript in the web interface of the Trosnoth server. But I don’t feel this invalidates my assertion.
2 If you followed the link, I’m sure you saw what I did there.
3 CPython is the reference implementation of the Python language, available from python.org.