I’ve had to write and maintain Python unit tests both for work and for Trosnoth. I’ve played around with py.test and nose, and have done a tiny bit with trial (for twisted). As a result I’ve come up with the following wish list for a Python unit testing framework.
- Being able to constrain test order—in particular, I’d like to be able to say “If test A fails, do not run test B.” This should make it easier to identify the true cause of certain problems.
- Test code should be readable—some unit testing frameworks use lots of magic which can make it easy to write tests, but when you come back and read the tests later, it can be really confusing to find where certain things are defined (e.g. py.test funcargs).
- Generating many similar tests should be easy—if you want to run the same test with x=1, 2, 5, 15 and 28, you should not have to write 5 separate functions. You should not have to write a class. You should be able to write a function and specify that it should be called with multiple values.
- Setup and teardown for groups of tests—pretty much every test framework has this, but I thought it was worth mentioning anyway. It could be done better too. For instance, in some situations it would be useful to have nested scopes where each scope has a setup and teardown.
Those are the main things I can think of right now. At some stage I may implement a simple framework based on these ideas, just for fun. If I do, I shall post about it on this blog.