Automated Testing in Gamedev

click here to view this on my blog with some nicer styling

Most developers that have had some experience with programming business software are familiar with testing, especially unit testing. But that experience usually doesn’t transfer neatly into gamedev due to the fact that games are more about interaction between game elements rather than the behavior of a single element alone. Never the less, unit testing can be a powerful tool, even when it comes to game development. Test with a larger scope than unit tests can also provide interesting possibilities that aren’t talked about enough. I’ll try to change that and hope to inspire you to try out some different forms of automated testing.

Starting simple – Unit Tests

Let’s first get some basics out of the way and start with unit tests and when to use them. Unit tests are the most useful whenever you have some complex logic that is at least somewhat isolated. As an example from my current game/prototype I had to write a method that computes if and when in the future two moving circular hit boxes will overlap.

This method is used to predict when two units would collide and helps steer them apart. The result isn’t perfect (yet) but is already a lot more natural than any collision based system.

The method signature looks something like this (Warning! Two letter variables ahead):

fun computeCollision(
    // position, velocity and radius of first hitbox
    p0: Vector, v0: Vector, r0: Double, 
    // position, velocity and radius of second hitbox
    p1: Vector, v1: Vector, r1: Double
): Double? {
    /* complex logic goes here */
}

Writing some unit tests helped me iron out some edge cases that such e piece of code might have:

• What happens if the circles have intersected in the past?
• What happens if the paths the circles take never intersect? (e.g. parallel movement)
• What happens if the paths never intersect but the circles do (e.g. parallel movement but they always intersect)
• What happens if the circles touch but never intersect?

Given that the logic completely depends on the inputs it is very easy to write unit tests:

@Test
fun `two circles with crossing paths at different times should never collide`() {
    val t = computeCollision(
        // Will pass (0,0) after 5 seconds
        p0 = Vector(-5f, 0f), v0 = Vector(1f, 0f), r0 = 1.0,
        // Will pass (0,0) after 8 seconds
        p1 = Vector(0f, -8f), v1 = Vector(0f, 1f), r1 = 1.0
    )

    expectThat(t) { isNull() }
}

This is nice for isolated methods, but can get complex and convoluted if you want to test interactions between different things in your game. Which is why we need to take this…

One level higher – Gameplay Tests

The prerequisites to efficiently test gameplay is that your game engine supports it. Sadly that isn’t a given by most common game engines, so you’ll have to make do with some weird hacks depending on your choice of tooling. In my case it was rather easy because I’m not using a real game engine but something like a game framework (libgdx + ktx). In addition I’m using the Entity Component Systems (ECS) design pattern (powered by fleks) for my game logic, which makes it quite easy to run my game as a simulation without any user interface.

Separating out any logic that depends on graphical context (e.g. OpenGL) was as simple as setting up a game world without all rendering systems. The game loop is also completely in my control, so I can easily simulate thousands of frames per second. Any complexity is extracted into easy to use methods. A simple test looks something like this:

@Test
fun `heroes with starter weapons should beat level 1`() {
    // Arrange - setupWorld() uses the same methods to setup the world as the 'real' game
    //           but without any rendering logic. It also mocks user input to cast
    //           skills at enemies.
    val (world, gameHandler) = setupWorld(level = 1, defaultBow, defaultSword)

    // Act - simulate the gameloop at 30fps until the heroes win or lose
    val result = world.simulateLevel(gameHandler)

    // Assert - check if simulation finished with the expected result
    expectThat(result) { isEqualTo(GameOutcome.Win) }
}

This simulates a full run of the first level in about 100ms. With this setup it is quite simple to add various scenarios for different enemies, heroes and item combinations and run them in a short amount of time in order to ensure everything works as expected. But why stop here?

Let’s turn it around – Game Balance Tests

Given that we can run through multiple levels under a second, why not automate some more tasks that you would normally do by hand? Let’s test some simple game balance assumptions that should always hold true.

A test to check that a very good item crafted at level X should be able to beat level X+1 could look like this:

// Parameterized tests run the same test multiple times with different inputs
@ParameterizedTest(name = "should beat level {0}")
@ValueSource(ints = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
fun `best crafted item from previous level`(level: Int) {
    // getBestCraftedItem() simulates randomly crafting items at a given level
    // the best item out of 100 crafted items from the previous level is selected
    val (world, gameHandler) = setupWorld(
        level,
        getBestCraftedItem(level - 1, ItemType.Bow, numItemsToGenerate = 100),
        getBestCraftedItem(level - 1, ItemType.Sword, numItemsToGenerate = 100)
    )

    val result = world.simulateLevel(gameHandler)

    expectThat(result) { isEqualTo(GameOutcome.Win) }
}

There are many other possible balance tests one could imagine writing:

• Crafted items should never be so good that it enables skipping a level entirely
• Heroes should be balanced in a way that multiple of the same hero cannot beat levels but different heroes together can (e.g. 3 knights should be a worse than 1 knight + 1 archer + 1 healer)
• Different builds should be able to beat levels (e.g. always selecting weapons that deal poison damage should be as viable as picking weapons that deal fire damage)

It’s also quite easy to collect some additional metrics from the world by adding some systems that are only used in tests and that monitor various entities (e.g. count how much damage a certain hero deals compared to other ones). The possibilities are almost endless!

Conclusion

Why would one even go through all this effort? I see the possibilities this brings when used in conjunction with manual gameplay testing. Playing a game to see if it is fun will always be required. But imagine switching some things around and prototyping different ideas while immediately being able to see what consequences that has is immensely powerful. How far one can push this concept is something that I will have to learn as my game grows in features and complexity.