Validation ‐ User Testing of Alien Fauna Features and Overall Gameplay

Introduction

User testing is an important validation activity in game development that can be used to inform design decisions and improve the overall quality of the game. This page documents the user testing conducted during Sprint 4 by team 4 to validate the "Gardener's of the Galaxy" game. The primary goal of the user testing was to leverage user feedback to identify the changes that need to be made regarding design, game mechanics and usability in order to make the game as fun as possible for player. The testing focused on the alien fauna features and the overall gameplay.

Methodology

The testing consisted of two main activities that the users had to complete - a think-aloud activity and a System Usability Scale (SUS) style activity. The think-aloud was conducted to test the general gameplay, and the SUS was conducted to test the alien fauna features. This division of the tests was necessary as the game was too long for users to play through sequentially and still interact with all the alien fauna features. It should be noted that the SUS activity only adapted the format of SUS and did not use the standard SUS questions. This was done to make the users' responses more specific to the game, and therefore more useful for validation. The think-aloud was performed before the SUS in order to preserve the user's first impressions for the think-aloud. Thi was done as a think-aloud is more effective at capturing first-impressions due to it being live and less restrictive than the SUS.

The think-aloud activity involved users playing the game uninterrupted, from the beginning, and vocalising their thoughts to the testing invigilators. The invigilators noted down meaningful comments made by the users and the context in which said comments were made. Each think-aloud lasted for five minutes. A think-aloud was chosen as it provides direct access to the thought process of users, which is critical in designing intuitive gameplay. Moreover, as games are driven by emotion and fun, qualitative data about the user's thoughts is key for maximising user enjoyment. It was also noted that a more rigid testing method (e.g., a questionnaire) may restrict the quantity and kind of feedback that users can provide, which is undesirable for a first impression test.

To test the alien fauna features, users were required to perform a list of in-game tasks including feeding, taming, and fighting hostiles. To be comprehensive, all key animal fauna interaction was covered in these tasks. After completing each task, users completed a SUS style response sheet. The statements constituting the SUS all related to the tasks the user just completed and were designed to gather insight into the tasks enjoyability, usability and cohesion with the game. The statements also focused on areas of the alien fauna features that team 4 considered users may be very sensitive to e.g., combat difficulty. This activity was designed to provide more targeted feedback than the think-aloud by directing players to the specific tasks alien fauna tasks and having them respond to statements about enjoyability, usability and design. A more specific test than the think-aloud was performed for the alien fauna features, as team 4 developed this code and could therefore make the technical improvements corresponding to the specific feedback.

User breakdown

The user testing was performed by four users who had no experience with the game and who were all within the age range of 20 to 22 years old. Two of the users were male and two were female. The users were all experienceed with technology and games. These users were chosen as to be as representative of the game's expected demograhpic as possible. As the game is still in very early development, the feedback of the expected demographic was prioritised as to ensure the game can at least appeal to its most likely customers.

Data collection methods

The specific tasks that the users had to complete and the SUS template that they had to fill out afterward can be found here. The template used to record the think-aloud can be found here.

Results

Think-aloud

The results from the think-aloud are summarised below. Similar comments are grouped together.

Feedback	Context / Stage of Game	Evidence
Movement speed is too slow for map size	Very start of game when users are exploring the map	"Wish the sprint was faster", "Finding the animals takes ages"
There is not much going on to keep players engaged	After completing first few quests but before the hostiles spawn	"This is just a bunch of fetch quests"
Controls are intuitive to use	Players were comfortable walking, accessing the inventory, and using items	No comments on difficulty regarding controls
The UI is overly cluttered	Throughout the game	"All the different bars are a bit much"
The colour palette is too dark and may not be aethetically pleasing	Throughout the game	"Everything looks muddy", "The colours could match better"

List of tasks

Discussion and Response

Recommendation for Future Testing