Validation ‐ User Testing of Alien Fauna Features and Overall Gameplay

Introduction

User testing is an important validation activity in game development that can be used to inform design decisions and improve the overall quality of the game. This page documents the user testing conducted during Sprint 4 by team 4 to validate the "Gardener's of the Galaxy" game. The primary goal of the user testing was to leverage user feedback to identify the changes that need to be made regarding design, game mechanics and usability in order to make the game as fun as possible for player. The testing focused on the alien fauna features and the overall gameplay.

Methodology

The testing consisted of two main activities that the users had to complete - a think-aloud activity and a System Usability Scale (SUS) style activity. The think-aloud was conducted to test the general gameplay, and the SUS was conducted to test the alien fauna features. This division of the tests was necessary as the game was too long for users to play through sequentially and still interact with all the alien fauna features. It should be noted that the SUS activity only adapted the format of SUS and did not use the standard SUS questions. This was done to make the users' responses more specific to the game, and therefore more useful for validation of the game. The think-aloud was performed before the SUS in order to preserve the user's first impressions for the think-aloud. This was done as a think-aloud is more effective at capturing first-impressions due to it being live and less restrictive than the SUS.

The think-aloud activity involved users playing the game uninterrupted, from the beginning, and vocalising their thoughts to the testing invigilators. The invigilators noted down meaningful comments made by the users and the context in which said comments were made. Each think-aloud lasted for five minutes. A think-aloud was chosen as it provides direct access to the thought process of users, which is critical in designing intuitive gameplay. Moreover, as games are driven by emotion and fun, qualitative data about the user's thoughts is key for maximising user enjoyment. It was also noted that a more rigid testing method (e.g., a questionnaire) may restrict the quantity and kind of feedback that users can provide, which is undesirable for a first impression test.

To test the alien fauna features, users were required to perform a list of in-game tasks including feeding, taming, and fighting hostiles. To be comprehensive, all key animal fauna interaction was covered in these tasks. After completing each task, users completed a SUS style response sheet. The statements constituting the SUS all related to the tasks the user just completed and were designed to gather insight into the tasks enjoyability, usability and cohesion with the game. The statements also focused on areas of the alien fauna features that team 4 considered players may be very sensitive to e.g., combat difficulty. This activity was designed to provide more targeted feedback than the think-aloud by directing players to the specific tasks alien fauna tasks and having them respond to statements about enjoyability, usability and design. A more specific test than the think-aloud was performed for the alien fauna features, as team 4 developed this code and could therefore make the technical improvements corresponding to the specific feedback. Additionally, as there were many tasks to complete, a SUS was the quickest way to get through them while still providing valuable insights.

User breakdown

The user testing was performed by four users who had no experience with the game and who were all within the age range of 20 to 22 years old. Two of the users were male and two were female. The users were all experienceed with technology and games. These users were chosen as to be as representative of the game's expected demograhpic as possible. As the game is still in very early development, the feedback of the expected demographic was prioritised as to ensure the game can at least appeal to its most likely customers.

Data collection methods

The specific tasks that the users had to complete and the SUS template that they had to fill out afterward can be found here. The template used to record the think-aloud can be found here.

Results

Think-aloud key results

The results from the think-aloud are summarised below. Similar comments are grouped together.

Feedback	Context / Stage of Game	Evidence	Insight
1. Movement speed is too slow for map size	Very start of game when users are exploring the map	"Wish the sprint was faster", "Finding the animals takes ages"	The player needs a faster way to move around the map
2. There is not much going on to keep players engaged	After completing first few quests	"This is just a bunch of fetch quests"	More activity to be added to the game to keep the player engaged. This should not just be fetch quests though
3. Controls are intuitive to use	Players were comfortable walking, accessing the inventory, and using items	No comments on difficulty regarding controls	The controls do not need to be changed.
4. The UI is overly cluttered	Throughout the game	"All the different bars are a bit much"	A minimalist redesign of the UI is necessary to make it more user-friendly
5. The colour palette is too dark and may not be aethetically pleasing	Throughout the game	"Everything looks muddy", "The colours could match better"	A visual redesign of the game may be necessary. A change of colour palette should be considered.
6. The bats (hostile mob) do too much damage	After hostiles start spawning	"The bats are way too strong"	Rebalance bat damage and consider rebalancing all hostile's stats
7. Much of the animal drops do not have a purpose	Feeding drops and collecting passive drops from animals	"How is drinking milk different to just eating beef"	More functionality needs to be added to the items to make them stand out and justify their presence in the game

SUS key results

The averaged SUS results for each task are listed below.

Discussion and Response

Recommendation for Future Testing