-
Notifications
You must be signed in to change notification settings - Fork 7
Validation ‐ User Testing of Alien Fauna Features and Overall Gameplay
User testing is an important validation activity in game development that can be used to inform design decisions and improve the overall quality of the game. This page documents the user testing conducted during Sprint 4 by team 4 to validate the "Gardener's of the Galaxy" game. The primary goal of the user testing was to leverage user feedback to identify the changes that need to be made regarding design, game mechanics and usability in order to make the game as fun as possible for player. The testing focused on the alien fauna features and the overall gameplay.
The testing consisted of two main activities that the users had to complete - a think-aloud activity and a System Usability Scale (SUS) style activity. The think-aloud was conducted to test the general gameplay, and the SUS was conducted to test the alien fauna features. This division of the tests was necessary as the game was too long for users to play through sequentially and still interact with all the alien fauna features. It should be noted that the SUS activity only adapted the format of SUS and did not use the standard SUS questions. This was done to make the users' responses more specific to the game, and therefore more useful for validation of the game. The think-aloud was performed before the SUS in order to preserve the user's first impressions for the think-aloud. This was done as a think-aloud is more effective at capturing first-impressions due to it being live and less restrictive than the SUS.
The think-aloud activity involved users playing the game uninterrupted, from the beginning, and vocalising their thoughts to the testing invigilators. The invigilators noted down meaningful comments made by the users and the context in which said comments were made. Each think-aloud lasted for five minutes. A think-aloud was chosen as it provides direct access to the thought process of users, which is critical in designing intuitive gameplay. Moreover, as games are driven by emotion and fun, qualitative data about the user's thoughts is key for maximising user enjoyment. It was also noted that a more rigid testing method (e.g., a questionnaire) may restrict the quantity and kind of feedback that users can provide, which is undesirable for a first impression test.
To test the alien fauna features, users were required to perform a list of in-game tasks including feeding, taming, and fighting hostiles. To be comprehensive, all key animal fauna interaction was covered in these tasks. After completing each task, users completed a SUS style response sheet. The statements constituting the SUS all related to the tasks the user just completed and were designed to gather insight into the tasks enjoyability, usability and intuitiveness. The statements also focused on areas of the alien fauna features that team 4 considered players may be very sensitive to e.g., combat difficulty. This activity was designed to provide more targeted feedback than the think-aloud by directing players to the specific tasks alien fauna tasks and having them respond to statements about enjoyability, usability and design. A more specific test than the think-aloud was performed for the alien fauna features, as team 4 developed this code and could therefore make the technical improvements corresponding to the specific feedback. Additionally, as there were many tasks to complete, a SUS was the quickest way to get through them while still providing valuable insights.
The user testing was performed by four users who had no experience with the game and who were all within the age range of 20 to 22 years old. Two of the users were male and two were female. The users were all experienceed with technology and games. These users were chosen as to be as representative of the game's expected demograhpic as possible. As the game is still in very early development, the feedback of the expected demographic was prioritised as to ensure the game can at least appeal to its most likely customers.
The specific tasks that the users had to complete and the SUS template that they had to fill out afterward can be found here. The template used to record the think-aloud can be found here.
The results from the think-aloud are summarised below. Similar comments are grouped together.
| Feedback | Context / Stage of Game | Evidence | Insight |
|---|---|---|---|
| 1. Movement speed is too slow for map size | Very start of game when users are exploring the map | "Wish the sprint was faster", "Finding the animals takes ages" | The player needs a faster way to move around the map to prevent them losing interest travelling long distances |
| 2. There is not much going on to keep players engaged | After completing first few quests | "This is just a bunch of fetch quests" | More activity to be added to the game to keep the player engaged. This should not just be fetch quests though |
| 3. Controls are intuitive to use | Players were comfortable walking, accessing the inventory, and using items | No comments on difficulty regarding controls | The controls do not need to be changed and future controls should follow the current scheme |
| 4. The UI is overly cluttered | Throughout the game | "All the different bars are a bit much" | A minimalist redesign of the UI is necessary to make it more user-friendly |
| 5. The colour palette is too dark and may not be aethetically pleasing | Throughout the game | "Everything looks muddy", "The colours could match better" | A visual redesign of the game may be necessary. A change of colour palette should be considered. |
| 6. The bats (hostile mob) do too much damage | After hostiles start spawning | "The bats are way too strong" | Rebalance bat damage and consider rebalancing all hostile's stats |
| 7. Much of the animal drops do not have a purpose | Feeding drops and collecting passive drops from animals | "How is drinking milk different to just eating beef" | More functionality needs to be added to the items to make them stand out and justify their presence in the game |
The averaged SUS results for each task are listed below.
- Find cow food and feed a cow
- It was easy to obtain cow food - Agree
- It was difficult to feed a cow - Strongly disagree
- Tame a chicken
- The chickens are too fast to catch - Neutral
- It was easy to understand taming and identify when an animal was tamed - Strongly Agree
- It was difficult to tame a chicken - Agree
- Kill one of each hostile
- It was difficult to differentiate hostiles from passive fauna - Neutral
- The combat was dynamic and fun - Disagree
- The game would benefit from higher hostile spawn rates - Agree
- Eat an animal
- Obtaining food was intuitive - Agree
- It was easy to find an animal that dropped food - Disagree
- The food made sense relative to the hunger system - Strongly Agree
The user testing revealed various insights as to what users thought the game did well and did poorly. Firstly, the users found the controls to be intuitive and easy to use "i key for inventory makes sense". Therefore, future control mechanisms will follow the same scheme. Another point of feedback was that there is not much to keep the player engaged per the feedback "you need more going on to keep us on our toes". This feedback
In response to the feedback from both the think-aloud and the SUS, team 4 implemented a new feature in sprint 4, the buff system. The buff system involved creating certain items which when consumed, provided unique buffs for the player. This system was designed to address a multitude of issues highlighted in the user testing. More specifically, the think-aloud revealed that users felt that the map was too big or that the player needed a faster way to traverse the map. Concequently, a buff to increase player speed was devised as to prevent players from getting bored when travelling long distances. Additionally, the think-aloud also highlighted that much of the fauna drops did not have any unqiue behaviour. Hence, the buff system utilised the item drops from passive fauna once tamed - the items that previously had no use. Finally, the SUS indicated that users did not find combat to be dynamic or fun. From this, buffs related to comabat were designed. With this feedback, two buffs were added to existing items for the buff system. Golden eggs link now increase the players speed - to decrease travel time and make fighting fast hostiles more fun, and drinking milk now adds a strength buff - to make combat more dyanmic.