Command Naming Test Script

Introduction

Hi, _____________. My name is ___________ and I'm going to be walking you through this session.

You probably already know, but let me explain why we've asked you to come here today: We're testing a slackbot that we're working on to see what it's like for actual people to use it.

I want to make it clear right away that we're testing the bot, not you. You can't do anything wrong here. In fact, this is probably the one place today where you don't have to worry about making mistakes.

We want to hear exactly what you think, so please don't worry that you're going to hurt our feelings. We want to improve it, so we need to know honestly what you think.

As we go along, I'm going to ask you to think out loud, to tell me what's going through your mind. This will help us.

If you have questions, just ask. I may not be able to answer them right away, since we're interested in how people do when they don't have someone sitting next to them, but I will try to answer any questions you still have when we're done.

We have a lot to do, and I'm going to try to keep us moving, but we'll try to make sure that it's fun, too.

Do you have any questions before we begin?

Background Information Questions

Before we look at the site, I'd like to ask you just a few quick questions. First, what's your role at Bocoup?

How do you (if you use it at all) currently use Robocoop? What kind of information are you looking for? What do you like about the experience? What could be improved on?

OK, great. We're done with the questions, and we can start looking at things.

Usability Test

Talk to Skills Bot:

• Press Cmd-K, then type in "skills" and press Enter.
• Type "help"

Perform a few tasks:

• Which person (or people) has the highest possible level of both interest and experience for the "work scoping" skill?
  • Tip: If the tester needs some assistance, have them type "help" and/or "find"
  • The user will need to determine which is higher, 1 or 5
• What do the numbers 1-5 for experience and interest mean?
  • If they don't know, ask them to try to figure it out.
  • Are the command descriptions obvious enough for them to figure this out?
• What might you do with this data?
• What percentage of people have the highest possible interest level for the "javascript" skill? how about for experience?
• How many programming language related skills are there?
• What skills have you rated at the maximum possible level for both experience and interest?
• What skills have you not yet rated? Are there any?
• Update your experience and interest level for the "User Research"

And again, as much as possible, it will help us if you can try to think out loud so we know what you're thinking about.

Conclusion

Thank you for participating in the usability test. Do you have any additional feedback regarding the bot? We will share the results at Grunch and on github. If you are interested in staying involved in the project, you can take a look at the Skillsbot repository.