User Testing Webinars by Tomer Sharon and Steve Krug
I’m watching a series of webinars on usertesting.com led by Tomer Sharon and Steve Krug. This one covers Remote Moderated testing. To see them all go here >
There are three types of testing:
Both people are in the same location, “Usability Lab”
Participant going through a script on their own, no moderator. Not happening at the same time, unsynchronized. Can be quantitative (using services like UserZoom), or qualitative (using services like User Testing.com)
Combo of the two. Synchronized. Uses screen sharing. Qualitative ONLY.
Helps with recruiting because you can recruit from ANYWHERE.
Tomer Sharon on recruiting participants: Are the people you want to use this around you, or do you want to rule the world? If the answer is rule the world, do remote Moderated testing!
Feature- can you share your screen with participants and have them control and use the product? This is not trivial, you need this.
Awesome User Testing Script from Steve Krug:
http://www.sensible.com/downloads/test-script.pdf
He recommends reading it verbatim, otherwise you can sometimes drift into language that makes it seem as if you are looking for specific feedback rather than just observing them. You are trying to stay away from opinions, but rather watch HOW they use it.
- There is no right way to do this.
- Please think out loud for me as you go through
- Looking for honest reactions, don’t worry about hurting our feelings.
First have participant describe overall impression of product or service, and then asking them to do a specific task.
During the task, simply make sure the participant is on-task and thinking aloud, but aside from that, keep yourself out of it. Hold your tongue.
Why moderated than un-moderated?
- It’s easier to capture what people feel. Not a lot of opportunity to probe or ask questions during un-moderated. This is the key for remote moderated.
- Ability to course-correct the participant to keep them on-task.
- Allows you to give participants control of your machine (must have screen share on the tool)- if you have a product you can not allow
- Allows you to give participants control of your machine (must have screen share on the tool)- if you have a product you can not allow out into the world.
Why Remote Moderated than local?
- Makes recruiting MUCH easier.
- Faster, and easier to schedule.
- Get a sampling of people that is more realistic as it’s distributed.
- Only deal with people 100% evaluating the product, all the scheduling and logistics are online. Less overhead.
Advice on remote moderated testing:
-Maintain rapport. Makes sure they are comfortable. You have to ask them directly when it is remote to make sure they are ok. What you can do with a look in person must be vocalized remotely.
-Less body language cues when you are not in the same room, you have to pay more attention if doing it remotely.
- Managing observers: Tomer’s view- invite team members to watch session in same room. Invites them to interact with him or participant. Steve’s view- observers sitting in conference room down the hall. You are in another room solo- you don’t have to keep them quiet.
-When to interfere? How long do you let them be silent?
-How certain are you that you still understand what they are thinking? If you feel you don’t, prompt them to vocalize again.
- Not interfering is KEY.
-“9 out of 10 times, when you want to say something, you shouldn’t.” Tomer
-If you interfere, you are going to miss an opportunity to see what REALLY HAPPENS.
-When you learned what you needed to learn, go ahead and stop them.
You can do a screen body hug, where you have them turn the computer camera to be looking at their phone
Use reflector with Air Play, and share screens.
Try not to make it a report- make it a team sport- Allow everyone to watch together, use the Rainbow spreadsheet, allows all the entire team to be involved and turns itself into a report.
How to get people not to just search for “hotspots” for the right move?
- Make it all clickable, but just got to the next page ans say, we don’t have that available yet.
“OK is a very useful utterance in usability testing” sole purpose is to forward the conversation. It’s ok for your words to be detached.
Qualitative research, all you are doing is harvesting insights. You can change the test between users if there is a fatal flaw in the usability and there is a whole type of testing that advocates this: RITE testing
