User Research, Usability Testing
This was an evaluative study to understand issues first time visitors of the website run into while searching for training and certification exams.
Tools Used: Ovo Studios, Powerpoint, SurveyMonkey
Team Size: 4
In addition to understanding the experience of first time users, it was important for us to understand the users who were already visiting the site. We wanted to ensure that we focus our work on realistic users who would potentially visit the site on their own versus a complete non-user.
Microsoft shared detailed data from our sponsors. Based on this data, we created user groups. These user groups helped us in creating our personas for the system.
We carefully selected the persona’s for our study based on the demographics of users who have visited the site in the past. This helped us in two ways (1) Creating a baseline about what certain persona’s will do when confronted with different tasks on the website and (2) recruiting participants similar to the persona so that we understand how we can make the experience better.
The Image on the right has been modified to hide data.
We developed our screener to map our sample against it and select participants by considering factors such as eliminating professional user research participants, minimizing no-shows, eliminating the strong -silent types, learning about their familiarity with Microsoft Learning website and ensuring that people are physically able to participate.
- Pre-session interview
- A warm up task
- Two sets of scenario-based tasks completed with the website
- Post-session interview and debrief
For each task, we collected the following data:
The Image below has been modified to hide data.
The study revealed some interesting findings and we were able to complement our quantitative data with verbatim comments from our participants. From these comments, we were able to dive deeper into the issues that caused task failure or difficulty. All the recommendations have been implemented by the Microsoft team and are live now.
The usability study was a great learning experience. Few of the high level learnings were:
- Test the scenario, prior to the actual test, on the same computer as you would use for the participant, and do not log in from the corporate account. There are some interfaces which are available to employees only and might impact your results.
- Keep a print out of the tasks for the participant and ask them to read it aloud and ask them to repeat it in their own words. It helps to see if the participant has understood the task correctly.
- Always test a new environment a few hours before the test is about to begin. Give enough room to fix any technical glitches
- Run a pilot study (at least) a day in advance. Have enough time to fix the script or other issues you might face.