Testing the flow
As part of my MA in User Experience Design, I completed a module called UX Design.
This blog post was originally published as part of the reflective piece I wrote throughout that module.
Having created a user flow based on a persona and scenario, I wanted to understand how it compared with the experiences of real users trying to accomplish the same task on the existing website. To do this, I decided to carry out some user tests.
User tests
I was a little unsure about carrying out user tests this early on in my project. In the course content for this module, user testing is positioned later on. However, I really want to analyse the existing website so that I can understand the current situation and establish “what does and doesn’t work and why” (Pernice 2021). User tests are a way to do this. More specifically, I want to:
Produce a snapshot of how users interact with the existing site
Identify improvements that I can incorporate into the design of my prototype
Create a benchmark to use as a comparison when I design my own prototype
Thinking aloud
The think aloud protocol is a very commonly used technique in user tests. This is because it is “cheap, robust, flexible, and easy to learn” (Nielsen 2012). According to Nielsen, it “should be the first tool in your UX toolbox” and it’s something I’m keen to learn more about.
This technique is a qualitative user research method which involves asking participants to use a system while encouraging them to verbalise their thoughts throughout the process. It’s something that I can definitely do with the resources I have.
Designing the test
I started by defining my goal. This was to see how users were able to use the existing website to find out more about the charity and then how they might either volunteer or donate.
With that goal in mind, I created three tasks:
Find out about the charity
Sign up to volunteer
Donate to the charity
This process follows the advice given by Pernice in her article for Nielsen Norman Group (2021).
Checklist and moderator script
In another Nielsen Norman Group article, Rosala (2022) recommends creating a checklist to support the faciliator when they carry out user tests. Baxter et al. (2015) recommend a similar thing but they use the word “protocol” instead of script or checklist.
This is something I’ve found really useful in the past as there are so many things to remember to do, share and say. You can see the checklist/script I created below.
I’ve written facilitator guides for face-to-face workshops before. Through doing that, I’ve learned there are ways to present and format guides in ways that help the facilitator. In the guide I’ve created for myself, I’ve tried to carry some of these ideas across. For example, where I want to read text out almost verbatim, I’ve used green font to highlight those sections.
Recruiting participants
I recruited three participants using my own connections. All three were either current or past Sheffield residents aged between 30 and 39. Two participants were women and one was a man.
Conducting tests
I wanted to be able to:
Conduct the tests remotely, at times that were convenient for me and participants
See participants’ faces and screens during the tests
Record each test so that I could watch back later to analyse and pick out the key points
For these reasons and because it is widely used, I decided to conduct the user tests via MS Teams.
Reflections
Having designed and conducted user tests on the existing Roundaboutwebsite, here are my reflections.
What went well
In a previous blog post, I reflected on my interviewing and facilitation skills and decided this was an area where I wanted to develop my skills and experience. Including user tests at this point in the project gave me an opportunity to do this.
The user tests I designed and conducted achieved my goals which were to:
Produce a snapshot of how users interact with the existing site
Identify improvements that I can incorporate into the design of my prototype
Create a benchmark to use as a comparison when I design my own prototype
In her article for Nielsen Norman Group, Therese Fessenden (2021) outlines common mistakes that people make when conducting user research interviews. I was able to avoid the following mistakes:
Insufficient rapport-building
I am good at building rapport, although I sometimes spend too much timedoing this when I need to get on with the interview
Multi-tasking and note-taking
Partly due to Teams’ recording functionality, I did not take notes during the interviews and was able to focus all of my attention on the participants.
It was really interesting to see how real users attempted to complete the task I had outlined in the user flow. This gave me lots of useful insights which I plan to analyse shortly.
What could be improved
All of the participants I recruited were white, aged between 30 and 39, with a similar socio-economic background. Ideally, I should have recruited people who more closely match the Jackie persona (Kuniavsky 2003),(Barnum 2011),(Baxter et al. 2015). I would also like to have recruited a more diverse mix of participants. However, I was a limited by the time available and needed to interview the people who I was able to persuade to help on the evenings I had free to do the tests.
Looking at the common mistakes article from Therese Fessenden again, here are the mistakes I did make:
Not enough probing questions
Leading the participant
During the interviews, I thought I had avoided these mistakes. However, when I watched the recordings, I realised that I made these mistakes throughout.
Future plans
As I work on this project, I plan to continue focusing on practicing and mproving my interviewing and facilitationskills, looking for as many opportunities to do this as possible.
-
BARNUM, Carol M. 2011. Usability Testing Essentials: Ready, Set-- Test! Amsterdam: Morgan Kaufmann Publishers.
BAXTER, Kathy, Catherine COURAGE and Kelly CAINE. 2015. Understanding Your Users: A Practical Guide to User Research Methods. Second edition. Amsterdam Boston Heidelberg: Elsevier, MK, Morgan Kaufmann.
FESSENDEN, Therese. 2021. ‘5 Facilitation Mistakes to Avoid During User Interviews’. Nielsen Norman Group [online]. Available at: https://www.nngroup.com/articles/interview-facilitation-mistakes/ [accessed 22 Mar 2023].
NIELSEN, Jakob. 2000. ‘Why You Only Need to Test with 5 Users’. Nielsen Norman Group [online]. Available at: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ [accessed 22 Mar 2023].
NIELSEN, Jakob. 2012. ‘Thinking Aloud: The #1 Usability Tool’. Nielsen Norman Group [online]. Available at: https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/ [accessed 22 Mar 2023].
PERNICE, Kara. 2021. ‘5 Facilitation Principles for Both UX Workshops and User Tests’. Nielsen Norman Group [online]. Available at: https://www.nngroup.com/articles/facilitate-ux-workshop-user-test/ [accessed 21 Mar 2023].
ROSALA, Maria. 2022. ‘Checklist for Moderating a Usability Test’. Nielsen Norman Group [online]. Available at: https://www.nngroup.com/articles/usability-checklist/ [accessed 22 Mar 2023].
Thank you for reading.
