OVERVIEW
PROJECT TYPE
Heuristic evaluation
Moderated user testing
DATE
Nov 2022
HEURISTIC EVALUATION
PROJECT CONTEXT
As part of my graduate assistantship with UMD’s Division of IT, I was asked to help assess the usability of the University’s app, focusing especially on layout and ease of navigation. This work also served to fulfill a project requirement for course INST 631 - Intro to Human-Computer Interaction.
NIELSON’S 10 USABILITY HEURISTICS
Before beginning the user testing I conducted a heuristic evaluation to become familiar with the app and to identify problem areas to focus on in the testing. I examined the app from the perspective of each of Nielson’s 10 Usability Heuristics which provide general principles for successful interaction design.
I uncovered a number of issues during the heuristic eval, but the high level issues that stood out were confusing navigation and inconsistent layouts/designs. For example:
Heuristic 1 - Visibility of system status - UMD app lacks a navigational tool such as a breadcrumb trail to let users know where they are and provide an easy way to backtrack. Users could easily get lost in the app’s many layers of links and pages.
Heuristic 3 - User control and freedom - The behavior of the back arrow in the upper left corner of the screen is inconsistent across the app, sometimes bringing the user back to the previous page (as desired) and sometimes bringing them all the way back to the homepage, skipping all of the pages in between. This takes control away from the user and introduces unpleasant navigational surprises.
Heuristic 6 - Recognition rather than recall - Only the top level links are visible from the homepage, making it difficult to remember where subpages are located. (eg. Are STAMP events found under Calendar or Get Involved?) Creating a side menu with toggles so the next level links can also be displayed would allow users to easily find the page they’re looking for without having to click into each link to find it.
VIEW FULL HEURISTIC EVALUATION
USER TESTING
For the second half of my evaluation of the UMD campus app, I conducted remote, moderated usability testing with five participants using UserZoom Go.
As the moderator I was able to see the participants’ phone screen as well video from their front facing camera so that I could watch their facial expressions and body language.
I emphasized to each participant that our goal was not to test their skills or abilities as a user, but to test the app interface and its ease of use.
METHOD
SOFTWARE
PARTICIPANTS
All of the participants recruited were current students or had graduated within six months of the study (two as undergraduates and three as master’s students), and ranged in age from 20 - 28.
The IT team managing the app was interested in revamping the app and relaunching it at new student orientation, so I chose not to focus on current students for this particular study. Because none of the participants attended UMD, they approximated the knowledge level of students new to the university, either as Freshmen, first year grad students, or transfer students.
PARTICIPANT DATA
I chose tasks based on two main factors: I wanted to test features that I thought students would likely use most frequently, as well as less frequently used but important functions such as contacting the University Police. One of the tasks involved customizing the app menu settings and the other nine tasks were centered on information seeking.
I focused especially on tasks that students are likely to perform as they walk around campus (looking up dining hall information, finding study spaces, requesting a walking escort, etc.) Several of the tasks involving questions about intramurals and campus events were deliberately included to explore areas where I had identified potential problems in the heuristic evaluation.
Usability testing sessions ranged from 22 - 30 minutes in length. Each task attempt was timed and given one of three performance ratings: S for a successfully completed task; U for tasks that were completed unsuccessfully (participant completed the task, but gave the wrong answer); and A for tasks that were attempted but then abandoned.
After each session I rewatched the recording, writing down key quotes from participants and making note of observations such as navigational struggles and facial expressions.
The usability testing uncovered a number of issues with the app interface which resulted in frustrating user experiences.
Many of the problems identified in the heuristic evaluation such as the lack of navigational breadcrumbs and the inconsistent behavior of the back arrow also showed up during user testing. Examples of other issues included:
CALENDAR
All five participants were repeatedly confused when they looked for information on events and found that the Calendar on the homepage was blank except for a few holidays. They were further confused by the fact that the Get Involved page includes an ecalendar as well as additional events under TerpLink.
P01: “Is there nothing happening on that day?”
P05: “Is there just nothing on this calendar, or do I not know how to use it? Because there’s nothing showing up on this calendar for me.”
P05: “Oh there’s a whole other calendar, weird.”
Recommendation - Centralize all events on the main calendar and provide filters and a drop down menu so that users can narrow down the number of events they’re seeing at once.
HOMEPAGE VS MENU
Participants were very confused about the differences between the homepage and the menu page. They did not immediately realize that the menu page contained more icon options than the homepage, and struggled to find pages (such as the Police page) which were on the menu page, but not the homepage.
P03: “I feel like home and menu are kinda the same thing because there’s a lot of overlap.”
P04: “I really don’t like that you have this home with these icons and then there’s like four additional, or eight or however many additional ones on this page (menu). I feel like they should just be the same.”
P05: “I didn’t realize that the menu had other icons that the home page didn’t because they mostly have the same icons. So I figured if it wasn’t on the homepage that I wouldn’t be able to find it. Why are there two of them?”
Recommendation - Use a slide out menu in order to make the location of the menu more obvious and to provide more detail about the sub-links within each page. This would help alleviate the confusion between the homepage and current menu page, and would make it easier for users to quickly scan the menu and recognize the links they are looking for, rather than having to recall which page contained which links.
POLICE CONTACT NUMBER
Several users struggled to find the non-emergency number for the University Police. It is currently located under the Campus section of the Contact Us page which is not an intuitive categorization.
P03: “I feel like it’s not immediately obvious that police is under campus.”
Recommendation - Create a section specifically for the Police Department phone numbers so that users do not have to hunt for the information.
RECWELL PAGE DESIGN
Participants had trouble reading the small text on the RecWell page and were distracted by the many video ads. The busy layout caused some participants to click on things they didn’t intend to click on.
P04: “It’s really small and I accidentally clicked on a bunch of things I didn’t mean to.”
P05: “Now we have weird, outside Google ads.”
Recommendation - Clean up the content on the RecWell page and utilize the layout that’s already in use for the Dining and UMD Police pages. Using consistent layouts across pages also helps reduce navigational friction.
HELP DESK ICON
Although none of the tasks involved the Help Desk page (which brings users to the IT Service Desk), several of the usability testing sessions revealed that participants were unsure of what kind of information the page contained due to the ambiguous icon. Three participants clicked on Help Desk when looking for other information.
P02: “I’m gonna do Help Desk because I feel like that might be frequently asked questions.”
P04: “Oh wait, this is IT.”
Recommendation - Label the page as IT Service Desk and change the icon to a computer to make the page’s content more obvious to users.
VIEW FULL USER TESTING REPORT
The IT team managing the app is currently using the findings from the heuristic eval and user testing report to improve the app and revamp the homepage.
The plan is to conduct another round of user testing with the new and improved design in the near future.
CLAIRE NIEMEIER
