Usability Research
How does the usability of stationary eye-tracking systems (ETS) compare to mobile counterparts in providing an accessible entry point for novice users?
-
Timeframe
March 2022 - June 2022
-
Role
UX Researcher
-
Team
A team of four UX Researchers
01. Background
The growing significance of product usability highlights the need to evaluate and enhance user experiences. Eyetracking systems play a pivotal role in assessing product usability, offering insights into aspects such as app clarity, ease of settings navigation, and user distractions. Paradoxically, these very eyetracking systems often exhibit poor usability.
02. Objective
This study aimed to explore whether a stationary eyetracking system, in comparison to its mobile counterpart, provides a superior entry point for inexperienced users. The hypothesis was that the use of stationary eyetracking technology has a better usability (satisfaction, effectiveness, and efficiency) in capturing relevant metrics.
"Dikablis Essential" from "Ergoneers" as Mobile ETS
"SMI-RED250" from "SensoMotoric Instruments" as Stationary ETS
„When a stationary ET technology is used in contrast to a mobile ET technology, this leads to better usability and user experience.“
– The Hypothesis
03. Methodology
To address the research question, a study comparing stationary and mobile eyetracking systems was conducted. Eight diverse groups consisting of in total 23 subjects underwent both stationary and mobile eyetracking processes, divided into three phases: calibration, use case, and export. Each phase involved time measurements, and participants completed User Experience Questionnaires (UEQs) after each phase. Post the entire eyetracking process, individual group members filled out a questionnaire, followed by a small group discussion.
Calibration
Participants configured the respective technology for precise eye movement tracking.
Post-configuration, the executing operator filled out an UEQ.
Use Case
Participants performed a specific use-case guided by the moderator.
The task involved navigating the "otto.de" website, adding an "iPad Air 2022" in "space-grey" with a "36-month OTTO Extended Warranty" to the cart, and finding the cheapest pair of size 40 socks to add to the cart.
The choice of the "Otto" website was due to the stationary ETS running on Windows XP, necessitating a site with minimal JavaScript.
After completing the use case, the participant filled out an UEQ, evaluating their experience during the task execution.
Export
Participants exported a gaze path and a heatmap.
Post-export, the operator completed an UEQ, assessing the experience during file export.
Questionnaire
The eyetracking process concluded with a questionnaire capturing demographic data and qualitative insights.
Group Discussion
A small group discussion followed, where eyetracking features were ranked by importance, and positive/negative attributes of both eyetrackers were discussed and documented.
04. Qualitative Results
In terms of usability application, both eyetracking systems were effective, with all participants achieving the set goals with both technologies. While overall efficiency showed minimal differences, participants using the stationary eyetracking system tended to reach their goals slightly faster. Survey responses and UEQ analyses revealed a substantial preference for the stationary eyetracking system. Nineteen out of 23 participants stated in the questionnaire that they believed they achieved better results with the stationary eyetracker.
05. Quantitative Results
To measure the user experience of the participants, they filled out a User Experience Questionnaire (UEQ) after each of the three phases, and the collected data from the UEQs was subsequently analyzed. Despite both Eye Tracking Systems (ETS) not exhibiting an outstanding user experience, the stationary Eyetracker consistently outperformed the mobile ETS in each phase.
Mobile ETS in the use case phase
Stationary ETS in the use case phase
06. Conclusion
The study validated the hypothesis, indicating that utilizing stationary eyetracking systems, as opposed to mobile eyetracking systems, leads to enhanced usability and user experience in capturing relevant metrics. The findings underscore the importance of considering the usability of the tools themselves, even in applications designed to evaluate the usability of other products. This case study demonstrates the value of a comparative analysis in informing technology choices for optimizing user experiences.