Usability Evaluation of the Spatial OLAP Visualization and Analysis Tool (SOVAT)  

Increasingly sophisticated technologies, such as On-Line Analytical Processing (OLAP) and Geospatial Information Systems (GIS), are being leveraged for conducting community health assessments (CHA). Little is known about the usability of OLAP and GIS interfaces with respect to CHA. We conducted an iterative usability evaluation of the Spatial OLAP Visualization and Analysis Tool (SOVAT), a software application that combines OLAP and GIS. A total of nine graduate students and six community health researchers were asked to think-aloud while completing five CHA questions using SOVAT. The sessions were analyzed after every three participants and changes to the interface were made based on the findings. Measures included elapsed time, answers provided, erroneous actions, and satisfaction. Traditional OLAP interface features were poorly understood by participants, and combined OLAP-GIS features needed to be better emphasized. The results suggest that the changes made to the SOVAT interface resulted in increases in both usability and user satisfaction.

Scotch, Mathew, Bambang Parmanto and Valerie Monaco. Journal of Usability Studies (2007). Articles>Usability>EPSS>Case Studies

Usability for the Masses    

The biggest problem facing the usability field is how to scale up massively so that we can impact all the user interface designs in the world. How big is this challenge? As of November 2005, there are about 75 million websites on the Internet. There are also about 30 million intranets inside corporate firewalls. Thus, there are more than 100 million user interface designs, just in the online space.

Nielsen, Jakob. Journal of Usability Studies (2005). Articles>Usability>International

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing    

Usability testing a mobile application in the laboratory seems to be sufficient when studying user interface and navigation issues. The usability of a consumer application was tested in two environments: in a laboratory and in a field with a total of 40 test users. The same problems were found in both environments, differences occurred in the frequency of findings between the contexts. Results indicate that conducting a time-consuming field test may not be worthwhile when searching user interface flaws to improve user interaction. In spite of this, it is possible that field testing is worthwhile when combining usability tests with a field pilot or contextual study where user behavior is investigated in a natural context.

Kaikkonen, Anne, Aki Kekäläinen, Mihael Cankar, Titti Kallio and Anu Kankainen. Journal of Usability Studies (2005). Articles>Usability>Testing

Usability Testing of Travel Websites    

A usability study was conducted to identify usability problems as well as recommendations for improvement for three travel sales websites. The study performed testing on twenty participants, between the ages of 19 and 65, recruited from the university campus consisting of students, faculty, and staff. The three websites tested were Expedia.com, Orbitz.com, and Travelocity.com. Each participant was given general instructions and a pre-survey to determine their demographics and level of Internet experience. The usability study tested participants on the task of finding the same itinerary on each travel website. The participant during testing was under observation of the experimenter that maintained an observation log. A post-survey along with a debriefing session was conducted to gather additional feedback. The average testing time for participants was 30 minutes. The results of this study are presented as well as a future research discussion consisting of the development of usability guidelines for designers of travel websites.

Carstens, Deborah S. and Pauline Patterson. Journal of Usability Studies (2006). Articles>Web Design>Usability>E Commerce

User Research of a Voting Machine: Preliminary Findings and Experiences    

This paper describes a usability study of the Nedap voting machine in the Netherlands. On the day of the national elections, 566 voters participated in our study immediately after having cast their real vote. The research focused on the correspondence between voter intents and voting results, distinguishing between usability (correspondence between voter intents and voter input) and machine reliability (correspondence between voter input and machine output). For the sake of comparison, participants also cast their votes using a paper ballot.

de Jong, Menno D.T., Joris van Hoof and Jordy Gosselt. Journal of Usability Studies (2007). Articles>Usability>Government>Netherlands

Using Eye Tracking to Compare Web Page Designs: A Case Study    

A proposed design for the American Society of Clinical Oncology (ASCO) Web site was evaluated against the original design in terms of the ease with which the right starting points for key tasks were located and processed. This report focuses on the eye tracking methodology that accompanied other conventional usability practices used in the evaluation. Twelve ASCO members were asked to complete several search tasks using each design. Performance measures such as click accuracy and time on task were supplemented with eye movements which allowed for an assessment of the processes that led to both the failures and the successes. The report details three task examples in which eye tracking helped diagnose errors and identify the better of the two designs (and the reasons for its superiority) when both were equally highly successful. Advantages and limitations of the application of eye tracking to design comparison are also discussed.

Bojko, Agnieszka. Journal of Usability Studies (2006). Articles>Web Design>Usability>Eye Tracking

WAP and Accountability: Shortcomings of the Mobile Internet as an Interactional Problem  

Wireless Application Protocol (WAP) is designed to allow access to the Internet on a mobile phone. Attempts to explain its limited success have focused on attitudinal and cognitive reasons for non-use, finding that although people recognize the benefits of WAP, issues like lack of content, privacy concerns, and reference group behavior account for non-use. Such explanations have also been incomplete in that they have not addressed problems related to actual use and interaction with the technology. Our article studies the use of WAP as situated action. We focus on how users make sense of WAP pages and how they disambiguate in situ the responses from the service, i.e., new pages and new menus. Our method of transcribing videos of WAP use following the conventions of conversation analysis offers a cost-effective tool for understanding user interaction with technology and provides useful implications for design.

Koskinen, Ilpo, Petteri Repo and Kaarina Hyvönen. Journal of Usability Studies (2006). Design>Web Design>Wireless Web>Interaction Design

When 100% Really Isn't 100%: Improving the Accuracy of Small-Sample Estimates of Completion Rates    

Small sample sizes are a fact of life for most usability practitioners. This can lead to serious measurement problems, especially when making binary measurements such as successful task completion rates (p). The computation of confidence intervals helps by establishing the likely boundaries of measurement, but there is still a question of how to compute the best point estimate, especially for extreme outcomes. In this paper, we report the results of investigations of the accuracy of different estimation methods for two hypothetical distributions and one empirical distribution of p. If a practitioner has no expectation about the value of p, then the Laplace method ((x+1)/(n+2)) is the best estimator. If practitioners are reasonably sure that p will range between .5 and 1.0, then they should use the Wilson method if the observed value of p is less than .5, Laplace when p is greater than .9, and maximum likelihood (x/n) otherwise.

Lewis, James R. and Jeff Sauro. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods

Creating Effective Decision Aids for Complex Tasks  

Engineering design tasks require designers to continually compare, weigh, and choose among many complex alternatives. The quality of these selection decisions directly impacts the quality, cost, and safety of the final product. Because of the high degree of uncertainty in predicting the performance of alternatives while they are still just sketches on the drawing board, and the high cost of poor choices, mathematical decision methods incorporating uncertainty have long held much appeal for product designers, at least from a theoretical standpoint.

Clarke Hayes, Caroline and Farnaz Akhavi. Journal of Usability Studies (2008). Articles>Usability>Knowledge Management>EPSS

Switching Between Tools in Complex Applications  

Usability practice needs a procedure to identify, record, count, and highlight tool switch events for study. This paper describes one that supports the trained observers on which User-Centered Design relies to detect problems and causes, and evaluate design changes.

Schroeder, Will. Journal of Usability Studies (2008). Articles>Usability>User Centered Design>Methods

Unexpected Complexity in a Traditional Usability Study  

This article is a case study of a demonstration project intended to prove the value of usability testing to a large textbook publishing house. In working with a new client, however, the research team discovered that what our client thought were simple problems for their users were actually complex problems that required the users to evaluate potential solutions in a surprisingly complex context of use. As Redish (2007) predicted, traditional ease of use measures were "not sufficient" indicators and failed to reveal the complex nature of the tasks. Users reported high levels of satisfaction with products being tested and believed they had successfully completed tasks which they judged as easy to complete when, in fact, they unknowingly suffered failure rates as high as 100%. The study recommends that usability specialists expand our definition of traditional usability measures so that measures include external assessment by content experts of the completeness and correctness of users' performance. The study also found that it is strategically indispensable for new clients to comprehend the upper end of complexity in their products because doing so creates a new space for product innovation. In this case, improving our clients' understanding of complexity enabled them to perceive and to take advantage of a new market niche that had been unrealized for decades.

Howard, Tharon W. Journal of Usability Studies (2008). Articles>Usability>Testing>Methods

User Experience Design: The Evolution of a Multi-Disciplinary Approach  

Easy task completion (traditional usability) is not enough in the Web world. Appealing visual site design is not enough. A site visitor needs to not only be attracted to a site and able to figure out how to buy (or register, sign up, etc.)-they need in addition to be able to tell quickly that a site will meet their needs, and they need to want to buy from this site, as opposed to a competitor's site. This is a key aspect of overall Web site success.

Mayhew, Deborah J. Journal of Usability Studies (2008). Articles>Web Design>User Experience>Usability

An Empirical Investigation of Color Temperature and Gender Effects on Web Aesthetics  

Limited research exists on the relevance of hedonic dimensions of human-computer interaction to usability, with only a small set of this research being empirical in nature. Furthermore, previous research has obtained mixed support for gender differences regarding perceptions of attractiveness and usability in Web site design. This empirical research addresses the above gap by studying the effects of color temperature and gender on perceptions of Web site aesthetics.

Coursaris, Constantinos K., Sarah J. Sweirenga and Ethan Watrall. Journal of Usability Studies (2008). Articles>Web Design>Aesthetics>Gender

Examining Users on News Provider Web Sites: A Review of Methodology  

This project implemented and reviewed several methods to collect data about users' information seeking behavior on news provider Web sites. While browsing news sites, participants exhibited a tendency toward a breadth-first search approach where they used the home page or a search results page as a hub to which they returned and then linked to other pages. Generally, they browsed before using search. Information seeking patterns were consistent within-user but varied somewhat across users. Most behaviors were characterized as visually scanning with users spending much time scrolling.

Gibbs, William. Journal of Usability Studies (2008). Articles>Web Design>User Centered Design

A Voyage to Maturing Usability  

In this article, the chief editor of the recently published book Maturing Usability: Quality in Software, Interaction and Value reports her experiences, from the very beginning when the book project was conceived to the time when the book was delivered.

Law, Effie Lai-Chong. Journal of Usability Studies (2008). Articles>Usability>Publishing

A Low-Cost Test Environment for Usability Studies of Head-Mounted Virtual Reality Systems  

There is a need to develop new usability testing environments and methodologies for unconventional interactive systems. Pursuant to that need, we developed a low-cost test environment for a Head-Mounted Display (HMD)-based, virtual reality system called Osmose. Osmose was difficult to test for many reasons, one of which was its style of interaction. We began setting up the testing environment about two weeks before the start of the usability testing. We learned many lessons throughout the experience. This paper summarizes the study findings, both methodological - how to setup and conduct a usability lab for such an environment - as well as conceptual -the human experiences and behavioral patterns involved in using an immersive environment.

Seffah, Ahmed, Jonathan Benn and Halima Habieb Mammar. Journal of Usability Studies (2008). Articles>Usability>Testing>3D

How May I Help You? An Ethnographic View of Contact-Center HCI  

This study used an applied ethnographic research method to investigate human-computer interaction (HCI) between call center agents and agent-facing software in the context of contact-center culture. Twenty semi-structured interviews were completed, along with non-participant observation at two contact centers, one that followed a user-centered design (UCD) process for software development and another that did not. Agent productivity and satisfaction at the non-UCD center were hampered by poor task-UI integration, ambiguous text labels, and inadequate UI standardization. Agents required multiple applications to complete a single task, leading to long task times and cognitive strain. In contrast, the UCD center used a unified UI that reduced task times and decreased cognitive strain. In both centers, the workflow was reported to be stressful at times; however, management at both companies employed high involvement work processes that mitigated this stress. Implications for possible high-involvement UI design are considered and a strategy for applied ethnographic research is discussed.

Kiewe, Howard. Journal of Usability Studies (2008). Articles>Human Computer Interaction>Usability>Contextual Inquiry

A New Approach to Analyse Human-Mobile Computer Interaction  

This paper describes a tool for log file recording and a method for quickly and easily analysing human-computer interaction with mobile devices. The tool logs screenshots and quantitative interaction data, such as number of clicks and timestamps. The analysing tool provides the ability to evaluate the interaction sequences and to export an MS Excel®-sheet for statistical analysis. To evaluate the tool, a usability study was conducted comparing the effectiveness of this tool in the laboratory and in the mobile context. Findings show that the tool is the first step toward a very effective, unobtrusive analysing method for user interaction in the mobile context. Combined with debriefing methods, it would be an optimized way for usability testing with mobile devices.

Kawalek, Jurgen, Annegret Stark and Marcel Riebeck. Journal of Usability Studies (2008). Articles>Human Computer Interaction>Wireless Web>Log Analysis

Using Eye Tracking to Compare Web Page Designs: A Case Study  

A proposed design for the American Society of Clinical Oncology (ASCO) Web site was evaluated against the original design in terms of the ease with which the right starting points for key tasks were located and processed. This report focuses on the eye tracking methodology that accompanied other conventional usability practices used in the evaluation. Twelve ASCO members were asked to complete several search tasks using each design. Performance measures such as click accuracy and time on task were supplemented with eye movements which allowed for an assessment of the processes that led to both the failures and the successes. The report details three task examples in which eye tracking helped diagnose errors and identify the better of the two designs (and the reasons for its superiority) when both were equally highly successful. Advantages and limitations of the application of eye tracking to design comparison are also discussed.

Bojko, Agnieszka. Journal of Usability Studies (2006). Articles>Web Design>Usability>Eye Tracking

Can Collaboration Help Redefine Usability?    

There are countless usability blogs, message boards and listservers. But to my knowledge, no one has attempted to integrate all this information into a single, collaborative knowledge space. I believe that creating such a knowledge space would be of immense benefit to the usability profession and would be a wonderful platform on which to refine our understanding of social computing and knowledge management.

Kreitzberg, Charles B. Journal of Usability Studies (2006). Articles>Collaboration>Usability>Help

International Standards for Usability Should Be More Widely Used  

Despite the authoritative nature of international standards for usability, many of them are not widely used. This paper explains both the benefits and some of the potential problems in using usability standards in areas including user interface design, usability assurance, software quality, and usability process improvement.

Bevan, Nigel. Journal of Usability Studies (2009). Articles>Usability>Standards>International

Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale  

The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including Web sites, cell phones, interactive voice response systems, TV applications, and more. It provides an easy-to-understand score from 0 (negative) to 100 (positive). While a 100-point scale is intuitive in many respects and allows for relative judgments, information describing how the numeric score translates into an absolute judgment of usability is not known. To help answer that question, a seven-point adjective-anchored Likert scale was added as an eleventh question to nearly 1,000 SUS surveys. Results show that the Likert scale scores correlate extremely well with the SUS scores (r=0.822). The addition of the adjective rating scale to the SUS may help practitioners interpret individual SUS scores and aid in explaining the results to non-human factors professionals.

Bangor, Aaron, Philip Kortum and James Miller. Journal of Usability Studies (2009). Articles>Usability>Assessment>Standards

Extremely Rapid Usability Testing  

The trade show booth on the exhibit floor of a conference is traditionally used for company representatives to sell their products and services. However, the trade booth environment also creates an opportunity, for it can give the development team easy access to many varied participants for usability testing. The question is can we adapt usability testing methods to work in such an environment? Extremely rapid usability testing (ERUT) does just this, where we deploy a combination of questionnaires, interviews, storyboarding, co-discovery, and usability testing in a trade show booth environment. We illustrate ERUT in actual use during a busy photographic trade show. It proved effective for actively gathering real-world user feedback in a rapid paced environment where time is of the essence.

Pawson, Mark and Saul Greenberg. Journal of Usability Studies (2009). Articles>Usability>Testing>Methods

The Effect of Culture on Usability: Comparing the Perceptions and Performance of Taiwanese and North American MP3 Player Users  

A study of how 23 Taiwanese and North American subjects use a consumer electronic product shows that culture strongly affects the usability of the product. Survey data shows that North American users had much lower levels of user satisfaction and perceptions of effectiveness and efficiency than Taiwanese users. On the other hand, results on performance were unclear, indicating similar levels of effectiveness for both cultural groups and conflicting results on levels of efficiency.

Wallace, Steve and Hsiao-Cheng Yu. Journal of Usability Studies (2009). Articles>Usability>Assessment>International