Ninety Percent of All Usability Testing is Useless

Ninety percent of all usability testing performed on Web sites is useless. This is not to say that it doesn’t have a significant role to play in user experience design. When done right, usability testing will improve your Web site and your development process, but the current culture surrounding Web site usability testing is such that it rarely benefits the design. Worse, this misapplication can undermine the acceptance of this important technique throughout an organization.

Becker, Lane. Adaptive Path (2004). Articles>Usability>Testing

Outliers and Luck in User Performance

6% of task attempts are extremely slow and constitute outliers in measured user performance. These sad incidents are caused by bad luck that designers can -- and should -- eradicate.

Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods

Planning Usability Tests For Maximum Impact  

Usability tests make products better. Those of us who have seen their results understand their value, but we sometimes have difficulty convincing managers or developers of their worth. The pay-off of a usability test depends on its context. Both the type of test you perform and the timing of that test influence what you can do with the results. At best, you can guide the design of a product or clearly demonstrate its usability. At worst, you can ruin your credibility and have no effect on the product whatsoever. This paper explores what you get from a usability test at each stage of the user interface design process. It also provides advice on selling testings to management.

McDaniel, Scott M. and Laura Snyder. STC Proceedings (2004). Articles>Usability>Testing

Protocol Considerations for Using Eye-Tracking in Website Usability Testing  

Eye-tracking systems can enrich a Website usability study by providing an additional method for observing users' behavior. While eye-trackers can provide valuable data, the pros and cons of adding eye-tracking to a usability study need to be considered before designing the study's protocol. This paper discusses the kinds of usability questions that benefit from eye-tracking data and considerations for designing and running the study. Our findings are based on work done in the Laboratory for Usability Testing and Evaluation (LUTE) at the University of Washington, which is equipped with the Eye-gaze Response Computer Aid (ERICA) system controlled by the Gaze Tracker software, both of which were developed by Eye Response Technologies, Inc.

Cuddihy, Elisabeth, Zhiwei Guan and Judith A. Ramey. STC Proceedings (2005). Articles>Usability>Testing>Eye Tracking

Quantitative Studies: How Many Users to Test?

We can define usability in terms of quality metrics, such as learning time, efficiency of use, memorability, user errors, and subjective satisfaction. Sadly, few projects collect such metrics because doing so is expensive: it requires four times as many users as simple user testing. Many users are required because of the substantial individual differences in user performance. When you measure people, you'll always get some who are really fast and some who are really slow. Given this, you need to average these measures across a fairly large number of observations to smooth over the variability.

Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods

Recording Screen Activity During Usability Testing

Recording what users do is a crucial aspect of usability testing. Fortunately, recording screen activity doesn’t necessarily cost much. Three Windows-based software programs range between $30 and $150 and offer excellent performance.

Fast, Karl. Boxes and Arrows (2002). Articles>Usability>Methods>Testing

Recruiting User Testing Participants

To meet your users’ needs, it is essential to know your audience and to design for them. A key way to do this is by identifying your Web site’s primary users and recruiting a sample for usability testing. Consider these four aspects.

Martin, Beth A. Usability.gov (2006). Articles>Usability>Testing>Methods

Remote Usability: Insight into New Tools

A review of software packages for moderated usability testing in a native environment via a telephone/web interface.

Madaio, Mike. Usability Professionals Association (2006). Articles>Usability>Testing>Online

The Return on Usability Testing

Usability testing is the qualitative measurement of user experience with a product or service, such as a website. Usability testing is for anyone who wants to make their website, software, device, etc. easier to use. Whether you employ thousands of people such as Microsoft or fall into the small business category, usability testing can play a significant role in giving you direct feedback from your customers.

Subramanian, Subha. GotoMedia (2005). Articles>Usability>Testing

Review: A Review of Morae 2.0 for Usability Testing

TechSmith's recent release, Morae 2.0, features a new graphing tool, integrated satisfaction survey, and embedded task definitions. The editable marker log in Observer and the improved timeline controls in the Manager improve operator efficiency. This article highlights these and other new features of the new 2.0.

Shrestha, Sav. Usability News (2007). Articles>Reviews>Usability>Testing

A Review of Morae for Usability Testing

TechSmith's Morae is a powerful tool that is useful for any usability study. Its simplicity, efficiency, cost effectiveness and portability has redefined the data collection process and allows researchers to focus on the user experience rather than the data collection tools. This article discusses some of the features of this tool and proposes some enhancements to make it even more powerful to usability practitioners.

Shrestha, Sav. Usability News (2007). Articles>Usability>Testing>Software

Rigor in Usability Testing    

Usability testing is an evaluation method used by technical communicators that can combine aspects of both quantitative and qualitative research methodologies. This article compares and contrasts the standards and techniques between these two methods of inquiry with particular emphasis on maintaining rigorous tests in regard to validity and reliability of the findings. Whether an evaluator relies on quantitative methods, qualitative methods, or both, should depend on the questions the research or evaluation seeks to answer. Both methods have well-established practices meant to ensure the validity and reliability of their findings. Not only should usability evaluators be competent within the method of inquiry they apply, they also need to help clients understand the legitimate application and limitations of their findings.

Hughes, Michael A. Technical Communication Online (1999). Articles>Usability>Testing>Methods

Role of the Highlights Video in Usability Testing: Rhetorical and Generic Expectations    

Despite the proliferation of books and articles dealing with usability testing as a sub-field of technical communication, there remains one artifact that is underdefined: the usability highlights video. Several sources suggest that usability testers should deliver a video to their clients, but there is no common description of the contents of this video or the rhetorical purpose it fulfills. In this article, we examine the use of the highlights video as described in the literature, but we go further to discover some ways practicing usability professionals understand the role of the highlights video. Through the use of generic conventions, rhetorical theory, and industrial practices, we attempt to draw conclusions that point to some common uses of the highlights video that can instruct both teachers in the usability classroom and practicing usability experts as they create videos for client projects.

Yeats, Dave and Locke Carter. Technical Communication Online (2005). Articles>Usability>Testing>Video

RSVP in Review: A Comparison of Programs for the PC and Handheld Devices

RSVP is a method of displaying text one word (or sentence) at a time in the same area of a screen. RSVP has been used as a method of teaching reading, speed reading, as well as assessing reading speed and comprehension in students and adults. At SURL, we have been interested in the use of RSVP with small-screen interfaces, such as those used on handheld devices.

Russell, Mark C. and Shannon Riley. Usability News (2003). Articles>Usability>Testing

Scheduling Hard-to-Find Users

Developers may hesitate to start usability testing because they worry that their product poses special problems in finding, scheduling, or compensating the right users. This shouldn’t stop them. We successfully find and test hundreds of users a year and about 10% of these require special tactics for scheduling.

User Interface Engineering (1998). Articles>Usability>Methods>Testing

Six Slick Tests for Docs and Help

Usability testing isn’t just for software and web sites. Testing documentation can ensure that it includes — and accurately conveys — all the information users expect and need. Testing gives you accurate information on how well your documentation and Help work. It can even uncover problems that are better solved by changing the interface.

User Interface Engineering (1998). Articles>Documentation>Usability>Testing

Six Tips to Ensure a Successful Usability Test

Success in usability testing is learning what you need to know. That includes finding out both what is working well and what is not working well. Focusing on formative tests—with an eye toward identifying problems and bringing the issues to the team—is the secret to successful usability testing.

Redish, Janice C. 'Ginny'. Usability Interface (2005). Articles>Usability>Testing

Stratégie Web: Le Test d'Utilisabilité

La facilité d'utilisation conditionne la rentabilité d'un site web. Le test d'utilisabilité permet de la mesurer et d'identifier les problèmes que rencontre l'utilisateur. Grâce au test, il est possible d'améliorer le design d'un site web en éliminant ces difficultés qui freine l'internaute dans son achat.

Nogier, Jean-Francois. Usabilis (2002). (French) Articles>Usability>Testing

Students Perform Usability Testing for Industry Clients  

This paper describes how undergraduate students at Mercer University designed and performed hands-on usability tests on seven different products from a wide range of outside clients. Though the projects were challenging and quite difficult for undergraduates, they resulted in significant learning for students and important usability data for the clients. The professor describes the course design and discusses the clients and projects, and three undergraduates report their experiences serving on project teams as project leader, usability specialist, and technical specialist.

Davis, Marjorie T., Corley S. Brown, Erik T. Lindborg and Katherine T. Watson. STC Proceedings (2004). Articles>Education>Usability>Testing

Super Easy Usability Testing

Self-described as the absolute [sic] easiest introduction to usability testing you could possibly find anywhere.

Rhodes, John S. WebWord (1999). Articles>Usability>Methods>Testing

Taking Usability Testing to the Field    

Know your audiences, comes the repeated message for technical communicators and in response, more and more technical communicators have turned to usability testing to learn more about their readers and to improve their communications. Technical communicators produce manuals, instructions, and warnings for hand tools, medical equipment, lawn mowers, tractors, pesticide sprayers, and thousands of different products. Most manuals, instructions, and products can benefit from usability testing. This case study provides guidance for technical communicators who are novices to usability testing. The lessons we learned can be of value to technical communicators beginning their first usability testing on a wide range of technical communications and products.

Zimmerman, Donald E., Michel Lynn Muraski and Michael D. Slater. Technical Communication Online (1999). Articles>Usability>Testing>Methods

Test That Information!  

Software products are tested. Hardware products are tested. But, how often and how thoroughly are the information deliverables tested? In the rush to bring products to market, the full and thorough testing of information often gets overlooked. Last minute changes are crammed in. Procedures are not thoroughly tested. Even something as simple as testing links can be ignored due to lack of time or resources.

Crawford, Vanadis. STC Proceedings (2005). Articles>Documentation>Usability>Testing

Testing Incentives: The Best Way to Pay

The topic of test subject compensation generates a lot of conversation...how do you motivate test participants?

Anderson, Clifford. Boxes and Arrows (2007). Articles>Usability>Testing>Methods

Towards the Design of Effective Formative Test Reports    

Many usability practitioners conduct most of their usability evaluations to improve a product during its design and development. We call these 'formative' evaluations to distinguish them from 'summative' (validation) usability tests at the end of development. A standard for reporting summative usability test results has been adopted by international standards organizations. But that standard is not intended for the broader range of techniques and business contexts in formative work. This paper reports on a new industry project to identify best practices in reports of formative usability evaluations. The initial work focused on gathering examples of reports used in a variety of business contexts. We define elements in these reports and present some early guidelines on making design decisions for a formative report. These guidelines are based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation. Future work will continue to investigate industry practice and conduct evaluations of proposed guidelines or templates.

Theofanos, Mary Frances and Whitney Quesenbery. Journal of Usability Studies (2006). Articles>Usability>Testing>Reports