Eight Guidelines for Usability Testing

Eight essential guidelines for usability testing, helping you to plan, run and analyse usability tests.

Fidgeon, Tim. Webcredible (2005). Design>Web Design>Usability>Testing

Examining the Relationship Between Quality Writing and Quality Reading    

This article introduces the ISTE approach as a way to measure the relationship between the design of a document and the way that users handle it. ISTE is an acronym for Information SubTypes and Effects. The article describes the classification of information into a general typology as well as into a specific one, to illustrate how the approach can be adapted to specific usability questions. ISTE also requires a typology for categorizing possible user reactions. This article first positions ISTE within the broader context of usability testing and then describes its basic approach and some recent applications. Unlike most formal research methods, ISTE does not require an inordinate amount of time and effort for analyzing high-fidelity data. Electronic versions such as the ERR instrument allow for the creation of a complete and detailed profile of the use of a document 10 minutes after task completion. These characteristics make ISTE a viable instrument for gathering usability metrics in industrial settings.

van der Meij, Hans. Technical Communication Online (2000). Articles>Usability>Testing>Assessment

Experience Equity and Universal Access: Designing Clinical Studies for Low Vision  

In this paper, I describe web page design for those interested in conducting clinical, low vision studies. Ideally, web pages should be accessible and usable for all readers; however, the web is a highly visual medium for communication and creates serious accessibility issues for specialized (diverse needs) those with vision needs. Therefore, I propose that researchers consider a paramount and concurrent user-centered design approach when creating stimulus materials for these specialized audiences. This paper introduces readers to this design approach for a low vision audience as described in the WebText Study.

Reece, Gloria A. STC Proceedings (2005). Articles>Usability>Testing>Visual

Experiences in Remote Usability Evaluations  

Traditional usability testing occurs in a laboratory setting in which users are directly observed by test facilitators. However, the costs associated with accessing users for usability testing as well as the limited availability of many users make it necessary to consider new methods for gathering usability feedback for software products. Online, remote testing is a key to achieving a larger, more diverse pool of participants.

Cheatham, Deane and Velda Bartek. IBM (2004). Articles>Usability>Testing

Eye Gaze Patterns while Searching vs. Browsing a Website

This article discusses users' visual scan paths of web pages containing text and/or pictures while conducting browsing and searching tasks. User performance on three usability tasks on an e-commerce website is described. Results show that users follow a fairly uniform scan path when browsing through pictures, and a more random path while specifically searching through them. Additionally, users appeared to follow Nielsen's 'F' pattern (2006) while both browsing and searching through text-based pages.

Shrestha, Sav and Kelsi Lenz. Usability News (2007). Articles>Usability>Testing>Eye Tracking

Fast and Simple Usability Testing

Everyone knows by now that they should test the usability of their applications, but still hardly anybody actually does it. In this article I'll share some tips I've picked up for doing usability tests quickly and effectively. Relatively recent tools like Django and Ruby on Rails allow us to develop projects faster and to make significant changes later in the project timeline. Usability testing methods should now be adapted to fit this modern approach to development.

Downe, Natalie. 24 Ways to impress Your Friends (2005). Articles>Usability>Testing>Methods

Fast, Cheap, and Good: Yes, You Can Have It All

The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.

Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods

Getting the Complete Picture with Usability Testing

Good usability testing definitely provides an opportunity for clear-cut improvements in the usability of Web sites. In order to increase the chances of success with usability testing, it is important to measure effectiveness, efficiency, and satisfaction–they all measure different aspects of the usability of a Web site. If only one or two of these measures are used, it would provide an incomplete or partial picture of the possible human performance and user satisfaction results.

Bailey, Robert. Usability.gov (2006). Articles>Usability>Testing

Giveaways to Thank Helpful Users--What's Best?

What is the best way to thank helpful users for participating in a usability study? Carl Myhill asked this question to a popular discussion group of the Usability community. The following are some of the replies.

Dick, David J. Usability Interface (2004). Articles>Usability>Testing

Hotspots and Hyperlinks: Using Eye-Tracking to Supplement Usability Testing

This article discusses how eye-tracking can be used to supplement traditional usability test measures. User performance on two usability tasks with three e-commerce websites is described. Results show that eye-tracking data can be used to better understand how users initiate a search for a targeted link or web object. Frequency, duration and order of visual attention to Areas of Interest (AOIs) in particular are informative as supplemental information to standard usability testing in understanding user expectations and making design recommendations.

Russell, Mark C. Usability News (2005). Articles>Usability>Testing>Eye Tracking

Hotspots and Hyperlinks: Using Eye-Tracking to Supplement Usability Testing

This article discusses how eye-tracking can be used to supplement traditional usability test measures. User performance on two usability tasks with three e-commerce websites is described. Results show that eye-tracking data can be used to better understand how users initiate a search for a targeted link or web object. Frequency, duration and order of visual attention to Areas of Interest (AOIs) in particular are informative as supplemental information to standard usability testing in understanding user expectations and making design recommendations.

Russell, Mark C. uiGarden (2006). Articles>Usability>Testing>Eye Tracking

How Many Users Should You Test With in Usability Testing?

Doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?

Spillers, Frank. Demystifying Usability (2005). Articles>Usability>Testing>Methods

How Much Interaction is Too Much?

Watching a colleague facilitate usability testing, the author asks what constitutes too much interaction by the facilitator. What his colleagues had to say got me thinking.

Anderson, Clifford. Usability Interface (2004). Articles>Usability>Testing

Improving Information Quality Through Iterative Usability Testing  

Testing documents at each step of their development is one way to ensure that the final document is of high quality. It is not necessary to wait until a document is complete before we start testing; we can incorporate iterative testing into the information-development process so that we can build in quality each step of the way.

Grice, Roger A. STC Proceedings (1994). Articles>Usability>Testing

In Praise of Usability Testing

Documentation usability testing has been with us at least since 1983. But, to all intents and purposes, it seems that there is still very little testing being done. Of all the documentation quality assurance activities that we can carry out, usability testing represents one of the best values in the process chain. Without devaluing any of the other quality assurance functions, it is safe to say that money spent on usability testing is extremely well spent.

Hosier, William J. Boston Broadside (1991). Articles>Usability>Testing

Instructions for Branch Office Testing

These are the instructions we gave to the people at various Sun branch offices in Europe and Asia for their user testing of a new design for the company's web pages. In a few places, these instructions refer to web-specific issues, so they will have to be modified slightly for use in other projects. These instructions were sent by electronic mail to those local Sun reps who had volunteered to lead a test.

Nielsen, Jakob. Alertbox (1996). Articles>Usability>Methods>Testing

Introduction to Eyetracking: Seeing Through Your Users' Eyes

Over the coming months, I'll use eyetracking to evaluate a lot of world-renowned user interfaces--including Web sites like Amazon.com, Google News, and eBay; Rich Internet Applications (RIAs); and desktop applications--and analyze quantitative eyetracking data to provide best practices for designing user interface elements like navigation systems, menus, and forms, and for effective ad placement.

Penzo, Matteo. UXmatters (2005). Design>Usability>Testing>Eye Tracking

Is A Lab Essential For User Testing?

Once an organisation decides to go ahead with a user testing programme, the questions really begin. Is it really necessary to undertake testing in a 'usability lab'? And what exactly should a fully functioning lab consist of anyway? As one might imagine, opinion is divided on these issues. We take a quick look at what a typical lab might consist of and the pros and cons of lab-based testing.

Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods

Iterative Usability Research Methods: Why Testing Isn't Enough  

Discusses how to choose different usability methods for iterative research. Slides only.

Rosenbaum, Stephanie L. Tec-Ed, Inc. (1999). Articles>Usability>Testing>Methods

Iterative Usability Testing as Continuous Feedback: A Control Systems Perspective    

This paper argues that in the field of usability, debates about number of users, the use of statistics, etc. in the abstract are pointless and even counter-productive. We propose that the answers depend on the research questions and business objectives of each project and thus cannot be discussed in absolute terms. Sometimes usability testing is done with an implicit or explicit hypothesis in mind. At other times the purpose of testing is to guide iterative design. These two approaches call for different study designs and treatment of data. We apply control systems theory to the topic of usability to highlight and frame the value of iterative usability testing in the design lifecycle. Within this new metaphor, iterative testing is a form of feedback which is most effective and resource-efficient if done as often as practically possible with project resources and timelines in mind.

Genov, Alex. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods

Journaled Sessions

Journaled sessions bridges usability inquiry, where you ask people about their experiences with a product, and usability testing, where you observe people experiencing the product's user interface. Journaled sessions are often used as a remote inquiry method for software user interface evaluation. A disk is distributed to a number of test subjects containing a prototype of the software product, as well as additional code to capture (or journalize) the subjects' actions when using the prototype. Users perform several tasks with the prototype, much as in formal usability tests, and their actions are captured with the journalizing software. Upon completion of the series of tasks, the users return the disks to you for you to evaluate. Because the journaling portion of the evaluation is largely automated, this approach to remote, hands-off inquiry is certainly more 'usable' then self-reporting logging, where users are requested to write down their observations and comments and send them back to you.

Hom, James. VWH.net. Articles>Usability>Methods>Testing

Lessons Learned from Usability Testing of the Documentation  

This presentation and demonstration will first establish the principles behind usability testing of the documentation, then show examples of lessons learned from testing both print and online documentation. Video clips of actual tests will be used to make some compelling points. The session will be especially useful to those who are interested in usability testing but haven't done it yet.

Barnum, Carol M. STC Proceedings (1998). Articles>Documentation>Usability>Testing

Low Budget Usability Testing Can Work

With just a modest amount of equipment and effort, you can uncover some serious product glitches and potentially earn your company thousands of dollars and a sterling reputation. The first step is to get a video camera, a product prototype, and a small sample of your target audience.

Umbhua, Kurt. Boston Broadside (1991). Articles>Usability>Testing

Low-Cost Usability Testing

Southwest Airlines' low-cost methods for formal usability testing.

Hurst, Bob. Usability Professionals Association (2005). Articles>Usability>Testing