Journaled Sessions

Journaled sessions bridges usability inquiry, where you ask people about their experiences with a product, and usability testing, where you observe people experiencing the product's user interface. Journaled sessions are often used as a remote inquiry method for software user interface evaluation. A disk is distributed to a number of test subjects containing a prototype of the software product, as well as additional code to capture (or journalize) the subjects' actions when using the prototype. Users perform several tasks with the prototype, much as in formal usability tests, and their actions are captured with the journalizing software. Upon completion of the series of tasks, the users return the disks to you for you to evaluate. Because the journaling portion of the evaluation is largely automated, this approach to remote, hands-off inquiry is certainly more 'usable' then self-reporting logging, where users are requested to write down their observations and comments and send them back to you.

Hom, James. VWH.net. Articles>Usability>Methods>Testing

Lessons Learned from Usability Testing of the Documentation  

This presentation and demonstration will first establish the principles behind usability testing of the documentation, then show examples of lessons learned from testing both print and online documentation. Video clips of actual tests will be used to make some compelling points. The session will be especially useful to those who are interested in usability testing but haven't done it yet.

Barnum, Carol M. STC Proceedings (1998). Articles>Documentation>Usability>Testing

Low Budget Usability Testing Can Work

With just a modest amount of equipment and effort, you can uncover some serious product glitches and potentially earn your company thousands of dollars and a sterling reputation. The first step is to get a video camera, a product prototype, and a small sample of your target audience.

Umbhua, Kurt. Boston Broadside (1991). Articles>Usability>Testing

Low-Cost Usability Testing

Southwest Airlines' low-cost methods for formal usability testing.

Hurst, Bob. Usability Professionals Association (2005). Articles>Usability>Testing

Measuring Online Experience: It's About More Than Time!

Users' specific online activities, their knowledge of computers and the Internet, and how they feel about being online could differ significantly. In order to obtain a more robust measure, researchers may want to consider how and what users think, feel, and do online when they assess level of experience. In this article we explore each of these areas and present a new measure of measuring online experience.

Rogers, Bonnie Lida. Usability News (2003). Articles>Usability>Testing>Online

A Methodology for Testing Voting Systems  

This paper compares the relative merit in realistic versus lab style experiments for testing voting technology. By analyzing three voting experiments, we describe the value of realistic settings in showing the enormous challenges for voting process control and consistent voting experiences. The methodology developed for this type of experiment will help other researchers to test polling place protocols and administration. Comparing the results from laboratory experiments with voter verification and realistic voting experiments further validates the procedure of testing equipment in laboratory settings. The methodology and protocol for testing voting systems can be applied to any voting technology. This protocol matches the real-world conditions of voting by replicating them for the experiment.

Selker, Ted, Elizabeth Rozenwieg and Anna Pandolfo. Journal of Usability Studies (2006). Articles>Usability>Testing>Civic

Methods for Successful 'Thinking Out Loud' Procedure

When you are screening the participants for a study, notice how they respond to your questions. Decide on a strategy for engaging the participant before they arrive for the usability study. Be careful of the social dynamics you set up with the participant.

Ramey, Judith A. STC Usability SIG (1998). Resources>Usability>Testing>Methods

Multiple-User Simultaneous Testing (MUST)

Testing 5-10 users at once lets you conduct large-scale usability testing and still meet your deadlines.

Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods

The Myth of the Genius Designer

Having a good designer doesn't eliminate the need for a systematic usability process. Risk reduction and quality improvement both require user testing and other usability methods.

Nielsen, Jakob. Alertbox (2007). Articles>Web Design>Usability>Testing

The Need for Usability Analysis  

An overview of methods for usability testing and analysis.

Xerox (1995). Presentations>Usability>Testing>Methods

Ninety Percent of All Usability Testing is Useless

Ninety percent of all usability testing performed on Web sites is useless. This is not to say that it doesn’t have a significant role to play in user experience design. When done right, usability testing will improve your Web site and your development process, but the current culture surrounding Web site usability testing is such that it rarely benefits the design. Worse, this misapplication can undermine the acceptance of this important technique throughout an organization.

Becker, Lane. Adaptive Path (2004). Articles>Usability>Testing

Outliers and Luck in User Performance

6% of task attempts are extremely slow and constitute outliers in measured user performance. These sad incidents are caused by bad luck that designers can -- and should -- eradicate.

Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods

Performance Testing

Performance testing is a rigorous usability evaluation of a working system under realistic conditions to identify usability problems and to compare measures such as success rate, task time and user satisfaction with requirements.

UsabilityNet. Resources>Usability>Testing

Planning Usability Tests For Maximum Impact  

Usability tests make products better. Those of us who have seen their results understand their value, but we sometimes have difficulty convincing managers or developers of their worth. The pay-off of a usability test depends on its context. Both the type of test you perform and the timing of that test influence what you can do with the results. At best, you can guide the design of a product or clearly demonstrate its usability. At worst, you can ruin your credibility and have no effect on the product whatsoever. This paper explores what you get from a usability test at each stage of the user interface design process. It also provides advice on selling testings to management.

McDaniel, Scott M. and Laura Snyder. STC Proceedings (2004). Articles>Usability>Testing

Protocol Considerations for Using Eye-Tracking in Website Usability Testing  

Eye-tracking systems can enrich a Website usability study by providing an additional method for observing users' behavior. While eye-trackers can provide valuable data, the pros and cons of adding eye-tracking to a usability study need to be considered before designing the study's protocol. This paper discusses the kinds of usability questions that benefit from eye-tracking data and considerations for designing and running the study. Our findings are based on work done in the Laboratory for Usability Testing and Evaluation (LUTE) at the University of Washington, which is equipped with the Eye-gaze Response Computer Aid (ERICA) system controlled by the Gaze Tracker software, both of which were developed by Eye Response Technologies, Inc.

Cuddihy, Elisabeth, Zhiwei Guan and Judith A. Ramey. STC Proceedings (2005). Articles>Usability>Testing>Eye Tracking

Quantitative Studies: How Many Users to Test?

We can define usability in terms of quality metrics, such as learning time, efficiency of use, memorability, user errors, and subjective satisfaction. Sadly, few projects collect such metrics because doing so is expensive: it requires four times as many users as simple user testing. Many users are required because of the substantial individual differences in user performance. When you measure people, you'll always get some who are really fast and some who are really slow. Given this, you need to average these measures across a fairly large number of observations to smooth over the variability.

Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods

Reading to Decide: Designing for Usability with a Needs, Users, and Learnings (NUL) Analysis  

Information products that help users make decisions are a challenge to design, develop, and test for usability. We can begin to formulate a strategy for producing such documents effectively by doing a comprehensive preassessment of the specific project and by adapting testing methodologies from studies of creative thinking processes and expert system design.

Knodel, Elinor L. STC Proceedings (1995). Design>Usability>Methods>Testing

Recording Screen Activity During Usability Testing

Recording what users do is a crucial aspect of usability testing. Fortunately, recording screen activity doesn’t necessarily cost much. Three Windows-based software programs range between $30 and $150 and offer excellent performance.

Fast, Karl. Boxes and Arrows (2002). Articles>Usability>Methods>Testing

Recruiting User Testing Participants

To meet your users’ needs, it is essential to know your audience and to design for them. A key way to do this is by identifying your Web site’s primary users and recruiting a sample for usability testing. Consider these four aspects.

Martin, Beth A. Usability.gov (2006). Articles>Usability>Testing>Methods

Remote Usability: Insight into New Tools

A review of software packages for moderated usability testing in a native environment via a telephone/web interface.

Madaio, Mike. Usability Professionals Association (2006). Articles>Usability>Testing>Online

The Return on Usability Testing

Usability testing is the qualitative measurement of user experience with a product or service, such as a website. Usability testing is for anyone who wants to make their website, software, device, etc. easier to use. Whether you employ thousands of people such as Microsoft or fall into the small business category, usability testing can play a significant role in giving you direct feedback from your customers.

Subramanian, Subha. GotoMedia (2005). Articles>Usability>Testing

Review: A Review of Morae 2.0 for Usability Testing

TechSmith's recent release, Morae 2.0, features a new graphing tool, integrated satisfaction survey, and embedded task definitions. The editable marker log in Observer and the improved timeline controls in the Manager improve operator efficiency. This article highlights these and other new features of the new 2.0.

Shrestha, Sav. Usability News (2007). Articles>Reviews>Usability>Testing

A Review of Morae for Usability Testing

TechSmith's Morae is a powerful tool that is useful for any usability study. Its simplicity, efficiency, cost effectiveness and portability has redefined the data collection process and allows researchers to focus on the user experience rather than the data collection tools. This article discusses some of the features of this tool and proposes some enhancements to make it even more powerful to usability practitioners.

Shrestha, Sav. Usability News (2007). Articles>Usability>Testing>Software

Rigor in Usability Testing    

Usability testing is an evaluation method used by technical communicators that can combine aspects of both quantitative and qualitative research methodologies. This article compares and contrasts the standards and techniques between these two methods of inquiry with particular emphasis on maintaining rigorous tests in regard to validity and reliability of the findings. Whether an evaluator relies on quantitative methods, qualitative methods, or both, should depend on the questions the research or evaluation seeks to answer. Both methods have well-established practices meant to ensure the validity and reliability of their findings. Not only should usability evaluators be competent within the method of inquiry they apply, they also need to help clients understand the legitimate application and limitations of their findings.

Hughes, Michael A. Technical Communication Online (1999). Articles>Usability>Testing>Methods