#10318
Accentuate the Negative: Obtaining Effective Reviews Through Focused Questions 
How you ask a question strongly determines the type of answer that you will obtain. For effective documentation reviews, whether they are conducted internally or externally as part of usability testing, it's important to use precise questions that will provide concrete information on which to base revisions. This paper proposes an approach to obtaining useful feedback that emphasizes negative, 'what did we do wrong?' questions. This approach focuses limited resources on areas that need improvement rather than areas that already work well and that don't require immediate improvement.
Hart, Geoffrey J.S. Technical Communication Online (1997). Articles>Usability>Methods>Testing
#31652
This paper focuses on the interaction between test participants and test facilitator in two variants of the think-aloud method. In a first, explorative study, we analyzed think-aloud transcripts from two usability tests: a concurrent think-aloud test and a constructive interaction test. The results of our analysis show that while the participants in both studies never explicitly addressed the facilitator, the think-aloud participants showed more signs of awareness of the facilitator than the participants in the constructive interaction test. This finding may have practical implications for the validity of the two methods.
van den Haak, Maaike J. and Menno D.T. de Jong. IEEE PCS (2005). Articles>Usability>Testing>Methods
#19290
Usability testing is a powerful tool in identifying problems and issues that users may have with a website or software application. But for all its benefits, traditional testing does not necessarily give a complete picture at how effective a site or application is in terms of meeting business goals.
Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods
#28102
Contrasts the unique aspects of expert reviews and usability testing. The usability goals they address are different. Know when to use which one, and when to use both.
Straub, Kathleen. Usability Professionals Association (2006). Articles>Usability>Methods>Testing
#30436
Clustering for Usability Participant Selection
User satisfaction and usefulness are measured using usability studies that involve real customers. Given the nature of software development and delivery, having to conduct usability studies can become a costly expense in the overall budget. A major part of this expense is the participant costs. Under this condition, it is desirable to reduce the number of participants without sacrificing the quality of the experiment. If a company could use a smaller participant pool and get the same results as the entire pool; this would result in significant savings. Given a participant pool of size N, is there a subset of N that would yield the same results as the entire population? This research addresses this question using a data-mining clustering tool called Applications Quest.
Gilbert, Juan E., Andrea Williams, and Cheryl D. Seals. Journal of Usability Studies (2007). Articles>Usability>Testing>Methods
#29636
Creative Low-Budget Usability Testing Methods 
Usability testing doesn't come cheap. You can however, follow test models that will help you improve the quality of your products, including websites. Usability professionals agree that some testing is better than none, and traditional formal usability testing can be adapted to fit your needs and your budget. This paper discusses how all four of these methods: low-cost usability testing, heuristic evaluations, expert reviews, and checkpoints in the development process were used to analyze subsites and applications at a federally funded public health website.
Reilly, Maggie T. and Cathy N. Hogan. STC Proceedings (2005). Articles>Usability>Testing>Methods
#19606
Discount Usability: Time To Push Back the Clock?
Discount usability techniques are a great way to eradicate usability problems. But they can never answer the question, 'How usable is this system?' We blow the dust off some techniques commonly used in the early days of usability testing to see if they can provide an answer.
#28020
Do Usability Expert Evaluation and Testing Provide Novel and Useful Data for Game Development?
A case study was done to study whether usability expert evaluation and testing are suitable for game development. In the study, a computer game under development was first evaluated and then tested. Game developers were then asked to rate the findings and give other feedback about the methods used and the results gained. It was found that the usability expert evaluation and testing provided both novel and useful data for game development. Based on these and the other results it is argued that the usability expert evaluation and testing have considerable face validity in game development. In addition to the usefulness and face validity of the methods it was studied whether the usability experts participating in the game usability expert evaluation should be double experts. It was found that there was no significant difference in the number or the rated relevancy of the problem the gamer and non-gamer usability specialists found.
Laitinen, Sauli. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods
#20894
Don't Test Users, Test Hypotheses
User testing typically consists of a sort of fishing trip. We lower a lure (the user) into the water (the application or site) and see what critters (defects) bite. This is a valuable and time-tested approach. But when we start fishing for defects, we are left with some tough questions. For instance: When are we finished? How many defects do we need to find before we have fully tested the site or application? If we find a defect, how do we know how severe it is, and by what measure? In iterative testing, how do we compare results from the test of the current version with results from testing earlier versions?
Soudack, Avi. Boxes and Arrows (2003). Articles>Usability>Methods>Testing
#28909
Fast and Simple Usability Testing
Everyone knows by now that they should test the usability of their applications, but still hardly anybody actually does it. In this article I'll share some tips I've picked up for doing usability tests quickly and effectively. Relatively recent tools like Django and Ruby on Rails allow us to develop projects faster and to make significant changes later in the project timeline. Usability testing methods should now be adapted to fit this modern approach to development.
Downe, Natalie. 24 Ways to impress Your Friends (2005). Articles>Usability>Testing>Methods
#28700
Fast, Cheap, and Good: Yes, You Can Have It All
The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.
Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods
#27405
How Many Users Should You Test With in Usability Testing?
Doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?
Spillers, Frank. Demystifying Usability (2005). Articles>Usability>Testing>Methods
#20855
Instructions for Branch Office Testing
These are the instructions we gave to the people at various Sun branch offices in Europe and Asia for their user testing of a new design for the company's web pages. In a few places, these instructions refer to web-specific issues, so they will have to be modified slightly for use in other projects. These instructions were sent by electronic mail to those local Sun reps who had volunteered to lead a test.
Nielsen, Jakob. Alertbox (1996). Articles>Usability>Methods>Testing
#19040
Is A Lab Essential For User Testing?
Once an organisation decides to go ahead with a user testing programme, the questions really begin. Is it really necessary to undertake testing in a 'usability lab'? And what exactly should a fully functioning lab consist of anyway? As one might imagine, opinion is divided on these issues. We take a quick look at what a typical lab might consist of and the pros and cons of lab-based testing.
Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods
#15003
Iterative Usability Research Methods: Why Testing Isn't Enough
Discusses how to choose different usability methods for iterative research. Slides only.
Rosenbaum, Stephanie L. Tec-Ed, Inc. (1999). Articles>Usability>Testing>Methods
#28023
Iterative Usability Testing as Continuous Feedback: A Control Systems Perspective
This paper argues that in the field of usability, debates about number of users, the use of statistics, etc. in the abstract are pointless and even counter-productive. We propose that the answers depend on the research questions and business objectives of each project and thus cannot be discussed in absolute terms. Sometimes usability testing is done with an implicit or explicit hypothesis in mind. At other times the purpose of testing is to guide iterative design. These two approaches call for different study designs and treatment of data. We apply control systems theory to the topic of usability to highlight and frame the value of iterative usability testing in the design lifecycle. Within this new metaphor, iterative testing is a form of feedback which is most effective and resource-efficient if done as often as practically possible with project resources and timelines in mind.
Genov, Alex. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods
#18622
Journaled sessions bridges usability inquiry, where you ask people about their experiences with a product, and usability testing, where you observe people experiencing the product's user interface. Journaled sessions are often used as a remote inquiry method for software user interface evaluation. A disk is distributed to a number of test subjects containing a prototype of the software product, as well as additional code to capture (or journalize) the subjects' actions when using the prototype. Users perform several tasks with the prototype, much as in formal usability tests, and their actions are captured with the journalizing software. Upon completion of the series of tasks, the users return the disks to you for you to evaluate. Because the journaling portion of the evaluation is largely automated, this approach to remote, hands-off inquiry is certainly more 'usable' then self-reporting logging, where users are requested to write down their observations and comments and send them back to you.
#30198
Multiple-User Simultaneous Testing (MUST)
Testing 5-10 users at once lets you conduct large-scale usability testing and still meet your deadlines.
Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods
#27941
Outliers and Luck in User Performance
6% of task attempts are extremely slow and constitute outliers in measured user performance. These sad incidents are caused by bad luck that designers can -- and should -- eradicate.
Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods
#27897
Quantitative Studies: How Many Users to Test?
We can define usability in terms of quality metrics, such as learning time, efficiency of use, memorability, user errors, and subjective satisfaction. Sadly, few projects collect such metrics because doing so is expensive: it requires four times as many users as simple user testing. Many users are required because of the substantial individual differences in user performance. When you measure people, you'll always get some who are really fast and some who are really slow. Given this, you need to average these measures across a fairly large number of observations to smooth over the variability.
Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing>Methods
#21366
Recording Screen Activity During Usability Testing
Recording what users do is a crucial aspect of usability testing. Fortunately, recording screen activity doesn’t necessarily cost much. Three Windows-based software programs range between $30 and $150 and offer excellent performance.
Fast, Karl. Boxes and Arrows (2002). Articles>Usability>Methods>Testing
#27147
Recruiting User Testing Participants
To meet your users’ needs, it is essential to know your audience and to design for them. A key way to do this is by identifying your Web site’s primary users and recruiting a sample for usability testing. Consider these four aspects.
Martin, Beth A. Usability.gov (2006). Articles>Usability>Testing>Methods
#10383
Usability testing is an evaluation method used by technical communicators that can combine aspects of both quantitative and qualitative research methodologies. This article compares and contrasts the standards and techniques between these two methods of inquiry with particular emphasis on maintaining rigorous tests in regard to validity and reliability of the findings. Whether an evaluator relies on quantitative methods, qualitative methods, or both, should depend on the questions the research or evaluation seeks to answer. Both methods have well-established practices meant to ensure the validity and reliability of their findings. Not only should usability evaluators be competent within the method of inquiry they apply, they also need to help clients understand the legitimate application and limitations of their findings.
Hughes, Michael A. Technical Communication Online (1999). Articles>Usability>Testing>Methods
#10578
Developers may hesitate to start usability testing because they worry that their product poses special problems in finding, scheduling, or compensating the right users. This shouldn’t stop them. We successfully find and test hundreds of users a year and about 10% of these require special tactics for scheduling.
User Interface Engineering (1998). Articles>Usability>Methods>Testing
