#18808
Usability Testing: Getting It Right The First Time 
User-centered product design is a design approach that focuses on the users’ job tasks, skills, and abilities. Usability testing has emerged as a critical component in the user-centered design process to assure that a product meets the needs of the user. Implemented correctly, usability testing can increase customer satisfaction and acceptance, improve product image, and reduce development costs. A variety of information is available to help you get started in developing a usability testing process. This workshop will provide sample questionnaires, checklists, scenarios, scripts, etc. However, the main focus of the workshop will be to train participants in the following two areas: (1) writing measurable usability test goals; and (2) collecting and interpreting the test data. These activities are critical because they relate directly to the usefulness of the test results.
Cope, Michael and Kevin Uliano. STC Proceedings (1994). Articles>Usability>Methods>Testing
#31625
Usability Testing: Now and the Future
All councils and public sector organisations have a clear remit to make their information available and accessible to their citizens, and increasingly they are applying usability testing to gather empirical evidence of the effectiveness of their sites. Whether this is done by internal teams or external, independent organisations and consultants, usability testing is now part of a process applied by many site managers. Indeed usability is considered by SOCITM in their annual UK council site reviews, and is a critical part to citizen fining services and transacting online.
Rourke, Chris. User Vision (2007). Articles>Usability>Testing
#30790
Usability, Testing and Documentation
The three disciplines of usability, testing, and documentation are important to the success of hi-tech products. These three areas are often overlooked, but they have a profound affect on the end user's experience of a software product. As customers become more perceptive about IT, their expectations rise. Since customers now have more choice than ever, to be successful, a software product needs to stand out from the crowd. This article explains why the three disciplines are important to the overall success of a software solution. It concludes with recommendations for obtaining the best return on investment.
Unwalla, Mike, Andrew Swartz and Bill Matthews. TechScribe (2006). Articles>Usability>Testing>Documentation
#26258
The User Advocate: Interactive Prototyping, Part 1: Easy PDF Prototyping
I've often observed that once wireframing begins, it's off to the races! In the rush to launch, we sometimes forget end-users. Is there a way to ensure that they get a voice during this always-hectic phase?
Rogers, David J. GotoMedia (2005). Articles>User Centered Design>Usability>Testing
#24521
User Observation Testing is Mandatory
Without user observation testing, the usability of your web site is virtually unknown. Surveys are worth little, since those surveyed tend to tell you what they think you want to hear. Staff opinions are nice, but biased and they are not typical users. Heuristic (general guidelines-based) evaluation is helpful, but remains theoretical until tested on actual, representative users as they attempt to find information or perform tasks at your site.
Streight, Steven. Blogger.com (2004). Articles>Usability>Human Computer Interaction>Testing
#24519
User Observation Tests: Forms and Procedures
Detailed explanation of how to conduct a web usability user observation test. Simple, step-by-step instructions for professional administration of testing program. How to select and supervise test subjects. How to design test task assignments. Suggested forms to use: test subject selection computer skills level telesurvey, link strategy survey, system usability scale questionnaire, site satisfaction survey.
Streight, Steven. Usability Interface (2004). Articles>Usability>Testing>Web Design
#28092
User Testing is Not Entertainment
Don't run your studies for the benefit of the people in the observation room. Test to discover the truth about the design, even when user tasks are boring to watch.
Nielsen, Jakob. Alertbox (2006). Articles>Usability>Testing
#23782
Using Benchmark Testing to Measure Ease of Use Improvements: A Case Study
In order to measure how future releases improve on the current store development scenario in our product, IBM® WebSphere® Commerce, a multidisciplinary team of user experience analysts, information developers, and testers decided to create a benchmark test that we can apply to each new version of the product. This paper discusses how we prepared and planned for the test, as well as the execution and results of the first test. We also discuss lessons learned and how we can improve our benchmarking process.
Markova, Uliyana, Diba Bot-Roche, Erin Heximer and Justine Yoon. STC Proceedings (2003). Articles>Usability>Testing
#24758
Using Microsoft Excel to Collect Usability Data
For many usability engineers and human factors researchers, basic note-taking on a paper form or laptop computer represents the common data collection strategy. Unfortunately, this approach can be cumbersome and slow to compile the data following the completion of a study.
Zazelenchuk, Todd. Userfocus (2003). Articles>Usability>Testing>Microsoft Excel
#27701
Variability in User Performance
When doing website tasks, the slowest 25% of users take 2.4 times as long as the fastest 25% of users. This difference is much higher than for other types of computer use; only programming shows a greater disparity.
Nielsen, Jakob. Alertbox (2006). Articles>Web Design>Usability>Testing
#31903
Weekly User Testing: TiVo Did It, You Can, Too
TiVo ran 12 user tests in 12 weeks while designing its new website. As TiVo's experience shows, frequent and regular testing keeps the design usability focused.
Nielsen, Jakob. Alertbox (2008). Articles>Usability>Testing
#24695
What Really Goes on at a Usability Test
The term 'usability testing' means different things to different people. In its 'high fidelity' form, it connotes two-way mirrors, fancy electronic equipment, sophisticated logging software, and tomes of test data. However, high fidelity options aren’t always required for learning about or performing substantive usability testing. Even in a 'low fidelity' environment, you can hit all of the usability high points and provide a valuable service to your software developers and users.
Hyman, Francine N. and Barton A. Evans. STC Proceedings (1996). Articles>Usability>Testing
#28017
When 100% Really Isn't 100%: Improving the Accuracy of Small-Sample Estimates of Completion Rates 
Small sample sizes are a fact of life for most usability practitioners. This can lead to serious measurement problems, especially when making binary measurements such as successful task completion rates (p). The computation of confidence intervals helps by establishing the likely boundaries of measurement, but there is still a question of how to compute the best point estimate, especially for extreme outcomes. In this paper, we report the results of investigations of the accuracy of different estimation methods for two hypothetical distributions and one empirical distribution of p. If a practitioner has no expectation about the value of p, then the Laplace method ((x+1)/(n+2)) is the best estimator. If practitioners are reasonably sure that p will range between .5 and 1.0, then they should use the Wilson method if the observed value of p is less than .5, Laplace when p is greater than .9, and maximum likelihood (x/n) otherwise.
Lewis, James R. and Jeff Sauro. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods
#28915
When Observing Users Is Not Enough: 10 Guidelines for Getting More Out of Users' Verbal Comments
One of the principles underlying usability testing is that observing a user perform a task provides more reliable information than simply asking the user how easy it would be to perform the task. By observing users, you can assess whether they are actually able to use a product. By asking them, you simply cannot.
Peyrichoux, Isabelle. UXmatters (2007). Articles>Usability>Testing>Methods
#27413
Why You Only Need to Test With Five Users
Some people think that usability is very costly and complex and that user tests should be reserved for the rare web design project with a huge budget and a lavish time schedule. Not true. Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.
Nielsen, Jakob. Alertbox (2000). Articles>Usability>Testing>Methods
#28378
#32357
Unexpected Complexity in a Traditional Usability Study
This article is a case study of a demonstration project intended to prove the value of usability testing to a large textbook publishing house. In working with a new client, however, the research team discovered that what our client thought were simple problems for their users were actually complex problems that required the users to evaluate potential solutions in a surprisingly complex context of use. As Redish (2007) predicted, traditional ease of use measures were "not sufficient" indicators and failed to reveal the complex nature of the tasks. Users reported high levels of satisfaction with products being tested and believed they had successfully completed tasks which they judged as easy to complete when, in fact, they unknowingly suffered failure rates as high as 100%. The study recommends that usability specialists expand our definition of traditional usability measures so that measures include external assessment by content experts of the completeness and correctness of users' performance. The study also found that it is strategically indispensable for new clients to comprehend the upper end of complexity in their products because doing so creates a new space for product innovation. In this case, improving our clients' understanding of complexity enabled them to perceive and to take advantage of a new market niche that had been unrealized for decades.
Howard, Tharon W. Journal of Usability Studies (2008). Articles>Usability>Testing>Methods
#32362
A Low-Cost Test Environment for Usability Studies of Head-Mounted Virtual Reality Systems
There is a need to develop new usability testing environments and methodologies for unconventional interactive systems. Pursuant to that need, we developed a low-cost test environment for a Head-Mounted Display (HMD)-based, virtual reality system called Osmose. Osmose was difficult to test for many reasons, one of which was its style of interaction. We began setting up the testing environment about two weeks before the start of the usability testing. We learned many lessons throughout the experience. This paper summarizes the study findings, both methodological - how to setup and conduct a usability lab for such an environment - as well as conceptual -the human experiences and behavioral patterns involved in using an immersive environment.
Seffah, Ahmed, Jonathan Benn and Halima Habieb Mammar. Journal of Usability Studies (2008). Articles>Usability>Testing>3D
#32378
Quick Turnaround Usability Testing
Completing usability testing quickly is a challenge anywhere but especially in consultancies, which have to overcome additional challenges, such as learning a new application. To assure success on these projects, I’ve developed a quick turnaround usability testing methodology (QTUT) that minimizes the time needed to complete testing.
Nuschke, Paul. Boxes and Arrows (2008). Articles>Usability>Testing
#32546
The Sixty-Minute Guide to Evaluating Comparative Test Data
Mary wants to compare the average registration times between the two versions to see if the second version is faster than the first. The method typically used when comparing averages is called a t-test of independent means.
Hughes, Michael A. STC Proceedings (2008). Articles>Usability>Testing>Assessment
#32592
Usability testing makes use of a lot of role-playing scenarios like this one, and many findings and design recommendations result from participants’ responses to these scenarios. But an over-reliance on role playing when testing a product and making design recommendations can have major downsides and risks
Peyrichoux, Isabelle. UXmatters (2008). Articles>Usability>Testing>Personas
#32901
Guidelines for Usability Testing with Children
Although user-centered design is a well-supported concept in the literature on adult computer products, not until recently have publications begun to appear addressing the need to include the user in the design process of children’s computer products.
#33007
Get Out of Your Lab, and Into Their Lives
The proliferation of usability labs is a sign of success for the field of user-centered design. Whether it’s a low-rent lab comprised of a couple adjacent conference rooms, a video camera, and a television, or a fully decked-out space with remote-control cameras, two-way mirrors, an observation room, and bowls of M&Ms — more and more companies are investing in such set-ups. Conducting user tests in labs is probably the most common means of getting user input on projects. That’s a shame, because standard user testing practice is remarkably out of sync with reality.
Merholtz, Peter. Adaptive Path (2006). Articles>Usability>Testing>Contextual Inquiry
#33163
Why Doing User Observations First is Wrong
Field studies, user observations, contextual analyses, and all procedures which aim at determining true human needs are still just as important as ever – but they should all be done outside of the product process. This is the information needed to determine what product to build, which projects to fund. Do not insist on doing them after the project has been initiated. Then it is too late, then you are holding everyone back.
Norman, Donald A. JND.org. Articles>Usability>Testing>Contextual Inquiry
#33186
Conducting International Usability
User testing is a valuable tool, but how does one conduct user tests internationally? This essay draws from my experience leading an international user testing project, and I hope you can learn from my mistakes and successes.
Merholtz, Peter. Adaptive Path (2003). Articles>Usability>Testing>International
