Accentuate the Negative: Obtaining Effective Reviews Through Focused Questions    

How you ask a question strongly determines the type of answer that you will obtain. For effective documentation reviews, whether they are conducted internally or externally as part of usability testing, it's important to use precise questions that will provide concrete information on which to base revisions. This paper proposes an approach to obtaining useful feedback that emphasizes negative, 'what did we do wrong?' questions. This approach focuses limited resources on areas that need improvement rather than areas that already work well and that don't require immediate improvement.

Hart, Geoffrey J.S. Technical Communication Online (1997). Articles>Usability>Methods>Testing

Accessibility Testing: Case History of Blind Testers of Enterprise Software    

How do software companies evaluate whether accessibility criteria mandated by law are met? Confirmation is often provided by filling out a checklist. However, the method used for determining compliance to the checklist is not specified. Typically the task of filling out the checklist is done by accessibility specialists, usability professionals, quality assurance testers, or, in one case we know of, the development team that wrote the software. We have conducted several types of accessibility evaluations, walkthroughs, and testing with scenarios by sighted test participants and testing by blind test participants. While testing with blind participants takes considerable preparation time, we have uncovered important findings that were not revealed with sighted participants. We consider accessibility testing by blind participants an important component of our evaluations.

Bayer, Nancy L. and Lisa Pappas. Technical Communication Online (2006). Articles>Accessibility>Testing>Visual

Agile Documentation (Using Tests as Documentation)  

Storytelling can make documentation more exciting for both writers and readers. Stories provide context and people tend to remember them. More all-∆around fun when stories are tests.

Gheorghiu, Grig. Business Information Review. Articles>Documentation>Agile>Testing

Analyzing and Reporting Usability Data

The Just-In-Time (JIT) method of data analysis has the virtue of immediacy, rapid turn-around, and team involvement; however there are several disadvantages. First, this type of analysis is problem-focused, rather than goal-focused. Long lists of problems are generated, but there is no clear relation to specific usability goals. Second, developers may not be able to fix things immediately so the context of the problem may be lost when it is time to fix the problem. Third, the JIT analysis requires that the entire development team observe the testing sessions since problems may occur that are the responsibility of different developers.

Wilson, Chauncey E. Usability Interface (1997). Articles>Usability>Testing>Reports

Analyzing the Interaction Between Facilitator and Participants in Two Variants of the Think-Aloud Method    

This paper focuses on the interaction between test participants and test facilitator in two variants of the think-aloud method. In a first, explorative study, we analyzed think-aloud transcripts from two usability tests: a concurrent think-aloud test and a constructive interaction test. The results of our analysis show that while the participants in both studies never explicitly addressed the facilitator, the think-aloud participants showed more signs of awareness of the facilitator than the participants in the constructive interaction test. This finding may have practical implications for the validity of the two methods.

van den Haak, Maaike J. and Menno D.T. de Jong. IEEE PCS (2005). Articles>Usability>Testing>Methods

Authentic Behavior in User Testing

Despite being an artificial situation, user testing generates realistic findings because people engage strongly with the tasks and suspend their disbelief.

Nielsen, Jakob. Alertbox (2005). Articles>Usability>Testing

Better Reports: How to Communicate the Results of Usability Testing  

You've spent several days setting up a usability test, recruiting the participants and running it. Then you've pored over the data. What next? If you are doing usability testing as part of user-centred design within a business setting, then there are many ways that you can communicate the results. This paper looks at reports and then considers presentation and observation as alternatives to reports.

Jarrett, Caroline. STC Proceedings (2004). Articles>Usability>Testing>Reports

Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing    

It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.

Faulkner, Laura. Behavior Research Methods, Instruments and Computers (2003). Articles>Usability>Testing

Beyond the Universal User: How to Design for the Universe of Users  

Current 'user-centered' design methods place great value on design for the user. In this, I examine how investigation methods for user-centered design like usability testing and field methods are often only used to design for the universal user and not the universe of users. I critically explore the universalizing of the user that occurs during these investigation methods. I then address the problems with designing for a universal user and finally present ways to design for the universe of users.

Bowie, Jennifer L. STC Proceedings (2003). Articles>Usability>Testing

Beyond Usability Testing

Usability testing is a powerful tool in identifying problems and issues that users may have with a website or software application. But for all its benefits, traditional testing does not necessarily give a complete picture at how effective a site or application is in terms of meeting business goals.

Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods

Capturing the Usability Chimera: Practical Applications of Usability Testing  

How do I effectively include usability as a part of developing online and printed documentation?

Holmes, Fredrick and Melissa Wallentine. STC Proceedings (1994). Articles>Usability>Testing>Documentation

Cleaning Up for the Housekeeper, or, Why it Makes Sense to do Both Expert Review and Usability Testing

Contrasts the unique aspects of expert reviews and usability testing. The usability goals they address are different. Know when to use which one, and when to use both.

Straub, Kathleen. Usability Professionals Association (2006). Articles>Usability>Methods>Testing

Clustering for Usability Participant Selection  

User satisfaction and usefulness are measured using usability studies that involve real customers. Given the nature of software development and delivery, having to conduct usability studies can become a costly expense in the overall budget. A major part of this expense is the participant costs. Under this condition, it is desirable to reduce the number of participants without sacrificing the quality of the experiment. If a company could use a smaller participant pool and get the same results as the entire pool; this would result in significant savings. Given a participant pool of size N, is there a subset of N that would yield the same results as the entire population? This research addresses this question using a data-mining clustering tool called Applications Quest.

Gilbert, Juan E., Andrea Williams, and Cheryl D. Seals. Journal of Usability Studies (2007). Articles>Usability>Testing>Methods

A Comparative Analysis of Heuristic and Usability Evaluation Methods  

Usability testing and heuristic evaluation are two methods for detecting usability problems, or bugs in software user interfaces. Usability testing identifies bugs that impair user per$ormance. It provides a realistic context for the product evaluation, whereas heuristic evaluation does not. Further, it provides an estimate of bug severity, while heuristic evaluation does not. To its detriment, usability testing is more expensive and time intensive than heuristic evaluation. It also tends to overlook bugs that may not affect user pelformance but may negatively impact the user’s perception of product quality. Recently, we have incorporated the bestfeatures of these two methods into one procedure which seems to work better than either method alone.

Simeral, Elizabeth J. and Russell J. Branaghan. STC Proceedings (1997). Articles>Usability>Testing

Comparing Assessment Techniques  

In just the last few years, we have begun to see research studies comparing usability testing to other techniques for assessing usability. In general, usability testing has found more of the most serious usability problems and fewer of the least serious problems than other methods. Heuristic evaluation--having people evaluate the interface either from their own expertise or from a set of guidelines--has achieved mixed results. Although usability testing seems expensive compared to other methods, it may be less expensive when considered on the basis of 'cost per problem that needs to be fixed.'

Redish, Janice C. 'Ginny'. STC Proceedings (1993). Articles>Usability>Testing>Assessment

A Comparison of Eye Tracking Tools in Usability Testing  

Eye tracking tools have recently attracted attention from usability professionals. Eye tracking offers usability researchers a new way to identify very fine-grained behaviors that indicate usability problems. This paper is a comparison of different types of eye tracking tools and their potential usefulness in usability testing. Specifically, the paper examines the cost of the systems, system types, sampling rate, and some system limitations. The paper aims to provide a basic introduction to technical communicators who are considering adding an eye- tracking system to their toolkit.

DeSantis, Rich, Quan Zhou and Judith A. Ramey. STC Proceedings (2005). Articles>Usability>Testing>Eye Tracking

Complexities Of Usability Testing  

Usability testing has proven itself in improving product usability, but actually planning, doing testing, and interpreting results are not always straightforward. Interpretation of the results of usability testing, changes to improves usability, and general inferences to be drawn from specific tests are extremely difficult to make with accuracy. After working through the practicalities and politics of usability testing itself you must then draw conclusions and support them People who have done a lot of testing will find these problems familiar.

Ridgway, Lenore S. STC Proceedings (1994). Presentations>Usability>Methods>Testing

Conducting Usability Studies at User Group Meetings  

In this day and age, getting closer to your user base is imperative for creating user-centric documentation. This paper discusses how the Technical Publications group at Mentor Graphics tapped into their annual User Group meeting (MUG) to conduct usability studies. We cover: Convincing management of the ROI of participating in the User Group meeting; establishing relationships with meeting organizers; defining proper "protocol" for interacting with users and other meeting attendees; planning for and dealing with equipment setup; recruiting users to the usability lab; considering and acquiring incentives for usability lab participants.

Yaspo, Catherine and Sarah E. Leritz-Higgins. STC Proceedings (2005). Articles>Usability>Testing>User Centered Design

Conducting Usability Studies with Users Who Are Elderly or Have Disabilities    

Many disabled or elderly users have embraced the Web as a way to get easier access to information, services, contacts with others, and entertainment. But very often, Web sites are creating barriers for them, at the same time as they are reducing barriers of access. We can safely assume that Web sites are made by designers who have no intention to exclude groups of users from using the site. Our studies, however, have proved that good intentions are not enough to create Web content that is accessible and also usable for people with various kinds of physical, sensory, or cognitive disabilities. Web designers need to familiarize themselves with accessibility guidelines and apply them properly, but that goal is seldom achieved. Even when the guidelines are applied, a site's ease of use for people with disabilities needs to be confirmed in a usability test with actual users who have disabilities. This article describes how to conduct user-focused tests with test participants who are elderly or who have disabilities.

van der Geest, Thea. Technical Communication Online (2006). Articles>Usability>Testing>Elderly

Creative Low-Budget Usability Testing Methods  

Usability testing doesn't come cheap. You can however, follow test models that will help you improve the quality of your products, including websites. Usability professionals agree that some testing is better than none, and traditional formal usability testing can be adapted to fit your needs and your budget. This paper discusses how all four of these methods: low-cost usability testing, heuristic evaluations, expert reviews, and checkpoints in the development process were used to analyze subsites and applications at a federally funded public health website.

Reilly, Maggie T. and Cathy N. Hogan. STC Proceedings (2005). Articles>Usability>Testing>Methods

Cross-Browser Web Application Testing Made Easy

'Test on multiple browsers' has been a mantra ever since there have been multiple browsers to test on. Testing them all--especially these days--is impossible. But you can come a lot closer than you may think. In this article, learn a variety of techniques for cross-browser testing, from the very thorough to the quick and dirty. The choice you make will depend on your resources, but this is an issue you can't ignore.

Fruhlinger, Joshua A. IBM (2007). Articles>Web Design>Programming>Testing

Designing and Testing Customer Satisfaction Surveys on WWW Sites  

The types of surveys being conducted on the web fall into two categories: surveys that determine who is using the WWW and surveys that determine customer satisfaction with the product or service. To the survey guidelines described by GVU, we add five guidelines for designing web customer satisfaction surveys based on reviews of recent surveys and browsing the web: (1) Begin with a clear mission statement. (2) Classlfy current users of the site. (3) Report the results online. (4) Limit the length of the questionnaire to no more than 25 multiple-part questions. (5) Limit big graphics.

Feinberg, Susan G. and Peter Y. Johnson. STC Proceedings (1997). Articles>Usability>Testing>Web Design

Discount Usability: Time To Push Back the Clock?

Discount usability techniques are a great way to eradicate usability problems. But they can never answer the question, 'How usable is this system?' We blow the dust off some techniques commonly used in the early days of usability testing to see if they can provide an answer.

Userfocus (2003). Articles>Usability>Methods>Testing