Clustering for Usability Participant Selection  

User satisfaction and usefulness are measured using usability studies that involve real customers. Given the nature of software development and delivery, having to conduct usability studies can become a costly expense in the overall budget. A major part of this expense is the participant costs. Under this condition, it is desirable to reduce the number of participants without sacrificing the quality of the experiment. If a company could use a smaller participant pool and get the same results as the entire pool; this would result in significant savings. Given a participant pool of size N, is there a subset of N that would yield the same results as the entire population? This research addresses this question using a data-mining clustering tool called Applications Quest.

Gilbert, Juan E., Andrea Williams, and Cheryl D. Seals. Journal of Usability Studies (2007). Articles>Usability>Testing>Methods

Cognitive Walkthrough

Cognitive walkthrough is a review technique where expert evaluators construct task scenarios from a specification or early prototype and then role play the part of a user working with that interface--'walking through' the interface. They act as if the interface was actually built and they (in the role of a typical user) was working through the tasks. Each step the user would take is scrutinized: impasses where the interface blocks the 'user' from completing the task indicate that the interface is missing something. Convoluted, circuitous paths through function sequences indicate that the interface needs a new function that simplifies the task and collapses the function sequence.

Hom, James. VWH.net. Articles>Usability>Methods

Conducting Mail, Telephone, and Online Surveys: 1998  

While surveys appear surprisingly easy to plan and conduct, they are fraught with pitfalls for the unwary. That said, technical communicators can use surveys as a tool to enhance their understanding of audiences, assess the effectiveness of their communication products, and determine the value of technical communications to their company--if they follow wellestablished social science and communication science methodologies. This workshop will provide you with the foundations you need for developing, conducting, and managing surveys; analyzing the data, interpreting surveys and reporting your results.

Zimmerman, Donald E. STC Proceedings (1998). Articles>Usability>Methods>Surveys

Conducting Mail, Telephone, and Online Surveys: 1999  

While surveys appear surprisingly easy to plan and conduct, they are fraught with pitfalls for the unwary. That said, technical communicators can use surveys as a tool to enhance their understanding of audiences, assess the effectiveness of their communication products, and determine the value of technical communication to their company--if they follow well-established social science and communication science methodologies. This workshop will provide you with the foundations you need for developing, conducting, and managing surveys; analyzing the data, interpreting surveys and reporting your results.

Zimmerman, Donald E. STC Proceedings (1999). Articles>Usability>Methods>Surveys

Contextual Inquiry

Contextual inquiry is basically a structured field interviewing method, based on a few core principles that differentiate this method from plain, journalistic interviewing. Contextual inquiry is more a discovery process than an evaluative process; more like learning than testing. Contextual inquiry is based on three core principles: that understanding the context in which a product is used (the work being performed) is essential for elegant design, that the user is a partner in the design process, and that the usability design process, including assessment methods like contextual inquiry and usability testing, must have a focus. For example, suppose you need to assess the usability of a wrench for automotive repair. Using contextual inquiry, you'd visit mechanics at auto repair shops and see how they work. You'd take in not only physical arrangements such as the location of the tool chests, or cramped conditions inside engine compartments, but also environmental concerns, such as the level of cleanliness of their hands, or the noise level in the shop, or the tight schedules imposed by their bosses. All of these would help define a context for their work--and thus a context for the usage of your product, the wrench. You'd also listen to their gripes about your product; how it slips out of their hands if they've been working on greasy stuff, how it gnaws the corners off stubborn bolts. You'd ask them what would make their jobs easier; what design changes would help them. They're a partner in the design process. Of course, you'd conduct all this research centering on the one thing you're analyzing: the wrench. This focus is important--it sets the goals for the visit ('We need to know how they store their wrenches'). Once you're done with your site visit, you can assess from your notes whether you found out what you needed to know.

Hom, James. VWH.net. Articles>Usability>Methods>Contextual Inquiry

Crafting a User Research Plan

Every piece of user research is part of an ongoing research program, even if that program is informal. However, making a program formal provides a number of advantages: It gives you a set of goals, a schedule that stretches limited user-research resources, and results when they’re needed most. It also helps you avoid unnecessary, redundant, or hurried research.

Kuniavsky, Mike. uiGarden (2005). Articles>Usability>Methods

Creative Low-Budget Usability Testing Methods  

Usability testing doesn't come cheap. You can however, follow test models that will help you improve the quality of your products, including websites. Usability professionals agree that some testing is better than none, and traditional formal usability testing can be adapted to fit your needs and your budget. This paper discusses how all four of these methods: low-cost usability testing, heuristic evaluations, expert reviews, and checkpoints in the development process were used to analyze subsites and applications at a federally funded public health website.

Reilly, Maggie T. and Cathy N. Hogan. STC Proceedings (2005). Articles>Usability>Testing>Methods

Criteria for Focused Data Collection  

The objective of this task is to propose and validate a mechanism whereby projects can identify their needs for software measurement data and focus their data collection activities using a minimum standardized set of software measures. The purpose of this strategy is to evolve a process that will enable NASA projects to tailor with their data collection activities to their unique needs for effective management control indicators, but also encourages consistent data collection that will facilitate statistical analysis across NASA domains.

Wilson, William M. NASA (2001). Articles>Usability>Methods>Assessment

Demand Characteristics

Demand Characteristics is a term used in Cognitive Psychology to denote the situation where the results of an experiment are biased because the experimenters' expectancies regarding the performance of the participants on a particular task create an implicit demand for the participants to perform as expected.

Soegaard, Mads. Interaction-Design.org. Articles>Usability>Methods

Designing an Effective User Study  

When it comes to learning about your users, a plethora of methods await you. But which one is best for your situation? The answer depends on many factors, including the kind of information you hope to discover, the time and budget you have available, and your access to users.

Hammar, Molly and Dawn Stevens. STC Proceedings (1998). Articles>Usability>Methods>User Centered Design

Discount Usability: Time To Push Back the Clock?

Discount usability techniques are a great way to eradicate usability problems. But they can never answer the question, 'How usable is this system?' We blow the dust off some techniques commonly used in the early days of usability testing to see if they can provide an answer.

Userfocus (2003). Articles>Usability>Methods>Testing

Do Usability Expert Evaluation and Testing Provide Novel and Useful Data for Game Development?    

A case study was done to study whether usability expert evaluation and testing are suitable for game development. In the study, a computer game under development was first evaluated and then tested. Game developers were then asked to rate the findings and give other feedback about the methods used and the results gained. It was found that the usability expert evaluation and testing provided both novel and useful data for game development. Based on these and the other results it is argued that the usability expert evaluation and testing have considerable face validity in game development. In addition to the usefulness and face validity of the methods it was studied whether the usability experts participating in the game usability expert evaluation should be double experts. It was found that there was no significant difference in the number or the rated relevancy of the problem the gamer and non-gamer usability specialists found.

Laitinen, Sauli. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods

Don't Test Users, Test Hypotheses

User testing typically consists of a sort of fishing trip. We lower a lure (the user) into the water (the application or site) and see what critters (defects) bite. This is a valuable and time-tested approach. But when we start fishing for defects, we are left with some tough questions. For instance: When are we finished? How many defects do we need to find before we have fully tested the site or application? If we find a defect, how do we know how severe it is, and by what measure? In iterative testing, how do we compare results from the test of the current version with results from testing earlier versions?

Soudack, Avi. Boxes and Arrows (2003). Articles>Usability>Methods>Testing

Driving Innovation and Creativity through Customer Data

This article explores the foundations of designing for innovation. Karen Holtzblatt has created contextual inquiry, a practical, customer-centered approach that helps designers develop creative solutions that dominate the competition.

Perfetti, Christine. User Interface Engineering (2002). Articles>Usability>Methods>Contextual Inquiry

Ethnographic Study / Field Observation

Observing users in the field is often the best way to determine their usability requirements. Traditional usability testing, while providing a laboratory environment that makes data collection and recording easy, also removes the user and the product from the context of the workplace. Sometimes, it's best to see exactly how things are done in the real world.

Hom, James. VWH.net. Articles>Usability>Methods>Contextual Inquiry

Evaluation of an Informational Web Site: Three Variants of the Think-aloud Method Compared    

To evaluate Web sites, usability experts often use methods that were originally employed for the evaluation of software applications. In doing so, they assume that these methods will work exactly the same for both types of test objects. However, there is a major difference between transactional software applications and informational Web sites, a difference that could have an effect on the workings of various usability methods. As such, we felt that it was valuable to repeat one of our previous studies in which we compared concurrent think-aloud protocols, retrospective think-aloud protocols, and constructive interaction to evaluate a Web application, this time using a Web site. The results of our study showed that in some respects, the methods did work differently depending on the test object they were applied to. However, we conclude that the three methods are largely interchangeable and that the decision to choose one variant of the think-aloud method over the other should be based on practical considerations.

van den Haak, Maaike J., Menno de Jong, D.T. and Peter JanSchellens. Technical Communication Online (2007). Articles>Web Design>Usability>Methods

Examining how Users Interact with Hypermedia Using a Neural Network  

Users of hypermedia systems, including the Web, are known to produce distinctive patterns depending upon what it is that they are trying to achieve with the medium. For example, someone who is seeking specific information produces a different browsing pattern than someone who is browsing generally. However, it is also known that people using hypermedia for similar purposes produce similar, but not identical, patterns. Such information would be useful for a browsing aid, since it would enable the better selection of links, for offering to the user, based upon what the user’s task is. This paper describes the architecture and training of a neural network system designed to recognise hypermedia browsing patterns in a prototype hypermedia environment. A further fuzzy-logic based system, which is used to record trends in browsing patterns, is then discussed in outline. Both systems have performed well in small-scale studies, with both real users and simulated-data. Further, both systems have proved robust to the potentially complex behaviour of users. These issues are discussed further.

Mullier, D.J. Proceedings of ICAI (2000). Articles>Usability>Methods>Cognitive Psychology

Eye Tracking

Eye tracking is a technique used in cognitive science, psychology (notably psycholinguistics), human-computer interaction (HCI), marketing research, medical research, and other areas. The most widely used current designs are video based eye trackers. A camera focuses on one or both eyes and records their movement as the viewer looks at some kind of stimulus. Most modern eye-trackers use contrast to locate the center of the pupil and use infrared and near-infrared non-collumnated light to create a corneal reflection (CR). The vector between these two features can be used to compute gaze intersection with a surface after a simple calibration for an individual.

Wikipedia. Articles>Usability>Methods>Eye Tracking

Eye-Tracking Studies: Usability Holy Grail?

The reality is that eye-tracking, while valuable, doesn't make usability testing any more powerful. It's what you do with the observations and the usability test data that counts.

Spillers, Frank. Demystifying Usability (2004). Articles>Usability>Methods>Eye Tracking

Fast and Simple Usability Testing

Everyone knows by now that they should test the usability of their applications, but still hardly anybody actually does it. In this article I'll share some tips I've picked up for doing usability tests quickly and effectively. Relatively recent tools like Django and Ruby on Rails allow us to develop projects faster and to make significant changes later in the project timeline. Usability testing methods should now be adapted to fit this modern approach to development.

Downe, Natalie. 24 Ways to impress Your Friends (2005). Articles>Usability>Testing>Methods

Fast, Cheap, and Good: Yes, You Can Have It All

The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.

Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods

Fast, Cheap, and Good: Yes, You Can Have It All

The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.

Nielsen, Jakob. Alertbox (2007). Articles>Usability>Methods

Field Studies Done Right: Fast and Observational

Field studies should emphasize the observation of real user behavior. Simple field studies are fast and easy to conduct, and do not require a posse of anthropologists: All members of a design team should go on customer visits.

Nielsen, Jakob. Alertbox (2002). Articles>Usability>Methods