Accentuate the Negative: Obtaining Effective Reviews Through Focused Questions

How you ask a question strongly determines the type of answer that you will obtain. For effective documentation reviews, whether they are conducted internally or externally as part of usability testing, it's important to use precise questions that will provide concrete information on which to base revisions. This paper proposes an approach to obtaining useful feedback that emphasizes negative, 'what did we do wrong?' questions. This approach focuses limited resources on areas that need improvement rather than areas that already work well and that don't require immediate improvement.
Hart, Geoffrey J.S. Technical Communication Online (1997). Articles>Usability>Methods>Testing
Analyzing Card Sort Results with a Spreadsheet Template
This article explains how to quickly derive easily-read, quantitative results from a card-sort activity by entering data into a spreadsheet template that is adaptable to any set of cards and categories.
Lamantia, Joe. Boxes and Arrows (2003). Articles>Usability>Methods>Card Sorting
This paper focuses on the interaction between test participants and test facilitator in two variants of the think-aloud method. In a first, explorative study, we analyzed think-aloud transcripts from two usability tests: a concurrent think-aloud test and a constructive interaction test. The results of our analysis show that while the participants in both studies never explicitly addressed the facilitator, the think-aloud participants showed more signs of awareness of the facilitator than the participants in the constructive interaction test. This finding may have practical implications for the validity of the two methods.
van den Haak, Maaike J. and Menno D.T. de Jong. IEEE PCS (2005). Articles>Usability>Testing>Methods
Anthropologists Go Native in the Corporate Village
Anthropologist Elizabeth Briody earned her PhD studying communities of Mexican-American farm workers and Catholic nuns. For the past 11 years, though, she's been studying a different community -- the men and women of General Motors. As GM's 'industrial anthropologist,' Briody explores the intricacies of life at the company. It's not all that different from her previous work. 'Anthropologists help elicit the cultural patterns of an organization,' she says. 'What rules do people have about appropriate and inappropriate behavior? How do they learn those rules and pass them on to others?' Briody is a pioneer in a growing and influential field -- corporate anthropology. What began as an experiment in a handful of companies such as GM has become an explosion. In recent years, some of the biggest names in business have recruited highly trained anthropologists to understand their workers and customers better, and to help design products that better reflect emerging cultural trends. These companies are convinced that the tools of ethnographic research -- minute observation, subtle interviewing, systematic documentation -- can answer questions about organizations and markets that traditional research tools can't.
Kane, Kate A. Fast Company (1996). Articles>Usability>Methods>Contextual Inquiry
Most usability practitioners don't derive full value from their user tests because they don't systematically archive the reports. An intranet-based usability archive offers four substantial benefits.
Nielsen, Jakob. Alertbox (2005). Articles>Usability>Methods
The Art of Usability Benchmarking
One common concern raised by managers and engineers alike is this: how easy to use is enough? This question, and the absence of an easy answer, is often the first defense people offer against investing in usability and ease of use. The smart usability engineer or designer has at least one response: the usability benchmark. By capturing the current level of ease of use of the current product or website, a reference point is created that can be measured against in the future. It doesn't answer the question of how usable is enough, but if the benchmark is done properly, it does enable someone to set goals and expectations around ease of use for the future.
Berkun, Scott. ScottBerkun.com (2006). Articles>Usability>Methods
Avoiding Bias from the Survivor Effect
Only a few of the survey sites we analyzed in 2000 are still around. We can safely assume that the surviving sites are not a random sample of the original group, but rather that significant differences exist between the sites that made it and those that died. Survival might be due partly to luck, but it is mainly a result of good management and an understanding of Internet fundamentals. Thus, the surviving sites are likely to be disproportionately clued-in about what it takes to run an online business.
Nielsen, Jakob. Alertbox (2002). Articles>Usability>Methods>Web Design
Just what do we mean by usability? Before we can set out to achieve it, we need to understand what it is we are trying to achieve. It's not enough to declare that from here on, our software will be more user friendly or that we will now be customer focused.
Quesenbery, Whitney. Cutter IT Journal (2004). Articles>Usability>Methods
Being User-Centered When Implementing a UCD Process
For those who are interested in usability – whether long-time advocates or newly introduced – this is a good time to introduce a user-centered design process.
Quesenbery, Whitney. WQusability (2001). Articles>User Centered Design>Methods>Usability
The Best of Eyetrack III: What We Saw When We Looked Through Their Eyes
In Eyetrack III, we observed 46 people for one hour as their eyes followed mock news websites and real multimedia content. In this article we'll provide an overview of what we observed.
Outing, Steve and Laura Ruel. Eyetrack III. Articles>Usability>Methods>Eye Tracking
Focus groups are popular amongst marketing professionals for good reason. They are relatively quick to organise and the feedback is instantaneous. A wide range of views can be assembled from people from a wide range of backgrounds. When focus groups go well, the data can be extremely useful in identifying profitable design routes. Plus any technique that gets companies closer to their customers can't be all bad.
System Concepts (2005). Articles>Usability>Methods>Focus Groups
Usability testing is a powerful tool in identifying problems and issues that users may have with a website or software application. But for all its benefits, traditional testing does not necessarily give a complete picture at how effective a site or application is in terms of meeting business goals.
Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods
Brainstorming is an individual or group process for generating alternative ideas or solutions for a specific topic. Good brainstorming focuses on the quantity and creativity of ideas: the quality of ideas is much less important than the sheer quantity. After ideas are generated, they are often grouped into categories and prioritized for subsequent research or application.
Usability Body of Knowledge (2007). Articles>Usability>Methods>Collaboration
Method acting can take your personas from the page to the stage. Think beyond traditional practice to give emotional life to your personas.
Fugaz, Zef. Boxes and Arrows (2006). Articles>Usability>Methods>Personas
Bringing Your Personas to Life in Real Life
The way you communicate the personas and present your deliverables is key to ensuring consistency of vision. Without that consistency, you'll spend far too much time arguing with your colleagues about who your users are rather than how to meet their needs.
Freydenson, Elan. Boxes and Arrows (2002). Articles>Usability>Methods>Personas
Building Effective Customer Surveys
Well-designed customer surveys can yield valuable information for your business. Unfortunately, though, a poorly worded survey can set you marching off in exactly the wrong direction. Below are some tips on designing surveys to get reliable, useful data.
Building Usability in from the Beginning: Analyzing Users and Their Tasks 
In this interactive session, attendees will practice their skills in interviewing users, creating task scenarios from the users’ perspective, and turning the task scenarios into designs for information products.
Hackos, JoAnn T. and Janice C. 'Ginny' Redish. STC Proceedings (1996). Articles>Usability>Methods
Card Sorting Tools: Final Summary
A summary of how IBM's USort/EzCalc and CardZort worked for results entry and analysis.
Maurer, Donna. DonnaM (2004). Articles>Usability>Methods>Card Sorting
Card Sorting: A Definitive Guide
Card sorting is a simple user-centered technique for obtaining insight into the structure of a site. But is it really so simple? This definitive guide to card sorting includes detailed instructions on how to execute and analyze a sort, plus helpful hints to improve your sorts. It is the first in a series of articles about card sorting.
Maurer, Donna and Todd Warfel. Boxes and Arrows (2004). Articles>Usability>Methods>Card Sorting
Card Sorting: An Inexpensive and Practical Usability Technique

Card sorting is often inexpensive, quick, and easy. Learn when to use this method and how to perform a card sort of your own within your company.
Kaufman, Joshua. Intercom (2006). Articles>Usability>Methods>Card Sorting
Card Sorting: How Many Users to Test
Testing ever-more users in card sorting has diminishing returns, but you should still use three times more participants than you would in traditional usability tests.
Nielsen, Jakob. Alertbox (2004). Articles>Usability>Methods>Card Sorting
Card Sorting: Mistakes Made and Lessons Learned
Card sorting is a simple and effective method with which most of us are familiar. There are already some excellent resources on how to run a card sort and why you should do card sorting. This article, on the other hand, is a frank discussion of the lessons I've learned from running numerous card sorts over the years. By sharing these lessons learned along the way, I hope to enable others to dodge similar potholes when they venture down the card sorting path.
Ng, Sam. UXmatters (2007). Articles>Usability>Methods>Card Sorting
Card-Based Classification Evaluation 
We hear and talk a lot about card sorting in various forms, and how it can be used as input on a hierarchy or classification system (or a taxonomy, if you like more technical words). We hear that we should test our hierarchies, but we don’t talk about how.
Maurer, Donna. Boxes and Arrows (2003). Articles>Usability>Methods>Card Sorting
Contrasts the unique aspects of expert reviews and usability testing. The usability goals they address are different. Know when to use which one, and when to use both.
Straub, Kathleen. Usability Professionals Association (2006). Articles>Usability>Methods>Testing
Once in a while a client will tilt their head and look at me with one of those smiles. “You want to do expert review and then also usability testing?” they say. “Is this one of those consulting tricks? Why would I need to do both?” It’s a fair question. To the casual observer, usability testing and expert review probably look very similar.
Straub, Kathleen. Usability Professionals Association (2004). Articles>Usability>Methods
There are 16 readers currently online: 1 registered user and 15 guests. Register.

![]()
![]()


![]()
![]()
![]()