Discovering User-Generated Metaphors Through Usability Testing

A post-hoc analysis of data collected from a usability test on the Fluke ScopeMeter 97â (a diagnostic instrument for analyzing electrical signals) revealed that subjects miss-hit certain interface keys significantly more than other keys. Comments that subjects made during the usability test through the thinking-aloud protocol led us to the hypothesis that miss-hits were caused by subjects' internal metaphors (user-generated) that were unsupported by the design of the keys. A review of literature revealed a lack of research in the area of user-generated metaphor and its effects on user performance. With further analysis we found aspects of the ScopeMeter keys and subject characteristics that provide a strong foundation for this hypothesis. We conclude that formal experiments should be set up to test this and other hypotheses that address the role user-generated metaphors play in building mental models and influencing user behavior.

Schulz, Erin Leanne, Judith A. Ramey, Maarten van Alphen and William Rasnake. City University of Hong Kong (1995). Articles>Usability>Testing>Tropes

Do Usability Expert Evaluation and Testing Provide Novel and Useful Data for Game Development?    

A case study was done to study whether usability expert evaluation and testing are suitable for game development. In the study, a computer game under development was first evaluated and then tested. Game developers were then asked to rate the findings and give other feedback about the methods used and the results gained. It was found that the usability expert evaluation and testing provided both novel and useful data for game development. Based on these and the other results it is argued that the usability expert evaluation and testing have considerable face validity in game development. In addition to the usefulness and face validity of the methods it was studied whether the usability experts participating in the game usability expert evaluation should be double experts. It was found that there was no significant difference in the number or the rated relevancy of the problem the gamer and non-gamer usability specialists found.

Laitinen, Sauli. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods

Don't Test Users, Test Hypotheses

User testing typically consists of a sort of fishing trip. We lower a lure (the user) into the water (the application or site) and see what critters (defects) bite. This is a valuable and time-tested approach. But when we start fishing for defects, we are left with some tough questions. For instance: When are we finished? How many defects do we need to find before we have fully tested the site or application? If we find a defect, how do we know how severe it is, and by what measure? In iterative testing, how do we compare results from the test of the current version with results from testing earlier versions?

Soudack, Avi. Boxes and Arrows (2003). Articles>Usability>Methods>Testing

Eight Guidelines for Usability Testing

Eight essential guidelines for usability testing, helping you to plan, run and analyse usability tests.

Fidgeon, Tim. Webcredible (2005). Design>Web Design>Usability>Testing

Examining the Relationship Between Quality Writing and Quality Reading    

This article introduces the ISTE approach as a way to measure the relationship between the design of a document and the way that users handle it. ISTE is an acronym for Information SubTypes and Effects. The article describes the classification of information into a general typology as well as into a specific one, to illustrate how the approach can be adapted to specific usability questions. ISTE also requires a typology for categorizing possible user reactions. This article first positions ISTE within the broader context of usability testing and then describes its basic approach and some recent applications. Unlike most formal research methods, ISTE does not require an inordinate amount of time and effort for analyzing high-fidelity data. Electronic versions such as the ERR instrument allow for the creation of a complete and detailed profile of the use of a document 10 minutes after task completion. These characteristics make ISTE a viable instrument for gathering usability metrics in industrial settings.

van der Meij, Hans. Technical Communication Online (2000). Articles>Usability>Testing>Assessment

Experience Equity and Universal Access: Designing Clinical Studies for Low Vision  

In this paper, I describe web page design for those interested in conducting clinical, low vision studies. Ideally, web pages should be accessible and usable for all readers; however, the web is a highly visual medium for communication and creates serious accessibility issues for specialized (diverse needs) those with vision needs. Therefore, I propose that researchers consider a paramount and concurrent user-centered design approach when creating stimulus materials for these specialized audiences. This paper introduces readers to this design approach for a low vision audience as described in the WebText Study.

Reece, Gloria A. STC Proceedings (2005). Articles>Usability>Testing>Visual

Experiences in Remote Usability Evaluations  

Traditional usability testing occurs in a laboratory setting in which users are directly observed by test facilitators. However, the costs associated with accessing users for usability testing as well as the limited availability of many users make it necessary to consider new methods for gathering usability feedback for software products. Online, remote testing is a key to achieving a larger, more diverse pool of participants.

Cheatham, Deane and Velda Bartek. IBM (2004). Articles>Usability>Testing

Eye Gaze Patterns while Searching vs. Browsing a Website

This article discusses users' visual scan paths of web pages containing text and/or pictures while conducting browsing and searching tasks. User performance on three usability tasks on an e-commerce website is described. Results show that users follow a fairly uniform scan path when browsing through pictures, and a more random path while specifically searching through them. Additionally, users appeared to follow Nielsen's 'F' pattern (2006) while both browsing and searching through text-based pages.

Shrestha, Sav and Kelsi Lenz. Usability News (2007). Articles>Usability>Testing>Eye Tracking

Fast and Simple Usability Testing

Everyone knows by now that they should test the usability of their applications, but still hardly anybody actually does it. In this article I'll share some tips I've picked up for doing usability tests quickly and effectively. Relatively recent tools like Django and Ruby on Rails allow us to develop projects faster and to make significant changes later in the project timeline. Usability testing methods should now be adapted to fit this modern approach to development.

Downe, Natalie. 24 Ways to impress Your Friends (2005). Articles>Usability>Testing>Methods

Fast, Cheap, and Good: Yes, You Can Have It All

The sooner you complete a usability study, the higher its impact on the design process. Slower methods should be deferred to an annual usability checkup.

Nielsen, Jakob. Alertbox (2007). Articles>Usability>Testing>Methods

Getting the Complete Picture with Usability Testing

Good usability testing definitely provides an opportunity for clear-cut improvements in the usability of Web sites. In order to increase the chances of success with usability testing, it is important to measure effectiveness, efficiency, and satisfaction–they all measure different aspects of the usability of a Web site. If only one or two of these measures are used, it would provide an incomplete or partial picture of the possible human performance and user satisfaction results.

Bailey, Robert. Usability.gov (2006). Articles>Usability>Testing

Giveaways to Thank Helpful Users--What's Best?

What is the best way to thank helpful users for participating in a usability study? Carl Myhill asked this question to a popular discussion group of the Usability community. The following are some of the replies.

Dick, David J. Usability Interface (2004). Articles>Usability>Testing

Hotspots and Hyperlinks: Using Eye-Tracking to Supplement Usability Testing

This article discusses how eye-tracking can be used to supplement traditional usability test measures. User performance on two usability tasks with three e-commerce websites is described. Results show that eye-tracking data can be used to better understand how users initiate a search for a targeted link or web object. Frequency, duration and order of visual attention to Areas of Interest (AOIs) in particular are informative as supplemental information to standard usability testing in understanding user expectations and making design recommendations.

Russell, Mark C. Usability News (2005). Articles>Usability>Testing>Eye Tracking

Hotspots and Hyperlinks: Using Eye-Tracking to Supplement Usability Testing

This article discusses how eye-tracking can be used to supplement traditional usability test measures. User performance on two usability tasks with three e-commerce websites is described. Results show that eye-tracking data can be used to better understand how users initiate a search for a targeted link or web object. Frequency, duration and order of visual attention to Areas of Interest (AOIs) in particular are informative as supplemental information to standard usability testing in understanding user expectations and making design recommendations.

Russell, Mark C. uiGarden (2006). Articles>Usability>Testing>Eye Tracking

How Many Users Should You Test With in Usability Testing?

Doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?

Spillers, Frank. Demystifying Usability (2005). Articles>Usability>Testing>Methods

How Much Interaction is Too Much?

Watching a colleague facilitate usability testing, the author asks what constitutes too much interaction by the facilitator. What his colleagues had to say got me thinking.

Anderson, Clifford. Usability Interface (2004). Articles>Usability>Testing

Improving Information Quality Through Iterative Usability Testing  

Testing documents at each step of their development is one way to ensure that the final document is of high quality. It is not necessary to wait until a document is complete before we start testing; we can incorporate iterative testing into the information-development process so that we can build in quality each step of the way.

Grice, Roger A. STC Proceedings (1994). Articles>Usability>Testing

In Defense of Cheating

I am not in favor of deception, trickery, fraud, or swindle. What I wish to change are the curriculum and examination practices of our school systems that insist on unaided work, arbitrary learning of irrelevant and uninteresting facts. I'd like to move them toward an emphasis on understanding, on knowing how to get to an answer rather than knowing the answer, and on cooperation rather than isolation. Cheating that involves deceit is, of course wrong, but we should examine the school practices that lead to cheating: change the practices, and the deceit will naturally diminish.

Norman, Donald A. JND.org (2001). Articles>Education>Instructional Design>Testing

In Praise of Usability Testing

Documentation usability testing has been with us at least since 1983. But, to all intents and purposes, it seems that there is still very little testing being done. Of all the documentation quality assurance activities that we can carry out, usability testing represents one of the best values in the process chain. Without devaluing any of the other quality assurance functions, it is safe to say that money spent on usability testing is extremely well spent.

Hosier, William J. Boston Broadside (1991). Articles>Usability>Testing

Instructions for Branch Office Testing

These are the instructions we gave to the people at various Sun branch offices in Europe and Asia for their user testing of a new design for the company's web pages. In a few places, these instructions refer to web-specific issues, so they will have to be modified slightly for use in other projects. These instructions were sent by electronic mail to those local Sun reps who had volunteered to lead a test.

Nielsen, Jakob. Alertbox (1996). Articles>Usability>Methods>Testing

Introduction to Eyetracking: Seeing Through Your Users' Eyes

Over the coming months, I'll use eyetracking to evaluate a lot of world-renowned user interfaces--including Web sites like Amazon.com, Google News, and eBay; Rich Internet Applications (RIAs); and desktop applications--and analyze quantitative eyetracking data to provide best practices for designing user interface elements like navigation systems, menus, and forms, and for effective ad placement.

Penzo, Matteo. UXmatters (2005). Design>Usability>Testing>Eye Tracking

Is A Lab Essential For User Testing?

Once an organisation decides to go ahead with a user testing programme, the questions really begin. Is it really necessary to undertake testing in a 'usability lab'? And what exactly should a fully functioning lab consist of anyway? As one might imagine, opinion is divided on these issues. We take a quick look at what a typical lab might consist of and the pros and cons of lab-based testing.

Farrell, Tom. Frontend Infocentre (2001). Articles>Usability>Testing>Methods

Iterative Usability Research Methods: Why Testing Isn't Enough  

Discusses how to choose different usability methods for iterative research. Slides only.

Rosenbaum, Stephanie L. Tec-Ed, Inc. (1999). Articles>Usability>Testing>Methods

Iterative Usability Testing as Continuous Feedback: A Control Systems Perspective    

This paper argues that in the field of usability, debates about number of users, the use of statistics, etc. in the abstract are pointless and even counter-productive. We propose that the answers depend on the research questions and business objectives of each project and thus cannot be discussed in absolute terms. Sometimes usability testing is done with an implicit or explicit hypothesis in mind. At other times the purpose of testing is to guide iterative design. These two approaches call for different study designs and treatment of data. We apply control systems theory to the topic of usability to highlight and frame the value of iterative usability testing in the design lifecycle. Within this new metaphor, iterative testing is a form of feedback which is most effective and resource-efficient if done as often as practically possible with project resources and timelines in mind.

Genov, Alex. Journal of Usability Studies (2006). Articles>Usability>Testing>Methods