Analyze Your Web Site Traffic

If you run a Web site, you're probably already thinking about tracking and analyzing the traffic it gets. Knowing how many pages are accessed, when, by whom, and for what purposes can mean the difference between simply having a Web site and building a sound Web strategy. Understanding how people use your site can help you--and your sales and marketing team--generate more traffic. If you can track your audience, learn which pages and resources are most popular, and identify technical problems and system bottlenecks, you can deliver a better experience. And that's the best way to keep people coming back to your site.

Aviram, Mariva H. Builder.com (1998). Articles>Usability>Assessment

Are Organizations Doing Enough to Improve Customer Satisfaction

Time-to-market pressure can diminish product testing time and quality. The results are product recalls, shoddy merchandise, and apologies by CEOs about poor quality. The consequence is the loss of consumer confidence. Don’t these companies realise that there’s no compromise on quality? I’m sure that these companies are ISO 9000 certified or have a Total Quality Management (TQM) program, so what is the problem? Perhaps the problem is not with ISO 9000 or TQM but with the way it is used.

Dick, David J. and Shelby Rosiak. Usability Interface (2000). Articles>Usability>Assessment>ISO 9000

Comparative Usability Evaluation: CUE

CUE-1 is a comparative usability test of a Windows calendar program (Task Timer for Windows, version 2) carried out by four professional teams. The results were published at UPA98 in Washington DC in June 1998. CUE-2 is a comparative usability test of the popular www.hotmail.com web-site. Nine teams have simultaneously usability tested this web-site.

Dialog Design (1998). Articles>Usability>Assessment

Comparing Assessment Techniques  

In just the last few years, we have begun to see research studies comparing usability testing to other techniques for assessing usability. In general, usability testing has found more of the most serious usability problems and fewer of the least serious problems than other methods. Heuristic evaluation--having people evaluate the interface either from their own expertise or from a set of guidelines--has achieved mixed results. Although usability testing seems expensive compared to other methods, it may be less expensive when considered on the basis of 'cost per problem that needs to be fixed.'

Redish, Janice C. 'Ginny'. STC Proceedings (1993). Articles>Usability>Testing>Assessment

A Comparison of Two Evaluation Techniques for Technical Documentation

This study compared two evaluation techniques, Usability Testing and Cognitive Walkthrough, in their ability to identify errors in aviation maintenance documentation. The techniques were evaluated to see how much unique information they each produced as well as the type of errors identified. Results showed that the techniques were complementary in their findings and both are recommended in the development of technical documentation.

Rogers, Bonnie Lida, Chris Hamblin and Alex Chaparro. Usability News (2005). Articles>Documentation>Assessment>Usability

Criteria for Focused Data Collection  

The objective of this task is to propose and validate a mechanism whereby projects can identify their needs for software measurement data and focus their data collection activities using a minimum standardized set of software measures. The purpose of this strategy is to evolve a process that will enable NASA projects to tailor with their data collection activities to their unique needs for effective management control indicators, but also encourages consistent data collection that will facilitate statistical analysis across NASA domains.

Wilson, William M. NASA (2001). Articles>Usability>Methods>Assessment

Decision Models for Comparative Usability Evaluation of Mobile Phones Using the Mobile Phone Usability Questionnaire (MPUQ)  

A comparative usability evaluation was performed using various subjective evaluation methods, including Mobile Phone Usability Questionnaire (MPUQ). Further, decision-making models using Analytic Hierarchy Process (AHP) and multiple linear regression were developed and applied. Although the mean rankings of the four phones were not significantly different across the evaluation methods, there were variations across the methods in terms of the number of rank orderings, preference proportions, and methods to select their initial preference. Thus, this study provided a useful insight into how users make different decisions through different evaluation methods. Also, the result showed that answering a usability questionnaire affected a user's decision-making process for comparative evaluation.

Ryu, Young Sam, Kari Babski-Reeves, Tonya L. Smith-Jackson and Maury A. Nussbaum. Journal of Usability Studies (2007). Articles>Usability>Assessment>Surveys

Evaluation Toolbox for Aviation Technical Publications

This article describes the Evaluation Toolbox (Chaparro et al., 2004) - an aid to understand the process of evaluating the usability of aviation maintenance documentation -- from the initial development stage through the final pre-publication stage. This toolbox provides techniques to help technical writers better understand their users and to evaluate their documentation more effectively and efficiently.

Rogers, Bonnie Lida, Chris Hamblin and Alex Chaparro. Usability News (2005). Articles>Documentation>Assessment>Usability

Examining the Relationship Between Quality Writing and Quality Reading    

This article introduces the ISTE approach as a way to measure the relationship between the design of a document and the way that users handle it. ISTE is an acronym for Information SubTypes and Effects. The article describes the classification of information into a general typology as well as into a specific one, to illustrate how the approach can be adapted to specific usability questions. ISTE also requires a typology for categorizing possible user reactions. This article first positions ISTE within the broader context of usability testing and then describes its basic approach and some recent applications. Unlike most formal research methods, ISTE does not require an inordinate amount of time and effort for analyzing high-fidelity data. Electronic versions such as the ERR instrument allow for the creation of a complete and detailed profile of the use of a document 10 minutes after task completion. These characteristics make ISTE a viable instrument for gathering usability metrics in industrial settings.

van der Meij, Hans. Technical Communication Online (2000). Articles>Usability>Testing>Assessment

First Principles

Principles fundamental to the design and implementation of effective interfaces, whether for traditional GUI environments or the web. Of late, many web applications have reflected a lack of understanding of many of these principles of design, to their great detriment. Because an application or service appears on the web, the principles do not change. If anything, applying these principles become even more important.

Tognazzini, Bruce. Nielsen Norman Group (1994). Articles>Usability>Assessment

From Writing Documents to Meeting User Information Needs  

As professional Technical Communicators, we constantly struggle with the question 'Is what we produce being used and is it making a difference?' Too often, we base our measures of success on our own views of what makes a good document, Help system, or other information product. Through broader and more detailed analysis, and by measuring success based on how our information products affect others, we can increase the effectiveness of our information products and increase our value to our users and companies.

Hurst, Bob, Cindy Line and Kerry Newberry. STC Proceedings (2003). Articles>Writing>Usability>Assessment

Heuristic Evaluation - a Step By Step Guide

Evaluation and testing is an important part of your website development process. Usability tests gather data about the usability of your site by a group of users performing specific tasks.

Danino, Nicky. SitePoint (2001). Articles>Usability>Methods>Assessment

Heuristic Evaluation Quality Score (HEQS): A Measure of Heuristic Evaluation Skills  

Heuristic Evaluation is a discount usability engineering method involving three or more evaluators who evaluate the compliance of an interface based on a set of heuristics. Because the quality of the evaluation is highly dependent on their skills, it is critical to measure these skills to ensure evaluations are of a certain standard. This study provides a framework to quantify heuristic evaluation skills. Quantification is based on the number of unique issues identified by the evaluators as well as the severity of each issue. Unique issues are categorized into eight user interface parameters and severity is categorized into three. A benchmark computed from the collated evaluations is used to compare skills across applications as well as within applications. The result of this skill measurement divides the evaluators into levels of expertise. Two case studies illustrate the process, as well as its applications. Further studies will help define an expert's profile.

Kirmani, Shazeeye and Shamugam Rajasekaran. Journal of Usability Studies (2007). Articles>Usability>Assessment>Heuristic Evaluation

Making Usability Recommendations Useful and Usable    

This paper evaluates the quality of recommendations for improving a user interface resulting from a usability evaluation. The study compares usability comments written by different authors, but describing similar usability issues. The usability comments were provided by 17 professional teams who independently evaluated the usability of the website for the Hotel Pennsylvania in New York. The study finds that only 14 of the 84 studied comments (17%) addressing six usability problems contained recommendations that were both useful and usable. Fourteen recommendations were not useful at all. Sixteen recommendations were not usable at all. Quality problems include recommendations that are vague or not actionable, and ones that may not improve the overall usability of the application. The paper suggests characteristics for "useful and usable recommendations," that is, recommendations for solving usability problems that lead to changes that efficiently improve the usability of a product.

Molich, Rolf, Robin Jeffries and Joseph Dumas. Journal of Usability Studies (2007). Articles>Usability>Assessment>Reports

Not Getting Personal: Assessing Website Effectiveness  

Websites are sometimes evaluated primarily on first impressions or personal preference. More difficult to ascertain is their success in terms of communication. Assessments of websites can benefit from research and developments from fields such as usability studies, linguistics, professional writing, and rhetoric.

Durham, Marsha. STC Proceedings (1999). Articles>Web Design>Assessment>Usability

Readable Computer Documentation    

A retrospective look shows earlier advice still relevant to both predicting and producing readable writing. For prediction, refined readability formulas with stronger criterion passages and updated familiar -word lists have appeared, although the computerization of readability tests sometimes encourages misapplying or misinterpreting them when screening text. For production, attention to sentence construction, word characteristics, and information density remains relevant to both drafting and revising computer documentation for readability, especially since reading speed and reader preference often interact with comprehension in practical settings.

Klare, George R. Journal of Computer Documentation (2000). Articles>Documentation>Assessment>Usability

Results on a Study of Usability Testing

A study I conducted as part of my graduate work at the University of Houston shows that technical communicators find many benefits in usability testing of documentation but cannot quantify them. The study’s purpose was to identify an exact return on investment figure that could be used to convince otherwise unwilling management to initiate usability studies. While the data failed to produce such a figure, impact analysis indicated that the return on investment is probably high.

Ostrander, Elaine. Usability Interface (2000). Articles>Usability>Assessment

Return on Investment for Usability

Development projects should spend 10% of their budget on usability. Following a usability redesign, websites increase usability by 135% on average; intranets improve slightly less.

Nielsen, Jakob. Usability News (2003). Articles>Usability>Assessment

Rolling Assessment: Observing On-Going User Responses to a Next Generation Internet Telemedicine Application Through Successive Stages of Development  

Industry and academia have developed disparate models for evaluating the success of a given project. Academia applies a scientific research model, with an emphasis on environmental control and static reproducibility. Industry, software development in particular, has employed a more user-centered approach, gathering user feedback to inform successive waves of redesign. This study describes how the blending of these two approaches can result in quantifiable descriptive data, which can be immediately applied to the design process.

Eliot, Matthew J., Tristan Robinson, Robin Maberry, Judith A. Ramey, Brent Stewart. STC Proceedings (2002). Articles>Usability>Assessment

Selling Usability

This is a collection of presentations, white papers and other resources on selling usability. Links are provided where they are available.

Usability Professionals Association. Articles>Usability>Assessment

Success Rate: The Simplest Usability Metric

In addition to being expensive, collecting usability metrics interferes with the goal of gathering qualitative insights to drive design decisions. As a compromise, you can measure users' ability to complete tasks. Success rates are easy to understand and represent usability's bottom line.

Nielsen, Jakob. Alertbox (2001). Articles>Usability>Assessment

System Usability Scale and Non-Native English Speakers  

The System Usability Scale (SUS) was administered verbally to native English and non-native English speakers for several internally deployed applications. It was found that a significant proportion of non-native English speakers failed to understand the word 'cumbersome' in Item 8 of the SUS (that is, 'I found the system to be very cumbersome to use.') This finding has implications for reliability and validity when the questionnaire is distributed electronically in multinational usability efforts.

Finstad, Kraig. Journal of Usability Studies (2006). Articles>Usability>Assessment>Methods

Ten Usability Heuristics

A collection of ten heuristics applicable to most usability evaluations. Nielsen originally developed the heuristics for heuristic evaluation in collaboration with Rolf Molich in 1990.

Nielsen, Jakob. Alertbox (1994). Articles>Usability>Assessment