http://tc.eserver.org/publisher/Journal_of_Usability_Studies A listing of works published by Journal of Usability Studies in the field of technical communication. en-us Copyright (c) 2005-08 by the EServer. All rights reserved. tclib-editorial@eserver.org (TC Library Editorial Board) webmaster@eserver.org (Geoffrey Sauer) http://tc.eserver.org/images/newlogo.gif http://tc.eserver.org/dir/Journal_of_Usability_Studies http://tc.eserver.org/34873.html http://tc.eserver.org/34873.html Despite the authoritative nature of international standards for usability, many of them are not widely used. This paper explains both the benefits and some of the potential problems in using usability standards in areas including user interface design, usability assurance, software quality, and usability process improvement. http://tc.eserver.org/34874.html http://tc.eserver.org/34874.html The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including Web sites, cell phones, interactive voice response systems, TV applications, and more. It provides an easy-to-understand score from 0 (negative) to 100 (positive). While a 100-point scale is intuitive in many respects and allows for relative judgments, information describing how the numeric score translates into an absolute judgment of usability is not known. To help answer that question, a seven-point adjective-anchored Likert scale was added as an eleventh question to nearly 1,000 SUS surveys. Results show that the Likert scale scores correlate extremely well with the SUS scores (r=0.822). The addition of the adjective rating scale to the SUS may help practitioners interpret individual SUS scores and aid in explaining the results to non-human factors professionals. http://tc.eserver.org/34875.html http://tc.eserver.org/34875.html The trade show booth on the exhibit floor of a conference is traditionally used for company representatives to sell their products and services. However, the trade booth environment also creates an opportunity, for it can give the development team easy access to many varied participants for usability testing. The question is can we adapt usability testing methods to work in such an environment? Extremely rapid usability testing (ERUT) does just this, where we deploy a combination of questionnaires, interviews, storyboarding, co-discovery, and usability testing in a trade show booth environment. We illustrate ERUT in actual use during a busy photographic trade show. It proved effective for actively gathering real-world user feedback in a rapid paced environment where time is of the essence. http://tc.eserver.org/34876.html http://tc.eserver.org/34876.html A study of how 23 Taiwanese and North American subjects use a consumer electronic product shows that culture strongly affects the usability of the product. Survey data shows that North American users had much lower levels of user satisfaction and perceptions of effectiveness and efficiency than Taiwanese users. On the other hand, results on performance were unclear, indicating similar levels of effectiveness for both cultural groups and conflicting results on levels of efficiency. http://tc.eserver.org/33357.html http://tc.eserver.org/33357.html There are countless usability blogs, message boards and listservers. But to my knowledge, no one has attempted to integrate all this information into a single, collaborative knowledge space. I believe that creating such a knowledge space would be of immense benefit to the usability profession and would be a wonderful platform on which to refine our understanding of social computing and knowledge management. http://tc.eserver.org/33235.html http://tc.eserver.org/33235.html A proposed design for the American Society of Clinical Oncology (ASCO) Web site was evaluated against the original design in terms of the ease with which the right starting points for key tasks were located and processed. This report focuses on the eye tracking methodology that accompanied other conventional usability practices used in the evaluation. Twelve ASCO members were asked to complete several search tasks using each design. Performance measures such as click accuracy and time on task were supplemented with eye movements which allowed for an assessment of the processes that led to both the failures and the successes. The report details three task examples in which eye tracking helped diagnose errors and identify the better of the two designs (and the reasons for its superiority) when both were equally highly successful. Advantages and limitations of the application of eye tracking to design comparison are also discussed. http://tc.eserver.org/32360.html http://tc.eserver.org/32360.html This project implemented and reviewed several methods to collect data about users' information seeking behavior on news provider Web sites. While browsing news sites, participants exhibited a tendency toward a breadth-first search approach where they used the home page or a search results page as a hub to which they returned and then linked to other pages. Generally, they browsed before using search. Information seeking patterns were consistent within-user but varied somewhat across users. Most behaviors were characterized as visually scanning with users spending much time scrolling. http://tc.eserver.org/32361.html http://tc.eserver.org/32361.html In this article, the chief editor of the recently published book Maturing Usability: Quality in Software, Interaction and Value reports her experiences, from the very beginning when the book project was conceived to the time when the book was delivered. http://tc.eserver.org/32362.html http://tc.eserver.org/32362.html There is a need to develop new usability testing environments and methodologies for unconventional interactive systems. Pursuant to that need, we developed a low-cost test environment for a Head-Mounted Display (HMD)-based, virtual reality system called Osmose. Osmose was difficult to test for many reasons, one of which was its style of interaction. We began setting up the testing environment about two weeks before the start of the usability testing. We learned many lessons throughout the experience. This paper summarizes the study findings, both methodological - how to setup and conduct a usability lab for such an environment - as well as conceptual -the human experiences and behavioral patterns involved in using an immersive environment. http://tc.eserver.org/32363.html http://tc.eserver.org/32363.html This study used an applied ethnographic research method to investigate human-computer interaction (HCI) between call center agents and agent-facing software in the context of contact-center culture. Twenty semi-structured interviews were completed, along with non-participant observation at two contact centers, one that followed a user-centered design (UCD) process for software development and another that did not. Agent productivity and satisfaction at the non-UCD center were hampered by poor task-UI integration, ambiguous text labels, and inadequate UI standardization. Agents required multiple applications to complete a single task, leading to long task times and cognitive strain. In contrast, the UCD center used a unified UI that reduced task times and decreased cognitive strain. In both centers, the workflow was reported to be stressful at times; however, management at both companies employed high involvement work processes that mitigated this stress. Implications for possible high-involvement UI design are considered and a strategy for applied ethnographic research is discussed. http://tc.eserver.org/32364.html http://tc.eserver.org/32364.html This paper describes a tool for log file recording and a method for quickly and easily analysing human-computer interaction with mobile devices. The tool logs screenshots and quantitative interaction data, such as number of clicks and timestamps. The analysing tool provides the ability to evaluate the interaction sequences and to export an MS Excel®-sheet for statistical analysis. To evaluate the tool, a usability study was conducted comparing the effectiveness of this tool in the laboratory and in the mobile context. Findings show that the tool is the first step toward a very effective, unobtrusive analysing method for user interaction in the mobile context. Combined with debriefing methods, it would be an optimized way for usability testing with mobile devices. http://tc.eserver.org/32355.html http://tc.eserver.org/32355.html Engineering design tasks require designers to continually compare, weigh, and choose among many complex alternatives. The quality of these selection decisions directly impacts the quality, cost, and safety of the final product. Because of the high degree of uncertainty in predicting the performance of alternatives while they are still just sketches on the drawing board, and the high cost of poor choices, mathematical decision methods incorporating uncertainty have long held much appeal for product designers, at least from a theoretical standpoint. http://tc.eserver.org/32356.html http://tc.eserver.org/32356.html Usability practice needs a procedure to identify, record, count, and highlight tool switch events for study. This paper describes one that supports the trained observers on which User-Centered Design relies to detect problems and causes, and evaluate design changes. http://tc.eserver.org/32357.html http://tc.eserver.org/32357.html This article is a case study of a demonstration project intended to prove the value of usability testing to a large textbook publishing house. In working with a new client, however, the research team discovered that what our client thought were simple problems for their users were actually complex problems that required the users to evaluate potential solutions in a surprisingly complex context of use. As Redish (2007) predicted, traditional ease of use measures were "not sufficient" indicators and failed to reveal the complex nature of the tasks. Users reported high levels of satisfaction with products being tested and believed they had successfully completed tasks which they judged as easy to complete when, in fact, they unknowingly suffered failure rates as high as 100%. The study recommends that usability specialists expand our definition of traditional usability measures so that measures include external assessment by content experts of the completeness and correctness of users' performance. The study also found that it is strategically indispensable for new clients to comprehend the upper end of complexity in their products because doing so creates a new space for product innovation. In this case, improving our clients' understanding of complexity enabled them to perceive and to take advantage of a new market niche that had been unrealized for decades. http://tc.eserver.org/32358.html http://tc.eserver.org/32358.html Easy task completion (traditional usability) is not enough in the Web world. Appealing visual site design is not enough. A site visitor needs to not only be attracted to a site and able to figure out how to buy (or register, sign up, etc.)-they need in addition to be able to tell quickly that a site will meet their needs, and they need to want to buy from this site, as opposed to a competitor's site. This is a key aspect of overall Web site success. http://tc.eserver.org/32359.html http://tc.eserver.org/32359.html Limited research exists on the relevance of hedonic dimensions of human-computer interaction to usability, with only a small set of this research being empirical in nature. Furthermore, previous research has obtained mixed support for gender differences regarding perceptions of attractiveness and usability in Web site design. This empirical research addresses the above gap by studying the effects of color temperature and gender on perceptions of Web site aesthetics. http://tc.eserver.org/30436.html http://tc.eserver.org/30436.html User satisfaction and usefulness are measured using usability studies that involve real customers. Given the nature of software development and delivery, having to conduct usability studies can become a costly expense in the overall budget. A major part of this expense is the participant costs. Under this condition, it is desirable to reduce the number of participants without sacrificing the quality of the experiment. If a company could use a smaller participant pool and get the same results as the entire pool; this would result in significant savings. Given a participant pool of size N, is there a subset of N that would yield the same results as the entire population? This research addresses this question using a data-mining clustering tool called Applications Quest. http://tc.eserver.org/30435.html http://tc.eserver.org/30435.html A comparative usability evaluation was performed using various subjective evaluation methods, including Mobile Phone Usability Questionnaire (MPUQ). Further, decision-making models using Analytic Hierarchy Process (AHP) and multiple linear regression were developed and applied. Although the mean rankings of the four phones were not significantly different across the evaluation methods, there were variations across the methods in terms of the number of rank orderings, preference proportions, and methods to select their initial preference. Thus, this study provided a useful insight into how users make different decisions through different evaluation methods. Also, the result showed that answering a usability questionnaire affected a user's decision-making process for comparative evaluation. http://tc.eserver.org/30437.html http://tc.eserver.org/30437.html Discusses reasons that practitioners read research papers and the obstacles that they face when reading research papers. Jarrett provides several examples and suggestions for improving the accessibility of research papers for practitioners. Her suggestions include writing clear titles, ensuring that the abstract states the study population and limitations of the study, and ensuring that the conclusions are written clearly. She also discusses her criteria for determining whether or not a research paper is relevant to her work. http://tc.eserver.org/30434.html http://tc.eserver.org/30434.html Much research has been devoted to developing usability evaluation methods that are used in evaluating interaction designs. More recently, however, research has shifted away from evaluation methods and comparisons of evaluation methods to issues of how to use the raw usability data generated by these methods. Associated with this focus is the assumption that the transformation of the raw usability data into usability information is relatively straightforward. We would argue that this assumption is incorrect, especially for novice usability practitioners. In this article, we present a structured process for transforming raw usability data into usability information that is based on a new way of thinking about usability problem data. The results of a study of this structured process indicate that it helps improve the effectiveness of novice usability practitioners. http://tc.eserver.org/30049.html http://tc.eserver.org/30049.html Animated characters have been a popular research theme, but the respective desktop applications have not been well-received by end-users. The objective of this study was to evaluate the use of an animated character for presenting information and navigating music videos within an interactive television (ITV) application. Information was displayed over music video clips with two alternative user interfaces: 1) semi-transparent information overlays, 2) an animated character. For this purpose, the differences between ITV and desktop computing motivated the adaptation of the traditional usability evaluation techniques. The evaluation revealed that users reported higher affective quality with the animated character user interface. Although the success of animated characters in desktop productivity applications has been limited, there is growing evidence that animated characters might be viable in a domestic environment for leisure activities, such as interactive TV. http://tc.eserver.org/30043.html http://tc.eserver.org/30043.html In the article, 'A Methodology for Testing Voting Systems' (JUS, November 2006, pp7-21), Selker, Rosenzweig, and Pandolfo discuss their methodology for usability testing of voting systems. With so much at stake in the usability of our ballots and voting systems, we can only applaud any research in this field. There is little history of research in this area, so discussions of test protocols are especially valuable. Unfortunately, although this article sets out to compare 'the relative merit in realistic versus lab style experiments for testing voting technology,' it falls short of this goal. If their point is that real-world testing is important because real election environments add burdens that are not present in lab settings, this conclusion is not supported by any of the work described. http://tc.eserver.org/30048.html http://tc.eserver.org/30048.html A major impediment in global user interface development is that there is inadequate empirical evidence for the effects of culture in the usability engineering methods used for developing these global user interfaces. This paper presents a controlled study investigating the effects of culture on the effectiveness of structured interviews in international usability evaluation. The experiment consisted of a usability evaluation of a website with two independent groups of Indian participants. Each group had a different interviewer; one belonging to the Indian culture and the other to the Anglo-American culture. The results show that participants found more usability problems and made more suggestions to an interviewer who was a member of the same (Indian) culture than to the foreign (Anglo-American) interviewer. The results of the study empirically establish that culture significantly affects the efficacy of structured interviews during international user testing. The implications of this work for usability engineering are discussed. http://tc.eserver.org/30040.html http://tc.eserver.org/30040.html In this editorial, I describe our birth and some personal experiences as I lived through those times. I present these observations, not as a historian, but as a usability professional viewing events of 15 years ago through my personal filter. http://tc.eserver.org/30041.html http://tc.eserver.org/30041.html Heuristic Evaluation is a discount usability engineering method involving three or more evaluators who evaluate the compliance of an interface based on a set of heuristics. Because the quality of the evaluation is highly dependent on their skills, it is critical to measure these skills to ensure evaluations are of a certain standard. This study provides a framework to quantify heuristic evaluation skills. Quantification is based on the number of unique issues identified by the evaluators as well as the severity of each issue. Unique issues are categorized into eight user interface parameters and severity is categorized into three. A benchmark computed from the collated evaluations is used to compare skills across applications as well as within applications. The result of this skill measurement divides the evaluators into levels of expertise. Two case studies illustrate the process, as well as its applications. Further studies will help define an expert's profile. http://tc.eserver.org/30045.html http://tc.eserver.org/30045.html This paper compares the relative merit in realistic versus lab style experiments for testing voting technology. By analyzing three voting experiments, we describe the value of realistic settings in showing the enormous challenges for voting process control and consistent voting experiences. The methodology developed for this type of experiment will help other researchers to test polling place protocols and administration. Comparing the results from laboratory experiments with voter verification and realistic voting experiments further validates the procedure of testing equipment in laboratory settings. The methodology and protocol for testing voting systems can be applied to any voting technology. This protocol matches the real-world conditions of voting by replicating them for the experiment. http://tc.eserver.org/30044.html http://tc.eserver.org/30044.html "Don't people in your field actually know anything?" From what I read on the list-servs, blogs, and other places where usability professionals hang out, there are many people even in our field who would answer "No!" They tend to see usability as a craft and question whether anything can be known about human-system design in a way that can be codified, or at least codified in a useful way. I would disagree, but I do think our field is passing through an evolutionary stage, where we have been working toward satisfying the business requirements for improved productivity. http://tc.eserver.org/30047.html http://tc.eserver.org/30047.html This study was a follow-up to determine the psychometric quality of the usability questionnaire items derived from a previous study (Ryu and Smith-Jackson, 2005), and to find a subset of items that represents a higher measure of reliability and validity. To evaluate the items, the questionnaire was administered to a representative sample involving approximately 300 participants. The findings revealed a six-factor structure, including (1) Ease of learning and use, (2) Assistance with operation and problem solving, (3) Emotional aspect and multimedia capabilities, (4) Commands and minimal memory load, (5) Efficiency and control, and (6) Typical tasks for mobile phones. The appropriate 72 items constituted the Mobile Phone Usability Questionnaire (MPUQ), which evaluates the usability of mobile phones for the purpose of making decisions among competing variations in the end-user market, determining alternatives of prototypes during the development process, and evolving versions during an iterative design process. http://tc.eserver.org/30050.html http://tc.eserver.org/30050.html The System Usability Scale (SUS) was administered verbally to native English and non-native English speakers for several internally deployed applications. It was found that a significant proportion of non-native English speakers failed to understand the word 'cumbersome' in Item 8 of the SUS (that is, 'I found the system to be very cumbersome to use.') This finding has implications for reliability and validity when the questionnaire is distributed electronically in multinational usability efforts. http://tc.eserver.org/30042.html http://tc.eserver.org/30042.html Increasingly sophisticated technologies, such as On-Line Analytical Processing (OLAP) and Geospatial Information Systems (GIS), are being leveraged for conducting community health assessments (CHA). Little is known about the usability of OLAP and GIS interfaces with respect to CHA. We conducted an iterative usability evaluation of the Spatial OLAP Visualization and Analysis Tool (SOVAT), a software application that combines OLAP and GIS. A total of nine graduate students and six community health researchers were asked to think-aloud while completing five CHA questions using SOVAT. The sessions were analyzed after every three participants and changes to the interface were made based on the findings. Measures included elapsed time, answers provided, erroneous actions, and satisfaction. Traditional OLAP interface features were poorly understood by participants, and combined OLAP-GIS features needed to be better emphasized. The results suggest that the changes made to the SOVAT interface resulted in increases in both usability and user satisfaction. http://tc.eserver.org/30046.html http://tc.eserver.org/30046.html Wireless Application Protocol (WAP) is designed to allow access to the Internet on a mobile phone. Attempts to explain its limited success have focused on attitudinal and cognitive reasons for non-use, finding that although people recognize the benefits of WAP, issues like lack of content, privacy concerns, and reference group behavior account for non-use. Such explanations have also been incomplete in that they have not addressed problems related to actual use and interaction with the technology. Our article studies the use of WAP as situated action. We focus on how users make sense of WAP pages and how they disambiguate in situ the responses from the service, i.e., new pages and new menus. Our method of transcribing videos of WAP use following the conventions of conversation analysis offers a cost-effective tool for understanding user interaction with technology and provides useful implications for design. http://tc.eserver.org/29452.html http://tc.eserver.org/29452.html This paper evaluates the quality of recommendations for improving a user interface resulting from a usability evaluation. The study compares usability comments written by different authors, but describing similar usability issues. The usability comments were provided by 17 professional teams who independently evaluated the usability of the website for the Hotel Pennsylvania in New York. The study finds that only 14 of the 84 studied comments (17%) addressing six usability problems contained recommendations that were both useful and usable. Fourteen recommendations were not useful at all. Sixteen recommendations were not usable at all. Quality problems include recommendations that are vague or not actionable, and ones that may not improve the overall usability of the application. The paper suggests characteristics for "useful and usable recommendations," that is, recommendations for solving usability problems that lead to changes that efficiently improve the usability of a product. http://tc.eserver.org/29454.html http://tc.eserver.org/29454.html This paper describes work on developing usable interfaces for creating and editing methods for high-throughput screening of chemical and biological compounds in the domain of life sciences automation. A modified approach to metaphor-based interface design was used as a framework for developing a screening method editor prototype analogous to the presentation of a recipe in a cookbook. The prototype was compared to an existing screening method editor application in terms of effectiveness, efficiency, and satisfaction of novice users and was found to be superior. http://tc.eserver.org/29451.html http://tc.eserver.org/29451.html The world of usability practitioners is undergoing massive changes. I know because I read it in the New York Times. http://tc.eserver.org/29453.html http://tc.eserver.org/29453.html This paper describes a usability study of the Nedap voting machine in the Netherlands. On the day of the national elections, 566 voters participated in our study immediately after having cast their real vote. The research focused on the correspondence between voter intents and voting results, distinguishing between usability (correspondence between voter intents and voter input) and machine reliability (correspondence between voter input and machine output). For the sake of comparison, participants also cast their votes using a paper ballot. http://tc.eserver.org/28014.html http://tc.eserver.org/28014.html A collaborative knowledge space would provide great value to the usability community. In particular it would: Help define the field and give it a presence that provides professionals and the public with a single source for theoretical, practical and speculative information about usability; encourage the integration of research and practice; invite colleagues in related fields to participate and share their perspectives; serve as a platform to advance our understanding of collaboration and knowledge management tools. Most of the tools needed to implement a collaborative knowledge space are already available and there are a number of related activities already underway that could feed into this project. It would be a great deal of work but I believe it would also yield a great deal of benefit. http://tc.eserver.org/28016.html http://tc.eserver.org/28016.html IBM was contracted to provide a new Air Defence Command and Control (ADCC) system for the Royal Air Force. The IBM Human Factors (HF) team was responsible for the design of the operations room, workstations and the graphical user interfaces. Because the project was safety-related, IBM had to produce a safety case. One aspect of the safety case was a demonstration of the operational effectiveness of the new system. This paper is an in-depth case study of the user testing that was carried out to demonstrate the effectiveness of the system. Due to time constraints the HF team had to observe five participants working simultaneously. Further, to provide a realistic operational environment, up to twenty-eight operators were required for each test. The total effort for this activity was four person-years. The paper will detail the considerations, challenges and lessons learned in the creation and execution of these multi-user user tests. http://tc.eserver.org/28021.html http://tc.eserver.org/28021.html Is culture dead as a topic of interest to usability and user-interface usability and design professionals? One European anthropologist/ethnographer wrote recently that 'culture is dead' and only of interest to people in the USA (who seemingly have little or no understanding of other cultures around the world). On the other hand, another (USA) usability/design professional recently stated that she thought cross-cultural issues were one of the most important and potent trends in product/service development. Who is right? http://tc.eserver.org/28020.html http://tc.eserver.org/28020.html A case study was done to study whether usability expert evaluation and testing are suitable for game development. In the study, a computer game under development was first evaluated and then tested. Game developers were then asked to rate the findings and give other feedback about the methods used and the results gained. It was found that the usability expert evaluation and testing provided both novel and useful data for game development. Based on these and the other results it is argued that the usability expert evaluation and testing have considerable face validity in game development. In addition to the usefulness and face validity of the methods it was studied whether the usability experts participating in the game usability expert evaluation should be double experts. It was found that there was no significant difference in the number or the rated relevancy of the problem the gamer and non-gamer usability specialists found. http://tc.eserver.org/28018.html http://tc.eserver.org/28018.html A usability assessment entailing a paper prototype was conducted to examine menu selection theories on a small screen device by determining the effectiveness, efficiency, and user satisfaction of a popular cellular phone's menu system. Outcomes of this study suggest that users prefer a less extensive menu structure on a small screen device. The investigation also covered factors of category classification and item labeling influencing user performance in menu selection. Research findings suggest that proper modifications in these areas could significantly enhance the system's usability and demonstrate the validity of paper-prototyping which is capable of detecting significant differences in usability measures among various model designs. http://tc.eserver.org/28023.html http://tc.eserver.org/28023.html This paper argues that in the field of usability, debates about number of users, the use of statistics, etc. in the abstract are pointless and even counter-productive. We propose that the answers depend on the research questions and business objectives of each project and thus cannot be discussed in absolute terms. Sometimes usability testing is done with an implicit or explicit hypothesis in mind. At other times the purpose of testing is to guide iterative design. These two approaches call for different study designs and treatment of data. We apply control systems theory to the topic of usability to highlight and frame the value of iterative usability testing in the design lifecycle. Within this new metaphor, iterative testing is a form of feedback which is most effective and resource-efficient if done as often as practically possible with project resources and timelines in mind. http://tc.eserver.org/28019.html http://tc.eserver.org/28019.html Knowledge gained from usability testing is often applied merely to the immediate product under test and then forgotten--at least at an organizational level. This article describes a usability knowledge management system (KMS) based on principles of pattern language and use-case writing that offers a way to turn lessons learned from usability testing into organizational knowledge that can be leveraged across different projects and different design teams. http://tc.eserver.org/28024.html http://tc.eserver.org/28024.html Many usability practitioners conduct most of their usability evaluations to improve a product during its design and development. We call these 'formative' evaluations to distinguish them from 'summative' (validation) usability tests at the end of development. A standard for reporting summative usability test results has been adopted by international standards organizations. But that standard is not intended for the broader range of techniques and business contexts in formative work. This paper reports on a new industry project to identify best practices in reports of formative usability evaluations. The initial work focused on gathering examples of reports used in a variety of business contexts. We define elements in these reports and present some early guidelines on making design decisions for a formative report. These guidelines are based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation. Future work will continue to investigate industry practice and conduct evaluations of proposed guidelines or templates. http://tc.eserver.org/28022.html http://tc.eserver.org/28022.html Usability testing a mobile application in the laboratory seems to be sufficient when studying user interface and navigation issues. The usability of a consumer application was tested in two environments: in a laboratory and in a field with a total of 40 test users. The same problems were found in both environments, differences occurred in the frequency of findings between the contexts. Results indicate that conducting a time-consuming field test may not be worthwhile when searching user interface flaws to improve user interaction. In spite of this, it is possible that field testing is worthwhile when combining usability tests with a field pilot or contextual study where user behavior is investigated in a natural context. http://tc.eserver.org/28025.html http://tc.eserver.org/28025.html A usability study was conducted to identify usability problems as well as recommendations for improvement for three travel sales websites. The study performed testing on twenty participants, between the ages of 19 and 65, recruited from the university campus consisting of students, faculty, and staff. The three websites tested were Expedia.com, Orbitz.com, and Travelocity.com. Each participant was given general instructions and a pre-survey to determine their demographics and level of Internet experience. The usability study tested participants on the task of finding the same itinerary on each travel website. The participant during testing was under observation of the experimenter that maintained an observation log. A post-survey along with a debriefing session was conducted to gather additional feedback. The average testing time for participants was 30 minutes. The results of this study are presented as well as a future research discussion consisting of the development of usability guidelines for designers of travel websites. http://tc.eserver.org/28015.html http://tc.eserver.org/28015.html A proposed design for the American Society of Clinical Oncology (ASCO) Web site was evaluated against the original design in terms of the ease with which the right starting points for key tasks were located and processed. This report focuses on the eye tracking methodology that accompanied other conventional usability practices used in the evaluation. Twelve ASCO members were asked to complete several search tasks using each design. Performance measures such as click accuracy and time on task were supplemented with eye movements which allowed for an assessment of the processes that led to both the failures and the successes. The report details three task examples in which eye tracking helped diagnose errors and identify the better of the two designs (and the reasons for its superiority) when both were equally highly successful. Advantages and limitations of the application of eye tracking to design comparison are also discussed. http://tc.eserver.org/28017.html http://tc.eserver.org/28017.html Small sample sizes are a fact of life for most usability practitioners. This can lead to serious measurement problems, especially when making binary measurements such as successful task completion rates (p). The computation of confidence intervals helps by establishing the likely boundaries of measurement, but there is still a question of how to compute the best point estimate, especially for extreme outcomes. In this paper, we report the results of investigations of the accuracy of different estimation methods for two hypothetical distributions and one empirical distribution of p. If a practitioner has no expectation about the value of p, then the Laplace method ((x+1)/(n+2)) is the best estimator. If practitioners are reasonably sure that p will range between .5 and 1.0, then they should use the Wilson method if the observed value of p is less than .5, Laplace when p is greater than .9, and maximum likelihood (x/n) otherwise. http://tc.eserver.org/26614.html http://tc.eserver.org/26614.html The biggest problem facing the usability field is how to scale up massively so that we can impact all the user interface designs in the world. How big is this challenge? As of November 2005, there are about 75 million websites on the Internet. There are also about 30 million intranets inside corporate firewalls. Thus, there are more than 100 million user interface designs, just in the online space. http://tc.eserver.org/25396.html http://tc.eserver.org/25396.html This peer-reviewed publication is dedicated to promoting and enhancing the practice, research, and methods of usability engineering. Its aim is to provide usability practitioners and researchers with a forum to share usability research case studies, empirical findings, opinions and experiences in the practice and teaching of usability engineering, and good practices in usability engineering.