Analyzing and Reporting Usability Data

The Just-In-Time (JIT) method of data analysis has the virtue of immediacy, rapid turn-around, and team involvement; however there are several disadvantages. First, this type of analysis is problem-focused, rather than goal-focused. Long lists of problems are generated, but there is no clear relation to specific usability goals. Second, developers may not be able to fix things immediately so the context of the problem may be lost when it is time to fix the problem. Third, the JIT analysis requires that the entire development team observe the testing sessions since problems may occur that are the responsibility of different developers.

Wilson, Chauncey E. Usability Interface (1997). Articles>Usability>Testing>Reports

Better Reports: How to Communicate the Results of Usability Testing  

You've spent several days setting up a usability test, recruiting the participants and running it. Then you've pored over the data. What next? If you are doing usability testing as part of user-centred design within a business setting, then there are many ways that you can communicate the results. This paper looks at reports and then considers presentation and observation as alternatives to reports.

Jarrett, Caroline. STC Proceedings (2004). Articles>Usability>Testing>Reports

Towards the Design of Effective Formative Test Reports    

Many usability practitioners conduct most of their usability evaluations to improve a product during its design and development. We call these 'formative' evaluations to distinguish them from 'summative' (validation) usability tests at the end of development. A standard for reporting summative usability test results has been adopted by international standards organizations. But that standard is not intended for the broader range of techniques and business contexts in formative work. This paper reports on a new industry project to identify best practices in reports of formative usability evaluations. The initial work focused on gathering examples of reports used in a variety of business contexts. We define elements in these reports and present some early guidelines on making design decisions for a formative report. These guidelines are based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation. Future work will continue to investigate industry practice and conduct evaluations of proposed guidelines or templates.

Theofanos, Mary Frances and Whitney Quesenbery. Journal of Usability Studies (2006). Articles>Usability>Testing>Reports