Expert Judgments Versus Reader Feedback: A Comparison of Text Evaluation Techniques    

Are technical writers able to predict the results of a reader-focused text evaluation? In this article we report a study with fifteen technical writers, who were asked to point out the reader problems in a public information brochure. The brochure was also evaluated with thirty readers from the target audience (using a combination of the plus-minus method, a questionnaire, and user protocols). The results of both kinds of text evaluation show little overlap. The technical writers only predicted a small proportion of the reader feedback, and produced a lot of new problem detections. In addition, there was little agreement among the technical writers with regard to their problem detections.

de Jong, Menno D.T. and Leo R. Lentz. Journal of Technical Writing and Communication (1996). Articles>Writing>Assessment>Methods

Using Scenarios for the Evaluation of Municipal Web Sites  

It appears to be difficult for experts to predict the problems users of documents and websites experience. Realistic usage scenarios may help experts to achieve better prediction rates, since they focus the experts' attention explicitly on the users and their use situations. We developed a set of scenarios for evaluating municipal websites, and used them to evaluate 15 websites. In this paper, we will describe the scenario evaluation method, and the feedback it helps to provide. The results suggest that a scenario-based evaluation method may be a fruitful way of enhancing experts' sensitivity for detecting user problems. open, except for the instruction to concentrate on potential reader problems.

de Jong, Menno D.T. and Leo R. Lentz. STC Proceedings (2005). Articles>Web Design>Assessment>Government