Evaluation of Digital Libraries Using Snowball Sampling  

This article describes how snowball sampling was applied in two different cases to evaluate digital collections. The first digital library was evaluated by conducting in–person interviews with survey participants. For the second digital library, an e–mail survey was mailed to site users. The results are compared and a cost–benefit analysis is provided. The author concludes that the convenience of an e–mail survey is not necessarily the most effective way to survey users.

Peterson, Elaine. First Monday (2005). Articles>Web Design>Assessment>Surveys

Evaluation of Training Programs in Technical Communication  

To remain viable in this economy, executives and administrators must produce efficiently and hence must assure sound evaluation of training programs in technical communication. These decision-makers can benefit from the insights of professional evaluators of educational programs so as to establish goals, secure resources, review the activities, and report results. Described and then illustrated here is the CIPP-model to review the activities, that is, the contexts, input, processes, and products. Well-done evaluations lift the level of communication skills, the morale of the students and faculty, and the organization’s products.

Battle, Mary V. STC Proceedings (1993). Presentations>Education>Assessment

Evaluation Toolbox for Aviation Technical Publications

This article describes the Evaluation Toolbox (Chaparro et al., 2004) - an aid to understand the process of evaluating the usability of aviation maintenance documentation -- from the initial development stage through the final pre-publication stage. This toolbox provides techniques to help technical writers better understand their users and to evaluate their documentation more effectively and efficiently.

Rogers, Bonnie Lida, Chris Hamblin and Alex Chaparro. Usability News (2005). Articles>Documentation>Assessment>Usability

Even Excellent Sites Benefit from Expert Reviews

Get the flavor of an Expert Review as Dr. Schaffer points out the strengths and weaknesses of 11 award-winning Web sites.

Schaffer, Eric M. and Phil Goddard. Human Factors International (2006). Presentations>Web Design>Assessment

Examining the Relationship Between Quality Writing and Quality Reading    

This article introduces the ISTE approach as a way to measure the relationship between the design of a document and the way that users handle it. ISTE is an acronym for Information SubTypes and Effects. The article describes the classification of information into a general typology as well as into a specific one, to illustrate how the approach can be adapted to specific usability questions. ISTE also requires a typology for categorizing possible user reactions. This article first positions ISTE within the broader context of usability testing and then describes its basic approach and some recent applications. Unlike most formal research methods, ISTE does not require an inordinate amount of time and effort for analyzing high-fidelity data. Electronic versions such as the ERR instrument allow for the creation of a complete and detailed profile of the use of a document 10 minutes after task completion. These characteristics make ISTE a viable instrument for gathering usability metrics in industrial settings.

van der Meij, Hans. Technical Communication Online (2000). Articles>Usability>Testing>Assessment

Expert Judgments Versus Reader Feedback: A Comparison of Text Evaluation Techniques    

Are technical writers able to predict the results of a reader-focused text evaluation? In this article we report a study with fifteen technical writers, who were asked to point out the reader problems in a public information brochure. The brochure was also evaluated with thirty readers from the target audience (using a combination of the plus-minus method, a questionnaire, and user protocols). The results of both kinds of text evaluation show little overlap. The technical writers only predicted a small proportion of the reader feedback, and produced a lot of new problem detections. In addition, there was little agreement among the technical writers with regard to their problem detections.

de Jong, Menno D.T. and Leo R. Lentz. Journal of Technical Writing and Communication (1996). Articles>Writing>Assessment>Methods

An Exploratory Study Of Adoption Of Software and Hardware By Faculty in The Liberal Arts and Sciences    

Universities and colleges are investing millions of dollars in information technology infrastructure to support teaching, research, and service, and thousands of dollars annually in faculty training programs. And yet, many college graduates entering the workforce lack adequate technology skills. To ascertain the frequency of faculty adoption of information technology, we surveyed a random sample of faculty in the liberal arts and sciences departments in our university. Overall faculty members (n = 174) reported a low usage of information technology for teaching, though the rate of software adoption is higher than the rate of hardware adoption. While opportunities to learn technology are available, about two-thirds of the faculty members have not completed the available seminars and workshops on information technologies but prefer more informal ways of learning information technology, such as talking with other faculty members.

Yohon, Teresa and Donald E. Zimmerman. Journal of Technical Writing and Communication (2006). Academic>Computing>Assessment

Factors Affecting Readability

No one has found a way to really help writers create readable prose. Robert Gunning developed a method for calculating the 'Fog Index' and Rudolph Flesch worked out more than one formula for measuring the simplicity of writing. By one of Flesch's formulas (the one without personal pronouns), Ronald S. Lemos in the February, 1985 issue of Communications of the ACM (CACM) was able to prove that CACM required two less years of school to read than Datamation. Statistics can prove anything. I have no idea what Sophomore in High School could read the CACM cover to cover and even understand most of it. Flesch's book 'The Art of Plain Talk' was given to me at a Yourdon Systems Analysis course. The Instructor handed it to each of us, saying something like 'read this and you'll be a manager in no time' (supposedly, management is handed to the least efficient person who can also write well). The book is full of examples, mostly journalistic, showing how good writers evoke human interest. Of course, these writers had human events, thoughts and feelings as their focal points, not software, I doubt whether any of the graduates of that week ever used Flesch as a reference for grading their own documentation. How would Bernard Shaw have documented software? Or Mingus played it? This paper addresses these burning issues.

Hallgren, Chris. SIGDOC Proceedings (1986). Articles>Writing>Assessment>Formulas

Faculty Integration of Technology into Instruction and Students' Perceptions of Computer Technology to Improve Student Learning  

There has been a remarkable improvement in access and rate of adoption of technology in higher education. Even so, reports indicate that faculty members are not integrating technology into instruction in ways that make a difference in student learning. To help faculty make informed decisions on student learning, there is need for current knowledge of faculty integration practices. Therefore, the purpose of this study was to examine the nature of the relationship between faculty integration of technology into classroom instruction and students' perceptions of the effect of computer technology to improve their learning. A sample of at least 800 undergraduate students at a participating medium-sized midwest public university was selected using a stratified random sampling technique. The researcher delivered and administered the surveys to the participating students and collected them after completion. 98% of the questionnaires were complete and retained for analysis.

Keengwe, Jared. Journal of Information Technology Education (2007). Articles>Education>Online>Assessment

Fifteen Steps to Select a Content Management System

These are suggested steps to research Content Management System options for your organization, large or small. They can take you from knowing nothing about CMS to final vendor and product selection. Even if you are replacing a CMS with a new tool, you should follow these steps in the CMS lifecycle.

Doyle, Bob. CMS Review (2004). Articles>Content Management>Assessment

Finding a Home for Technical Communication in the Academy    

The placement of technical communication within an academic curriculum presents an interesting challenge for university administrators and faculty. Technical communication is a young discipline that borrows content from several older, more established disciplines. As a younger discipline, technical communication must combine its borrowed ingredients from other areas into a new and complete offering that can attract research funding for professionals in the academy and deliver job opportunities for its students preparing to enter industry. The credibility of technical communication as a new discipline is dependent on its ability to develop a cohesive body of basic and applied research, its ability to manage technological change, and its ability to promote its identity among an army of competing disciplines.

Carver, Michael. ACM SIGDOC (1998). Academic>Education>Assessment

First Principles

Principles fundamental to the design and implementation of effective interfaces, whether for traditional GUI environments or the web. Of late, many web applications have reflected a lack of understanding of many of these principles of design, to their great detriment. Because an application or service appears on the web, the principles do not change. If anything, applying these principles become even more important.

Tognazzini, Bruce. Nielsen Norman Group (1994). Articles>Usability>Assessment

The Five Biggest Mistakes in CMS Selection

Describes the big 'gotchas' in choosing a web content management system.

Welchman, Lisa. CMSwatch (2003). Articles>Content Management>Assessment

Five Tips for Measuring Public Relations and Catching the CEO's Attention

Measuring the effectiveness of PR is critical to moving PR from a tactical function to a strategic component of your company's plan for success. But the old ways of counting clips just aren't good enough to convince today's management executives that their investment in PR and overall communications is paying off. Here are 5 Tips about how to measure PR in ways that will catch the CEO's attention and increase the stature of PR in any organization:

McNamara, Julie. Communication World Bulletin (2003). Articles>Business Communication>Public Relations>Assessment

Focus Research on Your Most Valuable "Capital"

The entire concept of human "capital" seems to have arisen during the last several years of booming economy and scarce availability of skilled employees. When any resource is scarce, it's valuable. Now with the highest rates of layoffs being announced in the U.S. since 1991, let's hope the mindset of management is not on the order of getting the most out of the human "liabilities" they're forced to retain.

Sinickas, Angela D. Sinickas Communications (2001). Careers>Management>Assessment

Forget ROI, Let's Show How We're Making Money

Throw a stone in a room full of communication professionals and there's a good chance you'll hit one that will back up this statement: senior management loves to see ROI measurements, but seeing how communication initiatives create sales trumps all other measurements. From a marketing communication perspective, simply receiving feedback from a sales team can help your team answer most senior-level frustrations. From the perspective of a sales force, understanding marketing efforts (and how those efforts actually work) aids in everyone's ultimate objective: securing sales.

Cohen, Ephraim. Communication World Bulletin (2003). Articles>Management>Communication>Assessment

From Soup to Nuts: Fashioning the Menu for a New Program in Technical Communication  

The process of revising an English Communications emphasis proceeded smoothly for the most part because of good planning by a Curriculum Committee. However, unseen pitfalls and departmental politics hindered some aspects of the experience. It will be necessary to apply lessons learned to continue the revision process and create a successful emphasis.

Allen, Lori A. STC Proceedings (2002). Academic>Education>Instructional Design>Assessment

From Writing Documents to Meeting User Information Needs  

As professional Technical Communicators, we constantly struggle with the question 'Is what we produce being used and is it making a difference?' Too often, we base our measures of success on our own views of what makes a good document, Help system, or other information product. Through broader and more detailed analysis, and by measuring success based on how our information products affect others, we can increase the effectiveness of our information products and increase our value to our users and companies.

Hurst, Bob, Cindy Line and Kerry Newberry. STC Proceedings (2003). Articles>Writing>Usability>Assessment

Fun With Measurement Math!

Recent research tells us that 97 percent of all public relations departments are involved in media relations, and 88 percent evaluate their campaigns using media analysis. On one hand, industry leaders urge us to measure the results of our work via business outcomes; yet on the other, communicators are still asked to supply output results as 'proof of performance.' Is there some link between the two that can cover both? Here are some relatively easy ways to make your media results speak with numbers that management will respect and understand.

Jeffrey, Angela. Communication World Bulletin (2003). Articles>Business Communication>Public Relations>Assessment

Getting the Most from the Quality Improvement Process  

The Quality Improvement Process can be used to improve customer satisfaction and reduce cost and cycle time. The “R.U.S.T.” four-step action process provides a helpful guideline to begin quality improvement by Recognizing the opportunity for improvement, Understanding requirements and expectations, Starting to apply systematic process changes, and Testing the results by measuring.

McDonald, Audrey M. STC Proceedings (1996). Presentations>Workplace>Assessment

The Grading System of the Real World

At the beginning of each semester, the instructor hands out a syllabus packet which often contains a course outline or schedule and an explanation of the grading policy. The work world has grading systems too, and you need to know about them in advance so you can prepare for Performance Reviews.

Perry, Lynellen D.S. ACM Crossroads (2001). Careers>Workplace>Assessment

Great Indexes

The American Society of Indexers identifies criteria for measuring a great index. An index is not an inverted table of contents, nor is it a simple listing of where certain terms appear in a document. An index consists of a 'compiled list of topics covered in the work, prepared with the reader’s needs in mind.'

Brown, Fred. Allegro Time! (2001). Articles>Indexing>Assessment

Guidelines for Designing and Evaluating the Display of Information on the Web    

These guidelines are intended to assist Web designers, authors, and editors in their efforts to create Web pages that effectively reveal—rather than obscure or confuse—the information they are trying to present. These guidelines are also intended to be used to assist in the evaluation of existing Web sites. Of course, the design of a Web site can, to some degree, be modified by the user or by the characteristics of the browser or monitor enlisted to display it. The guidelines, consequently, acknowledge that in a very real sense, users may also assume the role of designer. The guidelines, therefore, are also intended to help users make informed decisions about how to make a display easier to use.

Williams, Thomas R. Technical Communication Online (2000). Articles>Web Design>Assessment>Visual Rhetoric

Guidelines for Evaluating Work with Digital Media

These guidelines from the MLA Committee on Computers and Emerging Technologies (CCET) deal with the hiring, reappointment, tenure, and promotion processes, are designed to help departments and faculty members implement effective evaluation procedures. These guidelines were approved by the MLA Executive Council at its 19-20 May 2000 meeting.

Modern Language Association (2000). Academic>Research>Assessment

Hard Measures are Key to Gauging the Effectiveness of Communication on the Bottom Line

In conducting its landmark 2003 Communication ROI Study, which focuses on the relationship between an organization's internal communication strategy and practices and its shareholder returns, Watson Wyatt made some surprising findings regarding the relationship between effective external and internal communication.

Vogt, Peter. Communication World Bulletin (2003). Articles>Business Communication>Workplace>Assessment