Case Study: Conducting Large-Scale Multi-User User Tests on the United Kingdom Air Defence Command and Control System
http://www.usabilityprofessionals.org/upa_publications/jus/2006_may/hey_air_defence.pdf
  peer-reviewed

Hey, Elliott
Journal of Usability Studies
2006

Abstract:

IBM was contracted to provide a new Air Defence Command and Control (ADCC) system for the Royal Air Force. The IBM Human Factors (HF) team was responsible for the design of the operations room, workstations and the graphical user interfaces. Because the project was safety-related, IBM had to produce a safety case. One aspect of the safety case was a demonstration of the operational effectiveness of the new system. This paper is an in-depth case study of the user testing that was carried out to demonstrate the effectiveness of the system. Due to time constraints the HF team had to observe five participants working simultaneously. Further, to provide a realistic operational environment, up to twenty-eight operators were required for each test. The total effort for this activity was four person-years. The paper will detail the considerations, challenges and lessons learned in the creation and execution of these multi-user user tests.