Effectiveness of Non-Immersive vs. Immersive Scenario Training, 07-R9666

Printer Friendly Version

Principal Investigator
Doel A. Durieux

Inclusive Dates:  01/02/07 – Current

Background - Game-Based Learning (GBL) has become a popular topic in the instructional community, provoking much speculation on how it could be used and what it could accomplish. A review of the technical literature indicated that much has been written about what results GBL should produce, but little empirical data proves what actually works under controlled trials. Studies into the outcomes of applied GBL appeared to be nonexistent, and there was almost no empirical evidence that this method of delivering instructional content is more effective than traditional training methods. Gaming hardware, software, design techniques and data collection systems present a unique opportunity for technological transfer, considering the technology developed by the entertainment industry, if the potential benefits can be identified.

The objectives of this study are two-fold. First, the impact of GBL on several important training outcomes relative to a more traditional e-learning approach will be measured. Second, a prototype GBL immersive environment will be produced for demonstration to Air Mobility Command (AMC) senior management at Scott AFB, Ill.

Approach - The proposed approach for developing instructional programs is to implement a instructional systems design process that includes analysis, design, development, validation, and reporting. During the analysis phase, specific topics for which GBL offers the most promise, such as detailed factual material requiring extensive repetition and scenario-based practice, will be identified. During the design and development phases, GBL activities incorporating the needed features will be developed, such as challenging problems that can be solved only by using the relevant factual material to gain game-winning points. During the evaluation phase, trainees' learning will be assessed by analyzing data from several tests and surveys that will be part of both the immersive GBL and the non-immersive traditional version of the training. The research subjects (U.S. Air Force trainees) will be randomly assigned to one of two versions. Test results will be analyzed with standard statistical techniques, including significance tests.

The outcomes to be measured fall into three distinct categories and include the following items:

  • Pre-Learning Measures – including prior knowledge of the task and prior attitudes toward following prescribed procedures.
  • Learning Process Measures – including time to complete and number of repetitions and practices.
  • Data Measures – including accuracy and speed of filling out forms, knowledge of correct procedures and steps, preference and attitude toward learning from this kind of course (versus other training taken in the past), attitude toward the value of following specified procedures, and attitude toward further training on this subject.

Accomplishments - To date, the team has successfully developed two versions of the training course. The project team traveled to two U.S. Air Force bases designated by HQ AMC to conduct usability tests on both versions of the course. The team is currently analyzing the data gathered from each base. A final report will be prepared with the findings.

2007 Program Home