Effectiveness of Non-Immersive vs. Immersive Scenario Training, 07-R9666

Printer Friendly Version

Principal Investigator
Doel A. Durieux

Inclusive Dates:  10/04/06 – 01/31/08

Background - Game-Based Learning (GBL) has become a popular topic in the instructional community, provoking much speculation on how it could be used and what it could accomplish. A review of the technical literature indicated that much has been written about what results GBL should produce, but little empirical data proves what actually works under controlled trials. Studies into the outcomes of applied GBL appeared to be nonexistent, and there was almost no empirical evidence that this method of delivering instructional content is more effective than traditional training methods. Gaming hardware, software, design techniques and data collection systems present a unique opportunity for technological transfer, considering the technology developed by the entertainment industry, if the potential benefits can be identified.

The objectives of this study were two-fold. First, the impact of GBL on several important training outcomes relative to a more traditional e-learning approach was measured. Second, a prototype GBL immersive environment was developed for demonstration to Air Mobility Command (AMC) senior management at Scott AFB, Illinois.

Approach - The proposed approach for developing instructional programs was to implement an instructional systems design process that included analysis, design, development, validation and reporting. During the analysis phase, specific topics for which GBL offers the most promise, such as detailed factual material requiring extensive repetition and scenario-based practice, were identified. During the design and development phases, GBL activities incorporating the needed features were developed, such as challenging problems that could be solved only by using the relevant factual material to gain game-winning points. During the evaluation phase, trainees' learned competencies were assessed by analyzing data from several tests and surveys that were part of both the immersive GBL and the non-immersive traditional version of the training. The research subjects (U.S. Air Force trainees) were randomly assigned to one of two versions. Test results were analyzed with standard statistical techniques, including significance tests.

The outcomes measured fell into three distinct categories and included the following items:

  • Pre-Learning Measures – including prior knowledge of task and prior attitudes toward following prescribed procedures.
  • Learning Process Measures – including time to complete and number of repetitions and practices.
  • Data Measures – including accuracy and speed of filling out forms, knowledge of correct procedures and steps, preference and attitude toward learning from this kind of course (versus other training taken in the past), attitude toward the value of following specified procedures and attitude toward further training on this subject.

Accomplishments – To date, the team has successfully developed two versions of the training course. The project team traveled to two U.S. Air Force bases designated by HQ AMC and conducted usability tests on both versions of the course. The team analyzed the gathered data from each base. Proctors observed and commented that the use of an immersive training scenario did indeed capture the students' attention for a longer period of time when compared with the data from the group that took the conventional web-based course. It was discovered that the students did not rush through the GBL course and focused more on the course content rather than on the objective of just finishing the course as quickly as they could.

During the validation period, a rather interesting situation developed. Students were actually competing against each other to see who got the best results. This is an excellent indication that the use of an immersive training scenario enhances the learning experience.

2008 Program Home