Using Mixed Reality with Haptic Feedback for Training Medical
Inclusive Dates: 10/01/00 - Current
Background - A significant amount of medical training is performed using actual patients, which can be costly to patients and hospitals. Medical simulations provide a low-cost, low-risk, readily available option. Improving current medical simulation techniques will enhance the quality of healthcare and reduce the chances for error, potentially saving hospitals and patients thousands of dollars from unnecessary follow-up treatments.
Approach - The purpose of this project is to design, develop, and demonstrate a prototype medical-training simulation. This project will focus on demonstrating the use of mixed reality technologies integrated with current medical simulation techniques to improve the realism and quality of medical training. As shown in the illustrations below, the device will allow a simulated procedure to be performed in a natural manner by permitting the user to work in his or her normal workspace. Haptic and force feedback will be provided in the workspace through the use of a mannequin and PHANTOMTM force feedback device. A charge couple device (CCD) camera will be mounted in the workspace to capture images of the user's hands. Chromakeying techniques will be applied to combine these real images with virtual images representing a simulated patient, which will be displayed on a liquid crystal display (LCD) monitor. The insertion and removal of a catheter introducer will be simulated to the degree necessary to demonstrate the potential of this approach for use as a medical training device.
Accomplishments - The project team has successfully built a preliminary prototype that demonstrates the feasibility of this approach. To date, a display stand has been constructed with the LCD monitor, CCD camera, mannequin hand, and PHANTOMTM force feedback device. Chromakey materials have been selected and integrated to allow for the merging of real and virtual objects. Software has been generated to compute and apply forces when the virtual introducer comes into contact with the virtual patient. This software allows the user to interactively set physical properties associated with the virtual patient's skin, bones and veins, including stiffness and puncture force threshold.