Navigating Virtual Worlds
A new device enables operators to use natural motions in virtual worlds
Virtual reality technologies immerse users in a realistic synthetic environment where they interact with simulated objects in response to simulated situations, making these technologies ideal for training applications. As these trainers increase the level of immersion, realistic actions from the user become increasingly important.
One of the biggest challenges of virtual reality technologies is developing noninvasive yet affordable ways to move in virtual worlds using motions similar to those in real life. Some methods for navigating virtual environments have been inexpensively implemented but are unnatural to the user. For example, moving in a direction by pointing a finger is effective, but when the real-life motion requires a person to walk in that direction the adequacy of the training simulator could be questioned.
Attempts to implement natural motion so far have required users to wear numerous sensors or to move on a moving surface, such as a treadmill. Both methods are effective but can distract or hinder the user. Southwest Research Institute (SwRI) has developed a unique prototype device to incorporate natural motions into virtual reality simulations.
Goal: Detecting Natural Motion
As part of an internal research program, the Advanced Interactive Technologies Department in the SwRI Training Systems and Simulators Division has developed an input device for virtual reality simulations called the Pressure MatTM. This inexpensive, nonintrusive hardware detects motions similar to those a person would use in the real world. The device detects whether a user is standing, walking forward and backward, or sidestepping left and right. Although originally developed to simulate the motions of ground soldiers, the device could be adapted for use in any application that requires user locomotion. For example, the mat could easily be expanded to accommodate crawling.
Using commercial off-the-shelf components, SwRI engineers attached pressure-sensitive resistors, arranged hexagonally to reduce directional bias, on a Lexan® sheet. An analog/digital (A/D) card connects the resistors to a standard personal computer.
Pressure-sensitive resistor applications typically require linear output, achieved by coupling the resistors with amplifiers. However, because the Pressure Mat only needs repeatable responses to recognize patterns, SwRI engineers could use the nonlinear output created by the resistors alone, greatly lowering the cost and simplifying the design. The prototype mat uses 64 resistors. Other applications may require larger mats, which would use many more resistors.
Recognizing gestures with pattern recognition
Pressure patterns on the mat enable the system to detect user motion. SwRI engineers defined a set of gestures for walking forward, backward, left and right and wrote software for a suite of visualization techniques to determine if the pressure patterns were distinct and repeatable. After confirming that they were, the team recorded data from several test subjects and created an algorithm for detecting the step patterns. The real-time software developed reads the Pressure Mat data and detects user gestures.
For some users, the gestures for walking forward and sidestepping are similar, yielding false readings. The mat software assumed that any gestures that were different from the preceding sequence were in error and filtered them out using a majority filter; that is, the system tracked the last three gestures returned by the gesture recognizer. Instead of using the gesture immediately returned by the recognizer, the mat software used the gesture most commonly returned for the previous three cycles.
To improve responsiveness when starting or stopping, filtering was not applied to transitions to or from standing. Standing is rarely reported as a false gesture, so this did not harm the effectiveness of the gesture filter.
The gesture recognition application detects the rate and direction the user is walking and transfers this information to an image generator. The image generator uses these data to change the viewpoint from which the virtual scene is rendered, giving the user the illusion of moving through the virtual environment.
The Pressure Mat system requires only two sensors to be attached to the user: one for head position and one for waist orientation. All other user motion data are derived entirely from sensors on the mat. This minimal tethering is a great advantage compared to other real-time rendering systems that require significantly more tethers.
With additional research and a larger mat, the SwRI team can detect other motions such as crawling without attaching additional sensors to the user. The system even could be expanded to recognize the pressure patterns of more than one person. With closer spacing of the sensors on the mat, it may be possible to eliminate the waist sensor and derive the user's heading from pressure patterns. Closer spacing of the resistors also will improve the responsiveness of the Pressure Mat, thereby allowing detection of patterns for starting and stopping as well.
Result: an intuitive interface
Using the Pressure Mat, subjects could select a target in the virtual world and move toward that target. Users had no difficulty determining how to use the mat: all they had to do was to move in the desired direction.
A curious phenomenon did occur, however, that made users feel they were moving slower than they should have been, even though their stride lengths were reasonable to the scale of the virtual environment. Research at the University of Virginia* suggests the problem may be caused by a limited field of view, which makes people feel they are moving slower in the virtual environment than they are in reality. Users reported fewer problems when they looked frequently at the ground or scanned the environment.
One potential use for the Pressure Mat is in virtual architectural walk-throughs. Because the Pressure Mat forces user exertion, designers could get a better sense of the distances represented in computer models. For example, architects designing malls and sports arenas could use the Pressure Mat to get a true sense of how far visitors must walk to reach a washroom.
The mat also could be used in medical rehabilitation to help accident victims regain motor skills by having them practice "normal" walking patterns as defined by the mat. Other applications include military and sports training programs, as well as video games.
Thus far, the Pressure Mat has proven to be an inexpensive and effective tool for making virtual reality simulators significantly more realistic. As virtual reality technologies continue to evolve, Southwest Research Institute will continue developing new devices and techniques to advance the cutting edge.
*Banton, T.A., Steve, J., Durgin, F.H., Proffitt, D.R. (2000). The calibration of optic flow and treadmill speed during treadmill walking in a virtual environment. Investigative Ophthalmology and Visual Science, 41(4), S718.
Comments about this article? Contact Couvillion at (210) 522-3471 or firstname.lastname@example.org.
Published in the Fall 2001 issue of Technology Today®, published by Southwest Research Institute. For more information, contact Maria Stothoff.