SUCCESS STORIES IN
WEARABLE IMMERSIVE MULTI-MEDIA INFORMATION SYSTEM

ARMY RESEARCH LAB (ARL)
Mobile Vision-Based Augmented Reality System Provides Soldiers Geographical Information in the Field

Since 1996, Rockwell Scientific Company (RSC) has teamed with the Army Research Lab (ARL) and the University of Illinois at Urbana-Champaign (UIUC) to research, design, and develop a mobile Augmented Reality (AR) system to provide geographic information to the soldier in the field.

For this purpose, the Army Research Lab, under the Advanced Displays and Interactive Displays Federated Laboratory program, funded the Integrated Displays Testbed to further the research of intuitive human computer interaction methods. Research has shown that judicial use of augmented reality techniques can improve the situational awareness for a soldier. One method is to show overlay information in a head-mounted display so as to appear registered with the view of the human observer. The information displayed may be tactical, mission related, or situation related.

HAfter years of experience in Head-mounted Display (HMD) technologies, 3D graphics representation, and wearable systems, RSC developed a Wearable Immersive Multi-Media Information System (WIMMIS). WIMMIS is built from commercial off-the-shelf (COTS) products. This wearable, tetherless system provides full user immersion in 3D terrain, together with an auditory experience in which sounds seem to emanate from particular locations in the 3D space surrounding the user.

VTree was chosen for its support of the TerraPage™ fast database paging format, flexible scene graph editing capabilities, and its support for various model formats.

The 3D terrain visualization, created with VTree, is tightly linked to PocketPC miniature map displays, a 2D battle plan display, and a 2D isometric display of the same terrain to provide a common operating picture across a range of display platforms.

The WIMMIS consists of a wearable computer, a sourceless tracking device, headphones, a variable opacity HMD, a gamepad, wireless data transmitters, and audio/video receivers, all integrated into a wearable vest unit and connected through a wireless communication link to the Integrated Displays Testbed. Specifically a CyberTrack II® head tracker, attached to a set of headphones worn by the user sends the head-orientation data through a wireless internet connection to a nearby scene server for correct rendering of the surrounding terrain at a selected location. The video signal is then transmitted back to the user via an analog RF connection. A head-worn SONY Glasstron® display provides immersive visualization. This display has an adjustable opacity for true Augmented Reality display. The game pad provides interfaces for navigation in the virtual terrain and the selection of various display modes. Alternatively, a GPS device can be used to provide positional data to the scene server. The headphones receive a wirelessly transmitted audio signal that is rendered by a 3D audio server for correct spatial auralization. Significant objects in the scene are configured as sources of 3D audio. These include tanks, helicopters, and distributed sensors. The audio track associated with an object may change depending on the object’s current status. For example, a sensor may play different tracks depending on whether a target has been detected, and if so, the type of detected targets. With 3D audio, situation awareness is enhanced because a soldier can be alerted about threats that are currently out of his field of view.

Quantum3D’s VTree®
ARscape, the scene server of the WIMMIS tetherless system, was created using the VTree API from Quantum3D. VTree was chosen for its support of the TerraPage fast database paging format, flexible scene graph editing capabilities, platform independence, and its support for various model/image formats. “VTree’s rapid 3D application prototyping framework allowed the visualization portion of WIMMIS to be developed and integrated in just under four months,” said Clement Tam, developer of the ARscape application.

Balancing the amount of information needed, graphical detail, and area covered, while maintaining performance goals was also critical to the success of the project. The terrain database alone used four levels of detail with 512x512 pixel tiles. The number of polygons at any one time can vary from 50,000 to 80,000.

Initially ARscape was a MFC application developed on an SGI 320 NT workstation. It has since been deployed on a variety of PC workstations, including a notebook computer. ARscape supports two interaction modes: 1) egocentric and 2) exocentric. Egocentric view is the quintessential VR first-person view. Exocentric view provides a bird’s eye view of the environment in which the user is always shown at the center of the display. An optional transitional movement between the two modes is also supported. Terrain-hugging mode, in which the observer is kept at a constant altitude from the ground (1.7m), can also be selected while moving over the terrain. To further enhance scene realism, clear and simulated foggy weather conditions are supported. The field of view and terrain resolution are adjustable. The locations and movements of various battle units, along with tactical information such as avenue of approach and line of defensible terrain, are also displayed in the ARscape view. “Look-to-query” of such objects results from simply aiming at the interested object, placing it in the center reticle. Pertinent information about the object then appears next to it. A user can also mark an interesting terrain location and return to it later by the press of a button. A choice of snap teleport or gradual movement from the current location to that landmark is available.

ARscape has been deployed on the following platforms successfully:
As a part of the Integrated Displays Testbed, this program was demonstrated at the Federated Laboratories (FEDLAB) Symposium in March 2001.

Summary
This ambitious project has seen the computer industry change many times during its five-year history. Initially research was performed on a desktop Pentium 166 MHz machine running under Windows 95. Now the wearable platform is a Xybernaut® PC MA IV with a 233 MHz Pentium processor. Possible applications of this technique include manufacturing or maintenance of complex products, information displays for a soldier in battlefield scenarios, or fire fighters. The information to be displayed may be tactical, mission related, or situation related.

The demonstration of the feasibility of high quality 3D visualization on PC platforms with ARscape is significant for mobile Augmented Reality applications. Currently, a WIMMIS user has an operational range of 100 feet from the workstation. With high performance graphics cards being incorporated into notebook computers, and perhaps eventually in wearable computers, the WIMMIS setup may achieve true mobility fairly soon.

For more information on ARscape or the Integrated Displays Testbed contact:

Rockwell Scientific Company (RSC)
Attn: Clement Tam
1049 Camino Dos Rios, P.O. Box 1085
Thousand Oaks, CA 91358
(805) 373-4858
ctam@rwsc.com

Learn more about how Quantum3D can optimize your synthetic environment for training and simulation applications. Visit our website at www.quantum3d.com; email us at salesinfo@quantum3d.com, or call us at U.S. toll free 1-800-827-1980, or internationally 408-361-9999.