Beyond Visual Augmented Reality

Overview

• Project Code: BeyVAR
• Start Date: November 1, 2023
• End Date: October 31, 2027
• Reference: DAP/22-01
• Funding: Defense Funded Research (DFR)
• Royal Military Academy Involvement: Director
• Quad Chart

Team

Staff

Partners

• Coordinator: Rob Haelterman
• Coordinator Affiliation: Royal Military Academy, Department of Mathematics (MWMW)
• Project Co-promotor: Mathias Becquaert (CISS)
• RMA Researchers: Charles Hamesse, Tiago Troccoli Cunha (CISS)

• Project Partners (RMA): Signal and Image Centre (CISS)
• Project Partners (BE-DEF): N/A
• Project Partners (Other): N/A

Context

The convergence of Augmented Reality (AR) headset technology and miniaturized computing/sensor systems presents novel opportunities for connected soldier applications. Advances in AR technology, surpassing earlier iterations like Google Glass, have led to current implementations in various industrial sectors, enhancing efficiency in processes ranging from manufacturing to construction inspection. Integrating advanced sensing capabilities such as multi-spectral cameras and radio frequency detectors with state-of-the-art AR hardware constitutes a significant research area for the Belgian Defence, potentially extending the limits of human sensory perception in operational contexts.

Objectives

The goal of this study is to integrate a) the most recent compact infrared cameras with AR headsets for real-time, wearable and hands-free usage; b) a RF spectrum sensing device together with our AR system. Through the development of ad-hoc algorithms for sensor fusion, threat or anomaly detection and IR/RF signal visualization, we would effectively extend human vision to IR and RF.

Methodology

For IR: Development of an automated multi-spectral calibration framework, development of multi-spectral SLAM system, development of onboard processing threat detection capabitilies. For RF: Development of an automated RF spectrum sensing with detection of arrival, development of a SLAM-based RF triangulation system, development of onboard processing threat detection capabilities.