Innovative technology for effective mitigation of system latency and image alignment error in Next Generation Helmet Mounted Display Systems (NGHMDS)

Many existing combat platforms such as the F-16 and the F/A-18 are being retrofitted with Helmet Mounted Display (HMD) systems. New advanced aircraft such as the F-35 Joint Strike Fighter (JSF) are committed to using HMDs in combat operations. System latency and image alignment error are major issues associated with their use in flight simulators and these errors must be addressed in order to ensure effective training. System latency manifests itself through the occurrence of physiological disturbances similar to symptoms of simulator sickness and includes eyestrain, headache, nausea, sweating, dizziness, and a general sensation of not feeling well. Additionally, simulator sickness can be a significant distraction during training and may result in ineffective training, negative training, reduced user acceptance, and a reduction in simulator usage. Image alignment error manifests itself by reducing the accuracy of the training environment and may result in ineffective training and negative transfer of training to the real world. Innovative solutions to address latency and alignment error problems must be developed so that training can be optimized as aircrews are afforded the capability to "train as they fight" using Next Generation HMDs in a simulation environment. The current project developed a number of innovative technologies that effectively mitigate both system latency and image alignment error. The technologies developed include: 1) a customized Kalman predictive filter, 2) a learning predictive neural network, and 3) image warping technology. These technologies operate independently yet in concert to continually sample, compare, and adjust their outputs to produce the most accurate prediction of future image placement possible using current available head movement and position data. Results indicate that effective system latency was reduced an average of 65% and image alignment error was reduced an average of 45% from the baseline condition.