Verification of Video Frame Latency Telemetry for UAV Systems Using a Secondary Optical Method

Research output: Contribution to journalArticlepeer-review

Abstract

This paper presents preliminary work and a prototype computer vision optical method for latency measurement for an UAS (Uninhabited Aerial System) digital video capture, encode, transport, decode, and presentation subsystem. Challenges in this type of latency measurement include a no-touch policy for the camera and encoder as well as the decoder and player because the methods developed must not interfere with the system under test. The goal is to measure the true latency of displayed frames compared to observed scenes (and targets in those scenes) and provide an indication of latency to operators that can be verified and compared to true optical latency from scene to display. Latency measurement using this optical computer vision method was prototyped using both flight side cameras and H.264 encoding using off-the-shelf equivalent equipment to the actual UAS and off-the-shelf ground systems running the Linux operating system and employing a Graphics Processor Unit to accelerate video decode. The key transport latency indicator to be verified on the real UAS is the KLV (Key Length Value) time-stamp which is an air-to-ground transport latency that measures transmission time between the UAS encoder elementary video stream encapsulation and transmission interface to the ground receiver and ground network analyzer interface. The KLV time-stamp is GPS (Global Positioning System) synchronized and employs serial or UDP (User Datagram Protocol) injection of that GPS clock time into the H.264 transport stream at the encoder, prior to transport over an RF (Radio Frequency) or laboratory RF-emulated transmission path on coaxial cable. The hypothesis of this testing is that the majority of capture-to-display latency comes from transport due to satellite relay as well as lower latency line-of-sight transmission. The encoder likewise must set PTS/DTS (Presentation Time Stamp / Decode Time Stamp) to estimate bandwidth-delay in transmission and in some cases may either over or underestimate this time resulting in either undue added display latency or frame drop-out in the latter case. Preliminary analysis using a typical off-the-shelf encoder showed that a majority of observed frame latency is not due to path latency, but rather due to encoder PTS/DTS settings that are overly pessimistic. The method and preliminary results will be presented along with concepts for future work to better tune PTS/DTS in UAS H.264 video transport streams.

Original languageAmerican English
JournalAIAA Science and Technology Conference
StatePublished - Jan 12 2014

Keywords

  • UAV
  • UAS
  • digital video
  • real-time
  • latency

Disciplines

  • Aeronautical Vehicles
  • Computer and Systems Architecture
  • Systems and Communications

Cite this