Tracking UAVs is becoming increasingly critical, in both the civilian and military spheres. Take for instance, recent mysterious sightings of UAPs over the U.S. eastern seaboard. Meanwhile, on the other side of the world, drone warfare has enlarged the front line to include remote cities and defensive positions. Monitoring drones is becoming a significant challenge.

There are plenty of techniques to do so, but most suffer from slow updates or inaccurate detections, both of which struggle with fast-moving small drones. A team from the Vienna University of Technology thinks they might have found a way to eliminate the weaknesses of previous drone tracking technologies, and they proved it by using a telescope.

Existing tracking technologies

Drone tracking technologies often fall into two main categories: detectors and trackers. While there are many multi-spectral methods of detecting drones, such as via radar, acoustic or other, optical tracking is the most widely used, as it is the easiest to improve using non-expert human feedback.

Traditional detection methods suffer from high false alarm rates, making it difficult to accurately identify drones. Recently, deep learning algorithms have made progress in drone detection accuracy. However, while these are more accurate than the previous classifier and filter models, they are also painfully slow. Some drones can move up to 20 m/s, leaving the detection algorithm in the dust. By the time a drone is recognized, it's long gone

Tracking algorithms tend to operate faster, but they aren't as accurate as detectors. Tracking algorithms compare image frames of the object to chart a trajectory and speed. This is easily disrupted by foreign objects, occlusions, weather or misclassification. Tracking models can be easily confused and lose sight of the drone, even if it is still in the imaging system's field of vision.

Reinforcing detection and tracking algorithms

The Vienna University of Technology team thinks they have found a way to combine the accuracy provided by the detectors with the speed provided by the trackers into a faster and more accurate system than either system alone.

The system relies on a reliability metric to help decide when to begin or end object tracking. Deep learning algorithms provide a confidence value for each detection. Tracker algorithms can offer the likelihood they will correctly track and estimate an object's position. A higher tracker reliability enables smarter collaboration between the detection and tracking components in a system. When the confidence of a detection from a deep learning-based detector is higher than the tracker’s reliability, the system can reinitialize the tracker, ensuring the most reliable information is used.

Additionally, tracker reliability allows the system to dynamically adapt to changes in the tracking environment. For instance, if factors like lighting conditions or drone behavior cause the tracker’s reliability to decrease, the system can adjust by relying more on the detector or reinitializing the tracker as needed.

Hardware implementation and results

One important consideration is what to do with the new tracking information. A telescope's field of view is relatively tiny, and a fast-moving drone can quickly move out of it. The team built a control system that moves the telescope to track the drone. The calculated trajectory is fed into the telescope's pan and tilt controllers.

To prove their system's effectiveness, the researchers tested it in the field by having it track a series of drones in front of both clear backgrounds (the sky) and complex backgrounds, such as a series of buildings or trees. They then compared their results to those of a stand-alone detector system and a stand-alone tracker system.

Their system scored about the same as existing methods for experiments on a clear background but significantly improved over simple detector-based (6% more accurate) or tracker-based (14% more accurate) algorithms on complex backgrounds. It also had another advantage that neither could match.

Estimating flight paths is key for tracking and goes above and beyond simply detecting the presence of a drone in an image. To estimate a flight path, another algorithm specializing in flight path calculation must be fed two localizations — accurate depictions of where a drone is from frame to frame — by the detection algorithm. In this metric, the parallel detector/tracker approach outperformed the detector-only approach by a whopping 49%. While the tracker-only approach performed better, it did not necessarily have the accuracy to predict a flight path. Impressively, this was also capable as the drone was flying away from the detector up to a distance of 4 km.

Even incremental improvements in drone detection could save lives in some cases, such as on the battlefields of Ukraine, but for now, this system isn't yet ready for prime time. It still requires a massive computer with a GPU to run its complete algorithm, and future work is planned to integrate an additional telescope with a larger field of view to improve detection accuracy and tracking even more.

For now, accurate tracking in all scenarios is still a difficult nut to crack. However, given the interest and incentive in doing so, both from the military and civilian law enforcement, plenty of improvements will come in this technology in the near future.