Hardware-Software Co-Design for Low-Latency Object Detection in Autonomous Drones

Autonomous aerial vehicles increasingly rely on real-time visual perception for navigation, obstacle avoidance, surveillance, and inspection tasks. Object detection is the cornerstone of this perception pipeline, yet the strict size, weight, and power (SWaP) constraints of drones, combined with the sub-30-millisecond latency budgets demanded by high-speed flight, make conventional software-only deployment of deep convolutional detectors impractical. This paper presents a hardware-software co-design framework for low-latency object detection on autonomous drones. We review the evolution of lightweight detection architectures and edge accelerator platforms, propose a heterogeneous system-on-chip (SoC) architecture that combines a neural processing unit (NPU), a field-programmable gate array (FPGA) pre/post-processing pipeline, and general-purpose cores, and describe a closed-loop co-optimization methodology spanning network architecture search, quantization, pruning, dataflow scheduling, and memory hierarchy configuration. We evaluate the proposed framework on a representative aerial object detection benchmark using five hardware targets, demonstrating that the co-designed configuration achieves an end-to-end latency of 6.8 ms per frame (147 frames per second), a 14x speed-up over an embedded ARM CPU baseline, while retaining 97% of the mean average precision (mAP) of the floating-point reference model and reducing energy per inference by more than 20x relative to a discrete mobile GPU. The results confirm that jointly optimizing model structure and accelerator architecture, rather than treating either in isolation, is essential to meeting the latency, accuracy, and energy requirements of real-time drone perception.

Cited via DOI — this article is permanently archived on Zenodo.