The Defense Systems Information Analysis Center (DSIAC) was asked to identify and locate candidate sense and avoid (SAA) systems for small unmanned aerial vehicles (UAVs) to enable a cargo UAV to deconflict with other air vehicles during delivery missions. SAA systems could be integrated into an existing UAV or be a modular, platform-agnostic system. The inquirer also requested information about organizations performing research and development or that have made progress on the challenges surrounding the inquiry. DSIAC staff performed open-source and Defense Technical Information Center searches to obtain the requested information. DSIAC then compiled a response report for the inquirer consisting of a list of candidate technologies and associated production companies, as well as a summary of relevant organizations and references for further research.
1.0 Introduction
The inquirer requested information on SAA systems to be integrated into small (Group 1–3) UAVs to enable a cargo UAV to deconflict with other air vehicles during delivery missions. Information about organizations and programs conducting relevant research was also requested. The Defense Systems Information Analysis Center (DSAIC) staff searched open-source databases and the Defense Technical Information Center database for relevant information about SAA systems to provide to the inquirer. Descriptions of candidate technologies, associated production companies, and research programs and organizations relevant to the inquiry topic were compiled.
2.0 Key Technologies and Research
An SAA system is a combination of various programmed sensors and controllers that can be integrated in a variety of ways on small UAVs, typically in the UAV airframe. The SAA system enables the UAV to detect any obstacle in its path and avoid it completely while continuing to fly in the airspace. Technologies used in SAA systems may include a combination of cameras, radar, light detection and ranging (LIDAR), and other components. Sections 2.1.1 through 2.1.6 discuss examples of technologies being used or researched in SAA systems for small UAVs.
2.1 Sensors
Sensors are typically employed in a UAV to collect and record data along the flying path, detecting and identifying any obstacles or threats, depending on their specifications. Sensors can provide environmental mapping information to a collision-avoidance program embedded in the main processor. There are multiple types of collision-avoidance programs:
- Nonlinear Model Predictive Control
- Graph Search Algorithm
- Field Method
- Vision-Based Neural Network
The UAV can execute the avoidance action based on the information processed from the obstacle [1].
2.1.1 Cooperative and Noncooperative Sensors
An SAA system typically has two sensor options for small UAV applications: cooperative and noncooperative. Noncooperative sensors include both passive and active technologies to detect obstacles, shown in Table 1.
Table 1: Passive and Active Sensor Technologies for Use in Noncooperative Sensors [1, 2]
| Active Sensors | · Laser ranging · Radar · Sonar |
| Passive Sensors | · Electro-optical · Infrared · Thermal Imaging · Motion Detectors |
Cooperative systems typically include one or more of the following [3]:
- Traffic Collision Avoidance System (TCAS)
- Automatic Dependent Surveillance-Broadcast (ADS-B) System
- Mode C Transponders
ADS-B and TCAS are being miniaturized to be beneficial to small UAVs in current research.
2.1.2 Sensor Fusion
A sensor fusion technique is being researched to overcome limitations of individual sensors in SAA systems for small UAVs. This technique is necessary to improve detection and minimize error in tracking [1]. Advanced development in miniaturization technology is predicted to promote research efforts in integrated sensor fusion and sensor architecture of SAA systems.
2.2 Forward-Looking Sensors and Low-Cost Navigation and Guidance System
Forward-Looking Sensors (FLS) and a low-cost Navigation and Guidance System (NGS) consist of a Vision-Based Navigation (VBN) sensor integrated with a Global Navigation Satellite System (GNSS) [4]. This SAA system performs obstacle detection and tracking by combining measures from different FLS types, including passive and active systems.
The detection and tracking process starts with navigation-based image stabilization and uses image morphology operations to extract relevant features that may represent a collision threat. Obstacle tracking is performed via low-level or high-level tracking. Low-level tracking uses an algorithm with four filter branches allowing obstacle classification, in addition to position and heading estimation. High-level tracking can estimate the velocity of the obstacle by considering the position and heading estimates from the algorithm in use (Viterbi). Risk of collision is assessed using the relative position of the intruding obstacle and the risk area of the UAV using the joint Probability Density Function.
2.3 Patent for Collision and Conflict Avoidance System for UAVs
A collision and conflict avoidance system for UAVs was patented to use accessible, onboard sensors to generate an image of the surrounding airspace, which is then analyzed for imminent conflicts [5]. This process is then followed by a search for avoidance options.
2.4 Airborne SAA Radar Panel
The Massachusetts Institute of Technology (MIT) Lincoln Laboratory developed a phased-array antenna [6], which uses a lightweight, very reliable sensor that enables an onboard SAA system to perform quick and repeatable scanning of the search region. It also satisfies the requirements for small UAVs and supports both aircraft-detecting and weather-sensing modes in a single aperture.
2.5 Safe2Ditch Technology
Safe2Ditch is a crash-management system that resides on a small processor onboard a small UAV [7]. The technology uses intelligent algorithms, knowledge of the local area, and knowledge of the disabled vehicle’s remaining control authority to select and steer to a crash location that minimizes risk to people and property.
2.6 Acoustic SAA Systems
Acoustic SAA systems provide small UAVs with 360° field-of-view SAA capabilities [8]. The systems allow the UAV, equipped with appropriate acoustic sensors and processing, to detect and accurately track other manned or unmanned aircraft, determine if they are a threat, and take appropriate, autonomous measures to avoid them.
3.0 Production Companies
3.1 Intel
Intel conducts research in all areas of SAA using active sensors. In January 2016, they acquired the German drone manufacturer, Ascending Technologies (AscTec), and demonstrated their Intel® RealSense™ technology integrated into an AscTec drone that showcased how it can avoid obstacles and continue to follow the subject [9].
Intel is also working on projects that can create a 3-D map of a UAV’s surroundings, then use it to autonomously navigate through its environment, including rerouting itself around obstacles. This technology is present in Yuneec’s Typhoon H.
3.2 PrecisionHawk
PrecisionHawk offers a range of drone hardware, software, and packages for surveying and mapping [10]. One such product, Low Altitude and Avoidance System (LATAS), provides tracking and avoidance for every drone in the sky using real-time flight data transmission based on world-wide cellular networks.
3.3 Parrot
Parrot S.L.A.M.dunk is a new drone development kit for the creation of autonomous, obstacle-avoidance drones and robots [11]. It integrates software applications based on robotic mapping called “simultaneous localization and mapping,” or SLAM. This capability enables a drone to sense and map its surroundings in 3-D and to localize itself in environments with multiple barriers and where GPS signals are not available, performing obstacle avoidance and using active sensors. It is powered by a Robot Operating System to allow 3-D mapping and uses onboard cameras and sensors for data gathering.
3.4 Neurala
Neurala has developed a passive software solution that analyzes images from off-the-shelf cameras to enhance drone navigation [12]. This software can help identify safe areas to travel and land, while full collision avoidance is still under development [9]. Neurala also launched the Bots Software Development Kit, allowing manufacturers to install artificial intelligence “neural” software into their applications without the need for additional hardware [12].
3.5 DJI
DJI offers the Guidance and Phantom 4 PRO V2.0 drones that are applicable to the desired request. Table 2 describes the difference between the two systems [9].
Table 2: DJI Product Descriptions
| Products | Description |
|---|---|
| Guidance | · Combination of ultrasonic sensors and stereo cameras that allow the drone to detect objects up to 65 ft away and avoid objects at a preconfigured distance. · Made available to the Matrice 100 drone development platform and was incorporated into DJI’s flagship Phantom 4 prosumer drone. |
| Phantom 4 PRO V2.0 | · Equipped with a sensor capable of shooting video and burst mode stills. · FlightAutonomy system includes dual, rear-vision sensors and infrared sensing systems for a total of five directions of obstacle sensing and four directions of obstacle avoidance. |
3.6 Embention
Embention offers SAA with their Veronte Autopilot [13], a mini-high-reliability avionics system for advanced control of unmanned systems. For detection of cooperative elements, it is compatible with mode S transponders, which identify the presence of other aircraft to avoid collisions. For detection of noncooperative sources, the SAA system incorporates sensors capable of detecting potential obstacles in its environment over long distances. The sensors range from directional laser sensors and LIDAR to radar sensors and work together with sensors for weather detection to guarantee safety during the operation. The latest development includes a vision camera to detect the surrounding environment.
3.7 Vigilant Aerospace
Vigilant Aerospace is an innovator in SAA technology for UAVs [14]. They offer the FlightHorizon detect-and-avoid (DAA) system, which is based on National Aeronautics and Space Administration (NASA)-tested and patented technology. FlightHorizon provides a complete, autonomous collision-avoidance solution for both piloted and fully autonomous UAVs to deliver situational awareness, self-separation commands, and collision avoidance.
FlightHorizon also completed a successful Beyond Line-of-Sight (BLOS) flight test, which demonstrated the system’s ability to provide BLOS flight safety for small and mid-sized UAVs. The system detected and tracked intruder aircraft and provided traffic alerts and collision warnings for 100% of the encounters.
3.8 SURVICE Engineering
SURVICE Engineering is working with Near Earth Autonomy to develop autonomy packages which include collision detection and avoidance, as well as landing zone assessment [15]. The technology can identify and classify hazards to UAVs while keeping the UAV safe during all phases of flight. It uses sensors to map the area around an aircraft, including underneath and behind it. This technology automatically recommends an area safely away from obstructions and hazardous terrain that may interfere with the successful landing of the UAV.
3.9 FLARM and MicroPilot
MicroPilot has successfully integrated FLARM’s SAA system with its autopilot, allowing for reliable, autonomous collision avoidance for a fully autonomous UAV [16]. The system alerts the autopilot of nearby aircraft, along with their velocity, altitude, and future trajectory. Using this information, the autopilot decides how to avoid the other aircraft, preventing a collision autonomously.
3.10 Autonomous Solutions, Inc.
Vantage is an obstacle DAA system that can detect and avoid potential hazards with an advanced suite of software and sensors [17]. Vantage dynamically plans the safest and most efficient pathway around obstacles identified through sensor data, with no operator approval necessary. Fuse sensor feedback with existing satellite imagery is used to create accurate terrain maps of indoor and outdoor environments. It uses a Forecast 3D Laser System, which turns raw LIDAR data into a detailed drivability map.
3.11 Aerobits
Aerobits offers two receivers for use in SAA/DAA systems:
- TIM-MC 1: This high-performance receiver series offers the possibility of receiving and decoding ADS-B and Mode-A/C/S for application in SAA/DAA systems.
- TIM-SC1: This high-quality and low-price ADS-B receiver series allows for high-speed, radio-frequency data processing with a significantly reduced number of electronic components. It can be applied to SAA/DAA systems and UAS collision-avoidance systems.
4.0 Organizations and Their SAA Projects for Further Research
The following organizations are participating in ongoing research in SAA systems:
- University of Zurich’s Robotics and Perception Group
- NASA
- Johns Hopkins University Applied Physics Laboratory (JHU APL)
- Carnegie Mellon University
- The Office of the Secretary of Defense (OSD)
- UAV Battlelab
- Air Force Research Laboratory (AFRL)
- The Office of Naval Research (ONR)
4.1 University of Zurich’s Robotics and Perception Group
The University of Zurich developed a drone that uses a camera and an onboard Visual-Inertial Odometry system to see an incoming ball and dodge out of the way by using a sensor called an event camera [18]. Rather than recording frames each second and passing them along to be analyzed, event cameras only send data when the pixels shift or spike in intensity. This capability reduces processing bottlenecks that restrict conventional SAA systems and results in faster response times.
4.2 NASA
NASA is conducting tests on SAA systems at the Armstrong Flight Research Center along with General Atomics Aeronautical Systems, Inc., Honeywell International, Inc., and the U.S. Federal Aviation Administration (FAA) [19]. The Environmental Research Aircraft and Sensor Technology (ERAST) program is also performing relevant research under NASA.
4.3 Carnegie Mellon University
Carnegie Mellon has developed prototypes and continues to research field-deployable SAA systems for UAVs using passive vision as the main sensing modality [20].
4.4 OSD
OSD has developed a Mid-Air Collision Assessment Tool (MARCAT) under the joint OSD-FAA airspace effort [21]. It is a software application used to quantify SAA parameters and provides data on collision occurrences, analyzes near misses, evaluates possible resolution maneuvers, and enables visualization of the parameters to view the multivariable problem more easily.
4.5 UAV Battlelab
UAV Battlelab has an Air Traffic Detection Sensor System (ATDSS) that uses passive, non-cooperative moving target detection (PMTD) technology developed by Defense Research Associates (DRA) in coordination with AFRL [21]. The system has the following features:
- Target-detection algorithms.
- High-resolution sensors.
- High-density, field-programmable gate arrays that provide the means to implement the algorithms while still meeting space, weight, and power constraints associated with UAVs.
4.6 AFRL
AFRL is supporting sensor development work in optical flow technology by its subcontractor AFRL/SNJW, particularly work on detecting noncooperative traffic [21].
The Air Vehicle Directorate (AFRL/VA) is supporting efforts to integrate SAA capability with flight-control software designed by Geneva Aerospace. They also are contributing to the Autonomous Flight Control Sensing Technology (AFCST) effort.
4.7 ONR
The Midair Collision Avoidance System (MCAS) is a software application that was developed to address SAA requirements. It integrates collision avoidance into existing flight-critical aircraft transponders [21].
References
[1] Zhahir, A., A. Razali, and M. R. Mohd Ajir. “Current Development of UAV Sense and Avoid System.” IOP Conference Series: Materials Science and Engineering, vol. 152, no. 1, https://iopscience.iop.org/article/10.1088/1757-899X/152/1/012035/pdf, 2016.
[2] Kemkemian, S., M. Nouvel-Fiani, P. Cornic, P. Le Bihan, and P. Garrec. “Radar Systems for ‘Sense and Avoid’ on UAV.” 2009 International Radar Conference “Surveillance for a Safer World” (RADAR 2009), https://ieeexplore.ieee.org/abstract/document/5438372,
25 March 2010.
[3] Ramasamy, S., R. Sabatini, and A. Gardi. “Avionics Sensor Fusion for Small Size Unmanned Aircraft Sense-and-Avoid.” 2014 IEEE Metrology for Aerospace (MetroAeroSpace), https://ieeexplore.ieee.org/abstract/document/6865933, 29 May 2014.
[4] Salazar, L. R., R. Sabatini, A. Gardi, and S. Ramasamy. “A Novel System for Non-Cooperative UAV Sense-and-Avoid.” European Navigation Conference 2013, Vienna, Austria, https://www.researchgate.net/publication/264742784_A_Novel_System_for_Non-Cooperative_UAV_Sense-and-Avoid, April 2013.
[5] Tooren, J. V., M. Heni, A. Knoll, and J. Beck. “Collision and Conflict Avoidance System for Autonomous Unmanned Air Vehicles (UAVs).” U.S. Patent 7737878B2, 15 June 2010.
[6] Aerospace and Defense Technology. “Airborne Sense-and-Avoid Radar for UAVs.” https://www.aerodefensetech.com/component/content/article/adt/features/insider/27354, 2 August 2017.
[7] NASA. “Safe2Ditch Technology.” https://technology.nasa.gov/patent/LAR-TOPS-243, accessed 18 June 2019.
[8] Finn, A., and S. Franklin. “Acoustic Sense & Avoid UAVs.” 2011 Seventh International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Adelaide, SA, Australia, https://ieeexplore.ieee.org/document/6146555, 9 February 2012.
[9] Phillips, A. “Sense and Avoid for Drones is No Easy Feat.” https://dronelife.com/2016/09/20/sense-and-avoid-for-drones-is-no-easy-feat/, 20 September 2016.
[10] PrecisionHawk. “Technology.” https://www.precisionhawk.com/technology, accessed 18 June 2019.
[11] Rouffineau, T. “Welcoming the Parrot S.L.A.M.dunk: The New Drone Development Kit.” https://blog.ubuntu.com/2016/09/07/welcoming-the-parrot-s-l-a-m-dunk-the-new-drone-development-kit, 7 September 2016.
[12] Reagan, J. “InterDrone 2016: Neurala Brings Deep Learning to Drones.” https://dronelife.com/2016/09/12/interdrone-2016-neurala-brings-deep-learning-drones/, 12 September 2016.
[13] Embention. “Veronte Autopilots,” https://products.embention.com/veronte/uav-autopilot, accessed 18 June 2019.
[14] Vigilant Aerospace Systems. “New Global UAV Sense-and-Avoid Systems Report Names Vigilant Aerospace a Key Vendor and Forecasts Market at $3.6Bn by 2022.” https://vigilantaerospace.com/new-global-uav-sense-and-avoid-systems-report-names-vigilant-aerospace-a-key-vendor-and-forecasts-market-at-3-6bn-by-2022/, 18 April 2018.
[15] Near Earth Autonomy. “Technologies.” https://www.nearearth.aero/technology, accessed 17 June 2019.
[16] FLARM. “MicroPilot and FLARM’s Autonomous Sense & Avoid System.” https://flarm.com/micropilot-and-flarms-autonomous-sense-avoid-system/, 3 November 2017.
[17] Autonomous Solutions, Inc. “Improve Vehicle Safety with Vantage.” https://www.asirobots.com/platforms/vantage/?tracking=Products+Page, accessed 12 June 2019.
[18] Murison, M. “Zurich Researchers Teach Drone to Dodge with Dynamic Obstacle Avoidance.” https://dronelife.com/2019/05/16/zurich-researchers-teach-drone-to-dodge-with-dynamic-obstacle-avoidance/, 19 May 2019.
[19] Drone Life. “NASA to Test Sense-and-Avoid Technology for Drones.” https://dronelife.com/2015/06/29/nasa-to-test-sense-and-avoid-technology-for-drones/, 29 June 2015.
[20] Geyer, C., D. Dey, and S. Singh. “Prototype Sense-and-Avoid System for UAVs,” Carnegie Mellon University Research Showcase, Pittsburgh, Pennsylvania, https://pdfs.semanticscholar.org/71ce/58ed8e85e8998bdfefecb61f1460ec51f5b2.pdf, February 2009.
[21] Office of the Secretary of Defense for Acquisition Technology and Logistics. “Airspace Integration Plan for Unmanned Aviation.” Distribution A, DTIC Accession Number: ADA431348, November 2004.