The Defense Systems Information Analysis Center (DSIAC) was asked to locate long-wave infrared (LWIR) imagery of threat small unmanned aerial vehicles (sUAVs) to use for testing models in development. Detecting sUAVs in flight can be difficult due to low visibility, night conditions, or clutter in the environment. Thermal imaging cameras and methodologies are currently being researched to increase the ability of IR imaging to detect sUAVs in a variety of conditions. DSIAC identified current research being conducted by the U.S. Army Research Laboratory (ARL) as well as researchers at the University of Zagreb, Croatia. ARL is producing imagery using LWIR thermal and corresponding polarimetric imagery, while the University of Zagreb research tested the applicability of a low-cost LWIR sensor for detection of various sUAVs in flight.
1.0 Introduction
The inquirer was particularly interested in the following: (1) airborne long-wave infrared (LWIR) imagery of threat sUAVs with weather conditions, (2) airborne LWIR imagery of threat sUAVs with a clear sky background, (3) ground-based LWIR imagery of threat sUAVs with background clutter, and (4) ground-based LWIR imagery of threat sUAVs with a clear sky.
The Defense Systems Information Analysis Center (DSIAC) searched open-source documents and the Defense Technical Information Center’s repository for publications containing information about IR imagery of threat sUAVs (i.e., UAVs weighing less than 55 pounds). This research is critical to the Department of Defense’s ability to counter foreign sUAV threats. Being able to detect sUAVs at a variety of ranges helps to increase the safety and security of the United States.
Detecting sUAVs in flight can be difficult due to low visibility, night conditions, or clutter in the environment. Thermal imaging cameras and methodologies are currently being researched to increase the ability of IR imaging to detect sUAVs in a variety of conditions.
2.0 U.S. Army Research Laboratory (ARL)
The U.S. Army Research Laboratory (ARL) is researching methodologies for producing imagery using LWIR thermal and corresponding polarimetric imagery [1]. This research helps evaluate the ability to distinguish between sUAVs and birds of similar size and dimensions.
Initial research has been limited to LWIR thermal imaging to be effective for both day and night imaging. Preliminary sets of calibrated radiometric and polarimetric images with both sUAVs and birds in the scene were developed, followed by targets (i.e., UAVs and/or birds) within each image being segmented. Regions of interest were generated, and pixel values were then averaged to yield a radiometric value (watts/cm2) and a polarimetric value to quantify similarities and differences between the sUAVs and birds. Because thermal polarimetric methods and analyses are currently being developed, the well-researched Stokes image methodology was used in ARL’s research. The Stokes methodology effectively yields the percentage of the overall radiance emitted/reflected from the object that is linearly polarized.
To measure the polarization state for emitted or reflected light, the Stokes parameters are measured (S0, S1, S2, and S3). These parameters are determined on a pixel-by-pixel basis to reconstruct a 2D image by measuring the intensity of radiance projected through a polarizer/wave-plate pair oriented at various angles to measure the intensity of a particular state [1]. There are multiple optical approaches and methods used in various polarimetric camera designs appropriate for polarization state filtering and analysis in the thermal IR. These approaches include 1) the division-of-time-approach, which uses a spinning-achromatic-retarder (SAR) arrangement; 2) a division-of-amplitude (DoA) in which a polarized beam-splitting-plate is used to project different polarization states of the same scene onto one, two, or more focal-plane-arrays (FPAs); and 3) the division-of-focal-plane (DoFP) design in which micropolarizers oriented at different angles are attached to individual pixels that make up the FPAs.
Figure 1 displays an example of an LWIR thermal image and its corresponding polarimetric DoFP image. In the conventional thermal image (left), the vehicle (man-made object) is difficult to discern from the background. The corresponding polarimetric image (right) clearly shows the vehicle. Producing a set of well-calibrated imagery, both conventional LWIR thermal (watts/cm 2-sr) and its corresponding polarimetric imagery (percent radiance polarized), helps distinguish between sUAVs and birds of similar size and dimensions [1].
Figure 1: Example of Conventional LWIR Thermal Image (Left) and Resultant LWIR Polarimetric Image (Right) in Which a Vehicle at Ambient Temperature Is Parked on a Hillside [1].
3.0 Using High-Definition Cameras for Airborne Reconnaissance
There are two types of thermal imaging cameras: LWIR and mid-wave IR (MWIR). LWIR cameras detect light in the 8−14-µm range, and MWIR cameras detect light in the 3−5-µm range [2]. MWIR cameras offer increased range and less disturbance by atmospheric conditions but typically require cooling systems that can be heavy, bulky, and expensive. Uncooled LWIR cameras are lighter, smaller, and cheaper than cooled MWIR or LWIR cameras, which makes them good candidates for use in airborne reconnaissance using drones.
Sierra-Olympic Technologies, Inc. has developed a high-definition (HD) thermal IR camera for aerial surveillance, the Vayu HD [2]. It features a vanadium oxide microbolometer sensor with a capacity of over 2.3 million pixels on a 12-micron pixel pitch. A hard-carbon coated lens allows the Vayu HD to be used outdoors, while its electronic image stabilization helps to compensate for any in-flight turbulence and produces high-quality images.
4.0 Nighttime Detection of sUAVs Using Thermal IR Cameras
Researchers at the University of Zagreb, Croatia tested the applicability of a low-cost LWIR sensor for detection of various sUAVs in flight. The research presented an analysis of thermal images obtained with FLIR’s Lepton microthermal camera that was mounted on a Raspberry Pi processing unit [3]. The imaging was tested on three different sUAVs: the DJI Phantom 4, the Parrot AR.drone 2, and a custom hexacopter. Specifications of the tested sUAVs are shown in Table 1.
Table 1: sUAV Specifications [3]
4.1 Testing
The test drones were flown over a 100-m-long test track to determine at what distance the sUAVs could be detected without trying to identify them [3]. The sUAVs were flown approximately 10 m above ground level at a steady velocity of 2 m/s. Testing was completed against a clear sky with no wind with grassy terrain, 2 hours after nautical twilight.
The LWIR sensor used in the study, Lepton, is produced by FLIR. The version used in this test had a horizontal field of view of 25°, with a maximum length of 11.7 mm and weighed .55 gm [3]. The spectral range is 8−14 μm with the best spectral response from 9.5 μm to 12.5 μm. With the sensor array consisting of only 80×60 elements, the spatial resolution is very low, with each sensor element covering 0.3125° horizontally. At the distance of 100 m, each pixel represented approximately 0.55 m of horizontal distance, which is more than the length of some of the sUAVs tested.
Figure 2 shows an increased graininess of the images on the right compared to those on the left. This graininess is due to the sensor automatically adjusting the range of brightness to show image features with higher contrast [3]. The lower part of the images on the right is shown as brighter than the same part of the images on the left. The brighter area is not the ground, but a layer of a higher temperature air near the ground.
Figure 2: Upscaled and Resampled Thermal Images of sUAVs at 5 m (Left) and at Limit of Detection (Right) for (a) Parrot AR.drone 2, (b) Phantom 4, and (c) Hexacopter [3].
4.2 Results
Results showed that small, electrically powered multirotor sUAVs can be detected with low-cost thermal sensors in some conditions. Due to noise, a human interpreter was necessary for detection. Researchers doubted that an algorithm could have outperformed the human interpreter in the given conditions [3].
The main source of heat was batteries, not motors. Motor visibility in the thermal spectrum was diminished due to being well cooled by rapid air circulation. Batteries were easily discernible in thermal imaging due to larger bulk and receiving moderate air circulation [3].
The maximum range of detection was shorter than calculations assumed. The limit of detection was on average 41 m for the Parrot AR.drone 2, 51 m for the Phantom 4, and beyond the 100-m line of the test track for the hexacopter [3]. Possible causes for the shorter distances could be the orientation of the sUAVs relative to the sensor and/or the cooling of external parts of the sUAVs due to rapid air circulation.
Future work will include 1) testing sUAVs against more diverse backgrounds, 2) more rigorous testing of sUAV detectability from unknown directions, and 3) testing methods for reducing the heat signature of tested sUAVs.
References
[1] Gurton, K. P. “Calibrated Long-Wave Infrared (LWIR) Thermal and Polarimetric Imagery of Small Unmanned Aerial Vehicles (UAVs) and Birds.” U.S. Army Research Laboratory, https://www.aerodefensetech.com/component/content/article/adt/tech-briefs/unmanned-vehicles-robotics/39046, 21 May 2021.
[2] Sierra-Olympic Technologies Inc. “Using HD Thermal Cameras for Airborne Reconnaissance.” https://sierraolympic.com/thermal-imaging-articles/solutions/using-hd-thermal-cameras-for-airborne-reconnaissance/, 16 September 2018.
[3] Andrasi, P., T. Radisic, M. Mustra, and J. Ivosevic. “Night-time Detection of UAVs Using Thermal Infrared Camera.” International Conference on Air Transport – INAIR 2017, Transportation Research Procedia 28, pp. 183–190, https://www.sciencedirect.com/science/article/pii/S2352146517311043, 2017.