2.4.1. FLIR ADAS Dataset

The FLIR Thermal Starter Dataset [47] was originally designed to supply a thermal image and a set of RGB images for training and validating neural networks for object detection. It provides thermal and RGB images simultaneously, making it optimal for applying pansharpening methods.

The dataset was acquired via a RGB and thermal camera mounted on a vehicle (car). It contains 14,452 annotated thermal images with 10,228 images sampled from short videos, and 4224 images from a continuous 144 s video. All videos were taken on streets and highways in Santa Barbara, CA, USA, under generally clear-sky conditions during both day and night.

Thermal images were acquired with a FLIR Tau2 (13 mm f/1.0, 45-degree horizontal field of view (HFOV) and a vertical field of view (VFOV) of 37 degrees). RGB images were acquired with a FLIR BlackFly at 1280 × 512 pixels (4–8 mm f/1.4–16 megapixel lens with the field of view (FOV) set to match Tau2). The cameras were 48 ± 2 mm apart in a single enclosure.

As both sensors were mounted on the same structure with different lenses and resolutions, a previous work of alignment is essential [48]. Image alignment (also known as image registration) is the technique of warping one image (or sometimes both images) to ensure the features in the two images line up perfectly so that both images show the same field. We calculated an affine transformation to resolve this by identifying clearly-distinguished common points in both images. The result of this transformation is that both images are now aligned in preparation for further pansharpening analysis.

Once the performance of the algorithms was confirmed, we obtained our own dataset with the requirements needed for our application, with a focus on aerial surveying.