Model

This detection model is a deep learning neural network (Fig. ​(Fig.13),13), more specifically an encoder with a VGG16-like structure³⁵, that given an image (input image) return the probability of presence of Pleroma trees with flowers in the image. The model inputs are 4 bands RGB-NIR images made up of 136 × 136 pixels at 10 m of spatial resolution (Fig. 13). Sentinel-2 tiles of 10240 × 10240 pixels were cropped based on a regular grid of 128 × 128 pixels, and 4 neighbouring pixels were added on each side to create an overlap between the patches. The resulting images are 136 × 136 pixels in size. However, in the training, the presence or absence of blooming Pleroma was given only for the images of 128 × 128 pixels without consideration of the borders. This enable to avoidance of the border effect that is common in convolutional neural networks. Each image of 136 × 136 pixels goes through a data augmentation process that consists in random vertical and horizontal flips. No additional data augmentation necessary due to the natural data augmentation provided by atmospheric conditions and illumination. After data augmentation, the images were then fed to the detection encoder. The encoder was made up of 5 consecutive convolution and pooling blocks, one fully connected layer (dense 100) and a final output layer with a softmax activation that provided the probability of presence of blooming Pleroma trees in the image (Fig. 13). Additionally, one drop-out layer was used at the end of the fully connected layer to perform further implicit data augmentation and avoid overfitting during training. The model has a total of 25,448,686 parameters, of which 25,440,622 are trainable. The model was coded in R language⁵⁵ with Rstudio interface to Keras and TensorFlow 2.2^56–59.

Architecture of the Pleroma blossom detection model.

To make the training sample, a manual sample was produced for the Sentinel-2 tile 23KMQ, in the area where we had previously made a high resolution map of blooming Pleroma⁶, and for five other tiles where flowering Pleroma were detected visually from high resolution Google Earth images (22JFQ, 22JGQ, 23KLP, 23KLQ and 23KNQ, respectively). What is identified in the Sentinel-2 images are forest stands dominated by Pleroma and not single individuals. Pleroma trees have a small stature (8–12 m height) and crown of less than 10 m and one tree alone cannot influence sufficiently the reflectance to be clearly detectable in Sentinel-2 images. However, they occur very frequently clumped together, a common behaviour of this pioneer Genus. These flowering Pleroma dominated forest stands were easy to identify visually in the Sentinel-2 images because they combined several very distinctive features. First, an intense magenta-to-deep-purple color, which is an unusual color for other land covers in this ecological domain. Second, these identified Pleroma pixels should be undoubtedly identified as forested pixels and have a green color outside the blooming season. Third, Pleroma dominated forests often formed continuous magenta-to-deep-purple patches of size ranging from some 10 m × 10 m pixels to more than thousands of pixels and the shape of the patches tend to present linear features, likely representing the border of the space that was colonized by the Pleroma trees. Fourth, individual crowns were not visible, and the texture of the patches was very smooth during the blooming season with sometimes some inclusions of tree crowns of green color. Finally, texture of the Pleroma dominated forest stands outside the blooming season shown a smooth green texture, more homogeneous and with less shade than other forests. The first sample was constituted of images of background and of blooming Pleroma dominated forest stands that were following the previously described criteria. From this sample, we train a first model and applied it to the complete time series of Sentinel-2. From the results of this model, we obtained a first map of Pleroma trees and were also able to identify the main detection errors of this model, mainly clouds and dirt roads proximity with some unidentified agriculture fields or sometimes Eucalyptus plantation. The results of this first model were checked visually to produce a second larger sample (which was used for the results presented in this study) made up of images containing blooming Pleroma dominant to monospecific forest stands, a set of background images without blooming Pleroma and images identified erroneously by the first model as containing blooming Pleroma. While a large majority of the detected pixels were undoubtedly forest stands dominated by Pleroma trees, some other isolated trees of the genus Handroanthus (Ipê in Brazil or Lapacho in Argentina) with pink flowers and large crowns covering several pixels of 10 m × 10 m were also detected and kept in the training sample. For these particular Handroanthus trees, crowns were visible during and sometimes also outside the blooming season, which was not the case for detected Pleroma dominated forest stands. Finally, as our model detected also large Handroanthus trees, we must acknowledge that other tree species with highly similar features could also potentially being detected.

The final training samples comprised a total of 158,612 images of 136 × 136 pixels. Among them, 35,541 contained blooming Pleroma trees and 123,071 images contained only background. Among the background images, there was nine different images types: images without blooming Pleroma, i.e., background such as other land covers, urban structures, water surfaces and agriculture and other land uses (57,007), images with forests containing Pleroma but outside the flowering period (23,427), images with clearly identified detection errors mainly located in the east of the São Paulo state (12,965), clouds and detection errors in clouds (10,991), images clearly identified as detection errors near the State of Bahia (9030), other detection errors over Atlantic Forest (5843), images of forests without Pleroma trees during the season of blooming (2170), images with identified detection errors in the northern part of Atlantic Forest (1126) and images with no data (512). Of these images, 80% (126,890) were used for training and 20% (31,722) used for validation.

During network training, we used a standard stochastic gradient descent optimization, a binary cross-entropy loss and the optimizer RMSprop⁶⁰ with a learning rate of 1e-4. We used the accuracy (i.e. the frequency with which the prediction matches the observed value) as the metrics of the model. However, due to the imbalance between the number of blooming Pleroma and background images, the metric of the model was weighted by one for the background and, for the Pleroma, by the ratio between the number of background images and the number of images containing blooming Pleroma: that is, ~3.5. The network was trained for 5000 epochs, where each epoch was made of 61 batches with 2048 images per batch and the model with the best weighted accuracy was kept for prediction (epoch 4331 and weighted accuracy of 99.58%). The training of the models took approximately 9 hours using a Nvidia RTX2080 Graphics Processing Unit (GPU) with an 8 GB memory.

To avoid border effects, each 10240 × 10240 pixels Sentinel-2 image was cropped on a regular grid of 128 × 128 pixels (1280 × 1280 m), and 4 neighboring pixels were added on each side to create an overlap between the patches. The function gdal_retile⁶¹ was used for this operation. Prediction was then made for each subset image: for each image, the detection model returned 0 or 1 if a blooming Pleroma was found in the image. Then the results were spatialized again using the grid, but this time, each cell of the grid only received 1 value, the prediction, resulting in a raster of 80 columns and 80 rows and a spatial resolution of 1280 m, of the same extent as the Sentinel-2 image. The value of the pixels (1 or 0) indicated the presence or absence of blooming Pleroma trees in this squared area of 1280 m of side. Prediction using GPU of a single tile of 10240 × 10240 pixels took approximately 1 minute on a Nvidia GTX1080 with an 8 GB memory and 45 s on a Nvidia RTX2080 with an 8 GB memory. The prediction for the complete Sentinel-2 time series presented in this work took approximately 22 days using a Nvidia GTX1080 GPU—from the 30 October 2020 to the 20 November 2020.