This project aims at helping the current program of bear re-introduction in the Pyrénées mountains, by developping an automatic bear detector that will trigger some scaring device, in order to protect sheep from bear attacks.
Sample of images taken by one of the camera trap set by the OFB.
Foreword
Governmental organizations started reintroducing the brown bear in the French Pyrénées in 1996. The population remains very fragile as bears are targetted by hunters and are subject to allegations that they cause a lot of damage to mountain pastures, for example in the case of a fall of a hundred sheeps in 2016. Damage caused by bears is indeed estimated to 1200 sheep per year and only on the french side of the Pyrénées. The french government has voted a vast program for the defense of the brown bear starting from 2018 up to 2028 to help work on different axes for the brown bear defense. One of them is to help protect pastures with shepherds as mentioned on their website.
It also involves research funding for “automatic shepherds” which consist of triggering bear scaring devices when the approach of a bear is detected. Actors on this subject are the DREAL which is the french bureau for industrial and natural risks, the OFB and the DDT, who are both responsible for biodiversity conservation and actual developments of departments in the Pyrénées.
Why is Cerfacs interested in a project like this ? Because it is not only applied research with Deep Learning on High power computing devices, it is also a challenge to embark it in-situ and answer a lot of challenges:
- to be able to run a deep learning library on a portable device,
- find the energy resource to make it last at least a month in a difficult environment,
- adapt the deep learning detection to the constraints of the environment such as rain or fog during the night.
Objective
The DREAL provided us pictures taken on the field of all the wildlife that can be encountered in the mountains. These pictures come from the resource project Ours brun, a network of 450 people that collect information on wildlife. The objective of the project is to train a neural network to recognize a bear amongst all of the other animals that are likely to be passing by. Once trained, the neural network will be exported on a portable computer that will be connected to a camera trap. When an animal is passing by, a picture will be taken and analyzed by the neural network (this is called inference). The inference result, bear or not bear will trigger a frightening device when necessary.
The choice of deep learning
Deep learning and convolutional neural networks (CNN) are very effective in image treatment. Deep learning is called deep because it uses networks that pile layers, each layer learning a different feature, more and more complex as the output of the network is reached. The CNN will analyse the picture and perform a convolution operation on it. This will reduce the size of the image whilst enhancing top features such as fur patterns, eyes or ears in the last layers in the neural network. Since eyes, ears of animals have already been an object of work for several existing CNNs that exist online, it is possible to use pre-trained networks that will avoid our neural network the difficult task to have to learn again to recognize an eye or an ear. This is called transfer-learning. This leads us to the workflow below that represents a classical deep learning project.
Deep learning architecture
The CNN will be trained on the images that have been labeled by experts and classified into species. The CNN will be using pre-trained weights coming from ImageNet. It means that we don’t have to train all the layers from our neural network but only the one that leads to our final output (which is bear or not). The pictures that the network will be trained on are divided into a training set and a validation set. For these two groups, the labels are known. This is called supervised learning. At each epoch we will have trained the network on all the batches of pictures of the training set, the neural network will be tested for accuracy on the validation set, and each time some correction into the weights of the neural network will be done thanks to adjusting the performance given by comparison to the labels of the validation set.
Deep learning library chosen for the project
Fastai is an opensource API developed by Jeremy Howard on PyTorch. It is very well adapted to image classification.
For the network architecture we will choose a classical CNN with a ResNet of 18 layers.
How to develop a neural network able to recognize a bear
The data and the means for training the network
The DREAL provided several thousands hundreds of data that need to be organized and read by the neural network. Since the data available is taking a huge amount of memory, it is necessary to run on GPU nodes, the network will be trained on a Nvidia Tesla V100 card on the cluster Kraken owned by Cerfacs.
The neural network
A binary classifier is going to detect bears versus a class “other” that gathers all the remaining species, including humans and pictures where there is nothing.
Sample of images labeled as bear or other.
How to validate the model
Result on a validation set
The performance of the network can be visualized thanks to a confusion matrix which shows well classified pictures on the diagonal and failures elsewhere. In this particular case 4 bears were mistaken for another, and 1 other was mistaken for a bear and would be frightened.
Failures
The top most miscategorized images can be looked at to improve the reliability of the network, it can be seen that falsely labeled images are images where there is no animal to be found. This is due to either: - badly classified images initially - the choice of setup for the camera trap that takes several images when a motion is detected with a time delay, which leads to a picture taken each time at the end without any animal. If they are labeled, it can induce a high error rate in the network training.
Example of images badly labeled by the neural network
Further fine training focused training on a cleaned and restricted database of 3242 images, among them 1258 bears. The pictures were cleaned from pictures from the year 2019, and the bears were filled in with more pictures from the year 2020. Hence, when the network will be tested on other years before 2019, it will be for sure the first time that the network sees the pictures.
Visualization of detection
The vizualisation shows on a heatmap superposed on the image which pixels on the image are lit on, which answers the question: 1- what part of the image made the network say that this is a bear or another species.
These three images show a bear in the three different positions, it was primordial for us to check that the tree is not the reason why the network is identifying to label the images as bear. We can see that the bear is identified by the shape of its paw of the fur texture.
How to export the model
Now that the model is capable of identifying a bear it has to be used in the wild, at real time. The model can be exported and installed on a portable computer, such as a Rapsberry Pi 4 which has been chosen for this project:
The Raspberry Pi 4 is a tiny computer with 4Gb of RAM, with low consumption and easy to carry anywhere.
As shown on the diagram below, Once a camera is connected to it, a program can be installed to get the camera running and take a picture when an animal is passing by. The image is sent to the bear detection software, and depending on the outcome, a frightening device will be triggered.
This project is on-going and in currently in the hardware compatibility assessment phase.
The author wishes to thank Henri Toulotte, Bear protection project manager at DREAL, Sébastien Forest, regional director at DREAL, Corentin Lapeyre, data-science expert, the OFB for providing the pictures, and Isabelle d’Ast for technical support for libraries installation on the CERFACS clusters.
