The Floating Farm in Rotterdam is the world’s first floating farm. Its priorities are animal welfare, circularity, sustainability and innovation. Once interesting for us to explore what you can do with computer vision on a farm. The goal? To gain insight into the feeding process in order to optimize the process and ultimately prevent wastage of feed.
Getting started, start collecting data
To get started with computer vision, we need images. We started by collecting pictures of the feeding areas for the cows. We installed Hikvision cameras because this fits well with the integration with Python. What is special about this project is the use of two different camera angles. This allowed us to collect data from two perspectives, immediately adding an extra challenge to our computer vision model.
The two different camera angles
Time to classify
Now that we are collecting images, it is time to start classifying the images, a.k.a. image classification. But what is image classification? With image classification, computers learn to place images into different categories. This is done with machine learning, such as deep learning, which can recognize patterns and features. The trained model can thus analyze new unknown images and then predict which category they belong to.
For the techies among us. I hear you thinking… Have you thought about masking techniques yet? Yes, definitely, only image classification was better suited to the complex environment of a floating farm. In addition, it was more efficient and robust.
In this case, the model must be able to properly distinguish three categories.
- Sufficient feed
- Insufficient feed
- And must be swept up
The classification allows the computer to analyze images of feeding areas for the cows and assess what to do.
If there is enough feed, it is characterized as sufficient feed. No action is required. If there is too little or no feed, it is recognized as insufficient feed. Feed needs to be added. If there is plenty of feed, but the cows cannot reach it, it is characterized as needing to be swept.
Data labeling
Each image collected should be labeled to identify the three categories sufficient feed, insufficient feed and there is need to supplement. This labeling process is essential for training the image classification model. It must be done very accurately and consistently for the reliability of the model.
Training models
We investigated different machine learning models to do this as accurately as possible. The CNN model was the best fit. Our data consists of two different camera angles and because of this we trained three models separately. One for each camera angle and the third model combines the data from both angles.
Architecture of a CNN
To measure is to know
Throughout the process, we tracked everything in metrics to assess the presentations of the model. We looked at things like how often it was right, how many times it went wrong (accuracy, precision, recall and F1 score were measured), all to test how effective the model is in identifying feed status.
Is computer vision something for the farm?
The Floating Farm is a perfect example of how innovation and technology can be used to drive sustainability, efficiency and productivity in agriculture. Using image classification is not only a smart way to manage feed for cows, but it also opens the door to a new world of possibilities for the agricultural sector.
On average, all three models perform above 85% accuracy in classifying feed status. Interestingly, the models trained with data from separate camera angles perform slightly better than the combined model.
Here are some test examples on which the combined model made predictions. Below each image is the model’s certainty percentage.
99.99% sufficient feed
100% insufficient feed
99.46% must be swept up
This innovative approach can have many benefits for modern agriculture. It makes the process much more efficient, reduces feed waste and manual labor is reduced here. More importantly, it ensures that feed is always available, which promotes cow health and productivity. A great step in which smart technology advances the agricultural sector. Or computer vision on a farm? Absolutely.
Do you have a great idea to help advance the agricultural sector? Contact us and we will be happy to think with you.
