Computer vision on a farm?

The Floating Farm in Rotterdam is the world’s first floating farm. Its priorities are animal welfare, circularity, sustainability and innovation. We wanted to explore what computer vision could do on a farm. The aim? To understand the feeding process so we can optimise it and ultimately prevent feed waste.  

Getting started: collecting data

To get started with computer vision, we needed images. We began collecting photos of the cows’ feeding areas. We installed Hikvision cameras because they integrate well with Python. What makes this project unusual is the use of two different camera angles. This allowed us to collect data from two perspectives, giving our computer vision model an extra challenge from the outset.  

The two different camera angles

Time to classify

Now that we’re collecting images, it’s time to classify them, a.k.a. image classification. But what is image classification? It teaches computers to sort images into different categories. This uses machine learning techniques, such as deep learning, to recognise patterns and features. The trained model can then analyse new, unfamiliar images and predict which category they belong to.

For the techies among us. I can hear you thinking… Did you consider masking techniques? Yes, we did, but image classification was better suited to the complex environment of a floating farm. It was also more efficient and robust.

In this case, the model needs to distinguish reliably between three categories.

1. Sufficient feed
2. Insufficient feed
3. Feed needs to be swept back

With this classification, the computer can analyse images of the cows’ feeding areas and determine what needs to happen.

If there is enough feed, the image is classified as sufficient feed. No action is needed. If there is too little feed or none at all, it is classified as insufficient feed. More feed is needed. If there is plenty of feed but the cows cannot reach it, the image is classified as needing the feed to be swept closer.

Data labelling

Every photo collected must be labelled to identify the three categories: sufficient feed, insufficient feed and feed needs to be topped up. This labelling process is essential for training the image classification model. It must be done accurately and consistently to ensure the model is reliable.

Training the models

We explored several machine learning models to do this as accurately as possible. The CNN model was the best fit. Our data comes from two different camera angles, so we trained three models separately: one for each camera angle and a third that combines data from both angles.

Architecture of a CNN

Measuring performance

Throughout the process, we tracked metrics to assess the model’s performance. We looked at how often it was right and how often it got things wrong (measuring accuracy, precision, recall and F1-score), all to test how effectively the model identifies feed status.

Does computer vision have a place on the farm?

The Floating Farm is an example of how innovation and technology can help improve sustainability, efficiency and productivity in agriculture. Using image classification is not only a way to manage feed for cows; it also opens up new possibilities for the agricultural sector.

All three models achieve an average accuracy of over 85% when classifying feed status. Notably, the models trained on data from individual camera angles perform slightly better than the combined model.

Here are some test examples on which the combined model made predictions. The model’s confidence percentage appears below each image.

99.99% sufficient feed

100% insufficient feed

99.46% feed needs to be pushed closer to the cows

This approach could bring many benefits to modern farming. It makes the process much more efficient, reduces feed waste and cuts down on manual work. More importantly, it ensures feed is always available, supporting the cows’ health and productivity. It’s a step forward that shows how smart technology can help the agricultural sector. In other words, computer vision on a farm? Absolutely.

Do you have an idea that could help the agricultural sector move forward? Get in touch and we’ll be happy to explore it with you.