Get a recommendation system built

A recommendation system, also known as a recommendation engine or recommender system, selects what is likely to be relevant to each user from a large range of options: a product, an article, an event or a job opening. It learns from past behaviour, such as purchases, clicks and viewing time, as well as from the characteristics of the available items.

Product recommendations in an online shop are the best-known example. Personalised recommendations in a newsletter or app work in the same way. This is one of the AI applications we build.

Collaborative filtering finds customers with similar purchasing behaviour and recommends things they bought that you have not. It does not need a customer profile. Content-based filtering looks at the characteristics of things someone has chosen before, such as category, brand or genre.

A hybrid recommendation system combines both approaches, as our Recommend.ai does. Large systems often work in three steps: selecting candidates from the full range, scoring them and then reordering the list, for example to add variety or include new items.

This is called the cold-start problem: a new customer or product has no history yet, so collaborative filtering has nothing to learn from. In such cases, the ALS model in Apache Spark, the processing engine behind Databricks, deliberately makes no prediction, leaving you to choose a fallback.

A common approach is to combine methods: new visitors see popular or seasonal products, while new products are recommended based on characteristics such as category and price range. Once clicks and purchases come in, the model takes over.

You need interactions you can link to a customer or session, such as orders, clicks, views or ticket purchases, plus a product catalogue with characteristics and availability. This is usually implicit feedback: a click or purchase, rather than a star rating. Models such as ALS have a variant designed for this.

In a shop without a loyalty card, you cannot link a purchase to an individual, but you can still run a market basket analysis on receipts. If your data is spread across systems, start with the data foundation.

Start offline: train on older data and check whether the model predicts later purchases better than a simple baseline, such as a bestseller list. Common metrics include precision at k, the proportion of the first k recommendations that are relevant, and NDCG, which also takes their order into account.

Then test live, using a comparison group that continues to see the old selection. Agree in advance whether you are optimising for clicks, purchases or margin. Lots of clicks on cheap products are not always what you want.

A model that relies only on past behaviour reinforces what is already popular and overlooks new or less familiar items. That is why you reorder the list: limit the number of products from any one category, give new items a chance and exclude things someone has already bought.

Business rules belong in the same step, such as not showing out-of-stock products. After launch, you retrain the model so it takes new products and seasons into account. You can read about the technical side under MLOps.

Making recommendations based on someone’s behaviour is profiling. The European Data Protection Board describes this as the automated analysis of personal data to assess personal aspects, such as behaviour, preferences and interests. You need a purpose and a legal basis under the GDPR, and customers can object to direct marketing.

The Digital Services Act requires online platforms to be transparent about how their recommendation systems work. The largest platforms must also offer a feed that is not personalised. Your legal adviser or privacy officer can assess whether this applies to your organisation. Together, we determine which data is genuinely necessary.

When your product range is small, customers rarely return or purchases cannot be linked to a customer or session. In those cases, a fixed bestseller list does much the same job with less maintenance. If you first want to understand who your customers are and what value they bring, start with customer analysis.

If your product data is messy, with missing categories or duplicate items, address that first. If an off-the-shelf approach fits, we will point you towards Recommend.ai. If you are unsure where the problem lies, start with a maturity assessment.