NLP and text analysis

Here, NLP stands for natural language processing: techniques that enable a computer to process language. It does not refer to neurolinguistic programming used in coaching. Text mining and text analysis are applications of NLP: they help you find out what large volumes of text are about, how people experience something, and which names or concepts appear in them.

For us, NLP falls under data science, one of our AI applications.

Computers work with numbers, so the text must first be converted. Embeddings turn a word or sentence into a sequence of numbers that represents its meaning, placing ‘animal’ and ‘creature’ close together. You can then train a classification model using examples for each topic, or let a topic model such as BERTopic find groups on its own.

For WUA, we combined NLP, embeddings and machine learning in a topic classification model. You can read how embeddings work in our blog post The power of embeddings.

Preferably, the platform where your data already resides. In Databricks, you can use AI Functions in SQL, such as ai_classify to classify text, ai_analyze_sentiment to analyse sentiment and ai_mask to mask data. Microsoft’s Azure Language offers, among other things, PII detection and named entity recognition, which identifies names and concepts.

We build our own models in Python, for example using the Dutch language model RobBERT. As a Databricks and Microsoft partner, we choose what fits your platform and how you manage it.

By testing it on texts that people have already classified and that the model has never seen. For each topic, you look at precision: when the model selects that topic, how often is it right? And at recall: how many of the texts about that topic does it find?

Ask two colleagues to classify the same set as well. Where they disagree, the categories are not yet clear enough for a model. Assess a topic with few examples separately, because an average across all topics will hide its performance.

Open-ended text may contain more than you asked for: a name, a customer number or something about someone’s health. According to the Autoriteit Persoonsgegevens, customer and employee numbers are also personal data, and processing health data is prohibited unless a legal exception applies.

We identify and mask such data before the text goes into the model, and document where processing takes place. Your privacy officer or data protection officer assesses whether the processing is permitted.

That depends on what decisions the output helps to make. The European Commission cites software that sorts CVs during recruitment as an example of a high-risk application. Following the AI omnibus, the stricter rules for these applications apply from 2 December 2027. Classifying complaints for a report is different from selecting job applicants.

A text model can also pick up biases from its training data, as we describe in our blog post on gender data bias. Legal interpretation is up to your legal counsel or compliance department. We can help you take stock of applications, prepare the risk classification and establish AI governance.

Text analysis looks at many texts at once: which topics recur and how sentiment changes. If you need to extract fields from a single document, such as an invoice amount, document processing with AI is a better fit.

If you want employees to ask questions about internal documents, that falls under generative AI, as with Anthony Chat for Fokker Services Group. If AI needs to carry out steps in a process itself, read about agentic AI.

When there is little text, or when a selection list solves the problem: let people choose a category themselves and keep the open field for additional explanation. A model also adds nothing if nobody acts on its output.

Sometimes a dashboard based on existing selection fields is enough. If you are unsure where the problem lies — in the data, the processes or the organisation — start with a maturity assessment.