Data engineering consultancy in Utrecht

It brings data from all your systems together reliably in one place, using pipelines and a platform that it designs, builds and maintains. Models in production need more: the same data for training and production, checks when data is ingested, and a defined model input. The models themselves fall under data science. Read more about data engineering.

If you would rather have a data engineer from Utrecht or the surrounding area on your own team, you can hire a data engineer. They remain employed by us.

Because the model relies on what it saw during training. If a source system is replaced, a unit changes from metres to centimetres, or a column takes on a different meaning, the model will still produce results, but they will be less accurate. No one gets an error message.

So put checks at the point of ingestion. Delta tables on Databricks enforce the schema when data is written: columns the table does not recognise are rejected, and data types must match. Schema changes then become deliberate. You can validate data arriving as a file or through an API in Python with Pydantic.

By storing the input and output of every prediction alongside the training data. On Databricks, inference tables do this: they record the requests and responses of a model serving endpoint in a Delta table in Unity Catalog. You can then use data profiling to track performance and data drift.

Rijkswaterstaat has several AI models in production. For these, we developed a generic package for drift and outlier detection, using tools including Alibi Detect, a Python library for outlier and drift detection. Read the Rijkswaterstaat case study or find out more about MLOps.

With lineage: a record of which queries and files populate a table, which jobs and notebooks modify it, and which dashboards and model versions use it. Unity Catalog records this automatically for queries on Databricks, down to column level. If the person responsible for a source wants to remove a column, you can see in advance who will be affected.

Without a catalogue like this, you can keep a list for each pipeline, but it quickly becomes outdated. Read more about setting up Unity Catalog.

For high-risk AI systems, Article 10 requires training, validation and test data to be subject to appropriate data governance and management practices. The datasets must be relevant, sufficiently representative, and as free of errors and complete as possible for their intended purpose. Pipelines that record where data comes from help you demonstrate this.

Under the AI Omnibus, the requirements for Annex III do not apply until 2 December 2027. We can help you take stock; your legal counsel or compliance department decides whether your model is high-risk. Read more about the AI Act.

That depends on your existing IT landscape. We build on Databricks and Microsoft Fabric. For Fokker Services Group, we are building a data platform using Databricks, Azure and dbt, among other tools, modelled on the medallion architecture. To see how bronze, silver and gold form a lakehouse together, read about building a data lakehouse.

If you mainly need to bring together structured sources and exports, and no models run on the platform, a data warehouse is often enough.

If a single report runs on a single system, or if the problem is not technical but that no one owns a data source. In that case, start by agreeing on data governance, not by building a pipeline. If you mainly need more people on your own team, bringing in a specialist makes more sense than starting a project.

If you would rather compare a few consultancies first, use the comparison of data engineering consultancies. If you are based in South Holland, read about our data engineering consultancy in Rotterdam. If you first need to decide which model to build and who will manage it, see AI consultant Utrecht.