Data engineering consultancy in Tilburg
We build data pipelines that follow a standard path from testing to production. That’s how we automated the release process for SNG.
For SNG, we automated the release process across the DTAP environments in Google Cloud Platform
The SNG DevOps team has fewer manual steps to take, and every release follows the same controlled process using Terraform and Kubernetes. We apply the same approach to data pipelines.
Why is the dashboard no longer accurate after the data changes?
A new column in the source, a quick fix in production, or a loading step that fails silently. Without a standard process and checks, the user is the first to notice. With the right setup, you spot it sooner.
Every change tested, deployed and reversible
A consistent release process
Development, testing, acceptance and production as separate environments, as we set up the release process for SNG.
Infrastructure as code
Storage, compute and permissions defined in Terraform or a similar tool, so you can rebuild an environment.
Checks at every loading step
Tests for record counts, empty fields and keys, with an alert if a loading step fails or runs late.
Validation before publication
For Zorginstituut Nederland, we developed a validation and publication pipeline in GitLab for controlled releases of KIK-V products.
We bring the pipeline that breaks most often into the deployment workflow first
-
Assessment
Which sources, loading steps and environments exist, who manages them and where things currently go wrong.
-
One pipeline in the deployment workflow
Version control, tests and automated deployment for the loading step with the most failures.
-
Expand and hand over
The remaining pipelines follow, with documentation and monitoring for your own team.
Is a data release pipeline right for a small team?
A good fit if
Several people work on the same pipelines, reports or models depend on them, and investigating a production error now takes a whole morning.
Not a good fit if
One person manages a single data source and a single report, and changes are rare. A simple backup with documentation is enough in that case.
What determines the scope
How many data sources and loading steps there are, which platform you use, whether version control is already in place and who will manage it afterwards.
Would we be tied to you?
No. Everything is documented and kept in your own version control and cloud environment, so your team or another consultancy can take it forward.
Manual deployment, a script for each pipeline or one workflow for everything?
Manual deployment
Choose this if little changes and one person knows how everything works. The risk is that knowledge and mistakes remain with that person.
A script for each pipeline
Choose this if you have a few loading steps that run independently. Each script then has its own approach to testing and deployment.
One workflow for everything
Choose this if more people work with the same data. Our advice: start with version control and tests for the most important loading step.
We built release workflows and pipelines for public sector organisations
Data Science Lab works from Westerdoksdijk in Amsterdam, including for organisations much further afield. We automated releases and publications for SNG and Zorginstituut Nederland. We are a Microsoft Partner and Databricks Partner.

The loading step you fix most often is the start of the first conversation
Stef Stoelwinder
Business DevelopmentStef is your point of contact. In the first conversation, we’ll discuss your sources, your pipelines and where they’re breaking now.
E-mail info@datasciencelab.nl
Phone +31 (0)20 3636 724
Which pipeline do you want to make reliable first?
Tell us which sources and platform you use. We’ll reply by email or arrange an initial conversation.
Message sent
Thanks for your message. We’ll be in touch as soon as possible.
Frequently asked questions
It builds and manages the pipelines that extract data from your sources, clean it and prepare it for reporting and models. It also makes sure those pipelines keep running reliably.
Read about how we approach this in data engineering.
It consists of separate environments for development, testing, acceptance and production. A change only goes into production once it has worked properly in the other environments.
For data, this also means having a test dataset that resembles the real data without exposing sensitive information.
Test the number of rows, required fields, unique keys and values within an expected range. Also check whether the data load finished on time.
With dbt, we define these tests alongside the models.
We work with Azure, Databricks, Microsoft Fabric and Google Cloud, as well as tools such as Terraform, GitLab and dbt. For SNG, we worked on Google Cloud Platform with Terraform and Kubernetes.
We choose what fits your existing environment.
Data Science Lab is based in Amsterdam. We come to you for the start of the project and key milestones, and work remotely with your team the rest of the time.
Much of the work takes place in your own environment, so this works well.
Yes. If you would prefer someone to build alongside your team for a while, read about hiring a data engineer.
They will set up the DTAP environments together with your team.
Then we build the foundation first, incorporating the DTAP setup from the start. That's easier than adding it later. If reporting is your main priority, take a look at our Power BI specialist for Tilburg.
For models built on that data, you can turn to our data science consultancy.
We hand over the work with documentation and monitoring, or we continue to manage the pipelines. If you later need models, read about our data science consultancy for Tilburg.
We agree on the approach in advance.