An effective MLOps workflow developed within the Datalab team
Waternet
AI in the process industry
We start with one plant and the process and lab data you already have, then deploy the model to production, as we did for Waternet.
Nitrous oxide and ferric chloride: developing models for Waternet to reduce emissions and consumption
As part of Waternet’s Datalab team, we set up and maintain the MLOps workflow, from the model registry through to deployment in production. We developed a model to minimise nitrous oxide emissions from wastewater treatment. Another model, designed to reduce ferric chloride use in the pretreatment of river water for drinking water, is in development.
Why is the operator still relying on this morning's lab sample to adjust the process?
Product quality is measured in the lab a few times per shift, while temperature, pressure and flow rate are recorded every second in the historian. A soft sensor estimates quality between lab tests using this process data, so the operator can make adjustments sooner.
Process models that run alongside the operator and are monitored after go-live
Emissions from the treatment plant
As part of Waternet’s Datalab team, we developed a machine learning model to minimise nitrous oxide emissions at the wastewater treatment plant.
Condition-based maintenance
For Fokker Services, we combined maintenance history, sensor data and operational data into a model that shows when maintenance is actually needed.
One platform for all data sources
For Fokker Services Group, we are bringing fragmented data sources together in a data platform based on the medallion architecture, with integrated data quality checks.
Monitored for drift
Processes change, and models become outdated. For Rijkswaterstaat, we built a package that monitors models in production for drift and outliers.
We start with one control loop or piece of equipment, then move on to the rest of the historian
-
Decision and measurement points
Which setting, dose or maintenance decision needs improving, and which tags, laboratory results and fault reports are relevant?
-
Prototype model using historical data
We train the model on earlier production periods, test it on a later period and compare it with laboratory results and the current way of working.
-
Advice where the operator looks
The estimate appears on the control screen, with drift monitoring in place. The operator remains in control of the equipment.
Is our process data sufficient for a soft sensor or a model?
A good fit if
You have historical data in a historian or SCADA, timestamped lab results and a recurring decision, such as a setpoint, a dosage, when to clean or when to replace a pump.
Not a good fit if
Sensors have been replaced or recalibrated without anyone recording it, the installation runs a different recipe each time, or a well-configured controller already solves the problem. In those cases, we start with the data or the control system.
What determines the scope
The number of installations and tags, whether lab results and fault reports can be linked, how the OT environment allows data to be exported, whether the work stops at a trial model, and who will manage the model afterwards.
Can a model control the process?
We build it to provide recommendations alongside the process control system. If you want to automate control later, first establish who will supervise it, the limits within which the model may act and how to fall back on the existing control system.
Soft sensor, process optimisation or maintenance model: where can data deliver value first?
Soft sensor
Choose this if a quality measurement only comes back from the lab hours later. We recommend starting here: process and lab data are usually already available, and the results are straightforward to assess.
Process optimisation
Choose this if you know which outcome needs to improve, such as energy use per tonne or the consumption of auxiliary materials, and the installation runs steadily enough to compare settings.
Maintenance model
Choose this if faults in pumps, compressors or heat exchangers halt production and you have a history of those faults. See predictive maintenance.
From model to production: we set up the MLOps workflow at Waternet
Data Science Lab has worked with data and AI from Amsterdam since 2016, as a Databricks and Microsoft partner. As part of Waternet’s Datalab team, we set up the MLOps workflow that brings code and models into production.
One installation and an export from the historian are enough for an initial conversation
Stef Stoelwinder
Business DevelopmentStef is your first point of contact. In our first conversation, we’ll discuss which installation to start with and what process and lab data you have.
E-mail info@datasciencelab.nl
Phone +31 (0)20 3636 724
Which process variable would you like to know sooner or more accurately?
Describe the installation, your question and the data you already have. We’ll reply by email or arrange an initial conversation.
Message sent
Thanks for your message. We’ll be in touch as soon as possible.
Frequently asked questions
In the process industry, you make a product through a continuous process or in batches, as in food, chemicals, paper, drinking water and energy. AI helps in three areas: estimating a value that you currently measure only in the lab, choosing better settings and spotting faults earlier.
The starting point is almost always the process data you already have. These are among the AI applications we build. For power generation and grids, read more about data and AI for the energy sector.
A soft sensor is a model that estimates a process variable that is difficult to measure, such as concentration or viscosity, from measurements that arrive continuously, such as temperature, pressure and flow rate. A review by Kadlec, Gabrys and Strandt (Computers and Chemical Engineering, 2009) describes soft sensors as a valuable alternative for measuring, monitoring and controlling critical process variables.
The pitfall is timing: a lab result belongs to the moment the sample was taken, not the moment the result was entered. Mix those up, and you train a model on the wrong process state.
Usually, you need tags from the historian, populated by the process control system or SCADA, lab results from the LIMS, and faults and work orders from the maintenance system. The ISA-95 model divides a company into layers, from equipment on the factory floor to ERP, with MES and SCADA in between.
We bring data from those layers together on a data and AI platform, often through the historian or a connection using OPC UA, the platform-independent standard for industrial data exchange.
The OT environment — the networks and systems that control the plant — is usually separated from the office network, and it should stay that way. The ISA/IEC 62443 series sets out requirements and processes for keeping industrial automation and control systems digitally secure.
That is why we do not read data directly from the control system. Instead, we use a copy of the historian or a connection approved by your OT management team. A model that only provides advice does not need to write back to the control system.
In a continuous process, a single pump, compressor or heat exchanger can sometimes stop the entire line. A maintenance model tracks signals such as vibration, temperature and pressure drop, and learns from fault history which patterns precede a failure. In a heat exchanger, you can often spot fouling through a gradually increasing pressure drop.
For Fokker Services, we combined maintenance data, sensor data and operational data into a model that shows when maintenance is actually needed. Read more about the approach under predictive maintenance.
That depends on what the model does. The AI Act classifies AI used as a safety component in the supply of water, gas, heating or electricity as high-risk. A soft sensor that advises an operator is different from a model that operates a gate or valve itself.
Your legal counsel or compliance department must assess whether your application is high-risk. We help you take stock and prepare the risk classification; you can find the timeline under AI Act.
A different raw material, a replaced sensor or a new cleaning cycle shifts the data. A model that does not detect the change can start making inaccurate estimates without anyone noticing. That is why you need to monitor models in production. For Rijkswaterstaat, we built a general-purpose package for detecting drift and outliers.
For Waternet, we set up an MLOps workflow with an MLflow model registry and CI/CD templates, giving code and models a defined route into production. Read more about MLOps and management and continuous improvement.
When a properly configured controller or an additional lab measurement already solves the problem, or when you cannot link historical data to lab results and faults. In that case, you start with the controls, the data or a Power BI dashboard — and we will tell you so.
If you make discrete products on a production line rather than a product through a process, the questions are different. In that case, see the AI assessment for manufacturing.
Applications, maintenance or the energy sector: what would you like to read about next?
- AI applicationsOur AI applications at a glance, from data science to business intelligence.
- Predictive maintenanceHow asset data helps you schedule maintenance sooner and more precisely than a fixed calendar would.
- Data and AI for the energy sectorHow we bring information from across the chain together so planners can spot anomalies sooner.