Email classification with AI

Email classification automatically sorts incoming messages. A model assigns each message one or more labels, such as complaint, invoice or fault, and sends it to the right team. It often also extracts information from the text and determines which emails take priority. With AI, this goes beyond rules based on the sender or subject line.

Routing email with AI is one of the AI applications we build.

From examples: emails your team has already labelled. The model learns which words and sentences belong to which category. Microsoft describes two approaches for Azure Language: one label per text, or multiple labels at once. An email about both an invoice and a complaint would get both labels.

The quality of the labelled examples matters more than the choice of algorithm. For Olympia, we built an automated classification model to match candidates with suitable job openings.

Yes, if you define what should take priority. A fault, a cancellation or a complaint about an earlier complaint can go to the top of the queue. The model predicts the category; the order comes from rules your team chooses, sometimes supplemented with CRM data, such as the type of contract.

Keep the prioritisation explainable. A member of staff needs to be able to see why an email is at the top.

Usually a customer number, order number, invoice number, address or date. A model recognises these in free text, even when they are written in different ways. The system can then use that information to look up the customer or order in advance.

If the information is mainly in attachments, such as invoices or PDF forms, document processing with AI is a better fit.

For Microsoft 365, we use Microsoft Graph to read messages from a mailbox, assign them a category or move them to a folder. We connect a ticketing system or CRM through its API so that the label and information appear there.

Ideally, the model runs on the platform you already use, such as Azure or Databricks. As a Microsoft and Databricks partner, we choose an approach that fits how you manage your systems.

Before launch, we test the model on emails it has not seen, measuring two things for each category: how often the label is correct, and how many emails of that type it identifies. Together with your team, we choose a confidence threshold below which an email goes to a member of staff.

After launch, we track how often staff change a label. If that rises, for example because of a new product or a new type of complaint, we retrain the model using the new examples.

Email almost always contains personal data: a name, an email address, sometimes a customer number or information about someone’s health. That means the GDPR applies. Document where the email is processed, whether a language model outside your own environment has access to it, and how long the data is kept.

Your data protection officer should assess whether a DPIA is needed. You can read what that involves in DPIA for AI.

When you receive few emails, when the sender or subject line already makes the category clear, or when a web form with selection fields can replace email. In those cases, mailbox rules are enough.

If you want to analyse large volumes of text by topic and sentiment, read about NLP and text analysis. If you also want to answer questions, building an AI chatbot may be a better fit. Generating a draft reply for each email falls under generative AI.