How do you choose the right LLM for your organisation?

Large language models (LLMs) such as ChatGPT are popular, not only for personal use but also within organisations. Searching company documents for specific information, summarising large amounts of text and processing customer data more efficiently are some applications where it is easy to see their value.

But can you simply use LLMs within your organisation? What do they cost? And what happens to your data? LLMs come with risks, but these depend on many factors. An obvious starting point is which LLM you choose and how you configure it. This depends on the LLM and its configuration. In this blog, we explain the types of LLM available, how you can use your data safely with this technology, and provide an overview to help you choose the right LLM for your organisation.

LLM risks

Before we look at the different types of LLM, we’ll discuss two risks that organisations often raise with us. They frequently ask about data confidentiality (do storage and use meet GDPR requirements?) and harmful or incorrect output.

1. Data confidentiality

Negligent data protection can have costly consequences. Sensitive company data may no longer be stored within the organisation and could even become accessible to third parties. In 2023, we saw this negligence lead to data breaches at several companies, including Samsung. As a result, crucial trade secrets were leaked to OpenAI’s ChatGPT. Not only were company secrets exposed, but the European privacy regulation, GDPR (General Data Protection Regulation), was also breached.

1. Harmful or incorrect output

The unintended generation of harmful text can be a risk when using LLMs. Because of their training data, models may contain unconscious biases and stereotypes. This can lead them to produce text that is offensive, discriminatory or even hateful. Beyond the ethical concerns, harmful output can also have legal consequences. Companies may be held liable for harm caused by text they generate or distribute.

These risks can be challenging, but there is a solution to each of them.

Different types of LLMs

ChatGPT

ChatGPT is the online version of OpenAI’s LLM models. Its user-friendly interface makes it easy to use an LLM without a technical background. If you use ChatGPT without changing any settings, you agree to everything you enter being stored on OpenAI’s servers. If you share company secrets here, you breach GDPR rules because the data is stored in the United States (by OpenAI), rather than in Europe. The cost of using ChatGPT depends on how you choose to use it. You can pay through a monthly subscription or per use. The exact rate depends on the model you select and how intensively you use it.

OpenAI API

Another way to use OpenAI models, with more attention to privacy, is to use OpenAI’s Application Programming Interface (API) directly. This lets you interact with ChatGPT without going through a user interface (UI). The UI is geared towards simple interactions, while the API gives you more control. This is particularly useful for organisations that want to integrate OpenAI models into their own apps. The model generating the text is the same as in ChatGPT, but you use your own interface.

Since 1 March 2023, OpenAI’s API has been updated: “prompts” and responses are no longer used to improve the models. This means your documents stay confidential, without information leaking out. That is good news for protecting company information. To protect personal data, OpenAI offers an important option: you can ask OpenAI to amend its standard terms with its Data Processing Addendum. As with many large US tech companies, you cannot insist on your own data processing agreement and must accept OpenAI’s version. Even so, a protective data processing agreement is an important GDPR requirement when sharing personal data with “processors” such as OpenAI. The API uses pay-as-you-go billing, so you pay for each call.

Azure OpenAI service

This Microsoft service uses a hosting platform: an all-in-one environment where you can store, manage and access applications while the infrastructure is taken care of for you. Many organisations already use the Microsoft Azure cloud platform. With the Azure OpenAI Service, you can expect the same security standards as with other Microsoft Azure services. So yes, you use the same models as ChatGPT, but you manage them through that hosting platform.

With the Azure OpenAI Service, you have more control over the text generated by OpenAI models. By default, responses are stored temporarily in the same region as the resource, for up to 30 days. This data is used for debugging and investigating potential misuse of the service. If you prefer not to have responses stored, you can request this from Microsoft. For data protection, you can rely on the standard data processing agreement provided by Azure, including its standard contractual clauses. Payment is also pay-as-you-go.

Databricks

Databricks is also a hosting platform where you can deploy and manage LLMs. It offers plenty of options, including models from OpenAI and other providers. It is a versatile environment where you can fine-tune your LLM, integrate it with data pipelines and handle large-scale deployments efficiently. That makes it well suited to production environments that require high performance. It is more technical than Azure, though, so you need knowledge of data science and data engineering. This makes it less user-friendly for non-technical users. Databricks also has a comprehensive data security platform and meets GDPR requirements. The cost of using LLMs through Databricks is generally higher than the alternatives. Here too, you pay on a pay-as-you-go basis.

Open-source model/your own LLM

Here, other models form the basis of the LLM. You can choose from a wide range of open-source models, differing in size, for example. Using a local LLM instead of an API-based (closed-source) model gives you more control. With a local LLM, all data is processed within your own organisation, reducing the risk of data breaches. You also control security yourself, because you can manage access to the model. This helps minimise security risks and create a bespoke approach. It also makes it easier to comply with GDPR rules, as you have direct control over how data is handled and stored.

This LLM application takes the most time to implement because you have to set up the entire infrastructure. If you want to use an open-source LLM, you need to host it on your own infrastructure (on-premise), with powerful GPUs and sufficient storage. It is important to assess this carefully and invest in it so that your application does not perform poorly or run slowly. Unlike with the closed-source models mentioned above, you also have to take more responsibility for the output of open-source models. That is both an advantage and a disadvantage. You need to consider not only the accuracy of the answers but also whether they might be offensive. You can do this by applying various LLM techniques that act as guardrails. Of course, you should always check the accuracy of other LLMs too and use your own judgement.

Costs

Without carefully considering all the options, costs can quickly spiral. That is not the aim: it can soon put your project’s profitability and viability at risk. But don’t worry: with a careful approach, you can get the most out of your investment!

The cost of using LLMs depends on several factors:

• Model choice: More advanced models with greater capabilities often cost more. But they have also been trained on more data or can process more data as input.*
• Usage intensity: How often and how extensively you use the LLM affects the cost. But consider how much value you could get from those hours of use.
• Infrastructure: The cost of the infrastructure needed to host and manage the LLM. You are investing in your LLM’s performance and reliability.

Different funding models are available for different LLM implementations:

• Subscriptions: Monthly or annual payments for unlimited use.
• Pay-as-you-go: Pay per generated token or API call. You usually pay per 1,000 tokens, which is roughly 750 words.**
• On-premise: Deploying the LLM on your own infrastructure, which requires an initial investment.

We have several LLM applications, each with its own characteristics. Depending on your needs, we can find the best match that meets your criteria.

Characteristics by service

? Depending on settings and level of use

Scalability: Scalability means LLMs can respond flexibly to a range of situations and tasks. They are not limited to what they have already learnt: they can also adapt to new data and circumstances. This makes them useful and versatile for a range of specific applications.

Data privacy: Data privacy concerns how well an LLM protects the privacy of user data. It means keeping, processing and sharing personal information securely, in accordance with privacy legislation.

SotA performance: “State-of-the-Art” (SotA) means that, at that point in time, the model is considered the best available in terms of accuracy, relevance and overall performance.

Costs: This refers to the costs involved in using an LLM, including infrastructure, production, maintenance and any licences.

Fine-tuning: Fine-tuning means adapting an LLM to specific requirements, data or tasks to improve its performance for a particular use case. For example, you can add specific domain knowledge. Fine-tuning makes the model better suited to certain applications.

Conclusion

LLMs can help organisations become more efficient and innovative. The choice of LLM depends on your organisation’s needs. In many cases, the Microsoft Azure OpenAI service is a strong option. It is easy to use and provides adequate data security, especially for organisations that already use Azure. If you need more technical control over your LLM, Databricks is a good alternative that also safeguards your data. If you want to retain full control over data processing, hosting an open-source LLM on your own infrastructure may be a good choice. That way, you have complete control over how your LLM is used, your data and its output.

In short, with the right approach, LLMs can be a valuable addition to any organisation. Would you like advice tailored to your organisation? Get in touch with us and we’ll be happy to help.

* For an indication of current prices per token/word/character, see this website, which has a useful calculator: https://docsbot.ai/tools/gpt-openai-api-pricing-calculator

** For an indication of LLM performance, see this website: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard