GenAI in your organisation: Internal LLMs and the power of RAG

What can GenAI and Large Language Models (LLMs) do for an organisation’s internal operations? How can we implement an LLM effectively for organisations? Not just as a ‘nice to have’, but to make sure it helps them work more efficiently and effectively. In other words, a step forward.

Many time-consuming processes can be handled efficiently by an LLM. We’ve previously discussed using an LLM for your knowledge management. It’s a good example of how to make your knowledge base accessible to your organisation efficiently. In this blog, I, Dave Meijdam (Data Scientist), will look at the value of an internal LLM combined with RAG for Fokker.

Fokker case study in focus

Imagine a Programme Manager at Fokker needs specific details from a contract drawn up several years ago. Instead of searching through folders and documents, they can simply ask the LLM a question, such as “What are the main terms of the contract with supplier X for project Y?”. Supported by RAG, the LLM retrieves the relevant document and summarises the requested information. This not only speeds up the process but also provides accurate, traceable information.

Another example is the Customer Support team needing quick access to information about the warranty terms for a specific aircraft part. With a centralised knowledge base and an LLM, they can ask questions such as “What are the warranty terms for part Z?”. They receive an accurate answer straight away, based on the most recent and relevant documents.

Ideally, Fokker would have a central knowledge base where anyone with questions about processes and agreements could find answers. This can be achieved by implementing an LLM combined with RAG and a knowledge base. With this solution, it is important that the organisation can validate the LLM’s answers. This helps build trust and makes it easier to spot errors. For example, the LLM can be instructed to always cite the document name and paragraph number in its answer. The organisation can also see which passages the RAG system retrieves. DSL has implemented part of this system at Fokker, called Anthony Chat.

The power of Retrieval-Augmented Generation (RAG)

One promising technology in this area is RAG. RAG combines a generative model, such as an LLM, with a retrieval system. This system searches a predefined knowledge base for relevant documents or information. The LLM then uses that information to generate an accurate answer. This makes the answers not only relevant to the context but also factually correct, because they are based on the most recent and specific information in the knowledge base.

A knowledge base is an organised collection of information that is easy to search. In an organisation like Fokker, it contains legal documents, contracts, manuals, technical specifications and other relevant company documents. By centralising this information, employees can find what they need quickly and efficiently, without having to search through multiple folders and documents. These documents are stored in a vector store.

A vector store is a database designed specifically to store and search text information using vectors. Instead of traditional keyword-based search methods, a vector store uses numerical representations of text (vectors). These vectors are generated by an embeddings model, which converts each sentence or document into a high-dimensional vector.

The RAG process explained

Here is a brief explanation of the RAG process:

Embedding documents: Each document in the knowledge base is split into smaller sections (chunks). These chunks are then passed through an embeddings model, which converts them into vectors. The vectors represent the semantic meaning of the text, so similar texts are close to each other in vector space.

Storing vectors: The generated vectors are stored in the vector store together with the associated document information (metadata).

Searching with vectors: When a user asks a question, the same embeddings model converts it into a vector. This question vector is then compared with the vectors in the vector store to find the most relevant chunks. These comparisons often use techniques such as cosine similarity, which measures how close vectors are to each other.

Retrieving documents: The most relevant chunks are retrieved and used by the LLM to generate an accurate answer.

Benefits of internal LLMs combined with a RAG implementation and a vector store as a knowledge base

Implementing GenAI, or internal LLMs, in combination with RAG and a vector store as a knowledge base can bring significant benefits to Fokker:

Precision and relevance: Vector-based search retrieves the most semantically relevant chunks, even when they do not contain exactly the same keywords as the query.

Scalability: A vector store can easily handle large amounts of data and search quickly, even across a very large knowledge base.

Context-aware searches: Because vectors represent the semantic content of the text, the system can better understand and answer complex questions that depend on context.

Improved efficiency: A knowledge base gives employees faster access to the information they need. This reduces the time spent searching for documents and improves overall productivity.

Accurate information: An LLM that uses RAG bases its answers on the most relevant information from the knowledge base. This reduces the risk of errors and inconsistencies in the information provided.

Knowledge retention: Centralising information helps retain knowledge within the organisation, especially when employees leave or change roles. This supports continuity and reduces reliance on individual employees’ knowledge.

Lessons learned & what’s next 

Building a RAG system does not come without challenges. For example, the organisation wanted to ask dozens of questions about a document at once to retrieve a large amount of specific information quickly. This did not work optimally with the initial RAG implementation: because it created a generic embedding of the input, it could not retrieve all the relevant chunks needed to answer every question. As a result, the LLM could answer some questions, but not all of them. To solve this, we added an intermediate step: when needed, the user’s input is first broken down into several specific questions. Each question is then answered in turn, and all the answers are used as chunks for the final response.

Another challenge is that contracts can look similar, especially when they have been split into chunks in a vector store. It is essential to prevent information from another client being used accidentally as a chunk in the LLM’s answer. To do this, documents and chunks need clear metadata. This could indicate which client a document relates to or, in Fokker’s case, which aircraft parts it concerns. For now, to avoid this risk, the user has to select or upload a document manually. In the future, we want to build a robust knowledge base that removes the need to select the right document manually and creates the ideal situation described earlier.

Conclusion

Integrating internal LLMs and RAG at Fokker offers an innovative solution to the challenges of storing information across multiple locations. It makes various tasks more efficient, provides consistent and accurate information, supports better decision-making, and helps retain knowledge within the organisation. By adopting this technology, Fokker can simplify day-to-day work and strengthen its overall competitive position. At a time when information is key to success, internal LLMs and RAG give Fokker the tools it needs to stay ahead. Want to stay ahead too? Get in touch with us.