Retrieval Augmented Generation

The Basics

Retrieval Augmented Generation is an advanced system that combines information retrieval with natural language processing techniques to generate contextually relevant responses, based on a semantic search over a vectorised knowledge base.

The OpenDialog RAG service allows you to

• upload & manage your own documents, data and information

• convert it into numerical representations through vectorisation

• query your data via tailored prompting

• generate contextually relevant responses to user queries

In Action

Use your own data

Generate response from your own data, making sure that responses are accurate and relevant to your user's queries.

Dynamic content generation

Rather than depending on statically defined messaging, make your content more dynamic by generating responses based on your up-to-date knowledge.

Up-to-date responses

By relying on up-to-date information from retrieval mechanisms, the system can provide more accurate and current responses.

Where to find

OpenDialog's RAG service allows you to manage your content for use in your AI application. In addition, the OpenDialog RAG feature provides an overview to manage your different knowledge sources, sorted by relevant topic.

To access the RAG service feature in your workspace:

Structure

The OpenDialog service consists of two main components : the settings of your RAG service, and the topics component.

The RAG service settings section allows you to name and provide a description for your RAG service to easily recognise it in the Language Services Dashboard. Once it is being used in your applications, you can also visualise the scenarios that are actively using this particular RAG service.

The Topics section allows you to define, manage, edit and test the main semantic topics of your RAG service.

To action the generation of responses based on your RAG service, you will need to use an additional feature of OpenDialog: LLM Actions. The setup of a topic in your RAG service will generate a knowledge string that you can reference in your LLM Actions prompt, to ensure that the generated answer uses the knowledge sources you have provided.

How to use

Overview

[Video tutorial coming soon]

Vocabulary

RAG

Retrieval Augmented Generation is an advanced system that combines information retrieval with natural language processing techniques to generate contextually relevant responses, based on a semantic search over a vectorised knowledge base.

Topics

In the context of a Retrieval-Augmented Generation (RAG) service, a "topic" refers to a specific subject area or theme around which information is organised, retrieved, and generated. Topics help in structuring the knowledge base and guiding the retrieval mechanism to fetch relevant information, which the language model then uses to generate responses.

Topic Sources

All of the information that your RAG service topic relies upon is stored in topic sources. The data saved in each of these Topic Sources can be provided in one of several formats: uploading a document, linking to a website, or simply written in as text.

When a topic is queried, behind the scenes, all of the topic sources belonging to that topic will be sifted through to find the segments most relevant to the query.

Knowledge strings

A knowledge string is a piece of syntax you will use to reference a specific RAG service, and one of its topics within an LLM action, or for testing purposes in the test panel.

Set up your RAG service

Basic settings

Advanced settings

If you are an advanced user of RAG services, and know your way around vectorisation - the Advanced settings section is where you can update the following settings:

Chunk size

Chunk size refers to the number of tokens or words that a piece of text is divided into during processing. This setting determines how large each segment, or "chunk," of text will be when it is broken down for analysis or computation. For example, a chunk size of 512 tokens will divide the text into segments of 512 tokens each. Adjusting the chunk size can directly impact the performance and accuracy of text processing tasks like vectorization and information retrieval.

Chunk overlap

Chunk overlap is the number of tokens that overlap between consecutive chunks of text. It ensures that important context is preserved across chunks, improving the accuracy of downstream tasks such as information retrieval or machine learning models. For example, if your chunk size is 512 tokens and your chunk overlap is 50 tokens, each chunk will share 50 tokens with the previous chunk. Adjusting chunk overlap can help balance between context preservation and computational efficiency.

Top K

Top K refers to the number of top results or items to retrieve during a search or query operation within Retrieval-Augmented Generation (RAG) services. Specifically, it determines how many of the most relevant documents or pieces of information are returned from the database or knowledge base for further processing. For example, setting Top K to 5 will retrieve the 5 most relevant documents based on the search criteria. Adjusting this setting can influence the quality and relevance of the retrieved information, balancing between precision and recall in information retrieval tasks.

Add your topics and relevant data sources

Once you have created your RAG service setup, you will get taken to the Topics page. This page will be empty when first creating your service. You can start setting up your topics, by clicking on the Add topic button in the top right corner of the middle panel of your screen.

Setting up your topics

Adding topic sources

Once you have created a topic, you can start adding your documents, data and information to it by using the 'Adding sources' section.

Adding a text source

Once you have selected text, the text upload modal will pop up. From here you can add your text source name (so that it is easily recognisable from within the topic sources table) and a space where you can add your text source.

Adding a URL source

To add a URL source, you first need to select the 'URL' option from the topic source drop down menu.

Once you have selected 'URL' from the topic source dropdown, the URL upload modal will appear. From here, you can add in your source. You can either add in one URL, or you also have the option to add in multiple URL's, separated by a comma list.

You also have the option to include exclusion ID's and Classes. These optional fields allow you to select parts of your URL that you would not like to be referenced. Then when your URL source is vectorised, these ID's and Classes will not be included.

Once you are happy with the URL`s selected, you can click the 'Upload source' button to add them to your topic.

Adding a document source

To add a document source, you first need to select the 'Document' option from the topic source drop down menu.

The types of document that can be uploaded to your topic as a source are:

• HTML

• CSV

• PDF

• DOCX

• DOC

• TXT

• YAML

• XLSX

Once you have selected Document, the document upload modal will pop up. From here you can either click to upload a document of your choice, or you can drag and drop files onto the modal. You can upload multiple files at once if you wish to.

After you have selected the files that you want to upload, you can see them listed underneath the document upload modal. If you have accidentally selected a file that you don't wish to upload, you can click the 'X' button next to it's name to remove it from the list.

Once you are happy with the files selected, you can click the 'Upload source' button to add them to your topic.

Vectorise your topic sources

When you add topic sources to your knowledge service topics, they get stored to the topic sources table. From within the topic source table, you can see all of the topic sources that you have added.

Your topic sources now need to be converted to their numerical representation, using vectorisation.

A topic source can be in one of the following vectorisation statuses :

• Requires Vectorisation : your topic source has not yet been vectorised since its last upload or edit. Your topic source might also be put in this status when it has become obsolete or when the RAG settings have changed. Select the topic source and click the vectorise button to start the process.

• Vectorisation in progress : your topic source is currently being vectorised. Depending on the size of your topic source, this might take a few minutes.

• Vectorised : your topic source is vectorised and ready to be used by an LLM

• Vectorisation failed : the vectorisation process of the selected topic source has failed. Verify your settings, and retry.

Test your RAG service

Once you have vectorised your different topic sources for a given topic, you can now start testing the responses that will get generated based on this information and a user query, using the test panel.

Retrieve your knowledge string

You can use your RAG service in your AI application by referencing it through a knowledge string in other services, like LLM Actions. In order to do so, you need to retrieve or construct the relevant knowledge strings.

Retrieving a knowledge string

The easiest way to retrieve your knowledge string is by going into the test panel of a specific topic. The default system prompt already has your pre-formatted knowledge string in it. You can simply copy/paste it for use in an LLM action.

Example:

%%MyKnowledgeService.topic?{utterance_text}%%

Constructing a knowledge string from scratch

You can construct your knowledge string from scratch, and use it elsewhere in the OpenDialog platform by typing %% followed by your knowledge string syntax.

To use your RAG service from within a system prompt you should use the special syntax %% [service].[topic] %% to reference the data you want to use. For example if you have a knowledge service called 'countries_ks' and you want to reference the 'capital_cities' topic data you would use %% countries_ks.capital_cities%%.

You should also add a query to only return the topic data that is useful to the user's question. To do this, the syntax is %% [service].[topic] ? [query] %%

An example knowledge service string, used in a system prompt:

"You are a chatbot answering user questions. You may only find your answer from within the following data: %% countries_ks.capital_cities ? France %%

Set up an LLM action referencing a RAG service

In order to use the knowledge from within a specific RAG service to generate responses in your OpenDialog application, you will need to:

Using a knowledge string in your LLM action

For example:

You are a helpful customer service assistant. 

The user will provide you with questions
related to the organisation, its founders, its originis and its vision. 

Your goal is to provide a relevant response to their queries.  

Construct your response using the following data: 
%%MyKnowledgeService.AboutCompany?{utterance_text}%%

Add your RAG-based LLM Action to your conversation design

In order to use your RAG-based generated response in your conversation design, you will need to update it to run the LLM action on the relevant intent and reference its output in the relevant message.

Adding your RAG-based LLM Action to the conversation design

Just like other LLM actions, in order for them to run, they need to be added to the intent they are relevant for.

Adding your RAG-based LLM Action output to the messages

Your LLM action's response based on the RAG knowledge will be stored against an output attribute. If you have not defined a specific output attribute in your LLM action, the default output attribute will be <llm_response>.