Integrate with Microsoft Azure OpenAI chat to create multiple chatbots using your Company’s data.  Leverage “Ask Q” directly from your Q360 window and find additional information using included references.

Using Azure OpenAI on your data enhances the ability to retrieve and utilize data in a way that improves Ask Q’s output. You can choose to deploy multiple chatbots, each with different capabilities.  These chatbots can use the base GPT-3 or GPT-4 models, where responses are similar to human-like conversation and based on pre-trained data, or a chatbot that is limited to delivering results based on Retrieval Augmented Generation (RAG) where either only your data is used, or your data is used in conjunction with the rest of the model’s data.

This integration does not pull data directly from Q360.  Instead, any data used in the RAG must be loaded into Microsoft Azure and indexed on a Microsoft Cognitive Search Service.  This process ensures the security and privacy of your data while enhancing the capabilities of the Ask Q chatbot. There are features built into various areas that will allow you to “Ask Q” about that data using a right-click + selection, or Extended Menu option.  In those cases, the data is sent through your prompt to the language model, where it is used to aid in the response.

1. Go to Maintenance > AI Configuration
2. Enter a Description for the chatbot. This description will appear under the Ask Q Chat Bubble to an end User with the proper permissions
3. In the API Key field, enter your Azure OpenAI API Key
4. In the Resource Name field, enter the name of your Azure OpenAI Resource
5. In the Deployment Name field, enter the name of the model deployment that you have previously created
6. Note: Type is currently limited to “Assistant”

Chat Settings (optional)

• Modify the Chat Settings as necessary for your configuration
• System Prompt: The Prompt field denotes the prompt that is sent with every request to the API.  Utilize best practices in prompt engineering in an effort to achieve the desired responses from your chat box
• Max Tokens: The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens can’t exceed the model’s context length
• Temperature: What sampling temperature to use, between 0 and 2. Higher values mean the model takes more risks. Try 0.9 for more creative applications, and 0 for ones with a well-defined answer. Best practices recommend altering this OR probability threshold, but not both
• Probability Threshold: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this OR temperature, but not both
• Search Service Name: The name of the Azure Cognitive Search service configured in your Azure Service Portal. This is necessary when using Retrieval Augmented Generation on your data
• Search Index Name: The name of the Azure Cognitive Search Index created for your data
• Search API Key: The API Key created under the Azure Search Service -allows access from Q360
• Internal Docs Only: Checking this box limits the chatbot to only return results based on your data
• Top # Documents: This parameter is an integer that can be set to 3, 5, 10, or 20, and controls the number of document chunks provided to the large language model for formulating the final response. By default, this is set to 5. The search process can be noisy and sometimes, due to chunking, relevant information might be spread across multiple chunks in the search index. Selecting a top-K number, like 5, ensures that the model can extract relevant information, despite the inherent limitations of search and chunking. However, increasing the number too high can potentially distract the model. Additionally, the maximum number of documents that can be effectively used depends on the version of the model, as each has a different context size and capacity for handling documents
• Query Type: Options are SIMPLE or SEMANTIC. In Azure AI Search, semantic ranking measurably improves search relevance by using language understanding to re-rank search results. The search index must be enabled with semantic search in Azure to support this
• Semantic Config: The name of the semantic configuration for the Azure Search Index set above.  Please note that any settings in this section are limited by the configuration of your Azure services

Permissions:

• Assign permission Q360AI to the Users or Groups for which you would like to permit access to your Ask Q chat bot
• The created chat bot will be available in the Ask Q bubble upon the User’s next login to Q360

Reporting:

• All calls to the API and the associated responses are logged into a table.  This table can be accessed via the Live Data Designer using the view LDView_AIChatLog, assuming the report designer has the appropriate permissions

Sending Data to Ask Q:

• On any grid, you can highlight data and then, via a right-click + select “Ask Q about Highlighted Data.”  The data selected will be sent to the currently selected Assistant in json format.  Token limits set on your Azure Open AI model deployment and in your Q360 AI Config options must be taken into account when using this feature

Service calls also have an Extended Menu function to “Ask Q.”  This will copy the text in the Problem Description to the chat window of the currently select chatbot Assistant