OpenDialog Docs
opendialog.aiStart BuildingTalk to an expert
  • GETTING STARTED
    • Introduction
    • Getting ready
    • Billing and plans
    • Quick Start AI Agents
      • Quick Start AI Agent
      • The "Start from Scratch" AI Agent
        • Chat Management Conversation
        • Welcome Conversation
        • Topic Conversation
        • Global No Match Conversation
        • Supporting LLM Actions
        • Semantic Classifier: Query Classifier
      • A Process Handling AI Agent
  • STEP BY STEP GUIDES
    • AI Agent Creation Overview
    • Add a new topic of discussion
    • Use knowledge sources via RAG
    • Adding a structured conversation
    • Add a 3rd party integration
    • Test and tweak your AI Agent
    • Publish your AI Agent
  • CORE CONCEPTS
    • OpenDialog Approach
      • Designing Conversational AI Agents
    • OpenDialog Platform
      • Scenarios
        • Conversations
        • Scenes
        • Turns and intents
      • Language Services
      • OpenDialog Account Management
        • Creating and managing users
        • Deleting OpenDialog account
        • Account Security
    • OpenDialog Conversation Engine
    • Contexts and attributes
      • Contexts
      • Attributes
      • Attribute Management
      • Conditions and operators
      • Composite Attributes
  • CREATE AI APPLICATIONS
    • Designing your application
      • Conversation Design
        • Conversational Patterns
          • Introduction to conversational patterns
          • Building robust assistants
            • Contextual help
            • Restart
            • End chat
            • Contextual and Global No Match
            • Contextual FAQ
          • Openings
            • Anatomy of an opening
            • Transactional openings
            • Additional information
          • Authentication
            • Components
            • Example dialog
            • Using in OpenDialog
          • Information collection
            • Components
            • Example dialog
            • Using in OpenDialog
            • Additional information
          • Recommendations
            • Components
            • Example dialog
            • Additional information
          • Extended telling
            • Components
            • Example dialog
            • Additional information
          • Repair
            • Types of repair
            • User request not understood
            • Example dialog
            • Additional information
          • Transfer
            • Components
            • Example dialog
            • Additional information
          • Closing
            • Components
            • Example dialog
            • Using in OpenDialog
            • Additional information
        • Best practices
          • Use Case
          • Subject Matter Expertise
          • Business Goals
          • User needs
            • Primary research
            • Secondary research
            • Outcome: user profile
          • Assistant personality
          • Sample dialogs
          • Conversation structure
          • API Integration Capabilities
          • NLU modeling
          • Testing strategy
          • The team
            • What does a conversation designer do
          • Select resources
      • Message Design
        • Message editor
        • Constructing Messages
        • Message Conditions
        • Messages best practices
        • Subsequent Messages - Virtual Intents
        • Using Attributes in Messages
        • Using Markdown in messages
        • Message Types
          • Text Message
          • Image Message
          • Button Message
          • Date Picker Message
          • Audio Message
          • Form Message
          • Full Page Message
          • Conversation Handover message
          • Autocomplete Message
          • Address Autocomplete Message
          • List Message
          • Rich Message
          • Location Message
          • E-Sign Message
          • File Upload Message
          • Meta Messages
            • Progress Bar Message
          • Attribute Message
      • Webchat Interface design
        • Webchat Interface Settings
        • Webchat Controls
      • Accessibility
      • Inclusive design
    • Leveraging Generative AI
      • Language Services
        • Semantic Intent Classifier
          • OpenAI
          • Azure
          • Google Gemini
          • Output attributes
        • Retrieval Augmented Generation
        • Example-based intent classification [Deprecated]
      • Interpreters
        • Available interpreters
          • OpenDialog interpreter
          • Amazon Lex interpreter
          • Google Dialogflow
            • Google Dialogflow interpreter
            • Google Dialogflow Knowledge Base
          • OpenAI interpreter
        • Using a language service interpreter
        • Interpreter Orchestration
        • Troubleshooting interpreters
      • LLM Actions
        • OpenAI
        • Azure OpenAI
        • Output attributes
        • Using conversation history (memory) in LLM actions
        • LLM Action Analytics
    • 3rd party Integrations in your application
      • Webhook actions
      • Actions from library
        • Freshdesk Action
        • Send to Email Action
        • Set Attributes Action
      • Conversation Hand-off
        • Chatwoot
    • Previewing your application
    • Launching your application
    • Monitoring your application
    • Debugging your application
    • Translating your application
    • FAQ
    • Troubleshooting and Common Problems
  • Developing With OpenDialog
    • Integrating with OpenDialog
    • Actions
      • Webhook actions
      • LLM actions
    • WebChat
      • Chat API
      • WebChat authentication
      • User Tracking
      • Load Webchat within page Element
      • How to enable JavaScript in your browser
      • SDK
        • Methods
        • Events
        • Custom Components
    • External APIs
  • Release Notes
    • Version 3 Upgrade Guide
    • Release Notes
Powered by GitBook
On this page
  • The basics
  • Analyse in action
  • Where to find
  • Structure
  • How to use
  • Project Usage
  • Interaction logs
  • Message log table
  • Detailed conversation logs
  • Frequently asked questions
  1. CREATE AI APPLICATIONS

Monitoring your application

Use the OpenDialog Analyse feature to make informed decisions about the past, present and future of your conversational application.

PreviousLaunching your applicationNextDebugging your application

Last updated 7 months ago

This section explains how to track user interactions with your conversational application and identify trends in project usage.

The OpenDialog Analyse dashboard provides a comprehensive view of recent interactions, a log of the latest messages, and a high-level summary of usage patterns over time.

The basics

Analyse lets you visualise your application's conversational data set, which you can use to improve your conversational application where needed.

It allows you to answer the following questions:

  • How many conversations are happening by how many users?

  • What is happening in the conversations?

  • Which conversations had errors and where?

  • What was the logic that led to specific conversational paths & errors?

Analyse provides you with a visualisation of the application's interaction logs, project usage over time, message logs, detailed conversational history per user, flow visualisation and dialogue management insights.

Analyse in action

Actionable insights

Viewing specific conversations and their logic, you can make informed decisions and improve your application's user experience.

Application health

By viewing the number of fallbacks and filtering conversations based on them, you can check your application's health and identify improvement areas.

Diagnose & improve discovery

By viewing the number of users that started a conversation and the number of conversations per user, you can diagnose any discoverability & desirability issues and fix them.

Monitor ROI

By visualising usage trends, you can see the evident uptake or downturn in the number of users and expand or improve the application's use case accordingly.

Where to find

Analyse allows you to explore how your application has been performing. It is, therefore, part of the testing suite of a given scenario.

To access the Analyse dashboard:

  • Navigate to your workspace dashboard

  • Select a scenario

  • Using the left-hand sidebar menu, expand the 'Test' section

  • Select 'Analyse'

Structure

To use the Analyse feature within OpenDialog, navigate to the 'Analyse' dashboard within the sidebar, where you can review, explore, and expand different data snapshots of your application.

Analyse has three main components: project usage, interaction logs, and message log table:

How to use

Project Usage

The project usage functionality allows you to visualise project usage trends over time.

You can expand or minimize the project usage graph to your needs using the Expand/Collapse icons in the top right corner of your screen, depending on how focussed you wish the view to be.

Project usage can be viewed for a specific day, week, month or other selected date range via the date picker in the top right corner. This allows you to hone in on specific date ranges and check your conversational application's usage for that time.

This can be particularly useful, for example, if you have made specific changes to where your users can find your application, the call-to-action, or the conversational experience at a given time, and you wish to visualise and communicate its impact on the number of users.

Interaction logs

The list of interaction logs allow you to visualise a list of conversations had by specific users, at a specific time and it's related data. Available data points per interaction log are date & time , user-ID , timezone , platform , browser , number of fallbacks in the conversation, and number of messages . Each of these can be used to sort through the interaction logs. We will specify the use of a few of them below.

In addition, you can use the date picker on the top of the screen to select conversations for a specific date range, or search the interaction log for a specific user-ID thanks to the user-ID search field in the top right corner.

Date & time gives you an indication of the timestamps of users interacting with your application. Filtering by this field allows you to visualise the most recent conversations first or, on the other hand, the oldest conversations. In addition, the timestamp can help you identify a specific conversation for a user, in case you need to review it. Bearing in mind that an interaction spans over a period of time, the single value in the Date and Time column show us the date and time of the first message sent within an interaction (within the date range specified by the filters).

User-ID indicates the user's unique user-ID which can help you identify a specific user. OpenDialog automatically generates a user-ID for each start of your application, if one is not provided. However, this user-ID can also be user-specific like user e-mails if your scenario is set up with this information.

You can also sort by fallback to visualise which logs contain more fallbacks versus which ones did not. It will be interesting to dig further into those conversations with many fallbacks to identify areas of improvement for your conversational model or your language service.

The number of messages will indicate how many exchanges a user had with your application within the provided date range. This can indicate users' engagement with your application and its capacity to retain users and help them accomplish their goals.

You can click through each item in the interaction log list to visualise its detailed conversation logs.

Message log table

The message log table is an essential tool for reviewing and understanding the full history of conversations within your OpenDialog scenarios. Combined with its filtering capabilities, the table offers a powerful method for monitoring and analysing the flow of messages. It acts as a detailed log of every message exchanged between users and the application, providing valuable insights into the progression of conversations.

There is an export button in the top left hand side of the table. Clicking this button will export data with the current filters applied into a CSV file for you to download.

By examining this data, you can evaluate the performance of your application, identify patterns, and uncover potential areas for improvement.

Columns in the message log table

The table presents a variety of fields that categorise each message within your application:

Date and time

Indicates when the message was created

User ID

User-ID indicates the user's unique user-ID which can help you identify a specific user.

Conversation, Scene, Turn and Intent

Indicates where in your scenario this message was sent

Author

Specifies the message origin, which will be either user or system (your application).

Type

Indicates the message type, such as text, image, or button.

Message

Displays the actual content of the message, either from the user or the system. This is where you can review exactly what was communicated in each turn of the conversation.

Filtering Messages

There are many different ways to filter the message log table to find exactly what you are looking for. You can filter by date and time by clicking on the calender and selecting the specific time frame you would like to filter the messages by.

You can also filter your messages based on User ID. To do this, you can start to manually type in a User ID into the search bar. Or, you can click on the User ID attached to a certain message in the table and this will populate the search bar with this User ID for you.

Another way you can filter the table is with the preview filter. This allows you to select what type of messages you would like to view in your message log table. There are three different options to choose from:

  • Preview data - this filters all the messages that have been generated using the 'Preview' application internally within OpenDialog

  • Live data - this filters all the messages that have been generated from a live conversation with a user

  • All data - this shows you all of the available data

The message log table includes a breadcrumb filter to make message analysis quicker and more effective. This feature allows you to filter the messages based on specific criteria, ensuring that you can narrow down the data to the most relevant interactions. The filter works in a hierarchical fashion, helping you to further investigate the data in a descending order:

  1. Conversation: The first filter layer is the conversation itself, allowing you to isolate a particular session with a user. This is helpful when you want to analyse all messages associated with a single conversation from start to finish.

  2. Scene: Once you've selected a conversation, you can further filter by scene. This allows you to examine messages within a particular phase of the interaction, providing context on how users move through the various stages of the conversation.

  3. Turn: With scene filtering applied, you can then add an additional filter for specific turns. This level of filtering helps you focus on moments (where either the user or system sent a message) enabling you to further investigate the flow of interaction at a more granular level.

  4. Intent: Intents display the exact message contents from either the user or the system, allowing you to review what was communicated within each turn of your conversation.

Detailed conversation logs

The detailed conversational logs for each item in the interaction logs are composed of four sections:

  • The user details

  • The conversational log history

  • The conversational flow visualisation

  • The conversational logic logs

User information gives you a quick summary of the interaction log item you are viewing. You also have the option to copy the User-ID from here to search for further interaction logs by this same user in the interaction logs list.

The conversation logs contain the entire conversational history for this interaction log, tracking back all user and application exchanges. You can use the 'play' button to visualize the individual exchanges and their logic in the middle panel of the screen. You can also view the intent details for each message by using the 'View intent details' toggle at the top of the Conversation Log section. If you see any issues with the displayed message, you can thus click on the linked intent and go edit the selected message.

The considered conversational paths are displayed in the middle panel for the selected user-application exchange. You can play through them via the Play button on the conversational logs or walk through them step by-step to review the conversation engine logic by using the arrows at the bottom of the screen. This helps you to understand what incoming and outgoing intents, and messages were selected by the conversation engine and why. If any virtual intents were encountered within the request you are viewing, a further cycle of frames will be displayed. These frames will display the incoming and outgoing intents that were selected as a consequence of reaching the virtual intent.

Frequently asked questions

I need more granular data. How can I access this information?

If you need more granular data, you can contact us via support@opendialog.ai.

I do not see the conversational flow visualisation. Is this normal?

Conversational flow visualisation is currently unavailable for exceptionally large scenarios. We are working on alternative visualisations to cater for those as well.

I want to access the analytics data via an API rather than the OpenDialog application. How can I do this?

I got a warning message for the conversational flow visualization indicating 'No Request ID provided'. What does this mean?

This can happen if the event storage was not active or turned off when the user made a request. In the future, you should not encounter this any more as this is related to the roll-out of the Analyse feature.

Finally, the right-hand panel displays the conversational logic behind the selected conversational paths, displaying the available, selected and rejected intents and the in-depth reasoning behind each decision made by the conversation engine. This is particularly useful to debug conversations and, for example, to better understand why certain user utterances might have caused an unexpected answer. The detailed conversational logic contains information regarding the of the intents via the interpreters set up for your scenario, the passing of on conversational components, the reasoning behind intent selection, transitions between conversational components and what are attributes are being saved during that interaction.

The enables retrieving information about interactions between the user and OpenDialog scenarios. The data is requested for over a time period ( from , to ) and is split by user_id . You can check out the Interaction API documentation through this .

interpretation
Interaction API
link
The OpenDialog Analyse dashboard gives you an overview of the latest interactions with your application and a bird-eye overview of project usage over time.
You can find Analyse, in the Test section of your scenario menu.
Expanded view of project use over the course of a week
List of interaction logs
List of message logs
By using the breadcrumb filter, you can narrow down you search by conversation, scene, turn and intent
Detailed conversation log, including user information, conversational history, conversational flow visualization, and conversational logic.
Cover

The project usage graph shows both the number of users and the number of requests for any given date or range of dates. It can also be expanded for a more focused and in-depth visualisation.

Cover

The interaction logs are listed and filtered by date, with the most recent appearing at the top. Each interaction log can be expanded to review specific conversations.

Cover

The message logs are listed and filtered by date, with the most recent appearing at the top. Messages can also be filtered by Conversation, Scene, Turn and Intent to find specific results.

log table

Project usage
Interaction logs
Message