Analyse AI

Overview

Once your AI Agent is active and handling customer conversations, it's essential to track its performance to understand its effectiveness and identify areas for improvement. The Analyse page provides a comprehensive dashboard with key metrics, insights, and logs to give you a clear picture of your AI's impact.

To access the dashboard, navigate to AI Agent > Analyse from the side menu.

Dashboard Overview

At the top of the page, you can filter the data you want to see:

• Integrations: Select the specific channels (e.g., Facebook, Instagram, LINE) you want to analyze. You can choose one, several, or all of them.

• Date Range: Choose the time period for which you want to view the performance data, such as the last 7 days, last 30 days, or a custom range.

Understanding the Metrics

The dashboard is divided into several sections, each providing valuable insights.

AI Agent Summary

This section gives you a high-level overview of your AI's activity and success rate.

• AI Messages Sent: The total number of messages your AI Agent has sent to customers within the selected period.

• AI Full Resolution Rate: This is a key success metric. It shows the percentage of conversations that were handled entirely by the AI from start to finish, without any human agent interaction. This is measured from chat open to chat closure.

• AI Partial Resolution Rate: The percentage of conversations where the AI handled at least one message but a human agent also participated in the chat. This is measured from chat open to chat closure.

Important Note: The metrics and logs on this dashboard only include data from conversations that have been marked as closed. Open or ongoing chats are not reflected in these analytics.

ROI Summary

This section helps you quantify the business value your AI Agent is delivering.

• Agent Time Saved: An estimate of the time your team has saved by letting the AI handle messages. This is calculated based on the number of messages the AI sends, assuming each message saves one minute of a human agent's time.

• Money Saved: An estimate of the costs saved based on the agent time saved. To make this calculation accurate, you need to click "Update average monthly salary" and input your team's average salary. The formula divides this salary by 160 working hours to get an hourly rate, then multiplies it by the time saved.

AI Agent Insights

These charts provide a more detailed breakdown of your AI's activity.

• AI Messages Sent by Channels: A bar chart showing the distribution of AI messages across your different integrations. This helps you see which channels your AI is most active on.

• Case Handling Breakdown: This chart visualizes the proportion of cases that were fully resolved by the AI, partially resolved, or had no AI involvement.

AI Chats Logs: Dive into the Details

Below the main dashboard, you'll find the AI Chats Logs. This is a powerful tool for qualitative analysis. It provides a complete list of every conversation the AI participated in during the selected period.

From here, you can:

• Filter conversations: You can filter the log to see only "AI Full" resolutions, "AI Partial" resolutions, or all chats.

• Review full transcripts: Click on any conversation to see the full chat history between the customer and the AI.

• Analyse the AI's thinking: Just like on the Test page, you can click "Show thinking" below any AI message to understand exactly why it gave that response.

Reviewing these logs is the best way to find opportunities to improve your AI. If you see the AI making a mistake, you can analyze its thought process and go back to the Train page to update the relevant knowledge source or scenario.