It’s easy to track Resolution Bot’s impact on your business. In the Resolution Bot report, you can measure:
How many conversations Resolution Bot covers for your team.
What percentage of conversations it successfully resolves.
The time it saves your customers, teammates, and more.
To access this report, go to Reports > Customer support > Resolution Bot:
You can choose any date range you’d like to measure:
Along with the type of conversations to show in the report:
Inbound conversations are started when customers click ‘New conversation’ in the Messenger.
Outbound conversations are started when customers reply to an ongoing message, or Custom Bot.
1. See Resolution Bot’s overall impact
You can get a quick overview of Resolution Bot’s performance - how many conversations it resolved, and the time it saved your customers and teammates.
These metrics will help you quickly gauge Resolution Bot’s impact on your support.
What do these metrics mean?
Conversations resolved immediately
‘Conversations resolved immediately’ is how many conversations Resolution Bot closed after providing your customer with a successful answer. This includes conversations where:
The customer clicked “That helped”.
The customer didn’t reply after Resolution Bot answered their question.
Resolution Bot automatically closes these conversations.
Customer hours saved
‘Customer hours saved’ shows you how much time Resolution Bot’s instant resolutions saved customers, as they didn’t have to wait for your team to reply. Here’s how it’s calculated:
The number of conversations Resolution Bot resolved, multiplied by your team’s median resolution time (defined as 'Time To Last Close' for conversations started in the last seven days) for inbound conversations.
Important: ‘Customer hours saved’ is not impacted by your selected date range. It’s always calculated from the last 7 days.
Teammate hours saved
‘Teammate hours saved’ measures how much time Resolution Bot saved your team, by resolving common questions for them. Here’s how it’s calculated:
The number of conversations resolved by Resolution Bot, multiplied by the average ‘one-touch handling time’ for all Intercom customers.
One-touch handling time refers to the time it takes teammates to resolve conversations that only require one reply. These are the types of conversations Resolution Bot covers for your teammates.
Note: We use the average “one-touch handling time” across all Intercom customers in the calculation, which is 12 minutes. If this time is lower than your median resolution time, then median resolution time is used in the calculation instead.
2. Track Resolution Bot’s coverage
You can see how many conversations Resolution Bot covers for your team over time (the x/time-axis is the date the conversation was started):
You can track:
The total number of conversations in which Resolution Bot provided at least one answer in the selected period.
What percentage of conversations Resolution Bot covers (coverage rate). You should aim to increase your coverage rate by creating more answers.
Coverage rate is calculated as the total number of conversations in which Resolution Bot provided at least one answer, divided by the total number of inbound conversations in the selected time period.
Coverage rate includes Articles as well as Answers. It’s not possible to see which articles are included in the reporting.
Ready to get started?
Automatically resolve 33% of your common questions!
3. Improve Resolution Bot’s resolution rate
Crucially, you can track how many instant resolutions Resolution Bot provides your customers.
The resolution rate bar shows you three key metrics:
Resolved, customer closed: The dark green bar shows the percentage of conversations where the user clicked “That helped”. Resolution Bot instantly closes these conversations.
Resolved, customer left: The light green bar shows the conversations where Resolution Bot provided an answer, but the customer didn’t reply. This usually means their question was successfully answered. Resolution Bot closes these conversations for your team, if a customer or teammate doesn’t reply within three minutes.
Routed to team: The gray bar shows the conversations where the customer clicked “Wait for team” or replied with text. Resolution Bot leaves these conversations open, so a teammate can reply.
This section also explains what percentage of total inbound conversations have been resolved by Resolution Bot.
Note: Resolution Bot routes conversations in accordance with your assignment rules.
You can also see the average resolution rate across all Intercom customers, so you can compare your Resolution Bot’s performance with other Intercom customers.
Finally, you can check the percentage of total conversations Resolution Bot resolves, so you can track an increase in this figure as you create more answers.
☝️ This is the percent of total conversations that Resolution Bot resolved, this includes conversations where Resolution Bot had no matching answer to suggest.
Reduce conversations where Resolution Bot can't find an answer
Under "Conversations that were not covered and customer left", you'll see a list of questions that Resolution Bot failed to answer and then the customer left:
Important: This only shows question asked when looping mode was enabled.
You can use these questions to train Resolution Bot by creating a new answer, or adding the question to an existing answer.
Just click "Train Resolution Bot" next to an answer:
To add the question to an existing answer, click "Train existing answer":
To create a new answer from scratch for this question, click "Create new answer":
How is the resolution rate calculated?
The resolution rate is calculated as the number of conversations that were resolved immediately by Resolution Bot, divided by the total number of conversations in which Resolution Bot provided at least one answer.
Your total resolution rate adds the ‘Resolved, customer closed’ rate with the ‘Resolved, customer left’ rate, as these are conversations where Resolution Bot has most likely provided a helpful answer.
Improve underperforming answers
One of the easiest ways to improve your resolution rate is to improve any underperforming answers. You can see these on your answers list by applying the “Underperforming” filter. Using the date picker, you can look at answers that are underperforming since they have been set live, or from a specific time frame.
Note: This option will only be present if you have underperforming answers.
For a more detailed look at how a single answer is performing, you can open the answer, and see its stats:
To ensure your answer is triggering for the right questions, open the ‘Routed to team’ tab and check the conversations that weren’t resolved:
Pro tip: If you notice your answer has triggered for an irrelevant question, you should do two things:
Create an answer for the specific question your customer asked. This will increase your answer’s accuracy, as Resolution Bot will have the right answers for similar questions.
You can add qualifiers to your answer, so it only sends when customers use specific words.
Metrics when Looping Bot is enabled
Teammates can report on the performance of Looping Bot in the existing ‘Resolution Bot’ reporting page. The top of the page will present itself with a new filter, allowing teammates to review the existing reporting page, through the lens of any Resolution Bot setting applied.
For Looping Bot we report ‘Resolved’ and ‘Routed to team’ on a conversational level rather than individual answer level. If an answer was part of a conversation that was either Resolved or Routed to team, we will allocate that outcome to the answer. The revised reporting calculations:
Sent: number of conversation where the answer has been served by the Resolution Bot
Resolved: Among "Sent", number of conversation that ended up with a "That helped" reply or soft resolution
Routed to team: Among "Sent", number of conversation that ended up being routed to the team
This means, that if two answers were sent in the conversation, regardless of which one resolved the customer's question, it will count to both answers "Resolved" rate. Same goes for "Sent" and "Routed to team".
Here are more resources to help you set up Resolution Bot for success:
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts