Fin AI Agent report

Find out how Fin AI Agent is performing in conversations and impacting your resolution rates.

Beth-Ann Sher avatar
Written by Beth-Ann Sher
Updated over a week ago

After you’ve set Fin AI Agent live, you’ll be able to monitor it's performance in the Fin AI Agent report.

To access this report, go to Reports > Automation > Fin AI Agent.

You can filter this report by date and conversation, user, company, and custom attributes.

Use this report to see:

  • Fin AI Agent’s involvement in all conversations.

  • The answer rate, deflection rate, and resolution rate of conversations in which Fin AI Agent is involved.

  • Fin AI Agent’s impact over time.

  • Breakdown of resolved and unresolved conversations.

  • List of conversations Fin AI Agent didn’t provide an answer for.

  • Fin AI Agent’s customer satisfaction score (CSAT).

Report definitions

We refer to the bot quick reply buttons That helped and Get more help in the definitions below but you can customize which options you use in your Fin AI Agent Settings.

Name

Definition

Involvement Rate

The percentage of conversations in which Fin AI Agent attempted to provide an answer out of all new conversations. Fin involvement is dependent on audience targeting rules. If Fin is shown to all customers, the involvement rate will be 100%.

New conversations include all new inbound and outbound interactions. These are only considered 'new' if the first message from the customer was received during the selected date range.

Answer Rate

The percentage of conversations in which Fin AI Agent sends at least one answer, whether it's an AI Answer or a Custom Answer, out of all conversations Fin AI Agent was involved in.

An answer from Fin AI Agent is defined as any content that could potentially answer the customer's question.

Deflection Rate

The percentage of all conversations involving Fin AI Agent, in which either:

  • The conversation was successfully resolved, or

  • The customer left before receiving an answer, or

  • The customer left after reacting negatively to the last delivered answer by Fin.

Resolution rate

This is the percentage of conversations resolved by Fin AI Agent, divided by conversations where Fin AI Agent attempted to provide an answer.


A resolved conversation constitutes either a confirmed OR assumed resolution.


A confirmed resolution is when a customer:

  • Clicked the That helped quick reply button to Fin AI Agent’s last reply, or

  • Entered an affirmative response such as ‘Yes’, ‘Sure’, etc. to Fin AI Agent’s last reply.

An assumed resolution (which is auto-closed after 3 minutes) is when a customer:

  • Did not click the Get more help or Talk to a person quick reply buttons.

  • Did not ask to speak to your human support team.

  • Did not enter a negative response such as ‘No’, ‘That didn’t answer my question’, etc.

Note: This is a new definition for resolution rate. Previously for Resolution Bot, this metric used 'Conversations with answers' as the denominator.

AI Answer resolutions

The number of conversations where Fin AI Agent's last answer was an AI Answer, and following this:

The customer indicates satisfaction with Fin AI Agent’s last reply by:

  • Clicking the that helped quick reply button.

  • Entering an affirmative response such as ‘Yes’, ‘Sure’, etc.

OR

The customer exits the conversation without indicating further need for assistance, as shown by:

  • Not clicking the get more help quick reply button.

  • Not asking to speak to your support team.

  • Not entering a negative response such as ‘No’, ‘That didn’t answer my question’, etc.

Custom Answer resolutions

The number of conversations where Fin AI Agent's last answer was a Custom Answer, and following this:

The customer indicates satisfaction with Fin AI Agent’s last reply by:

  • Clicking the that helped quick reply button.

  • Entering an affirmative response such as ‘Yes’, ‘Sure’, etc.

OR

The customer exits the conversation without indicating further need for assistance, as shown by:

  • Not clicking the get more help quick reply button.

  • Not asking to speak to your support team.

  • Not entering a negative response such as ‘No’, ‘That didn’t answer my question’, etc.

Assumed Resolution

The number of conversations in which Fin AI Agent provided an AI answer and the customer did not select the option that helped or get more help. We assume they got their resolution and dropped off the conversation.

Confirmed Resolution

The number of conversations in which the customer replied with positive feedback or clicked that helped quick reply button after the last answer was delivered by Fin AI Agent.

Abandonment rate

The percentage of conversations in which the customer either:

  • Left without receiving an answer from Fin AI Agent, or

  • Left after reacting negatively to the last answered delivered by Fin AI Agent,

out of all the Fin AI Agent involved conversations.

Routed to team rate

The percentage of conversations in which Fin AI Agent got involved and the customer either:

  • Requested to talk to a teammate, (e.g. 'I want to talk to a teammate') or

  • Clicked the Talk to a teammate quick reply button.

Fin CSAT

Percentage of positively rated (🤩 or 😀 ratings) conversations out of all Fin AI Agent rated conversations.

Fin rated conversations

Fin AI Agent conversations broken down by rating.

Remarks from customers

Customers’ conversation ratings and the optional comments they left with each rating.

Note: Only the date range filter affects remarks shown here.

Comparing Fin AI Agent to Resolution Bot

If you previously used Resolution Bot, you will be able to access the previous Resolution Bot report from the banner at the top of the page.

To compare Fin AI Agent’s performance with your previous Resolution Bot performance, use the date filters to set a custom date range.

Set the date to begin on the day you first set Fin AI Agent live, and select the end date.

Now, you'll be able to analyze the impact of Fin AI Agent compared to the previous period in which Resolution Bot was being used. For example, if you've selected a date range of 31 days, you'll see how this compares to the previous 31 day period.

You can also build your own custom report using Fin AI Agent metrics.

Using the API

Fin AI Agent metadata is also available through the Conversation model in the API’s unstable version.

This means that any time your AI agent becomes involved in a conversation, you will see information related to the nature of this involvement including the source (e.g. Workflow name), resolution state outcome, customer rating, as well as content sources used by Fin.

In addition to just providing you with this information, you can now use the Conversations Search endpoint to search for conversations with these new attributes, allowing you to build your own custom reports with the API data as described here.

Resolution state is only provided for conversations after August 2, 2023. We plan to update this for older conversations soon.


💡Tip

Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts


Did this answer your question?