{"id":30878,"date":"2024-11-14T16:06:32","date_gmt":"2024-11-14T16:06:32","guid":{"rendered":"https:\/\/www.intercom.com\/blog\/?p=30878"},"modified":"2024-11-14T16:06:32","modified_gmt":"2024-11-14T16:06:32","slug":"intercom-vs-zendesk-two-ai-agents-put-to-the-test","status":"publish","type":"post","link":"https:\/\/www.intercom.com\/blog\/intercom-vs-zendesk-two-ai-agents-put-to-the-test\/","title":{"rendered":"Intercom vs Zendesk: Two AI agents put to the test"},"content":{"rendered":"

We all know that generative AI is transforming the customer service industry.<\/p>\n

AI agents are already handling customer queries with impressive accuracy, and the teams using the right AI solutions are seeing remarkable results. They’re resolving more issues faster and delivering better customer experiences, which is allowing human support agents the freedom to focus on more complex, high-value interactions.<\/p>\n

However, identifying the right AI solution is not easy amid all the noise. Our research shows there are significant performance gaps when it comes to resolution rates, accuracy, and quality between different AI agents on the market.<\/p>\n

We\u2019ve repeatedly tested our Fin AI Agent<\/a> against competitors\u2019 offerings to ensure it performs optimally in every way. Here, we\u2019ll show you how Fin compares to Zendesk\u2019s AI agent and walk you through the research process to give you an in-depth understanding of why Fin is the superior choice.<\/p>\n

Fin is the best AI agent on the market \u2013 with stats to prove it<\/h2>\n

Let\u2019s start with some numbers.<\/p>\n

When we put Fin head-to-head against Zendesk’s AI agent, the difference isn\u2019t just noticeable \u2013 it\u2019s remarkable. Three things in particular stood out:<\/p>\n

<\/p>\n

In 80% of cases, Fin provided better answers across the board, demonstrating superior performance in accuracy, completeness, and overall quality. This isn’t just about getting the answers right; it’s about delivering the kind of experience that builds customer trust and loyalty.<\/p>\n

“Fin can handle twice the number of complex questions Zendesk\u2019s AI agent can, transforming what’s possible with automated support”<\/p><\/blockquote>\n

Fin is also much more capable at handling complexity. Unlike Zendesk’s AI agent, which defaults to basic responses when faced with challenging queries, Fin maintains natural conversations by asking clarifying questions. And now, Fin can answer more types of questions with the recent addition of actions<\/a>. This sophisticated capability means Fin can handle twice the number of complex questions Zendesk\u2019s AI agent can, transforming what’s possible with automated support.<\/p>\n

Perhaps most impressively, when dealing with questions that require pulling information from multiple sources \u2013 the kind of query that typically needs human intervention \u2013 Fin achieves a 96% answer rate, significantly outperforming Zendesk’s 78%. For support teams, this means more queries resolved automatically, faster response times, and happier customers.<\/p>\n

These numbers are compelling \u2013 but how did we get to them? Here\u2019s an overview of the research process we followed.<\/p>\n

How we compared Fin to Zendesk\u2019s AI agent: A look at our evaluation process<\/h2>\n

Step 1: Setting the stage<\/h3>\n

To evaluate the AI agents in an unbiased manner, we needed a completely neutral dataset of help articles and relevant questions that we knew were grounded in the articles.<\/p>\n

Using ChatGPT 4, we created a fictional bed and breakfast website with 48 comprehensive articles, all of which we loaded into Fin and Zendesk\u2019s AI agent, ensuring a fair playing field.<\/p>\n

We also generated 200 customer questions based on the 48 articles. Some were straightforward, while others required piecing together information from multiple articles.<\/p>\n

We asked all 200 questions to both Fin and Zendesk’s AI agent.<\/p>\n

Step 2: Checking the outputs for hallucinations<\/h3>\n

Before we started judging the outputs, we checked for any made-up information \u2013 hallucinations \u2013 in the responses. We found that there was no statistical difference in the hallucination levels between the two AI agents.<\/p>\n

Step 3: Judging the answers<\/h3>\n

We used four advanced AI models (Anthropic Claude 3 Opus, GPT-4, GPT-4 Turbo, and GPT-4 Omni) to act as impartial judges. These \u201cjudges\u201d had access to the articles and question bank, and were instructed to vote on the answers provided by both Fin and Zendesk\u2019s AI agent for given questions while considering the articles as the source of truth.<\/p>\n

To determine a winner, we applied the Elo rating system<\/a>, which calculates a score based on which AI agent delivered the better answer, according to the AI judges. Over hundreds of such \u201ccompetitions,\u201d a clear winner emerged.<\/p>\n

<\/p>\n

The results were clear: when pitted side-by-side, Fin\u2019s answers are almost always better than Zendesk AI agent\u2019s. <\/span><\/p>\n

Step 4: Digging into the details<\/h3>\n

We wanted to dig deeper into what specifically made Fin\u2019s answers better. So, we looked more closely at how Fin outperformed Zendesk AI agent in the following areas:<\/p>\n