Improving support quality while embracing AI: Strategies from Intercom and Klaus

AI may be transforming customer support, but a successful approach takes more than just a flick of a switch. In our latest webinar, we’ve covered how you can scale an AI-first support strategy while maintaining quality and consistency.

From swiftly handling common queries to assisting human agents in tackling complex issues, generative AI chatbots have swooped in and changed the game of customer support in just a matter of months. They instantly reduce support volume and response times, freeing up agents from the dreaded hamster wheel of repetitive queries to focus on the most high-impact parts of their jobs. Take our new AI chatbot, Fin. We were its first customers, and now, 70% of our inbound support conversations start with this AI bot experience, and Fin is able to resolve 33% of those queries straight off the bat. That’s a huge chunk of our workload solved *snaps fingers*just like that.

“The challenge lies in blending the efficiency of AI with the human touch that customers value”

AI also helps teams review the quality of the conversations by building better review samples, automating QA processes, and enabling better reporting:

“Since doing things automatically, you can get a hundred percent coverage with things like sentiment analysis and grammar analysis. AI-enriched analytics and reporting gives you much deeper insights into the big picture.”

Mervi Sepp Rei, Head Of ML and Data at Klaus

That is, of course, if AI is properly implemented. The challenge lies in blending the efficiency of AI with the human touch that customers value. Poorly executed integrations and inconsistencies in AI responses can result in bad experiences and frustration, which defeats the whole purpose.

In order to avoid it, support leaders must anticipate and address these potential issues through thoughtful implementation. That’s why we recently partnered with Klaus, the customer service quality management solution, to host a webinar on how to build a modern, AI-first customer support strategy. We were joined by:

• Bobby Stapleton, Director Of Customer Support at Intercom
• Diogo Costa, Customer Success Team Manager at Klaus
• Mervi Sepp Rei, Head Of ML and Data at Klaus
• Sean Reid, Customer Support Manager at Intercom

Here’s a brief recap on how to do it:

1. Careful planning and methodical execution

Deploying AI isn’t just plug and play, you can’t just click a button and be ready to go. You’re integrating with a spectrum of existing systems – both modern and legacy – setting up routing and handovers from bots to human agents, changing processes and org structures. A good way to set yourself up for success and reap the benefits of AI is to invest in a good implementation strategy. And sometimes, that can involve a phased deployment:

“We started by doing one step at a time. We realized Fin could help us in the out-of-office hours, something we were not doing until that point. We approach it as a test, ‘Let’s see how we can use Fin as an extra pair of arms to do support here.’”

Diogo Costa, Customer Success Team Manager at Klaus

By testing AI’s capabilities in specific, low-risk scenarios like out-of-office hours, organizations can alleviate burdens and deliver immediate value to customers. With Klaus, that translated to a resolution of 17% of out-of-office hours interactions, and they plan to increase that to one-third by the end of the year. And, of course, no decent implementation strategy is complete without an iterative approach for monitoring and continuously improving chatbots as you go.

2. Creating a strong knowledge base

“The biggest one is having a really comprehensive, up-to-date, and thorough help center or knowledge base. If what you have is wrong or out of date, it’s not going to give the right information to your customers. (…) We had all of our R&D engineers look at specific sections of the help center and make sure they were correct and if there was anything missing that they thought should be included.”

Bobby Stapleton, Director Of Customer Support at Intercom

AI chatbots like Fin work by consuming information in your knowledge base or help center to immediately offer your customers the answers they’re seeking. This means that if you want them to be accurate and trustworthy, you need to have well-written, well-structured help content that covers just about everything you want the bot to answer.

Before deploying it, make sure you audit your help center to ensure all of the information is accurate and up-to-date, optimize and update existing content, and create new content where needed. It takes a pretty significant upfront effort and some ongoing maintenance, but it’s going to pay dividends in the long run.

3. Keeping the human touch

Newer versions of these bots are pretty advanced, but they still can’t feel or display real emotions. They can’t empathize with a distressed customer or offer a sympathetic, timely discount and a sincere apology.

“Something that our customers really like about our service is the human aspect. Historically, it’s always been a plus. So, to lose that or put that at risk was a first concern.”

Diogo Costa, Customer Success Team Manager at Klaus

While AI can undoubtedly elevate efficiency, it’s the technology’s harmonious integration with human expertise and sensibility that offers the best of both worlds. By approaching AI as a supportive counterpart to human agents, businesses can maintain genuine customer relationships while reaping the rewards of AI’s capabilities. Not only can properly implemented bots help support agents focus on cultivating customer loyalty with top-tier support, but they require continuous human monitoring and intervention to ensure accurate interactions with customers.

4. Evolving QA practices for modern, AI-driven support

“Before, a traditional approach to quality assurance was basically just people-focused. You’re QAing the person within the structures you’ve created with your product and your processes and seeing if that person follows that correctly. With AI, you have to QA the entire customer journey.” Sean Reid, Customer Support Manager at Intercom

Simply put, your whole QA strategy will need a refresh. After all, you’re adding a lot of new, dynamic motions and complexities to the customer experience. As AI gets deployed, it’s important to shift to a more comprehensive approach that considers the entire customer journey – encompassing product limitations, process efficiencies, and the efficacy of AI to human handoffs. At Intercom, this translated into breaking our QA scorecard into three sections:

• People: The old-school way of making sure our specialists are doing the right thing;
• Processes: Looks at whether the processes we have in place are correct – this also looks at Fin’s handover to our specialists;
• Product: What can we do to make our product better for the customer experience? This also looks at Fin from a product point of view.

In order to make sure it’s all running as smoothly as possible, a key component is monitoring it and understanding how and when to intervene. After all, AI chatbots aren’t foolproof, especially if the content foundation isn’t solid. For example, if a conversation where Fin intervened received a negative CSAT score, what caused it? Maybe that knowledge base article needs a refresh.

“It’s very clever, but at the same time, a generative tool can go rogue. There are hallucinations; you have to monitor it. Monitoring what it does, understanding how you can interfere at a specific time and how it performs over time becomes way more critical. We were obviously very excited to start using Fin, but we knew our QA needed to adapt to this thing. We completely changed our core data pipeline so it treats Fin as a generative bot that you want to review because it acts as a person.” Mervi Sepp Rei, Head Of ML and Data at Klaus

Collaboration between AI tools, QA teams, and human agents is crucial. Plus, embracing automation for routine QA tasks like building samples or doing quality checks offers the potential to scale the process across the entire spectrum of customer interactions.

“It really comes down to checking what they’re doing. That involves manual things, but because they’re so abundant, it’s hard to check them manually. So, we do quality checks automatically. (…) In all the conversations, we see that Fin said something – what did it do? And then, this is surfaced in the reporting and you can understand where it is involved, what does it do, and it gives much, much deeper insight.” Mervi Sepp Rei, Head Of ML and Data at Klaus

5. Making space for new and improved roles

Many are concerned that these changes are going to drive us out of jobs. And while some support roles are going to change, we’ve seen how this new technology is also creating the need for new jobs and roles to appear. In the past few months at Intercom, we’ve hired a quality assurance manager, a process improvement manager, and a conversational designer. Even beyond that, we’ve seen just how existing roles are evolving:

“Yes, this new job was created, but it’s advancing and empowering your current support specialists as well. (…) Obviously, their bread and butter is helping our customers, but they do so much more. They QA new products, they write help center articles, they’re talking to our customers on our Intercom community.” Sean Reid, Customer Support Manager at Intercom

As businesses and technology evolve, so too must their support strategies. But implementing AI-driven support isn’t merely about adopting cutting-edge technology; it’s about orchestrating a strategy where chatbots and humans play to their unique strengths. By leveraging AI to streamline operations and optimize the customer experience while ensuring the authenticity of human connections, you can take your support to new heights.