Batch test lets you simulate how Fin will respond to real customer questions—before those responses reach actual customers. It helps you check content coverage, debug unexpected results, and refine Fin’s behavior across brands, users, languages, and automations.

Batch Test is designed for all customers who use Intercom to support conversations – whether you’re an existing Fin customer or simply looking to explore its potential.

Go to Fin AI Agent > Test from the main navigation and choose how you want to add questions.

The options for adding questions for a batch test are:

These options allow you to ensure that Fin is tested against real, relevant customer queries before going live, helping you spot content gaps and optimize responses for different audiences and scenarios.

Each approach to generating question groups comes with its own advantages and ideal use cases.

You can automatically generate up to 50 questions based on your most recent customer conversations (30-90 days), so the question set will accurately represent what customers are asking about right now.

This is a quick way to get a list of questions that accurately reflects current customer needs.

This method uses AI topics to generate topic-specific question sets based on real customer queries. It’s particularly useful when you want to focus on specific subject areas and understand how well they’re being handled.

Common use cases include:

If you don’t want to generate your questions from existing data, you can copy and paste a list of questions or add them one by one. This method gives you full control over the exact content and phrasing with no dependency on historical conversations.

Common use cases include:

This method lets you upload a set of questions generated elsewhere, such as from another support platform. You simply need a .csv file with a single column containing up to 50 test questions.

Common use cases include:

By selecting the right question-generation method for your needs, you can evaluate and refine Fin’s performance against relevant customer scenarios, helping you identify gaps and improve responses over time.

Your test questions will automatically start running using a generic preview user to access all content and trigger all automations without targeting. To configure these test settings, click the Manage dropdown at the top of the page and then select Settings.

Choose how to test Fin by simulating users or audiences. You can choose to test Fin as:

If you've set up multiple brands on your workspace, you can select the brand you would like to test. For example, the content might be different for this brand so the Fin response can be tested to ensure these nuances are being picked up.

Click Confirm to re-run the test with these settings.

Once you run a test, you can view Fin’s generated response for each question.

Use the "Evaluate answer" panel to see:

While you can’t directly edit automations triggered in a Batch test, the panel links out to the relevant configuration screen for you to review or quickly make adjustments.

To add a new question, go to Fin AI Agent > Test from the main navigation. Then, click the + add question button. A dropdown menu will appear, offering you various options for how you'd like to add your questions.

To help you get accurate responses in the correct language, Batch test makes it easy to check and update your Language support and Real-time translation settings during testing.

If you’re seeing a response in a different language than expected—for example, if a question in Russian gets a reply in English—it provides clear messages in yellow explaining why this might be happening and how to fix the issue.

Once you enable the appropriate settings and refresh the answer, Fin's responses will appear in the correct language.

You might need to enable just one or both settings, depending on your current configuration. Don’t worry—you can turn them on simultaneously and come back to this screen.

Review Fin’s responses and assess whether they meet expectations, checking for factual accuracy, appropriate tone, and whether the correct data was accessed or the right procedures were triggered.

If you select Acceptable, you can add an internal note to reference later when improving Fin’s responses. These notes will be included in the downloadable CSV report for the test.

If you select Poor, choose a reason for your rating from the list. This is where you need to conduct root cause analysis to correctly determine why Fin’s answer did not meet expectations so you can make immediate improvements.

The possible choices are as follows:

Read below for indicators to help you conduct root cause analysis for a Poor-rated answer.

Use when the answer is factually wrong because Fin relied on an irrelevant, outdated, or incomplete source.

Indicators:

Use when the question is ambiguous or missing key details and Fin answered prematurely instead of asking for context.

Indicators:

Use when the underlying content is correct, but Fin interpreted, combined, or ordered it the wrong way.

Indicators:

Use when the response doesn’t match your brand voice or the situation’s sensitivity.

Indicators:

Use when verbosity doesn’t fit the intent or channel.

Indicators:

Use when the response language doesn’t match the customer or region.

Indicators:

Use when the issue doesn’t fit the above categories (leave a clear note).

Examples:

Once you’ve selected the reason that best matches your rating, click Improve this answer to view dynamic recommendations tailored to the root cause you selected for that specific answer.

Adding or revising guidance to:

Adding a snippet for:

Creating or updating existing articles for:

Updating supported languages to:

This example demonstrates a hypothetical root cause analysis and solution for a Poor-rated question.

Question: “How do I add a new user without paying?”

What happened: Fin answered with generic steps to invite a user, but missed the plan-based billing nuance.

Root cause analysis: Answer details show Fin relied on the “Invite user” article which does not cover plan/billing specifics for free vs paid seats (knowledge gap).

Select reason: "Didn't use the correct content."

Suggestions: Update “Invite user” article to include relevant information; Add guidance to ensure Fin asks clarifying questions to identify the relevant plan type before answering.

Filter a test by Answer status:

Filter a test by Answer rating:

Make bulk updates using the checkboxes on the left of the question. This allows you to bulk update Q+A pairs to download certain questions, delete those questions, create a test group, or update the answer associated with that question.

You can use test groups to organize and save up to 50 questions and responses in the testing area. This is the maximum number you can upload at once, making it a handy way to group related questions for easier review and reuse. Each test group retains the settings you used during testing—like simulating a specific user—so you can re-run tests with the same configuration anytime.

Click Manage at the top of the page to select the option + Create new group.

From there, you have several flexible options to populate your new test group with questions:

Click Manage to rename a group or delete a test group.

Click the name of your test group to create additional groups, or select a different group you've previously saved.

Test groups are especially useful for organizing questions by topic. For example, if you’ve tested and reviewed a batch of insurance claim questions, you can save them as a group labeled “Insurance Claim Questions.” This makes it simple to revisit, rerun, or evaluate that content later.

They’re also great for managing team collaboration. Since Batch test is a workspace-level feature, using test groups lets teammates keep their test runs separate. Instead of deleting previous tests to make space, you can save them into groups to preserve everyone’s work.

A CSV file can be generated, compiling all questions, answers, user-applied ratings, and the sources utilized for each response. Simply click Manage at the top of the page and then select Get CSV report.

This is great for sharing results with your wider team for collaboration, or visibility for senior leaders to review.

Fin Vision (image recognition) isn’t supported in the Batch test section yet.

No, the Fin AI Agent > Test page is free to use and you won't be charged for AI answers generated through the Batch test. 👌

Yes, you can generate test questions automatically. However, for the "Generate from inbox" option to appear, you must have a minimum of 1 conversation in the last 90 days in your workspace.

No. Batch test is strictly for quality assurance—ratings help you identify areas to improve, not retrain Fin.

Yes. Batch test checks and flags any missing language or translation settings so you can resolve them easily.

Yes. Batch test allow you to select a user/lead in your workspace and see how Fin would respond based on their specific user attributes.

Resetting the test will allow you to choose another batch, either from conversation history or an upload. Re-running the test will re-generate answers based on any content changes or answer ratings you provided in the batch so you can continue to refine the performance.

The "Generate from inbox" option is disabled when there aren't enough conversations or conversations with relevant topics in your inbox.