This article explains when to use Fin's three testing tools — Simulations, Batch tests, and Previews.

Quick reference: which tool does what

	Simulations	Batch tests	Previews
What it is	Automated, AI-powered end-to-end testing of a single Procedure. AI acts as a simulated customer and judges the outcome against success criteria you define.	A question-and-answer testing tool for validating Fin's responses across up to 50 questions at once. Tests how Fin answers across content, guidance, users, languages, and automations.	A manual, interactive testing panel available throughout Fin's training areas. You type messages and see Fin's response and event log in real time.
Best for	Validating complex Procedure logic end-to-end before going live. Catching regressions after Procedure updates. Testing edge cases and branching paths.	Checking content coverage across many questions at once. Diagnosing gaps in knowledge, guidance, or automations. Pre-launch QA across audiences, brands, and languages.	Quick manual spot checks while building. Verifying how a specific change — to content, guidance, attributes, escalation, or a Procedure — affects Fin's response in the moment.
Works with	Procedures only.	All customers using Fin — whether using Procedures, Tasks, or just knowledge content.	All of Fin's training areas: Content, Guidance, Attributes, Escalation, Procedures, and Deploy.
Testing mode	Automated — AI generates the customer-side of the conversation and evaluates the result with a pass/fail judgment.	Automated — Fin generates a response to each question. You review and rate answers manually (Good / Acceptable / Poor).	Manual — you type messages yourself and observe Fin's response and event log.
Creates a real conversation?	No — runs entirely in the background. No inbox conversation is created.	No — simulation only. Does not create inbox conversations or affect customer data.	Yes — Preview conversations appear in your inbox, but are excluded from reporting.
Hits live APIs?	No — does not call live external APIs (e.g. Shopify, Stripe). Tests logic only.	No — data connectors use configured test/sample payloads. No live API calls.	Yes — can access real user data and live integrations when testing as a specific user or lead.
Saves & reruns	Yes — simulations are saved and can be rerun whenever the Procedure is updated. Supports Run all.	Yes — test groups (up to 50 questions each) can be saved, rerun, and shared across teammates.	No — one-off manual conversations. Not saved or rerunnable.
Pass/fail judgment	Yes — AI evaluates the conversation against success criteria and assigns Pass or Fail with full reasoning.	No — you manually rate each answer as Good, Acceptable, or Poor and choose a root cause.	No — no automated judgment. You assess the result yourself.
Usage limits	Yes — monthly run limit based on workspace conversation volume (50–2,500/month). Included with Procedures at no extra charge.	No charge. Up to 50 questions per test group.	No limits.
Availability	Procedures only — requires managed availability access.	Generally available to all Fin customers with a full seat and All conversations permission.	Generally available to all Fin customers.

When to use each tool

Use Simulations when

You need to validate a Procedure end-to-end before setting it live.
You want automated, repeatable tests that catch regressions after every update.
Your Procedure has branching logic, Conditions, or sub-procedures — and you need a separate test case per path.
You want AI to judge the outcome against defined success criteria (Fin reply, attribute values, data connector calls, instruction outcome).
You need to test time-sensitive logic (e.g. refund windows) by controlling the simulation time and data values.

Use Batch tests when

You want to validate Fin's answers across a broad set of questions — not just one scenario.
You're doing pre-launch QA across content, guidance, audiences, languages, or brands.
You want to diagnose content gaps by seeing which sources Fin pulls from for each answer.
You want to test questions generated from real past conversations or AI topics.
You need a downloadable CSV report to share results with your team or senior leaders.

Use Previews when

You're actively building and want to spot-check a change immediately — to content, guidance, an attribute, an escalation rule, or a Procedure.
You want to manually walk through a conversation as a specific user, audience, or in a specific language.
You need to see Fin's event log to understand exactly what it applied (personality, guidance, tasks, content sources).
You want to test Fin Vision by sending an image or screenshot.
You're validating a Procedure flow and want to see the full customer and teammate inbox experience (including downstream automations).

FAQs

What's the difference between Simulations and Previews?

Previews are manual and interactive: you type messages yourself and watch Fin respond in real time. They're great for quick spot checks while building.
Simulations are automated: AI acts as the customer, runs the full conversation, and judges the outcome against success criteria you set.
Simulations don't create inbox conversations; Preview does. Use Simulations when you need to validate Procedure logic at scale. Use Preview for in-context manual testing.

What's the difference between Simulations and Batch tests?

Batch tests check how Fin answers a broad set of questions — testing content coverage, guidance, and automations across many scenarios at once. You rate answers manually.
Simulations test a single Procedure end-to-end with automated AI judgment.
Batch tests work with all Fin setups. Simulations are exclusive to Procedures.

What's the difference between Batch tests and Previews?

Batch tests are structured, repeatable QA runs across up to 50 questions — useful for pre-launch checks and diagnosing content gaps.
Previews are unstructured and manual — one conversation at a time, useful for instant feedback while building.
Batch test results are saved and can be exported to CSV. Preview conversations appear in your inbox but aren't saved as test runs.

Do Simulations call live APIs or affect real data?

No. Simulations do not call live external APIs (like Shopify or Stripe) and do not affect any real data. They test logic only. If you need to test with real API calls, use Preview and select a specific user with real data in their external system.

Will I be charged for Simulations or Batch tests?

Simulations are included with Procedures at no extra charge. Batch tests are also free to use (you won't be charged for answers generated through Batch test).

Note: Simulations do have a monthly run limit (50–2,500 per month depending on conversation volume), which resets on the first of each month.

Can I use Batch tests to test a Procedure?

Batch tests show whether a Procedure would have been triggered for a given question, but they don't execute the Procedure or evaluate its end-to-end logic. For full Procedure validation (including branching, data connector calls, and instruction outcomes) use Simulations.

Which testing tool should I use before going live?

Use all three in combination: Preview for quick manual checks as you build; Simulations to validate Procedure logic end-to-end with automated pass/fail results; Batch tests to confirm Fin's answers across a broad set of real customer questions.

Running Simulations before every Procedure launch is strongly recommended.

Key facts to remember

Simulations are exclusive to Procedures. They cannot be used with Tasks, Workflows, or general content testing.
Batch tests work for all Fin customers — whether using Procedures, Tasks, or just knowledge content.
Preview conversations appear in the inbox but are excluded from reporting.
Only Preview can hit live APIs. Simulations and Batch tests are safe, sandboxed environments.
Simulation run limits are per workspace, not per teammate, and reset monthly.
Answer ratings in Batch test do not train Fin — they're for human review and root cause analysis only.
In Simulations, if a test passes but the success criteria are too rigid (e.g. requiring a specific question Fin found itself), the test may still fail. Focus success criteria on outcomes, not intermediate steps.

💡Tip

Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts

Batch test Fin AI Agent

Fin Procedures explained

Use Fin previews

Run Simulations for Fin Procedures

Procedures vs. Tasks vs. Workflows