Skip to main content

How to run and review Evals inside a Release

Find, run, and review Evals inside a Release, add more Evals to it, and learn why Batch Test is unavailable there.

Written by Dawn Perrott

Use this article to find the Evals that apply to a Release, run them, add more Evals to the Release, and understand why Batch Test is unavailable inside a Release.

How to see which Evals apply to a Release

To see which Evals apply to a Release, open the Release and click the Evals tab. The tab lists Evals in two groups:

  • Changed in this release: Evals that were edited or added as part of this Release. An Edited label marks an Eval that already existed and was changed.

  • From Main: Evals that already exist in Fin Main and are available to run against this Release. Fin Main is your live production version of Fin.

Each Eval row shows its name, its run status, and a … menu. The run status can be Queued, Running (for example, 6/6 Running), or Not run.

The Evals tab of a Release named Login query improvements. The Changed in this release group contains one edited Eval, and the From Main group lists Evals with Queued, Running, and Not run statuses.

To hide or show a group, click its name. The Evals tab also shows a count of the Evals in each group.

The Evals tab of a Release with the Changed in this release and From Main groups collapsed, and an Add more Evals button below them.

How to run an Eval inside a Release

To run an Eval inside a Release, click the run button on the Eval's row in the Evals tab. A run runs the Eval's Simulations (realistic test conversations) against your Release's changes instead of your live Fin Main setup. While an Eval runs, its row shows a stop button, which you can click to stop the run.

To add more Evals to the Release, click Add more Evals below the two groups.

How to see all Evals while working inside a Release

To see all Evals while working inside a Release, select the Release in the switcher at the top left of the nav, then go to Fin AI Agent > Test > Evals. The Evals page lists every Eval with its name, result, tags, last run date, and who last ran it. You can search Evals by name, add filters, and use the run or stop button next to an Eval name.

The Evals page opened inside a Release, with a search box, an Add Filter button, and a table of Evals with columns for Eval name, Result, Tags, Last run date, and Last run by.


Evals in a Release FAQs

What is the difference between Changed in this release and From Main?

In the Evals tab of a Release, Changed in this release lists Evals that were edited or added as part of the Release, while From Main lists Evals that already exist in Fin Main.

Can I use Batch Test inside a Release?

Batch Test can't be used inside a Release because it is only available in Fin Main. Switch to Fin Main to use it.


💡Tip

Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts


Did this answer your question?