A custom scorecard defines what good looks like for your team by explicitly setting the criteria you care about - such as accuracy, tone, or policy adherence. Scorecards work alongside Monitors: a Monitor defines which conversations get reviewed, and a scorecard defines how each one is evaluated. Monitors currently only evaluate Fin AI Agent conversations, not human teammates.

You can have multiple scorecards for different Monitors. Choose which scorecard to associate with a Monitor from the Monitor setup screen:

Go to Fin AI Agent > Analyze > Monitors and click Scorecards. You can use the out-of-the-box Fin Quality Scorecard or create your own by clicking + New scorecard:

Start by adding scorecard criteria. First, click New scorecard > + Criteria > Create new.

When creating a new attribute, work through the following steps:

Give the criteria a short, clear name (for example, Sentiment or Answer accuracy). This name appears in reports and will be used as a reference.

Add a clear description explaining what the criteria checks and how it should be scored. The description is the prompt the AI uses to score this criteria, and the more precise it is, the more accurately AI will evaluate conversations. It also helps human reviewers apply the same criteria consistently.

Decide whether the criteria should be automatically scored with AI, or manually scored by human reviewers. You can mix AI-scored and human-scored attributes within the same scorecard.

Add the possible rating values a reviewer or AI can select (for example: Good, Okay, Poor). Each attribute must have at least two rating options. For each rating option, you will:

The score you assign determines how that rating contributes to the overall review score.

You can toggle Include in review score on or off.

In this example, a scorecard attribute has been created to evaluate escalation ease:

You can automate the entire QA process for a scorecard by toggling on Auto-review scorecard.

When enabled:

After adding scorecard criteria, configure how they affect the overall review result.

You can mark an criteria as Critical. If a critical criteria receives a failing rating, the entire review fails:

Critical criteria are useful for non-negotiable standards such as compliance requirements, safety or policy adherence, and escalation handling.

Each criteria can be assigned a weight to define its relative importance.

Weights only apply to criteria included in the review score. Use weights to reflect what matters most — for example, a higher weight on Accuracy than Efficiency if correctness is more important than speed.

You can define a pass threshold — the minimum overall score required for a review to be considered passing. For example, if the pass threshold is 80%, any review scoring below 80% is marked as failed.

This is evaluated after weighted scoring, provided no critical criteria has already failed the review.

Here is an example of how three criteria combine into a final score:

Overall score = (100x60 + 50x30 + 100x10) / (60+30+10) = 85%

Once reviews are completed, scores are visible in both the conversation list and within each conversation.

In Monitor, the conversation list shows the overall review score (percentage or Fail) alongside the individual criteria ratings as columns. This makes it easy to scan performance across conversations and spot failures or low scores.

When you open a conversation and go to the Score tab, you can see the assigned scorecard, review status, overall score, and the selected rating for each criteria. This view shows exactly how the final score was determined.