Skip to main content

How Fin Vision understands images

Fin Vision instantly analyzes images to diagnose issues, provide solutions, or capture key details that move the conversation forward.

Beth-Ann Sher avatar
Written by Beth-Ann Sher
Updated today

Fin Vision is a built-in capability of Fin AI Agent that allows it to analyze and understand images sent by customers - screenshots, photos, documents, and more - directly within conversations via chat or email.

There’s no need to enable or configure anything, and there’s no additional cost.

Fin Vision helps:

  • Diagnose issues faster

  • Eliminate the need for lengthy customer explanations

  • Extract and understand visual content like error messages, receipts, product defects, and more

How Fin Vision works

Fin Vision uses multimodal large language models (LLMs) to understand images. When a customer sends an image, Fin processes it using a vision-enabled LLM to generate a structured textual description. This transcription includes:

  • Extracted text (OCR)

  • UI elements and associated labels

  • Reference numbers, product details, and key highlights

  • Context-aware insights derived from the image

This description is then added to the chat history, which allows Fin to incorporate visual context into its responses.

With this understanding, Fin can:

  • Search your knowledge base more effectively

  • Resolve Tasks that depend on visual information

  • Provide relevant, actionable answers - just like it would from a customer's written input

Note:

  • Fin does not train on or analyze images within your support content (e.g., images embedded in articles). It only processes images actively sent by customers during conversations.

  • Fin currently can't generate or send images when providing AI answers.

  • Fin currently can't read ALT text in images.

Ways to use Fin Vision

Industry

Example use cases

FinTech

  • Error troubleshooting: Screenshots of failed transfers or login issues help Fin provide targeted support.

  • Fraud alert review: Fin helps identify phishing screenshots or suspicious activity.

SaaS

  • Troubleshooting UI bugs: Customers share screenshots of errors or unexpected UI behavior; Fin extracts error messages and provides fixes.

  • Onboarding help: Fin can assist customers through unclear UI flows based on shared screenshots.

  • License verification: Fin reads license keys or account numbers from uploaded invoices.

ecommerce

  • Return/refund validation: Customers upload images of damaged or incorrect products; Fin evaluates eligibility based on Task instructions.

  • Shipping issues: Customers share photos of packaging or contents; Fin determines missing items or packaging damage.

  • Invoice processing: Fin extracts order numbers and dates from receipts or packing slips.

Gaming/Gambling

  • Bug reporting: Players send screenshots of glitches or crashes; Fin interprets the visuals and logs issues.

  • Withdrawal issues: Customers upload screenshots of failed transactions; Fin pulls timestamps, amounts, and transaction IDs.

  • Bet slip verification: Fin reads and confirms bet slip details from uploaded images.

Maximizing Fin Vision

To get the most from Fin Vision, combine it with Fin’s other features:

Use with Fin Guidance

1. Reading and Interpreting Receipts

Scenario:
A customer uploads a photo of a purchase receipt and asks, "Can you help me with a refund for this item?"

How Fin Vision and Guidance Work Together:

  • Fin Vision extracts key details from the image, such as the item name, purchase date, and total amount.

  • Fin Guidance provides custom instructions to Fin, such as:
    "If a customer asks about a refund and uploads a receipt, check that the purchase date is within 30 days. If so, guide them through the refund process. If not, politely explain the refund policy."

Result:
Fin can automatically verify eligibility and respond with the correct next steps, referencing the extracted receipt details.

2. Bug Reporting with Screenshots

Scenario:
A user submits a screenshot showing an error message in the app and says, "I'm getting this error—what should I do?"

How Fin Vision and Guidance Work Together:

  • Fin Vision analyzes the screenshot to identify the error code or message.

  • Fin Guidance instructs Fin to:
    "If an error code is detected in a screenshot, search the help center for that code and provide the relevant troubleshooting steps."

Result:
Fin can quickly match the error to known issues and deliver targeted support, reducing back-and-forth.

3. Device Identification for Support

Scenario:
A customer uploads a photo of their device and asks, "Is my device compatible with your service?"

How Fin Vision and Guidance Work Together:

  • Fin Vision identifies the device make and model from the image.

  • Fin Guidance tells Fin:
    "If a device model is recognized, check the compatibility list. If compatible, confirm and share setup instructions. If not, explain the limitations."

Result:
Fin provides a personalized answer based on the actual device, improving accuracy and customer satisfaction.

4. Document Verification

Scenario:
A user uploads a photo of their ID for account verification.

How Fin Vision and Guidance Work Together:

  • Fin Vision extracts the name, date of birth, and document type.

  • Fin Guidance instructs Fin to:
    "If the uploaded document is a valid ID and matches the account details, proceed with verification. If not, request a clearer image or additional documentation."

Result:
Fin can automate parts of the verification process, reducing manual review.

Guidance Strategies

  • Conditional Logic: Fin Guidance can set rules based on what Fin Vision detects (e.g., "If the receipt is older than 30 days, do X").

  • Fallbacks: If Fin Vision cannot extract needed information, Guidance can instruct Fin to ask the customer for clarification or a better image.

  • Personalization: Guidance can tailor responses based on visual context, making interactions feel more human and relevant.

Limitations and Ongoing Improvements

While these examples are based on current documented use cases, there is ongoing work to expand the library of real-world scenarios and best practices. Some advanced or niche examples may not yet be fully documented.

FAQs

What image formats does Fin Vision support?

Fin Vision supports standard image formats including JPG, PNG, and GIF files shared by customers.

How does Fin handle privacy and sensitive information in images?

Fin is designed with privacy in mind. The vision models are explicitly prompted not to extract any personal or sensitive information from images, such as credit card numbers, CVVs, or identification details. Additionally, images are stored temporarily and are automatically deleted after a short period.

Does Fin store images?

Images are temporarily stored in a secure cloud environment and automatically deleted after a short period.

Do customers need to send images in a certain way?

No, customers can upload or paste images into the chat or email. Fin handles the rest.

Can customers send multiple images?

Yes, Fin will analyze the latest five images individually and use the context to inform responses.

Does Fin generate or send images?

No, Fin cannot generate images when providing AI responses. It can only analyze images sent by customers.

Does Fin Vision support multiple languages?

Yes, Fin can extract text from images in many languages, though accuracy depends on clarity and complexity.

Can I turn off Fin Vision?

No, Fin Vision is built-in and cannot be disabled. It operates automatically as part of Fin’s understanding of conversations.

💡Tip

Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts

Related Articles
Fin AI Agent explained
Test Fin AI Agent
Best practices for Fin Tasks
How to automatically escalate conversations
Use Fin previews
Did this answer your question?