Evaluation metric example RAG document relevance
AI evaluation in n8n This is a template for n8n's evaluation feature. Evaluation is a technique for getting confidence that your AI workflow performs reliabl...
🔌 Nodes & Integrations Used (12)
📖 Implementation & Setup Guide
AI evaluation in n8n
This is a template for n8n's [evaluation feature](https://docs.n8n.io/advanced-ai/evaluations/overview).
Evaluation is a technique for getting confidence that your AI workflow performs reliably, by running a test dataset containing different inputs through the workflow.
By calculating a metric (score) for each input, you can see where the workflow is performing well and where it isn't.
How it works
This template shows how to calculate a workflow evaluation metric: **retrieved document relevance** (i.e. whether the information retrieved from a vector store is relevant to the question).
The workflow takes a question and checks whether the information retrieved to answer it is relevant.
To run this workflow, you need to insert documents into a vector data store, so that they can be retrieved by the agent to answer questions. You can do this by running the top part of the workflow once.
The main workflow works as follows:
- We use an evaluation trigger to read in our dataset
- It is wired up in parallel with the regular trigger so that the workflow can be started from either one. [More info](https://docs.n8n.io/advanced-ai/evaluations/tips-and-common-issues/#combining-multiple-triggers)
- We make sure that the agent outputs the list data from the tools that it used
- If we’re evaluating (i.e. the execution started from the evaluation trigger), we calculate the relevance metric using AI to compare the retrieved documents with the question
- We pass this information back to n8n as a metric
- If we’re not evaluating we avoid calculating the metric, to reduce cost
⚡ How to import this workflow into n8n:
- Click the Direct Download JSON button above to save
Evaluation metric example_ RAG document relevance.json. - Log in to your n8n Cloud or self-hosted n8n instance.
- In your workflow canvas, click the menu in top-right and select Import from File (or paste the JSON directly).
- Configure your credentials (e.g. API keys, database URLs) in the corresponding node settings.
- Click Execute Workflow to test and switch to Active!
Related AI Agents Workflows
View All in AI Agents →5 ways to process images PDFs with Gemini AI in n8n
How it works Many users have asked in the support forum about different methods to analyze images and PDF documents with Google Gemini AI in n8n. This workfl...
7 AI Sales Assistant
Production-ready n8n automation template for 7 AI Sales Assistant built by AhiXLight. Seamlessly connects Airtop Tool, Slack, Airtop with 24 configured nodes.
AAVE portfolio professional AI agent Telegram Email GPT 4o
A next-generation AI-powered DeFi health monitor that tracks wallet positions across Aave V3 using GPT-4o and LangChain. It delivers human-readable reports v...
Access control for AI agents RBAC using Airtable and Telegram
Purpose This workflow allows granular control over the access to tools connected to AI Agents (including Multi-Agent setups) using Role Based Access Control....