FM
FlowMarket
Video & Media Advanced ⚡ 21 n8n Nodes

Narrating over a video using multimodal AI

This n8n template takes a video and extracts frames from it which are used with a multimodal LLM to generate a script. The script is then passed to the same ...

🔌 Nodes & Integrations Used (7)

Lm Chat Open Ai HTTP Request Convert To File Google Drive Edit Image OpenAI Chain Llm

📖 Implementation & Setup Guide

This n8n template takes a video and extracts frames from it which are used with a multimodal LLM to generate a script. The script is then passed to the same multimodal LLM to generate a voiceover clip.

This template was inspired by [Processing and narrating a video with GPT's visual capabilities and the TTS API](https://cookbook.openai.com/examples/gpt_with_vision_for_video_understanding)

How it works * Video is downloaded using the HTTP node. * Python code node is used to extract the frames using OpenCV. * Loop node is used o batch the frames for the LLM to generate partial scripts. * All partial scripts are combined to form the full script which is then sent to OpenAI to generate audio from it. * The finished voiceover clip is uploaded to Google Drive.

Sample the finished product here: [https://drive.google.com/file/d/1-XCoii0leGB2MffBMPpCZoxboVyeyeIX/view?usp=sharing](https://drive.google.com/file/d/1-XCoii0leGB2MffBMPpCZoxboVyeyeIX/view?usp=sharing)

Requirements

* OpenAI for LLM * Ideally, a mid-range (16GB RAM) machine for acceptable performance!

Customising this workflow

* For larger videos, consider splitting into smaller clips for better performance * Use a multimodal LLM which supports fully video such as Google's Gemini.

⚡ How to import this workflow into n8n:

  1. Click the Direct Download JSON button above to save Narrating over a video using multimodal AI.json.
  2. Log in to your n8n Cloud or self-hosted n8n instance.
  3. In your workflow canvas, click the menu in top-right and select Import from File (or paste the JSON directly).
  4. Configure your credentials (e.g. API keys, database URLs) in the corresponding node settings.
  5. Click Execute Workflow to test and switch to Active!

Related Video & Media Workflows

View All in Video & Media →