Narrating over a video using multimodal AI
This n8n template takes a video and extracts frames from it which are used with a multimodal LLM to generate a script. The script is then passed to the same ...
🔌 Nodes & Integrations Used (7)
📖 Implementation & Setup Guide
This n8n template takes a video and extracts frames from it which are used with a multimodal LLM to generate a script. The script is then passed to the same multimodal LLM to generate a voiceover clip.
This template was inspired by [Processing and narrating a video with GPT's visual capabilities and the TTS API](https://cookbook.openai.com/examples/gpt_with_vision_for_video_understanding)
How it works * Video is downloaded using the HTTP node. * Python code node is used to extract the frames using OpenCV. * Loop node is used o batch the frames for the LLM to generate partial scripts. * All partial scripts are combined to form the full script which is then sent to OpenAI to generate audio from it. * The finished voiceover clip is uploaded to Google Drive.
Sample the finished product here: [https://drive.google.com/file/d/1-XCoii0leGB2MffBMPpCZoxboVyeyeIX/view?usp=sharing](https://drive.google.com/file/d/1-XCoii0leGB2MffBMPpCZoxboVyeyeIX/view?usp=sharing)
Requirements
* OpenAI for LLM * Ideally, a mid-range (16GB RAM) machine for acceptable performance!
Customising this workflow
* For larger videos, consider splitting into smaller clips for better performance * Use a multimodal LLM which supports fully video such as Google's Gemini.
⚡ How to import this workflow into n8n:
- Click the Direct Download JSON button above to save
Narrating over a video using multimodal AI.json. - Log in to your n8n Cloud or self-hosted n8n instance.
- In your workflow canvas, click the menu in top-right and select Import from File (or paste the JSON directly).
- Configure your credentials (e.g. API keys, database URLs) in the corresponding node settings.
- Click Execute Workflow to test and switch to Active!
Related Video & Media Workflows
View All in Video & Media →3-View Orthographic to Dynamic Video Converter
Production-ready n8n automation template for 3-View Orthographic to Dynamic Video Converter built by AhiXLight. Seamlessly connects HTTP Request with 13 configured nodes.
360 Virtual Try-on Video Generator
Production-ready n8n automation template for 360 Virtual Try-on Video Generator built by AhiXLight. Seamlessly connects HTTP Request with 16 configured nodes.
3D Model to Spinning Video Generator
Production-ready n8n automation template for 3D Model to Spinning Video Generator built by AhiXLight. Seamlessly connects HTTP Request with 15 configured nodes.
AI Motion Illustration Video Generator
Production-ready n8n automation template for AI Motion Illustration Video Generator built by AhiXLight. Seamlessly connects HTTP Request with 13 configured nodes.