Add an AI voiceover to a video with FFmpeg
FFmpeg can put any audio on any video, but it does not make natural speech: its optional flite filter is a robotic voice, and most builds leave it out. The practical pipeline is two steps: voice the script with a text to speech engine, then let FFmpeg place that audio on the video.
This guide does the first step with the free SuperClonica engine, offline, and gives the FFmpeg commands for the second. We ran every command on a test video: they keep the video stream untouched with -c:v copy, so they finish in seconds even for long clips.
Voice the script
Install the engine once with python -m pip install superclonica-cpu; it downloads its model on the first run and works offline after that. Save the narration as a text file and voice it:
superclonica --text-file script.txt --voice M3 --lang en --normalize --output voice.wav
Pick the voice first on its catalog page, where a line of 30 to 100 characters is voiced free with no account. Every voice speaks 31 languages; change --lang to voice the same script in Spanish, German or Japanese. Chinese is not supported yet.
Replace the audio track
For a screen recording or a silent clip, take the picture from the video and the sound from the voice:
ffmpeg -i video.mp4 -i voice.wav -map 0:v -map 1:a -c:v copy -c:a aac voiced.mp4
The video keeps its full length; if the voice is shorter, the rest is silent. Leave out -shortest here: with stream copy it can stall or cut the picture to the length of the voice.
Put the voice over music
To keep the original sound or music under the narration, lower it and mix both:
ffmpeg -i video.mp4 -i voice.wav -filter_complex "[0:a]volume=0.25[music];[music][1:a]amix=inputs=2:duration=first:normalize=0[a]" -map 0:v -map "[a]" -c:v copy -c:a aac voiced.mp4
volume=0.25 sets the music to a quarter, duration=first keeps the length of the video, and normalize=0 stops amix from halving the voice.
Duck the music only while the voice speaks
A sidechain compressor turns the music down when the voice is loud and lets it back up in the pauses, like an editor would:
ffmpeg -i video.mp4 -i voice.wav -filter_complex "[1:a]asplit=2[v][key];[0:a][key]sidechaincompress=threshold=0.02:ratio=10:attack=20:release=400[music];[music][v]amix=inputs=2:duration=first:normalize=0[a]" -map 0:v -map "[a]" -c:v copy -c:a aac voiced.mp4
Raise ratio for a deeper dip, and lengthen release if the music jumps back too fast.
Place several lines at exact times
Voice each line into its own file, then delay each one to its moment, in milliseconds, and mix them all:
ffmpeg -i video.mp4 -i line1.wav -i line2.wav -filter_complex "[1:a]adelay=500:all=1[l1];[2:a]adelay=4000:all=1[l2];[0:a]volume=0.25[music];[music][l1][l2]amix=inputs=3:duration=first:normalize=0[a]" -map 0:v -map "[a]" -c:v copy -c:a aac voiced.mp4
Here the first line starts at 0.5 s and the second at 4 s. Because each line is a separate file, a change in the script means re-voicing one line, not the whole track. For subtitles you already have, our SRT to speech guide builds the timed track for you.