The Best AI Tools for Video and Audio Post Production

The Best AI Tools for Video Editors

the best ai tools for video and audio post production

What are the best AI-powered tools for video and audio post-production that video editors, colorists and sound editors should be using in their workflows?

That’s the question I set out to answer in this article, yet with the explosion of “AI” based tools (really, it’s machine learning folks!) available today, there are a huge range of apps and services that you could be using, but which ones are actually worth your time and money?

Latest Update: April 2024

The 8 Best AI Tools for Post Production shortlist should help you get straight to the good stuff, while the long list that comes after that will give you a sense of everything that’s out there, although I left a few dubious options out, that were just ‘not good.’

Shortlist of the 9 Best AI Tools for Post-Production

I’ll be updating this list regularly as I find new and decent AI-based tools to add to it, but if you see something I’ve missed, do get in touch to let me know.

At the end of this article you’ll find a breakdown of some of the AI-powered tools already available on your timeline in DaVinci Resolve Studio and Adobe Premiere Pro.

If you’re after a list of tools that will actually help you get some editing done, such as speed up your rough cut, then check out this article on The Best AI Video Editing Tools.

Table of Contents

  • AI and Post-Production Frequently Asked Questions
  • The Best AI Based Tools for Video Post Production
  • AI Video Editing and VFX Tools
  • AI Colour Grading Tools
  • AI Audio Post-Production Tools
  • AI Transcription, Text Editing, Subtitles, Translation
  • AI Media and Metadata Search
  • AI CGI & 3D Animation Tools
  • AI Image & Video Creation Tools
  • AI Image Enhancement Tools
  • AI Voiceover Generation Tools
  • AI Tools Already in Your Video Editing Software
  • AI Creativity, Ethics and The End of Humanity
  • Actually Using AI in Post-Production
  • AI Video Generation Models Compared

FAQ about AI in Post-Production

With AI The Best UI Wins

My guess is that many of the services listed in this article are likely accessing the same AI and ML open source models underneath their browser-based UI. So, really, the race is on for whoever can develop the best user friendly interface, affordable pricing structure and win over enough paying users to hog market share and establish a sustainable business.

That said, some brands like, are actively developing their own proprietary models such as Stable Diffusion and have the VC-backed funds to develop new and existing features, market themselves well and play the long game.

Will AI Take My Job?

Whether you’re fearful of AI taking your job, or just want to get your day-to-day tasks done more efficiently, the AI toothpaste is already on it’s way out of the tube so we may as well make the most of it.

If you don’t, others will.

Personally, I’m not worried about creative video editors being relegated to the annals of history just yet, as the creative decision making, client relationships and cognitive processes involved in making excellent content just aren’t going to be beat by AI, at leat not just yet!

That said, obviously AI editing tools can slap clips and audio together and create an ‘edit’ but it’s not an apples to apples comparison.

What some of these tools excel at is removing the (human!) manual labour from complex tasks such as rotoscoping, transcription or asset tagging. Yet all of this work still needs to be managed, tweaked, corrected, edited and approved by a human.

AI Vs ML – Marketing Terms vs Technical Reality

To Nerd-splain for a minute, A.I. and M.L. are quite different things.

A.I. is mostly what you think of when you think of A.I.

Intelligent machines that can think and act upon their own objectives, usually involving world domination and the subjugation of the human race. They can do multiple things with dexterity.

Machine Learning, however is a branch of A.I. and far more constrained and it’s behind what most of these tools are doing.

Provide your image recognition algorithm with a large enough dataset of images, for example of cats, the computer program would be able to iteratively learn what a cat generally looks like based on patterns of pixels and so (eventually) predictively tag photos of cats with the metadata ‘cat’.

So there’s no need to start building your survival bunker just yet.

The Best AI-Based Tools for Video Post Production

As there are so many tools on this list, I’ve curated out my top 8 AI post-production tools at the start to save you from scrolling too far and to give you some more details on these specific services.

As I update this list, these might swap and change as new tools and services become available. I’ll also suggest why exactly a video editor like myself would want to leverage these in their workflow.

Update April 2024 – Adobe Previews Premiere Pro AI Tools

During NAB 2024, Adobe previewed some new AI-powered tools coming soon to Adobe Premiere Pro, including the ability to:

  • Add frames you don’t have – extend a shot with generative AI
  • Add objects – Text prompt to video within a selection
  • Remove objects – logos, text etc with roto/tracking built-in
  • Replace objects – swap costumes, props and much more
  • Content Credentials – know if AI was used in a video edit
  • Leverage third-party video models – Pika, Sora and Runway
  • Add shots – Text prompt to b-roll with the above models

TOP 9 AI POST-PRODUCTION TOOLS – Michael Cioni’s platform aims to leverage AI to solve many of Post Production’s core workflow problems. Having only just launched the company they are already rolling out a beta which does some incredibly impressive and really useful things such as:

Transfer all of your media data to the cloud and within the cloud into the Strada platform. This allows you to create virtual files where your footage is on but your audio files are in Dropbox yet you can play them both back in sync in Strada.

Transcription provides the backbone of metadata search as well as text-based editing and translation. Meanwhile, the parallel processing of the cloud means that transcoded files are generated in minutes rather than hours.

Their Tag/Analyse view of presenting in-frame object metadata is the best and most intuitive design that I’ve seen!

I’m really excited about Strada for a couple of reasons: Michael’s track record at LightIron, Panavision and, their creator-first development philosophy, the beautiful design aesthetic and mostly, that it’s a post-production-centric tool that solves real problems!

Sign up for the beta here. – The Swiss Army knife of AI powered browser-based tools, has a huge range of useful features such as rotoscoping, object removal, image creation, motion tracking, colour grading, audio noise removal, transcription, text to image, image to image and much more.

Imagine this as your (temp) VFX tool for rapidly fixing shots, creating new assets or backgrounds and solving problems for your clients like you are able to do magic.

If all goes well you could complete the shot inside of Runway’s online editor or download elements like Alpha mattes, new green screen elements, background plates and more.

Sign up for free to try for yourself here and save 10% with code: ELWYNJX10

DaVinci Resolve Studio – Less features than Runway but with the huge benefit of being available right there in your timeline, no uploading or downloading required. Also a one-time price of $295 for a lifetime of updates and usage it’s a pretty economical option.

DaVinci Resolve’s ‘Neural Engine’ features include facial recognition, magic mask, super scale, 3D depth maps, scene cut detection and a whole lot more (see the end of this article for more details!)

Buy DaVinci Resolve Studio on Amazon Global Stores

Descript – Offers an ‘all-in-one’ solution for text-based audio and video editing which includes the ability to ‘overdub’ words and phrases with an AI generated version of the speaker.

Descript is already really popular with podcasters and anyone who doesn’t mind a jump cut in their video. It can automatically remove pauses and filler words (ums, ahs etc.) and allows you to edit your transcribed video like a text document.

It also features a comprehensive browser-based multi-track editing tool, very impressive ‘Studio Sound‘ noise reduction and audio rebuilding, transcription, caption creation and the ability to export to professional applications such as Premiere, Final Cut Pro and others.

The Free account includes 1 hour of transcription, one 720p watermark free export, ums & ahs removed, 1000 word overdub vocabulary and 10 minutes of Studio Sound per month.

Useblanc.comBlanc’s super power is its ability to convert one transcription into multiple languages, there by exponentially increasing the reach of the content to new audiences.

Currently this means you can create 47 other versions of your video with either translated subtitles, AI-generated voice over, or even AI-generated replacement lipsync.

One of the cool things that Useblanc does is to translate the original text accurately, but also in a way that mitigates vast differences in word-length or sentence structure between languages, in order to deliver the most natural sounding results.

You can get $10 worth of free credit to test for yourself, with the promo code JONNYELWYN. & – OK so I’ve sort of cheated with this one, in that these are two completely different (competing) tools for AI powered colour grading.

ColourLab’s tools are machine based with a stand alone app and plugins for Adobe Premiere, Final Cut Pro, DaVinci Resolve and Pomfort Live Grade while works in your web browser and processes everything on their cloud-based servers.

The benefit of AI powered grading is that you can let the algorithm do the heavy lifting of matching references, extracting looks, making different cameras match each other, creating look variations on the fly and a whole lot more.

Both are being consistently developed and improved over the past few years with recently updating to v1.5, which paves the way for iPad and mobile friendly versions.

Likewise, ColourLab’s product range is now at version 2.3, with their offering split into three main tools (Colour Lab AI, Look Designer and GrainLab) although they all work cohesively together, depending on what you’re doing.

Update: FreeLab Plugin – have released a new free plugin to ‘replace LUTS’ for Premiere and Resolve. Essentially, it is a ”free version of Colourlab’s film native colour engine, allowing you to balance cameras and apply consistent look across a variety of materials.

Download it here to try it for yourself.

I’ve previously written more about both services in this post on Tools & Training for Pro Colorists. – A very affordable, and to date, the best sounding, speech to text AI voice generation platform I’ve heard. It also offers voice cloning based on a 1 minute sample and the results that I’ve heard are incredibly impressive.

The free account gives you 10,000 generated characters to play with, plus 5 cloned voices although you can only use it for non-commercial use and attribution is required.

However, the low-cost $5/month tier gives you 30,000 characters, 10 cloned voices and commercial use, not to mention API access, should you want it.

Unlike Altered Studio that charges by the downloaded minute, rather than the generation attempts, Elevenlabs charges per character generated.

Elevenlabs speech results are ideal for ebook, article and podcasts reads, although they are less ‘character driven’ than Altered Studio’s voices, the realism is much more believable.

In the video above I created an experimental episode of an issue of Cut/daily (my free weekly newsletter) which fit easily into the 10,000 character free quota, even with a few regeneration attempts, specific fixes and the like. All in all, incredibly easy to use and very impressed with the results.

Altered Studio – Actors in a box. That’s what Altered Studio can give you. Drop in your own line reading of some voice over or scripted lines into the system and let Altered Studio turn it into a complete multi-character production.

You can change your accent, add ‘gravitas and charisma’ with their 200+ voices, plus you can clone your own voice samples to be able to add text-to-speech additions to your edit.

Some of the voice samples in their demos sound more convincing than others but the ability to generate an entire table-reading, voice over or dramatic podcast from scratch with professional sounding voices is very compelling.

And while you can select different moods and stylistic approaches, I’m not sure there’s a way to ‘give direction’ to these AI voices in the same way that you can to a real actor – but at least you can create your first version, affordably.

Swell for Transcripts, Timestamps and Social Media

I’ve been using to create timestamps for my YouTube videos, which can often get pretty long, but that’s only scratching the surface of this incredibly useful tool.

You can upload video and audio and have it generate:

  • Complete transcripts
  • Export subtitles (.srt/vtt)
  • Shownotes
  • Timestamps (Show notes and YouTube chapters)
  • Summaries
  • Titles
  • Social media content (Twitter threads, blog posts, Newsletters, YouTube descriptions etc.)
  • Long-form articles
  • Chat-Agent

You can use the chat-agent to prompt Swell to do specific things, such as create a Twitter thread with a specific number of Tweets, or a summarise a portion of the interview etc.

Other cool features include watch folders for importing content into Swell from Dropbox, Google Drive, Apple podcast and other sources as well as outputting the results of Swell’s work back to similar locations.

Try Swell for yourself, for free here.

AI Video Editing and VFX Tools

In this section of the post I’ve curated a long list of AI/ML based tools that could come in handy as a video editor, colorist, sound designer or DIT.

For brevity, I’ve tried to keep things to the following formula:

Tool | Main Features | OS | Free Version? | Lowest monthly price | Rotoscoping, motion tracking, audio clean up, transcription and subtitles, export alpha mattes, in-painting, text to video, video to video, text to image, image to image, text to 3D texture and more! | Browser-based | Free Tier | $15/month – SAVE 10% with discount code: ELWYNJX10

Topaz Labs | Still image and video – upscaling, denoise, sharpening, deinterlacing | Windows & Mac | Watermarked Free Trial | Topaz Photo AI – $199, Topaz Video AI – $299 – One-time fee.

Project Blink | Adobe’s AI-powered browser-based video editor with content search, text-based editing, transcripts, social clips, filler-word removal | Browser-based | Free Beta Trial

Adobe Firefly | Recently announced in beta, this collection of online tools from Adobe does a lot of similar things to, but will eventually be embedded in Adobe’s ecosystem of apps and websites. Adobe are also trying to ‘train ethically’ | Browser-based | Free | Instantly create social media subclips with captions, resizing, chapters | Browser-based | 3 hours Free Tier with 720p exports | $19.99/month | AI face detection and replacement (plus a ton of other great tools) | Browser-based | No free trial | Pay for per-usage credits

AutoPod | AI powered multi-camera editor, shorts selection and silence removal plugins for Adobe Premiere Pro| Mac & PC | $29/month

Momento | AI generated social shorts selection and creation, media management, transcription and Chat-GPT trained article writing | Browser-based | $24.99/month

BigRoom.TV | Auto create 9:16 social media clips from 16:9 video with cuts and motion tracking | Browser-based with export to NLE | Free for now

Chat-GPT | The infamous chat bot that’s going ‘take over everything’ is actually useful for asking it for After Effects Expressions or DCTL code snippets in DaVinci Resolve | Browser-based | Free to use | “Sifts out blurry or shaky clips, selects the optimal shots, and auto-assembles them into engaging sequences (based on text prompt), ready for your final touch-ups.” | NLE-based plugin | Currently in Beta

AI Colour Grading Tools | AI grade extract, auto correct, colour matching, advanced technical matching, grade creation | Browser-based | Free Tier | $16/month | Auto Color, shot matching, reference extract, ‘timeline intelligence’ and advanced professional features | Mac (Windows coming soon) | Free Trial | $14/month | Online RAW still image grading built on ACES HDR colour management with “subtractive color modelling, physically accurate refraction and film density emulation.” Export your grades as 3D LUTs or custom Resolve DCTLs | Browser-based | Free Trial | $6.25/month | AI Colorise photos and video (plus a ton of other great tools) | Browser-based | No free trial | Pay for per-usage credits

AI Audio Post-Production Tools | Extract vocals, specific instruments from music, also audio clean up | Browser-based | 10 min Free Trial | 300 min/£15 fee | Updated their algorithm recently

Altered Studio | Voice to voice actor generation, transcription, text to speech, performance enhancement| Browser, Mac & PC | Free Trial | £49+Tax/month | Online podcast/video interview studio with AI transcription, auto-social media clips, silence removal and more | Browser-based | 2 hour free Tier| $15/month |20% off coupon code: jonnyelwyn

Read my in-depth post on why Riverside is the best video interview tool here and why I use it for AI-generated notes here. | Online AI voice generator with text-to-speech tools, voice clone and translation and dubbing in 29 languages l | Browser based | Free Trial | $5/month for 30,000 characters.

Descript | Voice enhancement with Studio Sound, voice cloning and AI voices with Overdub, filler word removal | Browser-based | Free Tier | $12/month | Using AI and DeepFake techniques to deliver ‘immersive lip sync‘ for seamless digital reshoots and translation | Bespoke service per project

I’ve written more about DeepFakes in Post Production here. | Create separate music stems (vocals, drums, bass etc.) from a single track | Browser-based | 2 free songs then $0.50 per song. | Remove vocals, create split music stems, cut and join audio files, adjust the tempo and pitch of a song | Browser-based | Free

Adobe Enhance Podcast | Online podcast studio with text editing, remote recording, AI ‘Enhance Speech’ and AI Mic check | Browser-based | Free | Update Enhance is now in Premiere Pro 24 beta.

TimeBolt | Automatically remove silence and gaps in multi-track video and audio, more features than Recut| Mac & PC | Free Tier | $17/month

Recut | Automatically remove silence and gaps in video and audio with export to XML | Free Trial | Mac & Windows | $89 one-time fee with this discount link

Momento | AI generated social shorts selection and creation, media management, transcription and Chat-GPT trained article writing | Browser-based | $24.99/month

AI Transcription, Text Editing, Subtitles, Translation and Search Tools

A lot of tools are appearing which will transcribe your material to make it text searchable or text editable. This also opens up opportunities for metadata extraction, tagging, translation, subtitles and more.

Swell | Auto-generate transcripts, summaries, timestamps, show notes, social media content; blog posts, Twitter threads and even AI-chat bot with the content | Browser-based | Free Trial | Price per minute ($8.99/hr) | $29/month (5 hours)

Blanc | Translate your video to 47+ languages with AI generated subtitles or AI voice over and even lip-sync’d video | Browser-based | Free Trial | Price per minute | $10 free credit with code: JONNYELWYN

Descript | Transcription, text editing, voice enhancement, voice cloning, filler word removal, screen recording, podcast recording | Browser-based | Free Tier | $12/month | Subtitles, translation, editing, social resizing and more | Browser-based | Free Tier | $10/month | AI subtitle translation into multiple languages (plus a ton of other great tools) | Browser-based | No free trial | Pay for per-usage credits

Podium | From audio auto-generate shownotes, chapters, highlights, social media posts full transcripts and more | Browser-based | 3 hours free | $16/month

AI Media and Metadata Search

Momento | AI generated social shorts selection and creation, media management, transcription and Chat-GPT trained article writing | Browser-based | $24.99/month

Limecraft | “keeps track of assets on local storage and that uses AI transcription and image recognition to index video and to automate repetitive work.” | Cloud-based | Free Tier | €85/month

Add Context | Turn any YouTube playlist into a Chatbot as it learns your content: ‘ask questions, find exact moments’ | Browser-based | $4/month 10 hours of content | AI search of collaborative asset management for visual references, mood boards etc | Browser-based | Free Tier | $12/month

Twelve Labs | Make your video internally searchable for objects, actions, people… | Browser-based | Free Tier

GrayMeta Curio | On-premises media search and metadata addition | Browser-based & Adobe Premiere Pro panel | Pricing by request

AI CGI & 3D Animation Tools

Wonder Studio | Drag and drop a CG character into a live action scene with AI-powered animation, lighting and composition. Export into Maya, Unreal, Blender for tweaking or export individual passes (Motion capture, Alpha mask, Clean plate, Camera track etc.) | Apply for Beta Access | Browser-based

AI Image & Video Creation Tools

“What AI looks like, in abstract” – according to MidJourney

These are most of the ones you’ve seen and heard of online already. Text prompt to image generation. Useful for generating random stuff in the style of different artists but also useful for creating background plates, storyboards and other visual assets.

Having played with these a bit text prompting does take a bit of practice but it doesn’t come with any of the joy, achievement, catharsis or skill that creating something ‘by hand’ conveys.

But the results are still remarkable!

Midjourney | Text to image generator in various artistic styles | Browser-based | 25 Free generations | $8/month | Video Transformation tool with animated styling including selecting camera moves and storyboard functionality. Text to video, video to video and image to video creation. | Browser-based | Free beta | $5/month

DreamStudio | Stability AI’s text to image generator | Browser-based | 100 free credits (500 images) | $10 for 1000 credits (approx 5,000 images) | Text to image, image to image, text to 3D texture and more! | Browser-based | Free Tier | $15/month

Synthesia | 85+ AI Avatars and 120+ language voice generation, video templates and more | Browser-based | £23/month

AI Image Enhancement Tools

Photoshop | Image generation, extension, augmentation on your desktop | Mac & PC | Part of Creative Cloud Subscription

LeiaPix | 2D image conversion to 3D Lightfield images. | Browser-based | Free Service

Nero ai | Image Upscaling, photo metadata tagging | Browser, Mac, PC | Free Online | Purchase credits.

AI Voiceover Generation Tools

Altered Studio | Voice to voice actor generation, transcription, text to speech, performance enhancement| Browser, Mac & PC | Free Trial | £49+Tax/month

Descript | Transcription, text editing, voice enhancement, voice cloning and AI voices, filler word removal, screen recording, podcast recording | Browser-based | Free Tier | $12/month

Blanc | Translate your video to 47+ languages with AI voice over and even lip-sync’d video | Browser-based | Free Trial | Price per minute | $10 free credit with code: JONNYELWYN | AI text to speech in multiple voices and languages (plus a ton of other great tools) | Browser-based | No free trial | Pay for per-usage credits

Synthesia | 85+ AI Avatars and 120+ language voice generation, video templates and more | Browser-based | £23/month

AI Tools Already in Your Video Editing Software

AI Tools in DaVinci Resolve Studio

If you don’t already own the $295 Studio version of DaVinci Resolve, you’re missing out on one of the best deals in the Post Production world.

Not only is it an incredible piece of software housing at least 3 industry leading apps (Fusion, Fairlight and Resolve’s Colour Grading) but every single update has been free – so you’re not tied into a costly subscription!

Not to mention their Cloud-based team collaboration and remote workflows, new iPad version, constant update cycles and excellent hardware solutions. There’s just a lot to love!

Also Blackmagic Design have been actively developing and implementing some great AI driven features directly into DaVinci Resolve Studio, with more to come in the future.

You can read all about the Studio only features in DaVinci Resolve here, but here’s a short list of the current features powered by DaVinci Resolve’s own AI/ML ‘Neural Engine’:

  • Facial recognition
  • Magic mask for object isolation and tracking
  • Object removal
  • 3D Depth Maps
  • Scene cut detection
  • Smart reframing
  • Speed warp retiming
  • Super scale upscaling
  • Auto colour
  • Auto colour matching

Buy DaVinci Resolve Studio on Amazon Global Stores | or on B&

AI Tools in Adobe Premiere Pro

While Adobe have to take care of a huge range of 40+ applications (so each app’s development seems to be a bit slower than Resolve’s relentless pace) they are also constantly pushing the boundaries of what’s possible through open beta’s and then later integrating those features into their Creative Cloud suite of apps.

If you’re a Premiere Pro editor like I am, then hopefully you’ve already had a chance to try out some of their current features powered by their AI/ML service Adobe Sensei but if not here’s a shortlist of some of the features in Adobe Premiere Pro currently powered by Adobe Sensei:

  • Automatic Music Editing (Remix)
  • Auto-ducking
  • Morph Cut
  • Colour Match
  • Auto Reframe
  • Content aware fill in After Effects

AI Creativity, Ethics and The End of Humanity

In this final section of this very long article, I wanted to share some insights on the ethics of AI creativity, where this all might be going and if we really are doomed by AI.

Kirby Ferguson’s intensely brilliant Everything is a Remix explores Artificial Creativity in the fourth part and final part of his recently re-made series.

Watch all four parts here.

There’s no going back

I won’t lie and say that I know where this is going, but there’s no going back.

Created for this short film explores the idea of ‘cultural ratcheting’ where “ideas that are distributed across a large network of minds can’t be lost, instead they accumulate and become more complex than any individual could grasp.” but now the disruption of AI is overtaking this uniquely human process.

AI Filmmaking is Not The Future. It’s a Grift

Patrick (H) Willems takes “AI filmmaking” to task in this extremely well researched video essay, responding to some of the viral Wes Anderson parody trailers created by Caleb Ward and arguing for the irreplaceable importance of human creativity.

The AI Dilemma

Presented at the Center for Humane Technology by Tristan Harris and Aza Raskin (creators of The Social Dilemma) explore the potentially disastrous negative effects of unleashing AI on an unsuspecting humanity.

Watch the presentation on Vimeo here.

Making Viral Videos with AI – Dave Clark

Director and AI filmmaker Dave Clark has created some slick-looking teaser trailers and concept films in various different genres using a wide range of AI-powered filmmaking tools.

If you want to get a sense of what’s possible – spend a few minutes browsing his YouTube channel!

In this conversation with Every CEO Dan Shipper, Dave talks about his process and makes a short film from scratch.

Actually Using AI in Post-Production

5 Things host Michael Kammes, showcases 5 AI Tools for Post Production that you may not have heard of including StoryToolKit AI, Twelve Labs, Curio Anywhere,, Pinokio.

These include AI models that work locally, include open-source solutions and allow you to select the kind of machine learning you want to use.

At a recent LACPUG event, Philip Hodgetts gave an overview of where we are in terms of AI progress and where this is all going. Philip covers everything AI might be touching from script to screen today.

Don’t miss the amazing, entirely AI-generated commercial 26 minutes in.

One particularly interesting thought:

AI Transcription now breaks the paradigm of only being able to have 2/3 from fast, cheap, good. It breaks that paradigm that has fundamentally underpinned my entire career.

Philip Hodgetts

Michael Kammes follows up with a look at some AI tools that you can already use today.

He gives examples from:

Applied AI in Post-Production Part 2

In Part two of his talk, Michael Kammes continues demonstrating some of the existing AI-powered Post tools available today, including rolling your own home-brew AI.

  • 01:00 Assistive color grading with
  • 04:42 Host your own free, local Chatbot that looks at your documents with Anything-LLM.
  • 08:20 Using OpenAI’s ChatGPT API key + cost examples
  • 11:27 Expanding what an LLM “knows” with your documents, using Embeddings
  • 12:12 Current easy use cases for AI
  • 13:46 Online resources for models and learning more about AI
  • 15:46 Why running LLMs locally is very tough (for now)
  • 17:44 Cool sites and Github repositories to check out

AI Video Generation Models Compared

When it comes to generating video clips from a text prompt or source image there are plenty of different models to choose from. The big ‘three’ (?) being OpenAI’s Sora, Pika Lab’s Pika and Runway’s Gen 2.

OpenAI (of ChatGPT fame) recently introduced their video model, Sora, which has some impressive-looking results in the video above. You can also watch some short films made using Sora here.


Pika now offers lipsync and sound effects as part of their video generation suite of tools, plus the ability to edit regions, reprompt generations and much more.

Runway Gen 2

One of Runway’s most useful tools is the motion brush, which allows you to add movement to still images both in terms of the camera and various layers of the image.

1 Comment

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.