Show HN: Revideo – Create Videos with Code

hkonsti · 2024-06-11T14:43:31.000Z

Hey HN! We’re building Revideo (https://github.com/redotvideo/revideo), an open source framework for programmatic video editing.Revideo lets you create video templates in Typescript and render them with dynamic inputs through an API. It also comes with a <Player /> component that lets you preview your projects in the browser and integrate video editing functionality into web apps.The project is useful for anyone who wants to build apps that automate certain video editing tasks. A lot of companies in the space build their own custom stack for this, like Opus (https://www.opus.pro/), which automatically creates highlight videos from podcasts, or Clueso (https://www.clueso.io/), which lets you create stutter-free product walkthroughs with AI voiceovers.Revideo is based on the HTML Canvas API and is forked from Motion Canvas (https://github.com/motion-canvas/motion-canvas), a tool that lets you create canvas animations. While Motion Canvas is intended by its maintainer to exclusively be a standalone application [1], we have turned Revideo into a library that developers can integrate into their apps, while specifically focusing on video use cases. To support this, we have, among other things, added the ability to do headless rendering, made video rendering much faster and added support for syncing and exporting audio.We’re excited about programmatic video editing because of the possibility to automate content creation with AI. One of our users is building StoriesByAngris (https://storiesbyangris.com/), which lets you create video-based RPG stories from language prompts. Other users are marketing-tech companies that help their customers generate and A/B test different versions of video ads.We started to work on video tooling because we ourselves explored a bunch of product ideas in the space of AI-based video creation earlier this year. For example, we built apps that automatically create educational short videos and tinkered with apps that let you create memes.While building these products, we were frustrated with the video editing frameworks we used: Moviepy (https://github.com/Zulko/moviepy), which we used initially, doesn’t work in the browser, so we’d often have to wait minutes for a video to render just to test our code changes. Remotion (https://github.com/remotion-dev/remotion), which we switched to later, is pretty good, but we didn’t want to rely on it as it is not FOSS (source-available only).We had already followed Motion Canvas for some time and really liked it, so we thought that extending it would get us to something useful much faster than building an animation library from scratch. We initially tried to build Revideo as a set of Motion Canvas plugins, but we soon realized that the changes we were making were too drastic and far too complex to fit into plugins. This is why we ultimately created a fork. We’re unsure if this is the right way to go in the long term, and would prefer to find a way to build Revideo without feeling like we’re dividing the community - if you have experience with this (keeping forks with complex changes in sync with upstream) or other suggestions on how to solve this, we’d love your input.Our current focus is improving the open source project. In the long term, we want to make money by building a rendering service for developers building apps with Revideo.We’d love to hear your feedback and suggestions on what we can improve! You can find our repo at https://github.com/redotvideo/revideo, and you can explore example projects at https://github.com/redotvideo/examples[1] “Motion Canvas is not a normal npm package. It's a standalone tool that happens to be distributed via npm.” - https://github.com/orgs/motion-canvas/discussions/1015

KhoomeiK 2 years ago |

Interesting—LangChain seemed kinda like unnecessary abstractions in natural language (since everything is just string manipulations), but with AI video, there's so many different abstractions that I'd need to handle (images, puppeting, facegen, voicegen, etc).

Seems like there might be room for a "LangChain for Video" in this space...

hkonsti 2 years ago | |

I agree! We were definitely motivated by the emergence of AI tools for video when building Revideo - as mentioned in the post, we think that a lot of video creation can be automated using AI. Currently, there are probably some higher-priority challenges related to the core rendering library that we need to solve, but we’ve definitely already thought about building a universal client library for common AI services that are useful for generating videos (e.g. text-to-speech, text-to-image, text-to-video)

Beefin 2 years ago |

Hey this is really cool, we just launched our video embedding model: https://learn.mixpeek.com/vuse-v1-release/

I wonder if there's opportunities for collaboration. It seems we're the only cloud-agnostic video embedding model that allows users to own their embeddings.

Here's a reverse video search tutorial: https://www.youtube.com/watch?v=H92cEhG9uMI&ab_channel=Mixpe...

franciscop 2 years ago |

Looks nice! Might want to disable pixel snapping for the text resizing since it yanks a little bit (Firefox on Mac at least).

I made an experiment in a similar style a while ago, but I decided it was too difficult to keep going as a "tiny" side project so never really released anything beyond a demo that you can see here:

https://francisco.io/demo/terminal/

hkonsti 2 years ago | |

Thank you, I'll try to disable it! I noticed the little yanks too but didn't know where to look and therefore didn't take the time to try and get rid of it.

Just checked out the demo, nice work!

thenorthbay 2 years ago |

Interesting stuff! Which use cases do you think developers will use you most for?

There could be really interesting abstractions that people might build on top of this. Like automatically creating and animating infographics, making background sounds, or video cutting and recycling. If you spin this 100x further an entire video creation studio might emerge.

Which parts of Video Infrastructure do you want to build first? Which other higher-level parts could be built by you or users? Where could this go?

hkonsti 2 years ago | |

Thank you! As of now, many people use Revideo to programatically create Youtube videos (primarily entertainment content, think of AI generated memes and Youtube Shorts) and for use cases in video marketing (think automatically generating and A/B testing ads from a product portfolio). As the project becomes more mature and better, we want to enable developers to build more complex, and especially interactive video editors using the framework.

I'm glad you raise the point of reusability and people building abstractions - what I find really exciting about code as an interface for video editing (rather than GUI) is that it makes it much easier to create and share isolated, reusable components and functions. Especially as we are open source, I hope that many developers will build abstractions and share them on Github

Out of curiosity, what do you mean by "Video Creation Studio"? Would you add a visual interface for editing to Revideo?

thenorthbay 2 years ago | | |

Maybe not for editing per se. But for providing meta-functionalities to services built on top of revideo. Rendering, for instance, but maybe even reusable embeds, video watch analytics, download links, programatic re-rendering with webhooks and their configuration, etc. What other functionalities could you offer?

sansseriff 2 years ago |

How does Jacob (aarthificial, creator of motion-canvas) feel about this? Will you compensate or include him in some way? I understand the license is MIT so you can do what you want. Just seems like it would be polite to maintain a good relationship with him and other motion-canvas maintainers.

hkonsti 2 years ago | |

We tried to reach out to Jacob but unfortunately didn't get a response from him. In general, Revideo and Motion Canvas have very different goals and we were very reluctant to go the route of forking his work. We are always open to contribute changes back into Motion Canvas if that is wanted, so far we haven't seen that to be the case.

supermatt 2 years ago | | |

You mention in your README that you had to fork it in order to make it so you could trigger renders programatically instead of having to click a button. Couldn't you have just used render from the core lib directly?

https://motioncanvas.io/api/core/app/Renderer#render

mattdesl 2 years ago |

Looks great. How are you encoding the video into MP4? Ffmpeg with wasm? Or WebCodecs?

I’ve struggled to find a pure client-side encoder that is as fast, lightweight and high quality (in terms of lossiness) as what I had going with mp4-h264[1]. I suspended the project out of legal concern but it seems like the patents are finally starting to run their course, and it might be worth exploring it again. I’ve been able to use it to stream massive 8k near-pixel-perfect MP4s for generative art (where quality is more important than file size), compared to WebCodecs which always left me with a too-lossy result.

[1] https://github.com/mattdesl/mp4-h264

hkonsti 2 years ago | |

Thanks! We're using a server side ffmpeg process for the encoding right now, so running a server is required for rendering. We are exploring fully client-side rendering too but I'm hesitant to use WebCodecs because of the limited compatibility with many browsers. Ffmpeg-Wasm does look promising so I'll give that a closer look very soon. I'll also check out your project, thanks for sharing.

mvoodarla 2 years ago |

Congrats on the launch! I work at Sieve (https://www.sievedata.com/). We do a bunch of stuff with AI and video. Excited to check this out :)

hkonsti 2 years ago | |

Hi Mokshith, we talked about releasing a project using the Sieve API and Revideo before! This is definitely something we still want to do. I'll reach out!

mike31fr 2 years ago |

> Remotion is pretty good, but we didn’t want to rely on it as it is not FOSS (source-available only).

Noob question : How would you explain in the simplest form the difference between FOSS and source-available. In other words, what does Remotion do not have that would make it FOSS?

justusm 2 years ago | |

In the simplest form: Revideo is MIT-licensed. That basically means that you can do anything you want with our code - you can modify it, fork it, build commercial products, resell it, etc., without owing us anything.

While you can look at Remotion's code on Github (that's what we mean by source-available), the project has a custom license that doesn't grant you the permission to do anything you want with it. Remotion retains ownership of the code (i.e. you cannot simply fork it and give it away for free), and when you are a company with more than three employees, you have to pay Remotion if you want to use their code to render videos

ramengrowth 2 years ago | | |

This is the one of the clearest explanations of the differences between the licenses I've seen, thanks!

LelouBil 2 years ago | |

A more restrictive license ?

Remotion has a custom license that requires you to have a Company license if you don't meet certain criterias for the free license.

Revideo just has the MIT license.

pavi2410 2 years ago |

I see that Revideo uses generator functions which seems intuitive to me as it linearizes frame sequences wrt time as the function yields.

How does this compare to Remotion^ which uses "React" mental model?

^: https://remotion.dev

hkonsti 2 years ago | |

Yes, exactly! We're also big fans of the generator function model. In Remotion, you get the current frame of the animation through a React hook called useCurrentFrame(). You then use React to build the UI based on the return value of the hook. By structuring it as a generator function, later parts of the function are for later parts of the animation, which makes it a bit easier to read and write. We found this to be a little more intuitive. Ultimately it probably comes down to personal preference.

hubraumhugo 2 years ago |

Just curious, are you the founders of https://haven.run (YC S23)? I noticed that the Linkedin company page now redirects to Revideo.

Would you mind sharing a bit about your pivot? I always find these stories interesting!

hkonsti 2 years ago | |

Yes, that's us! We started out in the open source LLM space, but realized that most companies didn't really have any real use cases for custom fine-tuned LLMs. Maybe we were too early, but everyone we talked to was best served by just using OpenAI. Late last year, we started looking at other areas and became fascinated with AI-generated social media content, which sent us down a rabbit hole that led us to build Revideo. It's been a very long and painful process, but we now have a lot of really cool and smart users building really useful stuff, which is something we haven't experienced with any of our previous pivots.

andrewstuart 2 years ago | | |

Removed - conversation tone not right.

matsemann 2 years ago |

Is it possible to render and export the video in browser, preferably faster than playback speed?

Use case is a service where people can upload certain data and I use that to generate a video. Let's say I gave you the option to make a speed gauge video, that display the values you input, one after another, for a second each. If you upload 60 values, that will be a minute video. But if you upload your speed each second for an hour, that will be an hour long video. But should ideally not take an hour to render. Unfortunately I've seen most browser based tools can't render faster than playback. So would have the user watch the whole video to actually download it.

rikroots 2 years ago |

I love mucking around with canvases and videos, so I will certainly be checking this out!

On a selfish note, as a canvas library developer/maintainer, I do have questions around your choice of Motion Canvas: what attracted you to that library in particular (I'm assuming it's the Editor, but could be wrong)?

On a broader note, my main interest in canvas+video center around responsive, interactive and accessible video displays in web pages. Have you had any thoughts on how you'd like to develop Revideo to support these sorts of functionalities?

justusm 2 years ago | |

Glad to hear that! I would say that the choice of Motion Canvas was motivated by their API for defining animations rather than the editor - we really like the approach of using generator functions and find the "procedural" API (the time of the yield corresponds to the time in the video) quite nice to work with.

Given that our goal is mainly to let developers build their own editors, the Motion Canvas editor is not that important for us - we only use it for previewing code changes, so merely projecting the canvas without any of the remaining editor interface would also be sufficient.

I also agree that interactivity is super important. We have not yet started to work on this, but something we definitely need to make easier with the Revideo player is building drag-and-drop editing features (i.e. moving elements around on the canvas to modify their position).

creativenolo 2 years ago |

This looks like lots of fun.

I’ve only skimmed the docs and nothing jumped out on this: would it be possible to use a 3d canvas context? For example, integrate a dynamic three.js layer/asset into the video?

hkonsti 2 years ago | |

We haven't played around with this yet but this is possible. There are people working on this stuff in the Motion Canvas community, which is the animation library we're based on. It should be pretty straight forward to adapt this for Revideo. https://github.com/motion-canvas/examples/tree/master/exampl...

epiccoleman 2 years ago |

This is really cool, I love this sort of thing.

hkonsti 2 years ago | |

Thank you! Love to hear it :)

ashia 2 years ago |

Looks promising - I've been using Shotstack's visual editor to create video templates but keep running into limitations. Looks like Revideo has an "editor" that allows previews but not edits? Is editing through the GUI on the roadmap?

justusm 2 years ago | |

Thanks for the feedback! Justus here, I am one of the co-creators of Revideo. Initially, we intended Revideo to be a developer tool and not necessarily something that could be modified through a user interface. However, we've heard this feedback many times at this point, so we're definitely considering building editing capabilities into the UI. One challenge here is to find a way to keep the UI changes and the code in sync.

andrewstuart 2 years ago | |

>> I've been using Shotstack's visual editor to create video templates but keep running into limitations

What limitations are you hitting?

What’s your functionality wishlist?

probson 2 years ago |

This looks very cool! I have built a project using remotion to bake in subtitles with some effects to a video from a .srt file, but this approach looks nicer and FOSS is amazing so I'll have a go at porting it. Thanks!

justusm 2 years ago | |

Thank you! If you have any questions or run into issues, please let us know! You can find out Discord server in our repo or can email me at justus at re.video

justusm 2 years ago | | |

Also, I forgot to mention: We have an example project that focuses specifically on subtitles with some visual effects, this could be helpful: https://github.com/redotvideo/examples/tree/main/youtube-sho...

earlyriser 2 years ago |

I have used Revideo for a personal project and I really like what you're doing.

justusm 2 years ago | |

Thanks a lot! Really glad to hear this feedback :)

Loiro 2 years ago |

Looks like a cool tool. Will play around a bit, thanks for sharing!

rjeli 2 years ago |

Very cool! I assume it uses WebCodec VideoEncoder to encode in browser, maybe with a wasm ffmpeg fallback? How reliable/easy to use have you found that?

hkonsti 2 years ago | |

Encoding the video in the browser is on the roadmap, but currently we stream the frames one by one to an ffmpeg backend process and concatenate them into a video there. This has been very reliable and fast so far. I've only just started looking at WebCodec, but it seems to be more challenging, especially as most of the APIs aren't supported by some of the mainstream browsers, so as you said, some fallback will be necessary.

albert_e 2 years ago |

Great idea.

When text-to-code capabilities of LLMs become more mature, libraries like these are going to create a lot of novel uses cases and opportunities.

justusm 2 years ago | |

Thank you! A lot of people have mentioned this to us already - we did not think about the LLM use case when getting started with Revideo, but it does make a lot of sense that it can be used to build text-to-video products and fully automate video editing.

As you mentioned, letting LLMs generate high-quality Revideo code is probably not yet possible, but something I believe could make a lot of sense is using LLMs to choose parameters for your video. Video parameters can get as complex as you like in Revideo (any JSON object is accepted), so you could build a very flexible template that lets you define pretty much any video by using a "universal" JSON representation (essentially, this would be a list of elements such as audio files, videos, images, texts, along with attributes describing the timing, position, etc. of the element)

Letting the LLM generate input variables for your template instead of the code would at least allow you to do type checking and ensure that there are no bugs in your video code - however, I still doubt that the resulting video will be of high quality, LLMs are probably not yet smart enough for this.

andrewstuart 2 years ago |

Are there commercial use cases for this?

hkonsti 2 years ago | |

Certainly! You can pretty much build full video editors with Revideo. There are many companies with video-editing at the core of their product that could have been built with us (think veed.io, invideo.io or the editor part of www.synthesia.io). We're thinking about building a template in this direction so people have a starting point when building these kinds of apps. In terms of video automation, we've seen people build apps for marketing and enterprise learning. One marketing tech company uses Revideo to generate video ads from their customers product catalogue.

darepublic 2 years ago |

The thing I am dubious about with many of these AI tools is having fine control over the details.

popalchemist 2 years ago |

Does it work with vue/vite? I am really hoping someone will make such a solution some day.

hkonsti 2 years ago | |

So far, we only have a prebuilt Player component for React, but it is definitely possible to integrate Revideo into a Vue project with a bit of hacking. I'd love to better understand your requirements and help you get started. I already did this for React so porting this over to another framework like Vue should be straight forward - feel free to message me in our Discord server (invite link is in our repository) or email me at konsti at re.video.

simonbarker87 2 years ago |

How does this compare to MoviePy beyond the JSX like syntax and being JS?

hkonsti 2 years ago | |

The fact that it is JS actually implies more than just being written in a different language. You can ship Revideo as part of your website and run it inside the browser. One of the reasons we struggled with Moviepy was that we couldn't preview our changes in real-time which made the dev-experience a little tedious.

fanfanfly 2 years ago |

Does Revideo equivalent with Pymovie or it has some advantages?

hkonsti 2 years ago | |

If you mean Moviepy, then yes, there are some advantages. As I mentioned in another comment, you can ship Revideo as part of your website and run it inside the browser. One of the reasons we struggled with Moviepy was that we couldn't preview our changes in real time which made the developer experience a bit tedious. We also like the syntax of our generator functions better, but that is of course subjective.