Non-Rectangular Video Cropping with FFMpeg(dragonquest64.blogspot.com) |
Non-Rectangular Video Cropping with FFMpeg(dragonquest64.blogspot.com) |
It even works on the web: https://files.catbox.moe/uu2ze0.webm (needs a non-black background)
I've been trying to find higher-performance methods of doing this with live footage that don't require openCV. Seems like FFmpeg doesn't have a great way to do this or I just haven't found it yet?
I guess I'm also looking for frameworks or tools that can handle live video "manipulation" like rendering shapes or graphics. OpenCV is painfully slow for most of this stuff, everything else is seemingly completely proprietary. But nVidia seems to have some incredible tools baked into their latest stream manipulation API's.
This is not to say there isn't scope for improvement. A couple of areas for improvement is the consistency of options naming across filters that implement the same feature within those filters, and a clearer syntax to identify the target of an option i.e. protocol, demuxer, decoder..etc
And it's not just N inputs, M outputs. You also have a processing graph for the intermediate filters. So ideally you want graph nodes represented by some identifier which can then be referenced by other nodes.
But I keep running out of memory. I like that the process just dies. And can usually be solved by sequential batching. But how can I automate via predictive instrumentation?
I just mean I'd like to "cap" ffmpeg's memory usage to ensure it never crashes.
I'll provide a simple example. Creating an image slideshow from a directory of huge images. I can downscale and compress to jpeg before processing. But even then, besides batching, it always fails. And this is just raw, without any filters or image processing. Have tried all the i/o techniques from the wiki:
https://trac.ffmpeg.org/wiki/Slideshow
All this local laptop based (4GB). I just want to get a better handle before devoting cloud resources (ffmpeg is standard on gcloud serverless) ;)
Unless I misunderstand you, this is already the case. Simple or complex filtergraphs are declared as a string* by the user as an arg to -vf/-af (per-output) or -filter_complex/-lavfi (global). Filter outputs can have link labels, which allow their consumption by other filters.
*can also be read from a file
*via shell too, if the arg is quoted/escaped.
I think this might make use of ffmpeg less daunting than people attempting to battle a single-line shell input, ordering-sensitive parameters and escaping.
Well, maybe it's redundant, there already is vaporsynth after all.
I regularly use a laptop with 4GB RAM to generate videos using the sequential method listed there, with 4-8k sized frames, producing video files of hundreds of GB, without a crash. There can be hundreds of thousands of them as well.
No batching required.
The final command in my pipeline is a simple:
ffmpeg \
-threads "$threads" \
-y \
-start_number 0 \
-i 'build_frames'"$unique"'/%09d.png' \
-c:v libvpx-vp9 \
-lossless 1 \
-qscale:v 2 \
-r "$fps" \
build_tmp"$unique"/media.webm
This can cause large memory spikes if your input has low fps (or is generated from JPEGs with large time difference), since frames between two input frames will be generated and then sent to the output in a single operation, spiking memory usage.
Which ffmpeg version are you using? This hasn't been the case since 2012. Both input and output -r are effected through fftools and not libs.
Switching to vp9 (from x264) worked for me!
Something is likely very off with how you are using ffmpeg. If you force more keyframes, use good quality settings etc. it should make short work of it.