Encoding on a PC is an edge case, especially on software. Just hardware encode H264 at that point

> I have been accustomed to any video playback taking just a few percents on any old computer with Intel graphics.

it's using like 160% of a hyperthreaded core, so not even completely saturating a laptop from 2015 (8 years ago) - I don't know why it needs to drop frames though (single core too slow?)

Yeah, I figured. But last time I toyed with decoding audio in wasm, the speed was I think 5x worse. Not sure if it's cause SIMD was not implemented or what.

Interesting you claim that - I actually put together a rather in-depth report at the time and VP8/H.264 performed very closely as formats (that is, quality metrics vs. filesize were very comparable even if encoding time of x264/libvpx wasn't) and theora was just awful (SSIM and similar quality metrics were behind for like 30-40% which was a lot).

The article you've linked it talking about some decoder implementation details which I'm not sure were all that relevant for end users - x264 was always the superior encoder, libvpx was "ok", libtheora was terrible when it came to actual encoded video results. Not sure what the article you posted really proves around that.

Theora is VP3 with some more leeway in the hope that the encoder can use the additional parameter freedom to produce better video (which never really happened; most of the 1.1 branch was, as I understand it, just cleaning up a lot of old mistakes around e.g. rate control). It's pretty clear that VP8 outperforms it.

Same use-case, we used to rely on system libraries but the video codec support is all over the place on each platform, and developers would test in their main platform and ship to others without fully testing and have a broken video near the end of their games because WMV runs on Windows but not on macOS, or the audio of the video was a variant of AAC that didn't run in the web port in some places and a bunch of other random differences between platforms to the point the only reliable way was to cut system video libraries and make sure all platforms have the exact same codecd. But then it turns out there isn't something better than Theora or mpg-1 without making the game player have 10x or more in size - the game player is around 3MB unzipped.

Does that hardware have the CPU power for a recent video codec? It doesn't sound like it.

Embedded machine with big enough CPU for video encoding, but not having 10-20 MB flash? Doesn't sound too plausible.

an embedded machine that 1. has a software decoder, 2. has presumably some ram for that decode and 3. cant afford like 128megs flash on its design for its rom?

I once made a custom Ogg stream for a simple multimedia document format [1]. The idea was that I wanted to have AV streams in the same file as basic vector graphics slides, and back in 2008 it was Ogg that had some open source momentum.

It worked, but my experience was that Ogg isn't really a well-designed container at all. Even QuickTime / MPEG-4 with its 1990s warts is more flexible and efficient. I would definitely pick Matroska today if I really wanted to torture myself with this kind of document format again.

Somebody else wrote more eloquently about Ogg's bizarre design choices:

https://hardwarebug.org/2010/03/03/ogg-objections/

"The variable overhead in the Ogg format comes from the page headers, mostly from the segment_table field. This field uses a most peculiar encoding, somewhat reminiscent of Roman numerals. In Roman times, numbers were written as a sequence of symbols, each representing a value, the combined value being the sum of the constituent values.

"The segment_table field lists the sizes of all packets in the page. Each value in the list is coded as a number of bytes equal to 255 followed by a final byte with a smaller value. The packet size is simply the sum of all these bytes. Any strictly additive encoding, such as this, has the distinct drawback of coded length being linearly proportional to the encoded value. A value of 5000, a reasonable packet size for video of moderate bitrate, requires no less than 20 bytes to encode."

- -

[1] https://github.com/pojala/twentytwenty/blob/master/twtw-ogg....

.logg - haha

A larger codec doesn't mean it's more inefficient in other areas than a smaller codec. A 400KB codec is not necessarily less laggy or janky than a 4MB codec.

Otherwise, why use a bunch of larger algorithms for map routing when BFS/DFS are so simple and small?

You wont be encoding any video with algorithm more computationally intensive than jpeg on esp32, and even mjpeg will be excruciatingly slow.

There's mjpeg for that. Is there any evidence anyone has streamed to such a platform over real video codecs? No there's not. People just make up endless shit to justify anti use cases.

Your primary concern is almost certainly power draw, so you're using a hardware encoder. The weight of a few more megabytes ROM would be inconsequential.

I mean the Ingenuity helicopter runs Linux, which seems to suggest that a few megabytes here and there isn't such a big deal; whatever the constraints of mars are, they seem to have a fair amount of onboard storage. I can't imagine that the improved video compression of AV1 compared to Theora or H.264 wouldn't outweigh the cost of that 4MB extra storage space.

It's not only about “handle”, it's about not eating through all of your battery before you're done watching the video. But no, “any” phone cannot reliably play 1080p AV1:

https://www.spiedigitallibrary.org/conference-proceedings-of...

“Overall AV1 real-time playback of 720p 30fps @ 2Mbps is feasible for low-end devices with 4 threads and 1080p 30fps @ 4Mbps is feasible for high-end and mid-range devices with 4 threads using Dav1d decoder.”

> Mobile doesn't have 4K screens . . .

The Sony Xperia 1 series does (along with a headphone jack, microSD card support, notification LED, a hardware camera button, no hole in the screen for the front camera, etc.).

The size of the codec is so unimportant as to be essentially irrelevant within reason outside of a few extremely niche cases. We care about video file size, we care about subjective quality, we care about power efficiency in compression and decompression, in many applications we care about latency; on all these metrics, modern codecs are stunningly well-optimised. Modern codecs are extremely complex, but that complexity is absolutely necessary for them to perform well on the dimensions that actually matter to users.

Nobody uses Theora, because it's a bad codec. It was worse than H.264 back in 2009 and it hasn't been updated since. Removing support for dead formats is generally a very good idea, particularly in a web browser, because it reduces the attack surface; we have recently seen a number of major vulnerabilities caused by archaic, neglected file formats and codecs that provided almost no value to users.

That's the very last patent, and I haven't looked at it in any detail to see whether it might even be relevant to a typical software implementation; a lot of patents are on special optimisations / hardware implementations, so may not be relevant to your use-case.

It's also noteworthy that of the full list of H.264 patents here:

https://scratchpad.fandom.com/wiki/MPEG_patent_lists

...the majority of them have already expired. IANAL but since the original H.264 spec became public 20 years ago, everything in it should be usable as prior art.

Also, all existing MPEG-4 part 2 (infamous DivX etc.) patents and anything older, e.g. H.263, MPEG-1/2 and H.261, have certainly expired by now.

The problem is that H.265's patent owners didn't form a single patent pool, so the licensing is a nightmare, with three separate pools, patent holders that never joined a pool, and lots of double-billing all over the place. H.266 isn't much better. So a lot of people just stuck with H.264 as the "good enough" codec as a result. The only people not using it were organizations who couldn't or wouldn't pay for any patented invention (e.g. most of the web standards people, Wikimedia, etc), who stuck with Theora and later VP8.

AV1 (itself derived from the On2 VP8 and VP9 formats) is supposed to be the answer to H.265's patent shenanigans, but support is very slow to manifest. Like, Apple only added it to the iPhone 15 - as in, the one that just came out a month ago. Implementations of AV1 in discrete GPUs similarly only landed last year with Nvidia 40 series, Intel Arc, and Radeon 7000 series cards.

> something else

Isn't AV1 the "preferred" option as a replacement, at least by those who are looking for something high quality and without patents encumbrance?

My entire music collection is encoded as ogg/vorbis (16k+ tracks).

Just about everything I own can play them. Including rockbox on my sansa clip+.

Writing about ogg vorbis as if it is a historical format is silly. Sure, it wasn't adopted by streamers, but everything on Bandcamp (for example) is available as ogg/vorbis.

Uh, Firefox? Chromium? Did everyone else but me give up on Vorbis? It works great. Opus is nicer for bitrate but for a long time Vorbis used way less CPU to decode than Opus.

All my music is Vorbis, Opus, or ripped m4as, and browsers take it all fine.

The first version of WebM actually used Vorbis. It's the newer ones that are Opus.

Probably VLC

Yet websites can run arbitrary JS?

Relaunched about a year ago.

The website is so unremittingly awful I didn't even spin up a Windows VM to try it. Massive, all bitmaps, almost no text, no actual info at all, mentions blockchain.

https://www.winamp.com/

It has always existed. The final release from years ago still works, and whoever owns it now has been updating it under version 5.9 fairly recently. There is a "Winamp 6" on the way supposedly...

There’s a modernized fork of it: https://getwacup.com/

The official one is still around, but last I checked it had a crash bug in the media library plugin.

I didn't say they did it wrong, I asked a question. It wasn't a rhetorical question.

Anyway, this point might be moot because dav1d has released several versions with further NEON optimizations after the paper

https://code.videolan.org/videolan/dav1d/-/tags/1.1.0

https://code.videolan.org/videolan/dav1d/-/releases/1.2.1

https://code.videolan.org/videolan/dav1d/-/releases/1.3.0

That's pretty cool, especially as my collection has a bunch of 90s-00s goth and indie bits and pieces which just aren't present on the streaming services.

This was something google play music was supposed to do as well, though that's dead now!

Is there one codec that's universally available on all supported versions of iOS and all versions of Android with significant use? With an encoder and a decoder? If not, something has to make up the difference, and that something is probably a server that can see all the content.

Is it space efficient? (Compared to whatever you're prepared to license and run on the cpu) Not shipping a codec to save app size but sending larger media doesn't help the user much.

Are all implementations secure? If you have to predecode to verify the file won't trigger buffer overruns etc, you've written half of a safe decoder, and maybe you should just include the rest of the owl. Media decoding is a bountiful area of security vulnerabilities... which brings risks to both using the system apis and using your own, but at least you have some control over your own.

VLC is a mistake?