> If AV2 was vibe coded, there would be no case.

…for copyright. Not for anything else. Patents would still apply.

What is your criteria to measure this by and over what time scale? In 5 years, I have doubts we will see even 5% penetration of riscv on platforms which end user software runs (outside of China). I wish I could be more optimistic, but after paying attention to the standards process due many years and seeing the landscape evolve, I've become pesemistic on the matter. Even after hardware is competitive, software support will continue to lag. This culminates in security, performance and power challenges that would regress user experience compared to arm and limit adoption by device makers.

And? It's common knowledge that the "reference" or "research" version of any codec is always quite high level to get development going and actually produce a working bitstream

Of course, static websites is the best solution to that problem.

Our documentation and a main website are not fronted by this protection, so they're still accessible for the scrapers.

wat. The protections in place that the OP is talking about are almost entirely due to (not government and company) bad actors.

Easily. It's a tradeoff.

It hasn't had any proper quality/speed improvements in years. Only thing that has changed is updating deps and some bug fixes.

I don't know this stuff super well but I imagine it's not necessarily about the lawyers being right or wrong so much as what they can convince people of. The ideal scenario for the patent troll is they can intimidate you into licensing with them. Another good outcome for them (though more costly) is they can convince some non-expert in court. In either case the big players behind the codec can defend themselves but a small one just picking it up downstream as OSS can't.

> There's plenty of improvement to put into a new format

Like what ? Lossy compression has nothing to offer since about 10 years ago.

Is there a flac for video ? Is there a codec able to "guarantee" DVD quality on today's internet ? Those "up to" really sound like "up your's".

I don’t think it’s unfounded. Media codecs have been one of the top sources for serious vulnerabilities. The code is incredibly complex, and takes complex input from untrusted sources.

Decoders are one of the best places for rust because they are both performance critical and security critical.

JPEG-XL couldn’t get off the ground until they recreated the decoder in Rust since none of the browsers wanted to touch it. And the apps that did utilise the C based libjxl ended up hit with vulnerabilities in it.

you're falling for and/or playing along with astroturfing, that's what's tedious

can't people coping about rust come up with something fresh? always the same dance:

- fake annoyance about <thing> not being written in rust (bonus points if nonsensical, like here)

- if merely insinuated, fake question about what do they mean exactly

- fake surprise about omg why are people like this, the rust community is so bad, wah wah wah

yawn

oh yeah, let's not forget the other classic:

- the rust community is so dumb for thinking <shit they don't think made up for an easy beatdown>

- yeah ikr haha so stupid

every fucking rust thread is like this, and it's unreadable. by intention of course, obviously.

but it's ai / corporations / the government ruining the internet guys! totally...

I think these conversations are directed by the parties funding the efforts. Example: "we (large company) want a fast AV2 decoder" -> they pay a specialized team to do it -> this team works in C for the most part, so it is done in C. If there were financial incentives to do it in Rust, they'd pay more for a Rust decoder.

Given Netflix's involvement with SV1-AV1, (not even that) indirectly, at least 1.

In cryptography, you want operations to run in constant time, even if it’s wasteful, otherwise an attacker could guess information about the key or plaintext by measuring execution times.

Modern compilers are extremely clever and will produce machine code that takes full advantage of modern CPU branch predictors, and reorder instructions to better take advantage of pipelining. This in itself will make the same code run at different speeds depending on the input data.

Then there is the whole issue of compiler version roulette. As a developer you have no idea which version of compilers your users and distros will use, and what new and wonderful optimisation they will bring.

> C compilers, Rust compilers, and assemblers are all deterministic.

Within a version, yes, but not cross version. Different versions of GCC/Clang etc can give you completely different code.

Then how can there be a decoder if it's not final ?

I see zero public evidence that they've filed any lawsuits against the members of AOM in any jurisdiction. I'm sure there's been a lot of threatening letters sent...

We need a more efficient way to eliminate bullshit patents or bullshit patent infringement claims than "violate them then spend millions on lawyers to fight them in court".

The disgusting part is that they are proud of how complicated and exploitable this patent situation is, acting as if they were the key experts in developing codecs when they are just experts in gating access to them. Like, their entire business model is based on negating the value of the inventions.

I'm sure there are many solutions for many problems, but expecting a small Foss development team to know or implement them all is rather unreasonable.

I think the world gains more if the VLAN team focuses on their amazing, free contribution to the world, than if they spend the same time trying to figure out how to save you two clicks.

We all hate that this is happening, but you don't need to attack everyone that is unfortunately caught up in it.

> I highly doubt there is no other technically feasible option to block the AI bots.

If you have discovered such an option, you could get very wealthy: minimizing friction for humans in e-commerce is valuable. If you're a drive-by critic not vested in the project, then yours is an instance of talk being cheap.

I'm all ears on how we can fix it otherwise.

Keep in mind that those kinds of services: - should not be MITMed by CDNs - are generally ran by volunteers with zero budget, money and time-wise

There's an order of magnitude difference in speed requirements between file format parsing and image decoding, then another order of magnitude difference to video decoding. Even rav1d reuses dav1d's assembly (most of the actual runtime) to approach its speed.

There was already attempt for dav1d to re-implement in rust by rav1d. It was hard to match performance: https://www.memorysafety.org/blog/rav1d-perf-bounty/

The people who write DSLs for video codec asm, or who claim that it's fine to use intrinsics or X higher-level language and it will still be fast enough to be usable, are simply wrong and have never been able to demonstrate otherwise.

Having said that I do think you could write a DSL to generate safe performant asm for a video codec. Just not a platform-independent one. It would still have to be asm.

At the very least, you go through all of their patents and try to get them invalidated by the hundreds.

I can see a number of ways to work around that limitation, without even lobbying and bribing. And I'm not even a lawyer or an accountant.

Eventually all the money and power will converge in a few sub 500, or sub 50, companies and nothing will change.

The commons in question is the internet itself.

Thank you for describing the tragedy of the commons

There’s definitely lots of problems with the ruling class and wealth disparity. Perhaps the defining problems of our current age.

That being said, so many of the plebs suck. Like 2% will ruin everything for everyone.

tragedy of the commons with your ideological buzzwords sprinkled in, truly innovative

> a gif is not a video.

They're not that different; the image codec WebP is derived from VP8's intra-frame coding.

You cant really cache the dynamic content produced by the forges like Gitlab and, say, web forums like phpbb. So it means every request gets through the slow path. Media/JS is of course cached on the edge, so it's not an issue.

Even when the amount of AI requests isnt that high - generally it's in hundreds per second tops for our services combined - that's still a load that causes issues for legitimate users/developers. We've seen it grow from somewhat reasonable to pretty much being 99% of responses we serve.

Can it be solved by throwing more hardware at the problem? Sure. But it's not sustainable, and the reasonable approach in our case is to filter off the parasitic traffic.

I think there's a few things at play here

- AI scrapers will pull a bunch of docs from many sites in parallel (so instead of a human request where someone picks a single Google result, it hits a bunch of sites)

- AI will crawl the site looking for the correct answer which may hit a handful of pages

- AI sends requests in quick succession (big bursts instead of small trickle over longer time)

- Personal assistants may crawl the site repeatedly scraping everything (we saw a fair bit of this at work, they announced themselves with user agents)

- At work (b2b SaaS webapp) we also found that the personal assistant variety tended to hammer really computationally expensive data export and reporting endpoints generally without filters. While our app technically supported it, it was very inorganic traffic

That said, I don't think the solution is blanket blocks. Really it's exposing sites are poorly optimized for emerging technology.

They are a scourge, they never rate-limit themselves, there are a hundred of them, and a significant number don’t respect robots.txt. Many of them also end up our meta:no-index,no-follow search pages leading to cost overruns on our Algolia usage. We spend way too much time adjusting WAF and other bot-controls than we should have.

Yes, it's that BIG of a load: https://status.sr.ht/issues/2025-03-17-git.sr.ht-llms/

You've gotten several comprehensive responses so far and I want to add a niche corner that people might assume might not have the bot problem but still does.

I run a website that hosts tools for my family: games and a TV interface for the kids, remote access to our family cloud and cameras, etc. Sensitive things require log in and have additional parameters required for access of course.

I specifically blocked bots from search engines so my site is never indexed, as I'm not selling anything nor want any attention, as well as some other public non-malicious bots in case they communicate with Google, just to be safe there, and my robots.txt doesn't allow anything.

I assume then, that the only way a bot could even find my site is to do what the indexers do: brute force try every single possible ipv4 address hoping to hear something back, as my domain should not be known (and isn't simple enough to be quickly guessed), and most traffic must be malicious, or indexing (AI overview and other scrapers won't be finding it via web search).

Since it isn't indexing, and keeping everything in simple black and white boxes, my remaining traffic is family or malicious bots, and 99.9% isn't family.

I currently have the most strict bot-blocking setup I could come up with, which nicely cut down on quite a bit of traffic, but I do still receive ~2k attempts per day, which as you can imagine, still is around 99% not traffic, as I have fewer than 20 kids, and my kids aren't using the site nonstop.

Conveniently, my setup has never accidentally blocked a family member, so I'm pleased with the setup.

Agree. Did some basic searching and looks like Gitlab is particularly bad. It ships with built in rate-limiting but the backend marks all pages as uncacheable on top of them being somewhat dynamically generated (I guess it caches "page fragments").

The only issues I found amounted to "here's how to use Anubis to block everything"

There's also some new but poorly supported standards around agents setting `Accept: text/markdown` and https://github.com/cloudflare/web-bot-auth

> Wordpress runs like 70% of the web and you can really feel it from the 1500ms+ TFFB most sites have. PhpBB is not much better.

At least phpBB died 15 years ago with most communities migrating to Xenforo. I'm not quite sure how or why WP is still around with so many SSGs and SaaS site builders floating around these days.

The funniest part about WordPress is that you can usually achieve at least a 50% speed boost or more by adding a plugin that just minifies and caches the ridiculous number of dynamic CSS and JS files that most themes and plugins add to every page. Set those up with HTTP 103 Early Hints preload headers (so the browser can start sending subresource requests in the background before the HTML is even sent out, exactly the kind of thing HTTP/2 and /3 were designed to make possible) and then throw Cloudflare or another decent CDN on top, and you're suddenly getting TTFBs much closer to a more "modern" stack.

The bizarre thing is that pretty much no CMS, even the "new" ones, seems to automate all of that by default. None of those steps are that difficult to implement, and provide a serious speed boost to everything from WordPress to MediaWiki in my experience, and yet the only service that seems to get close to offering it is Cloudflare.

Even then, Cloudflare's tooling only works its best if you're already emitting minified and compressed files and custom written preload headers on the origin side, since the hit on decompressing all the origin traffic to make those adjustments and analyses is way worse for performance than just forwarding your compressed responses directly, hence why they removed Auto Minify[1] and encourage sending pre-compressed Brotli level 11 responses from the origin[2] so people on recent browsers get pass-through compression without extra cycles being spent on Cloudflare's servers.

The solution seems pretty clear: aim to get as much stuff served statically, preferably pre-compressed, as you can. But it's still weird that actually implementing that is still a manual process on most CMSes, when it shouldn't be that hard to make it a standard feature.

And as for Git web interfaces, the correct solution is to require logins to view complete history. Nobody likes saying it, nobody likes hearing it. But Git is not efficient enough on its own to handle the constant bombardment of random history paginations and diffs that AI crawlers seem to love. It wasn't an issue before, because old crawlers for things like search engines were smart enough to ignore those types of pages, or at least to accept when the sysadmin says it should ignore those types of pages. AI crawlers have no limits, ignore signals from site operators, make no attempts to skip redundant content, and in general are very dumb about how they send requests (this is a large part of why Anubis works so well; it's not a particularly complex or hard to bypass proof of work system[3], but AI bots genuinely don't care about anything but consuming as many HTTP 200s as a server can return, and give up at the slightest hint of pushback (but do at least try randomizing IPs and User-Agents, since those are effectively zero-cost to attempt).

[1]: https://community.cloudflare.com/t/deprecating-auto-minify/6...

[2]: https://blog.cloudflare.com/this-is-brotli-from-origin/

[3]: https://lock.cmpxchg8b.com/anubis.html but see also https://news.ycombinator.com/item?id=45787775 and then https://news.ycombinator.com/item?id=43668433 and https://news.ycombinator.com/item?id=43864108 for how it's working in the real world. Clearly Anubis actually does work, given testimonials from admins and wide deployment numbers, but that can only mean that AI scrapers aren't actually implementing effective bypass measures. Which does seem pretty in line with what I've heard about AI scrapers, summarized well in https://news.ycombinator.com/item?id=43397361, in that they are basically making no attempt to actually optimize how they're crawling. The general consensus seems to be that if they were going to crawl optimally, they'd just pull down a copy of Common Crawl like every other major data analysis project has done for the last two decades, but all the AI companies are so desperate to get just slightly more training data than their competitors that they're repeatedly crawling near-identical Git diffs just on the off-chance they reveal some slightly different permutation of text to use. This is also why open source models have been able to almost keep pace with the state of the art models coming out of the big firms: they're just designing way more efficient training processes, while the big guys are desperately throwing hardware and crawlers at the problem in the desperate hope that they can will it into an Amazon model instead of a Ben and Jerry’s model[4].

[4]: https://www.joelonsoftware.com/2000/05/12/strategy-letter-i-... - still probably the single greatest blog post ever written, 26 years later.