On a Boat(moq.dev) |
I'm currently switching to a quic-based solution for other reasons, mainly that webrtc is a giant blackbox which provides very limited control[1], yet requires deep understanding of its implementation[2] and I'm tired[3].
I looked at moq-lite but decided against it for some reason. I think because I have <5 clients and don't need the fanout. The auth strategy is very different than what I currently use too.
[1] Why is firefox now picking that (wrong) ice candidate?
[2] rtp, ice, sdp, etc
[3] webrtc isn't bad for the video conferencing use-case but anything else is a pain
* Firefox support for WebCodecs is poor—none at all on Android [1], H.265 is behind a feature flag. [2]
* Mobile Safari doesn't support WebTransport. Or didn't...I just looked it up again and see it does in 26.4 TP. Progress! [3]
[1] https://searchfox.org/firefox-main/rev/da2bfb8bf7dc476186dfe...
[2] https://searchfox.org/firefox-main/rev/da2bfb8bf7dc476186dfe...
- libav.js for AudioEncoder/AudioDecoder. - QMux over WebSockets for WebTransport.
Both are NPM packages if you want to use them. @kixelated/libavjs-webcodecs-polyfill and @moq/qmux
26.4 removes the need for both so there's hope!
Just features/software need to be implemented?
If you are still struggling with WebRTC problems would love to help. Pion has a Discord and https://webrtcforthecurious.com helps a bit to understand the underlying stuff, makes it easier to debug.
[0] https://datatracker.ietf.org/doc/html/rfc8445#section-7.2.5....
You can convert any push-based protocol into a pull-based one with a custom protocol to toggle sources on/off. But it's a non-standard solution, and soon enough you have to control the entire stack.
The goal of MoQ is to split WebRTC into 3-4 standard layers for reusability. You can use QUIC for networking, moq-lite/moq-transport for pub/sub, hang/msf for media, etc. Or don't! The composability depends on your use case.
And yeah lemme know if you want some help/advice on your QUIC-based solution. Join the discord and DM @kixelated.
Probably never had to work with (live) video at all? I think using moq is the dream for anyone who does. The alternatives—DASH, HLS, MSE, WebRTC, SRT, etc.— are all ridiculously fussy and limiting in one way or another, where QUIC/WebTransport and WebCodecs just give you the primitives you want to use as you choose, and moq appears focused on using them in a reasonable, CDN-friendly way.
streaming might seem like a world to you by it is a small niche of live video;)
unless it has p2p in browser support, it has nothing against webrtc for live video calls which doesn't need a server.
Either side can abort a stream if it's taking too long, clearing the send buffer and officially dropping the data. It's a lot more flexible than opaque UDP send buffers and random packet loss.
FEC would make the most sense at the QUIC level because random packet loss is primarily hop-by-hop. But I'm not aware of any serious efforts to do that. There's a lot of ideas out there, but TBH MoQ is too young to have the production usage required to evaluate a FEC scheme.
I’ve been thinking about an application where people consume all their media, and having the ability to pick which tracks to pull for any content you want to stream would be great.
Edit: https://caniuse.com/?search=webtransport
Looks like the situation is the same as in 2024: "Yes, except for Apple devices?" If I'm reading this right, it looks like Safari will support it next week though...
But a huge difference is that there's a plan for congestion. We heavily rely on QUIC to drain network queues and prioritize/queue media based on importance. It's doable with multicast+unicast, but complicated.
Any idea what Firefox is waiting for? To me those lines I quoted seem entirely arbitrary, and a skim through bugzilla didn't help.
But WebCodecs is just really straightforward. It's hard to find anything to complain about.
If you have an IP camera sitting around, you can run a quick WebSocket+WebCodecs example I threw together: <https://github.com/scottlamb/retina> (try `cargo run --package client webcodecs ...`). For one of my cameras, it gives me <160ms glass-to-glass latency, [1] with most of that being the IP camera's encoder. Because WebCodecs doesn't supply a particular jitter buffer implementation, you can just not have one at all if you want to prioritize liveness, and that's what my example does. A welcome change from using MSE.
Skipping the jitter buffer also made me realize with one of my cameras, I had a weird pattern where up to six frames would pile up in the decode queue until a key frame and then start over, which without a jitter buffer is hard to miss at 10 fps. It turns out that even though this camera's H.264 encoder never reorders frames, they hadn't bothered to say that in their VUI bitstream restrictions, so the decoder had to introduce additional latency just in case. I added some logic to "fix" the VUI and now its live stream is more responsive too. So the problem I had wasn't MSE's fault exactly, but MSE made it hard to understand because all the buffering was a black box.
