Streaming Video and Audio with Low Delay(steinwurf.com) |
Streaming Video and Audio with Low Delay(steinwurf.com) |
Here's a Periscope post about LHLS: https://medium.com/@periscopecode/introducing-lhls-media-str...
Most systems that serve HLS media use fixed content-length segments, which requires knowledge of the length of a segment before the first byte can be sent over the wire. So, for a 5 second segment you would need to encode the entire 5 seconds before the first byte can be sent; this does not apply when streaming the segments with chunked transfer encoding.
Incidentally, at Mux we also use chunked transfer-encoding to stream video that is encoded on-the-fly with great performance.
That's incorrect. With DASH the latency depends on the fragment duration, not the segment duration. You can start sending the segment when its first fragment is generated, and use chunk-based HTTP transfer as mentioned in other comments.
Link for further details on low latency: https://www.gpac-licensing.com/2014/07/09/lowering-dash-live...
Each TS segment must start with a key-frame, and the GOP size can't exceed the duration of a segment (e.g. one second). Lowering the segment duration increases the frequency of key-frames, which has the effect of lowering the quality you can achieve at a given bitrate.
Is there an open source low latency video streaming solution for hobbyists?
All the code is on GitHub. I can get very low latency video. Most of the delay comes in the form of the speed of light being so slow.
On the web site in use jsmpeg.
Try using kurento [0].
Here’s a gif of RPi streaming through a CDN which itself is fast, showing timestamps, using janky ffmpeg I built (aka. Worst-case): https://twitter.com/iSpoogeDaily/status/986681611600084992?s...
Tuning is ongoing, learning as I go. Thanks for client buffering pointer.
Of course, sometimes peer-to-peer won't work for you. Maybe you have requirements that push you towards routing media through a server. (Content filtering, or compositing video or mixing audio, for example.) Or maybe you have more than a few people in a call. If so, upstream bandwidth and encoding become bottlenecks for mesh/peer-to-peer. Finally, some firewalls won't allow UDP traffic from/to computers behind them, so you'll need to route UDP through a central server, or (much worse) tunnel over TCP.
Back on the subject of latency and error correction in WebRTC, here are some fun links:
Draft spec for FEC in WebRTC: https://tools.ietf.org/html/draft-ietf-rtcweb-fec-08
Mozilla article from when they first turned on Opus FEC. Includes sample audio for calls with 19% packet loss. (19% packet loss is very, very bad. My startup makes a browser-to-browser video calling tool, and we try hard to deal well with packet loss that high, but it's a losing battle.) https://blog.mozilla.org/webrtc/audio-fec-experiments/
Notes from the very knowledgeable folks at Callstats.io about WebRTC FEC. Covers some of the same material as this thread's original post: https://www.callstats.io/2016/11/09/how-to-recover-lost-medi...
Tsahi Levent-Levi's benchmarks showing how a few different media servers perform in the context of 10% packet loss: https://testrtc.com/webrtc-media-server-packet-loss/
I haven't looked at RLNC, but it does seem to have some gains over more traditional FEC schemes.
In the article we simply show that substituting one "old" algorithm for a more modern one can give you much better efficiency (protection against packet loss) for the same bandwidth and latency/delay budget.
Did that make sense?
I have been working on a better web interface that has on screen controls, because most people seem to want to type in like twitch, but the arrow keys are what you use.
The video is very different, the overhead is a lot more, 128kbit vs 2Mbps for example. On top of that, a video player will try to buffer ahead enough video to make sure you won't get and buffering later on, which means it needs to measure your bandwidth and try to guess how much.
UDP can be nearly instant except in for b-frames, however if it's over UDP is probably over transport stream which has it's own buffering levels built in to make sure the decoder has a large enough buffer.
This isn't even remotely true. The brain will happily interpolate all kinds of visual information that is missing which allows you to drop frames with abandon as long as you display an older frame instead of a 'blank space'.
Audio isn't nearly as forgiving and even a relatively modest jitter or number of lost packets will result in an un-intelligible stream of audio.
I've spent many years of my life pumping video and audio across the net, even when that wasn't yet considered normal (or even an intended purpose of the net), and I've rarely heard complaints about video quality. But audio delivery requires top notch connectivity, low latency and sufficient throughput if you intend to hold a conversation with someone that does not result in irritation. Video is far more forgiving.
RTP is more commonly used for video calls, which doesn't have inherent buffering (but most implementations have a small jitter/reorder buffer if needed). Transport streams are better suited for broadcast, which has looser latency tolerances.
Audio doesn't have such a concept, you get 33ms worth of audio, you can play it right away.
Of course, you can opt to not have b-frames and negate that issue.
You do, because TCP packets get lost at roughly the same rate as UDP packets do (though, there are some caveats here: on some congested links UDP packets tend to be dropped earlier).
The only major difference is that TCP will cause the packet to be - eventually - resent. The price of this complexity is added latency due to the requirement to buffer for a longer time than it could conceivably take to re-transmit packets so you can continue to stream. If you ever exhaust that time a TCP based system will grind to a halt.
In live video / real time conferencing such latency is really annoying and if it gets too long can be killing.
In our testing of lots of internet connections in the US, we've found that many of the "free" modems/wifi-routers that cable and phone companies supply really, really discriminate against UDP. It's not completely clear whether this is intentional, at least in part, or whether it's just buggy behavior.
There are several models of routers that you can reliable cause to just stop delivering UDP for a while, if you send enough UDP through them. This happens on their little internal Ethernet switches as well as over WiFi, so it's a firmware thing. Some of these routers will perk up and behave perfectly well right away, if you access one of the well-known DSL speed test sites, while flooding them with UDP packets. :-)