Show HN: A Vulkan-Video-based game streaming tool for Linux

Show HN: A Vulkan-Video-based game streaming tool for Linux(github.com)

4 points by colinmarc 2 years ago | 3 comments

I've been working for about 6 months on a new game streaming/remote desktop tool for linux. Gaming on linux is pretty exciting right now, given all the improvements in compatibility, but game streaming isn't really there yet, and the remote desktop space is fairly underdeveloped too.

The server acts as a wayland compositor to get the streamed app or game to render completely off-screen (somewhat like Gamescope, if you're familiar), and it uses Vulkan Video to pass the app's rendered textures directly to the encoder ASIC on the same card. The result is very fast and very low-latency, without a lot of moving parts. Running on the same machine, I get about 15-20ms median latency round-trip (from user input to the frame responding to that input being displayed on the client), which is about one frame at 60fps.

The client is the weakest link right now, but I'm working on a Swift UI macOS/tvOS client that should be a bit nicer to use. You'll also need a reasonably strong GPU to run the server.

Thanks for reading and checking it out! And please leave a bug report if it doesn't work for you, it's quite new and I've basically only tested it on my setup. Running the server with `--bug-report` will dump a video capture and trace log to a folder in /tmp, which you can attach to the report.

JustinGarrison 2 years ago |

Would the Swift UI also work on an iPad? Do you have any comparisons with other tools (eg steam streaming, moonlight)

colinmarc 2 years ago | |

> Would the Swift UI also work on an iPad?

Yes, but probably not for the first version.

> Do you have any comparisons with other tools (eg steam streaming, moonlight)

Steam streaming just doesn't really work on linux. Moonlight is somewhat similar in terms of direction, and has an established client base. I know of at least two projects to build servers for the Moonlight protocol[1][2].

The Moonlight protocol is a bit weird, because it's an open-source reverse engineering of a NVIDIA project, GeForce now. There are fundamental limitations to the protocol, for example that the cursor must be rendered in-stream or simulated. Using my tool, the cursor is rendered locally, and custom cursor images can actually be pushed to the client, for a seamless experience. This sounds like a minor detail but it matters a lot for subjective latency. I'm also working on employing tricks like hierarchical coding using FEC in the protocol, because I hate VBR encoding for games (it makes text blurry and breaks immersion). Those tricks aren't really possible in Moonlight.

All of the Linux solutions I know about have significantly higher latency compared to Magic Mirror, although I don't have numbers for exactly how much higher. (I have a benchmark to test the latency of my tool, but the others don't.) I'd encourage you to try them out and get a feel for the difference.

Finally, I think Magic Mirror is the easiest to install and get going on the server. It has almost zero runtime library or service dependencies (there's a pesky dynamic link against libxkbcommon which I haven't managed to remove), so you don't need to mess with pipewire or docker or anything - it's completely self-contained.

All that said, the existing tools have the advantage of a larger user and contributor base, whereas Magic Mirror is just me on a mission so far :) So they're likely to be much more stable and usable.

[1]: https://github.com/LizardByte/Sunshine [2]: https://github.com/games-on-whales/wolf

Edit: an earlier version of this article mischaracterized the status of GeForce now :)

JustinGarrison 2 years ago | | |

Thanks for clarification how it compares. I've used Steam, Moonlight, and wolf with varying degrees of success.

I don't game very often, but I almost never do it from my desktop. I've been interested in low latency, GPU enabled remote desktop clients too. Happy to help test tvOS or iPadOS clients when you have something in test flight.