How to Animate Multiplayer Cursors(liveblocks.io) |
How to Animate Multiplayer Cursors(liveblocks.io) |
Instead of sampling the mouse position every 100ms, you'd save off all the mouse positions, and then send the latest batch every 100ms. The other side would then replay the exact positions, just delayed by 100ms. It'll end up with the same latency as these motion smoothing approaches, while only using slightly more bandwidth.
The marginal benefit of exact cursor positions is so low that sending all that data still feels like a waste.
Every 100ms you fit the best Beizer curve to the last batch of mouse positions and send that.
It seems like that would give a more precise reconstruction than fitting a Beizer curve only to one sample every 100ms on the server.
There is a lot of prior art in this space, btw. I remember Meteor.js having a great real-time demo over websockets that actually used predictive techniques to keep things (imperfectly, but still impressively) 0 latency.
And you also can't set up your system with a 5 second delay to send data every 5 seconds, because any jitter will result in hiccups.
You could set up your system to send out the new data each time the previous buffer is acknowledged, but that's kind of pointless, if you get lucky with a good connection and can send data to be rendered 4.970 to 5.000 seconds from now, what's the difference for the user between doing that versus reducing the network load by approximately a factor of 3 and waiting until you have data for 4.900 to 5.000 seconds?
I think 100ms is a reasonable minimum batch size.
X and Y can easily be 2 bytes each, 4 bytes total. 100 samples per second is a mere 400 bytes per second. You could do it from a dialup modem from the early 90s!
So the remote user packets up 100ms of mouse movement with timestamps, sends it with ~100ms latency. Your side now has a buffer of ~100ms to start playing the positions back.
This also removes all jitter in the playback from varying latency (up to the point the jitter stays under 100ms).
All of the above numbers are made up for this example. You can adjust the playback delay as much as needed for smooth playback.
Sampling X amount per second is enough, and send every clicking position in between as those are valuable information. The rest is noise.
Was thinking maybe the Phaser game framework, but that might be too heavy, especially if there needs to be 7 or 8 of them on the same page.
Nothing requires you send just a set of points across. A better approach is to transform and compress the data BEFORE sending it across the wire. This way you get the benefit of using all the available data to create a more accurate simplification.
For example, each update take your cursor point set and construct a bezier curve that best fits the data.
Another advantage of doing it this way is that the sender can downsample on large/gross movements that would render faithfully as splines with fewer points, and send more samples on tight movements.
If you're not sending uniformly-spaced (in time) samples, though, you'll want some kind of timing information encoded as well.
One additional parameter to keep in mind is how real-time does your simulation of remote players need to be? If you don't need real-time positioning there's a whole other dimension of shortcuts you can take by introducing what amounts to a 'streaming delay'.
In a lot of these apps the cursors can't interact with each other so you have no need for real-time positioning and its accompanying smoothing techniques. Cheat cheat cheat! That's how games get their performance, way more often than being smart they're clever opportunistic cheaters!
But I'm torn by this. The spline approach seems to look the most accurate. But when all three approaches are shown at the same time at the end, I think the spring animation might look more visually pleasing. But then, if the spring approach is only degrees better than CSS transitions, is it really worth all the extra code?
It's like the AAA studio equivalent of a tech article.
I would be cool to see an example of how a Kalman filter approach would compare in terms of precision and latency. My expectation is that it would be the best of both worlds.
I like the potential of this approach as it lets you get smoother results than just CSS transitions yet doesn't require you to use a RAF loop-based animation library.
I developed a multi player version of SimCity for X11 that I released in 1993 and demonstrated at the InterCHI '93 Interactive Experience, which showed you other player's cursors moving around and editing the map, but of course it required a fast network to run on and updated all the clients synchronously, due to the limitations of X-Windows, so there were no interpolation issues. (X-Windows clients aren't even capable of performing local computation and feedback the way NeWS clients could and web browser clients now can, so it was a moot point.)
https://www.donhopkins.com/home/catalog/simcity/simcity-anno...
https://www.donhopkins.com/home/catalog/simcity/simcitynet.h...
The multi player demo showing different kinds of voting dialogs, multiple cursors, the tool palette and pie menus, and the voting "yes" shortcut of building the same thing in the same place, starts at 5m45s:
https://www.youtube.com/watch?v=_fVl4dGwUrA&t=5m45s
One interesting thing about the SimCity cursor was that it was color and shape coded to show which tool was selected. The tool palette (and also the pie menus which had the same icons and layout as the tool palette) served as a legend for the cursor by showing the same color coded outline around the icons as the cursor used. So you could tell which tool other users had selected. You could hide the tool palette to make the map bigger, and use the pie menus instead, which were much more efficient.
Multi Player SimCityNet for X11 on Linux: Demo of the latest optimized Linux version of Multi Player SimCity for X11. Ported to Unix, optimized for Linux and demonstrated by Don Hopkins:
https://www.youtube.com/watch?v=_fVl4dGwUrA
Micropolis Online (SimCity) Web Demo: A demo of the open source Micropolis Online game (based on the original SimCity Classic source code from Maxis), running on a web server, written in C++ and Python, and displaying in a web browser, written in OpenLaszlo and JavaScript, running in the Flash player. Developed by Don Hopkins:
https://www.youtube.com/watch?v=8snnqQSI0GE
Source Code:
https://github.com/SimHacker/micropolis
HAR 2009 talk: Constructionist Educational Open Source SimCity:
https://donhopkins.medium.com/har-2009-lightning-talk-transc...
It is not uncommon to refer to pointers as "mouse cursors" or shorten it to just "cursor". In fact, Wikipedia considers pointers to be a subset of cursors.
https://en.wikipedia.org/wiki/Cursor_(user_interface)#Pointe...
"In computer user interfaces, a cursor is an indicator used to show the current position for user interaction on a computer monitor or other display device that will respond to input from a text input or pointing device."
If they had said pointers there would be no confusion.
cursor:pointer/crosshair
Not sure. Multiplayer games are easier to predict player movement. Once someone starts moving forward, you can predict that they'll move forward for a little while, start turning, continue forward and so on. Add in physics (like the motions/movements of a car, or the running of a human) and there will be constraints the player can't break (when you stop moving forward, you'll move forward slower and slower until you stop, maybe over 100ms or so)
But mouse movement is highly erratic. It'll be short of impossible to add any sort of prediction, as it'll be incorrect most of the time, instead of being mostly correct but sometimes not.
The penalty for a wrong prediction is rubber banding, so if you can get away with streaming their past at high latency, I would avoid predicting at all.
That's the model often used by async PvE games (think social games, invest/express, etc) and I think that approach would map best onto SaaS presence features.
Your example is only relevant if the player just loves to hold down the "move forward" key for long periods of time.
https://www.gabrielgambetta.com/client-server-game-architect...
Call of Duty MW2 used to replay what your opponent saw, after your death. That always made an inexplicable death much easier to swallow.
Of course, they probably had to add that feature since they did away with dedicated servers, but it was still nice.
I don't need to reliably send the TCP packets in 33 ms. I just need to be able to start sending a packet every 33ms. Assuming sending, acking, etc is non blocking and uses few enough resources asynchronously it's fine.
I made note for us to write a blog post about it. Stay tuned :)
{“x”:50,”y”:56}
Ok, I guess that’s still not that much.