Improving cursor rendering on Wayland(blog.vaxry.net) |
Improving cursor rendering on Wayland(blog.vaxry.net) |
1. Don't have a dependency on hyprlang. Other projects will probably be averse to that. Use a more common configuration language, using a common library.
2. Have a shim layer so that hyprcursor cursors can be used by applications that use the XCursor API (maybe have a drop in replacement library for libxcursor that delegates to hyprcursor).
3. Have a specification so other implementations can use the cursor format as well, in case other compositors/clients don't like your implementation for some reason.
Use XDG ini.
https://gitlab.gnome.org/GNOME/gtk/-/merge_requests?search=w...
https://gitlab.gnome.org/GNOME/gtk/-/issues/?search=wp-curso...
Nothing on Mutter either:
https://gitlab.gnome.org/GNOME/mutter/-/issues/?search=wp-cu...
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests?searc...
It looks like the old GTK mailing lists were moved to Discourse at: https://discourse.gnome.org - so I tried searching wp-cursor-shape:
https://discourse.gnome.org/search?expanded=true&q=wp-cursor...
Maybe it can be found under other terms of course, but all I'm saying is, before picking up the pitchforks, maybe it'd be worth asking if they'd just accept an MR. Yes, I know many (myself included) are not thrilled with GNOME's approach to Wayland, but it's not going to be any more productive to just assume bad faith, at some point you just gotta push forward.
Or if that’s not the reason, the behaviors of a lot of projects RH birthed or heavily influence make a whole lot more sense if you assume it is and are fairly lacking in explanation otherwise. If it’s not the case, they’ve managed to accidentally do something that’s in their interest, I guess.
I'm sure that why they invested in Flatpak, they just really want it to make it impossible for people to run non-standard packages on Red Hat. Makes total sense.
GNOME is also the one holdout that doesn't support server-side rendering of window decorations (like the title bar with buttons to close and minimize).
GNOME has pushed back on several wayland protocol extensions that all the other compositors supported.
If other compositors want to do it, they have their optional extension for that. Eventually, they will find out why they shouldn't do it. With mandatory extension, there would be no way back, with optional, there is.
Gnome are consistently the hold-out, and are the source of much drama and chagrin in Wayland.
macOS cursors are provided by an NSCursor class which can either be one of the standard cursor image or provided through an NSImage object, which supports a number of data types (among the usual pixel-based formats like PNG or JPEG also PDF), it looks like applications can also register their own data formats via a class called NSImageRep.
I heard somewhere that Haiku would use it for its own cursors, but my google searches for getting that verified have not been fruitful.
[1]: https://www.haiku-os.org/news/2006-11-06_icon_facts/
[2]: https://www.haiku-os.org/articles/2006-11-13_why_haiku_vecto...
https://patents.google.com/patent/US5471570A/en
EDIT: The linked patent seems to only cover 1 bit inversion, not full color inversion. I can't find the patent for the full color, but recall reading that a patent stopped the adoption of a nice color inverting cursor for Linux that Windows has had for so long. If Windows has had it for as long as you say, then perhaps any relevant patents have expired?
Screensharing is handled by pipewire [0], changing keyboard layouts aren't defined [1] by wayland, and generally anything Wayland devs think would 'corrupt' their protocol.
They leave most things to the compositor to implement, which leads to significant fragmentation as every compositor implements it differently.
Long gone are the days of xset and xdotool working across nearly every distro due to a common base, now the best you'll get is running a daemon as root to directly access `/dev/uinput` [2] or implementing each compositors accessibility settings (if they have them) as a workaround.
[0] https://superuser.com/questions/1221333/screensharing-under-...
[1] https://unix.stackexchange.com/questions/292868/how-to-custo...
Well, no, it's not that simple.
Rasters are annoying. Raster means an image is stored with pixels, for example a png, or an mp4. The problem with rasters is that when you resize them, the computer has to take a wild guess at what it should look like."
The problem with any display this software is going to run on is that it is also made with pixels. GTK foundation tried this approach for years, putting all their eggs into this basket, and recently having a reckoning and admitting this approach has not been working. Pixels are here to stay.
I found one issue (https://github.com/hyprwm/Hyprland/issues/2945) mentioning something similar, but no resolutions as of yet, so not sure what's going on.
Gnome3 doesn't manifest the same issue, so worth checking out if it happens to you if you're curious about moving from Gnome to Hyprland.
But encoding the images with PNG does make sense, given the wide availability of libpng and the potential file size savings. I don't know how the feel about SVG support. One the one hand using vector graphics for cursors seems like a great idea but SVG is a very complex format, with the only real parser available being librsvg, which massively increases toolchain complexity. Too bad PostScript doesn't support transparency.
[0]:https://www.x.org/releases/X11R7.7/doc/man/man3/Xcursor.3.xh...
I’m not sure we want a turing-complete language as a cursor format. ;)
But I agree that a simpler vector format than the kitchen sink that is SVG would be good.
At first, I thought it’s hlc [0], but I couldn’t find confirmation in the git repo either.
[0]: https://www.linode.com/docs/guides/introduction-to-hcl/
https://github.com/hyprwm/hyprcursor/blob/main/docs/MAKING_T...
Looking at the full description of the format, it includes the ability to use nested config sections with `{ ... }`, which has some obvious benefits for some use-cases. But for the complexity needed for defining cursors, I don't see the benefit of `.hl` over fdo `.desktop` configs. In fact, it looks like you could parse the hyprcursor `.hl` config files with a `.desktop` parser? Which might be more palatable for cross-desktop support?
Then again, I have no stake in the matter. I think if Vaxry wants this to be adopted though, dropping it onto the world like this is probably not the most likely case to get there. It probably would've been worth bugging maintainers for other compositors (kwin, sway?) to see if they had any input/thoughts on the matter. I can only imagine Hyprlang as a dependency would've come up immediately as a discussion topic (I could be very wrong though.)
Wait what. That's even more insane than client-side decorations.
s/SVG/a vector format/
There are some legacy X specifics in the theme format that would necessitate something more modern. IIRC, Xrender used a hash of hard-coded X pixmaps to find the cursor files to match. That's why some of the filenames are a long string of hex digits. The rest are predefined X cursor names.
For one, "network first" for a GUI is not a sane model, and that is only more and more true as more "compute" is pushed onto the GPU.
People like to say this, yet time and time again, X's design proves to be the superior one in the real world.
Some of it could use minor revisions (adding some image compression would be fine, etc), but it is hard to seriously say things are "highly dubious" compared to the competition.
Umm this does not pass smell test.. 96x96x4=36864 bytes of raw pixel data. How does that become 2MB?!
And what's wrong with that?
Basically, in the "old Wayland way", the Wayland compositor gave the client a surface and said "hey, draw something into this and I'll make it your cursor" and then the compositor well, composited it, using either true hardware cursor or an overlay layer of some kind. If the app froze, the cursor would still move, it just might not be updated. This seems clean architecturally but turns out to suck because now every client needs some kind of cursor drawing library in it, hopefully the same one everyone else is using so that cursor theming works and you don't get a different default mouse cursor in every app.
So then Wayland introduced wp_cursor_shape, which is where the client instead says "hey, draw a 'standard pointer' cursor over me please." But, the "old" client-drawn way is still supported and used, for example in GTK apps.
This proposal seems to just be a really complicated recommendation to change the file format used behind wp_cursor_shape, and a plea for "old" apps to either adopt this format/library or stop using client rendering. I think that switching to wp_cursor_shape pretty much Makes Sense for most apps, but the article didn't really sell me on this file format.
Most distributions still use a "Live CD" format, actually a "Live DVD" nowadays (with a few hacks to make it also work as a USB pen drive image), for their installers, and that limits the installation image size (which also includes enough packages for an offline install of a desktop environment) to a bit more than 4GB (they fit into a common 8GB USB pen drive). So it being the "age of Terabyte HDDs" does not matter, for packages expected to be installed on a normal desktop system.
1980: Personal Computers with mouse become popular, apple Lisa, Microsoft Windows 1
2000: Touch devices with stylus are popular
2007: Apple releases Iphone with finger touch
2015: Conversational interfaces like Alexa, Siri start becoming available.
2022: ChatGPT presents a huge tech improvement in conversational interfaces.
2024: "Actually let's rebuild this technology from the 80s from the bottom up. That's what we need"
I do not regret moving away from open source/free software.
This allows the application to customize the cursor, which happens more frequently than you might expect, e.g. dragging and dropping a file changes the "cursor" to include the little preview of the file. So the client already needs to learn how to render cursor images to composite the cursor image on top of the DnD preview.
One of Wayland's original core philosophies is to share behavior by shared library, rather than by protocol, because social agreement is already required, and is way better for a community long-term. So there is also a libwayland-cursor library that allows you to load Xcursors, which allows the client to do all the compositing and customization it needs, while still supporting a standard.
zwp_cursor_shape makes sense for a basic client without too many needs, but it's not a great idea for a full toolkit, otherwise DnD dragging (or any other case that requires compositing cursor images) will show a different cursor.
Also, SVG for cursors is not really a great idea.
Of course, because this is Wayland, one good idea suffers due to coexisting with other independently good ideas. The minds behind Wayland didn't want the protocol to ossify like X11 did, so they made extensions essential. If a given functionality doesn't need to be core, it's an extension.
Both great ideas in isolation, yet now, as a result, Wayland has trapped its ecosystem in a half-assed state of completeness. The one saving grace is that, unlike X11, things will actually improve over time as the gaps get filled in and older applications age out of the ecosystem.
In general, I think that is a good design.
The problem is the desktop compositors really struggle to agree on useful extensions. Which results in interoperability problems.
This is particularly important because most applications aren't designed to render at monitor framerate, whereas games make an exception because they tend to be full screen applications & tend to render at a high framerate already
I think it'd be best if "another vector format" was really "another cursor format which allows the actual cursor data to be tagged with a format". That way every time someone wanted to change the cursor protocol to allow compressed bitmaps or animated files or different vector formats it didn't require a whole new cursor file format to do it. Think something like MKV but for cursor data.
Because Gnome is RedHat and ...
1) Because RedHat is one of the few companies that puts sustained, long-term funding beind developers working on Linux. As such, they get an outsized say in what goes on because they are doing the work. If you would rather that Linux go a different direction, fund a bunch of programmers and take it that way.
2) RedHat put in the work, time, and money to get certified in ways that allows it to be used for big business. That means adhering to things like accessibility, auditing, etc. If you get the same certifications, big business accounts can use your stuff instead.
If you want people to ignore Gnome, all you have to do is fund a bunch of Linux developers to do all the work they are doing. Easy peasy.
It's not the first time I see people air their grievances like that so I gotta wonder why people keep working with them. Let red hat pay the developers to take Linux in whatever direction it wants. Other developers can go in some other direction that they agree with. The beauty of Linux is the kernel enables such a diverse user space. That's what I personally care about.
Of course issues still get stalled on this, because it takes a while to figure out that the disagreement is to fundamental. Often a compromise is actually found in the end.
> Will this ever be adopted?
> I don't know! All I know is that it's a clearly superior system that is easy to implement.
> The best path, in my opinion, would be to push for wp_cursor_shape adoption over at gtk. If gtk finally supports the protocol, almost all desktop apps that people might wanna use on Linux will be able to utilize hyprcursors via the compositor.
> Support from the toolkits themselves for hyprcursor itself is likely not going to happen, as hyprcursor is not made by a million-dollar non-profit entity via a 10-year-long bureaucratic process, unless somehow the developers over there decide it's more beneficial to use hyprcursors now rather than wait and develop their own, new standard. Who knows? :)
On one hand, I agree with toolkits simply adopting wp-cursor-shape. This makes a lot of sense, and unlike with the CSD argument, I don't really think it would place a particularly hard burden on Mutter to just implement this protocol.
On the other hand, though, if other compositors want a better solution than XCursor, they either have to adopt this or something else, or invent their own system. And Hyprland, by virtue of being a fairly popular and pragmatic Wayland compositor, will no doubt collect a fair number of themes in the Hyprcursor format.
And based on this particular wording:
> All I know is that it's a clearly superior system that is easy to implement. ... hyprcursor is not made by a million-dollar non-profit entity via a 10-year-long bureaucratic process, unless somehow the developers over there decide it's more beneficial to use hyprcursors now rather than wait and develop their own, new standard
This seems to suggest that Vaxry has:
- A belief that this really should be adopted, as a superior solution to cursor themes
- Some level of disdain for the idea that other developers might make their own standards instead of adopting this
Which honestly, is true... but that makes it all the more a shame that this probably won't happen for relatively unimportant reasons, meaning that likely, we'll end up with another cursor theme format later on that will be incompatible.
Not blaming them for not trying to do this as it seems Hyprland has mostly succeeded by simply blazing forward without gaining consensus, but there's a two way street to that approach IMO.
I mean, isn't that how Cinnamon and Panteon and Unity and Cosmic and MATE were born?
GPUs are highly complex and performance under generic wrappers.
Any USB drive is so super cheap these days, you can easily get something like 128 Gb for $15. Even if you live in an area where it’s not as easy (e.g. not very developed places), you’re likely to find, say, 32 Gb for less than $10. So I don’t know, maybe someone needs that 4 or 8 Gb limitation, but I believe it’s a non-issue in most cases.
Also, on top of that, we do have a very speedy Internet I’m so many places. Which means if you’re limited here, you can go with the net-installer. If you’re limited on both, more likely it’s a very niche case and you can have a spare large disk to download everything offline.
But neither does the "size on disk" because the ACTUAL size on disk might be reduced by gzip/bzip2/xz/zstd/whatever is invented later. Even if the program using the package doesn't support compression, it supports filesystems and filesystems often support some kind of transparent compression. It isn't a hard problem to solve.
1) Synchronization: when you are rendering decorations in a different process that the window content, you never get frame perfect. You can't synchronize both processes for each frame.
2) When painting using two processes, you need two surfaces. You just doubled your VRAM usage and the amount of blitting you need to do.
Now, decorations are not only the window chrome itself, but parts of them are font rendering (Qt renders fonts a little bit differently than Gtk, you can see the differences) and popup menus (when right clicking on the window chrome). The menus are a difficult thing; they basically pull in the entire respective UI framework (theming, sizing, font rendering, ...). So you do not really have a choice of unified look with server-side decorations; you have choice with cluttered mess between apps using different frameworks, or cluttered mess inside apps using different framework than the one used to render decorations.
So, how the other operating systems do it? They won't allow you to connect directly to compositor in the first place. You _must_ use the platform library, be it Cocoa or Win32 in order to be able to get a surface that gets displayed. And since you are already linking these libraries, they will paint the decorations for you -- unless you override it. All that is client-side, in the address space of your application, in your message queue.
Now, Windows does have a server-side fallback, when you app is not processing the message queue. Then it will paint the system decorations, but since your app is not responding, any synchronization is passe anyway.
So how to solve this in Linux? You will have as many looks, as many apps decide that they do not want to play ball, that they are a special snowflake, that need to avoid the "bloat" of frameworks. The cluttered mess of different UI styles and the permanently broken state of everything is a price you are paying for them to be able to do exactly that.
Otherwise, you would be able to reduce the looks to two (Qt and Gtk). When things would go very well, Qt could do what it does on other platforms and use Gtk+ look. That's about the only way to get unified look.
1. Different apps will draw decorations differently, leading to a lot inconsistency. 2. It means apps designed for other environments that don't draw their own decorations won't have any decorations. 3. If your client doesn't use one of the big all encomposing gui toolkits (not every app needs that), then it has to implement the decorations itself, which is non-trivial to do correctly (I know because I have worked on it for such an app). 4. You can't reliably configure the appearance and behavior of the decorations in a central place for all applications.
Fuethermore, if clients always draw their own decorations, then if you use a tiling wm and don't want decorations at all, you are stuck with them.
2) Only if they use server-side decorations only. In Wayland, client-side is mandatory and server-side optional. So if they do not have decorations, they are broken.
3) You client should use one of the big frameworks. On other operating systems, they are mandatory anyway. Only Linux allows your client connecting to compositor directly. Use that power responsibly. Frameworks bring lot of stuff implemented for you. If you reject that, it is up to you to implement that yourself.
4) You can't even with server-side decorations either.
Yes, with tiling, you get them. However, many apps have toolbar there, so you need it anyway.
Why?
https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/6212
No negative signals for either Mutter or GTK here, but no strong positive signals either. Let's make sure we add our thumbs up, at least.
This is just because people do not use Wayland. But, like in the case of SystemD, it seems to be pushed behind their backs.
If GNOME does anything I personally don’t like it’s usually the result of a grand conspiracy by Red Hat to make the Linux desktop worse, increasing their consulting profits. (/s)
I must admit, I have wondered about the plausibility of this in the past. It seems like ironically, making a problem-free Linux would be a conflict of interest to Redhat, which makes more money the more problems it has.
Is there any particular reason you think it couldn't be the case? I could definitely see it within the realm of possibility, especially since they maintain a huge chunk of the ecosystem nowadays (Fedora/RHEL, GNOME, LibInput, Kernel development, NetworkManager, Pulseaudio/Pipewire, Systemd, Wayland, and probably even more in the future.)
Many of the contributes to these project, even when being employed by Red Hat don't always do this as their primary job, many do it because they want to. It just so happens many people that love the linux ecosystem and linux itself work at Red Hat.
Other then that, many project that are very unifying and don't create much 'conflict' are also lead or worked on by Red Hat people. Pipewire for example has been the opposite of 'conflict' for the most part. LVFS is another good example. Flatpak/Flathub are another great example (Maybe Ubuntu is the evil agent of chaos). Systemd is also almost universally adopted by ever major distro even if it created much conflict.
Somehow people can't decide if Red Had is an evil dictator wanting to control everything or if they are deliberately creating 'conflict' so nobody is in control.
The simple reality is, there are a huge amount of people with incredibly different opinions in the Linux space. Even different people employed by Red Hat also don't always agree with each other. So who is really the true Red Hat conspiracy agent?
Difference on opinion is fine as long as people work on their own stuff, but when it comes to unification, like Wayland protocols it just gets really hard. The reason there is KDE and Gnome is because of fundamental disagreements about goals and that will be reflected in how they think wayland properties should be installed.
P.S:
> making a problem-free Linux would be a conflict of interest to Redhat, which makes more money the more problems it has.
What is your actual evidence for this statement? If that was true, why are they working on all those projects. Objectively things like Pipewire, LVFS, Flatpak, Systemd have been major improvements to the ecosystem.
It’s like saying you should use the same file access algorithms for a RAM disk and punch cards. No you shouldn’t!