Hacking the LG Monitor's EDID(gist.github.com) |
Hacking the LG Monitor's EDID(gist.github.com) |
EDID rewrites are 99% of the time blocked by the monitor firmware: https://notes.alinpanaitiu.com/Decoding-monitor-EDID-on-macO...
By the way, one helpful tool that helped me navigate the EDID dump was Kaitai Struct [2]. It shows a side by side view with the hex view and the EDID structure, and it highlights the hex values in real time as you navigate the structure. Unfortunately [3] it doesn't support the extension blocks that the author needs.
[0] https://notes.alinpanaitiu.com/Weird-monitor-bugs
[1] https://forums.macrumors.com/threads/external-displays-swapp...
In the Linux world, stable identifiers are made from a combination of product name and serial number.
Are you sure this is still true? I had this problem for a very long time but it seems to have been fixed with a recent macOS update. It must have been one of the Ventura point releases.
If that's all the author needs, that might be the path of least resistance.
2. This reminds me why I no longer use linux on the desktop
The monitor supports 3440x1440 @ 144hz and in the EDID it’s setting that mode as preferred. I checked online and LG market the monitor with those specs, so it’s not an error.
It would probably have been easier to flip the bit in the EDID that says the 60hz rate is preferred rather than messing about with the timings - since there is a perfectly good 60hz timing in the EDID.
To me (without any other evidence presented or investigation on my part) this is more an issue with the graphics card driver on Linux than an issue with LG.
In that case it's not so much the EDID that's wrong but something else in his setup that won't work with those capabilities, and either Windows and Apple just don't default to maxing out the refresh rate, or they do but are able to detect that it's the wrong cable. Or it's a graphics driver issue?
I wouldn't be surprised if MacOS and Windows simply default to 60Hz unless manually selected otherwise, just to reduce customer service tickets.
Debugging this issue on linux is maybe an exciting journey, debugging it over the phone with a end-user who only has one cable and one monitor is just a PITA.
On the first one (a 34" ultrawide), all of its inputs lose connection for a moment whenever it wakes from standby (including the USB ports, making them useless for external drives). This also has the effect of causing my computer to occasionally lock up on resume from hibernation unless I tap the power button on the monitor before I wake the system. Additionally, at refresh rates above 60Hz the gamma gets progressively lower making the image darker the higher you go, even though the gamma setting in its menu is exactly the same, and black frame insertion and game mode are both off. Others online have reported the same thing.
The other monitor I got second-hand and it mostly works. It has a horrid HDR implemenation however that just washes out everything. I also tried to use brightness and input control via DDC/CI, which is a fairly well known standard, but this causes it to shut off abruptly and I have to unplug the power cord it to get it back.
Wow you just off-handedly resolved an issue I’ve been dealing with for months on my Linux desktop. I lost an hour of Baldur’s Gate progress the other night after this happened after I hadn’t saved the game. Thank you!
Your fix also resolves a similar issue I have on the Mac side when using my work laptop. If I tap the keyboard to resume from sleep, my LG monitor wakes up but doesn’t display an image. I have to wait for it to go through the motions until it finally displays “no input” before tapping on my keyboard to make it display the lock screen. Meanwhile, my second monitor works from the get go.
I somehow never thought to hit the power button first. I wish there were a way to make the LG monitor work like my other one and not have to do that.
Awesome! Glad I could help.
> I wish there were a way to make the LG monitor work like my other one and not have to do that
Yeah unfortunately that would be on LG to release a firmware update. And from what I can tell, they'd rather you just buy the new revised model.
As someone else noted, I'm considering overwriting the EEPROM in the monitor but I'd like to be 100% certain that's correct before I try it (one of the reasons I posted to HN was to see if folks thought I was going down the wrong path). I'm actually going to try a completely new cable first in case it's a bandwidth issue.
Dumped the EDID from the other monitor, put in a file in /usr/lib/firmware/edid, and added this to the grub command:
drm.edid_firmware=DVI-D-2:edid/asus-1920x1200.bin
Long Live Linux!
And the reason it works on Windows or (sometimes) MacOS is just that those systems have arbitrarily different default choices (e.g. "see if it has 60 Hz and use that"). And it happens to work with Windows because the monitor manufacturer bothered to test with windows (and, sometimes, MacOS), where they didn't with Linux.
This happens everywhere in the tech industry. No one does QA to the standards. It's 100% routine to see devices with PCI capabilities advertised that don't work, to see phantom ACPI entries for hardware that doesn't exist, to see EFI BIOSes exposing function tables for capabilities that never worked and weren't tested, USB devices which advertise a standard class but which only work with a proprietary driver, etc...
And the only reason anything works anywhere is that, at the end of the day, they plug it in and test. And on Linux they don't.
You can use it to force RGB mode, force 4K60hz, etc.
[1] https://github.com/waydabber/BetterDisplay
[2] https://www.analogway.com/emea/products/software-tools/aw-ed...
[3] https://github.com/waydabber/BetterDisplay/discussions/1473
[1] https://www.monitortests.com/forum/Thread-Custom-Resolution-...
I know the author says later that they don't do C development, but note that most of this code is just reading two consecutive bytes as a 16-bit integer, except for x[9] and x[17] where the high bit has a special meaning.
I have to do more involved full EDID reconstruction surgery tho since I need to add DTD entries rather than just change existing ones. So I'm looking at [edid-generator] together with [cvt12]. The latter can calculate xrandr modelines for VESA standard timings that all seem to work with my TV. cvt12 adds the option to calculate NTSC (1/1.001) timings over regular cvt which is already in Debian.
[cvt12]: https://github.com/kevinlekiller/cvt_modeline_calculator_12
[edid-generator]: https://github.com/akatrevorjay/edid-generator (thanks Kodi wiki)
Thanks for the pointers for those programs. Someone else pointed out Kaitai Struct could help me do the hex editing which I'm planning on taking a look at later
- automatic display switching when an input is off (bad if you’re attempting to troubleshoot a non posting machine)
- LED flashing when the monitor is soft-powered off, barely noticeable during the day but blinding in a dark room - good luck sleeping
- OSD behaviour where it disappears to quickly, particularly in a soft-off status - no ability to know what the firmware behaviour is until you buy it and test for yourself
The bug is with the EDID values LG programmed into the monitor.
Windows is generally most forgiving, then Mac, finally Linux.
This brought a tear to my eye. What a beautiful attitude, kj800x! Don’t ever lose it; that spirit will take you far.
Want to play video games that only run in Windows, but don't want to leave Linux? You can run Windows in a virtual machine and "passthrough" your GPU's PCIe slot. The advantage is that Windows can get the full performance of your GPU. The disadvantage is that it took the whole GPU away from your Linux host, along with whatever display it's attached to. You can use a second GPU for your Linux host (i.e. your integrated CPU one), but that needs its own physical display.
You can use Looking Glass to copy the framebuffer from your Windows guest so that you can redraw it on your Linux host. That way you can have your Windows guest in a window, and never have to leave your Linux desktop again. The big caveat is that your Windows guest needs a valid display connected, or it will never draw any frames to begin with. You can plug in a spare monitor that you never actually look at, or you can spoof one.
Also loved that it integrated so well into OBS directly to stream from!
Out of curiosity, what is the performance like? Presumably good for desktop use, but how about fps sensitive games?
http://videoprocessor.org/lldv
https://www.avsforum.com/threads/alternative-devices-for-ena...
https://blog.semtech.com/demystifying-dolby-vision-for-pro-a...
I returned OP's model display for a similar reason - except there was no HDR possible on Windows, even with the LG custom drivers. I also can't unsee the parallax on those models. It's pretty atrocious for the price.
Not surprised there are similar for HDMI.
Unfortunately despite huge progress with resolutions, refresh rates and ability to use one cable for lots of stuff (usb-c) we went significantly backwards in KVMs. If all your devices speak hdmi 2.1 there are one or two 4way devices that claim 8K-hdmi 2.1 (made by easycoo?), but if you have usb-c or newer displayport devices that don't support passive hdmi dongles your out of luck.
As far as I know there is one device on the market (made by delock) that claims to support DP1.4 (4k@60hz),but it has no edid emulation and there are no pass through edid emulators for displayport 1.4... So every time you switch kvm your displays will act as if they're disconnected/reconnected. If you do actual work on two pcs and you switch a lot that's horrible.
So the tldr is: great modes, but why did nobody think of the kvm users when making those standards (by for example making monitor hot plug detect being able to be disabled from monitor menu?).
Lest anyone read this and nodded along: This was a defect in the LG monitor the OP worked around, not anything to do with Linux.
From the original article "It works great on both Mac and Windows, but on Linux it displays just a black panel" and, amusingly, after the fix: "One issue is that now my second monitor isn't displaying anything. When I go into Display Settings, it seems like my computer thinks that both monitors are sending this EDID so it's put the second monitor into the wrong mode."
Seems like 2023 isn't the Year of the Linux Desktop either.
* I say "mostly" because to 99+% of the people, telling them "this monitor works great on both Mac and Windows and shows nothing but a black screen on Linux, but don't worry; it has nothing to do with Linux" will get you some pretty quizzical looks.
And as far as I can tell from the article, OP just assumed the display supported 60hz because it happened to show an image when they set it to that.
Based on debugging some issues I was having in macOS, I discovered from various Reddit and forums that Linux doesn’t handle that well depending on your driver stack/how you connect. If you’re on very modern hardware, kernels, and the OEM drivers it’s a better situation. macOS also has partially broken support for Freesync (or ProMotion as Apple brands it) at the moment on Intel machines.
I have an Asus GSync monitor with 48-144hz compatibility. This works over DisplayPort on my M2 Pro MacBook. On my RX6800 equipped Intel Mac I get 100Hz over DisplayPort 1.4 and 120Hz over HDMI 2.0 due to a DSC bug affecting Intel Macs. Even without that bug, 120Hz sometimes presents the OP’s issues between startup and login so I use 100Hz. The betas for the next macOS apparently fix this.
Either way, it’s a fun patch
Nothing else in computing is like this. We just cannot help ourselves & each other in most realms of computing: we must be content with what we are given, as it is.
In almost all probability the linux system either doesn't have a GPU capable of running this high pixel clock or the cable/connector can't handle it. I'd love to know what the Mac & windows machines do; do they run 60Hz too or are they pushing all 144Hz here successfully? This seems very likely to be a cable issue, one I don't expect windows nor Mac deal with particularly excellently.
I do know that Linux X11 (amdgpu kernel driver, modesetting X11 driver) tends to drive my DVI 1080p display with too high of a pixel clock (too large blanking intervals) when connected over a HDMI-to-DVI cable from my GPU. I believe this is because there's actually a duplication of mode selection logic between the amdgpu kernel driver and X11. I've reported another (system hang) amdgpu/X11 resolution bug at https://gitlab.freedesktop.org/xorg/driver/xf86-video-amdgpu... with no progress towards being resolved so far. Neither bug appears on Wayland, but mainstream Wayland desktop environments (KDE/GNOME) do not allow adding custom resolutions through xrandr without overriding EDID files and either rebooting for the kernel to see it, or touching files in /proc/ (untested).
The patch is quite simple: https://lore.kernel.org/lkml/20230701120806.11812-1-julius@z...
But I haven't been able to make the time to sit down and 1. respond to the feedback (original author hasn't yet) and 2. try and apply it to the otherwise standard Debian kernel.
Aside from this issue linux on the desktop has been quite pleasant, and of course this issue is not the fault of Linux per say. Deb12/KDE/Wayland/AMDGPU
I find it so immensely "WTF!?" that a kernel needs to know about a monitor, to change its brightness.
Also it should be trivial to run this as an out-of-tree module until it's merged. With DKMS or put hid_bl.c into an empty directory and add the following makefile:
``` ifneq ($(KERNELRELEASE),) # kbuild part of makefile obj-m := hid_bl.o
else # normal makefile KDIR ?= /lib/modules/`uname -r`/build
modules:
%: $(MAKE) -C $(KDIR) M=$$PWD $@
endif ```
Julius has responded to comments as recently as a week ago and is now working on a (hopefully mergable) v4.
You will probably see it in 6.7 and probably backported to stable kernels in a few months.
It will support any monitor which uses this USB control scheme (presumably a few other apple monitors).
So don't worry about making the time to address the comments. The author seems to be on top of it. Two to three months for a medium sized driver patch (such as this) to be merged seems about right to me.
I get a lot of computers to use for a brief period of time (consultant), and fixing that on Macbooks every couple months is non-trivial. Sometimes it’s impossible if it’s an older version of MacOS because older versions required overriding system files and booting into single-user mode.
Normal sane Linux user would just force resolution into bootloader, and create global xorg.conf. There are like 10 far easier ways to solve this problem.
They will also pick 10bpc even if it results in power consumption increasing by 30W (hello stupid AMD GPUs idle power consumption heuristics).
I have the impression (untested) that Windows seem to be less excited to select modes outside of the common ones, even if they are advertised in the EDID.
I kind of feel MacaOS does the same - indeed a quick test shows my monitor at 60hz even though there is a free sync range from 40-90 in the refresh rate. But of course this is a weird situation since it’s a free sync thing too, so I couldn’t apply it to everything.
So on that note, I go back to my original theory that it’s the graphics driver (or xrand or one of those x-things) screwing the timings up.
The EDID advertises YCBR colors. But in reality it only accepts RGB.
The amdgpu driver in my AMD laptop is the only machine I own that actually sends YCBR, and I had to add an EDID override for it. It's a pain to do. It's confusing. And then the HDMI port is now only capable this specific screen.
Thankfully the amdgpu driver has now be fitted with few knobs allowing you to override some of the settings at runtime.
… so did I. There's several models all under the same "brand" name, or whatever hardware manufacturers call the idiocy of sell 8 products under the same name.
Some of them have max refresh rates of 120 Hz. Some can do higher. But the OP never states (unless I am missing it) what model monitor he has.
(There is a model number in the EDID dump but it doesn't seem to correspond in format to LG's model numbers…)
I even went as far as to buy one of those "weird" [0] LG screens that have 4096 horizontal resolution all the timings and such worked perfectly.
[0] https://www.lg.com/us/business/download/resources/BT00001837...
I definitely had the same expectation, & was most way through reading, expecting my expectation wasnt going to be mentioned, but I was far from upset. I was quite happy to hear there's kernel workarounds for exactly this kind of thing.
The main shortcoming I feel right now is that this only works if you only have one specific monitor you want to hack, or you are ok rebooting. If the kernel had some way to dynamically override the edid that would be excellent. Maybe a eBPF filter?
So in total, on the order of a millisecond additional latency if you pass vsync timing alignment down, and no variable refresh rate.
But the latter is barely compatible with compositing in general. All techniques for handling a VFR video stream in a window should also apply to the uncompressed/shared memory looking glass stream.
Resolving assignments of cloned screens in the absence of physical-plugging-involving hotplug (that thus only comes from the monitor(s) also turning off when the laptop goes to sleep) by random instead of remembering physical ports/paths to them, across sometime as tame as a suspend-to-RAM, is frankly terrible UX.
Trying to use uniqueness features to be sticky across port swaps is good.... but I'd not be surprised if my grandparents would actually assume the way to swap them is by swapping the ports they are plugged into. Especially I'd expect them to be confused/surprised by swapping the plugs _not_ resulting in swapped content.
Also people running a setup like this are using the Linux host as their main desktop. If you only have one monitor and are switching inputs you lose your whole workflow on Linux host when using the vm. It’s better with dual monitors but I’d still prefer looking glass.
Unless of course the monitor doesn't have additional inputs (but it was never specified the hypothetical monitor is ancient) or they are all already used (again wasn't specified).
Then you should definitely use Linux more, and buy hardware that explicitly supports Linux.
Yes, of course. The point wasn't that I would buy a device for my own use (though that is kind of appealing these days), but rather that your choices influence what the market provides.
If you want display devices to support Linux, you need to preferentially buy devices that explicitly support Linux, and make that fact known to vendors as well as you can.
Sometimes this may be intentional design to lower compatibility and cause these problems intentionally, but often it is just bad engineering, not malice.
> The Apple Studio Display does not have any physical buttons and the only
> way to get or set the brightness is by sending USB control transfers to a
> HID device exposed by the display.Problem is, regular DDC utils don't know how to communicate with this particular display.
Joke is on me, I shouldn't have spent $1,500 on a display that is pretty much exclusively designed to be connected to a Mac. But I was using it with a Mac for a while before building this new Linux box.
But it's the codebase that you patch, so "patching the kernel" doesn't mean the OP ruled out building it as a module in the end. Even though they may well be able to just build it against kernel headers and even load it at runtime.
https://github.com/torvalds/linux/blob/master/drivers/gpu/dr...
There certainly isn't any big philosophical point to make here, Linux doesn't take a hard stance of expecting correct HW.
Ultimately this is a symptom of (1) bugs happen (here, at LG) and (2) Linux has a lower market share and therefore sees less testing. It's certainly fine to say "then I will use Windows to run into statistically fewer problems", so long as you're aware the same argument applies to any entrenched incumbent. As mentioned earlier, it'd have applied to MSIE in 2005 as well.
What's funny is, I was around in 2005 and had already adopted Firefox well before it was called Firefox. (I was also around for the release of IE 4, and spent half a day downloading it on our 56k modem on release day! Exciting times.)
That's because the web is what I work on, and I am OK taking on buggy/beta stuff in the web domain because I learn useful things.
In the OS space, I was a linux user for a decade before I realized that I was wasting tremendous amounts of time and energy debugging stuff very like this monitor issue, and getting no transferable benefit out of it.
I switched to mac at the time, and have experienced vastly less of this sort of configuration nightmare since.
I love linux and I root for it, and occasionally I still try to switch again, before I end up having to figure out this sort of issue that just empirically doesn't exist on my mac, then I get sad and switch back.
For Linux, however, the equivalent might be "report the bug so an exception can be made for this specific monitor". In other words, it's less important that everybody be immediately happy, as long as bugs can be reported and eventually fixed.
* Shorthand for any OSS kernel and its associated ecosystem
Ok, but personally quite a lot of my job involves working around the bugs and quirks of old unmaintained Windows OSes. The customer rightly doesn't care, they just want your software to work. It is almost always possible to at least mitigate the problems of whatever buggy garbage hardware/software you have to deal with.
However they patched this on the Windows side was quite fragile, because it kept breaking after OS updates. After I could no longer get it working with the reinstall and pray method, I switched to Linux because of this issue. I fixed it once with an EDID in the initramfs, and haven't had any issues for the last 6 years.
Reasonable and practical, but which direction did get us more progress for the web?
EDID exists for a reason and is a good thing; monitors providing reliable, useful EDID data is something to strive for.
It's gotten a lot better over the years as the result of operating systems using and enforcing it more. Monitors with bogus/garbage EDID used to be a lot more common 10+ years ago.
At first, this seems a reason not to use Linux, but a future upgrade of win/osx will break your hardware. Tons of hardware gets obsoleted this way. Meanwhile, Linux will just keep on working.
Yep. I first and foremost look fir Linux support explicitly called out in the system requirements. If I can't find it, I look for reviews mentioning Linux.
But explicitly supported is always the best, ideally backed by reviews that confirm it.
Still, I think it does matter what's technically going on and where the fault lies. I also think that just like browser diversity, OS diversity is a net positive for the enforcement of good standards that makes things work better for users overall.
For example, I'm willing to bet (and it's because I know cases of it :-) that many PC peripherals work better on Macs because Linux existing has made HW-manufacturers more standards-conscious than a Windows-only world would have, especially since so many HW/embedded engineers run it.
> I was wasting tremendous amounts of time and energy debugging stuff very like this monitor issue, and getting no transferable benefit out of it
You got me there - in my case it lead to a career of making cars, phones, game consoles and other stuff running Linux, so I guess the over-under on the direct utility shakes out a bit differently here :-)
That said, it's been a very long time since I've done any fiddling/debugging to make any HW work privately. Ironically, I've had a lot more issues making HW work correctly on the M1 Mac I also have, e.g. my (quirky) Bluetooth earbuds work a lot better on PipeWire than macOS ...
I think the common case is that vendors test with the current version of Windows, MacOS and/or Linux (in decreasing priority order), then hope that it the hardware is EOL’ed before the drivers bit rot.
One thing I could imagine is that Linux is giving preference to using the values in the DisplayID block as it's the newer standard, and since EDID/DisplayID compliance has improved over time the logic may be "the newer one is more likely to be correct". In the meantime perhaps Win/Mac continue to look at the classic EDID data, and if they do, it likely gets less test coverage from manufacturers.
Pure speculation.
Edit: From https://learn.microsoft.com/en-us/windows-hardware/design/co...
> Windows does not support DisplayID 1.x blocks, and always ignores them.
No explanation is given.
The LG display sends a DisplayID 1.2 block.
Another oddity is that when I run edid-decode on the DisplayID 1.3 file created by CRU, edid-decode reports the block as "Version: 1.2" instead. Wonder what's up with that.
I work in the automotive industry, where we often use fancy unusual screen hardware a couple of years before it turns up in home consumer electronics or phones. For example special multi-axis curved stuff, dynamic angular privacy filters, or haptic feedback using electrostatic modulation of resistance instead of vibration motors (that allows you to make the screen feel rough and scaly or glidy, give UI elements a feel-able shape, etc.).
One time, we were told to use earplugs at work for a few days, because of a pre-release firmware bug that could in theory, if other safety mechanisms also failed, cause the haptics to potentially emit an ear-piercing low-frequency tone ...
Temporary EDID bugs, otoh, I've seen so many times. :)
1. Windows/macOS might have a "quirks table" that hardcodes fixes for spec-violating devices.
2. Windows/macOS might ignore some reported EDID values and derive their own when they determine that the display behaves differently from what is reported (e.g. by recording response packet timings).
Even modern packetized display interfaces like DisplayPort are still fundamentally a one-way firehose for the pixel data. There's no provision for acknowledging or retransmitting data packets, and forward error correction is optional. The bidirectional sideband channels used for stuff like EDID are not timing-sensitive like the pixel data stream.
Honestly though it could also be workarounds. Windows is king at making stuff like this work by hard coding overrides. E.g. a custom driver for this display.
Wow, on-demand tinnitus is one hell of a failure mode.