Dear Linux Kernel CNA, what have you done?(amanitasecurity.com) |
Dear Linux Kernel CNA, what have you done?(amanitasecurity.com) |
People who are burdened by requirements to ship (or produce rolling updates) to address every Linux kernel CVE are living in a state of sin. It doesn't make sense for the kernel CNA to alter its behavior to accommodate them.
The problem here seems to be fear over how EU legislators and judges will interpret CVEs more than anything the kernel is doing. Looks like a complex and legally risky situation; good luck to the EU devs, hope it stays as a theoretical concern. I imagine common sense will prevail and the law will be interpreted the sensible way.
I am not clear how the law affects people outside the EU whose software is distributed in the EU (including open source software that is not covered by the exemptions).
>I imagine common sense will prevail and the law will be interpreted the sensible way.
I hope it will, but I would hate to be in the position of having to depend on that.
Also if interpreted correctly it should help mitigate legal risks for EU companies that rely on Linux that update regularly.
The security team instituting those processes only have themselves to blame.
Have had to deal with too many rapid Jackson updates for "if you turn on the insecure mode that nobody turns on that lets the client specify the classes to instantiate and the documentation warns you about and requires a code change to enable, and include library X, then there's a new gadget that does RCE".
As far as I can tell, the signal to noise has only "gone down" in the sense of "from really low to really, really low".
> There’ll be a lot more time wasted in orgs triaging non-security-relevant bugs in the future
It seems to me that even before this change there was a lot of time wasted in orgs triaging non-security-relevant bugs, because CVEs didn't carry much useful information before.
CVEs should never be the input to anything except a triage pipeline, which in turn feeds other processes. If you don't have a competent pair of eyeballs (either internally or from a vendor) looking at CVEs with the context of how the impacted product is used in your organization, all you are doing is busy work.
Almost all end user organizations (not software vendors, OS distributors, etc) should pretend CVEs don't exist. Blindly apply all your OS and software patches within 24 hours of them being available and be done with it. You are much more likely to suffer a business loss as the result of a vulnerability than you are a patch application.
1. Most of the new CVEs have potential security concerns. At the very least, they would have been assigned a CVE if they were reported by an external researcher.
2. Many of them don't affect the older LTS branches.
3. I found 1 (1!) CVE that has a remote exploitation possibility. The rest are mostly local privilege exploit or DoS or crashes.
4. This Greg dude (heh) is backtracking to 2021 to flag bugs that have since been fixed. If your kernel repository is even semi up to date, you would have cut down the CVE count by 1/3.
In my company, the majority consensus amongst the kernel developers is 'patch your shit and do rolling release. We are too busy to evaluate all the CVEs.' On the opposite end, are embedded hardware kernel developers who barely had their kernels working on existing hardware. Both sides make good arguments and I don't have any opinion I'd share.
There are other ways to lessen the CVE workload.
1. Disable unused components with defconf or make menuconfig.
2. Don't stay on the bleeding edge branch (ie. 6.x)
3. Implement automatic minor version commit merges
Your mileage may vary, but with these implemented, the workload is managable imo.
> 1. Disable unused components with defconf or make menuconfig.
+1 for avoiding vulnerabilities, but were you saying this lessens the CVE evaluation workload? I'd love to hear about automation for evaluating CVEs based on a kernel config. I've done a fair amount of that manually and I'm not aware of any metadata in the CVE records (or in the CVE json in gregkh's new vulns repo) that includes config metadata.
Choosing a random selection of CVEs posted so far... they look reasonable. They're actual issues and they'll potentially affect someone.
This reminds me of the cookie banners situation. Many people complain about the cookie banners being visible rather than about the companies doing things that requires them to notify you. Now if you say you care about the published vulnerabilities, you get to actually see them all. And potentially change the policies around how you worked with them. (yes, it's not a great analogy, I'm not blaming linux for having each of those vulnerabilities)
So many vendors don't and it's tedious to say the least.
Maybe that's the aspiration, but it's clearly not the case in practice.
I reported a firefox bug 12 years ago where a malicious SVG could cause a hang - basically a 22-year-old XML bomb, adapted to SVG patterns. My bug turned out to be a duplicate of a 16 year old firefox bug.
No way of stealing user data. No sandbox escape. Not a crash that might indicate a buffer overrun. With a process per tab, it doesn't even crash the browser. It's just a file that takes a very long time to load - and it's not even an image type that user-generated-content sites like facebook and reddit allow you to upload. Reasonably enough, 12 years ago it was triaged as a minor performance issue.
Apparently in 2023, this counts as a CVE.
I should go see what happens if I load it now, since what changed was less that it stopped breaking them and more that I stopped having the conversation with many people...
That was always an option, as I recall. I think a non-default option, too. Not sure when they started adding the question about if you wanted to restore when you started up after a crash/unsafe shutdown.
Lots of corps would spend a non-trivial amount of effort to remediate something with such a score.
If you're already paying someone to maintain Linux for you, this shouldn't be causing that much trouble; it might need some contractual adjustments but you're already set up to get a stream of "good" updates. The patch frequency may be higher, but other people already do the majority of the work for you.
If you were just ingesting Linux "for free"… well, tough luck. You're profiting from the work of others already, you don't get to complain about not being spoon fed exactly what you need.
In practice, a small number of commercial entities (likely a mix of commercial distributions and designated security companies) will probably offer "Linux as a service". People could do the same work on their own, but that's not cost effective.
Either way, this shift in responsibilities has been long overdue.
grsecurity does this from a security angle specifically - in fact they're boasting about it on their homepage right now (fair enough!)
>Are Your Products Drowning in Linux Kernel CVE Noise?
>We know your products can't be updated every week based off unverified CVE information. Address true risk by protecting against entire classes of vulnerabilites and exploitation techniques. Our Pro Support ensures you make the most of attack surface reduction and our proactive defense in your products.
Then change that definition and stop operating off of it. It has never been correct.
Why should unpaid volunteers working on the Linux kernel do compliance work for FAANG-sized companies without getting paid for it? If these companies want the reports carefully triaged, they can send some employees to carefully triage them.
Having a large number of new, unscored, CVEs in the Linux kernel is going to make things... interesting. From their lists https://lore.kernel.org/linux-cve-announce/ these just have a CVE and not really enough detail for anyone to assign a score without a lot of additional analysis, which reduces their usefulness.
To an extent it could be suggested they're just exposing an existing flaw in the system (CVSS scores which may be taken to be scientifically applied, are actually just matters of opinion in many cases), but it will cause a lot of problems with automated tooling and compliance.
These Annexes explain the imposed Vulnerability Handling Processes imposed on manufacturers. The EU obviously only speaks about _exploitable_ vulnerabilities, because they know the problems of the CVE system all too well.
Best of all, open source projects are actively excluded by the CRA. [2] "Open source projects will not be required to directly implement the mandated processes described in the CRA. But every commercial product made available in the EU which is built on top of those open source projects will."
[1]: https://eur-lex.europa.eu/resource.html?uri=cellar:864f472b-...
[2]: https://eclipse-foundation.blog/2023/12/19/good-news-on-the-...
Edit: by that I mean filing bogus report or just non-security related CVEs. That is also reason why a lot of projects are trying to register themselves as CNA (see curl etc).
You can get FuSa (functional safety) certified Linux; to my knowledge this just does not exist for Windows. There may be other situations where the choice does exist, but considering Windows and Linux widely equivalent in this context is not possible.
> maybe even cause risk to human life
Neither Windows nor Linux are, to my knowledge, certified for SoL (safety-of-life) applications. And to no surprise considering this is close to (but not quite) a mathematical proof your system can't hang/crash/starve, which is pretty much impossible for anything beyond an RTOS with current tooling.
In their attempt to get revenge on security researchers for 'being more important', they have made it much easier for us to keep our exploits on the market for longer than before.
The idea that 'only the latest linux kernel is secure' is absolutely bananas, because it completely disregards the fact that legacy systems exist, they power very critical parts of our society, and they cannot be updated (as a general rule). Only components can be updated in-place.
So, again: thank you for the free money and for making exploits have a longer shelf life.
They have always said "Every bug is a security bug". I don't know about more global content, but at Kernel Recipes (2019?) gregkh took a Pixel that was running latest Google security patches that contained all CVEs. Then looked at non-CVE patches he merged in his LTS. And it took him less than an hour to find a DoS vulnerability.
I understand the author's frustration from the Linux Kernel community to not want to classify bugs, but the reality is that a huge portion of the bug fixes are actually security fixes [1], so between requiring 20% of the patches to be merged, and 100%, is there really a point?
The author mentions Cyber Resilience Act, and I believe that Linux Kernel team created this CNA /on purpose/ to have an impact on the CRA. They believe that the only way to have a secure Linux Kernel is to have an up-to-date Linux kernel. (cf https://social.kernel.org/notice/ARWvggnOvXny0CUCIa ). With the CRA enacted, doing such a every-bug-is-a-security-bug CNA is a way for them to enforce their view.
[1] FWIW, my personal opinion there is that this shows that Linux's monolith architecture is getting old, but I see nothing that could reasonably replace it. I think that "the dream" would be to have a LKL-like linux "arch" to compile every driver as an independent process of Hurd, with GKI-like stable-ish ABI.
The relevant part:
> Kroah-Hartman put up a slide showing possible "fixes" for CVE numbers. The first, "ignore them", is more-or-less what is happening today. The next, option, "burn them down", could be brought about by requesting a CVE number for every patch applied to the kernel.
They intend to burn down the cve system, and complaining about it is not a plan to stop it.
on one hand, it's true to the "all bugs are security bugs (and the converse is obviously true)" position.
on the other hand, it will demonstrably cause hassles for what i term the "checklist security" people. "oh noes we only have the budget for N CVEs per release and we must have NONE > $arbitrary-number". and so that's a very good thing because checklist-security like that is far worse than nothing at all.
on the the other-other hand, in more legitimate, well engineered downstream user, a CVE for every bug may conversely actually help real security. "all" you have to do is evaluate every single potential security flaw's real actual applicability and impact for your use case. and if you can't afford that - faking it with a check-the-box mentality is no substitute. but maybe you can afford that? if so better data will probably help, not hurt.
finally, the other concrete improvement is that the people closest to the product will be able to more accurately judge the CVE score and more quickly correct outliers. a local privilege escalation that requires enhanced privileges to execute is not an 8. i'd much rather have 1000 CVEs with appropriate scores than 10 that are massively overrated and stupid. of course, a simple one-dimensional metric is hogwash anyway, but - baby steps.
As and end user if you don’t have a script that causes an exploit for a given CVE i don’t care. I have had to patch too many systems from hypothetical issues ASAP!! because management wants to look like they know something
So perhaps this policy is a kind of spoiler response to efforts to require all security bugs to have a CVE allocated.
Cynically, I'd bet there are more than a few less technical but academic operators on the board who would run this play.
Linux Is a CVE Numbering Authority (CNA) - https://news.ycombinator.com/item?id=39406088 - Feb 2024 (10 comments)
The Linux kernel project becomes a CVE numbering authority - https://news.ycombinator.com/item?id=39361511 - Feb 2024 (24 comments)
CVEs assigned to every linux commit are more valid than each and every one of those bogus CVEs. Each and every one of them is associated with an actual change in a security-critical project.
If you want the flood of useless CVEs to stop, you have to clean your own house first.
Shouldn't this strategy lead to the opposite? By being overly cautious they should only assign CVEs for real demonstrable security issues.
Being cautious here means "it's better to assign a CVE when it's not a vulnerability, than to NOT assign a CVE when it's actually a vulnerability"
While the rate is low it does show that some bugs were indeed exploitable without that being known to the kernel devs. If an attacker is willing to invest more time than the kernel devs combing through commits to find vulnerabilities in the some older stable kernel then a big unlabeled pile saying "there's probably a vulnerability in there, go update" is correct.
There are easily hundreds of thousands of security vulnerabilities fixed every year that get no IDs because the current process is rooted in security from 1999 (the number is probably way way higher, but you get the idea)
Rather than obsessing over individual vulnerability IDs, we should be building systems that treat this data as one of many inputs to determining risk
I don't particularly think that CVE+CVSS has been the "right" way to do things ever (definitely not in the last 10 years) but my thoughts don't really matter whilst regulators and governments apply special significance to them, which they do.
Security bugs are special if a regulator can deem you in non-compliance if you have too many of them.
This is of course leaving the whole area of attackers who actively try to exploit them to one side :).
In summary, CVSS scores can be very objective, but in those cases they're also worthless.
If you can't reason about your codebase to a sufficient extent to actually determine that then something is very wrong.
If everything is a CVE, nothing is. That approach just wastes a lot of time and effort making people not so familiar with the codebase (as the maintainers) do triage.
I hope they get burnt quick by this approach.
Linux kernel developers are entirely capable of assessing this. They're just refusing to do it for someone else's definition of a "security bug".
"Every bug is a security bug" means "we fix things when they need fixing, categorizing the fixes is not our job and you'll need to do that yourself".
As such, the current new approach is in fact a concession, there's now a broad pre-categorizing of fixes you can work off.
> making people not so familiar with the codebase (as the maintainers) do triage
You seem to be under the impression that you hadn't needed to do that before. Which, to be fair, worked for a long time. From an engineering perspective this was always a case of "skipping inspections and verification", because the Linux community never agreed to do that work on top of providing the system.
> I hope they get burnt quick by this approach.
How would they get burnt by this? Social pressure from other kernel developers (or even outside) isn't going to have that effect. The only possible influence would be from employers paying for Linux work — in which case it's a perfectly reasonable discussion about spending paid time on security issues.
The environment where we write critical code the way we do now is very wrong. It's actually not that easy to figure out if something is exploitable or not. What if you add heap grooming? What if you enable another specific feature? What if an application fights for the same lock? What if measuring the time it takes to fail allows you to defeat aslr? People use exploit chains rather than independent ones these days and there are examples of clever cases of single-byte overflows turning into RCE.
Sure, there are going to be cases where you're really really sure something can't be used, because for example the bug only produces a null dereference and an oops. Then someone else comes along and proves you wrong https://googleprojectzero.blogspot.com/2023/01/exploiting-nu...
In my specific example /auth was reverse procured to a completely separate app, and /auth/login/bad wouldn't show the same content as / ... And even after explaining their test is invalid they still escalate rather than fixing or removing that test. Leaving me to explain 3 more times asking the way.
A vulnerability / bug in an upstream component may or may not end up being a vulnerability in a complex system incorporating that component. A vulnerability in upstream functionality that is disabled (or not used, with no possible path through which an attacker could trigger the upstream vulnerability) in a particular system is not a vulnerability in that system. Standards like ISO27001, SOC2, and PCI-DSS generally ask teams to have processes to discover vulnerabilities, but not necessarily to patch things that are not vulnerabilities in practice.
They've taken it upon themselves to assign a CVE to every bugfix, and it's being pointed out that that doesn't seem to be helping anyone.
Then the correct approach is not to cause "CVE fatigue" that can cause significant second-order effects. Not to mention the fact that who else is better suited to make that assessment? It's unavoidable that an assessment still has to be made because fundamentally there are use-cases where touching a working system has to have a really good reason. This will result in actually important things not getting patched because not-kernel-experts had to make that decision.
I also can't imagine large vendors being forced to follow a significantly more frequent update cadence also choosing to retain their current level of QA. Best case we're going to get more frequent less tested updates, worst case we're going to deploy an actual vulnerability due to some low-importance bugfix (with an assigned CVE).
I would require all CVE to ha attached exploit demo code. Otherwise it's shouldn't be CVE
And they're going to ask how much for the recertification for each CVE fixed? I doubt that'd be cheap.
> Neither Windows nor Linux are, to my knowledge, certified for SoL (safety-of-life) applications.
I didn't have exactly SoL applications in mind, there are plenty of other situations where the stability of a system could cause a risk. Be it just an emergency call center server or a field laptop for looking up license plates - can't leave them unpatched (especially with some of the new legislation) but also downtime from poor updates could be really bad.
FIPS has created an off-kilter perception about "recertification" because they require essentially the entire process when you change a single bit somewhere. Most certifications are not that harebrained.
Also if you need "certified" Linux, you are either already spending resources on it yourself, or paying someone else to do it. This might need adjusting for this new CVE practice, but it's going to be an adjustment and not a reset.
> […] can't leave them unpatched (especially with some of the new legislation) but also downtime from poor updates could be really bad.
Then pay someone to test and deliver.
That's the thing, resources aren't infinite. Linux offloading that work elsewhere will not have a net positive effect.
The path of least resistance will be taken, which is going to be proportionally less QA, if there was any to begin with.
I have to patch hundreds of CVEs in a month, and only a handful are actually valid. The vast majority is "CVSS 9.8: regex complexity explosion in $library" which my project only uses during build. But I've got to patch it, because it's definitely absolutely critical.
While the standard library bug that causes SSL connections to fall back to TLS1.1 instead of TLS1.3 by default is considered WONTFIX and gets REJECTED for a CVE.
Then instead of this, don't? It's utterly childish.
> How would they get burnt by this? Social pressure from other kernel developers (or even outside) isn't going to have that effect.
Fewer organisations willing to cooperate with them, for one? Social pressure comes in many forms and shapes, there's no way it won't have any effect.
> The only possible influence would be from employers paying for Linux work
They're going to be paying someone else to provide a clean feed instead of the organization that deliberately hinders these efforts.
And that's perfectly fine, it's open source software. Either way someone gets paid to look at the patches, which is my point.
If you want to do it in a cost-effective manner, you'll find other people with the same requirements, since the work result is "shareable".
> […] instead of the organization that deliberately hinders these efforts.
There is no such organization, and it feels like you have very little understanding of the organizational (and funding) structures behind the Linux kernel. I really can't extend my comments into a full-blown explanation of this, sorry.
(No, the Linux Foundation does not perform the role you're implying: they don't currently and likely never will sell a "clean feed".)
> Fewer organisations willing to cooperate with them[…]
I have no data on this but it is entirely reasonable (and I believe it likely) that the current behavior was requested (or encouraged) of involved organisations and people by cooperating organisations and people.
> There is no such organization
There is such an organization, the Linux Foundation is the CNA being the hindrance to these efforts. And yes, they won't perform the role, someone else will and they will be paid for it.
For some that's fine, I find it a significant amount of wasted effort, confusion and potential issues.
You were implying the Linux Foundation is attempting to get paid for providing said "clean feed".
Anyway, this has devolved far enough.
[Ed.: the Linux Foundation isn't even the CNA, shame on me for accepting that without verifying. The actual CNA is kernel.org. https://www.cve.org/Media/News/item/news/2024/02/13/kernel-o... ]
That's kind of how things work, and not much you or I can do about it.
So we need to think about "What is practical?" and "what is useful?", keeping these realities in mind, rather than insisting on "this is what it was meant to be".
Personally I think a good start would be to rethink the entire messaging and list them as "high impact bugs you probably want to get fixed ASAP", or something along those lines, rather than "security bugs". This should avoid the whole security wankery with memory issues that perhaps maybe possibly could perhaps someday lead a possible exploit maybe.
As 'arp242 says, we need to consider what is useful. Pretending that all CVEs are severe and must be addressed immediately is not useful. Spamming the CVE database with every bug in your tracker is not useful.
Replacing CVEs (and CPEs, which are equally terrible) with something new would be extremely helpful. My question is, who funds that work? NIST currently appear to have NVD resourcing issues, based on the banner on their website.
Undercutting my own point though, it doesn’t hurt to rerun a calculation if you think the public vectors is “lacking” or if temporal/environmental metrics matter in your context
Unfortunately, it does not appear that that proposal gained traction, since it's not discussed at all in the more recent LWN article.
I do have a problem with the kernel devs attempting to burn down the only system we presently have, however terrible it is, and with HN justifying the bonfire on the basis that CVEs were always broken anyway.
It's easy to see why people want CVE to work that way! It implies that people numbering potential security issues are doing a fuckload of work for you. But that work isn't free, and CVE has other purposes in the research community. So, no, I don't think anybody is going to talk the kernel people down from this. They're right.
If you want a feed of "CVEs" that clear a plausibility bar, put that together yourself. A lot of people would love to consume it and sell it to their customers; you'll get a lot of uptake.
This is a CNA responsibility.
I feel like I'm on reasonably safe ground when I say that my take on CVSS is a mainstream one in the field.
For example, what does Network vs Local vs Physical mean if it's an exploit in a cloud microservice?
Ooh let me consult the tea leaves. What's that? They consider it "Network" even though it's S3 mounted locally as a filesystem? Now that sev:med looks like a sev:crit.
The known alternative to CVSS is to rate severity levels entirely on vibes, and I find vibes to be more accurate.
> Network vs Local vs Physical
Network: It has to traverse the network stack. Adjacent: On the same physical network link, (usually this means the ability to send packets that are lower level than TCP/IP). Local: ability to execute code on the local machine as the starting point. Physical: You need to be able to touch the machine.
I'll be the first to admit that it can be difficult for some new players to correctly score their system. The "AV" refers to the attackers perspective, not how the software is used, this is a common mistake that quite a lot of vendors make.
I've been on both sides of bug bounty programs over the years.
I've been in corporate meetings where CVSS was summoned to downgrade the severity of high-sev security bugs, when the standard procedure wasn't to use CVSS at all.
I've published my fair share of security bugs.
Hell, I've even talked extensively with Steve Coley about how CVE and CWE intersect with my own experience doing security research.
And that's just some of the stuff I've done under this handle.
My experience with CVSS has consistently shown it to be misused.
Maybe you have enough discipline to use CVSS as it was intended by its designers. The rest of the world does not, by and large.
The main problem with the CVSS is that it's a one-dimensional numeric scale that's meant to measure the kind of complexity that warrants a formal threat model, not a 0-10 rating.
It’s so complicated you have to have a degree in CVSS to properly rate a vuln and it’s also highly subjective - which they want it to be.