Just the rumour of a bug is enough to find an exploit these days(anil.recoil.org) |
Just the rumour of a bug is enough to find an exploit these days(anil.recoil.org) |
In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review.
The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually.
I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment.
GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.
Not sure what the solution is, but it is definitely a problem for us.
I can't express in polite words how pathetic it is to see the only official blob storage bulk transfer CLI tool from a multi-trillion-dollar company fail to do the simplest, most essential functionality after ten major revisions.
Meanwhile, rclone Just Works(tm).
Thank you from me too!
Possibly some sort of ai agent code review system that churns through code looking for these vulns before the code is published. It feels like it's all about who has the resources to find bugs at the moment but that it should be a standard to catch issues before prod moving forward..
That said there are a number of people and companies working on more focused means of driving the LLM to look were bugs would be the most dangerous and in doing so reduce the token spend of each bug found.
In some ways the better you are at security stuff the more you can reduce your spend by better driving the LLM to problem spots.
- grab a group of (related) bugs/defects/vulns
- fix them on a branch like bug-batch-XXX
- run that group through the verification, landing in main, CI/CD flow to amortize process cost
- repeat as needed to process backlog
My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.
I felt that. The problem with doing that is you hate yourself afterwards so not a good solution either, gotta do it properly.
Thanks for working on rclone, Nick!
You can use those numbers in changelogs as soon as they're published
There is some human review, which takes time, before the GHSA is "GitHub-reviewed" which then allows security scanners to pick it up
Then, the CVE ID can be assigned to it after-the-fact if need be, but the GHSA should be a sufficient starting point
A strange bottleneck; anyone know why that would be so slow?
As mentioned by another commenter, they intentionally have a human reviewer in the process before a GitHub Security Advisory (GHSA) becomes "official" post-publish
(this is separate to getting CVE ID assigned, if wanted)
Can you discuss this? I might be able to help.
No matter how good AI gets at fixing bugs we'll never fix them when there's no will to fix things. Software will never be good if there's no will to make good software. The problem has always been about will. To many better products. It's insane that in a time where we can do better on speed and quality we still choose speed and tell ourselves it's velocity
Yes, no. Do not even think about doing that. You cannot, should not, and must not "microupdate" code running on users machines without their consent.
Also, the whole idea is completely insane. "We might have unpatched 0days in the field, so to mitigate that, we shall add arbitrary and remote code execution via the cloud"
___
And even if you were to think that taking a page out of googles book - you know, the company known for being laser focused on having the wrongest opinions - is a good idea, you shouldn't do so for free.
Live patching is something enterprises pay a lot of money for. Making it a Foss expectation is insane.
I do agree with the author's ideas, though; most of these are things that should have been done much sooner, and I suppose it's good in a sense that there is a forcing factor now.
True, but it used to take days or weeks of research, testing, and RE to get those PoCs.
Today the entire chain - reading commits, RE patch binaries, building exploit, scripting exploit scan, $profit - can be fully automated and happen in minutes or hours.
Is there a modern equivalent for AI enabled script kiddies?
slop kiddies?
Add to that the danger of supply-chain attacks where you don't even want automatic updates.
It’s a trap regardless:
A) run a known vuln B) accept and run any and all updates immediately… which could be compromised
Maybe A is worse because it’s a known vuln?
(By this I mean integrated in something like dependabot, not just some security companies doing it and publishing reports.)
I have heard of at least one project (c-lightning?) temporarily releasing a closed-source binary as a workaround until users could update safely.
now you can instruct an LLM to do this with MCP...
Information about the problem then gets released later, once everyone has had appropriate time to get up-to-date. Or not, and we are non the wiser.
I am not a fan of that, but I think many folks will take that away from this.
“I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.
So Fable knew about it. Maybe someone is running experiments again like in OpenAI's Huggingface hack.
Cute to see that the Glasswing apparatchiks still protect their income stream and hand out no accesse.
Another way is maybe something like saying there's a bug at some endpoint and thus manipulating a bunch of bots to DDOS that endpoint without having to pay for it?
LLMs are extremely good at finding corner-case vulnerabilities in C bindings even within a memory safe language; see for example the fixes in an OCaml crypto library here: https://discuss.ocaml.org/t/the-series-of-mirage-crypto-rele...
There was an article some days ago where someone was claiming "with AI only you decide how many bugs you have" — well no the same forces apply because tokens are not free and business wants to do stuff that directly earns money.
Well we have to make cases and measure where the bug costs money and how. Lots of bugs are irrelevant and not blocking people from using the software. If bug doesn't drop database but is "dropdown doesn't exactly align" or "given precodnitions A,B and C something bad will happen" while A, B and C have very small possibility of occuring.
> It is not "will" it always is money.
So it isn't money. You can argue that it is power, that the money is the proxy, but this still wouldn't answer the question.
The reason I'm pushing back hard here is these simplistic answers are just thought terminating cliches. They dismiss the problem, calling it inevitable and unsolvable. It only helps to preserve the status quo. It only helps empower those who seek to take our own. So I call bullshit
Now we've got an ability to push code out faster than ever, and absolutely zero innovation for QA. You simply cannot trust AI to verify your code is working. You can't have Quality Assurance without some form of assurance. So you're either hiring twice as many QA guys for the 10x output, or you're mostly ignoring the idea.
Economics has bubbles. Does computer science? Anyways, screw this. I'm moving to nursing.
With that said there are numerous companies that are very concerned about the situation. They know AI is finding bugs in their software at an accelerated rate, one they are having difficult times keeping up with because they want human understanding and review of the fixes to avoid introducing new bugs.
I'm hoping one of the unintended side effect of it being essentially free to find and exploit (and fix) software bugs is that companies become less cavalier about shipping bugs in their software. Unlike most of the industry I don't believe "bugs are inevitable." Bugs are a choice developers make when they're rushing and careless and when all of their incentives are to ship quickly. You can ship bug-free software but it takes (or used to take) a really long time and a lot of care, care that commercial software developers just don't ever seem to muster.
Maybe when their software is getting 0wned over and over and 30 security issues are published a day, they'll start caring and taking their time.
Very low signal information, preemptively trying to cover every rebuttal despite nobody ever planning on making one, in the few times someone does he plays devils advocate endlessly
Like bro just let us babysit these agents, everything’s going to happen
Today, while bitterly staring at Claude per my current employer's "use genAI or else" mandate, I got "bloviating."
The worst one was where I fixed a datetime bug and although it had been sending out false alerts, I was asked to dry run the 5 lines of code I changed, like a coding interview. In all this pressure I forgot what the code was meant to do, and was dismissed and asked to set up another meeting with an explanation of all the various cases that could happen...
FFS, we're living in a world where Linux has more than doubled in popularity mainly due to Microsoft actively fleecing their customers. It's crazy
/I use arch btw (and have been on it for over a decade)
> tech managers look at tech debt as a thing to be maintained at a certain level
> allowing us to achieve perfection
Look, I'll defend high quality code all day long. Good code allows you to move fast just like taking 30s to tie your shoe laces allows you to run faster.
But we can't write high quality if we pretend that perfection exists. Globally optional solutions are the exception, not the norm. There's almost always tradeoffs we need to make. The high quality code, the high quality engineering, is understating, tracking, triaging, and minimizing those tradeoffs. It is writing code that allows you to change those decisions as quick and effectively as possible. But perfection doesn't exist. It's a good thing to chase, like Utopia, but ultimately unobtainable. A good engineer knows how to triage.
Advocating for perfect code will be a losing battle. But we should not, even for a second, let that be interpreted as meaning quality doesn't matter. I'm still frustrated with people who say "don't let perfection be the enemy of good enough" as most of the people that say that believe "good enough" is "it looks like it is working" and call a "demo" a MVP
Huh?
I have pride in my work, and having to watch it be destroyed repeatedly is not a way to keep me around.
now none of those methods matter as you can just instruct an LLM to bang its head against the wall until the wall breaks.
I'm not sure if the cat and mouse game resolves clearly one way or another. Since the obfuscation can be hardened against LLMs during development.
To me, this is the craziest part about all of it. Why doesn't anyone seem to care?
This is the same as when junkies specifically seek batches of drugs on which others overdosed.
Alternatively, what about just doing the right thing? Either you convince them to take this stuff seriously or you find alternate employment. How can you subject yourself to the moral degradation and conflict of principles? I could understand for someone with no other financial options, or in some sort of oppressive culture/economy. But most in our field probably don’t fall into those
> But, isn’t it our job to impress upon the managers the importance, in a certain regard?
Our job is to engineer. Our job is to build (good) products. Information can't just flow top down, it has to go the other way too.
What seems weird to me is that during "the good times" in our field, that happened more frequently. A strong employee market (as opposed to an /employeer/ market) seemed to be good for employee, employeer, and the people buying everything. But myopia is quick to set in.
> Alternatively, what about just doing the right thing?
If you haven't heard it before, allow me to introduce you to Pournelle's Iron Law of Bureaucracy[0]. I think one of the important things it states is that the second group is actually bad for business. I think there's a common misconception. People are often saying "well it is good for business", pointing to all kinds of messed up shit. I don't buy that. It is myopically good for business, but not in any meaningful length of time. Though then again, as Buffet says "The market can stay irrational longer than you can stay solvent." Michael Burry famous learned this first hand.
[0] https://www.jerrypournelle.com/reports/jerryp/iron.html
In any bureaucratic organization there will be two kinds of people:
- First, there will be those who are devoted to the goals of the organization. Examples are dedicated classroom teachers in an educational bureaucracy, many of the engineers and launch technicians and scientists at NASA, even some agricultural scientists and advisors in the former Soviet Union collective farming administration.
- Secondly, there will be those dedicated to the organization itself. Examples are many of the administrators in the education system, many professors of education, many teachers union officials, much of the NASA headquarters staff, etc.
The Iron Law states that in every case the second group will gain and keep control of the organization. It will write the rules, and control promotions within the organization.Penny wise, pound foolish
The best is the explanations are in the PRs. They even have Claude these days to answer their questions... I swear, these people are more addicted to spending money than they are about making money.
* for people who don’t know the ref
Why is the word “you” in there, as-if that is inherently manual labour for me to carry out?
I know! Move is two operations under the hood.
Rclone automates this trivial two step action.
Azcopy does not.
For all the other companies fixing a bug has opportunity cost, they pick what to spend their resources on. Some 5 minute bug feels insignificant but if all devs start picking 5 minute bugs they fancy in a system company won't do what is needed to keep running I guess.
> Your answer makes sense for me only if your boss is Elon.
I know I picked very big numbers, but let's be honest, a more "modest" $50m and you have everything you could ask for. Put that in a safe 5% earning account and you're making $2.5m/yr, sitting on your ass doing nothing. Good luck spending all that in a year, consistently. There's nearly nothing out of reach.
There's far more people whose bosses are richer than that. But I think the problem is people don't understand what these levels of wealth actually mean. Money works differently at that level as it becomes harder to spend it faster than you earn it. Even those get in big tech jobs aren't living at those levels.
Clearly these people are addicted to something other than money. No billionaire actually cares about money. The money is a proxy
Being a billionaire is a conscious decision. You can always choose to give away your shares to employees or sell them and donate the profit like Warren Buffett. If you decide you want to keep all of it for yourself that means you are a selfish person and probably a psychopath. And for these types making money is a goal by itself.
Why not seek fame instead? Philanthropy? World records? Knowledge? Innovation? Adventure? Or many other things? Money makes all these things easier to achieve. These are less hated by the public. Using capital to seek these pursuits does not risk them losing their heads as the populous revolts. Power can still be had through many of these pursuits. To do the things they claim their companies do...
I don't think you're wrong, but it is far from a complete answer and that's why I ask
> Michael Burry managed just fine
Also, a line from a scene in the Matrix.
Like, yeah, RAID, several drives, one filesystem, right? Except no, because it's just one virtual device.
We've all seen developers who game the system. They hide the bugs just enough so checked out management doesn't see them. They convince themselves that those bugs don't matter, even when they trip over them later. They get rewarded because they appear to move fast, eventually become management, and the whole thing gets worse as time goes on.
It's a structural problem. Yes, the level of influence is higher the higher up in the org chart you go, but there is still "power" at every level. Even the most junior developer has power. The worst thing we can do is become apathetic, shrugging it off, saying "well what can we do?" That attitude is one of many factors that got us to this point. Importantly, it is a factor we actually can influence.
That's why I object to it. Not because I think it is going to solve the problem overnight, but because it is a thing we have some power over. And it is a very different situation when one engineer in a team is vocalizing "our software has issues" while most of the team silently agrees vs several members of the team simply vocalizing agreement. There's no magic single variable fix to problems like these, but we got here because a bunch of little problems added up. Unfortunately, or fortunately, the way to solve it is through solving a bunch of little problems. Each seems insignificant in isolation, but they accumulate
We've got a winner. Devs respond to the incentives set by management, and the message I have received loud and clear, my entire career, despite protestation, is that I should ship faster and not worry about bugs so much, big or small.
That I take pride in what I do is the only reason motivating me to not ship bugs. That and the thought of being the cause of an incident makes me physically ill, even though we have "blameless culture".
> Devs respond to the incentives set by management
> That I take pride in what I do
I just hope that more can. That they don't shirk off the part they play, just because it is small. The world is made by all of us, not just those "above" us.
Incredible strawman of dev behaviour, I can only assume you are a manager yourself. I assure you, that's not what we're doing.
Honestly, from the tone, I suspect the gp post is just trying to get a rise.
So I'm not just going to accept selection bias as the answer because while that explains why some people have an addition it doesn't explain why society is fixated on those chasing the arbitrary high score.