Aggressive Attack on PyPI Attempting to Deliver Rust Executable(blog.phylum.io) |
Aggressive Attack on PyPI Attempting to Deliver Rust Executable(blog.phylum.io) |
It's not an attack "on" PyPI, or even an attack at all: someone is just spamming the index with packages. There's no evidence that these packages are being downloaded by anyone at all, or that the person in question has made any serious effort to conceal their attentions (it's all stuffed in the setup script without any obfuscation, as the post says). The executable in question isn't even served through PyPI (for reasons that are unclear to me): it's downloaded by the dropper script. Ironically, serving the binary directly would probably raise fewer red flags.
Supply chain security is important; we should reserve phrases like "aggressive attack" for things that aren't script kiddie spam.
Lol, maybe, "chatgpt, give me a thousand feasible pypi package names"?
And in this way, malicious packages may be unintentionally downloaded by users even when those malicious packages did not yet exist when the LLM was trained. Just because the hallucinated package name was randomly later taken by someone malicious.
I wrote a comment on the NPM thread earlier (https://news.ycombinator.com/threads?id=freeqaz) that I'll quote here:
> "While being flooded with spam is never good, it gets immediately noticed and mitigated. It's harder for open source projects to spot and stop rare one-offs"
This is the real problem that NPM and other ecosystems face. A determined attacker that is trying to "poison" a popular Open Source package just has to feign as a maintainer long enough to succeed[0]. Defeating these types of attacks will require rethinking how we think about trust of packages.
Projects like Deno are one approach (fork the ecosystem) while projects like Packj (mentioned elsewhere here), Socket.dev, and LunaTrace[1] are taking the other angle (make it harder to install malware).
It's hard to say which approach is better right away. (Probably a hybrid of both, realistically) It's just non-trivial to fix this in one clean swoop. It's messy.
0: https://www.trendmicro.com/vinfo/us/security/news/cybercrime...
There's something beautiful in knowing you're using pure, clean Python. Much easier to install, also.
Attacking a popular repository like this does not have to have a high hit rate.
"Script kiddie spam" is now computers get compromised. Unsophisticated mass attack.
This sport of thing, combined with woeful security and fragile systems are causing havoc the world over.
It doesn't make plain Python code you blindly execute any safer, but at least you've explicitly given those packages your trust. I believe this is more geared toward detecting compromises of those packages you have given that trust.
Python used to have a "batteries included" philosophy which tried to put most important stuff into the distro, reducing the number of external dependencies any given app needed. They seem to have abandoned that now, leaving us to fend for ourselves against the malware.
NPM spam: https://www.scmagazine.com/analysis/devops/npm-repository-15...
Yes, along with reducing the stdlib and directing us to PyPI for "alternatives".
Maven sorted this out 20 years ago
what's a bit sad is the python packaging's authority survey from a few months ago seemed to be mostly interested in vision and mission statements
rather that building a functional set of tools
(This is not a reason not to add namespacing; just an observation that it's mostly irrelevant to contexts like this.)
example: the package named "aws" on pypi was created by some random guy and has been abandoned for years
if pypi/pip supported namespacing that would be info.randomdude.aws instead
and amazon's packages would be under com.amazon
not being able to namespace internal packages is another security issue that is substantially improved with proper namespacing
to be blunt: not supporting it at this point is reckless and irresponsible
(I note you're part of pypa!)
Maybe the list can be hosted on an internal server for other employees to reuse. Hosting all the packages internally is overkill. Trusting the world by default is overkill.
Now "pip install gooogle/package"
"Hey User, gooogle/package is not from a trusted namespace. Did you mean google/package which is similar and trusted? Or would you like to add gooogle to your local trust file?"
The lack of any kind of curated feeds that only lists verified or popular packges is tragedy. There should be a reasonable way of allowing clients to protect themselves from a typo.
https://journals.plos.org/plosone/article?id=10.1371/journal...
It's like NYC's side walks. Compare pedestrian behavior at say SoHo (daylight) and say LES (nighttime). Amazingly enough, the partying and inebrieted pedestrians at night all file politely in the correct bimodal L|R formation. During the day, it's a rather wild and somewhat uncivilized dynamic slalom formation. My theory: Fangs. The night creatures know someone potentially dangerous maybe in the midst.
In these cases I frankly assume that they don't either.
The closest thing is pattern/AST matching on the package's source, but trivial obfuscation defeats that. There's also no requirement that a package on PyPI is even uploaded with source (binary wheel-only packages are perfectly acceptable).
This is a little bit too strong, since packaging doesn't require arbitrary code execution. For example, Go doesn't permit arbitrary code execution during `go get`. Now - there have been bugs which permit code execution (like https://github.com/golang/go/issues/22125) but they are treated as security vulnerabilities and bugs.
Of course, you're right about Python.
Java's type system: ClassLoaders plus SecurityManager was impossible?
that's literally how Java applets worked, enforced through the type system
https://docstore.mik.ua/orelly/java-ent/security/ch03_01.htm
yes, SecurityManager was a poor implementation for many reasons, but it's definitely not "impossible" to sandbox downloaded code from the network while having it interact with other existing code, you can do it with typing alone
I worked a few years back on something like this but it went nowhere, but I still believe it would be doable and useful. The only trace I found back is https://wiki.python.org/moin/Testing%20Infrastructure, which contains almost no info...
Good namespacing (e.g. in Go), in practice, provides critical context about the development/publication of a software package.
But deliver anything more streamlined and secure? Hell, no!
They do look surprisingly convincing.
[0]: https://github.com/pyston/pyston/blob/1d65d4831912179c26bb27...
Python, Ruby, et al. are in an even worse position than that baseline, since they have both arbitrary code in the package itself and arbitrary code in the package's definition. But the problem is a universal one!
So yes, it's relevant.
> In general, the layout used by the Rust compiler depends on other factors in memory, so even having two different structs with the exact same size fields does not guarantee that the two will use the same memory layout in the final executable. This could cause difficulty for automated tools that make assumptions about layout and sizes in memory based on the constraints imposed by C. To work around these differences and allow interoperability with C via a foreign function interface, Rust does allow a compiler macro, #[repr(C)] to be placed before a struct to tell the compiler to use the typical C layout. While this is useful, it means that any given program might mix and match representations for memory layout, causing further analysis difficulty. Rust also supports a few other types of layouts including a packed representation that ignores alignment.
> We can see some effects of the above discussion in simple binary-code analysis tools, including the Ghidra software reverse engineering tool suite... Loading the resulting executable into Ghidra 10.2 results in Ghidra incorrectly identifying it as gcc-produced code (instead of rustc, which is based on LLVM). Running Ghidra’s standard analysis and decompilation routine takes an uncharacteristically long time for such a small program, and reports errors in p-code analysis, indicating some error in representing the program in Ghidra’s intermediate representation. The built-in C decompiler then incorrectly attempts to decompile the p-code to a function with about a dozen local variables and proceeds to execute a wide range of pointer arithmetic and bit-level operations, all for this function which returns a reference to a string. Strings themselves are often easy to locate in a C-compiled program; Ghidra includes a string search feature, and even POSIX utilities, such as strings, can dump a list of strings from executables. However, in this case, both Ghidra and strings dump both of the "Hello, World" strings in this program as one long run-on string that runs into error message text.
https://insights.sei.cmu.edu/blog/rust-vulnerability-analysi...
> I know there is a researcher out there who has retrieved and installed every single pip package to do an analysis, which is a good start.
You're probably talking about Moyix, who did indeed downloaded every package on PyPI[2], and unintentionally executed a bunch of arbitrary code on his local machine in the process.
[1]: https://cloud.google.com/blog/products/gcp/exploring-contain...
[2]: https://moyix.blogspot.com/2022/09/someones-been-messing-wit...
> Why isn't Google or Microsoft at least trying this?
They are: Google and Microsoft both spend (tens of) millions of dollars on hypervisor and VM isolation research each year. It's a huge field.
Simplify, don't use a VM.
Create an isolated network, hook your sacrificial machine up to it, have it install the package. Remotely kill it (network controlled power switch if needed). The machine's hard drive should be hooked up through a network controlled switch of some type. After the sacrificial machine is powered down, reroute the HD so it is connected to a machine that does forensics.
Now you have a clear "before" and "after" situation setup for analysis.
The sacrificial machine's network activity can be monitored by way of whatever switch/router it uses to connect to the Internet.
This article cites CVEs of a certain type, which were especially popular in the 2021 timeframe. These CVEs do not correspond to real vulnerabilities in real executables. Rather, they are reporting instances of rust programs violating the strictest possible interpretation of the rules of the rust language. For comparison, quite literally every single C program ever written would have to receive a CVE if C were judged by the same rules, because it isn't possible to write a C program which conforms to the standard as strictly as these Rust CVEs were requiring. CVEs of this nature are a bit of a meme in the rust community now, and no one takes them seriously as vulnerabilities. They are reporting ordinary, non-vulnerability bugs and should have been reported to issue trackers.
The whole discussion about layout order is completely irrelevant. When RE'ing unknown code you don't know the corresponding source anyways, so the one-to-many correspondence of source to layout is irrelevant. You are given the layout. You can always write a repr(C) which corresponds to it if you're trying to produce reversed source. This is no different than not knowing the original names of variables and having to supply your own.
The next objection is literally that rust does not use null-terminates strings, except the authors are so far out of their depth that they don't even identify this obvious root cause of their tools failing. Again, this has absolutely nothing to do with the reversibility of rust programs, except perhaps preventing some apparent script kiddies from figuring it out.
The authors do manage to struggle to shore, as it were, by the end of the article, and somehow they end up correctly identifying their tools and understanding and the root cause of their troubles, not Rust. I take it you didn't make it that far when you read it?
It also just kicks the can down the road: Amazon is the the easy case with `com.amazon`, but it isn't clear a priori whether you should trust `net.coolguy.importantpackage` or `net.cooldude.importantpackage`. These kinds of trust relationships require external communication of a kind that package indices are not equipped to supply, and should not attempt to solve haphazardly.
> (I note you're part of pypa!)
I am a member of PyPA, but I don't represent anyone's opinions but my own. It's a very loose collection of projects, and it would be incorrect to read a general opinion from mine.
For example in PHP/composer/packagist and node/npm they just have a vendor name that can be reserved.
It makes it very easy to distinguish “this package is from the (trusted vendor name here)” and prevents issues with namesquatting.
this is a classic example of not letting perfect be the enemy of good
there is no perfect solution, there never will be
piggybacking off of DNS works extremely well for Java and Go (and the tooling is a pleasure to work with)
meanwhile Python continues to be a complete disaster
Pypi could do this. Or, they could require that someone demonstrate proof of ownership for a namespace by signing it with a certificate tied to the domain name (so you couldn't claim the com.bigco namespace without having the certs, which you can't get without owning that domain). There could even be signature requirements/proof for each package and/or version uploaded.
> It’s also eminently not sustainable on PyPI’s scale, which is the context we’re talking about.
I started my software engineering career in testing before VMs were a thing, so large, very large, scale test setups like the one I outlined were common place. I wrote about some of my experiences at https://meanderingthoughts.hashnode.dev/how-microsoft-tested... and the physical hardware setup my team was using to run (millions of!) tests was tiny compared to what other teams in Microsoft did at the time.
Network controlled power and peripherals were exactly how automation was done back in the day. Instead of VM images, you got a bunch of identical(ish) hardware and you wrote fresh images to hard drives to reset your state.
Are VMs more convenient? Sure, but my reply was in context of ensuring malware can't detect it is running in a VM!
Depending on how something like this is implemented, maybe com.github could set it up to pull straight from the project repo.
Just because there's ways it could go poorly, doesn't mean it will go poorly.
Security needs to be evidence and outcome-driven, first and foremost. That takes a while, but improved outcomes make it worth it.
[1]: https://pyfound.blogspot.com/2019/06/pypi-now-supports-two-f...
[2]: https://pythoninsider.blogspot.com/2019/07/pypi-now-supports...
meanwhile the integrity of the supply chain continues to be compromised
> Your cynicism isn't warranted
it is: the python packaging situation is worse today than it was when I started writing Python in 2005
the legions of meetings, grandiose titles, conferences and mountains of unreadable proposals have produced tooling that is objectively worse than what Maven offered close to two decades ago
I have no opinions about titles, etc. But saying that Python packaging was better in 2005 is incorrect along all axes.