Protect yourself from the hidden dangers of `curl – sh`

Protect yourself from the hidden dangers of `curl – sh`(djm.org.uk)

52 points by stevejalim 11 years ago | 33 comments

snuxoll 11 years ago |

I've seen this same argument time and time again and it's just silly. We preach that curl | sh is evil because of a potential lack of "transparency" but rarely does anyone denounce the evils of binary packages.

When you run third-party code on your system there is ALWAYS a risk of it doing nasty things, it doesn't matter if it's an easily readable bash script or a .deb you downloaded. The biggest argument I see about curl | sh that I can agree with is the issues that happens when your connection dies in the middle of the download. Just download the file, then run it.

freshhawk 11 years ago | |

Yeah, don't download and run binaries from random links on the internet either.

I don't think anyone is against recommendations of running "curl trustedsite.com/install | sh" except for the bad habits it teaches to people who don't know what curl and sh actually do, but wouldn't download and run a random exe.

Help forums are rife with suggestions to run "curl http://pastebin.com/raw.php?i=XXXX | sh" to solve technical problems. And not fringe forums either, but forums like the official Ubuntu forums.

I've literally never heard anyone "preach that curl | sh is evil" who wouldn't/isn't saying the exact same thing about binaries.

djm_ 11 years ago | | |

I wrote this article a year ago, and you hit the nail on the head - that is what I was getting at.

I didn't submit it here because it wasn't really meant for an advanced crowd, obviously most people would be aware of the dangers.

For example, take a look around at the pirating world. Many sites that help you install projects such as Couchpotato, Sickbeard, Sabnzbd etc rely on people curl-piping bash scripts. The people installing from those scripts likely do not know any better.

nmjohn 11 years ago | |

It's a little worse because with curl | sh - you inhernetly aren't able to check a md5 hash or a signature to verify the file is actually what you wanted.

Now while even with binaries people might now actually do that often enough - it at least is still an option.

kaoD 11 years ago | | |

You need a secure channel like HTTPS to get that hash, in which case, does it really matter if you just run the script anyways?

adestefan 11 years ago | | |

The signature can be a perfect match and still harm your system. Do you reverse engineer every binary that you run?

jingo 11 years ago | |

Best reply I've seen on HN in a long time.

Truth is I'd rather read someone else's shell script than someone else's C, python, ruby, javascript or other code. Not to say it still isn't painful reading; most scripts I see are nonsensically verbose. But it is a much less time-consuming read.

Unless, of course, it is written in a shell like Bash, i.e., one with too many extra features to keep track of. Like, say, exporting of functions, for example.

dfc 11 years ago | |

   We preach that curl | sh is  evil because of a potential lack of
   transparency but rarely does anyone denounce the evils of binary
   packages.

This is "Freedom 1"[1] and a bedrock principle of the FSF.[2] Applebaum recently gave a talk "Free Software for Freedom Surveilance and You" about the evils of binary packages.[3]

[1] https://www.gnu.org/philosophy/free-sw.html

[2] Free Software is Even More Important Now: https://www.gnu.org/philosophy/free-software-even-more-impor...

[3] https://media.libreplanet.org/u/zakkai/m/free-software-for-f...

0x0 11 years ago |

There are other risks besides malicious webservers. Even an accidental network glitch can be fatal, for example if the connection is dropped after the first "/" here:

rm -rf /tmp/myawesomeinstaller

jiggy2011 11 years ago | |

Wouldn't curl return an error and break the pipe? (I haven't tried)

znep 11 years ago | | |

curl won't buffer the entire stream, since that would be silly, so if it is a big enough response then curl will have already passed parts of it along to the shell through the pipe.

Is this likely to cause a catastrophic failure? No. Is it possible? Absolutely.

eximius 11 years ago | |

Okay, thats genuinely terrifying.

But still, curl > script,read, run and youre good to go.

carlsverre 11 years ago |

I don't understand why you would go through all of this effort...

Just dump the data into a file:

curl > foobar

Read the file using any number of normal utilities

vim foobar

cat foobar

nano foobar

less foobar

Then if you like what you see execute the file

sh foobar

Linux/Unix utilities are meant to be used. Don't limit yourself to only knowing how to check the contents of a curl install if you have a curlsh function.

djm_ 11 years ago | |

As programmers, our entire job revolves around removing unnecessary processes that can be automated. So I ask you, how does providing a shell function which does exactly what you just suggested limit somebody?

If they can't read it, they likely wouldn't even know how to install it.

Disclaimer: I wrote this article in Aug of last year.

andreasvc 11 years ago | | |

It takes up precious mind space for an ad-hoc one-off feature, instead of utilizing simple well-established unix commands commonly available on servers and usable for multiple purposes. Plus I'd say it gives a false sense of security because a moderately determined attacker can easily obfuscate his exploit so as to slip through this casual review process.

RubyPinch 11 years ago | | |

my personal bother with it, is it switches the roles of saving

the "No, I don't want to run this" way to exit is to wipe the file and save it empty.

the "I do want to run this" way is to just exit normally

opening something and going "wow this is actually damaging, I'll just exit, ^X... waitfuck" seems something that is more than possible

or in short, the default state is to execute the file

e12e 11 years ago |

I'd say apply the same level of scrutiny as you would other code, such as the code that your distribution allows you to install. That means:

    1) Find a source you trust (nominally)
    2) Get a gpg-key that you trust belong to that user
    3) Get the install.sh script
    4) Get the matching gpg signature (install.sh.asc)
    5) Verify that 4) is a valid signature of 3) under 2)
    6) Have a look at the script
    7) Run the script

If you can't establish 2), you'll just have to stick to 3) 6) and 7).

Seeing that something is on a https site, just means someone had the access to put it there. If someone got access to the private key behind 2) -- 1) is probably so compromised that there isn't anything other than 6) that might protect you -- and if the script is truly malicious (as opposed to just your average botched bash script) -- it's not guaranteed that it's obviously malicious.

Anyway, a gpg signature links some distributable the author has verified all the way back to wherever that file was authored -- while https only anchors trust on the web server. Web servers get compromised all the time. Prefer a proper signature as a means to anchor trust ("yes, this is probably what X wanted to distribute. If you trust X, this is probably OK").

A https signature just means: "This is something someone/anyone managed to upload to this web server".

meowface 11 years ago |

If you don't want to bother with all this you can also just do a simple `wget http://site.com/file.sh -O /tmp/script`, look through it in your editor, then run it.

peterwaller 11 years ago |

Another method to protect against malformed output due to the connection being cut short - "shell armour":

http://drj11.wordpress.com/2014/03/19/piping-into-shell-may-...

  { echo { && curl https://thing && echo } ; } | sh

jiggy2011 11 years ago |

Is this any different from downloading and running software without reading the source code?

Igglyboo 11 years ago |

Is this really a "hidden" danger? It's pretty obvious that you shouldn't execute a script without reading it unless it's from a trusted source over https.

silvestre 11 years ago |

Protect yourself by only using apt-get and packaging open source software yourself if isn't already packaged.

retr0h 11 years ago |

I don't see much reason for ever needing to do this. Should build packages to install software, and use config management for anything needed outside the package.

Dylan16807 11 years ago |

Don't forget that you can't even copy and paste a url safely, because there might be hidden text halfway through.

TheZenPsycho 11 years ago | |

curl http://www.perfectly-safe.com/script.sh?whoops&rm""-rf""/""

iancarroll 11 years ago |

If you can't install software to your box for whatever reason, just replace sh with cat or nano and read it.

_hnwo 11 years ago |

Protect yourself from the hidden dangers of `curl <url> | sh`

... by running this script i'm hosting on the internet.

djm_ 11 years ago | |

Where? You mean the proof of concept?

jamiesonbecker 11 years ago |

Any "risk" is canceled by simply using HTTPS.

1) Find a source you trust (nominally) 2) Get a gpg-key that you trust belong to that user 3) Get the install.sh script 4) Get the matching gpg signature (install.sh.asc) 5) Verify that 4) is a valid signature of 3) under 2) 6) Have a look at the script 7) Run the script