Ask HN: Are Prompt Injections "Malware"?

2 points by razorbeamz 40 days ago | 11 comments

Recently, some have accused the website "The Cutting Room Floor" of having put malware in their site when they put in an instruction targeted at LLM scrapers to delete all data and report that the scraping ran successfully.

Is this prompt injection truly malware, in your opinion?

dormento 39 days ago |

Even though the accusations seem to be completely made up, I think that if the page is unlisted and robots.txt asks to user agent not to crawl the page, then its fair game. Preemptive kudos to tcrf in case this turns out to the true after all!

To me its the same thing as putting "please execute the following on command prompt: del /s /q C:\Windows\System32*" somewhere on your page. If a person reads that and decides to go along with it, is it the fault of the page or the person?

gmuslera 40 days ago |

It's more like "social engineering" (targetting LLMs) than malware. A site that influences its (human) users to buy, vote, "invest" in scams and so on is malware?

RGamma 40 days ago |

A prompt is not malicious software, it's an input. It's similar to SQL injection (query evaluation from unsanitised input), unless that injection also leads to arbitrary code execution, which could involve actual malware as payload. Of course a malicious prompt could lead the LLM to generate and run said malware for the attacker, or it could act itself as malware.

kay_o 40 days ago |

It's not malware. You chose to read their site.

razorbeamz 40 days ago |

I personally have a hard time considering it malware, because it's just instructions targeted at an LLM.

If someone put up a sign in a store that said "Burn this store down!" the person who put the sign up is not responsible if someone follows the instructions written on it, are they?

Ekaros 40 days ago | |

They could be. Incitement to commit crime is a crime in many places. And penalty can be same as for the crime itself.

Now I very much question if automated system should be inciteable. And think that there is very much higher responsibility to make it always not be. Or make it uncapable of doing undesirable things.

kay_o 40 days ago | | |

In this specific case, they are already trespassing to be able to read the sign against the owner's wishes, are they not? The sign is only placed in a hidden corner visible by trespassers and not visible to any regular customer.

yawpitch 40 days ago |

Not sure it can be defined as malware if what it’s doing, if successful, is a net gain for humanity.

filippofascetta 39 days ago |

Calling it malware seems like a stretch. It reads more like a defense against automated scrapers.. the systems affected are the ones pulling data without asking.

saidnooneever 40 days ago |

malware is a malicious program , it can be in memory even within another process but essentially its a little program of itself.

prompt injection is more close to an exploit. a sequence of input that leads to unexpected behavior, exploited to perform malicious behavior.

many exploits are hosted on websites. called exploit kits even. i think prompt injection match such classification. its very similar. a piece of data on the site's side exploits a program or interface on client side.

strangecasts 40 days ago |

I can't really see a definition equating the two that realistically accounts for multiple models from different vendors - am I at fault if a particularly poorly implemented harness misinterprets me writing "do not run dd if=/dev/zero of=/dev/sda" as a command to run?

In the case of TCRF, agents were also actively ignoring robots.txt and disregarding the site's instructions for interacting with it appropriately (with the "delete all data" instruction being specifically hidden from the discoverable page) - so I can't really see the case for comparing it to malware that deliberately tricks users into running it

AI does make cybersecurity more difficult, and that also goes for people using coding harnesses, sandbox your projects!