IDIV DoS (INT_MIN / -1)

111 points by alter8 13 years ago | 23 comments

makomk 13 years ago |

If anyone here has used Second Life's scripting language LSL, they had to work around this issue. Apparently they now calculate x / -1 as -x instead, which gets a counter-intuitive result when x=INT_MIN but doesn't crash. I believe once upon a time this was documented and everything.

loeg 13 years ago | |

But -1*INT_MIN is still 1 greater than INT_MAX in two's complement ... so that's signed overflow and thus undefined behavior in C.

Assuming x86 signed overflow behavior, you're just back to INT_MIN again? Or am I crazy?

bdonlan 13 years ago | | |

The language in question isn't C, so they can have whatever semantics they want.

0x0 13 years ago |

Reminds me about that fairly recent strtod() infinite loop that caused DoS in several languages.

http://www.exploringbinary.com/why-volatile-fixes-the-2-2250...

ctz 13 years ago | |

Did it affect any languages other than PHP? The linked article suggests that it was PHP only (and this agrees with zend_strtod being the problem function and not any C library implementation of strtod(3)).

0x0 13 years ago | | |

It affected a whole bunch of languages, libraries and compilers, apparently because the strtod() implementation in question has been a popular copy&paste piece.

This guy here over at http://blog.andreas.org/display?id=9 mentions at least: "Android libc, gcc libio, gcc java runtime, newlib libc, GNU Mono, Apple's libc, mozilla"

It was particularly nasty because it could easily be exploited simply by putting that specific decimal number into a web form or whatever, and for each request a thread on the backend server would lock a CPU to 100% usage until sysadmins discover and kill those threads, worst case.

jdlshore 13 years ago | | |

It affected Java too. I don't remember if it affected all versions of Java or not, but I reproduced (and fixed) it in my Let's Play TDD screencast series. Here's the episode: http://www.jamesshore.com/Blog/Lets-Play/Episode-89.html The "double of death" part starts at 3:33.

endianswap 13 years ago | | |

I believe Java did as well.

ScottBurson 13 years ago |

Fixed-precision integer arithmetic is evil.

malkia 13 years ago | |

And then a lot of game programmers would disagree :)

I for one just learned to love doubles...

huhsamovar 13 years ago |

I just tried the C version and it compiles and exits gracefully with gcc 4.2.1.

haberman 13 years ago | |

That's probably because the code in the article uses compile-time constants as operands. Try this one and pass "-1" as the command-line parameter:

  int main(int argc, char *argv[]) { return (1LL << 63) / atoi(argv[1]); }

Note that this issue has been written about before by Tavis Ormandy: http://my.opera.com/taviso/blog/show.dml/639454

huhsamovar 13 years ago | | |

Floating point exception: 8

ThisIsADogHello 13 years ago | |

Similarly, I tried the pgsql example on 8.4.13, although on FreeBSD rather than Windows like they suggest, and all I got was: ERROR: floating-point exception DETAIL: An invalid floating-point operation was signaled. This probably means an out-of-range result or an invalid operation, such as division by zero.

kevinnk 13 years ago | |

The example the article gives isn't a general C problem, it's a common bug in some (mostly older) C/C++ compilers. As you point out GCC does not have this bug.

haberman 13 years ago | | |

It's not a bug, it's intended to be undefined behavior; see http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html... (issue 1450)

afhof 13 years ago | |

Would you mind posting the output of gcc -S ?

Also, what optimization level did you run with?

huhsamovar 13 years ago | | |

No optimisation.

http://pastebin.com/NpUExgA2

derleth 13 years ago | |

Here's the LLVM code that clang-3.0-6ubuntu3 emits:

    define i64 @crash() nounwind uwtable readnone optsize {
      ret i64 undef
    }

In machine code terms, on x86-64 this turns into a function composed entirely of the ret opcode. gcc-4.6 adds an xor eax, eax before the ret.