An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl

An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl(page.mi.fu-berlin.de)

38 points by CaptainMorgan 16 years ago | 16 comments

koblas 16 years ago |

While this article is 10 years old, it contains a few very good points -- I've referenced it many times over the last 8 years. Specifically:

-- Scripting languages relative run time and memory consumption overhead will often be acceptable and they may offer significant advantages with respect to programmer productivity.

-- For all program aspects investigated, the performance variability due to different programmers is on average about as large or even larger than the variability due to different languages.

houseabsolute 16 years ago |

I've always kind of felt that any claim to empiricity in programming language comparison will be fatally undermined by a focus on only one task. The choice of task itself is very opinionated in a way that makes it difficult to apply the conclusions of this study more broadly.

m0th87 16 years ago | |

FTA: "It must be emphasized that the results are valid for the phone code problem only; generalizing to different application domains would be haphazard"

lg 16 years ago |

should maybe note that this was written 10 years ago.

cgranade 16 years ago | |

That's bound to throw off the Java and Python numbers quite a bit. JDK 1.2.1 was problematic, and I think a lot of its poor performance there could probably be fixed with no more than a simple recompile. The past 10 years have also seen a wide diversification of Python implementations, including IronPython, Jython and the JIT-based interpreters. It would be very interesting to compare the performance of the same programs across platform implementations to try to knock down the variations and get to the effects of the choice of language itself.

kllvql 16 years ago | | |

Both of those languages have seen vast improvements in speed throughout the past ten years. Java 1.4.2 was immensely faster than any previous Java implementation. I would be very interested in seeing this study redone with the newest Java, Python, Ruby, and something Lispy.

nomen 16 years ago |

Indeed; for some aspects of the study ten years ago was a different epoch.

antileet 16 years ago |

I've seen so many benchmark comparisons that only focus on conciseness (that too involving lines of code), and speed. Yeah, they all re-affirm what is already known - C is fast, python/ruby is concise, C#/Java is somewhere in between and there are the ones which do well in both - haskell, clojure, etc. (And then there's APL)

All the benchmarks are algorithm and processor intensive, while quite a few applications are idling most of the time. How about a comparison regarding the standard libraries, ease of debugging, external tools, code readability, available support, external libraries, etc. In short - developer productivity.

I'm not saying performance is not important - it is quite critical, but there are other things code does as well (At least most of my code is mundane, and is just calling APIs, updating files, serializing objects, etc. Nothing too fancy.) </rant>

dalke 16 years ago | |

And your rant is relevant to this paper precisely how? As the paper also covers time to implement and memory use it, it doesn't fall into the category you complain about.

antileet 16 years ago | | |

I guess it's rather off-topic from the paper's content. I apologize for that.

patrickgzill 16 years ago |

In many respects, TCL is dead at this point ... there is still a lot of legacy use for it but few new users in comparison to Python, Ruby, etc. Rexx is dead outside of IBM shops, and was ever since Amiga and OS/2 died.

davidw 16 years ago | |

It's written "Tcl", not "TCL". This has been true for at least 10 years, as the article nicely demonstrates.

Nitpicking aside, I agree that it's not real lively, but I don't think I'd call it 'dead' quite yet either, as there is still a fair amount of active development, and anything that widely used doesn't just up and go away from one day to the next. They even published a new Tcl book recently:-)

http://journal.dedasys.com/2009/09/15/tcl-and-the-tk-toolkit...

pmarin 16 years ago | |

I have never seen a dead high level language. Right now I am writting a toy scheme language with it: http://github.com/pmarin/Muddy-Scheme. Python has a Tcl interpreter built in its standart library for Tk, so Python is also dead?

thisrod 16 years ago |

One section of the paper describes which algorithms the programers used. It appears that the solutions in scripting languages made heavy use of associative arrays, so the results might change if you swapped Tcl's hashing algorithm for Perl's.

The algorithms described don't seem very clever. I expect you could do much better by splitting the input number into digraphs or trigraphs, sorting those by frequency in the dictionary, and matching the rarest ones first.

balu 16 years ago |

It's always nice to see papers published by my profs on HN. I would not have looked at those on my own.

cmallen 16 years ago |

Am I the only one tired of mathematical tests? I'd rather see a test comparing the relative speed and scalability of various web app implementations.

And don't give me that line about scaling not mattering, I have to cope with a cantankerous server and database at work, I know full well what my problem sets are.