Benchmark: C++ vs C#

78 points by prog 14 years ago | 35 comments

cageface 14 years ago |

The results seem conclusive: if you're developing software for the desktop, and (like me) you are equally skilled in C# and C++, it's safe to take advantage of the simpler syntax, rich standard libraries, stellar IntelliSense and reduced development effort that C# offers.

With the glaring exception of numeric code, which is the most crucial thing for a lot of apps that are now written in C++. You'd be crazy to write a native CRUD app in C++ now but most of the big C++ apps I can think of do heavy work in numerics (Photoshop, Ableton, Maya, etc).

kia 14 years ago | |

For these the best approach is to write the whole app in some higher-than-C++ level language with performance critical parts (usually less than 10 percent) in C/C++.

cageface 14 years ago | | |

That's often a good approach, but sometimes the overhead of marshaling data between the layers can kill you. For instance, Google warns people about assuming the apps written using the Android NDK will be faster, partly because of JNI overhead.

pragmatic 14 years ago | |

http://www.getpaint.net/

I'm not saying paint.net is a replacement for photoshop. However, for me, it's in the "good enough" category.

http://en.wikipedia.org/wiki/Paint.NET

"Paint.NET is primarily programmed in the C# programming language, with small amounts of C++ used for installation and shell-integration related functions."

Again this is good enough for a lot of peoples image manipulation. That area of "good enough" is getting larger all the time.

dagw 14 years ago | | |

Paint.NET is fine for small images, but performance really falls off a cliff when images get larger, whereas Photoshop hardly misses a beat. Which I guess is kind of your point, for doing 'normal' tasks under 'normal' loads C# is fine, but if you have to go beyond normal, then you really need the control and speed of something C-like.

cageface 14 years ago | | |

Certainly C++'s niche is getter smaller every year. It just happens to be a niche occupied by the most interesting (IMO) native apps.

The flood of low-powered mobile devices out there seems to have given it some new legs though.

bitdiffusion 14 years ago |

The poor guy is taking a beating in the code-project comments which is a pity since he obviously spent some time putting it together. In a comparison like this, you are always going to get people on either side of the fence claiming the results are invalid because there was some compiler switch or code construct that was missed.

The language-agnostic version of this article: you can write slow code in any language.

JoeAltmaier 14 years ago | |

In the article he illustrated that with a Microsoft C++ library example (a heap?). In the benchmark he fixed it by writing his own. You can do that in C++

S_A_P 14 years ago |

I have heard that you can expect a 10-15% difference in most cases. However, it seems that there are just some arenas that C# wont quite measure up. I'm thinking multimedia applications would not scale well. Mainly because there is no way via Microsoft's .NET implementation to access SSE or other processor intrinsics. I know that mono announced that this a year or two ago, anyone have experience with it?

profquail 14 years ago | |

You're thinking of Mono.Simd. I've toyed with it a bit, and it seemed pretty fast, but I haven't yet had the chance to write something "real-world" with it.

Miguel de Icaza wrote an article with some preliminary benchmarks when they announced the release a few years back: http://tirania.org/blog/archive/2008/Nov-03.html

ghenne 14 years ago |

We've been aware of how slow CF is on Windows CE for a while. Here are some benchmarks we have run:

CuWin 3500, CE5, NS Basic/CE 8 NS Basic/CE: 26529 loops/second Microsoft .NET: 133 loops/second

Acer beTouch E101 Windows Mobile 6.5, 528 mhz, NS Basic/CE 8.0 NS Basic/CE: NS Basic/CE: 16669 loops/second Microsoft .NET: 3060 loops/second

CuWin 5500, CE6, NS Basic/CE 8 NS Basic/CE: 14394 loops/second Microsoft .NET: 764 loops/second

i-mate 400mhz, WM5, NS Basic/CE 8.0 NS Basic/CE: 12632 loops/second Microsoft .NET: 1290 loops/second

Running the same test under NS Basic/App Studio, an iPhone 4 scores 282,674, Nexus One is 403,000 and the Motorola Xoom is 718,484. Plainly, Windows CE.NET has a long way to go!

scott_s 14 years ago |

He did not say if optimizations were turned on for the debug builds, and I'm not familiar enough with Visual C++ to know what the defaults are. You can still have DEBUG defined so that asserts are compiled in when optimizations are on. However, looking at the results, I suspect optimizations were turned off, which makes those results meaningless.

kevingadd 14 years ago |

The benchmark numbers for the Compact Framework on ARM make this a lot more interesting. I used it on PowerPC (the XBox 360 .NET runtime used by XNA is based on the Compact Framework), and I was uncertain whether the issues were related to the PowerPC architecture. It looks like the Compact Framework is just bad.

blub 14 years ago | |

I wonder if this is the same thing that's used in WP7. Bad news if it is, I guess...

Impossible 14 years ago | | |

The compact framework has traditionally been horrible and kind of a joke. I hope MS has either ported the proper .NET framework or greatly improved the compact framework for WP7, if not it makes the choice of no native code support even worse.

pragmatic 14 years ago |

I think C# is in the category of "fast enough".

When I was working through project Euler (http://projecteuler.net/) with various languages, the only one I really had to optimize was Python (vs C/C#/Haskell).

I really like Python, and for most stuff it's fast enough. Just saying the real penalty (that I noticed) is a scripting language, _if_ it's heavily arithmetically constrained.

Some (non-trivial) games in C#: http://www.arcengames.com/w/index.php/aiwar-alienbundle

http://murudai.com/solar/

However, it's possible and even practical to also write games in Python. See the pygame project.

adolgert 14 years ago |

This article raises interesting questions. He has done low-level tests which do not read much from the data or instruction pipelines, so these tests show compiled MSIL can be better optimized than C++. C++ is prone to accidental performance deficits from problems like aliasing, so that makes sense.

Now I want to see higher-level tests, where algorithms reading significant data or application-level benchmarks are run. The faster the CPU core, the less penalty there will be from C# compilation. The problem is that it would take lots of time to translate something like WORF, GAMESS, or other standard numerical benchmarks.

peacemaker 14 years ago |

I think he hit the nail on the head referring to c# programmers in general not being as "performance conscious" which leads the misconception that c# is a much slower language to use. I've worked in many places and have come across the same thing. As a c++ programmer I have also worked with c# coders and am often shocked at their slap-dash approach to software engineering. I'm not saying they're all like that but it seems to be a common trend in my opinion!

politician 14 years ago | |

Maybe that's true, but I'm working on a Windows Phone 7 app right now that struggling to hit its performance targets. The major culprit? Silverlight's data-binding. Microsoft shipped the framework without shipping high performance controls necessary to meet the certification requirements of their app store (ListBox, I'm looking at you), so while our code might return the results of the list in <20ms, it takes another 800-1500ms for the results to be rendered. It's frustrating.

yread 14 years ago | | |

Have you tried using the virtualizing listbox?

nxn 14 years ago | |

Do you think those programmers would suddenly become "performance conscious" if they switched to c++? What I'm saying is that I don't see the point in referring to them as "c# coders" they could be using any language just as badly.

shin_lao 14 years ago |

This benchmark is totally wrong as you can read in the comments of the page.

Basically he didn't turn on all optimizations on the C++ program and used the "slow but secure" version of the Microsoft STL.

Also his GenericSum uses array in C++ and lists in C#, so the C++ has to reallocate at each append.

I could go on for long, I appreciate the intent, but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

nxn 14 years ago | |

>> but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

Got any proof of that?

queensnake 14 years ago | | |

Not proof but, fwiw I read an article saying that.

shin_lao 14 years ago | | |

When they started Vista they wanted to go .NET all the way and had to go back using more native code.

chrisbennet 14 years ago | |

>I could go on for long, I appreciate the intent, but all you need to know is that Microsoft gave up on using C# everywhere because it's damn too slow.

As a data point, a friend of mine works for Microsoft on one of the Blend products. He's using C# for at least some of his work.

lutorm 14 years ago |

I don't know much C#, but it seems to me there are so many ways of writing "numeric" code in C++. Using the STL algorithms is one, using C-style looks on POD arrays another, and using an expression template math library like blitz yet another. You can't really say that any one of them, except possibly the expression template method, is uniquely C++.

omouse 14 years ago |

Even though the benchmark is incorrect, the flawed conclusion should lead you to see that Java is just as good a solution as C# (possibly better since it runs officially on more platforms).