Your code is fast if you're lucky(tiki.li) |
Your code is fast if you're lucky(tiki.li) |
if (BLQS_CMP(x, piv)) { *lwr = x; ++lwr; }
else { *rwr = x; --rwr; }
Although it's true compilers can be very sensitive to exact phrasing triggering specific optimization passes. So it still might not give the branchless version by changing it to pre increment (which is the same as a normal +=1).
The only way to really know is to dig into what optimization passes clang took in both cases and analyze the difference.
The computation is the same, but apparently, this tiny change prevents SimplifyCFGPass from turning the code into the branchless version later on. I'm not sure why this happens, perhaps because it messes something up in the pattern recognition of the pass?
So yeah, as you say, compilers can be very sensitive to this, and in the past (although more than 10 years ago now, so might not be relevant now), ICC often used to generate better (faster executing) code when using "val += 1" vs "++val".
The same meaning, but different performance based on notation—it's ultimately about entering LLVM's optimization pass, which likely comes down to differences in the internal IR pattern. It almost feels like a difference in innate talent...
I feel like I can build CRUD applications well enough, but I still seem to be weak at low-level processing.
Where can I learn these kinds of techniques? I'd appreciate any book recommendations.
Most software introduces a large number of unnecessary stalls where one part of the hardware is waiting on or bottlenecked by a different piece of hardware. Optimization is often about removing or minimizing these stalls to the extent possible. But to do this you need to understand why specific code choices cause the hardware to stall in the first place. The most basic form of this is CPU cache locality but there are many levels.
Compilers are great at localized micro-optimizations, except for SIMD. Most idiomatic code can be detected and transformed by the compiler into something that takes advantage of the hardware design. You don't need to do tricky bit-twiddling stuff, the compiler is better at it than you are in most cases (this wasn't always true). I always recommend people interested in optimization experiment running snippets of code through the compiler using godbolt with optimization turned on. You can learn a lot about how the compiler sees and understands your code by studying this. It is also a good way to became familiar with assembly.
Leveraging SIMD and vector ISAs is a mess. Different hardware architectures rely on different idioms and compilers cannot auto-vectorize most code that can be vectorized. You need to learn the idioms of each vector ISA and how to write them using intrinsics. A great way to learn this is reading other peoples' SIMD code. Modern SIMD code is wide in that you need to memorize a lot of things but conceptually pretty shallow. It mostly comes down to learning the idiomatic tricks and gadgets for an ISA -- code is composed from these conceptual primitives.
Thanks for the advice.
Personally I actually haven't read too many books on optimizations, I just absorbed information over years one thing at a time, but something like Computer Organization and Design is a pretty good intro to the low-level picture. If you want to drown in extremely dense technical topics that will give you a lot of jumping off points to search, read Agner Fog's microarchicture optimization guide (https://www.agner.org/optimize/). It won't tell you what LLVM is doing, but it'll tell you why it's doing it. Fair warning, it's dense and pretty dry.
Then it depends how interested you are in doing low-level nonsense. If you spend a lot of time writing performance oriented systems code, you'll come to use profiling tools that show you the assembly. If you stare at it long enough, you sometimes start to question why the compiler wrote it this way. And you're naturally led as you try to optimize your code to wonder how LLVM is coming up with this ASM that it spits out and why it sometimes gets it wrong.
There's nothing magical or that requires innate talent. You can learn all of this very naturally if you work close to the metal and take the time to question how the abstraction layer below you actually works. If you keep doing this, you eventually find out it's not that deep, it's just a lot of stuff accumulated over time, but none of it particularly difficult or inaccessible.
How ever, I will disagree slightly that all the optimizations compilers do are about optimizing for a given architecture; some transformations are just weird algorithmic black magic about optimizing the underlying code itself. Knowing how to make sure the compiler sees through a given construct to give you the low level expression you want is too much art and randomness; we need better ways to express optimization expectations so that if the compiler fails to match expectations it becomes a loud compiler error.
In this specific instance, at the hardware level it helps to understand how the branch predictor works and why quicksort in particular is essentially the worst case for the branch predictor, and then you'll understand why the cmov/csel optimization is such a big win.
Second step is to actually make it faster.
And I would say 99% of the time you can make applications faster without going down so deep on this low-level (which is still not easy) but sometimes you can only get faster via low-level optimizations.
In general, you must expect to try ten things and then one of them will help you; fewer if your ideas are bad. :-) Occasionally, you learn new things (either about your machine or your language or your code base) and then you can try that elsewhere in the code (but don't go overboard, every technique has its limits).
DO NOT FALL PREY TO SUPERSTITION. Always measure in one way or the other. Don't do stuff blindly just because someone on the Internet told you (there's a _lot_ of bad performance advice out there).
1. If you write CRUD apps, make sure that the database does the heavy lifting.
2. Take note of algorithm complexities, use hashtables as appropriate, and write good hash functions when you start using hashtables/dictionaries.
3. Avoid pointer-heavy datastructures. In high level languages like Java, an object reference is a pointer dereference that can stall the CPU waiting for memory. This is sometimes optimized, but you can’t depend on it. The true zealots call this “data oriented design”.
4. If you write C/C++,rust, or the like, you might want to learn to read assembly. Godbolt.com is a fun way to learn. Note that not all instructions are equally fast: Long division and trigonomic functions are slower than integer adds, even when they are both a single instruction.
5. The next level is probably going for vectorized instructions: SIMD (ARM Neon, AVX). The most original applications can be found at lemire.me: a professor exploring optimizing things like JSON parsing using the latest processor features.
Quicksort is supposed to be an algorithm that has O(n) to O(n²) performance and O(n log n) being only an average performance case. Test was made on random data coming from different archs (so I doubt it's characteristic would be remotely identical).
Given input size of 50M it means that performance could be between 50M (5e7) up to 2.5e15. That's like performance instability of 8 orders of magnitude.
I'm not sure here if we can't write instead that "Your code is fast if you picked fast case for it" especially since fix of 6 OOM is smaller than algorithm's performance range.
I think with this approach, we will only win if a language allows for:
1) ease of writing, and 2) fastness
Right now languages don't really combine both. We have ease of writing e. g. ruby or python, but they are slower than C, our godfather language. So far all languages that try to solve both problems, become mega-verbose and tend to gravitate more towards one than the other - usually e. g. "let's write a replacement for C". I wonder if combining both 1) and 2) is possible, kind of like select on your own what to combine, so if my time is precious, I write a quick prototype. If this must become faster, I write it with more details. That's still not really a language that combines 1) and 2) genuinely but perhaps it is an acceptable trade-off. Right now we kind of mix two languages here, say, ruby+java or python+C or any other similar combination.
else *rwr-- = x;
else{
*rwr--=x;
}That said, I'm not keen on "argue and you're off the project" work environment.
bool v = BLQS_CMP(x, piv);
int* ptr = v ? lwr : rwr;
*ptr = x;
ptr += int(v) * 2 - 1;
int* ptr = ((v != 0) * lwr) + ((v == 0) * rwr);
Can you do the same on the windows kernel or apple kernels?
Do you have those windows kernels?
Arch wiki says its okay.
You need alternative linux kernels with their modules.
*lwr = x; lwr++;
block
statement (assignment)
expression
operator (dereference)
variable
expression
variable
statement
expression
operator (post-increment)
variable
*lwr++ = x
block
statement (assignment)
expression
operator (post-increment)
operator (dereference)
variable
expression
variable
As for why the second form is faster (or rather, why the compiler can generate faster code): There is likely an optimisation pass somewhere in llvm that recognises a pattern that this fits into, which allows it to generate branchless instructions. For instance, in the second form, there is a pattern of:
operator (post-increment)
operator (dereference)
Finding which pass that is can be non-trivial, as it's more than a matter of enabling individual passes until one works. It might be that an earlier pass does some code reshaping that allows the relevant pass to work. My suggestion would be to dump the llvm ir at the end, and find the rough pattern that you're looking for, then re-run the compilation with `-mllvm -print-after-all` to see what the IR looks like after each pass, and then manually "look back" until you can't see the pattern any more.
block
statement (assignment)
expression
operator (post-increment)
operator (dereference)
variable
expression
variable
Or perhaps I am wrong and it would generate an AST like you describe and rely on later passes to actually create a proper dependency graph. My mental model of how these kinds of postfix operators work always assumed it must very early on turn it into two separate statements. Thank you for the suggestions.
Why restrict the optimisation unnecessarily like this? It's never the intention to artificially restrict optimisations, but they are difficult to test and debug as they can interact with other optimisations, and applying an optimisation too broadly leads to horrible correctness bugs. So if you cannot be certain that you understand all possible interactions now and in the future, it makes some sense to be conservative in applying them.
The SimplifyCFG pass in LLVM pulls the identical-in-all-but-one-operand store IR instructions into a successor block with a [select instruction](https://llvm.org/docs/LangRef.html#select-instruction) that will ultimately become a conditional move. But the pointer math differs by two operands (because there are two different pointers and one is being decremented and one is being incremented) so SimplifyCFG stops immediately and never reaches the stores.
This sort of brittle behavior where semantically identical code produces different optimization results is common in optimizing compilers unfortunately.
Edit: not saying they are different just that it is harder for the compiler to see the safety of the transform when they are quite different.
You can ask the compiler to dump its intermediate representations, so it might not be too hard to answer the question.
Both versions sort the same data using the same algorithm. Just a tiny change in the source code caused Clang to generate different machine code.
Using different seed values - (srand(1), srand(2), srand(time(NULL))) essentially leads to the same result. With a good choice of pivot, Quicksort is very close to O(n log n) in practice, so that’s not the key factor here.
The interesting thing is that the generated machine code changes significantly.
It's just rand() in test.c
Heapsort has the minimum number of comparisons. Search up "An optimal algorithm for deleting the root of a heap" by Svante Carlsson. Notice how Heapsort is commonly used only as a fallback sort? That's because minimizing comparisons isn't the most important.
I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set.
Also note that with a randomized pivoting you _might_ hit a O(n^2) worst case, it's just that it's incredibly rare and cannot be forced by an attacker controlling your input, so for most practical purposes can be ignored.
block
statement (assignment)
expression
operator (dereference)
operator (post-increment)
variable
expression
variableI accept that if you're extremely used to writing this style of C code, it might be something that you're used to, and understand implicitly. As a C++ engineer that infrequently comes across this precise pattern, having the precedence made explicit makes it much easier to understand.
>Knowing how to make sure the compiler sees through a given construct to give you the low level expression you want is too much art and randomness; we need better ways to express optimization expectations so that if the compiler fails to match expectations it becomes a loud compiler error.
There's a parallel with hardware there. Verilog is a kind of hardware language designed for an abstract simulator, in the same way than C is designed for a standard abstract machine for the sake of portability. You end up with an idea of the assembly/RTL you want the compiler/synthetizer to generate in your head, and then it's a game of writing the right pattern that will be recognized and generate the output you want.
I think this is partially unavoidable, because we're inherently asking the compiler to generate a non-portable target-specific output in what is supposed to be a portable high-level language. If you start injecting compiler hints or requirements in your "portable" code, it all becomes a bit of a mess. Part of the problem is also that the high-level languages we're using were designed at a time were many questions were still unsettled. Things like signed integers being two's complements is a recent change in C and C++. But I think some of it is intrinsic impedance mismatch between high-level code and machine code.
I'm not sure I would like a proliferation of annotations that direct exactly how the compiler should optimize (like "must use cmov/csel here"), because if internal optimizer choices become public API, people will rely on internals in their large legacy codebases. I expect this would be a force that ossifies the compiler and prevent optimizations from improving. The "register" and "inline" keywords in C used to mean something to the compiler. But they were misused, having them be a requirement would have held back performance more than anything.
Then again I accepted the same justification against Postgres planner hints, and now that the idea has been recast as a plan stability feature I'm actually very happy with that idea. I'm uncomfortable with letting old calcified codebases hold back compiler internal, but at the same time once you find a way to have the compiler generate what you want, there's a real need to not have it break silently when you upgrade.
As you say register and inline were wrong, but we have force inline and force inline so clearly the pendulum swung back a little bit because the compiler completely ignoring is also not good. We have ways to force the compiler to do an unconditional move because source level heuristics are completely incorrect for making such a decision. The die is already cast, we just keep living with a shitty status quo instead of something a bit more robust.
This sort of optimization is one that'd I'd not spend too much time trying to fix or catch, unless you are doing it for fun or you have a specific piece of code in a hot path that needs to go fast.
Where I'd spend time if I were trying to write very fast code which a compiler is unlikely to get right is SIMD optimizations. Specifically with floating point values.
One thing compilers can't and won't do is reorganize floating point optimizations (well, unless you explicitly give them permission to do that). That means the way you write your floating point code can really nerf performance and exclude you from much faster assembly.
The sure fire way to actually make such code faster is learning and using SIMD expressions. The compilers can sometimes get this right, but it's quite fragile.
> I'm not saying that optimization isn't valid, what I'm saying is that Quicksort shouldn't be optimized over randomized per run data set.
But yeah, this is correct. When optimizing, we want to pin every variable other than the change, as much as possible.
Big and small N different for different algorithms and hardware both.
I've definitely had to change things into SOA in order to eke out performance. My coworkers aren't thrilled seeing `double[] x; double[] y;` but that really is about the only way to get the JVM to play nice.