Cranelift, Part 4: A New Register Allocator(cfallin.org) |
Cranelift, Part 4: A New Register Allocator(cfallin.org) |
We also have an explicit focus on correctness, simplicity, and verification. One could argue that in practice LLVM is used everywhere and has dozens of active core contributors, that bugs and missed optimizations are shallow at that scale, and it's hard to compete with that; and there is some merit in that... but our codebase is two orders of magnitude smaller, and we're actively engaging with academics and designing things -- our lowering DSL, our regalloc's symbolic verifier, our fuzzing-first approach -- to get the most mileage we can out of our efforts. It seems to be working OK so far!
[0] https://github.com/bytecodealliance/wasmtime/blob/main/crane...
Does this mean Cranelift has a clearer view of what its IR really means than, say, LLVM? It seems to me that being very clear-eyed on this will be important as C and C++ get ready to bite the bullet (perhaps this decade) and formally document how pointer provenance works in their languages, and perhaps Aria's provenance "experiment" in Rust begins the journey to stabilisation.
I thought one of the huge advantages of using SSA is linear-time register allocation?
In practice, theoretical optimality matters far less than the actual heuristics you use (e.g. avoid spills inside of a loop.)
It would seem like there is still ample space to study the application of neural networks to register allocation. [2] They already gave pretty good results to branch prediction. [3]
What are your thoughts on the application of NN for RA and how would you structure the training set?
[1] http://incompleteideas.net/IncIdeas/BitterLesson.html
[2] https://www.semanticscholar.org/paper/Real-time-physical-reg... I haven't read the paper, just found in a quick search.
I'm also not sure when the last time I heard someone seriously make this claim was...
I'm not super-familiar with the pointer provenance work in Rust but I'll read more about this; thanks for the mention!
On that topic, NN for branch prediction are, in some sense, nothing new; perceptron and perceptron-based designs have existed for at least 20 years. But I'm not aware of anything concrete or specific in current BP designs as of recently (beyond marketing hype) but I haven't kept up with it; maybe some variation of TAGE with model-assistance is out there, but I'm not sure.
I do not know what a training set or neural network model for performing register allocation on real-world programs would look like at this moment.
TAGE is what I was thinking of [1,2]. Thanks for the reminder.
I found some relevant research in neural register allocation, "2020 LLVM in HPC Workshop: Deep Learning-based Approx. Graph-Coloring for Register Allocation"
https://www.youtube.com/watch?v=4FW7iznzIoE
There is also some nascent work on applying neural techniques to constraint optimization problems.
[1] https://www.semanticscholar.org/paper/A-case-for-(partially)...
[2] https://www.semanticscholar.org/search?q=TAGE%20branch%20pre...