Page Table Memory Consumption(frn.sh) |
Page Table Memory Consumption(frn.sh) |
For one, the hash on PowerPC is over the virtual address and a VSID (virtual segment ID) which functionally gets used as an address space ID[0]. That means that hashed page tables also have separate entries for each process mapping the same physical page.
Since the hashed page table is one (physically!) contiguous structure, that means that all of the same NUMA tradeoffs also apply to it as well.
Because it's based on a hash of the virtual address, you have the same issues with partial residency bloat of the structure as you either have to size the whole page table up in powers of two during hash collisions[1], or you have to treat the whole thing like a large last chance TLB and have a separate software page table to hold your ground truth (which will probably just be a tree based table anyway).
And on top of that, grabbing 8 spatially related PTEs at once rather than just 8 that happened to hash to the same thing also tends to reduce memory bandwidth rather than just latency for the same reasons that caching and prefetching works at all. There's a good chance the other seven PTEs are actually going to be needed soon if you grabbed their sibling, but a hashed page table has you generally grabbing 8 cache lines of PTEs for 8 virtually contiguous pages, for an 8x increase in bandwidth too for the same likely operations.
All of the references are great though; I really enjoyed reading through them. I hadn't stayed current with the state of the art since 2020ish.
0 - A little different than that since each VSID is just a subset of an address space and each process needs a handful of them rather than one, but close enough for this conversation.
1 - PowerPC hashed page tables give you two probes into buckets (PTEGs, or Page Table Entry Groups) that can each hold 8 PTEs.