Linux Memory Management FAQ(landley.net) |
Linux Memory Management FAQ(landley.net) |
I have made a few patches into the mm subsystem some simply inspired by researching for the articles.
I guess this is not the most up-to-date document?
it's also not correct. It doesn't have all 4GB "all to itself", because a portion of that (usually 1 or 2 GB) is mapped to the kernel.
A process does indeed have all 4GB of VIRTUAL adress space to itself. unless I'm misunderstanding you.
And regardless, I think the majority of systems running Linux today are phones, which usually have 4GB or less of RAM.
But I expect the FAQ was probably originally thinking about desktop or server systems, so, yeah, the intent there is probably out of date. Those types of systems are rarely 32-bit these days, and usually have a bit more than 4GB of RAM.
Even this is quickly becoming less and less true (for new phones). Even the Pinephone comes with 3 GB of RAM at a $200 price point, and that's inflated because of the niche, low volume nature of its production.
Samsung's "mid range" A series smartphones, for instance, start at 3GB at the absolute lowest end, with most models coming with 6 GB of memory. I expect this will be even more common in a year or two.
address sizes : 36 bits physical, 48 bits virtual
address sizes : 40 bits physical, 48 bits virtual
Other than that, I also think that even when outdated, computing history is worth reading anyway, since it gives you a natural understanding of _why_ we do what we do these days. In your day job, it also gives you a different appreciation for what people did and why they did it, and why 'this horrible code' may have made sense at the time.
Furthermore, performance engineering is fundamentally about opposing code and hardware limitations. If hardware limitations are different, you'll get different code, but the principles remain the same.
If you're curious, write a basic emulator for older hardware (the NES is a great choice) , it's both fun and eye-opening!
Edit: the NES emulator will answer 'how do you fit super mario bros in 32k, and how can it run on such limited hardware?'
Sometimes, but a description of the state of the art in the past does not become a historical tract with the passage of time. The better ones do; others just become outdated.
(And, of course, hardware transactional memory can be used to implement 'software' transactional memory faster than in software.)
However, STM only really works well in languages that are pure by default, like eg Haskell (or perhaps Erlang might be close enough). In a language with pervasive mutations and side effects, it's too annoying to use. Microsoft tried to make it work for .net for a while, and gave up.
In linux (in sane configurations) allocations are just preorders.
EDIT: I can't reply below due to rate limiting:
I'd argue that overcommit just makes the difference between allocation and backing very stark.
Your memory IS in fact allocated in the process VMA, it's just the anonymous pages cannot necessarily be backed.
This differs, obviously, in other OSes as pointed out. Also differs if you turn overcommit off but since so much in linux assumes it your system will soon break if you try it.
Not really. There are lots of holes in the physical address map. Look at /proc/iomem. Look at all of the gunk in there at addresses lower than the amount of RAM you have. Look at the highest “System RAM” address. It will be higher than the amount of actual physical RAM that you have.
That way userspace-to-kernel switch does not require changing active page table (and also avoids switcharoo each time kernel needs to access userspace memory).
In other words, we tend to focus on positive results, but negative ones ('don't do this or.. !') can be equally interesting and useful.
However newer incoming 5-level page intel chips [1] will allow up to 57 bits of address space, 128 PiB in theory though in practice 32 PiB of userland memory. See also [0] for discussion on practical limit for 5-page too!
[0]:https://github.com/lorenzo-stoakes/linux-mm-notes/blob/maste...
Having said that I did write a patch to ensure that the system would boot correctly with 256 TiB of RAM [0] so perhaps I am not always a realist... or dream of the day I can own that system ;)
[0]:https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-n...
This is actually kind of a cute way of dividing kernel and userland space as you just set the upper bit to 1 for kernel addresses and 0 for userland.
EDIT: Specifically talking about x86-64 here.
https://github.com/lorenzo-stoakes/linux-mm-notes/blob/maste...
Apple uses the high bits to cryptographicly sign the pointer value.
It's a useful optimisation technique where you can add some extra metadata without having to dereference a pointer.
As for storing tags in pointers on 64b platforms it is probably better to use the 3 low order bits. Another useful trick is what was used in PDP-10 MacLisp and is used by BDW GC: encode the type information in virtual memory layout itself.
An example of a split-the-difference approach is macOS, which AFAIU implements overcommit but also dynamically instantiates swap so that overcommit-induced OOM killing won't occur until your disk is full.
Also, it's worth mentioning that on all these systems process limits (see, e.g., setrlimit(2)) can still result in malloc returning NULL.
Not sure what you mean by this - I don't think Windows has overcommit in any form, whether opt-in or opt-out. What it does have is virtual address space reservation, but that's separate from commitment; reserved virtual memory is not backed by any page, no matter how much free RAM you have, until you explicitly tell the system to commit physical memory to it.
In fact I'm not even sure 'opt-in' to overcommit is possible in principle. Because if you opt-in to overcommit, you jeopardize other applications' integrity—who likely did not opt-in.
Are failures when accessing the allocated pointer due to overcommit substantially different than failures due to ECC errors or other hardware failure, with regard to what is specified in the c standard?
(FWIW I don't particularly like overcommit-by-default either)
I agree, reliance on overcommit has resulted in stability problems in Linux. But IME stability problems aren't induced by disabling overcommit, they're induced by disabling swap. The stability problems occur precisely because by relying on magical heuristics to save the day, we end up with an overall MM architecture that reacts extremely poorly under memory pressure. Whether or not overcommit is enabled, physical memory is a limited resource, and when Linux can't relieve physical memory pressure by dumping pages to disk, bad things happen, especially when under heavy I/O load (e.g. the buffer cache can grab pages faster than the OOM killer can free them).
Linux overcommitting memory and especially chrome/firefox beeing big-fat-memory-hogs are the problem. In fact every application which doesn't cope malloc beeing out of memory or assuming everybody has multiple gigs of memory to spare should "reevaluate".
I think Intel is adding CPU support for pointer tagging operations in the future which should make them a lot easier / safer / more efficient to work with, though I can't find a reference now, it doesn't refer to it as pointer tagging.
Any more information on encoding the type information in virtual memory layout? Sounds cool.
I guess you have different types allocated in specific regions?
And you are right that the tag inside address trick involves allocating objects of same type in different continuous regions. Usually such that whole page contains object of same type (as far as the tagging scheme is concerned) and by either masking off lower ten-ish bits of pointer you get to type header or you have some global out-of-line map of page frame->type.
I don't think pointer signing requires TBI though. Pointer signing uses the PAC instruction to sign a pointer, and the AUT instruction to verify and unpack the signed pointer, but in its signed/packed form it is not a usable pointer. So actual addressable pointers need not support non-canonical addresses.
A paper on it from Qualcomm: https://www.qualcomm.com/media/documents/files/whitepaper-po...
And there's also MTE which is upcoming.