Sally McKee, who coined the term "the memory wall", has died(online-tribute.com) |
Sally McKee, who coined the term "the memory wall", has died(online-tribute.com) |
https://dl.acm.org/doi/10.1145/216585.216588
on her obituary page, you will see a prominent "Memory Wall" link that is NOT a reference to her paper, but a place for sharing your thoughts about her life
I notice these things a bit more as she was my PhD thesis advisor
Wulf et al.
Wulf and McKee
It'd be interesting to see someone use the unabbreviated form; I have a hunch they wouldn't know to say "et alia".
Despite this, massively increased memory bandwidth does not translate to material performance improvements on non-parallel compute tasks because few tasks are actually memory bandwidth bound, instead being memory latency bound.
The best known general solutions for improving memory latency are per-compute element memory caches. Unfortunately, this increases the complexity and size of your compute elements forcing you to reduce the number of compute elements, but a large number of compute elements is the only way to saturate HBM memory bandwidth.
To keep up the best known techniques are either algorithmically batch which allows you to go wide using vector/batch instructions or you go the GPU route with memory latency-hiding parallelism.
Call your loved ones :(
Isn't that just a per-compute cache/local memory? You're proposing a scaled-up variety of NUMA where every compute core has its local memory and going outside that will cost you more.
It’s also not my proposal. The industry is exploring ways to cut down the energy requirements to do AI - 80-90% of the memory consumption is just moving memory back and forth across the memory controller. It has to read a row from a bank into a row buffer, access the specific cell being requested and then shuttle it over the bus to the compute and then write the data back to the cells. The current idea is to maybe do the processing on the entire row buffer but you could imagine scaling that up to do it at the bank level. The challenge is manufacturing complexity since DRAM is made different, heat from the ALU, etc.
[1] https://semiconductor.samsung.com/news-events/tech-blog/hbm-...
I really wish we had a better way to "name" papers. Big clinical trials often have an acronym (often hilariously forced: "CXCessoR4"). That takes the emphasis off (one) lead author but it's implausibly hard to make up one for every research paper.
Do you see the inherent tension in what you're claiming vs the lived experience of everyone in this post (including you!)?
One paper doesn’t make a career (she wrote many dozens), it’s not always cited weirdly, and even if it is, some people may remember the coauthors (as they should).
But since you mention lived experience, I’ll add that I’ve certainly been asked if I’m "even aware" of results from co-authored papers where my name was listed second—-and I don’t think this is very uncommon experience.
i'm not using software for this if this is off the top of my head, and it's the sort of thing that, at scale, hurts the forgotten author and their students
The author lists for economics papers are traditionally alphabetized, so more of your output will be known by your name if it occurs early in the alphabet. Abbie Ableson gets lots of mentions as "Ableson et al." while Zhang Zhu will almost always be relegated to the "et al". If name recognition matters, you’d expect successful academic economists to be clustered at the beginning of the alphabet—-and this appears to be true.
In most psychology journals, the author list is instead ordered by contribution/senority, and this effect disappears. https://www.aeaweb.org/articles?id=10.1257/08953300677652608...
Metrics are often attempts to formalize this but they’re not how most people actually make decisions: nobody is inviting seminar speakers or choosing collaborators because they have a high h-index. If anything, it goes the other way: name recognition gets you invited to speak or collaborate, which makes more people aware of your work, which boosts metrics.