> Do I really need to use React (a client-side framework) to generate static websites? What's wrong with Ruby + Liquid (that I'm familiar with), or anything similar (Python, PHP, etc)?
I think it depends on the other tools you're using in your tech stack. React is a popular choice for highly interactive webapps, so the people who are into building apps with React benefit from not having to switch between vastly different toolchains for their blogs and marketing sites. Rails or Laravel users who use templating might do the same, pick a site generator that's close to their main way of working.
> Assuming that the static website doesn't include a lot of JS in the first place, exactly how much faster is the Lighthouse/PageSpeed score for static websites built with Gatsby/Next JS? Are there any benchmarks available? Do these frameworks optimize the CSS delivery as well (remove unused CSS, inline the CSS, etc)?
Speed depends mostly on how you bundle and load the assets for your site, fonts, images, external JS marketing widgets and stuff like that. Gatsby comes with tools to optimize those things, and when people say that Gatsby or Next are fast it's mostly because of the way you link to your sites internally. They have a custom <Link> component able to fetch pages in the background, so as soon as users hover over links it's already loaded when they're clicking. That's happening so fast that you often don't realize that something has changed. Personally, from a user perspective it's not something that I need to have, I expect pages to load or a transition.
> - What are the chances that the JS community will jump to the "next new thing" in 2 years and I'll be stuck with a pile of unmaintained mess?
It's likely that your page will be outdated once React itself advances and the ecosystems jumps on a new way of doing things. You have to be part of the ecosystem and "go with the times". This ties back to my first answer, most users of JS based generators are part of the ecosystem and even enjoy rewriting their stuff with newer tools, it's part of their work and also hobby.
> - Are there frameworks for Ruby/Rails that do a similar job? Does Jekyll (or Hugo) come close?
Jekyll is a sane long term choice, it's battle tested, has a nice ecosystem and won't likely change that much in the future. It's only that builds are slow, so for larger sites with thousands of pages and frequent content updates maybe not the best choice if you can't wait hours for it to build.
No idea about Hugo, but an alternative to Jekyll with much faster build times is Eleventy, but it's also young and likely going to change, and it's ecosystem isn't that mature. If you're chosing Rails for stability reasons then you can't really go wrong with Jekyll, it fits well into that strategy.