The long and short of it is that it's basically impossible to do interventional studies on sleep deprivation in which you can see longitudinal as opposed to acute outcomes. Interventional studies are all short duration because no research subject is going to be willing or probably even able to comply with long-term sleep deprivation, and interventional studies on the order of decades over which things like cancer and death tend to happen are rare anyway simply because you can't intervene in a research subject's life for that long a span of time.
So understand the limitations of observational epidemiology studies. Correlations are observed between some outcome of interest (i.e. cancer and death) and a variable or variables these may or may not be impacted by (i.e. duration, quality, and regularity of sleep). If you observe a positive correlation, well, what causes it? Maybe poor sleep gives you cancer. Maybe cancer gives you poor sleep. Maybe a life full of terrible stress gives you both poor sleep and cancer. The researchers can't possibly know. This type of research is done to find interesting correlations that can then be investigated further by conducting interventional studies.
This is what the public doesn't seem to understand. Observational studies in nutrition and exercise science and what not are always being posted to Hacker News, but these are not really meant for public consumption. They're meant to suggest avenues of further research by other scientists. You see the same thing with various classifications by standards bodies, for instance, the IARC carcinogen thing. They classify compounds as possibly carcinogenic not to warn people off of using them, but to suggest to other researchers what they should bother and not bother looking into.
Unfortunately, as I stated in the first paragraph, it is hard to go further with sleep science. The problem with the book "Why We Sleep," caveating here that I haven't read it personally, is I'm pretty sure it goes beyond what the evidence warrants in order to state with far too much certainty that losing sleep causes a whole lot of bad outcomes when we do not know that to be true. My engagement with this largely goes back to the earlier Hacker News threads about this book and the Barbell Medicine and Stronger by Science podcasts going into fairly detailed debunkings of it and breakdowns of what we really know about sleep deprivation and poor sleep.
Treating insomnia is a different matter. People with insomnia will definitel consent to interventions, including long-term interventions. Whether or not any evidence exists for various interventions other than pharmaceutical sleep aids, I'm not all that aware of. Out of all the over the counter supplements people recommend, I'm pretty sure melatonin is the only one known to work, but it has drawbacks in that it tends to leave you drowsy when you wake and some people report wild and disturbing dreams. I'm sure it's worth trying, though.
There is plenty of pop sleep hygiene advice out there. I have no idea what sort of evidence other than anecdote it really has behind it, either, but it's also harmless to try. The standard keep your room cool, remove all lights, commit to a regular schedule, be in bed 30 minutes before you plan to sleep, all that that I'm sure you've heard before. I know my wife is trying a bunch. I'm not sure how well it's working. I used to be an insomniac when I was a kid. Now I'm not. I didn't really do anything. That seems to be common as well, or at least a lot of podcast hosts reported the same thing. They were insomniacs. Now they're not.
Granted, I run 60 miles a week, lift four days a week, and rock climb three days a week. I'm sure a lot of people will just say that much exercise is going to make you tired. I do believe evidence suggests that, but I was just as active as a kid.