Dynamical dark energy and the week that broke cosmology(perimeterinstitute.ca) |
Dynamical dark energy and the week that broke cosmology(perimeterinstitute.ca) |
https://preposterousuniverse.com/podcast/2026/06/22/358-solo...
https://preposterousuniverse.com/podcast/2026/06/29/359-solo...
Is this right? It would seem that adding another finding "in the same direction" should always increase the statistical significance, even if only slightly if the new evidence is week.
Under what circumstances could additional evidence for X reduce the estimated likelihood of X?
Suppose you've got a coin; null hypothesis is that it comes up heads and tails equally often. I flip the coin 5 times and get heads every time. Probability of at least this many heads on the null hypothesis is 1/32. Now I flip the coin another 9 times and get 5 heads / 4 tails: evidence in the same direction. Between the two experiments I have 10 heads / 4 tails. Probability of at least this many heads on the null hypothesis is [(14 choose 0) + ... + (14 choose 4)] / 2^14 ~= 0.09, much bigger than 1/32.
There are also other circumstances in which even additional strong evidence for X can reduce the probability of X.
Suppose I have three hypotheses A,B,C which initially I think are all equally likely. Then two things happen that both have probability 1/2 if A is true, probability 1/4 if B is true, and probability 0 if C is true. After one of them, I should think A is true with probability 2/3. After both, I should be more confident, right?
Nope. Suppose e.g. what I'm doing is pulling balls out of a bag. Hypothesis A is "I have either a bag of red balls or a bag of blue balls, with equal probability". Hypothesis B is "I have a bag with 25% red balls, 25% blue balls, and 50% green balls". Hypothesis C is "I have a bag containing only green balls".
So I pull out a ball from the bag and it's red. That happens half the time in scenario A, 1/4 the time in scenario B, and never in scenario C, like I claimed.
I put the ball back and shake things up so I'm starting afresh, and pull out another ball. This one's blue. Again: half the time in scenario A, 1/4 the time in scenario B, never in scenario C.
But those two things can't ever both happen in scenario A, because in that scenario I have a monochromatic bag. They can both happen in scenario B. And of course neither of them can happen in scenario C.
So I got a result that (on its own) was evidence for A over the other two hypotheses, and then another result that (on its own) was evidence for A over the other two hypotheses, and the effect of both together is that I know A is false and B is true.
1. In the first of those cases, with the biased coin, suppose that in the second phase you flip it _thousands_ of times and it comes up with N+1 heads and N tails. Then the statistical significance of the overall bias can become arbitrarily small, and if you had a concrete alternate hypothesis along the lines of "at least 5% more likely to come up H than T" the overall evidence can be extremely strong evidence against that.
2. In the second case, with the coloured balls, note that we can readily make each bit of evidence for A > B > C much stronger than the 2 : 1 : 0 above; the bag in hypothesis B could be, say, 1% red, 1% blue, 98% green. We'd still end up with B being the only one of our viable possibilities after both observations.
3. There are real-world situations with the sort of structure we see in that second case. Suppose hypothesis A is "there is a god who not infrequently works miracles", hypothesis B is "there are no gods, but people not infrequently fabricate evidence of miracles", and hypothesis C is "nothing at all miraculous-looking ever happens". Then you see what looks like a convincing case of evidence for one god, and then what looks like a convincing case of evidence for a different one. The two gods are from different religions each of which adamantly insists that the other one is pure invention.
4. The real world is never as neat as these examples. E.g., in the gods scenario you also need to consider possibilities like "actually both those religions have some truth to them, and rivalry between their gods is why each one's followers insist that the other religion is completely imaginary" and "there is a miracle-working god, but also followers of other gods fabricate miracles" and "there are no gods and very little fabrication, but sometimes incredibly unlikely things happen just by chance" and "there are no gods but there are super-advanced aliens who enjoy messing with us" and so forth. Even in the balls-from-bags scenario -- if it happens in the real world rather than in a textbook -- you have to consider things like "I was mistaken about what ball I picked the first time" and "actually sometimes one blue ball gets into a big bag of red balls by accident" and so forth.
(A lot of the most important work in, e.g., science consists not of testing rival hypotheses but of figuring out viable hypotheses to test.)
I mean that's how science works, discover something new, rewrite knowledge
not like religion where you make stuff up before you know anything about anything and then force everything new to adapt to the legacy of decisions thousands of years prior
also consider the dogmatic commitment that scientism has to methodological naturalism. this is the religious spirit of science today
The dark energy density is already a parameter in the model. The simplest case for such a parameter is that it's just a constant, so in the absence of evidence to the contrary, Occam's Razor led cosmologists to adopt it.
But now we have evidence that suggests that it's not a constant, so we're looking at the next simplest case, a function of time (but still constant everywhere in space at each instant of time). The article describes how the DESI data suggest that it's a slowly decreasing function of time.
What has not happened is people making up models and continuing to insist on them even after the data says otherwise. That's what the GP was saying religion does.
This isn't some slapdash random patching. Everyone involved knows that assuming that the λ in λCDM is a constant is shaky. It could be a function of time, or even location in some way. But it's simplest if it's a constant. So you start by modeling the universe as if it is. Then you determine what sorts of observations would support or contradict that, and you start making them. When you get results, you start examining what version of the model best explains those observations. And someone somewhere goes off to try find a better model than any version of λCDM. If they succeed, their model will eventually supplant it. This is how science progresses, even if the experiments are less under the control of the experimentors than they'd like. The important part that you make revisions in response to observations.
(FWIW, particle physicists are constantly frustrated that they can't find counterexamples to the Standard Model. They know it has to be incomplete, but the lack of contradictory observation leaves them no direction to try to improve it.)
If science operated like religion it would say "We dont know what dark energy is, but you have to accept it or your wrong".
Instead the argument is "Hmm... our current best understanding of the universe is lacking, when we add in these extra variables many of our predictions go from wildly inaccurate to nearly accurate. There must be something going on here we cannot observe, so lets call it "dark"."
Dark energy is "Wow, there's a factor at play here that we dont understand, without it our equations do not predict what we see, with it the equations are very accurate. Additionally we can predict multiple different things with this."
And the most important part, as soon as there is evidence that refutes the Dark Energy/Matter hypothesis scientists will quickly (over a few years, a decade at the most) walk away from it. Versus religious systems take centuries for people to shift their beliefs.
math doesn't try to explain why
math also allows predictions without understanding why
it's a placeholder to get things done
ie. correction factors
Einstein thought the universe was static but his math would not fit
So he made a "cosmological constant" to make static universe math fit
Hubble later used science to figure out the universe was expanding, not static
Einstein said "whoops" and threw out the constant
Except his math was right even without understanding why, it was just a model of the observed
The "cosmological constant" actually measures dark energy without even knowing what it was
Pretty amazing IMHO
What are you basing this on?
Here are a few excerpts from DESI:
* https://arxiv.org/html/2404.03002v3#:~:text=Although%20a%20c...
* https://arxiv.org/html/2404.03002v3#:~:text=Since%20the%20pa...
Scientists are also human and will tend to defend existing ideas. In cosmology the arguments over LCDM, dark matter and dark energy have been raging for decades. Dark matter people like to say that the Bullet Cluster is incontrovertible evidence of dark matter; MOND people don't agree and point to other aspects they claim refute dark matter! It's gonna take a long time for it all to shake out.
Moreover, your objection that science progresses one funeral at a time is exaggerated, even if it is real, it's not necessarily all that strong: https://pmc.ncbi.nlm.nih.gov/articles/PMC6814193/
The funeral thing isn't really a position I strongly hold; it's a pithy comment that points out science is a human endeavor and maybe not quite as objective as it is often presented.
But science as a whole is ultimately a self correcting methodology, even if it might take longer than we would want and doesnt always get it right.
It has always been a minority position - but it is interesting to see a different perspective. ΛCDM is not without it's own problems. Not least that when new observations contradict it, it just gets tweaked - since we still have no idea what the CDM, if it does exist, actually is. It does not have a great predictive record; it is good at explaining obervations after the fact (the CMB notwithstanding).
I am interested to know what the measurements within the solar system are that you refer to. I had thought this was far too small a scale to prove anything about MOND or CDM.
Sorry, not buying the argument from authority here.
> Fluids always have perturbations
Not sure I agree with this as a sweeping general claim; but in any case, my question was about what in the particular models under discussion you were basing your statement on.
> except in the special case of w = -1 (cosmological constant)
Yes, this part I agree with, a cosmological constant has to be, well, constant.
> otherwise, dropping them violates energy-momentum conservation and gauge invariance
I don't understand the argument here.
> Here are a few excerpts from DESI
Unfortunately these links don't seem to be showing me specific excerpts, just the whole paper. Can you give page/section references or equation numbers?
You asked what I based my answer on, and domain expertise is the answer. The rest was an actual argument.
> Yes, this part I agree with, a cosmological constant has to be, well, constant.
This is a nominal fallacy, since the reason it must be homogeneous (rather than just time independent) is actually the same reason all other (w != -1) fluids must not be homogeneous.
> Not sure I agree with this as a sweeping general claim > I don't understand the argument here.
The argument is general because it rests on energy-momentum conservation and gauge invariance. The perturbed energy-momentum equations for a fluid have source terms \propto (1 + w) * <metric perturbations>, and therefore cannot be solved by fluid perturbations that are zero at all time and locations unless w = -1 or the metric is also homogeneous. The same guarantee of dynamics underlies the gauge invariance argument: while one can choose a frame in which a single fluid is homogeneous ~~at any instant, that gauge choice is only valid at all times if the fluid's energy density is time-independent~~ EDIT: that property is only gauge invariant when w = - 1.
> Unfortunately these links don't seem to be showing me specific excerpts, just the whole paper. Can you give page/section references or equation numbers?
Open in a chromium based browser or search the article for "perturbations".
Can you elaborate on what you mean? LCDM hasn't been "tweaked" since \Lambda was established in the 90s (at least, not for large-scale cosmological observations). The current discrepancies are all at the precision level (<10%) rather than a qualitative O(1) difference. An important (but not dispositive) nuance is that LCDM well explains all observations individually; the percent-level tensions arise only between fits to datasets, meaning unknown systematics remain a viable explanation.
The unknown fundamental nature of CDM (as important a problem as it is) seems irrelevant to arguments about LCDM's predictivity (as a cosmological model).
1. LCDM predicted hierarchical formation of large galaxies through aggregation. JWST seems to show this is not what happens (large galaxies at early times). I am sure it can be made to work, but it was not a prediction.
2. Constant Lambda - recent obervations seem to show it may be changing (OK - that is more built into the theory rather than being a prediction).
By the way, I am not saying LCDM is not massively successful. It can clearly be used to explain a lot of things. I am saying that I am not aware of many things it successfully predicted ahead of observations being made.
CDM successfully predicted the CMB features that COBE could measure. LCDM certainly was a known model before supernova data in the 90s first provided evidence for it. CMB data shortly after detected a consistent fraction of \Lambda and have since provided evidence of multiple kinds (structure growth in addition to the original, the distance to the last scattering surface). Distances from baryon acoustic oscillations are another subsequent, independent test that corroborated the LCDM model.
2 and (especially) 1 are to me too speculative to place much weight on. I would also draw analogy to the Standard Model of particle physics: its predictivity and past success are not refuted by any deficiencies beyond the regimes we've been able to probe thusfar. (Of course, in the SM we fully expect those deficiencies.) Just to say that not every failed prediction falsifies a model's success in previously established regimes of energy/scale/dynamics.
> By the way, I am not saying LCDM is not massively successful. It can clearly be used to explain a lot of things.
Sure, I didn't think so.