Sister blog of Physicists of the Caribbean. Shorter, more focused posts specialising in astronomy and data visualisation.

Friday 7 June 2019

Oh look, another misleading press release, how amazeballs

If you ever see the phrase, "mystery solved" in a press release, you should immediately spit on it in disgust, burn it, and scatter the ashes to the four winds. Should you need a worked example of this, here's one.

If you've been reading this blog at all, you'll know that I've been closely following the tale of the galaxy (actually there are now two of them) without dark matter. Very recently there have been press releases proclaiming in the usual fashion that the mystery is solved, that there's a simple explanation for the puzzling initial result, hahhah, now we can all go home for tea and stop worrying about it.

The problem, of course, is that this is absolute bollocks.

The best way of explaining why these press releases are talking crap is to recount the main papers on this topic chronologically. The main scientific point to bear in mind is that this discovery depends on the distance to the galaxy - an issue which was raised in the very first paper. If the galaxy is far away, as originally estimated, then it doesn't have any dark matter and is really weird and interesting. If it's substantially closer, then it has a pretty typical dark matter content and is nothing particularly special.

To be fair, the problem is not so much what the press releases say so much as when they say it. Both of them do a nice job of explaining the findings of a paper by Trujillo et al., published in final form in March 2019, which claims the galaxy is closer than originally thought. Why the press release has come out three months later I don't know. It probably helps to know that publishing a paper goes in several stages:
Submission
-> revision
-> acceptance (if you're lucky)
-> proof
-> advanced online access
-> published online for all to see
-> published in print.
Usually, at any stage in this process authors can choose to put the paper on arXiv. They might do this to make the results known as soon as possible or to gain feedback from a wider audience during the reviewing process. Who knows. Anyway, the history of this discovery goes like this :
  • The van Dokkum team published their discovery of a galaxy without dark matter back in March 2018. This immediately came under criticism, though much of it was very silly and unjustified.
  • The Trujillo paper first appeared on arXiv prior to acceptance all the way back in July 2018. This was much better than most of the other criticism and was pretty compelling stuff. As you can see in the link, I myself thought this was a very credible and appealing explanation. It felt like a very careful and thorough analysis to me, and certainly a much more serious challenge than other criticism which had been raised.
The problem is that that's where the current press releases end, on results that are basically almost a year old. That's a problem, because the science didn't stop a year ago. There were plenty of advancements since then that mean this result is already out of date :
  • The van Dokkum team immediately hit back, publishing a paper in August 2018 saying the the Trujillo distance estimate, which would make the galaxy much closer, was incorrect, and their original much greater distance estimate (which would mean the galaxy was weird and lacking dark matter) was correct all along
  • A completely independent team also published a short paper in August 2018, finding that they agreed with the van Dokkum distance estimate, i.e. that the galaxy is far away and lacks dark matter. The Trujillo paper does not cite this. They do address van Dokkum's response, but only very briefly in a few sentences.
So to call this one "mystery solved" is flat-out wrong (for an even more complete history see this post). There's good evidence that the distance measurement is at best far from settled, and it's very credible that the original distance estimate is correct (for a really complete commentary see van Dokkum's blog, for a piece by Trujillo see this). 

To be clear, there's nothing wrong with Trujillo et al. publishing their result - it absolutely should be published so that people can debate it. At worst, it might be a mistake to do a press release on such a still controversial topic - but raising awareness of the controversy isn't the problem either. The problem is the press releases choosing, yet again, to bluntly declare a difficult problem solved based on one paper, without bothering to examine the history and controversy surrounding the topic. That gives a view which is both over-simplified and confusing at the same time. It paints a picture of scientists as a bunch of over-confident, error-prone, careless nutters. It's damaging for trust and frankly just stupid. 

It's time to stop using the phrase, "mystery solved". In scientific press releases, this is basically an open declaration that the author doesn't understand science at all.

No comments:

Post a Comment

Back from the grave ?

I'd thought that the controversy over NGC 1052-DF2 and DF4 was at least partly settled by now, but this paper would have you believe ot...