Confirmation what now ?

The article below is an old one of mine that attacks the charge that scientists are all, in a sense, in league with each other and refuse to consider alternative ideas because it's all a sort of old boy's club. There I concentrated on the fact that new ideas are, contrary to a bizarrely popular belief, generally welcomed - or at least as welcome as any others in science. Recent developments compel me to add just a few extra thoughts.

For my part, I see no evidence whatever of any special bias against any new ideas just because they're new. Instead, a lack of support for an idea generally just means that the evidence for that idea is lousy. That some ideas do occasionally rise to the top simply reflects the changing evidence, while survivorship bias means we quickly forget the hordes of other (often downright crazy) theories that had to be rejected.

It's a similar story with scientific methodologies. Inasmuch as it exists at all, which I rather doubt, the "replication crisis" is at worst due to a poor understanding of statistics and lack of rigour. This is entirely normal scientific practise : we find a problem and learn how to avoid it. That's exactly what we should be doing. If we take a more retroactive view, we'd quickly find ourselves hurling abuse at all previous experiments for not meeting today's improved understanding of how to achieve properly robust results. And yelling at Renaissance thinkers for not putting error bars on their plots just seems a bit, well, pointless to me.

The thing is, the process of refining cutting-edge practises always looks messy because cutting-edge practises are the least understood by definition. That mistakes are made is a completely normal, usually unavoidable part of the process. It's not worth panicking about because there's absolutely no way we can do anything about it, any more than a medieval peasant could have worried about the non-existence of cheap budget holidays or proper sanitation. Yes, mistakes are bad and should be corrected - and yes, sometimes the consequences can be extremely serious. But as the old meme goes :

Finding mistakes in the process is itself part of the scientific process. And guess what ? That means we're actually going to - shock, horror - find out that we've been making mistakes.

Which is, somewhat ironically, why I want to emphasise how much of a non-issue confirmation bias really is. When your very method rests upon criticising itself, paradigm shifts are inevitable. We might get stuck in a rut for a while, but nowhere near as long as if we took our techniques for granted and never bothered to update them at all.

What does this look like at the coalface of research ? Actually, sometimes it can be a pretty unpleasant experience. When you spend months checking a result is valid and exploring all the possible implications of it, write it up in a way that seems perfectly clear to you and your co-authors, only to have a referee decide (sometimes arbitrarily) that it's wrong, the self-correcting nature of the scientific process doesn't feel particularly nice. We often say that scientists like being wrong, and that's true - provisionally. There are definitely circumstances under which it's not much fun at all. Maybe at some point I'll try and generalise when being wrong feels rewarding and when it just plain sucks.

The tricky part is that with messy, front-line research, you never really know where it's going. So it can be hard to tell if the referee is just doing their job (they absolutely should attack things from multiple angles) or being over-zealous. Specifically, my current paper is now on its second referee, after we decided that the first was simply too inconsistent to try and reason with. I've previously said that this is in part due to the journal's lack of instructions as to what it is referee's actually do, but the second referee is also rather negative about the main results.

I'll describe the paper in more detail when it's finally published. The main thrust of it is that we found some short gas tails from galaxies in a region where we expect galaxies to be losing gas. This is hardly an Earth-shattering result. Detecting gas streams is something interesting enough to publish, but not something that would warrant a mass orgy. Finding a very small portion of the gas galaxies have lost isn't going to revolutionise physics. And yet to persuade two independent referees that these claimed detections are likely real is proving to be much more work than merely saying, "look, here's a picture", even though I personally think said picture is pretty damn compelling. Nor is this the first time I (or literally all of my colleagues) have had to try and convince someone of something which seemed to be bleedin' obvious.

For my part, I'm 100% convinced our results are correct. Some parts of the paper will change, but if the referee is a reasonable person then they'll become much more enthusiastic about our main findings on the next revision. We'll see.

What does this mean for the really big issues in contemporary physics - dark matter, dark energy, the standard model, that sort of thing ? It doesn't mean these paradigms are definitely correct, far from it. As I said, the self-criticising nature means these paradigms may well be overturned. Instead, what it means is that these paradigms are not, absolutely 100% certainly NO FUCKING WAY, enforced by confirmation bias. That is simply not a thing. If minor incremental advances are routinely subjected to such robust criticism, then the idea that the major findings have somehow evaded this onslaught is clearly utter bollocks. Trying to convince people of the bloomin' obvious may be (extremely !) annoying to deal with in day-to-day research, but the plus side is that we can be really, really, really, really sure the major findings aren't just because everyone likes to toe the line and hates novelty, or is feathering their own nest with all that lovely grant money or whatever other nonsense someone's on about.

Dark matter is just groupthink ? Total bollocks.
The standard model is full of holes and no-one wants to admit it ? Utter tripe.
Cosmology won't consider any alternatives to the Big Bang because of an obsession with publishing more and more papers ? Absolute rot, the lot of it.
That is not to say that individuals, and even individual institutions, can't be subject to all the usual human frailties that afflict us elsewhere : they can. But the idea that the whole global system enforces paradigms because it doesn't want to change or isn't seeing the problems with the models - that is pure fiction.

The reason paradigms change isn't because people suddenly decide they'd just got everything wrong. Real revolutions don't involve everyone apologising to people they'd once dismissed as cranks, because cranks overwhelmingly tend to be just, well, cranks. No, real change happens because of new evidence and discoveries. These could be new observational or experimental data, or the development of new statistical or mathematical techniques. They can take a while to become widely accepted and adopted, but that too is an unavoidable, desirable part of the process. You wouldn't really want to start off by flying passengers in prototype aircraft, after all.

So while it's absolutely normal to use techniques we expect are imperfect and will be improved, this is fundamentally different from the idea that there is widespread malpractice, i.e. knowingly using methods which are misleading in order to support a given result. I, for one, think that charge has no force to it at all. And while we should always seek to criticise current practise, what we should not do - unless there are quite exceptional circumstances - is pretend that this criticism is anything other than part and parcel of the normal scientific approach. Yes, it's extremely rewarding when this leads to major breakthroughs. The penalty we have to pay is that at the ground level it can often be tedious, boring, and genuinely annoying.

False Consensus

Of all the allegations made against mainstream science, the charge of "false consensus" is the one that's the most worrying. The idea is that we all want to agree with each other for fear of being seen as different, or worse, that we will lose research funding for being too unconventional.

A tale of two galaxies

Recently all the news in galaxy dynamics seems to have been dominated by a galaxy which looks to be free of the pesky "dark matter" that irritates people on the internet quite a lot. But we shouldn't forget that the same team (van Dokkum et al.), not so long ago, also found a galaxy with the opposite problem of having too much dark matter. And that's weird too - if a galaxy is really massive, it ought to be able to form plenty of stars. A galaxy which doesn't do that is potentially a major headache for galaxy evolutionary theories, one which no-one was expecting and we don't know how to solve.

While both objects have been mired in controversy, the dark matter deficient galaxy (NGC 1052-DF2) has stood up well to repeated scrutiny. Or, to be very strictly accurate, its measured dynamics are quite definitely different to other galaxies of comparable size and stellar mass. The stars in the Milky Way are rotating around the centre of the disc at ~200 km/s, so fast that without an enormous amount of dark matter to bind them together they ought to quickly be flung out into intergalactic space. Those of the galaxy without dark matter are sluggish, lazily milling about in random directions at a mere 10 km/s, like a swarm of partially anaesthetised bees. There's no need for this object to have any dark matter at all - indeed, if it did, the stars ought to be moving an awful lot faster. It's a simple relation : Mass => acceleration => speed.

On the other hand, the stars in Dragonfly 44, which seems to have too much dark matter, were previously estimated to be moving at about 47 km/s. You might think that implies it should have some dark matter, but quite a bit less than the Milky Way and certainly not an unusually high amount as was claimed. And you'd be right. Well, sort of.

When it comes to estimating total mass you need size as well as velocity dispersion, and "ultra diffuse galaxies" like these are very difficult to measure in their outermost regions (actually, exactly how far they really extend is a bit of an under-discussed issue in my opinion). But the main point here is that you should try and make a fair comparison : i.e. see how much dark matter they have at a fixed,well-defined point, rather than extrapolating to much larger distances to get the grand total. Dragonfly 44 might indeed have a huge total mass, but this relies on making an extrapolation to distances far beyond the stars we can actually see and measure. As I pointed out when it was discovered, the authors were very clear about that in their paper, but neglected this in the press release. Ho hum.

Anyway, if you do restrict it to the range where you have direct measurements, Dragonfly 44 does seem to have more dark matter than the Milky Way at least. Or perhaps not. This paper revisits the galaxy with new observations, and they find that this might not be the case. Their previous analysis looks to have overestimated the velocity dispersion, and the new value drops to about 33 km/s. That seems to be consistent with the original data : it was an mistake in the analysis that was responsible for the over-estimate.

It now looks far less likely that this galaxy has an unusual amount of dark matter, though it does depend on what comparisons you use. If you compare it based on its stellar density then it looks a bit unusual, but not much. If you use a simpler fixed radius then it looks completely normal : it's still heavily dominated by dark matter, but not to an unusual degree compared to other galaxies with similar stellar properties. So it probably doesn't have a total dark matter content as great as the Milky Way, as was previously claimed. It's more likely to be just another dwarf, albeit a fairly large one.

But wait ! This doesn't mean the galaxy isn't interesting. The most popular interpretation of UDGs is that they are "huge dwarfs" - that is, they have total masses the same as dwarf galaxies but are rotating much more quickly and hence are more extended. In that view, UDGs are just part of the normal galaxy population - they just have rather extreme properties. The alternative is that UDGs are massive galaxies that somehow fail to form stars. That now looks less likely for Dragonfly 44... but it shows no signs of rotating - its stellar motions are completely random. Which means the fast-spinning dwarf theory doesn't seem to work either ! We might be back to square one as far as their formation goes.

And the same analysis showing that Dragonfly 44 has a normal amount of dark matter for its size also reinforces the fact that NGC 1052-DF2 apparently has no dark matter at all. What's especially strange is that these objects are similar in terms of stellar mass and size. So the central problem remains : why do some galaxies apparently have very different amounts of dark matter and/or dynamical properties ?

It also potentially poses a problem for alternatives to dark matter that replace it with modified gravity. Since the two galaxies have very similar distributions of stars, they ought to show almost identical motions, but they clearly don't. Although it's not well understood, for the dark matter paradigm it's at least possible in principle to separate stars and dark matter (this can even be demonstrated for tidal dwarf galaxies). So we can at least conceive of galaxies with similar structural parameters but different dynamics in the dark matter paradigm, even if we don't fully understand them. But you can't do the same trick if you don't have any dark matter. If dynamics are due to baryonic mass distribution, then all objects with the same baryonic mass distribution ought to have the same dynamics. And they don't.

Checkmate, modified gravity ? Not quite. The major caveat is where the galaxy lives. NGC 1052-DF2 is in a small group, whereas Dragonfly 44 is in a big cluster. While Dragonfly 44 looks to be stable and in equilibrium (it doesn't show any signs of disturbance), in some modified gravity theories the presence of external masses can change dynamics considerably, as has been suggested is the case for NGC 1052-DF2. So the different environments may pose a problem for comparing dark matter and modified gravity predictions, at least if we want to make a direct comparison of these two particular galaxies. That said, a compelling case has been made for two dwarf galaxies in the same environment that do appear to favour the dark matter paradigm.

While these new observations, then, do make the exciting "failed giant galaxies" scenario less likely, they doesn't mean that UDGs are now boring, or that the results are just too uncertain and unclear to tell us anything interesting. Indeed, they show that the leading theory of their formation is still highly problematic. And they also open the door a little further to using UDGs as a way to test different theories of gravity. I'd say they remain as exciting and mysterious as ever.

Spatially-resolved stellar kinematics of the ultra diffuse galaxy Dragonfly 44. I. Observations, kinematics, and cold dark matter halo fits

We present spatially-resolved stellar kinematics of the well-studied ultra diffuse galaxy (UDG) Dragonfly 44, as determined from 25.3 hrs of observations with the Keck Cosmic Web Imager. The luminosity-weighted dispersion within the half-light radius is $σ_{1/2}=33^{+3}_{-3}$ km/s.

Meet the monster in Virgo

They did it !

We now have, as you can see, the first image of the event horizon of a black hole, a place of frozen time and gravity gone mad. The black hole at the heart of M87 has a mass billions of times greater than the Sun and measures 40 billion kilometres across - more than twice the distance from here to the Voyager 1 spacecraft. Fall inside this heart of utmost darkness and you enter an inescapable realm where space and time become interchangeable. At the very centre lurks the singularity, a point of near-infinite density where the known laws of physics fail. Nothing but death awaits you here. All you'll hear are the trapped screams of scientists who went mad at the sheer baffling complexity of the whole thing.

Apparently it would look like this.

But enough of that - I'll not feign expertise in general relativity, it's hellishly complicated and there are much more knowledgeable people who can explain the theoretical stuff. Instead I'll just say a few things about the observations themselves.

What's remarkable is just how close the real image looks to the predictions made years beforehand :

Clearly those relativity experts know what they're doing. The main difference seems to be that the real image has two bright spots while the simulation has just the one. My more expert colleagues (we were lucky enough to get a sneak peek at the real image) aren't sure why this is, so that might be interesting.

You might be wondering, though, why this image doesn't look as nice as previous models we've seen. Any why did they look at the black hole in M87 when the one in our own Milky Way is much closer ?

The second question can be partly answered by Red Dwarf :

Which is only slightly inaccurate. The black hole at the centre of the Milky Way is extremely inactive, meaning there's not much material falling in for it to devour. So it essentially is a case of black on black (or nearly so). The one in M87, however, is much more active, with huge jets of material extending well beyond the galaxy itself. That means that there's something for the hole itself to be silhouetted against.

M87 and its jet.

And the M87 black hole is so much larger than the one in the Milky Way that it compensates for the greater distance - they're about the same size on the sky. But the larger size of the M87 hole means that material takes a lot longer to orbit it than the much smaller one in the Milky Way, making observations much easier. The one in the Milky Way needs something more like high speed photography. We might still be able to get an image of it, but we'll have to wait longer for that one.

So why doesn't this first image look all that much like the more spectacular renderings we've been seeing ? For example the first attempt at visualisation was, despite the claims made about the movie Interstellar, done as far back as the 1970's :

With more modern simulations giving much more detailed predictions :

Those models look at optical wavelengths, but the Event Horizon Telescope operates over a broad range (continuum) sub-mm wavelengths. That means we can't see redshift effects causing one side to appear bluer than the other since we're not getting the precise frequency information. We couldn't get the same kind of optical colours anyway since it uses a very different wavelength of light. The reason it does this is that getting even the fuzzy image that we've got requires extreme resolution, equivalent, they say, to seeing a credit card on the Moon. For that we need an interferometer, a set of telescopes linked together to provide higher resolution.

Interferometry is something of a dark art in astronomy. The basic requirement is that you know the precise distance between each antenna. That means you know how much longer it takes light to reach one antenna than the other, and through some truly horrendous mathematics it's possible to reconstruct an image with much higher resolution than for a single reflector. The shorter the wavelength, the higher the accuracy you need, making this very difficult at optical wavelengths but easier in the sub-mm bands.

While a single-dish radio or sub-mm telescope works basically like a camera, measuring the brightness at many different points in space in order to construct an image, an interferometer is a much more subtle and vicious piece of work. The zeroth-order description is that you can combine telescopes and get an image of resolution equivalent to a gigantic single-dish telescope, one as big as the separation between the dishes (the baseline).

This is true, but it misses a lot of important additional subtleties. First, the interferometer won't be anywhere near as sensitive as an equivalent single-dish telescope. The most obvious explanation for this is that you don't have as much collecting area as a single-dish - you'd need to completely cover the ground between the dishes to do that, whereas in practice you usually only have a handful of receivers (otherwise you might as well just build that giant single-dish). But there's another, far less intuitive reason, one that only emerges from the mathematics behind how interferometry works.

With a single dish, effectively you have a single aperture of finite size through which you receive light. It sounds odd to think of a reflecting surface as an opening, but that's basically what it is. There are no gaps in the single dish aperture, so wavelengths of all scales can enter unimpeded (unless they're so large they can't, in effect, fit through the hole). An interferometer is not like that. It's like comparing the hole blasted in a wall by a single cannonball to those made by some machine-gun fire.

The simplest way of describing the effect of this is that there's also a lower limit on the resolution of the telescope. The longer the baselines, the higher the resolution and the sharper your images get. But if you don't have shorter baselines as well, you won't be able to see anything that's diffuse on larger scales - that material gets resolved out; you could see the fine details but not the big stuff. A photon of wavelength 10cm is not going to have any problems fitting through a hole 1m across, but clearly trying to squeeze it through two holes each 1mm across but separated by 1m is a very different prospect. It's not quite a simple as this in reality, but it gets the idea across.

So, while the upper limit on resolution is given by the longest baseline, the lower limit is given by the shortest baseline. This means the overall quality of the image is much more complicated than just the maximum separation of the antennas : maximum resolution and sensitivity are nice, but they aren't the whole story.

You don't have to worry about this for single dish telescopes, because remember they're like having one great big hole instead of lots of little ones. Consequently, while the Very Large Array has a collecting area about one fifth as much as Arecibo, and a resolution at least four times better, its sensitivity to diffuse gas is almost a thousand times worse.

To some extent you can compensate for this. The most obvious way is to observe for longer and collect more photons. That helps, but because sensitivity in that regard scales with the square root of the integration time, if you want to double your sensitivity you have to observe for four times longer. That's bad enough for single dishes, but the situation is more complicated for interferometers. It's the number of different length baselines that determine your sensitivity to structures of different angular scales, not how many photons you collect. These are limits you can never quite overcome.

You can make things better though, because while you're observing the world turns. What that means is that the configuration of the antennas changes from the point of view of the target. The longer you observe, the greater the change, and the more of the "uv plane" you fill in. That means you get much more even sensitivity to information on different scales, getting you a cleaner and more accurate image. It's a bit like looking through a sheet of cardboard full of holes - the more you rotate it, the more information you get about what's behind.

Instead of seeing a series of points, over time the target sees a series of tracks that sample the different baselines much more fully. These tracks vary depending on the configuration of the dishes and the position of the source on the sky.

Since the EHT needs as high a resolution as possible, it depends on telescopes which are as far apart as possible and so can't see the target for very long before the rotation of the Earth takes the source out of view. Add to that that it isn't a single dedicated facility but consists of many different institutes, each of which have different priorities and maintenance requirements - and also the not inconsiderable problem of weather concerns, which have to be suitable at every location at the same time. That's why getting this image took so long.

EHT uv tracks.

The crucial thing about interferometers is that unlike single dishes, they do not make direct measurements of their targets. They measure incoming light intensity, but that's not the same thing. Instead, the image has to be reconstructed from the data - and it doesn't give you a unique solution. For example, you can choose to weight the contributions form the longer baselines more strongly and get higher resolution, or prefer the shorter baselines and get better sensitivity - both are valid. There are of course limits to what you can legitimately do with the data, but a lot of choices are subjective and it's often said to be more of an art than a science. It's a far more difficult procedure than using a single dish - there is indeed no such thing as a free lunch.

That's where the "looking through holes" analogy breaks down. If you were looking through holes in a wall, it wouldn't matter if you looked through them one at a time and combined your measurements later. With interferometers it's crucial that you get the same data at the same time for every antenna - you can't just bang your data together and get a nice image out. Again, you get a reconstruction, not a measurement.

The good news is that means the image can only get better with time. The more uv coverage they have, the better the spatial sampling and the more uniform the sensitivity will be to different scales, and the closer the image will get to those detailed simulations. The bad news is that we'll all have to be patient. The interferometry behind the EHT is delivering astonishing results, but there is, as always, a high price to pay for that.