Get Those Mother**ing Satellites Off My Mother**ing Plane

It's been a good long while since I looked at any planes of satellites papers, so let's see what's new in this controversial arena.

For those not in the know, the idea is that there are significant numbers of galaxies whose small satellite companions orbit around them in thin planes. This is not a natural prediction of cosmological models, which show that they should orbit them in spheroidal clouds instead. So this has been seized on as a major challenge to the standard model and even the dark matter paradigm itself.

Now the plane of satellites around our own Milky Way is really very clear and unarguable. But as I go into at great length here, claims for other such planes are not in the least bit convincing, and I tend to view the whole field as awash with some bloody daft statistical biases from people who really should know better. Honestly I think it's just (ahem) plain silly. A very short recent summary that uses a lot of the same arguments that I do can be found here.

But today I turn my attention to this more substantial offering. This paper concentrates only on the plane around the Milky Way. As I've said, this feature is very interesting. I don't think it necessarily means that all of science is wrong, but I'd like to know how it formed all the same.

In this paper the authors use one of the big new all-singing all-dancing cosmological simulations, within which they try and look for Milky Way analogues that have planes. They impose limits on the mass of the host galaxy that are fairly generous, but they have a rather strict policy for considering the 14 brightest satellites of the galaxy. Together with the mass of the satellites, this ensures a direct like-for-like comparison with the observations. At this mass range the observations should be complete (meaning they haven't missed any) and the observations have sufficient resolution to simulate them accurately.

Actually, while their mass range for the host galaxy is quite large, I suspect their constraints on the satellites are if anything too strict. 14 is a pretty arbitrary number, really, and they only report on one single Milky Way analogue that has a plane in agreement with the observations. This bit is my only gripe with the paper : they state their selection criteria for the host galaxy very clearly, but they don't actually state their criteria for a match regarding the plane; my guess is that if they relaxed these parameters a bit, they might find considerably more planes. There is no need, after all, to insist on a perfect match of all parameters.

Anyway, of the 548 halos of the correct mass in their simulations, 404 are as isolated as the Milky Way, and of those 231 have the correct number of satellites. Only 1 of these has a compatible plane (the above caveats notwithstanding), but 1 in 200 is already quite a bit higher than some other estimates. Remarkably, this galaxy also has a very similar large-scale environment to the Milky Way, with its nearest massive neighbours matching the properties of Andromeda and Centaurus A pretty well. These environmental conditions weren't used as a selection criteria at all.

The plane itself is also very similar to that observed. Not only does it have a similar geometry but it's also rotating, and at a very high inclination angle with respect to the stellar disc of the host galaxy. This is really an excellent match; even if the mass of the Milky Way analogue is a bit low, it's still within observational constraints on the Milky Way. To match this closely on so many parameters at a rate of 1 in 200 is very impressive result indeed.

Now one of the frustrating aspects of many anti-standard-model papers is that they search simulations for analogues of the observed planes, find some, and then only compute the frequency at which they're found and nothing else. Observational planes, they say, are very common, whereas simulated planes are very rare. They conclude that this means there's a conflict between the standard model and observations. The problem here -  well, one of them - is that they seldom if ever attempt to examine how those few planes that they do find actually form in their simulations... and this can make a world of difference. 

What I mean by this is that saying that they're rare overall in the simulations might be correct but potentially irrelevant. For example, giant redwoods are very rare among trees, but if you walk through Giant Sequoia National Park you shouldn't be amazed that if you've found far more than random chance would suggest. The approach of those claiming the planes challenge the standard model is in essence entirely statistical, neglecting the physical processes at work, treating galaxies are random when in fact they're anything but.

Here the authors find that the plane results from two mechanisms. First, the satellites infall along large-scale filaments, preferentially leading to the formation of elongated structures. Second, while most of the satellites here are indeed orbiting around the galaxy, three are just coincidences : their true 3D velocities will take them in quite different directions, so it just so happens that at one particular moment, the plane appears to have more members than it really does. It's partly "real", but partly transient.

At this point I want to raise another issue. Some years back, having a protracted email discussion with a rather strongly pro-plane group, one response about the infall of filaments was rather brief :

"I had already talked to Libeskind, and he never even wanted to suggest that the filaments are related to the VPOS. They are far too thick anyway."

Well, Libeskind is one of the three authors of this paper, and not only do they here explicitly state that the filaments are part of the formation mechanism, but they several times cite his earlier papers as claiming that as well. The above quote is thus demonstrably not correct. You can see why these wholly erroneous claims tend to annoy me quite a lot.

Is this the last word on the matter ? No, though it probably should be. If planes really were as common as some suggest, with practically every nearby massive galaxy having a planar system, we'd likely have a big problem. The authors here are careful to state that their findings don't say anything about these other systems, but in my view, none of these other systems are even remotely comparable to the Milky Way system and really aren't worth bothering with. 

In short, models do predict planes. Not many, but more than some claim, and those which form seem to be in very good agreement with the observations both on large and small scales. By showing that there are physical processes leading to the formation of planes, this is a strong rebuttal of claims that planes pose a serious challenge to the standard model. I remain convinced that this, like many claims against the standard model, is a non-issue.

AI-Assisted Astronomy ?

Yesterday I decided to stop feeding the chatbot weird premises for crossover stories and try and use it for research. Not actual research, you understand, just a test. I did a similar exercise recently to see if a bot could properly fulfil its advertised ability to summarise papers and found it badly wanting, so my expectations were set pretty low. For this one I wanted instead to discuss something explicitly similar to my own research, so I wouldn't have to check all the answers because I pretty much already know them by heart. This was prompted by idle curiosity and learning that Bing AI is powered by GPT4, which is supposed to be a substantial improvement in terms of accuracy.

1) ChatGPT : Up to its old tricks again

First, I had a protracted discussion with ChatGPT about the possible nature of an extragalactic hydrogen cloud. I gave it some basic properties : line width, radius, distance, wavelength of detection. Then I asked it to speculate about its possible origins. Overall, it did pretty well at this, suggesting it might have been stripped from another galaxy or a primordial object that hadn't formed stars. I had to push it just a little to get it to estimate the dynamical mass and realise the key point that the object should be dark matter dominated, but it came up with results which were decently accurate. It came up with a genuinely good list of the other sources of motion in such a cloud besides rotation, noting that these were likely to be small in comparison, hence the need for dark matter.

At bit more prompting and it got a decent estimate for how long such a cloud could survive. It came up with a couple of correct examples of similar known objects too. I moved on to ask it about whether this high dynamical mass could be the result of stars and maybe the SDSS just wasn't sensitive enough to detect them. It produced a decent order-of-magnitude formulae to estimate this, but then it started to break down. It kept giving ever-more inconsistent numbers (I'm actually surprised it made it this far, since this is the free version which doesn't do actual calculations at all). When its mistakes were pointed out, it was pot luck as to whether its revised response would be any better or not. Still, its basic method for estimating the detectability of the stellar content, though crude, is something genuinely useful that I hadn't thought of before. And its list of suggestions for further research to help properly nail-down the cloud was absolutely 100% spot on.

After that it seriously degenerated when I asked it for a summary suitable for an academic paper. It started inventing all kinds of extraneous details, even deciding to give the cloud a plausible-sounding catalogue name, contradicting itself with regards to numbers, and deciding that the cloud had been detected in an optical survey despite explicitly being optically undetected. It even included irrelevant references for some reason. All in all, this part of the test was generally just unhelpful garbage. This was surprising and disappointing, because this is the sort of thing I'd expect ChatGPT to be good at.

In summary, it provided some useful ideas even at the expert level, but its specific numbers were, totally unsurprisingly, not at all reliable, and only when I prompted it did it admit it wasn't doing any calculations - something it absolutely should have been up-front about. I like that it's useful for exploring new ideas, but while this is beneficial, it's hardly revolutionary.

2) Bing Chat : A glimpse of the future or a freakishly coincidental hallucination ?

This one was truly strange, to the extent I almost wonder if I dreamed the whole thing. Bing AI is annoying for two huge reasons (besides, well, being Bing). First, you have to use Edge to run it (FFS, let me choose my own damn browser), and second because it gives you no easy way to save your history. It's either old-school copy+paste or nothing. And since I was on mobile, when I closed the app for a moment, all was lost. This is completely stupid.

At first it didn't look hopeful at all. In "balanced" mode it straight-up refused to give me anything useful in the way of an answer, shutting down the conversation completely so that you can't enter any more text, leaving you with no option but to start over. Why in the world anyone thought that giving it this "ability" was a good idea, I don't know. Again, this is stupid and frustrating, even for a preview tool.

And then... something truly amazing happened. In "precise" mode, it... did exactly what I wanted. True, it needed a little prodding, as ChatGPT did. But it also offered explanations that ChatGPT hadn't considered. It came up with citations as clickable links. It gave the formulae and, impressively, its numbers were absolutely self-consistent. It never messed up the masses of different components as ChatGPT did.

For an estimate of the optical detectability of the cloud it did (or at least appeared to do) something much more sophisticated than ChatGPT. It initially even said this was impossible without running a full population synthesis model with Staburst99, which it can't do and I'm not going to either. Then I told it to make a simpler estimate, and it required the stellar distribution (I told it to assume a standard IMF) and composition (I told it to assume solar metallicity). It then estimated the luminosity, assuming all the massing mass was stellar. It gave a value in Watts (I presume it defaults to SI units, which is not unreasonable) but had no problem converting into the more familiar solar luminosities and then apparent and absolute magnitudes.

I did not have the opportunity to check those numbers, but I do know they were perfectly credible. I'd really have love to scrutinise its calculations minutely, but I was sadly denied this opportunity. But they were certainly close to what I was expecting. I don't know if Bing AI has access to some mathematical tools (like the Wolfram Alpha plugin for paid versions of ChatGPT), but it certainly seemed like it was doing calculations and not just generating numbers statistically.

Bing AI stressed that these numbers were estimates and subject to a lot of uncertainty, something ChatGPT didn't do. I pushed it further, asking if it would be possible to alter the metallicity and/or IMF to render the galaxy undetectable, as with these simple assumptions the galaxy should be well above the SDSS sensitivity limit as I was expecting. It said yes, but when I asked to to check how, for example, the metallicity needed to do this compared to known galaxies, it found that the result was incompatible with known observations and gave me a reference to extreme metallicity values. Similarly for the IMF.

If correct, this is incredibly useful. A lot of tedious calculations and trawling papers... all gone, replaced with a quite natural style of conversation that gets right to the point.

And then all was lost forever. Worse, this morning Bing refuses to do any calculations at all (except in "creative" mode, which produces results which are wrong by many orders of magnitude). It won't give me the stellar mass estimate or even the dynamical mass. It comes up with formulae but its responses are partly garbled as it's very blatantly just scraping together bits of relevant text from different sources (it's at least honest about this and provides the links), and it point-blank refuses to admit it can do calculations at all. It even suggested I may have confused it with another chatbot. And to be fair, the experience is like using a different AI altogether, as though some bloke called Dave crawled around inside it and starting pulling out vital circuitry.

Well, I don't know what to make of all this. ChatGPT did better than I expected, and if that Wolfram Alpha plugin works as advertised... this could be extremely powerful. But as it stands, it's useful for discussions and ideas, but not actual analysis, and somewhat surprisingly, not for constructing replacement text either .

Bing, on the other hand... it might have been total garbage for all I know, that just happened to get things about right. But if (and I do stress "if" very strongly !) this is what using a language-model AI coupled with a genuine mathematical calculator is like, then it's transformative. I want this. Anyone saying it isn't useful is simply mad and wrong.