Science as a social network

""Rather than being heroic geniuses, Darwin and Wallace were in the same ‘cultural milieu’, both reading the same books and both travelling to biologically diverse island environments. While individual abilities vary, collective brains make each brain within it cleverer. The theory helps explain why there have been dramatic increases in IQ test scores over time.

Individuals copy other successful individuals – eating the foods they ate or hunting with the tools they used, for example – to become successful themselves without necessarily understanding why. Dr Muthukrishna added: “The processes of cumulative cultural evolution allow technologies and techniques to emerge, which no single individual could create on their own – because human brains, in isolation, aren’t actually all that smart."

You could probably say the same about most geniuses. Yes, there have been some people who have made stunning breakthroughs - often in large part thanks to their extreme intelligence and/or dedication. But it almost never happens only because of their own efforts. More usually it's because they were standing not on the shoulders of giants but on people of about average height who spent years working on apparently insignificant details.

http://www.telegraph.co.uk/news/science/12176959/Charles-Darwin-was-no-heroic-genius-say-scientists.html

Colour-corrected galaxies.

Colour-corrected galaxies selected from the ALFALFA HI catalogue, shown using SDSS optical data. The over-yellowness turned out to be because of changes in the way the SDSS assign colours and wasn't anything I'd done wrong. The new SDSS data fixes some artifacts but I preferred the old colour scheme. Easy enough to fix though !

The 3D version is looking nice, but I need to play with the camera offsets to get the best balance between a sense of depth and making my eyes go funny because of the multiple images.

22,935 galaxies

All shown at their true position in space with none of this crappy, "oh sod it let's just use a few hundred template images" whining you get from those losers at the SDSS. No sir !

I think something is off with the colours though. Everything seems too yellow, especially compared to the last time I made one of these. Which is why this is just a teaser gif. More will be forthcoming when I'm happy I've sorted everything out.

Original with explanation can be found here.

What a difference a display range makes

For those not following this regularly, I'm simulating what would happen if a dark galaxy (gas disc with dark matter but no stars) fell into a galaxy cluster. Last time we saw that very little happens to most of the discs, most of the time.

But that's not the whole story. Most of the discs do survive just fine, but some material get ripped off by other galaxies passing by. Some of this material gets pulled out into extremely long features. This gif is exactly the same data set as the last one, just with each galaxy shown at a much larger field of view and with the low density material enhanced with my shiny new method. That's important because although these streams are enormous - some of them are more than 3 million light years long - they have incredibly low density. If they exist in reality, there's no way to detect them.

But looking on this larger scale does give some clues as to features we could detect. For example, the famous VIRGOHI21 "dark galaxy" has been shown to possibly just be some sort of unusual tidal debris. VIRGOHI21 is a dense blob of hydrogen in the middle of a long stream originating from a spiral galaxy. Previously, it was thought impossible to produce long one-sided tails like this through tidal interactions until one now famous paper by Duc & Bouranud showed it was possible.

What this quick and dirty little analysis shows is that such features are indeed possible but they are very rare and not long-lived. Unfortunately, whenever any starless clouds of hydrogen are discovered there's a tendency to say, "it could be tidal debris, look at the Duc paper". However the Duc paper only shows that such things are possible - not that they are likely. From this small effort it appears that the setup has to be quite precise to produce something even close to VIRGOHI21. That other similar features exist may indicate that hand-waving about "tidal debris" is not an adequate explanation.

On the other hand this model doesn't use a very realistic galaxy - it's far too small and the density profile is all wrong. So this is just a preliminary examination, pointing the way to a proper study and indicating that it should be worthwhile. Fortunately, that's now a matter of some minor code adjustments and pressing "go". With any luck I'll soon have a nice controversial result to publish.

Galaxies Gone Wild On A Saturday Night

Galaxies Gone Wild On A Saturday Night

This is satisfying to watch. 25 gas discs embedded in dark matter halos being thrown into the chaotic environment that is a galaxy cluster. And they all survive pretty much intact !

All of the discs have the same initial conditions, though the movie for each one uses slightly different display parameters (brightness, field of view, etc.) since some of them are more disrupted than others. Each movie tracks the disc through the cluster. The harassing galaxies are not shown since the interaction times are very short. I actually ran 27 simulations, but 27 isn't a square number.

The hypothesis I'm testing is that dark galaxies could survive better than the "tidal debris" explanation that's used for several otherwise mysterious starless clouds of hydrogen seen in the Virgo cluster. Other simulations I've been running strongly suggest that producing clouds similar to those observed by removing gas from galaxies is almost impossible. Thus far, the dark galaxies hypothesis is coming up trumps.

There's still a huge amount of work to do here though. The next step, which will probably complete all the data I need for a publication, will be to try using smaller clouds with less dark matter. The ones shown here are using the largest possible sizes for the clouds allowed by observations, which also means the highest dark matter content.

A much more serious limitation of the current work is the lack of intracluster medium / ram pressure stripping. As the galaxies move through the hot gas inside the cluster itself, their gas gets removed. Combine that with the gravitational harassment and it might be possible to create something that looks like a dark galaxy but actually isn't. It's very hard to guess the result, but we have to take things one step at a time.

Animated SPH visualisation techniques

A few hours later and I've got this animated. Not as tricky or as slow as I thought it might be.

This is another disc falling through a galaxy cluster. Thanks to its massive dark matter halo it survives pretty much intact even after 5 billion years. It may look like it gets disrupted, and it does a little bit, but only about half its material gets removed. Observationally, that's not significant. Of course there would probably be triggering of star formation happening which I'm not modelling (although that would be possible), so possibly the dark galaxy wouldn't stay dark.

But the point of this is really the difference in visualisations. On the left each particle uses a fixed halo size. On the right the size varies according to the SPH kernel size, which accounts for how diffuse the material is. This completely avoids the problem of very dense regions which otherwise show up as "glowing orbs" (you can see one in the very centre on the right) which looks much larger than they actually are. Both the low density and high density material are shown equally clearly.

Ironically, the old version on the left is probably closer to what could be observationally detected. Sometimes, though, that's not the point - it's more useful to be able to examine all the structures formed in the simulation at all density levels. And it just looks nicer, too.

Improved SPH particle visualisation in Blender

Today's, "I'm waiting for my simulations to finish" side project : improve visualisation of particle data.

A long-standing issue I've had with using Blender's halos to display particles (raw data example shown on the left) is that when you have many halos in close proximity, the halo opacities sum up so the circular shape of the halos is revealed (I call this the "glowing orbs problem", middle panel). Normally you can't just lower the opacity so that the total is lower, because in other regions the particle density is much lower so that would make most particles invisible. Sometimes that's a good thing - maybe you don't care about the low-density stuff, or you want to emphasise the density contrast - but often you want to show everything that's going on and preserve the density contrast.

Fortunately (?) smoothed particle hydrodynamics is more complicated than just taking a bunch of particles with gravity and seeing what happens. Particle properties are smoothed over different kernel sizes, which depends on the particle density. So each particle is assigned a different "smoothing length", in this case chosen to contain 50 particles. The bottom line is that you can think of each particle as having a finite size, which can be used to set the halo size in Blender (right panel).

What that allows you to do is resolve very dense features and display them as the bright but small structures they really are, without losing the diffuse material. Lots of tweaking still to do to get this optimised, but it's clearly far better than using a fixed halo size.

Since in Blender creating 10,000 unique objects and materials is the equivalent of shouting rude words at your computer and questioning its parentage, I use the smoothing length to bin the halo size so there are only 100 unique objects, not 10,000 (but still 10,000 vertices, which Blender has no problem with). This is pretty fast for loading a single frame, not sure how well this will work for animations but I think it should be OK.

How not to build a galaxy.

And more importantly, how not to build a galaxy. Uniform discs of pure gas turn out to be almost impossible. Exponential discs - where there's a much greater density in the centre - work, but they don't match observations. By far and away the easiest solution is to add a gazillion tonnes of dark matter. That stabilises everything to the point where the disc becomes practically indestructible.

I also look at a claim by some mathematicians that you don't need dark matter because the gravitational field of a disc is very different to sphere. It turns out the field is very different - and this is often overlooked - but this cannot possibly explain why galaxies are rotating so fast. Although they do some mathematics that's way too complicated for me, I found a way to sidestep that and get the same results. The flaw in their argument is that they use a density profile that's completely ruled out by observations, and doesn't give that good a result anyway. They also neglect the fact that the gas is a fluid but actually you can't approximate this as a simply a collection of orbiting points - even if you use the prescribed formula that accounts for the gravity correctly, the thing tears itself apart in a wide variety of interesting ways.

Whatever its faults on other scales, dark matter really is the easiest solution here.

Placeholder post intended to be replaced with a better summary.

The academic system has its flaws, but let's not go nuts

I don't think it's impossible to get another Einstein in today's scientific environment, but it is difficult.

It's a tricky balance to get right. On the one hand, there is definitely too much pressure to publish. This results in relatively minor results being put on the same peer-reviewed pedestal as breakthrough discoveries, and a huge amount of time can be spend quibbling over minutia. The grant-based system of funding is even more dangerous since it reduces the amount of time doing useful science as much as possible. Postdoctoral fellowships (where you can do more or less whatever you want without worrying about funding) are not extinct, but they are far too rare. They should be the norm, not the exception.

On the other hand, science has become more complicated. We have access to a wealth of observational and theoretical data that we just didn't have access to before. So perhaps this rather more incremental approach is inevitable - there are many cases where the data leads naturally to contradictory conclusions. Resolving the paradoxes is not easy.

It's not a simple situation of either "academia is bad" or "academia is awful". It's more complicated than that.
http://astrorhysy.blogspot.cz/2015/11/when-worlds-collide-science-in-society.html


http://www.theguardian.com/commentisfree/2016/feb/12/einstein-gravitational-waves-physics

The Disc That Just Won't Die

OK, so I am now in the stage of doing science runs. Here's my stable gas disc suddenly finding itself in the very centre of a galaxy cluster. Unfortunately, the 400 massive galaxies surrounding it are difficult to visualise because they move so quickly. You'll just have to use your imagination. I also decided not to show the dark matter because it gets in the way and doesn't do anything.

But after some initial fireworks, the disc doesn't do a lot either. Initially the disc happens to be near the centre of one of the massive galaxies and it's moving away from it, so it experiences massive tidal forces. After that though, it never has such a close encounter and things settle down. About half of the mass that was originally in the disc stays in the disc. The thing is practically indestructible. Mind you, using this much dark matter is right on the limit of what the observations say it could contain. It could be smaller and need far less.

Next week I'll run more of these starting at different points within the cluster to see how statistically likely it is that such a dark galaxy could survive. Based on previous experiments I don't expect there to be much difference, but it's possible that sometimes the galaxy will experience much more violent encounters.

Of course, this is quite crude and missing a lot of physics like star formation and the gas inside the cluster itself. But if the quest to get a stable disc shows anything, it's that it pays to use a slow, steady, incremental approach.

No, this post is not about gravitational waves. Keep scrolling.

Same galaxy disc simulation as before but now with the dark matter shown in pink because Andres Soolo thinks that would be appropriate for some reason. Also, the gas temperature is more realistic now. Since it's hotter the disc initially expands in the vertical direction because it hasn't been setup to be in equilibrium, but after that it settles now quite nicely. I could probably fix that just by making the disc a bit thicker to begin with.

This temperature is a bit on the low side - 1500 K is about as cold as HI can get before it becomes molecular. At a more realistic 5000 K it still basically works, though the disc starts to evaporate after a few billion years. For the final runs I'll probably try both temperatures.

I'm slowly iterating down to re-creating the observational data. The current sim running has the correct velocity profile. The next step will be to shrink it down to the correct radius. Sometimes funny things can happen when making things smaller, hence I'm being very cautious and proceeding incrementally.

Simulations have to use a vastly smaller number of particles than in reality - in this case 10,000 gas particles and 10,000 dark matter. That means every particle is vastly more massive than in reality, so the gravitational field isn't nearly as uniform as it should be. When particles get very close together they can experience extremely high accelerations and scatter off into the void. That can't happen in reality because the masses are so much smaller. Simulations deal with this by using a "softening length" - when particles get within a certain radius, the gravitational force reaches a maximum instead of increasing to infinity. So it's important to check that scattering isn't important and adjust the softening length if needed.

So far, so good.

Behold, the disc that doesn't do anything !

Well, not very much at any rate. It definitely does not explode or disintegrate. It does make some rather nice rings, but that's OK. I don't need (or expect) it to be perfectly stable - not exploding is all I ask.

This one is embedded in a dark matter halo. I don't have a script to convert the dark matter particles into a format I can display but I'll try and get one.

A number of failed attempts proceeded this one, even with the dark matter. I tried spherical clouds first, which worked well at low masses but not at high masses. With a spherical system the gas is supported by its temperature, which for a massive cloud has to be extremely high. That means an extremely high pressure, which I think was tending to blast things apart.

With a disc, the support comes from rotation much more than pressure. So the gas can be cool and is far less prone to expanding. I also used a lower mass of gas than in the previous runs. With a pure gas disc that would have meant the rotation speed would also have been lower, making it difficult to judge if it was really stable against rotation without doing a very long simulation. Anyway I need a higher rotation speed to match the observations.

With the dark matter, I don't need to worry about finding a sweet spot in terms of being hot enough to avoid collapse but not so hot it explodes. It's bound by the mass of the dark matter, not the gas - so if I make it hotter, it doesn't matter so much. A lot of instabilities caused by making it colder are also avoided - there's less differential rotation across the disc because the extra mass changes the rotation curve.

This isn't quite the disc I need - it needs a bit more dark matter, be a bit smaller, spin a bit faster, and the gas needs to be a bit hotter. But it's very close. With luck, by next week I should be able to chuck this into a galaxy cluster and watch the carnage (or lack thereof).

Stable Uniform Pure Gas Discs Are Not A Thing

At least that's what I've concluded. After much tweaking, this is the best I could come up with. It's sooo close... and yet so far. The disc manages to make about a third of a rotation and it looks good... and then it collapses.

The problem is that in order to prevent fragmentation of the disc it needs to be hot. But if it's too hot it simply expands to infinity. According to some quick and dirty calculations I did this afternoon, there's no sweet spot - if it's hot enough to avoid fragmentation, it's also hot enough to evaporate.

Still, it's close - there's just one big fragment produced, while the outer parts slowly evaporate. Only about 1/3rd of the original particles remain within their starting radius by the end of the simulation, while the core is many times denser than the original disc.

Maybe if I change the parameters enough from my original requirements I could make this work, but it's not worth the effort. Uniform pure gas discs aren't found in nature anyway. Tomorrow I'll try adding a dark matter halo. Having a lot of extra mass near the centre should stabilise things much more easily - the dark matter should be much less prone to fragmenting, and the extra mass will keep everything bound so the gas can be hotter. At least that's the theory...

The Saga Continues

For those who haven't seen the previous efforts, I am trying to simulate a uniform, galaxy-sized disc of gas. Don't ask why. So far I've been plagued by my nice neat discs insisting on turning into rings, even when I tell them not to.

I realised that I was setting the velocity of the particles in a fundamentally wrong way. The classic equation v = sqrt(GM/r) assumes the system is a sphere, but it categorically does not work for a disc. The circular velocity (that is, the speed needed to stay in a stable circular obrit) depends strongly on the geometry. This is actually fiendishly difficult to calculate for a uniform disc - this paper (http://arxiv.org/ftp/arxiv/papers/0903/0903.1962.pdf) does it, but I don't care for all that mathematics.

What I've done instead is side-step the issue by letting the simulation code measure all the accelerations for the first frame, then I set the velocities to compensate for the radial acceleration. This gives me a result very similar to the analytic result, which is nice, albeit with a lot of scatter for reasons I'm still trying to determine.

Anyway, you can see above that this gives a different result to the earlier attempts. It still fails, but in a new way ! :) This time there is (at first) very little outward movement from the particles close to the centre of the disc. However, it still fragments due to the random motions and positions of the particles. These asymmetries build up and soon the thing is no longer even a disc or a ring at all, and consequently it breaks up. If I want the disc to be truly stable, I'll have to find a combination of parameters whereby the disc won't fragment but the particles aren't moving at escape velocity either.

I suspect that uniform pure gas discs, unlike spheres, need a lot of fine-tuning to remain stable. Inside a hollow shell a particle will stay at rest. This isn't the case for thin rings - particles inside the ring will move toward it. So rings are unstable, whereas shells are not. This means that whenever a ring develops (which can easily happen due to the random motions of the particles) it tends to get worse over time.

Of course in reality galaxies also have stars and probably a lot of dark matter as well which helps keep the gas stable. Intriguingly, the paper that calculates the velocity profile for a uniform gas disc also claims that far less dark matter is needed to keep discs stable than is normally supposed. That's something I can probably check. For now, it's fun to watch, at least.

They seek them here, they seek them there...

The missing satellite problem, a.k.a. the dwarf galaxy problem, is that standard cosmology predicts far more small galaxies than are actually observed. Specifically the missing satellite problem is that there are far fewer dwarf galaxies found around larger galaxies like the Milky Way.

This paper, which was first submitted two years and has only now been accepted, explores another variation on that. It turns out that there are also huge numbers (about a factor of five) of missing galaxies even in the field - that is, the general population of galaxies that aren't in groups or clusters. So even isolated galaxies appear to be missing. This is significant because in a galaxy group the interactions between galaxies could be very significant, so one could wave one's hands and say, "gas physics is haaaaard !". After all, large-scale simulations use just the dark matter, not the gas because it's tricky to model. They have to use clever prescriptions to work out how much gas and how many stars each dark matter halo would contain.

Galaxy groups offer another complication : satellite galaxies seem to be orbiting in planes, not orbiting at random as the theory predicts. That could potentially offer a way out of the missing satellite problem : maybe the gas flows down filaments into the central galaxy, so satellite dark matter halos outside the filament never accrete much gas and never form stars. Maybe. But this simply doesn't work at all for isolated galaxies. There's no reason to think that so many isolated dark matter halos should avoid accreting gas.

Perhaps one could still get away with saying, "gas physics is haaaaard !" for isolated galaxies. When the first stars form, stellar winds and supernovae might blow the remaining gas out and prevent much star formation from ever happening. Most galaxies would then be very hard to detect. The authors of this paper say no, this can't work. Although this might be OK for very small galaxies, it shouldn't work for larger galaxies - which are also missing (the old, "too big to fail" problem).

It's also worth noting that the authors try and minimize the effects of the still poorly-understood baryonic (that is, gas) physics. They compare theory and observations not through brightness but by looking at how fast the galaxies are rotating. Although they still have to do some corrections for the gas and stars, they say this is far less significant than if you look for galaxies of a certain brightness. Essentially, how fast everything is rotating should always be a good indicator of the mass of the galaxy, whereas brightness can vary for many more complicated reasons.

Unfortunately since the paper was submitted so long ago, the authors don't comment on the recent discoveries of huge numbers of very faint galaxies in clusters. As far as I know we don't yet have good measurements of the kinematics (rotation) of these new galaxies. So possibly a lot of very faint galaxies still remain to be discovered, even ones which should be too big to avoid forming stars. Of course those galaxies are also problematic.

In short, this exposes another bloody great gap in our ignorance.
http://arxiv.org/abs/1405.4523

*If at first you don't succeed..."

...send an email to someone who knows what's going on.

[More of my misguided attempts to simulate a stable rotating galactic gas disc without dark matter]

Yesterday I realised that I was setting the xy velocity components incorrectly. I fixed that, but alas that wasn't enough to get me a nice stable gas disc. No matter what I alter I always end up with a ring.

On the left : my standard setup now setting the velocities correctly. Some of the material in the outer disc escapes, but worse, pretty much all of the inner material moves outwards. Since most of the outer disc is still there, the result is a dense ring that fragments into a couple of massive blobs. With most of the mass now in two blobs rather than a disc or a ring, the system becomes unbound and the two blobs fly off to have wacky adventures.

In the middle : the same disc but with no temperature. I thought that maybe it was the thermal motions of the gas that were throwing everything off, but apparently not because the same thing happens. The difference is that there's much more fragmentation because there's no pressure preventing local collapse. Possibly because of this the resulting ring of fragments is fairly stable, since everything remains sort of symmetric.

The third one shows a desperate attempt using a more massive disc without any hydrodynamic effects. Apparently fluid effects aren't responsible either because the ring still forms. This time there are five large blobs formed which slowly merge into two.

At this point I admit defeat and emailed an expert. Maybe there's a well-known result that uniform discs aren't stable, though I can't see why they should turn into rings. Oh well, at least the failures look nice.

The stable disc failures continue unabated

Today's main effort was to try and set up a stable rotating disc of gas. It failed miserably but at least it looks nice.

What happens is that the outer parts of the disc expand into infinity, while the inner parts simultaneously collapse. When the density gets too high, numerical errors in the simulation become significant and momentum is not conserved. That's why you see a few dense clumps suddenly moving on straight lines (most noticeably the big bright one in the center at the end). That's not too much of a problem provided I can figure out why it's collapsing in the first place.

My hypothesis is that it's because the thermal motions of the gas (3.5 km/s) are significant compared to the maximum rotation speed (12.2 km/s) so simply by random motions, the gas particles move into regions where their carefully determined initial rotation velocities are no longer stable. If I make the gas much more massive (so the rotation speed higher) or colder it should fix the problem.

However, why so much of the gas is able to escape I do not know. None of the particles should ever reach the escape velocity. Hmmm.

Discs aren't supposed to grow tentacles, but this one does

More fun with viewing galaxy simulations with particle trails.

This time the disc of gas has temperature, so it doesn't collapse. In fact it's hot enough that the gas expands and the thing slowly dissipates. Something went funky when I tried to restrict the particle selection though, so for completely unknown reasons this only shows the particles leaving the disc vertically. I have no idea why that happened but it looks nice. In actuality the particles expand in all directions.