With all this stuff going around about some angry scientists writing a letter to some other angry scientists about how their science isn't really science, I thought I'd take a look at the popular topic of falsification. Being able to disprove your theory is certainly a good thing. You never make a theory worse if it's possible to disprove it. But is it absolutely essential ? I argue, "no" - and we're already in a era when in insisting on the possibility of falsification does more harm than good, at least in astronomy. "That's all there is to it" is, alas, woefully inadequate.
A simplified example : galaxies in very dense regions tend to have smooth, elliptical shapes, while those in less dense regions tend to be spirals and irregulars. We know there are varying processes which can act to change a galaxy's shape, but which one dominates ? We don't even want to try to falsify which ones happen - because we know they all do - it's just a case of establishing which one has the biggest effect. The effects of the different mechanisms are so complex (and observational errors so large) it's possible we could make any of them work, with enough effort. So which method gives the results closest to reality with the least amount of tweaking ? That's the question we try to answer, which has little or nothing to do with falsifying anything.
Here's another example - a computer claims to have proved an obscure mathematical theorem but its proof is far too long for any human to ever read. By necessity, this proof must be based on logical deductions, but if it's too long to check then is it really a proof ? This isn't really all that novel either - throughout history, stupid people have stubbornly refused to accept the proofs that cleverer people have come up with. Does that mean that clever people aren't being scientific if they can't explain their ideas to the mentally deficient ? With science becoming increasingly complex and requiring increasing amounts of time to fully understand, this is a real problem. And if scientists don't even fully understand their results, well...
Really extreme proponents of falsification often tend to be those of the anti-science ilk. Geology, astronomy and anything else which involves deep time, they say, are not really sciences because we can't actually prove anything - no-one left records for billions of years ago for us to check, and we can't wait around to see how galaxies evolve. In a very strict sense, the evolutionary history of life on Earth and the behaviour of stars over cosmic time really can't be falsified.
Such a way of thinking has many parallels with conspiracy theories. It's not that everyone is lying, exactly, it's just that they are demanding impossibly high standards from the evidence which can never be met. By demanding ludicrously high levels of confidence, by refusing to make even the most basic assumptions and give the data some rudimentary level of trust, in short by refusing to even entertain hypothesis for the sake of it, they prevent themselves from learning anything. And they rarely say why they have such confidence in their own senses, which is bizarre given the complexities and many, many demonstrable fallibilities of the human brain.
https://astrorhysy.blogspot.com/2017/05/i-told-you-he-was-tricksy.html
Subscribe to:
Post Comments (Atom)
-
In the last batch of simulations, we dropped a long gas stream into the gravitational potential of a cluster to see if it would get torn... -
Another day, another paper on how exciting Ultra Diffuse Galaxies are. At first, these large, faint galaxies were just wholly unexpected, a...
-
Of course you can prove a negative. In one sense this can be the easiest thing in the world : your theory predicts something which doesn...
An interesting perspective, but I'd wonder about the utility of a hypothesis that can't be falsified.
ReplyDeleteThe possibility of falsifiability doesn't need to be immediately accessible for it to be there, and observation of natural "experiments" is just as good for the purpose as anything we can do in the lab.
So Astronomers can rest easy: it's a science.
Something that I believe has to be taken into consideration is the breadth of coverage that a particular controversial claim can exhibit, without reliance upon "settled science" claims to rule things out.
ReplyDeleteThis is what I am working on: I'm setting up a crowdsourcing platform that provides tools to help the public to track claims against recent observations for all of the scientific controversies.
What I've observed is that many claims that are today considered "settled" are not settled in the sense that the observations do not support them; they are settled in the sense that the larger scientific framework cannot accommodate them. I intend to show that this is an anti-pattern for science which treats assumptions and starting-point hypotheses as facts, and I'll use crowdsourcing to show it.
As an example, I recently spent 9 days documenting the Halton Arp debate, with the intent of presenting it as an example of what I think that crowdsourcing scientific controversies should look like.
For example, on this ...
Re: "galaxies in very dense regions tend to have smooth, elliptical shapes, while those in less dense regions tend to be spirals and irregulars ... "
Halton Arp's quasar ejection hypothesis could help quite a bit to explain this observation. He of course argued that quasars are ejected from the active centers of AGN's, with high redshift. The redshifts over time normalize as the quasar evolves into a galaxy that usually looks remarkably like its parent.
It turns out that the Arp debate can be broken up into at least a dozen (probably many more) sub-claims which can all be independently tracked against modern observations -- things like redshift periodicity, quasar clustering, quasar proper motion (which very surprisingly is actually a thing), etc. These are observations which support Arp's arguments, and in many cases are anomalous to mainstream theory/framework, but Arp's claims of course violate the Big Bang. And I would argue that this is the primary reason for why the debate is considered by experts to be "settled" -- for there are in fact numerous observations which support his claim.
What stands out about the Arp debate, in particular, is its "breadth of coverage". This aspect is missed by those who avoid tracking the debate. The number of sub-claims seems a bit larger than other controversies I've tracked, and I would argue that this should be considered a metric of sorts: When a lot of sub-claims can be established for a controversy, the controversy would appear to be more a more serious challenge to textbook theory, and we'd be wise to pay special attention to it.
I would propose that this sort of approach needs to happen in parallel with the more "normal science" approach which you are suggesting, Rhys. We need to actually set up tools which can facilitate paradigm change in the sciences. And the public-access crowdsourcing of scientific controversies is the correct approach here (assuming some moderation of which controversies are crowdsourced).
Daniel Taylor Well, as I say, I think a model can still be utilised even if it can't be falsified : if you can say which one is in better or worse agreement with the observations, then that has value. Relative comparisons can be made without necessarily falsifying anything.
ReplyDeleteI agree that falsifiability doesn't need to be immediate, but I'd say the timescale on which falsifiability can occur is relevant : if it takes so long that there's a plausible chance the human race will be extinct by the time you can test it, that becomes "never" for all intents and purposes. And in astronomy, there are some systems that would take us to long to reach that they would changed beyond all recognition by the time we arrived to take the precise measurements needed to falsify the model.
Natural experiments are indeed useful, but they have the major caveat that we can't manipulate them. They can give results which are in excellent agreement with the models (even in detail - there's a link in the link about that) but those models can still be fundamentally wrong.
My main point is that falsifiability is a gold standard, and great to have whenever possible - but not an absolute requisite. A good theory should have at least some aspects which are falsifiable, but it needn't be a strict requirement for every novel prediction.
Rhys Taylor agreement with observation is falsifiability. New observations that disagree with past observations, unlikely as they are, mean having to reevaluate existing theories.
ReplyDeleteI think a lot of people look at falsifiability as resulting in having to throw out existing theories and hypotheses, when the more common result is a tweak here or there.
Newton's Laws have been falsified many times, after all.
Daniel Taylor I'd say "falsified" does strongly imply that something has been completely disproved. "Testability" is maybe more of a useful concept, IMHO. Some things are indeed shown to be wholly false and no longer worth discussing (the flat Earth), others are shown to be false but still useful (Newtonian gravity), while those more at the forefront of research can be compared to each other but one can't really say if any model is definitely false or not. And then there are models still in their infancy, which can often be falsified very easily if you take them too literally, but require more development and exploration rather than less. Lots of shades of grey !
ReplyDeleteRhys Taylor true, I like using the term falsifiability rather than testability, because if you accept testability you are accepting the possibility that your model could be proven completely wrong.
ReplyDeleteIt might be a vanishingly small possibility of total falsification (Newtonian mechanics are still true enough for Earthbound use), but I think it's a healthy mindset.
Of course, as you note, there are those who take it all too far. Black and white thinkers who are sure that every statement is perfectly true or false (or at least run very close to that extreme). I'm not sure that we can actually talk to such people productively.
Daniel Taylor I strongly agree. Once someone is genuinely certain of something, there's not much point talking to them. "Knowledge is the most perfect barrier against learning", as Frank Herbert put it.
ReplyDeleteI make a very few exceptions. One is for practicality : you can have arbitrary amounts of arbitrarily high quality evidence for and against a position. Beyond a certain threshold it's convenient to label things as "certain" and "impossible", though I wouldn't usually mean the terms to be taken literally (another, rather stranger variety of absolutist thinking I see is when people accept that nothing is truly certain or impossible but then for some reason go to absurd extremes and completely forget that you can still judge relative probabilities - and sometimes the differences can be extreme).
The second exception I make is more philosophical. There comes a point when your only way to save an idea from being truly impossible is if you reject the notion of an objective, measurable reality. At that point the ideas become unscientific, which is very interesting but have to be rejected within the scientific world view.
I guess my only very minor disagreement would be with terminology. To me, "falsifiable" has much stronger connotations than "testable".
Heh. A zillion experiments won't prove anything "right". But it only takes one experiment to prove a theory wrong. I think that's Einstein.
ReplyDeleteI'm coming at this issue from the perspective of an archaeologist who tries to do science. My attitude may be tied into the problem that very few archaeologists actually take a scientific approach toward their research. They look for data that proves their hypothesis/model without considering whether or not that data also supports a competing model.
ReplyDeleteIn this regard, I still think that we should try to falsify hypotheses/ models whenever possible. But, I do agree that their are hypotheses/ models that can't be disproven.
In this regard, I will note a debate (on usenet) I had with the late David Rindos regarding the domestication of plants. Rindos argued that the domestication of plants was an unintentional product of symbiosis between humans and plants (similar to the relationship between ants and acacias). I argue that domestication was an intentional act undertaken by humans as a result of population pressure. After a fairly long exchange, I finally realized that there was no evidence that Rindos could present that I would find convincing of his model, and I couldn't present evidence that he would find convincing. At that point, I stopped debating with him.
And, from reading the comments, I find that are attitudes aren't as different as I originally thought. But, it is a conversation that I think is worth having.
Dan Weese See links within link. :)
ReplyDeleteJeff Baker Yeah, as a total outsider I've heard there are different attitudes to archaeology within the field. Some view it as a science and try and test hypotheses while others view it as a pure branch of the humanities where it's find to just propose whatever they think best. I found Time Team (I only watched the UK version) did a good job of promoting the more scientific approach : let's map the area with geophysics, record surface finds statistically, and go and test our hypothesis by digging stuff up.
ReplyDeleteFalsification sounds like it's binary and the real world is often noisier and better modeled using probability and statistics. Unlike in math, when a single counterexample falsifies, a single "counterexample" in science might just be noise. If we look at actual replication failures it takes more than that to convince scientists to change their minds.
ReplyDeleteThere is also a saying in economics: "all models are false, but some are useful."
But, I think the spirit is that you do pay attention to observations, try to get better at gathering them, and theory is judged based on observation. Perhaps Feynman can be forgiven for simplifying a bit for dramatic effect?
In software development we like our tests to be pass/fail too, but inevitably some tests are flaky, often due to performance variation. It can take many more test runs to understand whether a flaky test indicates anything real, so we try to eliminate those.
Jeff Baker Re: "My attitude may be tied into the problem that very few archaeologists actually take a scientific approach toward their research. They look for data that proves their hypothesis/model without considering whether or not that data also supports a competing model."
ReplyDeleteThis is of course not a problem that specifically relates to just archaeology, or even science. Each person exists at a particular subject-object relation: They are subject to some things (meaning that they cannot think outside of them), and some things are objects to them (meaning that these are things that the person can mentally manipulate or discuss).
The essence of the problem which is occurring throughout the sciences is that there exists an epistemological disagreement about the importance of and probably even the meaning of a worldview. And it can be confusing because the definition can even possibly change depending on who a person is talking to.
When speaking to the public about science, the traditional academic culture oftentimes casts the worldview in academia as the scientific worldview. But, a worldview cannot be perfectly consistent amongst all scientists across even a single domain because it is in part formed by ones' experiences and exposure to information.
What is missing toady from at least the public-facing academic culture is the treatment of worldviews as objects. What we see today is a culture which oftentimes exists within the worldview -- in many cases not just refusing to critique or think outside of it, but actually many times lacking the daily practice necessary to excel at these habits.
In my own view, a lot of these problems are reinforced by the science journalism of the day. In fact, there is a science journalism history to this which emerged as a reaction to the emergence of Relativity and quantum mechanics. Up to that point, the popularization of science had relied entirely upon its explanatory appeal. What many people have missed is that these newer theories undermined that mode of operation, and threw science journalism into a state of chaos. It had to adapt in order to survive. Science historian John C Burnham has done a great job of explaining this complex transition in How Superstition Won and Science Lost: Popularizing Science and Health in the United States (a book which, it seems, nobody has actually read) ...
"The result of this challenge to the unity of science was to encourage popularizers to redouble the emphasis that had been developing on portraying the results, rather than the ideas, of science. The Science Year Book published in the 1940s, for example, usually had 'aviation' as a major category and consisted almost entirely of articles about applied science developments. As early as 1903, W.S. Franklin, a physicist at Lehigh, observed: 'Everything that appears in the name of science in our newspapers and magazines relates only to results. Have any of you seen in our newspapers or popular magazines any detailed description of the principles and methods used by Marconi in his wireless telegraphy?' More than half a century later, Palmer Wright, a Dow Chemical chemist, wondered if popularizers should not build on applied science for understanding. 'The popular thrust,' he observed, 'is toward the what, not the why of science.'
So it was that popularized science in the twentieth century continued as in the nineteenth to emphasize progress -- but progress now in terms more exclusively of the 'applications mankind can make from [the] marvelous findings' of 'pure science,' as a 1926 writer put it. But the new context of progress had further implications. Any facts at all could be digested into the popular science of results;
they needed no further context than that they were part of the advance of science. Thus a discovery at any level of science reported in the 1920s was a fact and a part of the progress of science; so, too, was a machine of the 1930s or a cure of the 1940s a fact and a part of the progress of science. This approach to popularizing was compatible with the rise of general science in the schools, in which teachers emphasized application and turned away from the work of researchers that was abstract and unpopular. Even in the 1960s, reported Howard E. Gruber, 'high school teachers generally approach[ed] science teaching as a matter of conveying science as established facts and doctrines' rather than 'science teaching in which science [was] treated as a way of thought.'"
ReplyDeleteThese reporting traditions continue with us today, and what they are doing is affecting the public's subject-object relationship with worldviews in science in possibly unintended ways. The reporting is narrowing the scope of our interactions with this scientific worldview.
What we will need to do is to replace this journalism with a format which is fundamentally designed to train people in the practice of manipulating worldviews.
And that is why scientific controversies really matter today, for this is where the worldviews clash.
A proper educator says "Look, here's the current working map of what we've explored. My job is to introduce you to the map. Furthermore, nobody knows it all. But lookit here... see that? There's a guy working on the problem of gasses in and around galaxies. Space isn't nearly as empty as we like to think it is... and that part of the map is woefully incomplete. But he's working on it and so are his colleagues. And one day, once you've come to terms with the map, you'll be in a position to add your own bits to it..."
ReplyDeleteArchaeologists know this much: once mankind started domesticating animals and planting crops, he was off to the races. All the other hominids, waiting around like today's simians, for food to come ripe, that's why they're endangered species and mankind is a dangerous species. Whether or not that process arose from increased population or vice versa - this much we do know: Chaco Canyon emptied out when those people ran out of pine nuts. And when the Sahara started drying out, that's about the time those big river cultures like the Egyptians and Nubians arose.
Science can become a cult, as surely as anything else. Truth as a concept is entirely overrated: that map from Paragraph 1 is entirely provisional. I entertain a genial contempt for Science Popularisers: they're Un-Teachers. To explain anything to a civilian requires simplifying the problem to the point of error at some level. At best, the Popularisers can pose rhetorical attacks upon Dumb Ideas, saying "The Ptolemaic universe actually had the math to support it: the genius who built the Antikythera Mechanism could correctly place those planets on his dial. But it's a lot simpler to build a heliocentric model..."
Dan Weese Hi, Dan. I think academia should spend more time studying sand. They always point to the warming events, with the implication that the sand just sort of appears or moves around the surface of the planet into the spot where it is observed. Close inspection of the actual details oftentimes raises more questions.
ReplyDeleteIn many cases, the sand appears to very suddenly appear across vast swaths of the planet. The amounts are sometimes perplexing.
Making matters worse, it oftentimes exhibits a strange iron oxide varnish, which under microscope is just a thin coating on a clear quartz interior.
And based on what I've seen to date, the textbook theories for sand's provenance appear conjectural for many large deposits. These conjectures may even be distracting people more broadly from recognizing that colored sand is oftentimes itself an anomaly: The ideas which try to establish sand's provenance (typically transport) would appear to oftentimes be in conflict with the existence of this thin iron-oxide varnish applied to each of the grains.
The varnish seems to rule out transport, and transport seems to rule out the varnish.
This is an example of a controversy which may or may not pan out in the long run. But, I'm gonna wager you hadn't heard the claim before (?).
Either way, our modern approach to science does not appear to involve tracking scientific controversies. The culture oftentimes discourages even their discussion, prioritizing upfront judgments as to whether or not the claims are true over just learning the debates. That's a mistake. We should all be experts in at least knowing what all of the controversies are, so that we can each recognize when challenges to textbook theory are exhibiting some observational successes.
To simply assume that all of the prior settled science claims are correct is probably wishful thinking. A more realistic approach admits that mistakes may have been made along the way, and simply crowdsources the process of tracking controversies.
Chris Reeve As synchronicity would have it, my childhood home in Dengas Niger was the last outpost of settled humanity before the Sahara Desert began in earnest, north of us. A huge erg of sand and dust for hundreds of kilometers, a treacherous sea, with few landmarks. I'd see the harmattan wall cloud arriving and cry in fear. We'd be in that harmattan for weeks at turns, barely able to see.
ReplyDeleteI've had time to think about sand. I spent time in the American Southwest: it's a landscape shaped by wind and to a surprising extent, by water. Here in this area of Wisconsin, sand is mined for gas well fracking. Trains go by, up the street, loaded with that sand.
Kissinger gets the credit but a dozen wags have made the joke before him: "Academic quarrels are vicious precisely because the stakes are so small." It's Talk Show Science: we put on two people known to fundamentally disagree - and expect something productive to come of the discussion. Doesn't work like that. As you say, the varnish on the sand is a confounder. And I'd never thought about it. But I do know this much about how to phrase an argument, I can always ask someone who disagrees with my conclusions "What leads you to your conclusions?" Can't learn anything whilst talking.
Dan Weese I am as well closely watching the efforts of the successful biology professor from Washington state Dr. Gerald Pollack, who has already advised the NIH and NSF on how to fix its gatekeeping problem. They did not enforce his recommendations, so he has realized the only way to fix this is to create a new privately-funded funding agency which only funds challenges to textbook theories. He has aggregated an impressive board of advisors, and he's proposed a novel system for peer review which seeks to remove the politics from the situation (outsider specialists are asked to help at a critical moment).
ReplyDeleteThe name of the fund he's trying to fund is the Institute for Venture Science.
If he succeeds, it will forever change the sciences -- because it would only take just one of these challenges to be true, in order for the world to take notice. And I can guarantee you that there is more than just one opportunity for the textbooks to be wrong.
Chris Reeve Here's my problem with Holistic and Psychic and Noetic anything. Alice in Wonderland: "Why, sometimes I've believed as many as six impossible things before breakfast.'
ReplyDeleteI'm a man of faith. Christian. And furthermore I make no excuses for it. This much I'll tell you about faith: it frankly admits there's no evidence to support it. Those who can't get that admission out of their mouths tend to indulge in Cosmic Woo and it's frankly embarrassing to listen to these would-be Aquinas-es wandering around, trying to square up religion with reason. Faith is as much doubt as belief.
As Rhys Taylor put it "There comes a point when your only way to save an idea from being truly impossible is if you reject the notion of an objective, measurable reality. At that point the ideas become unscientific, which is very interesting but have to be rejected within the scientific world view."
Psychics and holistic hooey and all the rest of that - well, there's another researcher who I genuinely admire, James Randi, a prophet for our times. Turn James Randi loose on these Holistic Hokum Artistes, he's got barrels full of Bollox Repellent.
Dan Weese I try to focus on the individual claims. The worldview basis is important to know, but a claim can be right for the wrong reasons. I think it's a mistake to reject all claims by a worldview because the worldview is rejected.
ReplyDeleteThat is treating the situation as an either-or binary selection, when in fact the real situation is a lot more messy than that -- with all sorts of worldviews competing for attention.
Dr. Pollack was told when he was in graduate school that he could study anything he wanted except for water.
They told him that of course because of the polywater debacle. Dr. Pollack ignored that warning, and eventually ended up studying water.
He is today famous for the discovery of EZ (exclusion zone), which is an observational fact that exists in any glass of water that is subjected to infrared (with a known peak). The infrared organizes the water into a structure at its surface, and this EZ water even exhibits a voltage potential when connected to the rest.
He's written a book which exposes a handful of problems with textbook biology -- with regards to the cell membrane, fundamental questions about how the body works, and the role of gels in cell function. His discoveries have largely yet to be incorporated into medical theory or practice. We're still waiting for that.
But, the scientific community is finally starting to understand that there is actually something worth looking at here with structured water. There is by now undeniable experimental evidence which can be repeated by anybody. They usually do not require expensive equipment.
Some people may misinterpret all of this activity as holistic medicine. I wouldn't call it that at all, personally. He's a water researcher, and his discoveries appear to have important ramifications for cell biology. He's basically discovered why life requires water.
Chris Reeve As someone with enough semi-obsolete tech manuals on his shelf to build a cellulose supertanker, I understand the pejorative attributive "textbook" well enough to have used the metaphor of a map, with thousands of cartographers, working away in their little plots. Often as not, they're obliged to redraw large-ish bits - huge foofaraw in the applicable scientific journals, to be sure. But today's textbooks are mere guides to the maps we have at present.
ReplyDeleteHave you ever returned to something you were quite proud to have created, long ago, only to find it wasn't half as great as you remember? If you don't look back at your life with a twinge of embarrassment, you haven't learned much, it seems to me.
Insofar as Pollack's claims about EZ Water have become the basis for all manner of homeopathic hoopla, much of which has been promulgated by Pollack himself, I just gotta say, a minute of googling hath revealed him saying EZ Water is a cure for kidney disease. And I have a very serious problem with that statement. Just sayin'
Dan Weese I don't know why somebody is singling out kidney disease. My own interpretation of what Dr. Pollack has discovered is that living tissue is alive because it is structuring the water. I am under the impression that MRI's work upon the same basic principle (?).
ReplyDeleteThe logic goes like this, just from the top of my head though ...
Structured water is the one-size-fits-all tool which gels use to maintain ionic gradients and do all sorts of functional tasks. The gel has a sort of transistor-like action embedded within it, where a small-input stimulus can transform the signal into a much larger action. Think muscles.
A situation in the biological sciences is that a lot of this theoretical functionality for the cell has been loaded into the cell membranes (a concoction of pumps and channels each somehow tailored for particular molecules).
The apparent mistake is that the cell is not a sack of fluid, but rather a gel. And what we have learned about gels over time is that they exhibit some remarkable similarities to living tissues. And my understanding is that many biologists will acknowledge that cells are gels.
But, the situation is that gels were discovered to have all sorts of functionality built into them which was accidentally loaded, in the textbooks, into the cell membrane. Gels can for instance control ionic gradients without any membrane at all. The complex system of pumps and channels in biological textbooks seems like a solution in search of a problem that's already been solved.
By not addressing the information problem of controversies, we are constantly seeking out shortcuts to winnow the field of ideas, as if we are trying to minimize our thinking.
The fad of the day is debunking. But, how hard is it really to "debunk" something when the evidence which supports it has never really been collected together into a single place?
We have a big information problem with controversies. There are way more controversies than people realize, and the information about them is extremely spread out. We need to bring it all together into a consistent format which is easily and rapidly digested.