Cherry Picking, Black Swans and Falsifiability
Whenever a skeptic points out a new paper or journal article refuting some claim made by the theory of anthropogenic global warming, climate change alarmists often shout “cherry picking!” Evidently, most climate change true believers do not understand how science works or how theories are tested. Scientific theories must make predictions by which they can be tested. Providing evidence that AGW has failed in its predictions is not cherry picking, it is refutation. Unfortunately, when confronted with failed predictions the standard alarmist answer is to disavow the predictions. They will say that those are not predictions at all, they are projections—and that means AGW is not a scientific theory at all.
I recently received a long email from a friend, who is a global warming believer, regarding my earlier post, “Why I Am A Global Warming Skeptic.” This friend is an educated person, a philosopher but not a scientist. In reading his objections and counter arguments to my points I found a number of common misunderstandings that arise when laymen try to jump into a scientific debate. In this report I will address some, but by no means all of my friend's objections.
Having written previously on how to judge global warming as a scientific theory, perhaps it is time to recap some of that discussion. In The Resilient Earth, we wrote about the philosophy of science and how scientific theories are formulated and then validated. Science is both a body of knowledge and an approach to understanding nature by gaining more knowledge. It is based on gathering empirical evidence.
Empirical means simply what belongs to or is the product of experience or observation. If you can touch it, smell it, feel it, see it or measure it, it's empirical. Collecting empirical data through observation or experimentation is how the correctness of theories is established. It is also how theories are found to be invalid.
An important point is that, in order to be testable, a theory must make predictions about how nature behaves. This idea comes from the work of Karl Popper, one of the important philosophers of the 20th century. Popper, who early in life was a communist, came to consider democracy the only form of government in which reason is celebrated. He identified Plato, Marx and Hegel as the greatest enemies of democracy in his 1945 book The Open Society and Its Enemies. As interesting as Popper's philosophy regarding society and politics was, it is his work on the philosophy of science that interests us here.
Popper made the following observations as to what makes a good scientific theory:
- It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations.
- Confirmations should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory.
- Every "good" scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is.
- A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice.
- Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; but there are degrees of testability: some theories are more testable, more exposed to refutation, than others; they take, as it were, greater risks.
- Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. (I now speak in such cases of "corroborating evidence.")
- Some genuinely testable theories, when found to be false, are still upheld by their admirers — for example by introducing ad hoc some auxiliary assumption, or by reinterpreting the theory ad hoc in such a way that it escapes refutation. Such a procedure is always possible, but it rescues the theory from refutation only at the price of destroying, or at least lowering, its scientific status.
Popper made a distinction between what he termed conditional scientific predictions, which have the form “If X takes place, then Y will take place,” and unconditional scientific prophecies, which have the form “Y will take place.” It is the former rather than the latter which are typical of the natural sciences. This means that predictions made by scientific theories are typically conditional and limited in scope—taking the form of a hypothetical assertion stating that certain specified changes will come about if particular preceding events take place. Conversely, if X takes place and Y does not, then the hypothesis must be false.
The Cherry Picking Argument
The first point of contention in this discussion is the time worn claim of “cherry picking.” This charge is often heard when a skeptical article cites papers in support of its position. The complaint as lodged by my friend goes like this:
You are surely here guilty of cherry picking a couple of published articles that go your way (at least when read in a certain way) out of all the thousands of articles in the literature, most of which don't go your way. To form a fair assessment of the state of the science you need to look at all of it, not just take one or two articles out of context.
The problem here is, if you don't cite some recent papers then you have provided no references and will be accused of making unsubstantiated claims (unless, of course, you are presenting experimental observations of your own). If you cite a few representative articles to underline your point you are accused of cherry picking. According to Wikipedia, cherry picking is the act of pointing at individual cases or data that seem to confirm a particular position, while ignoring a significant portion of related cases or data that may contradict that position. While this is certainly a bad thing when dealing with data and collections of repeated experimental results, the definition does not necessarily extend to general scientific argumentation.
The reason for this lies in the concept of falsifiability—a condition that must be met by all valid scientific theories. Popper noted that it is easy to obtain evidence in favor of virtually any theory, and he consequently held that such corroboration should count scientifically only if it is the positive result of a genuinely “risky” prediction. Risky here means that the prediction could conceivably have been false. For Popper, a theory is scientific only if it is refutable by a conceivable event. Every genuine test of a scientific theory, then, is logically an attempt to refute or to falsify it, and one genuine counter-instance falsifies the whole theory.
Basically, a theory must make predictions about how nature behaves so that the validity of the theory can be tested through experimentation and/or observation. For example, say I claim that all swans are white, based on my direct observation of a sample of swan populations. If someone finds a single black swan and is able to document the observation (e.g. by taking a picture or capturing the beast) then the white swan theory is disproved. In this case a single contradicting observation is sufficient to invalidate the theory.
It is possible that a disproved theory can be modified to better fit nature as observed—the white swan theory could be amended to say “most swans are white.” In this case the new theory could not be disproved by a single black swan siting, it would take finding a numerical majority of non-white swans to disprove it. Scientific philosophy would say that the first white swan theory is a stronger theory, the assertion that all swans are white being much more restrictive than the modified “most swans” theory. Simply put, the stronger the theory the simpler it is to disprove, the argument being that an easily disprovable theory which stands the test of time is stronger than a theory, which would take a much larger effort to debunk. It takes a deeper understanding of the assertions made by a theory to know what kind of argument is needed to disprove it.
When it comes to the AGW theory, which states that human generated CO2 is the reason for increasing world temperature, there is some wiggle room for its proponents, but not much. If it can be shown that the sum total of other contributing factors is more influential than CO2 then the theory is proven false. Any valid observation which shows CO2's influence is less important to climate change than other factors diminishes the validity of the theory. Moreover, if many of the predictions made by the theory are shown to be false then the theory is weakened—the death of a thousand cuts scenario.
If a theory claims to explain climate change and new work shows that there are phenomena that the theory does not explain then that theory is incomplete. If nature shows assertions made by the theory to be wrong then the theory is false. The papers I cited showed that there is dispute among scientists, that nature is still serving up big surprises that climate science is at a loss to explain, that the science is not settled. Given this evidence, for any layman to state otherwise is preposterous but as Popper himself said, “irrationalism will use reason too, but without any feeling of obligation.”
Furthermore, scientific arguments are not won by counting the number of papers written either for or against a particular theory. As was demonstrated above, a single counter example can be sufficient to disprove a theory, no matter how many scientists think the theory to be correct. In science there is an ultimate arbiter—nature.
Returning to the charge of cherry picking, the reason I chose the papers I did was because they were both recent, written by known climate scientists who are not fringe denier kooks, and (yes) they help to support my position. The intent is not to provide an argument from authority, another typical logical fallacy, but to show that the arguments presented do originate from trustworthy sources. Even so, that is immaterial to the real argument.
Those papers explicitly stated that mechanisms other than CO2 are noticeably involved in climate regulation. Moreover, the postulated positive feedback mechanism linking CO2 and water vapor does not respond in the expected way. Far from being a minor detail, this directly refutes part of the AGW theory as presented to the public.
If someone purposefully selects data or data sets so a study will give desired, predictable results—which may be misleading or even completely contrary to actuality—then the charge of cherry picking is valid. Michael Mann's use of selected tree ring data sets and then manipulating them statistically to produce the infamous “hockey stick” graph, that was cherry picking. Citing specific evidence that refutes fundamental assertions made by a theory, that is not cherry picking. Under the latter circumstance, the cherry picking retort is weak argumentation and intellectually lazy to boot. Dispute the articles, challenge the assertions, or provide some counter examples of your own but for heavens sake, argue the matter at hand.
Judged and found wanting
Returning to the subject of proving or disproving the theory of anthropogenic global warming, there are only three possibilities here: AGW makes no predictions and hence is not a scientific theory; AGW depends on vague feedback mechanisms that must be constantly reinterpreted, making AGW a very weak theory and scientifically useless; or the predictions made by climate scientists about the effects of AGW are just that, predictions, and if those predictions can be shown to not be true then AGW is a false theory.
It is hard to believe that any pro-AGW climate scientist would agree to the first interpretation, that AGW is simply not a scientific theory, so we can eliminate all that crap about AGW not making any predictions. If it is a theory it must make predictions, it must be falsifiable based on empirical evidence.
In the second scenario, the argument is often made that AGW is correct but not perfect, that it is getting better and its predictions more accurate every day. This fits Popper's final observation about theories, where ad hoc modifications are constantly needed to rescue a theory from falsification. Since such actions rescue a theory from refutation only at the price of destroying or lowering its scientific status—again an outcome that no AGW true believer would accept—we will move on to the final possibility.
The final scenario is that AGW as formulated does indeed qualify as a scientific theory. In this case it must make specific predictions that permit the theory to be falsified. So what are the strongest predictions made by the AGW theory? Let us first identify AGW's primary tenent. That is that human CO2 emissions are causing Earth's temperature to rise. What are the observed facts?
- Are CO2 levels rising? The evidence seems to indicate that this is true.
- Is the increase in atmospheric CO2 being caused by human activity? There is some debate over how much is due to nature and how much due to man but let's simply accept this assertion as well.
- What does AGW theory predict should happen as a result of these conditions? The obvious answer is that global temperature should rise.
In fact, back in 1988, NASA’s in house climate alarmist James Hansen presented a prediction of steadily rising temperatures to Congress. Has this prediction been substantiated, proving that AGW is a valid theory? It should be obvious from the graph below that Hansen got it wrong.
Not only was Hansen wrong back in 1988, all the models used by all the AGW proponents managed to miss the leveling off and slight downturn in temperature rise that started in 1995. In an email to Michael Mann, Kevin Trenberth, head of the Climate Analysis Section at the National Center for Atmospheric Research (NCAR), asked “where the heck is global warming?” He went on to explain:
The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t. The CERES data published in the August BAMS 09 supplement on 2008 shows there should be even more warming: but the data are surely wrong. Our observing system is inadequate.
Trenberth was a lead author of the 1995, 2001, and 2007 IPCC Scientific Assessment of Climate Change. If he had any sense of shame he would give back that Nobel Prize he shares (as a member of the IPCC) with Al Gore.
Furthermore, Dr. Phil Jones from the UK's Climate Research Unit (CRU), has recently conceded the possibility that the world was warmer in medieval times than now–suggesting global warming may not be a man-made phenomenon. He is quoted as saying that for the past 15 years there has been no “statistically significant” warming. Yet the yearly averaged atmospheric CO2 level has continued to steadily rise. What we have here, as Popper showed, is a false theory.
When contradicting evidence is found you must reevaluate the theory that made the discredited assertions. Based on direct measurement, AGW's fundamental prediction that an increase in CO2 levels causes an increase in temperature is false, therefore AGW is a false theory. The correct answer might not be known but the wrong answer is still wrong. In the mean time, climate scientists must stop making unsubstantiated claims or accept that their theory has been proven false by the very predictions they use to argue their case.
Be safe, enjoy the interglacial and stay skeptical.
Post Script: This is the second of two articles that centered on the philosophy of Karl Popper. The first article was “More Fictitious Hurricane Predictions.”
Addendum: There has been quite a flurry of comments regarding the “Hansen vs reality” graph I used in the article above. This graph has been criticised for presenting only the “A” scenario while Hansen actually presented three scenarios back in 1988 in his famous presentation to Congress. The prediction scenarios, labeled A, B and C, were based on a paper by Hansen et al. that had recently been published. Here is an updated graph showing all three scenarios plus NASA RSS satellite data and GISS surface temperature data.
Each of the three scenario described forcing projections for CO2, CH4, N2O, CFC11, CFC12 and the other Montreal Protocol traces gases as a group. Hansen is on record as saying that scenario B was the “most plausible,” and that it was his preferred scenario. As can be seen, there was substantial divergence between Scenario B in 2007 and both the RSS satellite temperature and the GISS temperature surface temperature series.
According to Steve McIntyre, “Hansen's statement that Scenario B was "used" in his 1988 testimony is very misleading: Hansen's oral testimony called Scenario A the "Business as Usual" scenario and mentioned Scenario B only in maps purportedly showing extraordinary projected warming in the SE USA.” Scenario C assumed that atmospheric CO2 levels were stabilized at 368 ppm in 2000 but that level has already been surpassed, rendering that prediction moot. If anything scenario C's curve should be below observed temperatures, but this is obviously not the case.
So there you have it: the graph I used in the article above does compare measured temps only with scenario A and I am guilty of not presenting the other scenarios. However, the assumptions on which C was based have not come to pass so it is now superfluous. And since Hansen chose to put the worst case scenario A forward, obviously trying to alarm members of Congress and the public, I feel absolutely no shame in using it in the comparison above. Scenario A was an exaggeration of Hansen's own creation and if you choose to persuade by exaggeration you should expect your case to die by exaggeration.
The graph and the background data presented here are taken from an article by Steve McIntyre at ClimateAudit.org. There is a lot of additional information at that link which is well worth reading. Thanks to the anonymous reader who posted this link and several others below.