Thursday, 30 April 2015

4th anniversary of Variable Variability

The previous birthdays I have simply forgotten, my apologies, but today is the 4th anniversary of Variable Variability.

More than usual birthdays, the day is a bit arbitrary. Many years I already wrote the occasional essay for my homepage. Ideas that I wanted to share, but were not big enough for an article or thoughts that I was afraid I would forget again with my terrible memory.

The first post of Variable Variability on the 30th of April 2011 was thus a republication of an essay from 2004 on the fractal nature of clouds and the limits of the fractal approximation.

The first two posts specially written for Variable Variability are: Is obesity bias evolutionary? and Good ideas, motivation and economics, both on the 28th of May 2011 somehow. Must have been a writing-fever day.

They are still good posts, but not much read because of the time they were published. To reduce your weight it may help to consciously eat less and move more, but that is an empirical question, it is not as obvious as many claim. Science is a creative profession and knowing where good ideas come from can thus be helpful. The second skill would be a good intuition, which of these problems to tackle. You also need technical skills, but they are overrated.

Another contender for the beginning of this blog would be January 2012, when I posted a press release on the famous homogenization validation study of the COST Action HOME. From a meeting several years before, I vaguely knew that Roger Pielke Sr. was interested in homogenization. Thus I asked him if he was interested in a repost. Roger referred me to someone called Anthony Watts of Watts Up With That. Coming from a good source, Watts asked me whether he could repost, maybe he noticed that I was not too enthusiastic (I had read some WUWT posts by that time) or maybe he read the post (and did not like the message that homogenization methods improve temperature trend estimates) because in the end he did not post my press release.

To promote the press release I started to look a bit more at other blogs and comment there. Now I see this as part of blogging, thus you could see this as the start of blogging.

Mitigation sceptics seem to have a tendency to think that scientific articles are written for the purpose of pissing them off. But they are not that important. Most scientists hardly know they exist. I did know that there was some trouble on the other side of the Atlantic in the USA, that a larger group there was sceptical about climate change. Being sceptical is sympathetic and there are so many real uncertainties. So why not? I was completely unprepared for the industrial production of deceit and misinformation and the rock bottom quality of the nonsense these people made up. I still wonder why mitigation sceptics do not orient themselves by the real uncertainties and problems; they are spelled out in detail in every IPCC report. It sometimes looks like winning a debate is less important to the most vocal mitigation sceptics than angering greenies, which is best done by stubbornly repeating the most stupid arguments you have. (In the light of the discussion we are just having at ATTP, let me add that I this is not an argument, I do not think this paragraph will convince anyone of climate change, I am just describing how I personally see the situation.)

Like the blogger of And Then There's Physics, I naively thought that it may be helpful to explain the mistakes in the posts of WUWT. Now keeping more of an eye on other blogs, I noticed a WUWT post about an erroneous conference abstract by Koutsoyiannis, which was called a "new peer reviewed paper". I wrote: "As a goal-oriented guy, Anthony Watts found the two most erroneous statements". That was the first time I got some readers and at least Watts admitted that there was no peer review. The other errors remained.

Earlier this year I wrote a short update:
Now two and a half year later, there is still no new activity on this [Koutsoyiannis] study. I guess that that means that Koutsoyiannis admits his conference abstract did not show what the State of the Climate and WUWT wanted you to believe. Together with all the other WUWT fails, it is clear that someone interested in climate change should never get his information from WUWT.
But well, who is listening to me, I do not have the right party book.

A little later Anthony Watts put WUWT on hold for the weekend because he was working on something big. He had written a manuscript and a press release and asked for blog reviews. Thus I wrote a blog review about this still unpublished manuscript. This was the first time this blog got a decent number of comments and the time people informed each other in comments elsewhere that there was this new blog on the block. That was fun.

I have somewhat given up on debunking misinformation and aim to write more about what is really important. Sometimes the misinformation is a nice hook to get people interested. I had wanted to announce the birth of the new Task Team on Homogenization (TT-HOM) anyway. By connecting this to the review of the UK Policy Foundation with silly loaded questions, more people now know about TT-HOM than otherwise; normally setting up a a new Task Team would have been a boring bureaucratic act not much people would be interested in.

Debunking does not seem to have much impact, even in cases of clear misquotations and when Watts deceives his readers about his number of readers, there is no bulge. The number of WUWT readers is a small matter, but it does not require any science skills to check, everyone can see this was deception. However, no one complained about being lied to.

America's problems with mitigation sceptics are not because of the science, mitigation sceptics seem to make this claim to make adult political debate about (free market) solutions impossible. By not complaining about deception they make clear that their issue is not the science. Their response to a new scientific paper is almost completely determined by the question: can it be spun into a case against mitigation? If they were interested in science, their response would be determined by the quality of the study. America's problems with mitigation sceptics can thus also not be solved by science or science communication. The American society will have to find a political solution for their toxic political atmosphere. Maybe that simply starts by talking to your neighbour, even if he has the wrong bumper sticker. Go on the street and make it clear to everyone this is important and something to think more about and make it clear to politicians that it is costly to peddle nonsense.


Blogging has been a lot of fun. I like writing, playing with ideas and debating.

The first two posts got 50 to 100 pageviews up to now. They were purely written for fun.

If you write a scientific article you can also be happy to get a 100 readers. But if it inspires someone else to continue the thought, to build on it, it is worth it.

Compared to the number of readers a scientist is used to, this blog is a huge megaphone. Thank you all for reading, your interest is very rewarding and nowadays another reason to write posts. With that megaphone also comes responsibility. I guess that mitigation sceptics will not like much of what I write, but when colleagues do not like something I hope for their honest feedback. That would be valuable and is much appreciated.

Blogging has given me a broader perspective on climatology and science in general. Where else would I have heard of the beautiful ideas of John Zyman on the social dimension of science? (Thanks Mark!)

A birthday is a good day to wish something. Do you have anything you would like me to write about? I will not promise anything, I still have about 100 drafts in the queue, but welcome new ideas. I am always surprised how much people read my posts about how science works and the intricacies of the scientific culture. It is my daily life, almost like writing about washing the dishes. It is easy for me to have a blind spot there, thus if you have any questions on science, just shoot.

Please do not wish a post why homogenization increased the trend for some station in Kyrgyzstan. Not on a birthday.

* Photo of fireworks by AndreasToerl (Own work) [CC BY-SA 3.0], via Wikimedia Commons

Monday, 27 April 2015

Two new reviews of the homogenization methods used to remove non-climatic changes

By coincidence this week two initiatives have been launched to review the methods to remove non-climatic changes from temperature data. One initiative was launched by the Global Warming Policy Foundation (GWPF), a UK free-market think tank. The other by the Task Team on Homogenization (TT-HOM) of the Commission for Climatology (CCl) of the World meteorological organization (WMO). Disclosure: I chair the TT-HOM.

The WMO is one of the oldest international organizations and has meteorological and hydrological services in almost all countries of the world as its members. The international exchange of weather data has always been important for understanding the weather and to make weather predictions. The main role of the WMO is to provide guidance and to define standards that make collaboration easier. The CCl coordinates climate research, especially when it comes to data measured by national weather services.

The review on homogenization, which the TT-HOM will write, is thus mainly aimed at helping national weather services produce better quality datasets to study climate change. This will allow weather services to provide better climate services to help their nations adapt to climate change.


Homogenization is necessary because much has happened in the world between the French and industrial revolutions, two world wars, the rise and fall of communism, and the start of the internet age. Inevitably many changes have occurred in climate monitoring practices. Many global datasets start in 1880, the year toilet paper was invented in the USA and 3 decades before the Ford Model T.

As a consequence, the instruments used to measure temperature have changed, the screens to protect the sensors from the weather have changed and the surrounding of the stations has often been changed and stations have been moved in response. These non-climatic changes in temperature have to be removed as well as possible to make more accurate assessments of how much the world has warmed.

Removing such non-climatic changes is called homogenization. For the land surface temperature measured at meteorological stations, homogenization is normally performed using relative statistical homogenizing methods. Here a station is compared to its neighbours. If the neighbour is sufficiently nearby, both stations should show about the same climatic changes. Strong jumps or gradual increases happening at only one of the stations indicate a non-climatic change.

If there is a bias in the trend, statistical homogenization can reduce it. How well trend biases can be removed depends on the density of the network. In industrialised countries a large part of the bias can be removed for the last century. In developing countries and in earlier times removing biases is more difficult and a large part may remain. Because many governments unfortunately limit the exchange of climate data, the global temperature collections can also remove only part of the trend biases.

Some differences

Some subtle differences. The Policy Foundation has six people from the UK, Canada and the USA, who do not work on homogenization. The WMO team has nine people who work on homogenization from Congo, Pakistan, Peru, Canada, the USA, Australia, Hungary, Germany, and Spain.

The TT-HOM team has simply started outlining their report. The Policy Foundation creates spin before they have results with publications in their newspapers and blogs and they showcase that they are biased to begin with when they write [on their homepage]:
But only when the full picture is in will it be possible to see just how far the scare over global warming has been driven by manipulation of figures accepted as reliable by the politicians who shape our energy policy, and much else besides. If the panel’s findings eventually confirm what we have seen so far, this really will be the “smoking gun”, in a scandal the scale and significance of which for all of us can scarcely be exaggerated.
My emphasis. Talk about hyperbole by the click-whore journalists of the Policy Foundation. Why buy newspapers when their articles are worse than a random page on the internet? The Policy Foundation gave their team a very bad start.

Aims of the Policy Foundation

Hopefully, the six team members of the Policy Foundation will realise just how naive and loaded the questions they were supposed to answer are. The WMO has asked us whether we as TT-HOM would like to update our Terms of Reference; we are the experts after all. I hope the review team will update theirs, as that would help them to be seen as scientists seriously interested in improving science. Their current terms of reference are printed in italics below.

The panel is asked to examine the preparation of data for the main surface temperature records: HadCRUT, GISS, NOAA and BEST. For this reason the satellite records are beyond the scope of this inquiry.

I fail to see the Policy Foundation asking something without arguments as a reason.

The satellite record is the most adjusted record of them all. The raw satellite data does not show much trend at all and initially even showed a cooling trend. Much of the warming in this uncertain and short dataset is thus introduced the moment the researchers remove the non-climatic changes (differences between satellites, drifts in their orbits and the height of the satellites, for example). A relatively small error in these adjustments, thus quickly leads to large trend errors.

While independent studies for the satellite record are sorely missing, a blind validation study for station data showed that homogenization methods work. They reduce any temperature trend biases a dataset may have for reasonable scenarios. For this blind validation study we produced a dataset that mimics a real climate network with known non-climatic changes, so that we knew what the answer should be. We have a similar blind validation of the method used by NOAA to homogenize its global land surface data.

The following questions will be addressed.

1. Are there aspects of surface temperature measurement procedures that potentially impair data quality or introduce bias and need to be critically re-examined?

Yes. A well-known aspect is the warming bias due to urbanization. This has been much studied and was found to produce only a small warming bias. A likely reason is that urban stations are regularly relocated to less urban locations.

On the other hand, the reasons for a cooling bias in land temperatures have been studied much too little. In a recent series, I mention several reasons why current measurements are cooler than those in the past: changes in thermometer screens, relocations and irrigation. At this time we cannot tell how important each of these individual reasons is. Any of these reasons is potentially important enough to explain the 0.2°C per century cooling trend bias found in the GHNv3 land temperatures. The reasons mentioned above could together explain a much larger cooling trend bias, which could dramatically change our assessment of the progress of global warming.

2. How widespread is the practice of adjusting original temperature records? What fraction of modern temperature data, as presented by HadCRUT/GISS/NOAA/BEST, are actual original measurements, and what fraction are subject to adjustments?

Or as Nick Stokes put it "How widespread is the practice of doing arithmetic?" (hat tip HotWhopper.)

Almost all longer station measurement series contain non-climatic changes. There is about one abrupt non-climatic change every 15 to 20 years. I know of two long series that are thought to be homogeneous: Potsdam in Germany and Mohonk Lake, New York, USA. There may be a few more. If you know more please write a comment below.

It is pretty amazing that the Policy Foundation knows so little about climate data that it asked its team to answer such a question. A question everyone working on the topic could have answered. A question that makes most sense when seen as an attempt to deceive the public and insinuate that there are problems.

3. Are warming and cooling adjustments equally prevalent?

Naturally not.

If we were sure that warming and cooling adjustments were of the same size, there would be no need to remove non-climatic changes from climate data before computing a global mean temperature signal.

It is known in the scientific literature that the land temperatures are adjusted upwards and the ocean temperatures are adjusted downwards.

It is pretty amazing that the Policy Foundation knows so little about climate data that it asked its team to answer such a question. A question everyone working on the topic could have answered. A question that makes most sense when seen as an attempt to deceive the public and insinuate that there are problems.

4. Are there any regions of the world where modifications appear to account for most or all of the apparent warming of recent decades?

The adjustments necessary for the USA land temperatures happen to be large, about 0.4°C.

That is explained by two major transitions: a change in the time of observation from afternoons to mornings (about 0.2°C) and the introduction of automatic weather stations (AWS), which in the USA happens to have produced a cooling bias of 0.2°C. (The bias due to the introduction of AWS depends on the design of the AWS and the local climate and thus differs a lot from network to network.)

The smaller the meteorological network or region you consider, the larger the biases you can find. Many of them average out on a global scale.

5. Are the adjustment procedures clearly documented, objective, reproducible and scientifically defensible? How much statistical uncertainty is introduced with each step in homogeneity adjustments and smoothing?

The adjustments to the global datasets are objective and reproducible. These datasets are so large that there is no option other than processing them automatically.

The GHCN raw land temperatures are published, the processing software is published, everyone can repeat it. The same goes for BEST and GISS. Clearly documented and defensible are matters if opinion and this can always be improved. But if the Policy Foundation is not willing to read the scientific literature, clear documentation does not help much.

Statistical homogenization reduces the uncertainty of large-scale trends. Another loaded question.

Announcement full of bias and errors

Also the article [by Christopher Booker in the Telegraph and reposted by the Policy Foundation] announcing the review by the Policy Foundation is full of errors.

Booker: The figures from the US National Oceanic and Atmospheric Administration (NOAA) were based, like all the other three official surface temperature records on which the world’s scientists and politicians rely, on data compiled from a network of weather stations by NOAA’s Global Historical Climate Network (GHCN).

No, the Climate Research Unit and BEST gather data themselves. They do also use GHCN land surface data, but would certainly notice if that data showed more or less global warming than their other data sources.

Also the data published by national weather services show warming. If someone assumes a conspiracy, it would be a very large one. Real conspiracies tend to be small and short.

Booker: But here there is a puzzle. These temperature records are not the only ones with official status. The other two, Remote Sensing Systems (RSS) and the University of Alabama (UAH), are based on a quite different method of measuring temperature data, by satellites. And these, as they have increasingly done in recent years, give a strikingly different picture.

The long-term trend is basically the same. The satellites see much stronger variability due to El Nino, which make them better suited for cherry picking short periods, if one is so inclined.

Booker:In particular, they will be wanting to establish a full and accurate picture of just how much of the published record has been adjusted in a way which gives the impression that temperatures have been rising faster and further than was indicated by the raw measured data.

None of the studies using the global mean temperature will match this criterion because contrary to WUWT-wisdom the adjustments reduce the temperature trend, which gives the "impression" that temperatures have been rising more slowly and less than was indicated by the raw measured data.

The homepage of the Policy Foundation team shows a graph for the USA (in Fahrenheit), reprinted below. This is an enormous cherry pick. The adjustments necessary for the USA land temperatures happen to be large and warming, about 0.4°C. The reasons for this were explained above in the answer to GWPF question 4.

That the US non-climatic changes are large relative to other regions should be known to somewhat knowledgeable people. Presented without context on the homepage of the Policy Foundation and The Telegraph, it will fool the casual reader by suggesting that this is typical.

[UPDATE. I have missed one rookie mistake. Independent expert Zeke Hausfather says: Its a bad sign that this new effort features one graph on their website: USHCN version 1 adjusted minus raw. Unfortunately, USHCN v1 was replaced by USHCN v2 (with the automated PHA rather than manual adjustments) about 8 years ago. The fact that they are highlighting an old out-of-date adjustment graph is, shall we say, not a good sign.]

For the global mean temperature, the net effect of all adjustments is a reduction in the warming. The raw records show a stronger warming due to non-climatic changes, which climatologists reduce by homogenization.

Thus what really happens is the opposite of what happens to the USA land temperatures shown by the Policy Foundation. They do not show this because it does not fit their narrative of activist scientists, but this is the relevant temperature record with which to assess the magnitude of global warming and thus the relevant adjustment.

Previous reviews

I am not expecting serious journalists to write about this. [UPDATE. Okay, I was wrong about that.] Maybe later, when the Policy Foundation shows their results and journalists can ask independent experts for feedback. However, just in case, here is an overview of real work to ascertain the quality of the station temperature trend.

In a blind validation study we showed that homogenization methods reduce any temperature trend biases for reasonable scenarios. For this blind validation study we produced a dataset that mimics a real climate network. Into this data we inserted known non-climatic changes, so that we knew what the answer should be and could judge how well the algorithms work. It is certainly possible to make a scenario in which the algorithms would not work, but to the best of our understanding such scenarios would be very unrealistic.

We have a similar blind validation of the method used by NOAA to homogenize its global land surface data.

The International Surface Temperature Initiative (ISTI) has collected a large dataset with temperature observations. It is now working on a global blind validation dataset, with which we will not only be able to say that homogenization methods improve trend estimates, but also to get a better numerical estimate of by how much. (In more data sparse regions in developing countries, the methods probably cannot improve the trend estimate much, the previous studies were for Europe and the USA).

Then we have BEST by physicist Richard Muller and his group of non-climatologists who started working on the quality of station data. They basically found the same result as the mainstream climatologists. This group actually put in work and developed an independent method to estimate the climatic trends, rather than just do a review. The homogenization method from this group was also applied to the NOAA blind validation dataset and produced similar results.

We have the review and guidance of the World Meteorological Organization on homogenization from 2003. The review of the Task Team on Homogenization will be an update of this classic report.

Research priorities

The TT-HOM has decided to focus on monthly mean data used to establish global warming. Being a volunteer effort we do not have the resources to tackle the more difficult topic of changes to extremes in detail. If someone has some money to spare, that is where I would do a review. That is a seriously difficult topic where we do not know well how accurately we can remove non-climatic problems.

And as mentioned above, a good review of the satellite microwave temperature data would be very valuable. Satellite data is affected by strong non-climatic changes and almost its entire trend is due to homogenization adjustments; a relatively small error in the adjustments thus quickly leads to large changes in their trend estimates. At the same time I do not know of a (blind) validation study nor of an estimate of the uncertainty in satellite temperature trends.

If someone has some money to spare, I hope it is someone interested in science, no matter the outcome, and not a Policy Foundation with an obvious stealth agenda, clearly interested in a certain outcome. It is good that we have science foundations and universities to fund most of the research; funders who are interested in the quality of the research rather than the outcome.

The interest is appreciated. Homogenization is too much of a blind spot in climate science. As Neville Nicholls, one of the heroes of the homogenization community, writes:
When this work began 25 years or more ago, not even our scientist colleagues were very interested. At the first seminar I presented about our attempts to identify the biases in Australian weather data, one colleague told me I was wasting my time. He reckoned that the raw weather data were sufficiently accurate for any possible use people might make of them.
One wonders how this colleague knew this without studying it.

In theory it is nice that some people find homogenization so important as to do another review. It would be better if those people were scientifically interested. The launch party of the Policy Foundation suggests that they are interested in spin, not science. The Policy Foundation review team will have to do a lot of work to recover from this launch party. I would have resigned.

Related reading

Just the facts, homogenization adjustments reduce global warming

HotWhopper must have a liberal billionaire and a science team behind her. A great, detailed post: Denier Weirdness: A mock delegation from the Heartland Institute and a fake enquiry from the GWPF

William M. Connolley gives his candid take at Stoat: Two new reviews of the homogenization methods used to remove non-climatic changes

Nick Stokes: GWPF inquiring into temperature adjustments

And Then There's physics: How many times do we have to do this?

The Independent: Leading group of climate change deniers accused of creating 'fake controversy' over claims global temperature data may be inaccurate

Phil Plait at Bad Astronomy comment on the Telegraph piece: No, Adjusting Temperature Measurements Is Not a Scandal

John Timmer at Ars Technica is also fed up with being served the same story about some upward adjusted stations every year: Temperature data is not “the biggest scientific scandal ever” Do we have to go through this every year?

The astronomer behind the blog "And Then There's Physics" writes why the removal of non-climatic effects makes sense. In the comments he talks about adjustments made to astronomical data. Probably every numerical observational discipline of science performs data processing to improve the accuracy of their analysis.

Steven Mosher, a climate "sceptic" who has studied the temperature record in detail and is no longer sceptical about that reminds of all the adjustments demanded by the "sceptics".

Nick Stokes, an Australian scientist, has a beautiful post that explains the small adjustments to the land surface temperature in more detail.

Statistical homogenisation for dummies

A short introduction to the time of observation bias and its correction

New article: Benchmarking homogenisation algorithms for monthly data

Bob Ward at the Guardian: Scepticism over rising temperatures? Lord Lawson peddles a fake controversy

Wednesday, 15 April 2015

Why raw temperatures show too little global warming

In the last few amonths I have written several posts why raw temperature observations may show too little global warming. Let's put it all in perspective.

People who have followed the climate "debate" have probably heard of two potential reasons why raw data shows too much global warming: urbanization and the quality of the siting. These are the two non-climatic changes that mitigation sceptics promote claiming that they are responsible for a large part of the observed warming in the global mean temperature records.

If you only know of biases producing a trend that is artificially too strong, it may come as a surprise that the raw measurements actually have too small a trend and that removing non-climatic changes increases the trend. For example, in the Global Historical Climate Network (GHCNv3) of NOAA, the land temperature change since 1880 is increased by about 0.2°C by the homogenization method that removes non-climatic changes. See figure below.

(If you also consider the adjustments made to ocean temperatures, the net effect of the adjustments is that they make the global temperature increase smaller.)

The global mean temperature estimates from the Global Historical Climate Network (GHCNv3) of NOAA, USA. The red curve shows the global average temperature in the raw data. The blue curve is the global mean temperature after removing non-climatic changes. (Figure by Zeke Hausfather.)

The adjustments are not always that "large". The Berkeley Earth group may much smaller adjustments. The global mean temperature of Berkeley Earth is shown below. However, as noted by Zeke Hausfather in the comments below, also the curve where the method did not explicitly detect breakpoints does homogenize the data partially because it penalises stations that have a very different trend than their neighbours. After removal of non-climatic changes BEST come to a similar climatic trend as seen in GHCNv3.

The global mean temperature estimates from the Berkeley Earth project (previously known as BEST), USA. The blue curve is computed without using their method to detect breakpoints, the red curve the temperature after adjusting for non-climatic changes. (Figure by Steven Mosher.)

Let's go over the reasons why the temperature trend may show too little warming.
Urbanization and siting
Urbanization warms the location of a station, but these stations also tend to move away from the centre to better locations. What matters is where the stations were in the beginning of the observation and where they are now. How much too warm the origin was and how much too warm the ending. This effect has been studied a lot and urban stations seem to have about the same trend as their surrounding (more) rural stations.
A recent study for two villages showed that the current location of the weather station is half a degree centigrade cooler than the centre of the village. Many stations started in villages (or cities), thermometers used to be expensive scientific instruments operated by highly educated people and they had to be read daily. Thus the siting of many stations may have improved, which would lead to a cooling bias.
When a city station moves to an airport, which happened a lot around WWII, this takes the station (largely) out of the urban heat island. Furthermore, cities are often located near the coast and in valleys. Airports may thus often be located at a higher altitude. Both reasons could lead to a considerable cooling for the fraction of stations that moved to airports.
Changes in thermometer screens
During the 20th century the Stevenson screen was established as the dominant thermometer screen. This screen protected the thermometer much better against radiation (solar and heat) than earlier designs. Deficits of earlier measurement methods have artificially warmed the temperatures in the 19th century.
Some claim that earlier Stevenson screens were painted with inferior paints. The sun consequently heats up the screen more, which again heats the incoming air. The introduction of modern durable white paints may thus have produced a cooling bias.
Currently we are in a transition to Automatic Weather Stations. This can show large changes in either direction for the network they are introduced in. What the net global effect is, is not clear at this moment.
Irrigation on average decreases the 2m-temperature by about 1 degree centigrade. At the same time, irrigation has spread enormously during the last century. People preferentially live in irrigated areas and weather stations serve agriculture. Thus it is possible that there is a higher likelihood that weather stations are erected in irrigated areas than elsewhere. In this case irrigation could lead to a spurious cooling trend. For suburban stations an increase of watering gardens could also produce a spurious cooling trend.
It is understandable that in the past the focus was on urbanization as a non-climatic change that could make the warming in the climate records too strong. Then the focus was on whether climate change was happening (detection). To make a strong case, science had to show that even the minimum climatic trend was too large to be due to chance.

Now that we know that the Earth is warming, we no longer just need a minimum estimate of the temperature trend, but the best estimate of the trend. For a realistic assessment of models and impacts we need the best estimate of the trend, not just the minimum possible trend. Thus we need to understand the reasons why raw records may show too little warming and quantify these effects.

Just because the mitigation skeptics are talking nonsense about the temperature record does not mean that there are no real issues with the data and it does not mean that statistical homogenization can remove trend errors sufficiently well. This is a strange blind spot in climate science. As Neville Nicholls, one of the heroes of the homogenization community, writes:
When this work began 25 years or more ago, not even our scientist colleagues were very interested. At the first seminar I presented about our attempts to identify the biases in Australian weather data, one colleague told me I was wasting my time. He reckoned that the raw weather data were sufficiently accurate for any possible use people might make of them.
One wonders how this colleague knew this without studying it.

The reasons for a cooling bias have been studied much too little. At this time we cannot tell which reason is how important. Any of these reasons is potentially important enough to be able to explain the 0.2°C per century trend bias found in GHNv3. Especially in the light of the large range of possible values, a range that we can often not even estimate at the moment. In fact, all the above mentioned reasons could together explain a much larger trend bias, which could dramatically change our assessment of the progress of global warming.

The fact is that we cannot quantify the various cooling biases at the moment and it is a travesty that we can't.

Other posts in this series

Irrigation and paint as reasons for a cooling bias

Temperature trend biases due to urbanization and siting quality changes

Changes in screen design leading to temperature trend biases

Temperature bias from the village heat island

Sunday, 12 April 2015

Flying to the geo science flash mob in Vienna #EGU2015

ICE Train by Rhine River (3619844036)

You get used to it, but actually it is very strange. One week in spring, suddenly over 10 thousand geo scientists turn up in a middle European town, go into a huge building, specially build for such mega flash mobs, talk about arcane topics and find others that want to do the same. Crazy.

Such events also have environmental consequences. Especially travelling to Vienna produces much additional pollution. The UK Tyndall Centre recently produced a report stating that climate scientists should set an example and fly less. Not that this is a large contribution to climate change, but you can claim that for every individual action of a small group. No, the reason is according to the first author Corinne Le Quéré:
“It’s a credibility issue, ... We’re trying to support a change in culture.”
I find the idea very sympathetic, but I could not disagree more.

I do not see less flying making an impact, I do not think it would or should change the credibility of climate scientists and I do not think it supports a culture change.

No impact

Climate change is a [[tragedy of the commons]]. The atmosphere is a resource that every one of us uses; it is overexploited because every individual has benefits from using it and the downsides are spread over everyone. Without coordination these incentives invariably lead to overexploitation. The term tragedy of the commons comes from the communal lands around a village, the commons. Every villager benefits by having too much cattle, all suffer because the land degrades, that is the tragedy. One villager reducing his herd does not solve this problem. The neighbours will just enlarge their herds. You need coordination.

Climate change, the ozone hole or biodiversity loss are extra tough versions of such tragedies of the commons because so many people are involved. That makes coordination difficult. To make the number of participants workable normally nations negotiate to solve these global tragedies.

If half of the population decides to stop eating meat of industrially maltreated animals, that reduces the problem by half. A tragedy of the commons is different. If half of the population stops using fossil fuels, this will at best delay the climate problem a few years and in the worst case also delay the solution by the same time span.

In the end, the solution to global environmental problems will come from social changes. The incentives need to change so that people automatically make other choices and do so because that is the best choice for them too. If climate scientists would act as if climate change is a personal choice problem, rather than a tragedy of the commons, that may give the wrong impression.

Especially for climate change, the preferred behaviour will be even easier once the incentives have changed. Now it may be hard to live without a car where you live, but once the incentives change more people will use public transport and get on their bikes. As a consequence, the quality of public transport will increase and more biking lanes will be build, your office may get showers and the dress code may change. Society needs to change.


I do not think that flying less makes climate scientists more credible, nor that it makes a stronger case that climate change is real, nor that people will suddenly realise that 97% of climate scientists agree humans are causing climate change.

Simply put.
If climate scientists fly the mitigation sceptics will call them hypocrites.
If climate scientists do not fly the mitigation sceptics will call them activists.
As always, the best advice is to ignore what the unreasonable say, but it is not a priori clear which option makes climate scientists more credible in the eyes of the public.

There is also nothing special about being a climate scientist, my job is to study the reasons for non-climatic changes in historical observations and to develop algorithms to remove these changes. If I would not be blogging, I would not be thinking often about the economic and human costs of climate change. No more than someone else. I am not suddenly also an expert on all the different impacts of climate change. There might be a few public scientists that have this broad overview, most are simply doing their job and do not have a better idea of the problem as someone reading a decent newspaper.

If people chose to reduce their environmental impact that is great. Some people living like that may also make the transition easier by showing that the behaviour is possible and that you can live a happy life that way. However, anyone who worries about climate change can do that, that is not specific to climatologists.

On credibility. I dare to write this post because I travelled the 880 km to Vienna by train. In doing so only emitted a quarter of the CO2 of a comparable flight, if you believe the calculator of the rail company. I did so because I love going by train, you have more space, you can walk around, see the landscape slowly change. When you fly you have to wait in line all the time and are interrupted continually: get to airport 2 hours in advance, check-in, security, boarding, disembarking, get your luggage, travel to centre. Flying is a waste of your time and it is loud, while travelling by train is a great moment to read books and articles.

Culture change

At the moment we already have the big problem, mainly in the USA, that people who do not like the political solutions claim not to accept the science of climate change. They know so little about the science and about scientists, that we can be sure it is not that the science is lacking or that the scientists are not behaving right. I would see it as a major improvement if these people would no longer come up with specious "arguments" why a science is wrong that they hardly know anything about.

Then we could have an adult conversation and negotiate. The liberals get renewable energy, the conservatives aircraft carriers or whatever they want to have. An interesting way to shift the tax burden to carbon would be to let the US Republicans decide which tax should be lowered the same amount as the carbon tax brings in revenue.

The science and the political solutions are two different things. Someone may hold the opinion that the solutions are worse than the problem. I do not agree, but that is just personal opinion. That is comparing apples and oranges, which is the realm of politics, not science.

This culture does not get better by telling those people that if they accept the science, like 97% of climate scientists, that they should then also take personal consequences and fly less, like 97% of climate scientists.

George Marshall

In a recent TEDx video George Marshall (of Talking Climate) makes a similar case as the Tyndall Centre about personal responsibility. I do not agree with him, but he sure can tell it well and makes many important points. He tells about meeting an expert in marine biodiversity who orders a grilled swordfish, a climate scientist who flies to South America for a summer skying holiday, and about having a conversation with an animal rights activist while Marshall was eating the biggest US hamburger he had ever seen. After which he explains the psychology that enables this disconnect in human minds.

Related reading

Climate Central: Can climate scientists make a difference by not flying?

Tyndall Centre: Towards a culture of low-carbon research for the 21st Century

Also by George Marshall: How to talk to uncle Bob, the climate ostrich

Do dissenters like climate change?

Boomerang effects in science communication: how motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies

Monday, 6 April 2015

Changing your mind

The grave of Isaac Newton. Newton made himself vulnerable, wrote up his mechanics and optics very clearly. So that it could be falsified and was falsified. While he never repented his errors in public on facebook, his erroneous work did science a great service.

Being a scientist is hard. No, not for the mathematics. Being a scientist is nothing for wimps because you have to make yourself vulnerable, write up your ideas so clearly that someone else can show them to be wrong. That is the background of Karl Popper's requirement that scientific hypothesis should be falsifiable.

Being shown to be wrong is not nice. That could be one of the reasons why science is a very recent invention, it does not come to humans easily.

Being shown wrong is central, however, to scientific progress.

Being trivially wrong, of the kind displayed on WUWT every day, does not help science and is very bad for your scientific reputation.

Being wrong in an interesting way is scientific progress. The moment someone shows why you were interestingly wrong, we understand the problem a little better. If you can make your colleagues think and understand things better, that is good. Like Albert Einstein who was wrong about quantum mechanics until he died, but his challenges were interested and enhance his reputation.

Because it is hard to make yourself vulnerable, because it is good for science when you do so, because it is good for science when others show you wrong, the scientific community has a culture that encourages people to be bold and allows them to be wrong in an interesting way. If scientists would receive a hostile fit every time someone shows them wrong, that would not encourage them much to contribute to scientific understanding.


It starts with the language. Instead of saying someone was wrong, scientists talk about "progressive insight". Actually, it is a bit more than just language, often scientists really could not have known that they were wrong because some experiment or observation had not been made yet. Often this observation was inspired by their ideas. Often "progressive insight" also an argument that was overlooked and could theoretically have been thought up before. You could call that being wrong, but "progressive insight" is much more helpful.

Such a change of language is apparently also important in eduction. A high school teacher wrote:
One of the biggest challenges is convincing teenagers that they can be wrong about anything, let alone that there's value in it, ... Therefore, a few weeks ago I assigned my 90 or so Freshmen the task of keeping a 9-week Surprise Journal. ...

I've noticed something, well, surprising. In the class culture, acknowledgement that you are mistaken about something has become dubbed a "moment of surprise" (followed by a student scrambling to retrieve their Journal to record it). As this is much more value neutral than "I screwed up," the atmosphere surrounding the topic is less stressful than in previous years (I suspect -- based on anecdotal evidence -- that they have a history of being punished for mistakes and so are justifiably skittish about the whole topic). That by itself makes me inclined to judge this experiment a success, and has already added richness to our subsequent activities and discussions...
[[Nassim Taleb]], who wrote The Black Swan, mentioned that it is easier for Arabs to admit they do not know something, which is much better than pretending you do, because they express this by saying: only God knows. Much more neutral. (It is highly recommended for anyone in the climate "debate" to read one of his books on new and unpredictable risks and human tendencies to be blind to chance.)

This amicable scientific culture is also the reason that scientific articles are clear on the work performed, but very subtle in pointing our problems with previous studies, to the extent that sometimes only an expert can even notice that a previous study was criticized.

A less subtle example of trying to be amicable would be the article that made me "famous". A study comparing the skill of almost all homogenization method used to remove non-climatic changes. In the tables you can find the numbers showing which method performed how well. In the text we only recommended the application of the five best methods and did not write much about the less good ones. The reader is smart enough to make that connection himself.

At conferences I do explicitly state that some methods should better not be used any more. In private conversions I may be even more blunt. I feel this is a good general rule in life: praise people in writing, so that they can read it over and over again, criticize people in person, so that is can be forgotten again and you can judge whether the message had the right impact and is well understood. Easier said than done; the other way around is easier.

The bar between a stupid error and an interesting one goes up during the process. In a brainstorm with your close colleagues, you should be able to mention any idea, a stupid idea may inspire someone else to find a better one. At an internal talk about a new idea, it is okay if some peers or seniors notice the problem. If all peers or too many juniors see the problem, your reputation may suffer.

Similarly, if all people at a workshop see the problem, your reputation takes a hit, but if only one sees it. If someone from another field, who has knowledge that you do not, that should not be a problem; that is what conferences are for, to get feedback from people with other backgrounds. Only few people heard it, it will normally not be sufficiently important to talk about that with people who were not present and the mistake will be quickly forgotten. Like it should, if you want to encourage scientists taking risks.

There is a limit to this amicable culture. Making trivial mistakes hurts your reputation. Making again them after being been shown wrong hurts your reputation even more. Being regularly trivially wrong is where the fun stops. There is no excuse for doing bad science. Quality matters and do not let any mitigation skeptics tell you that they are attacked for the political implications of their work, it is the quality. I am very critical of homogenization and doubt studies on changes in extremes from daily station data, but I have good arguments for it and being skeptical certainly did not hurt me, on the contrary.

The difference between writing text, fixed for eternity, open for critique forever, and spoken language is also the reason why I am not too enthusiastic about tweeting conferences. On the other hand, to reduce travel it would be a nice if the talks of conferences were broadcasted over the internet. To reduce the tension between these two requirements maybe we could make something like [[snapchat]] for scientific talks. You can only look at it once (in the week of the conference) and then the video is deleted forever.

The above is the ideal case. Scientists are also humans. Especially when people keep on making similar wrong claims, when you can no longer speak about mistakes, but really about problems, also scientists can become less friendly. One case where I could understand this well was during a conference about long range dependence. This term is defined in such a vague way that you can not show that a signal does not have it. Thus a colleague asked whether a PhD student working on this had read Karl Popper? His professor answered: "First of all you have to believe in long range dependence."

For scientific progress it is not necessary for scientists to publicly pronounce that the denounce their old work. It is more than sufficient that they reference the new work (which without critique is interpreted as signalling that the reference is worth reading), that they adopt the new ideas, use the new methods and again build on them. The situation is also normally not that clear, scientific progress is a continual "negotiation" process while evidence is gathered. Once the dust settles and the evidence is clear, it would be trivial and silly to ask for a denouncement.

Calls for walks to walks to Canossa only hinder scientific progress.


In politics the situation is different. If politically helpful, the opponent changing his or her mind is put in a bad light and is called flip-flopping. If someone once made a mistake, very unhelpful calls for public retractions, repentance and walks to [[Canossa]] are made. Anything to humiliate the political opponent. Politics does not have to be this way, when I was young politicians in The Netherlands still made an effort to understand and convince each other. As sovereigns we should encourage this and reward it with our votes.

A funny example in the political climate "debate" is that the mitigation skeptic FoxGoose thought the could smear BBD by revealing the BBD had once changed his mind, that BBD was once a mitigation skeptic. He had not counted on the reaction of more scientifically minded people, who praised BBD for his strength to admit having been wrong and to leave the building of lies.
I discovered that I was being lied to. This simply by comparing the “sceptic” narrative with the standard version. Unlike my fellow “sceptics” I was still just barely sceptical enough (though sunk in denial) to check both versions. Once I realised what was going on, that was the end of BBD the lukewarmer (NB: I was never so far gone as to deny the basic physics, only to pretend that S [the climate sensitivity] was very low). All horribly embarrassing now, of course, but you live and learn. Or at least, some of us do. ...

Always check. Fail to do this in business and you will end up bankrupt and in the courts. I failed to check, at least initially, and made a colossal prat out of myself. Oh, and never underestimate the power of denial (aka ‘wishful thinking’). It’s brought down better people than me. ...

There wasn’t a single, defining eureka moment, just a growing sense of unease because nothing seemed to add up. ... Once I eventually started to compare WUWT [Watts Up With That] with RC [RealClimate] and SkS [Skeptical Science], that was it, really.
Mitigation skeptics sometimes state something along the lines: I would believe climate scientists if only they would denounce the hockey stick of Michael Mann as a hoax. Firstly, I simply do not believe this. The mitigation skeptics form a political movement. When mitigation skeptics do not even tell their peers that it is nonsense to deny that the CO2 increases are man made, they signal that politics is more important to them than science. For the worst transgressions at WUWT you will sometimes see people complaining that that will make their movement look bad. Such a strategic argument is probably the strongest argument to make at WUWT, but the right reason to reject nonsense is because it is nonsense.

Secondly, the call to denounce past science also ignores that methods to compute past temperatures have progressed, which is an implicit sign that past methods could naturally be improved. An explicit denouncement would not change the science. To keep on making scientific progress, we should keep the scientific culture of mild indirect criticism alive. The disgusting political attack on science in the USA are part of their cultural wars and should be solved by the Americans. That is not reason to change science. Maybe the Americans could start by simply talking to each other and noticing how much they have in common. Stopping to consume hateful radio, TV and blogs will make the quality your life a lot better.

Let me close with a beautiful analogy by Tom Curtis:
We do not consider the Wright brothers efforts as shoddy because their engines were under powered, their planes flimsy, and their controls rudimentary. To do so would be to forget where they stand in the history of aviation – to apply standards to pioneers that develop on the basis of mature knowledge and experience in the field. Everybody including Michael Mann is certain that, with hindsight, there are things MBH98 could have done better – but we only have that hindsight because they did it first. So, the proper gloss is not “shoddy”, but pioneering.

Related reading

Are debatable scientific questions debatable?
On the difference between scientific disputes and political debates.

Falsifiable and falsification in science
Scientific theories need to be falsifiable. Falsification is not trivial, however, and a perceived discrepancy a reason for further study to get a better understanding, not for a rejection of everything. That is politics.

How climatology treats sceptics
I have been very critical of homogenization. I had good arguments and was royally rewarded for my skepticism. The skeptics that cry persecution may want to have a second look at the quality of their "arguments".

Scientific meetings. The freedom to tweet and the freedom not to be tweeted

Stop all harassment of all scientists now

Climatology is a mature field

* Top photo of Newtons grave from Wikimedia is in the public domain.

Saturday, 4 April 2015

Irrigation and paint as reasons for a cooling bias

Irrigation pump in India 1944

In previous posts on reasons why raw temperature data may show too little global warming I have examined improvements in the siting of stations, improvements in the protection of thermometers against the sun, and moves of urban stations to better locations, in particular to airports. This post will be about the influence of irrigation and watering, as well as improvements in the paints used for thermometer screens.

Irrigation and watering

Irrigation can decrease air temperature by up to 5 degrees and typically decreases the temperature by about 1°C (Cook et al., 2014). Because of irrigation more solar energy is used for evaporation and for transpiration by the plants, rather than for warming of the soil and air.

Over the last century we have seen a large 5 to 6 fold global increase in irrigation; see graph below.

The warming by the Urban Heat Island (UHI) is real. The reason we speak of a possible trend bias due to increases in the UHI is that an urban area has a higher probability of siting a weather station than rural areas. If only for the simple reason that that is where people live and want information on the weather.

The cooling due to increases in irrigation are also real. It seems to be a reasonable assumption that an irrigated area again has a higher probability of siting a weather station. People are more likely to live in irrigated areas and many weather stations are deployed to serve agriculture. While urbanization is a reason for stations to move to better locations, irrigation is no reason for a station to move away. On the contrary maybe even.

The author of the above dataset showing increases in irrigation, Stefan Siebert, writes: "Small irrigation areas are spread across almost all populated areas of the world." You can see this strong relation between irrigation and population on a large scale in the map below. It seems likely that this is also true on local scales.

Many stations are also in suburbs and these are likely watered more than they were in the past when water (energy) was more expensive or people even had to use hand pumps. In the same way as irrigation, watering could produce a cool bias due to more evaporation. Suburbs may thus be even cooler than the surrounding rural areas if there is no irrigation. Does anyone know of any literature about this?

I know of one station in Spain where the ground is watered to comply with WMO guidelines that weather stations should be installed on grass. The surrounding is dry and bare, but the station is lush and green. This could also cause a temperature trend bias under the reasonable assumption that this is a new idea. If anyone knows more about such stations, please let me know.

From whitewash to latex paint

Also the maintenance of the weather station can be important. Over the years better materials and paints may have been used for thermometer screens. If this makes the screens more white, they heat up less and they heat up the air flowing through the Louvres less. More regular cleaning and painting would have the same effect. It is possible that this has improved when climate change made weather services aware that high measurement accuracies are important. Unfortunately, it is also possible that good maintenance is nowadays seen as inefficient.

The mitigation skeptics somehow thought that the effect would go into the other direction. That the bad paints used in the past would be a cooling bias, rather than a warming bias. Something with infra-red albedo. Although most materials used have about the same infra-red albedo and the infra-red radiation fluxes are much smaller than the solar fluxes.

Anthony Watts started a paint experiment in his back garden in July 2007. The first picture below shows three Stevenson screens, a bare one, a screen with modern latex paint and one with whitewash, a chalk paint that quickly fades.

Already 5 months later in December 2007, the whitewash had deteriorated considerably; see below. This should lead to a warm bias for the whitewash screen, especially in summer.

Anthony Watts:
Compare the photo of the whitewash paint screen on 7/13/07 when it was new with one taken today on 12/27/07. No wonder the NWS dumped whitewash as the spec in the 70’s in favor of latex paint. Notice that the Latex painted shelter still looks good today while the Whitewashed shelter is already deteriorating.

In any event the statement of Patrick Michaels “Weather equipment is very high-maintenance. The standard temperature shelter is painted white. If the paint wears or discolors, the shelter absorbs more of the sun’s heat and the thermometer inside will read artificially high.” seems like a realistic statement in light of the photos above.
I have not seen any data from this experiment beyond a plot with one day of temperatures, which was a day one month after the start, showing no clear differences between the Stevenson screens. They were all up to 1°C warmer than the modern ventilated automatic weather station when the sun was shining. (That the most modern ventilated measurement had a cool bias was not emphasized in the article, as you can imagine.) Given that Anthony Watts maintains a stealth political blog against mitigation of climate change, I guess we can conclude that he probably did not like the results, that the old white wash screen was warmer and he did not want to publish that.

We may be able to make a rough estimate the size of the effect by looking at another experiment with a bad screen. In sunny Italy Giuseppina Lopardo and colleagues compared two old aged, yellowed and cracked screens of unventilated automatic weather stations that should have been replaced long ago with a good new screen. The picture to the right shows the screen after 3 years. They found a difference of 0.25°C after 3 years and 0.32°C after 5 years.

The main caveat is that the information on the whitewash comes from Anthony Watts. It may thus well misinformation that the American Weather Bureau used whitewash in the past. Lacquer paints are probably as old as 8000 years and I see no reason to use whitewash for a small and important weather screen. If anyone has a reliable source about paints used in the past, either inside or outside the USA, I would be very grateful.

Related posts

Changes in screen design leading to temperature trend biases

Temperature bias from the village heat island

Temperature trend biases due to urbanization and siting quality changes

Climatologists have manipulated data to REDUCE global warming

Homogenisation of monthly and annual data from surface stations


Cook, B.I., S.P. Shukla, M.J. Puma, L.S. Nazarenko, 2014: Irrigation as an historical climate forcing. Climate Dynamics, 10.1007/s00382-014-2204-7.

Siebert, Stefan, Jippe Hoogeveen, Petra Döll, Jean-Marc Faurès, Sebastian Feick and Karen Frenken, 2006: The Digital Global Map of Irrigation Areas – Development and Validation of Map Version 4. Conference on International Agricultural Research for Development. Tropentag 2006, University of Bonn, October 11-13, 2006.

Siebert, S., Kummu, M., Porkka, M., Döll, P., Ramankutty, N., and Scanlon, B.R., 2015: A global data set of the extent of irrigated land from 1900 to 2005. Hydrology and Earth System Sciences, 19, pp. 1521-1545, doi: 10.5194/hess-19-1521-2015.

See also: Zhou, D., D. Li, G. Sun, L. Zhang, Y. Liu, and L. Hao (2016), Contrasting effects of urbanization and agriculture on surface temperature in eastern China, J. Geophys. Res. Atmos., 121, doi: 10.1002/2016JD025359.