Thursday, May 01, 2008

Another decadal prediction...

...is appearing in Nature tomorrow, accompanied by some rather odd press (and blog) coverage. The paper itself also seems a little odd to me (but not as odd as the coverage). The authors have nudged sea surface temperatures to observations, which is probably the simplest plausible first step in coupled model initialisation (and I think has been used in seasonal prediction for some time). Mostly, they are looking at regional results (and predictions), especially the North Atlantic, but they also include some global analysis. For this, they are predicting very little change in mean temperature for the next few years, after which the trend will revert strongly upwards. Roger Pielke somehow saw "the world may cool over the next 20 years" which seems comprehensively contradicted by what the paper actually shows in their Fig 4. I've posted a comment on his blog.

Curiously, the global temperature prediction is contained in a graph which also appears to show (confirmed in the supplementary info) that the free-running model integrations actually tracked historical temperatures better than the nudged ones. This does not look like a good sign to me! In fact they only seem to show that their system has some skill in representing the phase of natural oscillations in some areas, and (unless I've missed something) never actually claim that it has any skill at predicting mean global temperatures (hence their use of correlations rather than the more usual RMS errors). They also have a rather odd graph of the IPCC results, which seems to imply that these models predicted a ~0.3C mean rise over the current decade (and the lead author quotes that value in the Telegraph article). I have not read the paper carefully enough to work out how they managed that, since as I just showed yesterday, the IPCC models on average generate a very linear response (and it's just under 0.2C/decade currently). It may be something to do with how they splice the 20th century simulation on to the A1B scenario projection - eg there could be an abrupt change in forcings, since the scenarios originally started in 1990. But anyway, that doesn't seem like a very fair comparison.

I suppose I should interpret this paper as implying that I'm less likely to win my bet than I thought yesterday. It would be biased of me to cling to Smith et al. as support and simply choose to ignore this less convenient one. But I honestly think Smith et al did a better job at demonstrating the value of their method, especially for predicting global temperatures over the 5 year time scale which is most directly relevant to me. It does seem that there is (almost certainly) a strong contradiction between the results, so one of them has to be wrong!

10 comments:

skanky said...

I may be looking at this wrong (I can't see the full article), but that jump in the RF/A1B line is a little odd. It seems to account for the main difference in the two lines, until 2010, when their forecast accelerates to catch up.

Using the 2007 CRU anomaly of 0.4 the temp. line extended looks like it would lie between the two projections. If they have the IPCC one artificially high, then it would be looking better than theirs.

William M. Connolley said...

"They also have a rather odd graph of the IPCC results, which seems to imply that these models predicted a ~0.3C mean rise over the current decade..."

Are those IPCC models (I assume you mean the black line in fig 4). Or their own?

James Annan said...

I don't know where they get it from, but Keenlyside is directly quoted in the Torygraph as "The IPCC would predict a 0.3°C warming over the next decade" which is flatly false and directly contradicted by the IPCC text.

If those free-running integrations really generate a jump as shown in their figure then it is either an artefact of their model (massive response to sudden reduction in aerosols?) or just noise due to an insufficient ensemble size. Either way, it's hard to support.

John Fleck said...

The Nature effect!

James Annan said...

Good grief what's up with the linkspam below..looks like someone is "borrowing" the scienceblogs output.

Anyway, I'm really not sure how much to blame the Nature effect. Of course it has had more attention than it warrants, but it seems to me that the authors carry plenty of responsibility for how it has been represented. Although as I said, I should be careful not to be too critical of it. You only have to look at Roger Pielke's antics to see how easy it is to fall into the trap of viewing things through the prism of one's own self-interest.

bi -- International Journal of Inactivism said...

What's this "bet", and where can I read about it?

-- bi, International Journal of Inactivism

James Annan said...

Frank,

Try here and here (and also the linked radio program).

Basically, I reckon we'll see the 1998 record broken by 2011, and David Whitehouse thinks we won't. However, he has not returned my email...

Alf said...

I have always found hindcast/forecast plots a little hard to comprehend. Am I right in thinking that each forecast should be compared to the observations 10 years before?

So the hindcast made in 1994 said little change (the globe actually warmed), and the forecasts made in 2000 and 2005 predict significant cooling (I don't think temperatures have dropped below the ten year mean before 2000). It would have been good to have put the annual values on their plot so we could see how those forecasts are doing, as you have pointed out it looks like their RF runs are better at predicting what will happen after 1994.

Alf said...

Keenlyside's figure 3 is also a bit odd. Is it normal when comparing models to observations and other models to show different axis ranges as is done in 3a and 3b?

James Annan said...

Here's a more thoroughly cynical take on it:

Their nudging method failed completely by most standard measures (eg RMS error). In an attempt to salvage something out of the work, they noticed that there is some synchronicity in the cyclical behaviour in some regions. That's an interesting result but hardly justifies so much attention. Given the length of the cycle (ie they only have one!) it is more of a weak hypothesis than a robust result. Moreover, the strength of the cycles is rather different (hence the different scales you noticed).

Most strikingly of all, their nudged model displays a clearly worse agreement with global mean temperature, so they have a lot of chutzpah to present their "forecast" of global temperatures and I think I am quite justified in not taking it too seriously. As Stoat pointed out, it was already shooting off in the wrong direction 10 years ago, according to their figure.

It will be interesting to see what spin RC puts on it - I'm sure they must be preparing a post.