Climate of the Past Discussions
www.clim-past-discuss.net
1814-9340
1814-9359
3
1
2007
10.5194/cpd-3-249-2007
http://www.clim-past-discuss.net/3/249/2007/
http://www.clim-past-discuss.net/3/249/2007/cpd-3-249-2007.html
http://www.clim-past-discuss.net/3/249/2007/cpd-3-249-2007.pdf
249
284
2007-01-31
On the verification of climate reconstructions
G. Bürger
gerd.buerger@met.fu-berlin.de
FU-Berlin, Institut für Meteorologie; Carl-Heinrich-Becker-Weg 6–10, 12165 Berlin, Germany
The skill of proxy-based reconstructions of Northern hemisphere
temperature is reassessed. Using an almost complete set of proxy and
instrumental data of the past 130 years a multi-crossvalidation is
conducted of a number of statistical methods, producing a distribution
of verification skill scores. The scores show considerable variation
for all methods, but previous estimates, such as a 50% reduction of
error (<i>RE</i>), appear as outliers and more realistic estimates vary
about 25%. It is shown that the overestimation of skill is possible
in the presence of strong persistence (trends). In that case, the
classical "early" or "late" calibration sets are not
representative for the intended (instrumental, millennial) domain. As
a consequence, <i>RE</i> scores are generally inflated, and the proxy
predictions are easily outperformed by random-based, a priori
skill-less predictions.
<br><br>
To obtain robust significance levels the multi-crossvalidation is
repeated using predictors based on red noise. Comparing both
distributions, it turns out that the proxies perform significantly
better for almost all methods. The nonsense predictor scores do not
vanish, nonetheless, with an estimated 10% of spurious skill based on
representative samples. I argue that this residual score is due to the
limited sample size of 130 years, where the memory of the processes
degrades the independence of calibration and validation sets. It is
likely that proxy prediction scores are inflated correspondingly, and
have to be adjusted further.
The consequences of the limited verification skill for millennial
reconstructions is briefly discussed.
Aldrich, J.: Correlations Genuine and Spurious in Pearson and Yule, Discussion Paper Series In Economics And Econometrics 9502, Economics Division, School of Social Sciences, University of Southampton, available at: http://ideas.repec.org/p/stn/sotoec/9502.html, 1995.
Bhansali, R J. and Kokoszka, P S.: Estimation of the long memory parameter: a review of recent developments and an extension, in: Selected proceedings of the symposium on inference for stochastic processes. IMS Lecture notes and monograph series, edited by: Basawa., I., Heyde, C C., and Taylor, R., 125–150, Institute of Mathematical Statistics, Ohio, USA, 2001.
Briffa, K R.: Annual climate variability in the holocene: intepreting the message of ancient trees, Quat. Sci. Rev., 19, 87–105, 2000.
Briffa, K R., Jones, P D., Pilcher, J R., and Hughes, M K.: Reconstructing Summer Temperatures in Northern Fennoscandinavia Back to A.D.1700 Using Tree Ring Data from Scots Pine, Arctic and Alpine Research, 385–94, 1988.
Briffa, K R., Bartholin, T S., Eckstein, D., Jones, P D., Karlen, W., Schweingruber, F H., and Zetterberg, P.: A 1,400-year tree-ring record of summer temperatures in fennoscandia, Nature, 346, pp. 434–439, 1990.
Briffa, K R., Jones, P D., and Schweingruber, F H.: Tree-ring density reconstructions of summer temperature patterns across western north america since 1600, J. Climate, 5, 735–754, 1992.
Briffa, K R., Osborn, T J., Schweingruber, F H., Harris, I C., Jones, P D., Shiyatov, S G., and Vaganov, E A.: Low-frequency temperature variations from a northern tree ring density network, J. Geophys. Res., 106(D3), 2929–2941, 2001.
Brockwell, P J. and Davis, R A.: Time series: theory and methods, 2nd edition, Springer Series in Statistics, 1998.
Bürger, G. and Cubasch, U.: Are multiproxy climate reconstructions robust?, Geophys. Res. Lett., L23711, doi:10.1029/2005GL0241 550, 2005.
Bürger, G., Fast, I., and Cubasch, U.: Climate reconstruction by regression – 32 variations on a theme, Tellus A, 227–35, 2006.
Cattin, P.: Estimation of the predictive power of a regression model, J. Appl. Psychol., 65(4), 407–414, 1980.
Cook, E R., Briffa, K R., and Jones, P D.: Spatial regression methods in dendroclimatology: a review and comparison of two techniques, Int. J. Clim., 379–402, 1994.
Cook, E R., Buckley, B M., D'Arrigo, R D., and Peterson, M J.: Warm-season temperatures since 1600 bc reconstructed from tasmanian tree rings and their relationship to large-scale sea surface temperature anomalies., Clim. Dyn., 16, 79–91, 2000.
Cooley, W W. and Lohnes, P R.: Multivariate data analysis, New York: Wiley, 1971.
Crowley, T J. and Lowery, T S.: How warm was the medieval warm period?, Ambio, 29, 54, 2000.
Dempster, A., Laird, N., and Rubin, D.: Maximum likelihood estimation from incomplete data via the EM algorithm, J. Royal Statist. Soc., B, 39, 1–38, 1977.
Efron, B.: Bootstrap methods: another look at the jackknife, Annals of Statistics, 17, 1–26, 1979.
Efron, B. and Gong, G.: A Leisurely Look at the Bootstrap, the Jackknife, and Cross-Validation, American Statistician, 36–48, 1983.
Esper, J., Cook, E R., and Schweingruber, F H.: Low frequency signals in long tree-ring chronologies for reconstructing past temperature variability, Science, 295, 2250–2253, 2002.
Esper, J., Frank, D C., Wilson, R. J S., and Briffa, K R.: Effect of scaling and regression on reconstructed temperature amplitude for the past millennium, Geophysical Research Letters, Volume 32, Issue 7, 32(7), L07711, 2005.
Evans, M N., Kaplan, A., and Cane, M A.: Pacific sea surface temperature field reconstruction from coral delta o-18 data using reduced space objective analysis, Paleoceanography, 17, 1007, 2002.
Fritts, H C.: Tree rings and climate, Academic Press, 1976.
Fritts, H C. and Guiot, J.: Methods of calibration, verification, and reconstruction, in: Methods Of Dendrochronology. Applications In The Environmental Sciences, edited by: Cook, E R. and Kairiukstis, L A., 163–217, Kluwer Academic Publishers, 1990.
Geweke, J. and Porter-Hudak, S.: The estimation and application of long-memory time series models, J. Time Series Analysis, 4, 221–238, 1983.
Golub, G H. and Loan, C. F V.: Matrix computations (3rd ed.), Johns Hopkins University Press, Baltimore MD USA, 1996.
Guiot, J., Nicault, A., Rathgeber, C., Edouard, J L., Guibal, E., Pichard, G., and Till, C.: Last-millennium summer-temperature variations in western europe based on proxy data, Holocene, 15, 500, 2005.
Hoerl, A E.: Application of ridge analysis to regression problems, Chem. Eng. Prog., 58, 54–59, 1962.
Hosking, J.: Modeling persistence in hydrological time series using fractional differencing, Water Resour. Res., 20(12), 1898–1908, 1984.
Huybers, P.: Comment on \textquotesingle\textquotesingleHockey sticks, principal components, and spurious significance\textquotesingle\textquotesingle, Geophys. Res. Lett., L20705, doi:10.1029/2005GL023395, 2005.
IPCC: Climate change 2001: the scientific basis. contribution of working group I to the third assessment report of the intergovernmental panel on climate change, Cambridge University Press, Cambridge, 2001.
Jones, P D., Briffa, K R., Barnett, T P., and Tett, S. F B.: High-Resolution Palaeoclimatic Records for the Last Millennium: Interpretation, Integration and Comparison with General Circulation Model Control-Run Temperatures, Holocene, 8, 455–71, 1998.
Krus, D J. and Fuller, E A.: Computer assisted multicrossvalidation in regression analysis, Educational and Psychological Measurement, 42, 187–193, 1982.
Little, R. J A. and Rubin, D B.: Statistical analysis with missing data, Wiley, 1987.
Lorenz, E N.: Empirical orthogonal functions and statistical weather prediction, Sci. Rept. No. 1, Dept. of Met., M. I. T., p. 49pp, 1956.
Luterbacher, J., Xoplaki, E., Dietrich, D., Rickli, R., Jacobeit, J., Beck, C., Gyalistras, D., Schmutz, C., and Wanner, H.: Reconstruction of sea level pressure fields over the eastern north atlantic and europe back to 1500, CLIMATE DYNAMICS, 18, 545–561, 2002.
Mann, M E. and Rutherford, S.: Climate reconstruction using \textquotesinglePseudoproxies\textquotesingle, Geophys. Res. Lett., p. 139, 2002.
Mann, M E., Bradley, R S., and Hughes, M K.: Global-scale temperature patterns and climate forcing over the past six centuries, Nature, 779–87, 1998.
Mann, M E., Bradley, R S., and Hughes, M K.: Northern hemisphere temperatures during the past millennium: inferences, uncertainties, and limitations, Geophys. Res. Lett., 759–762, 1999.
Mann, M E., Rutherford, S., Wahl, E., and Ammann, C.: Testing the Fidelity of Methods Used in Proxy-Based Reconstructions of Past Climate, J. Climate, 4097–107, 2005.
McIntyre, S. and McKitrick, R.: Corrections to the Mann et al (1998) proxy data base and northern hemispheric average temperature series, Energy Environ., 14(6), 751–771, 2003.
McIntyre, S. and McKitrick, R.: Hockey sticks, principal components and spurious significance, Geoph. Res. Let., 32, L03710, doi:10.1029/2004GL021750, 2005a.
McIntyre, S. and McKitrick, R.: Reply to comment by huybers on \textquotesingle\textquotesinglehockey sticks, principal components, and spurious significance\textquotesingle\textquotesingle, Geophys. Res. Lett., 32, L20713, doi:10.1029/2005GL023586, 2005b.
Moberg, A., Sonechkin, D M., Holmgren, K., Datsenko, N M., and Karlen, W.: Highly variable northern hemisphere temperatures reconstructed from low- and high-resolution proxy data, Nature, 433, 617, 2005.
Nash, J E. and Sutcliffe, J V.: River flow forecasting through conceptual models - Part I - A discussion of principles, J. Hydrol., (10), 282–90, 1970.
Nicholson, G E.: Prediction in future samples, in: Contributions to Probability and Statistics, edited by: Olkin, I., 322–330, 1960.
Overpeck, J., Hughen, K., Hardy, D., Bradley, R., Case, R., Douglas, M., Finney, B., Gajewski, K., Jacoby, G., Jennings, A., Lamoureux, S., Lasca, A., MacDonald, G., Moore, J., Retelle, M., Smith, S., Wolfe, A., and Zielinski, G.: Arctic Environmental Change of the Last Four Centuries, Science, 278, 1251–1256, 1997.
Pearson, K.: Mathematical Contributions to the Theory of Evolution – On a Form of Spurious Correlation Which May Arise When Indices Are Used in the Measurement of Organs, Proc. R. Soc., 60, 489–498, 1897.
Raju, N S., Bilgic, R., Edwards, J E., and Fleer, P F.: Methodology review: estimation of population validity and cross-validity, and the use of equal weights in prediction, J Appl. Psychol. Measurement, 21(4), 291–305, 1997.
Rubin, D B.: Inference and missing data, Biometrika, 63, 581–592, 1976.
Rutherford, S., Mann, M E., Delworth, T L., and Stouffer, R J.: Climate field reconstruction under stationary and nonstationary forcing, J. Climate, 16, 462–479, 2003.
Rutherford, S., Mann, M E., Osborn, T J., Bradley, R S., Briffa, K R., Hughes, M K., and Jones, P D.: Northern Hemisphere Surface Temperature Reconstructions: Sensitivity to Methodology, Predictor Network, Target Season and Target Domain, J. Climate, 2308–29, 2005.
Schneider, T.: Analysis of incomplete climate data: estimation of mean values and covariance matrices and imputation of missing values, J. Climate, 14, 853–871, 2001.
von Storch, H., Zorita, E., Jones, J M., Dmitriev, Y., and Tett, S. F B.: Reconstructing Past Climate from Noisy Data, Science, 679–82, 2004.
Wahl, E R. and Ammann, C M.: Robustness of the Mann, Bradley, Hughes reconstruction of Northern hemisphere surface temperatures: Examination of criticisms based on the nature and processing of proxy climate evidence, Climatic Change, in press, 2007.
Wahl, E R., Ritson, D M., and Ammann, C M.: Comment on "Reconstructing past climate from noisy data", Science, 312, p. 529b, http://www.sciencemag.org/cgi/content/abstract/312/5773/529b, doi:10.1126/science.1120866, 2006.
Wilks, D S.: Statistical methods in the atmospheric sciences. an introduction, Academic Press, San Diego, 1995.
Yule, G U.: Why do we sometimes get nonsense-correlations between time-series? – A study in sampling and the nature of time-series, J. Roy. Stat. Soc., 1–29, 1926.