Analog Forecasting: a Flexible and Parsimonious Alternative for Nonlinear Prediction

Features

  • Author: Patrick L. McDermott & Christopher K. Wikle
  • Date: 02 Aug 2016
  • Copyright: Image appears courtesy of Getty Images

Many complex spatio-temporal processes in the natural world evolve in a nonlinear fashion. As researchers attempt to forecast increasingly complex systems, there is a growing need to use so-called “mechanism-free models.” Traditional linear forecasting models may struggle to accurately forecast these types of complex nonlinear systems at even small time scales. In addition, parametric nonlinear statistical spatio-temporal dynamical models, such as those developed in Wikle and Hooten (2010) and summarized in Wikle (2015), as well as so-called “deep learning” methods such as recurrent neural networks, can suffer from a serious curse of dimensionality in the parameter space. This can make implementation difficult.

First used in the 1920s heuristically by operational weather forecasters, and re-popularized by Lorenz (1969), analog forecasting is a nonlinear method that has proven successful in a variety of fields (e.g., Zhao and Giannakis, 2014; McDermott and Wikle, 2016; Sugihara and May, 1990; Arora et al., 2013). The flexible nature of analog forecasting allows one to easily obtain dynamically realistic forecasts with large spatio-temporal datasets while only using a relatively small number of parameters.

thumbnail image: Analog Forecasting: a Flexible and Parsimonious Alternative for Nonlinear Prediction

Simple in nature, analog forecasting involves identifying past trajectories of the state of a system (i.e., analogs) that are similar to the current state of the system (commonly referred to as the initial condition). The current state of the system is then forecasted forward by assuming that the initial condition will evolve forward along a similar trajectory as the identified analogs from the past. In general, analog forecasting is most successful for systems that are governed by some underlying deterministic laws that lead the system to re-visit the same trajectories over time (see Figure 1). The Lorenz attractor (displayed in Figure 1) is an example of a dynamical system where analog forecasting has been demonstrated to work quite well. A key element of analog forecasting involves a famous theory from the study of dynamical systems due to Takens (1981). Although it is rarely possible to observe all of the state variables in a system, Takens’ theorem allows one to reconstruct a system through the use of so-called time delayed embedding vectors (e.g., see Chapter 3 of Cressie and Wikle, 2011), with only a subset of the state variables (i.e., state-space reconstruction). Further, state-space reconstruction is particularly useful for systems that tend to move along a particular subset of the entire phase space. As an illustrative example, suppose we have a univariate time series〖 {x_t }〗_(t=1)^T that we wish to forecast forward s periods starting from period T (i.e., x_T is the initial condition and we want to forecast x_(T+s)). We can construct an embedding vector such as

                                                                                z_t={x_T,x_(T-τ),x_(T-2τ) }^',

which contains the initial condition and the state lagged τ and 2τ periods. The essence of analog forecasting involves creating these embedding vectors for every period in the past and finding the vectors most similar to the embedding vector for the initial condition (i.e., z_t shown above). Generally, one then takes a weighted average of the future values from the identified (past) analogs (i.e., the values of the identified analogs s periods forward) to forecast the initial condition into the future.

While originally devised for short-term forecasting in the atmospheric sciences (e.g., see McDermott and Wikle, 2016, for an overview), analog forecasting has more recently been applied to short-term ecological forecasting (Perretti et al., 2013), as well as downscaling (Zorita and Von Storch, 1999) and data assimilation (Lguensat et al., 2016) in the atmospheric sciences. The first analog method to incorporate the ideas from Takens’ theorem was presented in Sugihara and May (1990), although their work focused on univariate time series. Moreover, the linear Constructed Analog (CA) method introduced in van den Dool (1994) was one of the first formal analog methods designed to forecast spatio-temporal data. The flexible and simple framework of analog forecasting makes it a particular appealing tool for modeling nonlinear dynamical systems that evolve over space and time. Nonlinear spatio-temporal models are often over parameterized and computationally burdensome, whereas analog forecasting only requires a small number of parameters and can be implemented with a relatively low computational cost.

Figure 1: Example of the famous Lorenz attractor in 3 dimensions. Each of the 3 points represents an embedding vector (e.g., Sugihara and May (1990)). The blue dot represents the initial condition and the two green dots are the analogs for the initial condition. Analog forecasting assumes the initial condition moves forward in a way similar to its analogs.

Until recently, nearly all of the research surrounding analog forecasting utilized an empirical framework, void of rigorous statistical modeling. The recent work of Zhao and Giannakis (2014) details a formal spatio-temporal analog forecasting method that is formed by using so- called “out-of-sample extension” techniques and dynamic dependent kernels, for the purpose of forecasting atmospheric dynamics. While rigorous in nature, the methods in Zhao and Giannakis (2014) are presented from more of a mathematical perspective than a traditional statistical perspective, and is thus not concerned with uncertainty quantification. To our knowledge, McDermott and Wikle (2016) presented the first analog forecasting model within a rigorous Bayesian statistical framework. Once placed within a Bayesian framework, analog forecasting becomes very appealing not only as an accurate forecasting model, but as a model to help practitioners properly quantify the certainty of various forecasts. In addition, through the use of dimension reduction techniques, analog models can easily handle large spatio-temporal datasets, while only employing a few number of parameters.

When developing an analog model, there are three main choices in determining the success of the model: the length of the embedding vectors, the number of analogs with which to apply weights, and how to determine these weights for the identified analogs. Finding robust analogs is critical to the success of any analog forecasting method; thus, the choice of how to form and compare embedding vectors can drastically affect the quality of the forecasts. Past analog methods have typically employed Euclidean distance to compare analogs, although, in practice, any valid distance metric can be used (e.g., see McDermott and Wikle, 2016). Furthermore, one can think of selecting the number of past analogs to weigh as a k-nearest neighbors problem. Finally, the identified analogs are traditionally weighed with a (possible parameter dependent) kernel function. As outlined in Zhao and Giannakis (2014), the choice of which kernel function to use is still a topic of research. Traditionally, analog forecasting methods have used heuristic rules to select these parameters that determine the analog model, instead of estimating the parameters outlined above. The Bayesian approach allows one to formalize the estimation of these parameters.

Figure 2: Illustrative example of using the analog approach to forecast 2005 mid-May averaged soil moisture anomalies in Iowa, USA, with a lead-time of 6 months using sea surface temperature (SST)-based analogs (from McDermott and Wikle, (2016)). Displayed are the four nearest neighbors to the SST embedding sequence for the initial condition (starting at 6 months prior to May 2005). Each nearest neighbor SST embedding sequence corresponds to a past soil moisture realization that can be weighted to construct a forecast for the period of interest (May 2005). Note how past SST embedding sequences (row 2 through 5), with a similar trajectory to the initial condition (top row), also have similar soil moisture anomaly fields 6 months in the future.

To demonstrate the power of analog forecasting we will briefly discuss two examples involving forecasts on real-world data. Both examples use Sea Surface Temperature (SST) from the Pacific Ocean to identify potential analogs. In particular, the SST dataset is comprised of monthly anomalies for 3,132 locations between 30.5◦S-60.5◦N latitude and 123.5◦E-290.5◦E longitude from the publicly available National Oceanic and Atmospheric Extended Reconstruction Sea Surface Temperature (ERSST) data. Furthermore, the first example involves forecasting mid-May soil moisture in Iowa over 4,053 locations by using the current trajectory of the SST in the Pacific Ocean starting in November of the previous year (see Figure 2). SST is employed as a predictor of soil moisture due to the known atmospheric teleconnective response between precipitation in North American and SST in the Pacific (e.g., Hoerling et al., 1997). By taking a weighted average of soil moisture values from past years that have SST trajectories (starting 6 months prior to the month of interest) similar to the current trajectory (as show in Figure 2), one can produce forecasts that improve on both standard climatology forecasts and linear time series models (e.g., see McDermott and Wikle, 2016).

Next, we illustrate an ecological example involving the forecasting of waterfowl settling patterns at a lead-time of 12 months. The waterfowl data comes from the Breeding Population Survey (BPS) conducted each spring (mid to late May) by the U.S. Fisheries and Wildlife Service (USFWS) in the northern United States and Canada. Due to the known relationship between habitat conditions and settling patterns (e.g., Hansen and McKnight, 1964), SST is again utilized to find past analogs. Using a model similar in nature to the Bayesian analog model outlined in McDermott and Wikle (2016), the posterior prediction and uncertainty maps for waterfowl counts in 2011 are displayed in Figure 3. These maps highlight the predictive power of analog forecasting within a statistical framework.

Figure 3: Summary of the posterior predictive results for the waterfowl application. (a) Observed waterfowl counts, (b) mean of the posterior predictive distribution, (c) lower 2.5th percentile from the posterior predictive distribution, (d) upper 2.5th percentile form the posterior

There are a vast number of potential ways analog forecasting can be extended and integrated with other scientific and statistical models. For example, analog models can easily be incorporated as a stage in a Bayesian hierarchical model (e.g., see Cressie and Wikle 2011 for a discussion of Bayesian hierarchical models for spatio-temporal data). Further, analog forecasting models can be combined with more traditional statistical forecasting models by integrating the two forecasting methods with a mixture model. To date, analog forecasting has only been applied to a small subset of scientific problems. There are many problems in areas such as ecology, environmental sciences, and atmospheric sciences where analog forecasting could be useful. As we continue to solve more difficult problems, a simple and flexible method such as analog forecasting can be tremendously useful.


References
Arora, S., Little, M., and McSharry, P. (2013), “Nonlinear and nonparametric modeling approaches for probabilistic forecasting of the US gross national product,” Studies in Nonlinear Dynamics and Econometrics, 17, 395–420.
Cressie, N. and Wikle, C. (2011), Statistics for Spatio-Temporal Data, New York: John Wiley & Sons.
Hansen, H. A. and McKnight, D. E. (1964), “Emigration of drought-displaced ducks to the Arctic,” in Transactions of the North American Wildlife and Natural Resources Conference, vol. 29, pp. 119–127.
Hoerling, M. P., Kumar, A., and Zhong, M. (1997), “El Nin ̃o, La Nin ̃a, and the nonlinearity of their teleconnections,” Journal of Climate, 10, 1769–1786.
Lguensat, R., Tandeo, P., Ailliot, P., Chapron, B., and Fablet, R. (2016), “Using archived datasets for missing data interpolation in ocean remote sensing observation series,” in OCEANS 2016 - Shanghai.
Lorenz, E. N. (1969), “Atmospheric predictability as revealed by naturally occurring analogues,” Journal of the Atmospheric sciences, 26, 636–646.
McDermott, P. and Wikle, C. (2016), “A model based approach for analog spatio-temporal dynamic forecasting,” Environmetrics.
Perretti, C., Sugihara, G., and Munch, S. (2013), “Nonparameter forecasting outperforms parameter methods for simulated multispecies system,” Ecology, 94, 794–800.
Sugihara, G. and May, R. (1990), “Nonlinear forecasting as a way of distinguishing from measurement error in time series,” Nature, 344, 734–741.
van den Dool, H. (1994), “Searching for Analogues, how long must we wait?” Tellus A, 46, 314–324.
Zhao, Z. and Giannakis, D. (2014), “Analog forecasting with dynamics-adapted kernels,” Nonlinearity, in review.
Zorita, E. and Von Storch, H. (1999), “The analog method as a simple statistical downscaling technique: comparison with more complicated methods,” Journal of climate, 12, 2474–2489.

Patrick L. McDermott & Christopher K. Wikle, Department of Statistics, University of Missouri, Columbia, MO 65211 U.S.A

Related Topics

Related Publications

Related Content

Site Footer

Address:

This website is provided by John Wiley & Sons Limited, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ (Company No: 00641132, VAT No: 376766987)

Published features on StatisticsViews.com are checked for statistical accuracy by a panel from the European Network for Business and Industrial Statistics (ENBIS)   to whom Wiley and StatisticsViews.com express their gratitude. This panel are: Ron Kenett, David Steinberg, Shirley Coleman, Irena Ograjenšek, Fabrizio Ruggeri, Rainer Göb, Philippe Castagliola, Xavier Tort-Martorell, Bart De Ketelaere, Antonio Pievatolo, Martina Vandebroek, Lance Mitchell, Gilbert Saporta, Helmut Waldl and Stelios Psarakis.