Mathematical Genesis of the Spatio-Temporal Covariance Functions

Problem statement: Obtaining new and flexible classes of nonseparable spatio-temporal covariances have resulted in a key point of research in the last years within the context of spatiotemporal Geostatistics. Approach:  In general, the literature has focused on the problem of full symmetry and the problem of anisotropy has been overcome. Results: By exploring mathematical properties of positive definite functions and their close connection to covariance functions we are able to develop new spatio-temporal covariance models taking into account the problem of spatial anisotropy. Conclusion/Recommendations:  The resulting structures are proved to have certain interesting mathematical properties, together with a considerable applicability.


INTRODUCTION
Spatial and spatio-temporal statistics recognizes and exploits the spatial locations of data when designing for, collecting, managing, analyzing and displaying such data. Spatial and spatio-temporal data are typically dependent, for which there are classes of spatial models available that allow process prediction and parameter estimation.
Spatially arranged measurements and spatial patterns occur in a surprisingly wide variety of scientific disciplines. The origins of human life link studies of the evolution of galaxies, the structure of biological cells and settlement patterns in archaeology. Ecologists study the interactions among plants and animals. Foresters and agriculturalists need to investigate plant competition and account for soil variations in their experiments.
The estimation of rainfall and of ore and petroleum reserves is of prime economic importance. Rocks, metals and tissue and blood cells are all studied at a microscopic level. Geology, soil science, image processing, epidemiology, crop science, ecology, forestry, astronomy, atmospheric science, or simply any discipline that works with data collected from different spatial locations, need to develop models that indicate when there is dependence between measurements at different locations.
The study of spatial and overall spatio-temporal variability is a relatively new area within Statistics, which explains the scarcity of spatial (spatio-temporal) statistical tools available 30 years ago. There has been a growing realization in the last 10 years that knowing where and when data were observed could help enormously in answering the substantive questions that precipitated their collection. One of the most powerful tools for spatial data analysis is the map. For example, in military applications, the battle space is mapped for command and control.
The sensors are both in situ and remote and they generate spatially distributed data of many different kinds (Fadda et al., 2008;Saberioon et al., 2010). Producing a statistically optimal map, together with measures of map uncertainty, which is always uptodate, is a complicated task. Once these types of statistical problems are solved, a Geographic Information System, or GIS, is well suited to forming the decision-making maps.
Recent literature persistently emphasizes the use of approximation methods and new methodologies for dealing with massive spatio-temporal data sets. When dealing with spatio-temporal data, calculation of the inverse of covariance matrices becomes a crucial problem. For instance, the inverse is needed for best linear unbiased prediction (also known as kriging) and is repeatedly calculated in maximum likelihood estimation or Bayesian inferences. Thus, large spatio-temporal sample sizes traduce into big challenges from the computational point of view.
As well known, interpolation-based kriging procedures strongly depend on the choice of the auto covariance associated to a space-time random field (Anuar et al., 2008;Gholizadeh et al. 2009, are two references in nonstandard fields). Thus, the geostatistical aim is to obtain permissible dependence structures for space and time; in other words, what is needed are nonseparable spatio-temporal covariance models associated to stationary or non-stationary random fields. This has been one of the most important challenges for the statistical community in the last ten years. But spatio-temporal covariance functions must be positive definite and this characteristic is no easy to prove and this is the reason why researchers have opted for using celebrated mathematical theorems from 20th Century that lead to positive definite covariance functions.
In this study we first deeply explore the connection between geostatistics and pure mathematical results proved several decades ago and which are now brought to light when dealing with spatial-temporal covariances. Connections between covariance functions and completely monotonic functions and Bernstein functions are highlighted. These are important tools for the construction of positive definite radial functions and have been used to define important inequalities involving characteristic functions.
In modern literature we can find several approaches to the construction of nonseparable covariance's. In the earlier contributions the focus was on extending spatial or temporal methods to spatiotemporal ones, considering the spatio-temporal dependency separately in most of the cases. Most of these contributions deal with stationary spatio-temporal covariance's assuming isotropy in space and time. However, isotropy is a very restrictive condition which is not reasonable in many real applications, but to which sometimes researchers need (are forced) to use, due to the lack of practical alternatives. This is why our approach here is to build new families of space-time covariance functions by removing the isotropy condition. In particular we present a new way of defining anisotropic models by assuming the condition of anisotropy by isotropy within components. This new definition needs the support of several important mathematical results appeared in the discipline of pure mathematical analysis and brought and adapted to the language used in the geostatistical community.
After this introductory section, we present the mathematical set up and background needed in the building of space-time covariance functions. We also include a spatio-temporal covariance function setup.
Next, we focus on the problem of modeling anisotropy through isotropy within components. We present some necessary definitions and theoretical background. We continue presenting the Bernstein class, a generalization of Gneiting (2002) approach to obtain new classes of space-time covariance functions which are spatially anisotropic. The study ends with some discussion and concluding remarks.

Mathematical setup and background on
for all finite sets of real coefficients i a and points ( ) This property is called positive definiteness, or with alternative notation, permissibility.
In this study we first deeply explore the connection between geostatistics and pure mathematical results proved several decades ago and which are now brought to light when dealing with spatial-temporal covariance's.
Connections between covariance functions and completely monotonic functions and Bernstein functions are highlighted. These are important tools for the construction of positive definite radial functions and have been used to define important inequalities involving characteristic functions. We then first recall the definition and some properties of two such important functions. A ≥ , for all n a natural number. For well known results, completely monotonic functions admit the representation in Bernstein's theorem, i.e.: for F a probability measure. In other words, a function ϕ such that (0) 1 ϕ = is a completely monotonic function if and only if it is the Laplace transform of a probability measure. In addition, Bernstein functions are positive functions whose first derivative is completely monotonic.
Finally, bivariate Laplace transforms, here denoted asL , admit the representation: . Obviously, if the probability measure is concentrated on the line 1 2 θ θ = , then L reduces to a completely monotonic function. The concept of positive-definiteness of a function defined on T T × for arbitrary T has played major role in the study of Gaussian processes. In addition, if T is a semi group with involution * , one can study the structure of positive definite functions. This can be related to problems in operator theory. Classically, continuous positive definite functions on , The connection with positive definite functions is provided by the Schoenberg theorem. Focusing on the construction of space-time covariance functions, in the last years there has been a great demand for models describing the evolution of a wide range of phenomena in space and time. In particular, there is a big need for models able to capture the simultaneous behavior of the spatial and temporal components. If we considered them separately, a large amount of information would be lost.
We refer to geostatistical spatio-temporal modeling as a leading instrument to analyze the evolution of such spatio-temporal processes.
As well known, interpolation-based kriging procedures strongly depend on the choice of the autocovariance associated to a space-time random field. Thus, the geostatistical perspective is to obtain permissible spatio-temporal covariances which take into account the interaction between space and time; in other words, what we need is nonseparable spatio-temporal covariance models associated to stationary or nonstationary random fields.
This has been one of the most important challenges for the statistical community in the last ten years.
Geostatistical approaches have been developed to fit Random Function (RF) models in continuous space and time, based on a limited number of spatially and/or temporally dispersed observations. These approaches model the observations as a partial realization of a spatio-temporal (typically Gaussian) random function.
Stationarity, isotropy, separability and full symmetry are then simplifying assumptions which are often needed for estimation and modeling.
Let ( , ), d Z t ∈ s s and t ∈ , be a stationary real valued Gaussian spatio -temporal RF with constant mean, which does not depend on the space-time coordinates and with constant and finite variance. Then, the function: called a stationary spatio-temporal covariance function of the process, as it exclusively depends on the spatial and temporal separation vectors, respectively h and u .
A stationary covariance function is called isotropic if it is rotation and translation-invariant, i.e.: ( ) where, with i we denote the usual Euclidean norm and st C is a positive definite function. Stationarity is only one of the simplifying assumptions which are often needed for estimation and modeling. Other very popular assumptions are that of separability and full symmetry. A space-time covariance function is called separable if we can factor: .
In other words, separability means that the spatiotemporal covariance structure factors into a purely spatial and purely temporal component, which allows for computational efficient estimation and inference. Consequently, separable covariance models have been used even in situations in which they are not physically justifiable. An interesting definition coined in Gneiting (2002) is full symmetry, which happens if: ( , ) ( , ) for every ( , ) . Equivalently, Lu and Zimmerman (2005) speak about reflection symmetry. It has been shown by Gneiting et al. (2005) that if C st is fully symmetric, so is its associated spectral density, if it exists. Another interesting aspect is that separable covariances are also fully symmetric, while viceversa is not necessarily true. Hence, covariance structures that are not fully symmetric are non-separable and tests for full symmetry can be used to reject separability. Gneiting et al. (2005) it is nicely summarized the relationships between the various notions in terms of classes of spatio-temporal covariance functions and an analogous scheme applies to correlation structures. The largest class is that of general, stationary or nonstationary covariance functions. A separable covariance can be stationary or nonstationary and similarly for fully symmetric covariances.
Experiments with time-forward kriging predictors suggest that the use of more complex and more realistic covariance models results in improved predictive performance. Thus, recent literature has emphasized the need for non-separable spatio-temporal covariances, in order to take into account the interaction between the spatial and temporal components.
In the earlier contributions the focus was on extending spatial or temporal methods to spatiotemporal ones, considering the spatio-temporal dependency separately in most of the cases. These extension approaches could be classified into two main categories: (a) the former considers a pure extension of multivariate spatial or temporal models and (b) the latter is based on considering a univariate spatiotemporal process. Even if there are many points in common between the two approaches, some distinctions are necessary in order to see the chance of applying one approach despite the other.
Some geometrically anisotropic models for spatiotemporal data were proposed in Dimitrakopoulos and Lou (1994). Guttorp et al. (1992) used separable covariance structures, obtained through the tensor product of a spatial and a temporal covariance. In the same context of separability, Rouhani and Hall (1989) proposed the so called sumvariogram.
Other approaches were based on the fact that the trend could entirely catch the temporal variability, in order to obtain a purely spatial random process. In the nonseparable context, efforts have been focus on obtaining general classes of spatio-temporal covariances. It is worth citing Jones and Zhang (1997), Cressie and Huang (1999), Christakos (2000); De Cesare et al. (2001); Gneiting (2002); Ma (2005), Stein (1999, Fernandez-Casal et al. (2003). Sahu et al., 2005, is also a recommended applied reference from the Bayesian perspective. Most of these contributions deal with stationary spatio-temporal covariances assuming isotropy in space and time. In particular, Cressie and Huang (1999) proposed a spectral approach to obtain spatio-temporal Covariances and Gneiting (2002) represents the natural generalization of this approach, obtained by using completely monotone functions and functions whose first derivative is completely monotonic. Stein (2005) puts emphasis on the spectral approach and on the fact that a spatio-temporal spectral density must be sufficiently smooth away from the origin, as well as on the problems of differentiability at the origin. Finally, Fernandez-Casal et al. (2003) extended the Shapiro-Botha approach to flexible variograms in the spatio-temporal context. In the non-stationary context, important contributions come from Christakos (2002); Fuentes (2002) and Kolovos et al. (2004). Mateu et al. (2007) and Porcu et al. (2006) introduced the following developments: (a) anisotropy by isotropy within components; (b) the inclusion of the Bernstein class for the generalization of Gneiting (2002) result. Chen et al. (2006) studied the non-stationary covariance functions based on mixtures of local orthogonal random functions. Porcu and Mateu (2007) obtained new nonstationary models through completely monotone functions. Gregori et al. (2008) developed covariances with negative values. And Porcu et al. (2009) had new insights about quasi-arithmetic means of covariance functions.
New families of anisotopic covariance functions: As the assumption of isotropy is very often unrealistic to model real phenomena, the researcher has to deal with the problem of relaxing this hypothesis. In other words, it is important to define covariance models that attain an anisotropic behavior in the spatial component.
There are two types of anisotropy. The simplest one, called geometric, is defined when the same covariance form and sill parameter is present in all directions, but the range changes with direction. In this case, there is a single sill, but the semivariogram reaches the sill in a shorter lag distance along a particular direction.
In the second case, called zonal anisotropy, either the form or the sill of the covariance structure or the scale parameter (or both) are different in distinct directions. Whilst in the case of geometric anisotropy, one can use an isotropic model simply by setting a deformation of the coordinates; the case of zonal anisotropy is mathematically much more complex and needs further caution.
Anisotropy through isotropy within components assumes that there is not necessarily isotropy in the complete space, but separately in subspaces of complete dimension. For instance, in 3 , if ( ) and (8) admits an integral representation as a scale mixture of Bessel functions. For the space-time setting, the element d has always cardinality |d|=d+1. Then, although we may force the anisotropy in the spatial components, all processes are considered as symmetric in time.
The bernstein class of space-time covariance functions: In this context of covariance functions in which anisotropy is obtained through isotropy within components, the Bernstein class with a covariance function of the form: In particular, note that an adaptation of Theorem 1 in Gneiting (2002) with, , , 1 ,2,3, i h u i ∈ = is a stationary nonseparable space-time covariance function with spatially anisotropic components, defined on 3 . × .
Observe that the arguments in (12) can be interchanged preserving the validity of the resulting covariance function. Another interesting point is that this covariance structure admits four trivariate margins, six bivariate and four univariate ones, so that starting from (12) Leading to a stationary nonseparable spatiotemporal covariance function in d × for any positive natural number d. Using random vectors ( )  Thus the corresponding covariance belonging to the Bernstein's class in Eq. 13 would be: … and μ ∈ . Observe that instead of the two Bernstein functions 1 2 , ψ ψ one could alternatively use two variogram models, say 1 2 , γ γ , provided that they do not vanish at the origin, which is equivalent to variograms with a nugget effect.

CONCLUSION
Spatio-temporal interactions usually show very complicated forms. We believe that analytical procedures which are easy to implement and interpretable can help in the specification of the spatiotemporal dependence. In this study we have presented an overview of possible solutions to the problem of modeling anisotropy in space-time data. The methods are quite general and allow building space-time covariances which are spatially anisotropic. The classes here presented are very rich as contain a wide range of particular cases. Note that the ingredients required for the construction of these covariances are easy-to-find. For instance, it is easy to find a list of Bernstein functions as indicated above and even easier to find increasing concave functions. In general, the new classes of covariance functions we present in this study are easy to build and allow for closed forms through simple bivariate or univariate mixtures.
The application of nonseparable anisotropic covariances to real space-time data is one of our main scopes for the future. Although the scope of this study is mainly theoretical, in the future we shall dedicate time for applications. Besides, simulation of space-time random fields is not at all easy for the meantime. Various computational problems are already encountered, even in very simple cases. But, obviously, it will be a very interesting research point for the future. In this context, it would be nice to consider our new proposed covariance families within a test of separability and analyze the power of such a test. Again, we leave this action for the next coming research.