I have encountered positive definite functions somewhere long before, and I think it should be during a probability course. These days when I attend a seminar on Brownian motions, this concept, positive definite function, recurs many times, and until then I realized that I did not know anything about this kind of function except perhaps its definition.
The most natural way to introduce positive definite function is perhaps to start from positive definite matrices. We all know what a positive definite matrix is. Here I assume always that the matrices are with real valued entries. Now consider a probability space, where the -algebra on the space is just generated by the Borel sets, and is a finite measure. Then for , we can define
Of course, we should assume that these quantities all exist. Then, we can easily show that the covariance matrix is semi-definite positive.
One can show that, conversely, for any semi-positive definite matrix , there is a measure on such that the above quantities exist and its covariance matrix is just .
Then, one can ask, what if these quantities, do not exist? Then there appears some generalization. Note that the essential obstacle for the existence of these quantities is that these functions are not bounded. Are there some important bounded functions related to a random variable or simply a finite measure? Yes, there are. And one of them is the characteristic function of a random variable or in other words, the Fourier transform of a finite measure:
where is the inner product on .
Then how could we translate the semi-positive-definiteness of the covariance matrix into the language of characteristic functions? It is not so obvious. I will give it directly. For any positive integer and for any real numbers and any vectors , we have that
This is the definition for a positive definite function: we call a complex valued function is semi-positive definite if for any positive integer and for any real numbers and any vectors , we have that
(a little remark is in order: there is another way around in Bayesian analysis which gives some more motivation for the introduction of (semi)positive definite functions, cf the wikipedia article). We drop the prefix ‘semi’ if the equality holds only for the case that . So it is clear that the function is a semi-positive definite function. If the measure admits the existence of the above quantities, then we can show that is positive definite if and only if is a positive definite matrix. So, in this sense, we say that positive definite functions are generalizations of positive definite matrices.
There is a rather remarkable result which inverses the above process, that is the Bochner’s theorem.
Theorem(Bochner) If is a positive definite function on , then there is a finite measure on such that is the Fourier transform of .
Note that, according to this theorem, if is positive definite, then tends to zero when tends to infinity. We can not see this point directly from the definition of positive definite functions.
Note that in the definition of positive definite functions, we use only the additive group structure of , this inspires us to again generalize the domain of definition to a general group. This will be the content of the next post.