The proof of the central limit theorem and Fourier analysis I.

by Péter Major

The Mathematical Institute of the Hungarian Academy of Sciences

Summary: The central limit theorem belongs to the most important results of probability theory. There is another classical result of the probability theory, the law of large numbers which states in a slightly informal language that under some mild conditions the average of the partial sums of independent and identically distributed random variables can be well approximated by the expected value of these random variables. The central limit theorem gives better, more substantial information about the fluctuation of the average of these partial sums around their expected value. It states that if we multiply this fluctuation by \sqrt{n}, where n denotes the number of random variables in the average, then the distribution of the fluctuation with this normalization is close to a (non-degenerated) distribution independent of the number n. Moreover, and this is a most remarkable fact, this approximating distribution does not depend on the distribution of the random variables whose average is taken. This distribution is a “universal law” which is called the normal distribution in the literature. In the series of problems described in this text we shall formulate the above result in a more precise and more general form.

The method of the proofs also deserves special attention. This method, which is called the characteristic function method, is actually the application of the Fourier analysis in the proof of limit theorems. One goal of this series of problems was to show that this is a natural approach. The main difference between my text and the discussion of the central limit theorem in usual text-books is that I tried to supply a more definitive and detailed explanation about the relation between the method of the proof of the central limit theorem and some basic ideas of the Fourier analysis.

The theory of the Fourier series gives a natural method to prove the local version of the central limit theorem, i.e. to describe the asymptotic behaviour of the density and not of the distribution function of the normalized partial sums of independent random variables. Furthermore, if we understand what kind of additional problems arise if we want to prove the central limit theorem in its original (and not in its local) form and also understand the answers supplied by the results of the Fourier analysis to these questions, then we can prove the desired results.

Moreover, in such a way we get a natural method to prove some refinements of the central limit theorem. We also can investigate the question how good approximation is supplied by the central limit theorem for the distribution of normalized partial sums of independent random variables, and what kind of better approximation can be given. But these problems are not discussed here. They will be the subject of the second part of this series of problems.

Finally, I mention some parts of this work which may be of special interest. They are the proof of the Stirling formula in a way suggested by the proof of the local central limit theorem, (problem 2), the necessary and sufficient condition of the relative compactness of a sequence of probability measures expressed by means of their characteristic functions (problem 22), a detailed discussion of the question how the properties of a function or a measure are reflected in the behaviour of their Fourier transform (problems 27–34 and the discussion about these problems).

CONTENT

problems solutions
A. Local limit theorems. . . . . . . . . . . . . . . . . . . . . . . . . . . .  3–10  42–50

B. The definition of the normal distribution function and of the characteristic function. Some important results related to these notions. . . . . . . . . . . . . . . . . . 10–13 51–53

C. The definition of the convolution and some of its

important properties. . . . . . . . . . . . . . . . . . . . . . . . . . . .  13–16  53–56
D. Convergence in distribution. . . . . . . . . . . . . . . . . . . . .  16–22  56–62

E. The relation between the convergence of distribution functions and the convergence of their characteristic functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22–24 62–65

F. The relation between the behaviour of a function

or measure and its Fourier transform. . . . . . . . . . . .  24–29  65–70
G. The central limit theorem. . . . . . . . . . . . . . . . . . . . . . .  29–35  70–75
H. The multi-dimensional central limit theorem. …  35–38  75–80
Some additional remarks. . . . . . . . . . . . . . . . . . . . . . . .  38–41
Appendix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  80–85

a.) The proof of the inversion formula for Fourier transforms together with the proof of the Parseval formula. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Central limit theorems and Fourier analysis

b.) The proof of Weierstrass second approximation theorem together with the proof of Fejér’s theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 A.) Local limit theorems.

Let us first consider the following problem: Let ξ_{j}, j = 1, 2, …, be a sequence of independent and identically distributed random variables which take integer values. We

introduce the notation P (ξ_{1} = k) = p(k), k = 0, ±1, ±2, …, ∑_{k=−∞}^{∞} p(k) = 1 and define

the partial sums S_{n} = ∑_{j=1}^{n} ξ_{j}, n = 1, 2, . . .. Let us consider the probabilities p_{n}(k) =

P (S_{n} = k), k = 0, ±1, ±2, …, n = 1, 2, …, and try to give a good approximation of the probabilities p_{n}(k) in the case of large parameters n.

A good estimation can be given for these probabilities p_{n}(k) by means of the following method. Let us define the Fourier series

P_{n}(t) = ∑_{k=−∞}^{∞} p_{n}(k)e^{ikt}, −π ≤ t ≤ π.  (1)

By a basic formula of the theory of Fourier series the coefficients of the above Fourier series can be expressed by means of the formula

p_{n}(k) = \frac{1}{2π} ∫_{−π}^{π} e^{−ikt}P_{n}(t) dt, k = 0, ±1, ±2, ….  (2)

Hence, if we give a good asymptotic formula for the Fourier series P_{n}(t) and estimate well the integral in formula (2), then we get a good estimate for the probabilities p_{n}(k) we are interested in. Let us observe that P_{n}(t) = Ee^{itS_{n}}, −π ≤ t ≤ π. Besides, as S_{n} is the sum of independent and identically distributed random variables,

P_{n}(t) = Ee^{it(ξ_{1}+···+ξ_{n})} = (Ee^{itξ_{1}})^{n} = (P_{1}(t))^{n} ,

where P_{1}(t) = ∑_{k=−∞}^{∞} P (ξ_{1} = k)e^{ikt}. Furthermore, P_{1}(0) = 1, and as we shall see,

|P_{1}(t)| < 1 if −π ≤ t ≤ π and t ≠ 0 under some natural conditions. Hence in the estimation of the probabilities p_{k}(n) a so-called singular integral appears in formula (2), in which only a small neighbourhood of the origin gives an essential contribution to the integral. Such expressions can be well bounded by some standard methods of the analysis. Let us first show the following identities which we later need.

1.) Let us show that

\frac{1}{\sqrt{2aπ}} ∫_{−∞}^{∞} e^{−u^{2}/2a} du = 1

for all real positive numbers a > 0. Furthermore,

\frac{1}{\sqrt{2aπ}} ∫_{−∞}^{∞} e^{−(u−z)^{2}/2a} du = 1

Péter Major

for all real positive numbers a > 0 and complex numbers z.

First we apply the above method in a special case. Let us consider the Poisson distribution with parameter λ = n, that is the distribution of a random variable η for which P (η = k) = P_{n}(k) = \frac{n^{k}}{k!} e^{−n}, k = 0, 1, 2, . . .. The distribution of a Poissonian random variable with parameter n agrees with the distribution of the sum of n independent Poissonian random variables with parameter 1. Hence the above sketched method makes possible to get a good estimate for the probabilities P_{n}(k). We get a really good estimate in the case when the number k is close to the expected value of the random variable η, i.e. if n ∼ k. In particular, by investigating the appropriate Fourier series we get a good bound on the number P_{n}(n). We do not exploit explicitly the probabilistic content of the coefficients P_{n}(k) of this Fourier series, but the calculations indicate what kind of technical problems has to be solved in the analogous problems of probability theory. Besides, we get in such a way the proof of an important result of the analysis, the proof of the Stirling formula. This is the content of the next problem.

2.) Let us calculate the value of the Poissonian distribution with parameter n in the point n with the help of the above discussed method by working with the Fourier series whose coefficients are the values of this Poisson distribution. Let us show with the help of this method that

n! = (\frac{n}{e})^{n} \frac{2π}{∫_{−π}^{π} e^{n(e^{it}−1−it)} dt}.  (3)

Let us show that

∫_{−π}^{π} e^{n(e^{it}−1−it)} dt = \begin{matrix}√ & \overline{2π} \\ √ & \overline{n} \end{matrix} (1 + O (\frac{1}{n}))  (4a)

and

n! = \sqrt{2πn} (\frac{n}{e})^{n} (1 + O (\frac{1}{n})) .  (4b)

Let us prove the following improvement of the previous two statements:

∫_{−π}^{π} e^{n(e^{it}−1−it)} dt = \begin{matrix}√ & \overline{2π} \\ √ & \overline{n} \end{matrix} (1 + \frac{c_{1}}{n^{1/2}} + \frac{c_{2}}{n} + ⋯ + \frac{c_{k}}{n^{k/2}} + O (\frac{1}{n^{(k+1)/2}} )) (4c)

for arbitrary integer k ≥ 1 with explicitly calculable coefficients c_{k}. In particular, c_{1} = 0. Furthermore,

n! = \sqrt{2πn} (\frac{n}{e})^{n} (1 + \frac{c̄_{1}}{n^{1/2}} + \frac{c̄_{2}}{n} + ⋯ + \frac{c̄_{k}}{n^{k/2}} + O (\frac{1}{n^{(k+1)/2}} )) (4d)

for arbitrary integer k ≥ 1 with explicitly calculable coefficients c̄_{k}. In particular, c̄_{1} = 0.

Similarly to the solution of problem 2 a good asymptotic formula can be given for the probability of the event that the sum of independent and identically distributed random variables takes a definite value. This investigation is based on formula (2) where the integral of a Fourier series with periodicity 2π has to be estimated. But such cases may appear when this Fourier series also has a smaller periodicity. Such an example appears for instance if the partial sums of such random variables are considered which take only even or only odd values. The Fourier series determined by the distribution of such random variables has a smaller periodicity π. If we want to study the asymptotic behaviour of the distribution of partial sums of independent and identically distributed random variables in a systematic way, then we have to clarify which is the smallest period of the Fourier series in formula (2). In particular, we want to tell when the smallest periodicity of this Fourier series is precisely 2π. This is the reason why we introduce the following definition. Then we study the case when the smallest periodicity of the Fourier series in formula (2) is 2π. The investigation of partial sums of independent and identically distributed random variables whose distribution takes values on a lattice kh + b, k = 0, ±1, ±2, …, with some numbers h and b can be relatively simply reduced to this special case.

Definition A. The values of a random variable ξ are concentrated on the lattice of

integers (as on the rarest lattice) if ∑_{k=−∞}^{∞} P (ξ = k) = 1, and for arbitrary integers

A > 1 and B ∑_{k=−∞}^{∞} P (ξ = Ak + B) < 1.

More generally, we call a random variable ξ lattice valued if its values are concentrated with probability one on a set of the form {b + kh,: k = 0, ±1, ±2, … } with some real numbers h > 0 and b. We say that the values of a random variable ξ are concentrated on a lattice of width h, h > 0, (as on the rarest lattice) if there exists a

real number b such that ∑_{k=−∞}^{∞} P (ξ = kh + b) = 1, and ∑_{k=−∞}^{∞} P (ξ = Akh + B) < 1 for

all integers A > 1 and real numbers B.

To carry out the further investigation we need the following result.

3.) If ξ is a non-constant random variable concentrated on a lattice, then there exist a smallest h > 0 number such that it is concentrated on a lattice of width h as the rarest lattice. Let ξ be a random variable concentrated on a lattice of width h (as on the rarest

lattice). Let us choose a real number b > 0 such that ∑_{n=−∞}^{∞} P (ξ = nh + b) = 1, and

let us consider the Fourier series P (t) = ∑_{n=−∞}^{∞} e^{inh}P (ξ − b = nh) determined by

the distribution of the random variable ξ − b. The periodicity of the Fourier series P (t) equals \frac{2π}{h}, P (0) = 1, |P (t)| ≤ 1 for all real numbers t, and |P (t)| < 1 if |t| ≤ \frac{π}{h} and t ≠ 0. If the absolute value of the random variable ξ −b has k-th moment, that is E|ξ − b|^{k} < ∞, then the function P (t) is k times continuously differentiable, and

\frac{dP^{k}(t)}{dt^{k}} |_{t=0} = i^{k}E(ξ − b)^{k}, (where i = \sqrt{−1}).

4.) Let ξ_{1}, ξ_{2}, …, be a sequence of independent and identically distributed random variables whose values are concentrated with probability one on the lattice of the integers (as on the rarest lattice). Let Eξ_{1} = m, Eξ_{1}^{2} = m_{2} < ∞, (that is we assume that the second moment of the random variable ξ_{1} is finite), and put σ^{2} = m_{2} −m_{1}^{2}. (The number σ^{2} denotes the variance of the random variable ξ_{1}.) Let us consider the partial sums S_{n} = ξ_{1} + ⋯ + ξ_{n}, n = 1, 2, . . .. Then

P (S_{n} = k) = \frac{1}{\sqrt{2πn}σ} exp \{−\frac{(k − nm)^{2}}{2nσ^{2}} \} + o (\frac{1}{\sqrt{n}}) , k = 0, ±1, ±2, …,

and o(·) is uniform in the variable k. 5.) Let us consider such a sequence ξ_{1}, ξ_{2}, …, of independent and identically dis-

tributed random variables which satisfies the conditions of the previous problem together with the condition E|ξ_{1}|^{3} < ∞. Under these conditions prove the following sharper form of the previous problem.

P (S_{n} = k) = \frac{1}{\sqrt{2πn}σ} exp \{−\frac{(k − nm)^{2}}{2nσ^{2}} \} + ε(n, k), k = 0, ±1, ±2, …,

where |ε(n, k)| ≤ \frac{K}{n}, and the constant K depends only on the distribution of the random variable ξ_{1}.

Historically first that special case of the above limit theorems was considered where the random variable ξ_{1} is binomially distributed, i.e. P (ξ_{1} = 1) = 1 − P (ξ_{1} = 0) = p, 0 < p < 1. In this case the distribution of the sum S_{n} can be expressed in a simple explicit form, and this expression can be well bounded with the help of the Stirling formula. As this special case has a particular importance in combinatorial applications, it deserves special discussion. This is done in the following problem.

5a.) Let us give an elementary proof of the statement of problem 5 in the special case when ξ_{1} is binomially distributed. (Here we want to give a proof where the Stirling formula is applied instead of the Fourier analysis.)

6.) Let a sequence ξ_{1}, ξ_{2}, … of independent and identically distributed random variables satisfy the conditions of problem 4 with the difference that now we assume that the values of the random variable ξ_{1} are concentrated with probability 1 on a lattice of width h, h > 0, (which is the width of the rarest lattice). Let the values of the random variable ξ_{1} be concentrated on the numbers kh + b, k = ±1, ±2, …, with some real number b. Then with the notations of problem 4

\begin{aligned}P (S_{n} = kh + nb) & = \frac{h}{\sqrt{2πn}σ} exp \{−\frac{(kh + nb − nm)^{2}}{2nσ^{2}} \} + o (\frac{1}{\sqrt{n}}) , \\ & k = 0, ±1, ±2, …, \end{aligned} (5)

where o(·) is uniform in the variable k. If also the condition E|ξ_{1}|^{3} < ∞ is satisfied, then

P (S_{n} = kh + nb) = \frac{h}{\sqrt{2πn}σ} exp \{−\frac{(kh + nb − nm)^{2}}{2nσ^{2}} \} + ε(k, n),
k = 0, ±1, ±2, …,

where |ε(n, k)| ≤ \frac{K}{n}, and the constant K depends only on the distribution of the random variable ξ_{1}.

7.) If a sequence of random variables S_{n}, n = 1, 2, …, satisfies relation (5), (it has no importance what additional properties these random variables have), then

lim_{n→∞} P (\frac{S_{n} − nm}{\sqrt{n}σ} < x) = ∫_{−∞}^{x} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du

for all real numbers x. The convergence is uniform with respect to the parameter x.

The above results supply good information on the probabilities the partial sums of lattice valued independent and identically distributed random variables take different values. Then by “integrating out” this relation in problem 7 we have obtained an estimate about the limit behaviour of the distribution of the appropriately normalized partial sums of independent and identically distributed random variables if the summands are lattice distributed. These results can be sharpened, but first we want to generalize the above result and to investigate the limit behaviour of the distribution or density (if the density function exists) function of appropriately normalized partial sums of independent random variables which are not necessarily lattice distributed.

Let ξ_{1}, ξ_{2}, …, be a sequence of independent random variables, and let us consider the partial sums S_{n} = ξ_{1} + ⋯ + ξ_{n}, n = 1, 2, . . .. We want to show that if the random variable ξ_{1} has a nice density function, then a good asymptotic formula can be given for the density function of the random sum S_{n}. Besides, we want to show under general conditions that the appropriate normalized version of the sums have a limit distribution as n → ∞, and also want to describe this limit distribution function. In the case when the random variables ξ_{n} are lattice valued then the corresponding problems could be solved by means of the investigation of an appropriate Fourier series. The question arises whether this method can be adapted to solve the analogous problem in the general case. To carry out such a program we should have a relatively simple inversion formula which enables to calculate a density or distribution function by means of its Fourier transform.

In case of nice density functions there exists such a simple inversion formula. In case of general distribution function there are only complicated inversion formulas which are not really well applicable in the investigations we have in mind. To prove limit theorems for the appropriately normalized partial sums of independent random variables first we have to understand what means exactly convergence in distribution. Then we can prove limit theorems for the normalized partial sums of independent random variables by means of some basic results of the Fourier analysis. Before doing this we formulate an inversion formula for the Fourier transform which is useful in the investigation of the density functions.

Inversion formula for Fourier transforms. Let f(u) be an integrable function on the real line, i.e. let us assume that ∫_{−∞}^{∞} |f(u)| du < ∞. Let us consider the Fourier transform

f̃(t) = ∫_{−∞}^{∞} e^{itu}f(u) du, −∞ < t < ∞  (6a)

of this function f(·). If the function f̃ is also integrable, i.e. ∫_{−∞}^{∞} |f̃(t)| dt < ∞, then the identity

f(u) = \frac{1}{2π} ∫_{−∞}^{∞} e^{−itu}f̃(t) dt  (6)

holds in almost all points of the real line with respect to the Lebesgue measure. Moreover, even the following stronger statement holds. Let μ be a finite measure, i.e. let μ(R^{1}) < ∞. Let f̃(t) = ∫ e^{itu} dμ(u) denote the Fourier transform of this measure μ. If the function f̃(·) is integrable, then the measure μ has a density function, and it agrees with the function f(·) defined in formula (6). This result can be slightly generalized. It also holds if μ is a signed measure with bounded variation, i.e. it is the difference of two finite measures.

The above definition of the Fourier transform slightly differs from the definition usually given in the literature, where the integral in formula (6a) and its analog which defines the Fourier transform of a measure μ is divided by \sqrt{2π}. With such a normalization the Fourier transform and the formula expressing its inverse are more similar to each other. For us the formula given in (6a) is more convenient. We shall prove this inversion formula for Fourier transforms in the Appendix. Let us remark that the restriction that formula (6) holds only for almost all points u with respect to the Lebesgue measure is natural. Indeed, by modifying a function on a set of Lebesgue measure zero we get a new function which has the same Fourier transform.

Let us finally remark that the formula about the calculation of the Fourier coefficients by means of the Fourier transform suggest the inversion formula (6). Namely, let us approximate a function in the points kε, k = 0, ±1, ±2, …, in a natural way and let us write the Fourier series with these coefficients (in this case we consider the Fourier series in the interval [−\frac{π}{ε} , \frac{π}{ε} ]). Then by expressing the Fourier coefficients of this Fourier series with the help of the Fourier series and by applying the limit procedure ε → 0 we get, at least on a formal level, formula (6).

8.) Let ξ_{1}, ξ_{2}, …, be a sequence of independent and identically distributed random variables, Eξ_{1} = 0,Eξ_{1}^{2} = 1, (i.e. we consider random variables with expectation zero and variance one), and put S_{n} = ξ_{1} +⋯+ξ_{n}, n = 1, 2, . . .. Let us also assume that the random variable ξ_{1} has a density function f(x), and the Fourier transform φ(t) of the density function f(x) is integrable or at least it satisfies the following weakened condition: There exists an integer k ≥ 1 such that φ^{k}(t) is integrable.

Then the random variable \frac{S_{n}}{\sqrt{n}} has a density function f_{n}(x) which satisfies the relation

lim_{n→∞} f_{n}(x) = \frac{1}{\sqrt{2π}} e^{−x^{2}/2} for all real numbers x,  (7)

and the convergence is uniform in the parameter x. 9.) If a sequence of distribution functions F_{n}(x), n = 1, 2, … have density functions

f_{n}(x), and they satisfy formula (7), then

lim_{n→∞} F_{n}(x) = ∫_{−∞}^{x} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du for all real numbers x.

The convergence is uniform with respect to the variable x. 10.) If the conditions of problem 8 are satisfied, and also the relation E|ξ_{1}|^{3} < ∞ holds,

then relation (7) holds with a better error term. Namely,

f_{n}(x) = \frac{1}{\sqrt{2π}} e^{−x^{2}/2} + O (\frac{1}{\sqrt{n}}) ,

and O(·) is uniform with respect to the parameter x.

Remark: We have proved in problems 6 and 10 that if the absolute value of the random variables have finite third moment, then the density function of the appropriately normalized sums can be approximated with the normal density function with an accuracy of order O (n^{−1/2}), where n is the number of summands in the sum. In the second part of this series of problems we shall prove an analog of this result about the approximation of the appropriately normalized distribution function of partial sums of independent and identically distributed random variables. This is an important result of the probability theory, which is called the Berry–Esseen inequality. It yields an estimate of the same order O (n^{−1/2}) for the approximation of the normalized distribution function by the standard normal distribution function. Nevertheless, there is an essential difference between the estimate supplied by the Berry–Esseen inequality and the results of problems 6 and 10.

The Berry–Esseen inequality gives an upper bound of the form const. μ_{3}n^{−1/2}, where μ_{3} is the third moment of the absolute value of the summands, and the const. is a universal number which can be chosen independently of the distribution of the summands. One cannot give a similar universal estimate on the normal approximation of the density function. Indeed, if for instance the density function of the summands take extremely large values in a small neighbourhood of the origin and the probability of this neighbourhood is relatively large, say larger than \frac{1}{10}, then for relatively small indices n the density function of the normalized sum of n independent random variables cannot be well approximated by a normal density function, although the third moment of the absolute value of the summands may be not large. A similar example can be given for the density function of the partial sums of lattice valued random variables if the width of the lattice may be very small.

The above sketched examples show that the analog of the Berry–Esseen inequality for density functions does not hold. This difference is related to the fact that in the first case we study distribution functions while in the second case their derivatives. It is a quite common experience in analysis that for the nice behaviour of a sequence of functions we have to impose much weaker conditions than for the nice behaviour of their derivatives.

A deficiency of the results of problems 8 and 9 is that it imposes conditions not for the density but for the characteristic function of the summands, while in typical applications we have some direct information about the density and not on the characteristic function. Hence we have to understand the relation between the properties of a density and the corresponding characteristic function. Later we shall return to this question. We shall see that relation (7) holds for the density function of normalized sums of independent and identically distributed random variables if the summands have “nice” density function.

B.) The definition of the normal distribution function and of the characteristic function. Some important results related to these notions.

Let us introduce the definition of the (standard) normal distribution. The solutions of the previous problems suggest that in limit distributions the normal limit distribution appears as the limit.

The definition of the normal distribution. The Φ(x) standard normal distribution function is the distribution function whose density function the standard density function φ(x) is of the form φ(x) = \frac{1}{\sqrt{2π}} e^{−x^{2}/2}, that is

Φ(x) = ∫_{−∞}^{x} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du.

More generally, a linear transform of the standard normal distribution function Φ(x) is called a normal distribution function. More explicitly, a normal distribution function can be characterized by two parameters, by a real number m and a positive real number σ. The normal distribution function with parameters m and σ equals Φ_{m,σ}(x) = Φ (\frac{x−m}{σ}). Its density function is φ_{m,σ}(x) = \frac{1}{σ} φ (\frac{x−m}{σ}) = \frac{1}{\sqrt{2π}σ} e^{−(x−m)^{2}/2σ^{2}}.

It follows from the solution of problem 1 that Φ(x) is really a distribution function. The normal distribution is also called the Gaussian distribution in the literature.

11.) Let us show that the expected value of a standard normal random variable equals zero, and its variance equals one. The expected value of a random variable with distribution function Φ_{m,σ}(x) equals m and its variance equals σ^{2}.

Let us consider the appropriately normalized partial sums of independent and identically distributed random variables with a nice density function. We have seen that the density functions of these normalized partial sums tend to the normal density and their distribution functions to the normal distribution function. A similar result holds for sums of lattice valued random variables. We say that the appropriate normalizations S̄_{n} = \frac{S_{n}−a_{n}}{b_{n}}, n = 1, 2, …, of a sequence of random variables S_{n}, n = 1, 2, …, satisfy the local central limit theorem if either the random variables S̄_{n} have a density function for large n, and they converge uniformly to the standard normal distribution function, or for all large n there exists a lattice kh_{n} + b_{n} of width h_{n}, k = 0, ±1, ±2, …, such that h_{n} → 0 if n → ∞, the random variable S̄_{n} is concentrated on this lattice (as on the rarest lattice), and P (S_{n} = kh_{n} + b_{n}) = h_{n}φ(kh_{n} + b_{n}) + o(h_{n}), k = 0, ±1, ±2, …, o(·) is uniform in the variable k, where φ(x) is the standard normal density function.

We have seen in problems 6 and 8 that the appropriately normalized sums of independent and identically distributed random variables with density function or lattice valued distribution satisfy the local central limit theorem under fairly general conditions. The normalization is made in a natural way. With the notation of the previous paragraph we choose a_{n} = nEξ_{1} = ES_{n}, b_{n} = \sqrt{n Var ξ_{n}} = \sqrt{Var S_{n}}, i.e. such a normalization is chosen with which the normalized partial sums have expected value zero and variance one. We also have seen that the local central limit theorem implies the (global) central limit theorem (problems 7 and 9). One would expect that the central limit theorem may hold also in such cases when its local version fails to hold. We want to show that this belief is right. Moreover, we want to prove the central limit theorem in more general cases when appropriately normalized partial sums of independent but not necessarily identically distributed random variables are considered.

We have proved the local central limit theorem with the help of the Fourier transform of the distribution functions we have investigated. We want to show that this method can be adapted to the proof of (global) central limit theorems. To do this we recall the definition of the Fourier transform of general (probability) measures. We shall call them, following the tradition of probability theory terminology, characteristic functions. Since later we also want to study partial sums of vector valued random variables, hence — to avoid some repetition — we define the multi-dimensional version of the characteristic functions.

The definition of characteristic functions. Let F (u) = F (u_{1}, …, u_{k}) = P (ξ_{1} < u_{1}, …, ξ_{k} < u_{k}) denote the distribution function of a k-dimensional random vector (ξ_{1}, …, ξ_{k}). The φ(t) = φ(t_{1}, …, t_{k}), t = (t_{1}, …, t_{k}), characteristic function of a kdimensional distribution function F or of an F distributed k-dimensional random vector ξ = (ξ_{1}, …, ξ_{k}), k ≥ 1, is defined by the formula

φ(t) = φ(t_{1}, …, t_{k}) = Ee^{i(t,ξ)} = Ee^{i(t_{1}ξ_{1}+···+t_{k}ξ_{k})}
= ∫ e^{i(t,u)}F ( du) = ∫ e^{i(t_{1}u_{1}+···+t_{k}u_{k})}F ( du_{1}, …, du_{k}),

where t = (t_{1}, …, t_{k}) is an arbitrary point of the k-dimensional Euclidean space. (Here we applied the following notation: If u = (u_{1}, …, u_{k}) and v = (v_{1}, …, v_{k}) are two k-dimensional vector, then (u, v) = u_{1}v_{1} + ⋯ + u_{k}v_{k} denotes the scalar product of the vectors u and v.)

12.) A characteristic function φ(t_{1}, …, t_{k}) is uniformly continuous in the k-dimensional Euclidean space R^{k}, φ(0, …, 0) = 1 and

|φ(t_{1}, …, t_{k})| ≤ 1 in all points (t_{1}, …, t_{k}) ∈ R^{k}.

Let φ(t) = φ(t_{1}, …, t_{k}), t = (t_{1}, …, t_{k}), be the characteristic function of a random vector ξ = (ξ_{1}, …, ξ_{k}). Let a be a real number and m = (m_{1}, …, m_{k}) a kdimensional vector. Then the characteristic function of the random vector aξ+m = (aξ_{1} + m_{1}, …, aξ_{k} + m_{k}) equals

e^{i(m,t)}φ(at) = e^{i(m_{1}t_{1}+···+m_{k}t_{k})}φ(at_{1}, …, at_{k}).

If ξ_{1}, …, ξ_{n}, 1 ≤ j ≤ n, is a sequence of independent k-dimensional random vectors with the notation ξ_{j} = (ξ_{j}^{(1)}, …, ξ_{j}^{(k)}), and φ_{j}(t) = φ_{j}(t_{1}, …, t_{k}), j = 1, …, n, t = (t_{1}, …, t_{k}), denotes their characteristic functions, then the characteristic function

of the sum ξ_{1} + ⋯ + ξ_{n} equals ∏_{j=1}^{n} φ_{j}(t) = ∏_{j=1}^{n} φ_{j}(t_{1}, …, t_{k}).

Let us calculate the characteristic function of some important distribution functions. 13.) Let us show that if the random variable ξ a.) has standard normal distribution, i.e. it has a density function of the form f(u) =

\frac{1}{\sqrt{2π}} e^{−u^{2}/2}, then its characteristic function equals φ(t) = e^{−t^{2}/2}.

b.) has uniform distribution in the interval [0, 1], i.e. it has a density function of the form f(u) = 1 if 0 ≤ u ≤ 1 and f(u) = 0 is otherwise, then its characteristic function equals φ(t) = \frac{e^{it}−1}{it}.

c.) has exponential distribution with parameter λ > 0, i.e. it has a density function of the form f(u) = λe^{−λu} if u ≥ 0 and f(u) = 0 if u < 0, then its characteristic function equals φ(t) = \frac{λ}{λ−it}.

d.) has Cauchy distribution, i.e. it has a density function of the form f(u) = \frac{1}{π} \frac{1}{1+u^{2}} , then its characteristic function equals φ(t) = e^{−|t|}.

e.) has Poisson distribution with parameter λ > 0, i.e. P (ξ = k) = \frac{λ^{k}}{k!} e^{−λ}, k =

0, 1, 2, …, then its characteristic function equals φ(t) = exp {λ(e^{it} − 1)}. f.) has binomial distribution with parameters n and p where n ≥ 1 is an integer,

0 < p < 1, i.e. P (ξ = k) = \binom{n}{k}p^{k}(1 − p)^{n−k}, k = 0, 1, …, n, then its characteristic function equals φ(t) = (1 − p + pe^{it})^{n}.

g.) has negative binomial distribution with parameters n and p, where n ≥ 1 is an integer, 0 < p < 1, i.e. P (ξ = k) = \binom{n+k−1}{k}p^{k}(1 − p)^{n}, k = 0, 1, 2, …, then its

characteristic function equals φ(t) = (\frac{1−p}{1−pe^{it}} )^{n}.

h.) has γ distribution with parameter s, s > 0, i.e., its density function is γ_{s}(u) = \frac{1}{Γ(s)}u^{s−1}e^{−u}, if u > 0, and γ_{s}(u) = 0, if u < 0, where Γ(s) = ∫_{0}^{∞} u^{s−1}e^{−u} du, then its characteristic function equals φ_{s}(t) = \frac{1}{(1−it)^{s}}.

It may be worth mentioning that the density function γ_{s}(u), considered in point h.) shows some similarity with the Poissonian distribution, and this can be exploited to prove, similarly to the solution of Problem 2, a good asymptotic formula for the function Γ(s) if s → ∞. This means a generalization of Stirling’s formula, since one prove with the help of some partial integration that Γ(n) = (n − 1)!.

Let us observe that γ_{s_{1}}(u) ∗ γ_{s_{2}}(u) = γ_{s_{1}+s_{2}}(u), where ∗ denotes convolution. This can be seen e.g. with the help of the form of the characteristic function of γ_{s} and some important properties of the characteristic functions discussed later. Further, we can write the identity γ_{s}(s) = \frac{s^{s−1}e^{−s}}{Γ(s)} = \frac{1}{2π} ∫_{−∞}^{∞} \frac{e^{−ts}}{(1−it)^{s}} ds with the help of the inverse Fourier transform formula for density functions. Then we get by giving a good asymptotic formula for the expression at right-hand side of this identity (similarly to the solution of Problem 2) that Γ(s) ∼ \sqrt{2π(s − 1)}(\frac{s−1}{e})^{s−1} if s → ∞.

C.) The definition of the convolution and some of its important properties.

In this series of problems we investigate the asymptotic behaviour of the distribution and density function of appropriately normalized partial sums of independent random variables. These distribution or density functions can be directly expressed by means of the distribution of density functions of the random variables in these partial sums. Hence the limit theorems discussed in this text also can be expressed in the language of distribution (and density) functions without speaking of sums of independent random variables. To do this we have to introduce the notion of the convolution operator. We introduce this notion in a slightly more general form and define the convolution of integrable (not necessary density) functions and signed measures. We shall not use the notion of the convolution in this series of problems. Hence this section could be omitted. Nevertheless, the discussion of limit theorems without the introduction of the convolution would not be complete, hence we introduce it. In several investigations of the analysis and probability theory this notion appears in a natural way, and it appears in the more general form introduced in this text.

The definition of the convolution operator. If f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{k}) are two k-dimensional measurable functions of k variables which are integrable, i.e. ∫ |f(x_{1}, …, x_{k})| dx_{1} … dx_{k} < ∞ and ∫ |g(x_{1}, …, x_{k})| dx_{1} … dx_{k} < ∞, then the convolution f ∗ g of the functions f and g is the function of k variables defined by the formula

f ∗ g(x_{1}, …, x_{k}) = ∫ f(u_{1}, …, u_{k})g(x_{1} − u_{1}, …, x_{k} − u_{k}) du_{1} … du_{k}  (8)

in all such points (x_{1}, …, x_{k}), where this integral is meaningful. (In the remaining points we can define the function f ∗ g in an arbitrary way.)

Let μ and ν be two signed measures of bounded variation on the measurable sets of the k-dimensional Euclidean space R^{k}. This means that we assume that there exist two representations μ = μ_{1} − μ_{2} and ν = ν_{1} − ν_{2} such that μ_{i} and ν_{i}, i = 1, 2, are finite measures on the measurable subsets of the space R^{k}, i.e. μ_{k}(R^{k}) < ∞ and ν_{k}(R^{k}) < ∞, k = 1, 2. Let μ × ν denote the direct product of these signed measures μ and ν on the product space R^{k} × R^{k} = R^{2k}. Then the convolution μ ∗ ν is the signed measure on the measurable subsets of the space R^{k} defined by the formula

μ ∗ ν(A) = μ × ν{(u, v): u + v ∈ A} for all measurable sets A ⊂ R^{k}.

In other words, μ ∗ ν is the pre-image of the product (signed) measure μ × ν induced by the transformation T: R^{k} × R^{k} → R^{k} defined by the formula T(u, v) = u + v.

Let f(x_{1}, …, x_{k}) be a measurable and integrable function of k variables, ν a measure of bounded variation on the measurable subsets of R^{k}. The convolution f ∗ν(x_{1}, …, x_{k}) is the following function of k variables:

f ∗ ν(x_{1}, …, x_{k}) = ∫ f(u_{1}, …, u_{k})ν(x_{1} − du_{1}, …, x_{k} − du_{k})

in all points (x_{1}, …, x_{k}) ∈ R^{j}, where this integral is meaningful. This means that we integrate the function f(·) with respect to the measure ν̄_{x_{1},...,x_{k}} defined by the formula

ν̄_{x_{1},...,x_{k}}(A) = ν((x_{1}, …, x_{k}) − A).

(In the remaining points where this integral is not meaningful we define the function f ∗ ν in an arbitrary way.)

We have not defined the convolution f ∗ g(x_{1}, …, x_{k}) of two functions f and g or the convolution f ∗ ν(x_{1}, …, x_{k}) of a function f and a measure ν in all points (x_{1}, …, x_{k}) ∈ R^{k}. But this restriction is not so disturbing as one might think at the first sight. As we shall see in problem 14, these convolutions exist for almost all points (x_{1}, …, x_{k}) ∈ R^{k} with respect to the Lebesgue measure in the k-dimensional Euclidean space. On the other hand, in typical applications these convolutions appear as the density function of a signed measure which is absolutely continuous with respect to the Lebesgue measure, and such density functions are determined only almost everywhere.

14.) If f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{k}) are two measurable and integrable functions on the space R^{k}, then the integral in formula (8) defining the convolution f ∗ g(x_{1}, …, x_{k}) is meaningful for almost all points (x_{1}, …, x_{k}) ∈ R^{k} with respect to the Lebesgue measure. The convolution f ∗ g is a finite and integrable function on R^{k}. If μ and ν are two signed measures of bounded variation on the space R^{k}, then their convolution μ ∗ ν has the same property. If μ and ν are two signed measures of bounded variation on the space R^{k}, and the measure μ has a density function f(u_{1}, …, u_{k}), that is μ(A) = ∫_{A} f(u_{1}, …, u_{k}) du for all measurable sets A ⊂ R^{k}, then the convolution of these signed measures, the signed measure μ ∗ ν has a density function, and it equals the function f ∗ ν. In particular, the function f ∗ ν(x) is integrable. If both signed measures μ and ν have a density function f(u_{1}, …, u_{k}) and g(u_{1}, …, u_{k}), then their convolution, the signed measure μ∗ν also has a density function, and it equals the convolution f ∗g.

15.) If ξ and η are two independent random vectors on the space R^{k}, the distribution of ξ is μ, the distribution of η is ν, then the distribution of the sum ξ + η is the convolution μ ∗ ν. If the random vector ξ has a density function f(u_{1}, …, u_{k}), then the sum ξ + η also has a density function, and it equals f ∗ ν. If ξ has a density function f and ν a density function g, then the sum ξ +η also has a density function, and it equals the convolution f ∗ g. As a consequence, if ξ_{j}, j = 1, …, n, are independent random vectors with distri-

bution functions F_{j}(x) = F_{j}(x_{1}, …, x_{k}), j = 1, …, n, S̄_{n} = ^{\frac{∑_{j=1}^{n} ξ_{j}−A}{B}} with some

norming factors A = (A_{1}, …, A_{k}) and B > 0, then the distribution of the expression S̄_{n} equals F_{1} ∗⋯∗F_{n}(Bx+A). If the random vectors ξ_{j} have density functions f_{j}, 1 ≤ j ≤ n, then S̄_{n} has a density function of the form Bf_{1} ∗ ⋯ ∗ f_{n}(Bx + A). f ∗ g = g ∗ f, μ ∗ ν = ν ∗ μ, (f ∗ g) ∗ h = f ∗ (g ∗ h), (μ_{1} ∗ μ_{2}) ∗ μ_{3} = μ_{1} ∗ (μ_{2} ∗ μ_{3}), i.e. the convolution operator is commutative and associative.

The next problem is about the relation between the convolution operator and Fourier transform.

16.) If f(u_{1}, …, u_{k}) and g(u_{1}, …, u_{k}) are two integrable functions on R^{k} with Fourier transforms

φ(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}u_{1}+···+t_{k}u_{k})}f(u_{1}, …, u_{k}) du_{1} … du_{k}

and

ψ(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}u_{1}+···+t_{k}u_{k})}g(u_{1}, …, u_{k}) du_{1} … du_{k},

then the Fourier transform of the convolution f ∗ g(u_{1}, …, u_{k}) is the function φ(t_{1}, …, t_{k})ψ(t_{1}, …, t_{k}). If μ and ν are two signed measures on R^{k} of bounded variation with Fourier transforms (or in other terminology with characteristic functions

φ(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}u_{1}+···+t_{k}u_{k})}μ( du_{1}, …, du_{k})

and

ψ(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}u_{1}+···+t_{k}u_{k})}ν( du_{1}, …, du_{k})

then the Fourier transform of the convolution μ∗ν equals φ(t_{1}, …, t_{k})ψ(t_{1}, …, t_{k}).

In an informal way the following problem can be formulated as the statement that the convolution is a smoothing operator. The convolution of two smooth functions is an even smoother function. For the sake of simplicity we shall consider only functions of one variable.

17.) Let f(u) and g(u) be two integrable functions. Let us assume that the derivatives \frac{d^{j}f(u)}{du^{j}} exist, they are integrable functions, and lim_{u→−∞} \frac{d^{j}f(u)}{du^{j}} = 0 for all integers

0 ≤ j ≤ k with some integer k ≥ 1. Let us also assume that the derivatives \frac{d^{j}g(u)}{du^{j}}

also exits, they are also integrable functions, and lim_{u→−∞} \frac{d^{j}g(u)}{du^{j}} = 0 for all integers

0 ≤ j ≤ l with all integers l ≥ 1. Then the derivative \frac{d^{k+l}f∗g(u)}{du^{k+l}} also exists, it is an

integrable function, and lim_{u→−∞} \frac{d^{k+l}f∗g(u)}{du^{k+l}} = 0.

Let f(u) and g(u) be two integrable functions such that the function f(u) has an analytic continuation to the domain {z: |Im z| < A} with some number A > 0. Let us further assume that ∫ |f(u + ix)| du < ∞ for all numbers |x| < A, and for all numbers ε > 0 and B > 0 there exists a constant K = K(A, B, ε) such that ∫_{|u|>K} |f(y − u + ix)g(u)| du < ε if |y| < B and |x| < A . Then the convolution f ∗ g is also an analytic function in the domain {z: |Im z| < A}.

D.) Convergence in distribution.

Although there is no convenient inversion formula for the expression of a distribution function by means of its characteristic function, the proof of the local limit theorems suggests that the convergence of a sequence of distribution functions can be proved by means of the convergence of their characteristic functions. This statement really holds, but to prove it we have to understand the situation better.

First we have to understand the meaning of the convergence of distribution functions. Before giving the formal definition let us consider a simple example which may explain some details of the definition.

Let us consider a sequence of negative numbers x_{n}, n = 1, 2, …, which satisfies the relation lim_{n→∞} x_{n} = 0, and put x_{0} = 0. Let us introduce the (degenerated) probability measures μ_{n}, n = 0, 1, 2, …, on the real line which are concentrated in the points x_{n}, n = 0, 1, 2, …, that is μ_{n}({x_{n}}) = 1, n = 0, 1, 2, . . .. It is natural to expect that by a right definition of convergence of probability distributions the sequence of these measures μ_{n} converges to the measure μ_{0} as n → ∞. Let us remark that the measures μ_{n}, n = 0, 1, 2, …, are determined by the distribution functions defined by the formula F_{n}(u) = 0 if u ≤ x_{n}, and F_{n}(u) = 1 if u > x_{n}, n = 0, 1, 2, …, i.e. μ_{n}([a, b)) = F_{n}(b) − F_{n}(a) for arbitrary pairs of numbers a < b. Observe that the relation lim_{n→∞} F_{n}(x) = F_{0}(x) holds for all numbers x ≠ 0, but this relation does not hold for x = 0, since F_{0}(0) = 0, and F_{n}(0) = 1 if n ≠ 0. This example shows that the naive picture by which the convergence of a sequence of distribution functions to a limit distribution function would mean that these distribution functions converge to the limit distribution function in all points of the real line would not supply a good definition. In the previous example the convergence does not hold in the point x = 0, where the limit distribution function is not continuous. The right definition of the convergence of distribution functions given below does not demand convergence in the points of discontinuity of the limit distribution function. We shall give this definition in the multi-dimensional case.

Definition of convergence of distribution functions. Let F_{n}(x_{1}, …, x_{k}), n = 0, 1, 2, …, be a sequence of k-dimensional, k ≥ 1, distribution functions. We say that the distribution functions F_{n} converge to a distribution function F_{0} in distribution, or in other words the probability measures μ_{n}, determined by the distribution functions F_{n}, n = 1, 2, …, converge in distribution to the measure μ_{0} determined by the distribution function F_{0}, or in a third terminology the random vectors ξ_{n} = (ξ_{n}^{(1)}, …, ξ_{n}^{(k)}), n = 1, 2, …, with distribution function F_{n} converge in distribution to a random vector ξ_{0} = (ξ_{0}^{(1)}, …, ξ_{0}^{(k)}), with distribution function F_{0} as n → ∞, if

lim_{n→∞} F_{n}(x_{1}, …, x_{k}) = F_{0}(x_{1}, …, x_{k})

in all points of continuity (x_{1}, …, x_{k}) ∈ R^{k} of the distribution function F_{0}(x_{1}, …, x_{k}). (Sometimes the convergence in distribution is called weak convergence in the literature.)

In later investigations the following Theorem A formulated below plays an important role. This result gives an equivalent condition for the convergence in distribution. The next problem is the proof of Theorem A. We remark that the same problem also appears in problem 1 of the series of problems Weak convergence of probability measures in metric space. (For the time being it exists only in Hungarian.)

Theorem A. The distribution functions F_{n}(x_{1}, …, x_{k}) converge in distribution to the distribution function F_{0}(x_{1}, …, x_{k}) if and only if

∫ f(x_{1}, …, x_{k}) dF_{n}(x_{1}, …, x_{k}) → ∫ f(x_{1}, …, x_{k}) dF_{0}(x_{1}, …, x_{k}) if n → ∞ (9)

for all continuous and bounded functions f(x_{1}, …, x_{k}) in the k-dimensional Euclidean space R^{k}.

18.) Let us prove Theorem A.

Let us remark that the characterization of the convergence in distribution given in Theorem A also plays an important role in other investigations. It helps to find the good definition of convergence in distribution in general topological spaces. The original definition is closely related to the notion of distribution functions which exploits the simple geometrical structure of the Euclidean space. Hence this definition has no natural generalization to more sophisticated spaces. The result of Theorem A helps to overcome this difficulty, and this is the reason why Theorem A appears in the series of problems mentioned above.

It is also worth mentioning that condition (9) in Theorem A can also be interpreted as a particular case of the weak convergence introduced in functional analysis. The probability measures in the k-dimensional Euclidean space can also be considered as continuous linear functionals on the space of bounded and continuous functions on R^{k} endowed with the usual supremum norm. Then relation (9) means that the probability measures μ_{n} determined by the distribution functions F_{n} weakly converge to the measure μ_{0} determined by the distribution function μ_{0}. Here we identify the measures μ_{n} with the linear functionals on the space of continuous and bounded functions they induce and apply the usual terminology of weak convergence in functional analysis. This fact may explain why convergence in distribution is sometimes called weak convergence.

Theorem A suggests a natural approach to study limit theorems in distribution with the help of Fourier analysis. For arbitrary k-dimensional vector the trigonometrical function e_{t}(x) = e_{t_{1},...,t_{k}}(x_{1}, …, x_{k}) = e^{i(t,x)} = e^{i(t_{1}x_{1}+···+t_{k}x_{k})} is continuous and bounded. Hence relation (9) demands in particular that for all vectors (t_{1}, …, t_{k}) the relation

φ_{n}(t_{1}, …, t_{k}) → φ_{0}(t_{1}, …, t_{k}), if n → ∞  (9′)

should hold, where φ_{n}(t_{1}, …, t_{k}) is the characteristic function of the distribution function F_{n}(t_{1}, …, t_{k}), n = 0, 1, 2, . . ..

Because of the nice properties of trigonometrical functions it is simpler to check relation (9′) than formula (9) dealing with general bounded and continuous functions. The question arises whether relation (9′) is sufficient to guarantee the validity of relation of (9) i.e. to prove convergence in distribution. We shall prove a more refined positive result in this direction, and this result serves as the basic tool in investigation of limit theorems in distribution. To prove this result we need such a result which states that the trigonometrical functions e_{t_{1},...,t_{k}}(x_{1}, …, x_{k}) = e^{i(t_{1}x_{1}+···+t_{k}x_{k})}, or more precisely their finite linear combinations are a sufficiently rich sub-class of the space of continuous and bounded functions. In our discussion we shall apply Weierstrass second approximation theorem which is a result in this spirit. In the Appendix we also supply a proof of Weierstrass second approximation theorem.

Weierstrass second approximation theorem. For all continuous and periodic by 2π functions f(t) and real numbers ε > 0 there exists a trigonometrical polynomial P_{n}(t) =

∑_{k=−n}^{n} a_{k}e^{ikt} such that

sup_{−∞<t<∞} |f(t) − P_{n}(t)| < ε.

(The degree of the polynomial P_{n} and the coefficients a_{k} in it depends both on the function f(·) and the real number ε > 0. If the function f(·) is real valued, then the coefficients a_{k} can be chosen in such a way that a_{−k} = ā_{k} for all indices k = 0, 1, …, n, where z̄ is the conjugate of the number z. Then also the polynomial P_{n}(t) is real valued.)

Also the following multi-dimensional version of this result holds. If f(t_{1}, …, t_{k}) is a continuous function in the k-dimensional Euclidean space which is periodic in all of its variables, i.e. f(t_{1} + 2j_{1}π, …, t_{k} + 2j_{k}π) = f(t_{1}, …, t_{k}) for all integers j_{1}, … j_{k}, and a real number ε > 0 is fixed, then there exists a trigonometrical polynomial of k variables

P_{n}(t_{1}, …, t_{k}) = ∑_{(j_{1},...,j_{k}): |j_{1}|+···+|j_{k}|≤n} a_{j_{1},...,j_{k}}e^{i(j_{1}t_{1}+···+j_{k}t_{k})},

where j_{1}, …, j_{k} are integers such that

|f(t_{1}, …, t_{k}) − P_{n}(t_{1}, …, t_{k})| < ε for all real numbers t_{1}, …, t_{k}.

An application of Weierstrass second approximation theorem together with an appropriate scaling makes possible to approximate a k-dimensional continuous function f(x_{1}, …, x_{k}) with arbitrary accuracy with a finite linear combination of trigonometrical sum ∑ a(t_{1}, …, t_{k})e^{i(t_{1}x_{1}+···+t_{k}x_{k})} in an arbitrary finite domain. On the other hand, given a probability measure μ the integral of the approximating trigonometrical can be expressed by means of the characteristic function φ(t_{1}, …, t_{k}) of the measure μ as ∑ a(t_{1}, …, t_{k})φ(t_{1}, …, t_{k}). If the domain where the trigonometrical sum well approximates the original function f is chosen sufficiently large, then the above expression gives a natural approximation of the integral ∫ f(x_{1}, …, x_{k}) dμ(x_{1}, …, x_{k}).

In such a way Weierstrass second approximation gives a good approximation of a continuous function in a finite domain by a trigonometrical sum whose integral with respect to a probability measure can be expressed by means of the characteristic function of this measure. But it yields no information about the contribution of a “small neighbourhood of infinity” to the integrals in formula (9). This is not the deficiency of Weierstrass second approximation theorem, but this difficulty belongs to the essence of the problem we investigate. In the following problems we shall show that a sequence of distribution functions may converge in distribution only if the contribution of an appropriate neighbourhood of the infinity in formula (9) is uniformly small for all distribution functions we consider. We shall formulate this statement in a more precise form, and we shall also show that the problem about convergence of distribution functions can be well investigated by means of their characteristic functions. First we introduce the following definition.

Definition of the relative compactness and tightness of distribution functions. Let a sequence of distribution functions F_{n}(t_{1}, …, t_{k}), n = 1, 2, …, be given in the k-dimensional Euclidean space, and let μ_{n} denote the probability measure on R^{k} determined by the distribution function F_{n}. We say that the sequence of distribution functions F_{n} or probability measures μ_{n} is relatively compact if all subsequences F_{n_{k}} (or μ_{n_{k}}), k = 1, 2, …, of the original sequence F_{n} or μ_{n} have a (sub)subsequence F_{n_{k_{j}}} (or μ_{n_{k_{j}}}), j = 1, 2, …, which is convergent in distribution.

We say that a sequence of distribution functions F_{n} or of probability measures μ_{n} determined by these distribution functions is tight if for all numbers ε > 0 there exists a number K = K(ε) such that the k-dimensional cube K(K)^{k} = \underbrace{[−K, K] × ⋯ × [−K, K]}_{k-fold product}

satisfies the inequality μ_{n}(K(K)^{k}) ≥ 1 − ε for all indices n = 1, 2, . . ..

19.) Let μ be a probability measure on the Borel measurable subsets of the k-dimensional Euclidean space, and let us fix a real number ε > 0. Then there exists a number K = K(μ, ε) > 0 in such a way that the k-dimensional cube K^{k}(K) = \underbrace{−[K, K] × ⋯ × [−K, K]}_{k-fold product} satisfies the inequality μ(K(K)^{k}) > 1 − ε. Let us show

with the help of this statement and Weierstrass second approximation theorem that a probability measure on R^{k} is uniquely determined by its characteristic function, that is if two probability measures μ_{1} and μ_{2} on R^{k} have the same characteristic function, then μ_{1} = μ_{2}. Furthermore, all signed measures μ of bounded variation on the k-dimensional space R^{k} is uniquely determined by their Fourier transform

φ(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}x_{1}+···+t_{k}x_{k})}μ( dx_{1}, …, dx_{k}), (t_{1}, …, t_{k}) ∈ R^{k}.

20.) Let us give an example which shows that the characteristic function of a distribution function in a finite interval does not determine this distribution function. That is let us show that for all numbers T > 0 there exist two different characteristic functions F_{1}(·) and F_{2}(·) such that their characteristic functions φ_{i}(t) = ∫ e^{itu} dF (u), i = 1, 2, satisfy the identity φ_{1}(t) = φ_{2}(t) for all real numbers −T ≤ t ≤ T.

21.) Let μ_{n}, n = 1, 2, …, be a sequence of probability measures on the k-dimensional Euclidean space R^{k}. This sequence of probability measures is relatively compact if and only if it is tight. In particular, all sequences of probability measures on R^{k} which are convergent in distribution are tight.

The hard part in the proof of problem 21 is to show that tightness implies relative compactness. To prove this the limit of an appropriate subsequence of distribution functions has to be constructed, and it has to be checked that the limit is really a distribution function. The proof could be simplified with the help of a classical result of functional analysis (Riesz theorem) which represents finite measures on a compact topological space as the linear functionals on the space of continuous functions on this topological space. This result together with a one-point compactification of the Euclidean space and the characterization of the convergence in distribution given in Theorem A may yield a simpler proof. The tightness condition would guarantee that the limit measure we obtain has no mass in the “infinity”. Nevertheless, I have described an elementary but more complicated proof.

I make a small détour. I describe a natural generalization of the results discussed in this section to sequences of probability measures on complete separable metric spaces. We shall not use these results in this note, but they are useful e.g. in the proof of the so-called functional central limit theorem. I shall explain these results without proof, although the results discussed in these section may be useful in these proofs.

Theorem A and formula (9) in it may give a natural idea for the definition of convergence in distribution of probability measures on a separable metric space. We say that a sequence of probability measures defined on a separable metric space (X, ρ) converges in distribution (or weakly, as this convergence is often called in the literature) to a probability measure μ on this metric space, if for all continuous and bounded functions on (X, ρ) the relation

lim_{n→∞} ∫ f(x)μ_{n}( dx) = ∫ f(x)μ( dx).

holds.

It may be worth remarking that a meausure μ on a separable metric space is uniquely determined by the integrals ∫ f(x)μ( dx) of all continuous, bounded functions f(x). This fact implies in particular, that the limit of probability measures on a metric space is uniquely determined. We can also define the relative compactness and tightness of sequences of probability measures on general metric spaces, and we can formulate theorems similar to the results of this section for them. At this point it is useful to restrict our attention to complete separable metric spaces.

A sequence of probability measurs μ_{n} on a complete separable metric space (X, ρ) is called relatively compact if each subsequence μ_{n_{k}} of μ_{n} has a sub-subsequence μ_{n_{k_{j}}} convergent in distribution (or in other words weakly). It can be proved that in a complete separable metric space of all probability measures μ and real numbers ε > 0 there exists a compact set K such that μ(K) > 1 − ε. Hence we can also define the tightness of a sequence of probability measures on a complete separable metric space. We say that a sequence μ_{n}, n = 1, 2, …, of probability meausures on a complete metric space (X, ρ) is tight if for all numbers ε > 0 there exists a compact set K = K(ε) such that μ_{n}(K) > 1 − ε for all indices n = 1, 2, . . .. The following theorem holds.

Theorem about the relation of tightness and relative compactness of probability measure sequences. Let μ_{n}, n = 1, 2, …, be a sequence of probability measures on a complete separable metric space (X, ρ). This sequence of probability meausures is relatively compact if and only if it is tight. In particular, all sequences of probability measures on a compact, complete separable metric space are (relatively) compact.

This result is useful in the study of convergence of probabilty measures on general metric spaces. (These problems are interesting, the functional central limit theorem e.g. can be obtained with the help of such an investigation.) To apply the above results in a concrete metric space we need a description of the compact sets of this metric space. The description of the compact sets in the C([0, 1]) space (in the Banach space of continuous functions on the interval [0, 1] with the supremum norm.) We shall describe them in the next theorem.

Theorem about the charactrization of the compact sets in the space C([0, 1]). The compact sets of C([0, 1]) (and in general of all metric spaces are closed.) A closed set F in the space C(0, 1]) is compact if and only if

(a) The set C = {x(0): x(·) ∈ F} is bounded, and

(b) The functions x(·) contained in the set F are universally uniformly continuous, i.e. for all numbers δ > 0 there is such a number η = η(δ) > 0 for which the relation

sup_{x(·)∈F} sup_{0≤s,t≤1, |t−s|≤η} |x(t) − x(s)| ≤ δ.

holds.

In certain investigations, e.g. in the proof of the functional central limit theorem the following consequence of the above results is useful.

Consequence. Let X_{n}(t), 0 ≤ t ≤ 1, n = 1, 2, …, be a sequence of C([0, 1])-valued random variables. The sequence X_{n}(t) converges weakly to some C([0, 1]) valued random variable X(t), 0 ≤ t ≤ 1, as n → ∞ if and only if

(a) For all finite sequences of numbers 0 ≤ t_{1} < ⋯ < t_{k} ≤ 1 the random vectors (X_{n}(t_{1}), …, X_{n}(t_{k})) converge in distribution as n → ∞.

(b) For all numbers ε > 0 and δ > 0 there is a number η = η(δ, ε) > 0 such that

P (ω: sup_{0≤s,t≤1, |t−s|≤η} |X_{n}(t, ω) − X_{n}(s, ω)|) ≤ δ) > 1 − ε

for all indices n = 1, 2, . . ..

Remark. Point (a) of the above consequence guarantees that the distributionos of all convergent subsequence of the C([0, 1]) valued random variables X_{n}(·) have the same limit. Furthermore, it can be proved with the help of point (b) and the convergence of the (real valued) random variables X_{n}(0), n = 1, 2, …, that the μ_{n} distributions of the random variables X_{n}(·) constitute a relatively compact set.

E.) The relation between the convergence of distribution functions and the convergence of their characteristic functions.

To express the conditions of convergence in distribution by means of characteristic functions it is useful to give the necessary and sufficient condition of tightness (or of relative compactness which property is equivalent to tightness by the result of problem 20) of probability measures with the help of distribution functions. This is the content of the next problem.

22.) Let F_{n}(u), n = 1, 2, …, be a sequence of distribution functions on the real line with characteristic functions φ_{n}(t), n = 1, 2, . . .. This sequence of distribution functions F_{n}(·) is tight if and only if

lim_{δ→0} lim sup_{n→∞} \frac{1}{2δ} ∫_{−δ}^{δ} Re (1 − φ_{n}(t)) dt = 0,  (10)

where Re z denotes the real part of the complex number z.

Although we shall not need the following observation let us also show that relation (10) is equivalent to the following relation (10′):

lim_{δ→0} sup_{n} \frac{1}{2δ} ∫_{−δ}^{δ} Re (1 − φ_{n}(t)) dt = 0.  (10′)

With the help of the result of problem 22 and Weierstrass second approximation theorem we can give the necessary and sufficient condition of convergence of a sequence of distribution function in distribution in the language of characteristic function. We formulate and prove this result which we call because of its importance fundamental theorem.

The Fundamental Theorem about the convergence of distribution functions. Let F_{n}(u_{1}, …, u_{k}) be a sequence of distribution functions on the k-dimensional Euclidean space R^{k} with characteristic functions φ_{n}(t_{1}, …, t_{k}), n = 1, 2, . . .. If the characteristic functions φ_{n}(t_{1}, …, t_{k}) have the limit φ_{0}(t_{1}, …, t_{k}) = lim_{n→∞} φ_{n}(t_{1}, …, t_{k}) in

all points (t_{1}, …, t_{k}) ∈ R^{k}, and the limit function φ_{0}(t_{1}, …, t_{k}) is continuous in the origin, then there exists a distribution function F_{0}(u_{1}, …, u_{k}) in the k-dimensional space whose characteristic function is this limit function φ_{0}(t_{1}, …, t_{k}). Besides, the distribution functions F_{n}(u_{1}, …, u_{k}) converge in distribution to this distribution function F_{0}(u_{1}, …, u_{k}). Moreover, the condition about the continuity of the limit function φ_{0}(t_{1}, …, t_{k}) in the origin can be slightly weakened. The above statements also hold if we only demand that the restriction of the function φ_{0}(t_{1}, …, t_{k}) to each coordinate axis is continuous in the origin.

In the converse direction we state that if a sequence of k-dimensional distribution functions F_{n}(u_{1}, …, u_{k}), n = 1, 2, …, converges in distribution to a k-dimensional distribution function F_{0}(u_{1}, …, u_{k}), and φ_{n}(t_{1}, …, t_{k}), n = 0, 1, …, denotes the characteristic function of the distribution function F_{n}(u_{1}, …, u_{k}), then φ_{0}(t_{1}, …, t_{k}) = lim_{n→∞} φ_{n}(t_{1}, …, t_{k}) in all points (t_{1}, …, t_{k}) ∈ R^{k}. Furthermore, this convergence is

uniform in all compact subsets of the space R^{k}.

We have formulated the above fundamental theorem in a slightly stronger form than it is done in the literature. We remarked that if we want to deduce the convergence of the distribution functions from the convergence of the characteristic functions, then it is enough to know that the restriction of this limit function to the coordinate axes is continuous in the origin. We have made this remark because its proof causes no problem, and it is useful in certain multi-dimensional application of the result. In particular, it simplifies the solution of problem 46 in this series of problems.

We want to prove the above formulated Fundamental Theorem. To do this let us first prove the following problem interesting in itself.

23.) Let ξ^{(n)} = (ξ_{1}^{(n)}, …, ξ_{k}^{(n)}) be a sequence of k-dimensional random vectors and let φ_{n}(t_{1}, …, t_{k}) denote their characteristic functions. Prove (with the help of problem 22) that if the characteristic functions φ_{n}(t_{1}, …, t_{k}) converge to a function φ(t_{1}, …, t_{k}) in a small neighbourhood of the origin which is continuous in the origin, then the distribution functions of the random vectors ξ^{(n)} = (ξ_{1}^{(n)}, …, ξ_{k}^{(n)}) are tight. Moreover, this tightness property also holds under the weaker condition that the restrictions of the limit function φ(t_{1}, …, t_{k}) to the coordinate axes are continuous in the origin.

24.) Let us prove the Fundamental Theorem about the convergence of distribution func-

tions. 25.) Let us show an example for a sequence of characteristic functions φ_{n}(t), n =

0, 1, 2, …, on the real line such that lim_{n→∞} φ_{n}(t) = φ_{0}(t), this convergence is uniform

on all finite intervals, but it is not uniform on the whole real line.

Let us make some comments about the above formulated Fundamental Theorem.

i.) The fundamental theorem gives conditions which automatically guarantee that the limit of characteristic functions is also a characteristic function. This result is very useful in a more systematic investigation of limit theorems. For instance it makes possible to describe all possible limit distributions which appear as the limit of appropriately normalized partial sums of independent random variables.

ii.) In the Fundamental Theorem we stated the limit of characteristic functions is again a characteristic function if it is continuous in the origin. On the other hand, we know that the characteristic functions are uniformly continuous in the whole space R^{k} (see for instances the result of problem 12). Let us remark that the class of the characteristic functions is a special class of all continuous functions. There is a nontrivial characterization of characteristic functions. A famous result of the analysis, the Bochner theorem yields such a characterization. This result plays an important role both in the analysis and in the probability theory. Although we shall not use this result in this series of problems, we shall prove Bochner’s theorem in the second part of this series of problems. The information obtained in the proof of limit theorems are useful in the proof of Bochner’s theorem.

iii.) The condition that the limit of the characteristic functions is continuous in the origin guarantees that the distribution functions corresponding to them are tight. Informally saying this means that “no mass flows out to the infinity”. If some mass “may flow out to the infinity”, then the asymptotic behaviour of the characteristic functions does not supply such a simple description of the asymptotic behaviour of the distribution functions corresponding to them as the Fundamental Theorem. The content of the next problem is to show this with the help of an example.

26.) Let us show an example for a sequence of probability measures μ_{n}, n = 1, 2, …, on the real line which satisfy the relation lim_{n→∞} μ_{n}(K) = 0 for all bounded sets K, and the characteristic functions φ_{n}(t) of the measures μ_{n} either

a.) have a limit in all points of the real line or b.) do not have a limit function on the real line.

F.) The relation between the behaviour of a function or measure and its Fourier transform.

If we want to investigate the behaviour of distribution functions by means of their characteristic functions, or more generally if we want to study the properties of a function or a measure by means of its Fourier transform, then we need a good “dictionary” which describes how the properties of a function or a measure are reflected in their Fourier transform, and what kind of properties of the original function or measure follow from certain properties of their Fourier transform. The “dictionary” described in this section will be by no means complete, but it also contains some results which we do not need in our further investigation. Actually, if we only wanted to get a proof of the central limit theorem and were not interested in the more intricate questions related to this problem, then the proof of the first statement in problem 27 in this section would suffice for our purposes. The results of this section will be formulated only in the one-dimensional case, although the proof of their multi-dimensional generalization would cause no essential difficulty.

Informally the content of this “dictionary” can be formulated in the following way: The smoother a function or a measure (more precisely its density function) on the real line is, the faster its Fourier transform tends to zero at infinity. The less mass a probability measure on the real line contains in a small neighbourhood of the infinity (or in other words the more moments the absolute value of a random variable with this distribution has), the smoother the Fourier transform of this probability measure is. Besides, the derivatives of the Fourier transform at the origin determine the moments of a random variable whose distribution this probability measure is.

In the other direction: If the characteristic function of a distribution function is smooth, and it is enough to assume this smoothness property in a small neighbourhood of the origin, then the smoother the characteristic function is the less mass the probability measure with this characteristic function contains in the vicinity of the infinity. On the other hand, the faster the characteristic function tends to zero as its argument tends to infinity, the smoother the original distribution function is. Furthermore, some results also show that if a function is very smooth everywhere except some points where the function has a singularity, then the behaviour of its Fourier transform in the vicinity of the infinity very well reflects the character of these singularities. We shall discuss this last statement, — which we shall not need later — only superficially, and omit the proof.

Actually, the statement of problem 22 can also be considered as an element of this dictionary, and its content is in full agreement with the above sketched heuristic picture. The statement that a class of distribution functions is tight means that there is an appropriate neighbourhood of the infinity which has a small measure with respect to all measures determined by this class of distribution functions. On the other hand, formula (10) which is equivalent to this property has the content that the characteristic functions of this distribution functions satisfy some sort of uniform continuity in a small neighbourhood of the origin. (Let us recall that 1 − φ(0) = 0.)

The results of problems 17 and 16 are also consistent with the above heuristics. The result of problem 17 formulates a statement by which the convolution of two smooth density functions is a function which is even smoother than the functions taking part in the convolution. On the other hand, the Fourier transform of the convolution of two functions equals the product of their Fourier transforms. By the above sketched heuristic argument the smoothness of a function depends on how fast its Fourier transform tends to zero in the neighbourhood of infinity. On the other hand, the product of two functions tending to zero in the vicinity of the infinity tends to zero faster than the functions in this product. This fact corresponds to the smoothing property of the convolution operator in the language of Fourier transforms.

27.) If the absolute value of the random variable ξ has finite k-th moment, i.e. E|ξ^{k}| < ∞, then the characteristic function φ(t) = Ee^{itξ} of the random variable ξ is k times continuously differentiable, and \frac{d^{j}φ(t)}{dt^{j}} |_{t=0} = i^{j}Eξ^{j} for all numbers 0 ≤ j ≤ k.

A random variable ξ has exponential moments in a small neighbourhood of the origin if and only if the distribution function of the random variable |ξ| tends to 1 exponentially fast in the infinity, i.e. Ee^{uξ} < ∞ for all numbers |u| ≤ t with some sufficiently small number t > 0 if and only if P (|ξ| > x) < Ce^{−αx} for all x > 0 with some appropriate constants C > 0 and α > 0. In this case the characteristic function φ(t) = Ee^{itξ} has an analytic extension to the domain {z: |Re z| < a} with some appropriate number a > 0.

28.) If the absolute value of the function f(u) is integrable on the real line, then the Fourier transform φ(t) = ∫ e^{itu}f(u) du of the function f(u) satisfies the relations lim_{t→∞} φ(t) = 0 and lim_{t→−∞} φ(t) = 0. (Riemann lemma.)

If the absolute value of the function f(u) is integrable, the function f(u) is k-times differentiable, and the absolute value of the functions \frac{d^{j}f(u)}{du^{j}} is integrable for all numbers 1 ≤ j ≤ k, then ∫_{−∞}^{∞} e^{itu}f(u) du = o ((1 + |t|)^{−k}) in case |t| → ±∞. If the function f(u) has an analytic continuation to the domain {z: |Re z| < A} with some A > 0, | ∫ |f(u + iv)| du < ∞, if |v| < A, then φ(t) = ∫ e^{itu}f(u) du = O (e^{−α|t|}) as t → ±∞ with some constant α > 0.

29.) Let ξ be a random variable with non-degenerated distribution, (i.e. assume that ξ does not equal a constant with probability 1), and let φ(t) = Ee^{itξ} denote its characteristic function. There exists a constant t ≠ 0 such that |φ(t)| = 1 if and only if ξ is a lattice valued random variable. (The notion of lattice valued random variable is introduced in Definition A.) If the values of the random variable ξ are concentrated on a lattice of width h (as on the rarest lattice), then the relation |φ(t)| = 1 holds for the characteristic function of the random variable ξ if and only if t = 2π \frac{k}{h} with some k = 0, ±1, ±2, . . .. If ξ is a lattice valued, non constant random variable, then there exits a rarest lattice of width h where the distibution of ξ is concentrated. If the random variable ξ is not lattice valued, then sup_{A≤|t|≤B} |φ(t)| < 1

for all pairs of numbers 0 < A < B < ∞. 30.) Let the distribution of the random variable ξ be given by the formula P (ξ = 1) =

P (ξ = \sqrt{2}) = P (ξ = −1) = P (ξ = −\sqrt{2}) = \frac{1}{4}. Then the characteristic function φ(t) = Ee^{itξ} satisfies the inequality |φ(t)| < 1 if t ≠ 0, and also the relation lim sup_{t→∞} |φ(t)| = 1 holds. The relations lim sup_{t→∞} |φ(t)| = 1 and |φ(t)| < 1 if t ≠ 0

also hold in the more general case when the values of the random variable ξ are concentrated in finitely or countably many points but ξ is not a lattice valued random variable.

Remark 1: By a classical result of measure theory all (finite) measures can be decomposed (in a unique way) as the sum of an absolute continuous, a discrete (i.e. concentrated in countably many points) and a singular measure. (An absolute continuous measure has a density function with respect to the Lebesgue measure, a singular measure is concentrated on a set of Lebesgue measure zero, but all points have measure zero with respect to it.) By the result of problem 28 the Fourier transform φ(t) of an absolute continuous measure tends to zero as t → ±∞, and the smoother density function the measure has the faster this convergence is. The results of problems 29 and 30 tell that the Fourier transform of a discrete measure behaves in the opposite way in the neighbourhood of infinity. The lim sup of the absolute value of the Fourier transform of a discrete measure equals the measure of the real line with respect to this measure. The behaviour of a singular measure in the neighbourhood of the infinity cannot be characterized in a similar simple way.

Remark 2: Let us consider in the neighbourhood of infinity the behaviour of the Fourier transform f̃(u) of an integrable function f(s) which is sufficiently many times differentiable everywhere except a point a ∈ R^{1}. Let the function f have a singularity of the form f(s) ∼ C|s − a|^{α}, α > −1, α ≠ 2k, k = 0, 1, 2, …, in the neighbourhood of the point a. In this case the Fourier transform of the function f behaves asymptotically as f̃(u) ∼ C̄e^{iua}u^{−α−1} if u → ∞, where the constant C̄ ≠ 0 can be given explicitly. We give a short heuristic explanation of this result.

Because of the smoothness properties of the function f(s) the asymptotic behaviour of its Fourier transform in the neighbourhood of the infinity is determined by the singularity of the function f in the point a, and the Fourier transform equals asymptotically the expression

g̃(u) = C ∫_{−∞}^{∞} e^{ius}|s − a|^{α} ds = Ce^{iau} ∫_{−∞}^{∞} e^{ius}|s|^{α} ds
= Ce^{iau}u^{−α−1} ∫_{−∞}^{∞} e^{is}|s|^{α} ds = C̄e^{iau}u^{−α−1}

as u → ∞.

The above calculation was rather incorrect. The main problem is that the integrals considered here are meaningless because of the factor |s|^{α} in the integrand, at least as usual Lebesgue integrals. Nevertheless, this calculation supplies a correct result. Moreover, by replacing the integral to the imaginary axis we can express the constant C̄ by means of the Γ(·) function. If the function f has several similar singularities, then their effects sum up if the Fourier transform of the function in the neighbourhood of the infinity is described.

Although we shall not apply the results sketched in this Remark 2, such kind of results play an important role in certain investigations of probability theory and analysis. The cases α = 2k, k = 0, 1, …, had to be excluded, because in this case the function f(s) is smooth in a small neighbourhood of the point a. The exceptional behaviour of the function |s − a|^{α} with such parameters α has deep consequences in certain problems of statistical physics.

The results of the subsequent problems 31–34 yield information about the behaviour of a probability measure by means of the properties of its Fourier transform.

31.) If the characteristic function φ(t) = Ee^{itξ} of a random variable ξ is twice differentiable in the origin, then the random variable ξ has finite second moment, i.e.

Eξ^{2} < ∞. Let us prove by induction that if the characteristic function of the random variable ξ is 2k-times differentiable in the origin, then Eξ^{2k} < ∞.

32.) If the characteristic function φ(t) = Ee^{itξ} of the random variable ξ is differentiable in a small neighbourhood of the origin, and the derivative is a Lipschitz α function in a small neighbourhood of the origin with some parameter α > 0, i.e. |φ′(t)−φ′(s)| < C|t − s|^{α} with an appropriate constant C > 0 if |s| < ε, |t| < ε with some number ε > 0, then E|ξ| < ∞. Under these conditions also the following somewhat stronger result holds: P (|ξ| > u) < const. u^{−1−α} for all numbers u > 0. If the characteristic function φ(t) is 2k + 1-times differentiable in a small neighbourhood of the origin, and the 2k + 1-th derivative is a Lipschitz α function with some constant α > 0, then E|ξ|^{2k+1} < ∞.

Remark: It follows from the results of problems 31 and 27 that the second moment of a random variable is finite if and only if the characteristic function of the random variable is twice differentiable in the origin. If the absolute value of a random variable has finite expected value, then the characteristic function of this random variable is differentiable in the origin (and everywhere). But to guarantee the existence of a finite expected value of the absolute value of the random variable ξ we have imposed a stronger condition than the differentiability of the characteristic function in the origin. (This would be the natural analog of the statement in problem 31.) Although the conditions of 32 could be weakened, the differentiability of the characteristic function of a random variable in the origin does not suffice for the finiteness of the expectation of the absolute value of this random variable.

Indeed, a necessary and sufficient condition of the differentiability of the characteristic function of a random variable can be expressed with the help of the distribution function of this random variable in a relatively simple way. Since this problem arises in a natural way in the investigation of the weak law of large numbers, this result is also contained in problem 12 of the series of problems Convergence in probability and with probability one. (For the time being it exists only in Hungarian). Let us formulate this result. The characteristic function of a random variable with distribution function F has a finite derivative ia, −∞ < a < ∞, in the origin if and only if

lim_{x→∞} x [F (−x) + (1 − F (x))] = 0, and lim_{u→∞} ∫_{−u}^{u} xF ( dx) = a.

33.) If the characteristic function of a random variable ξ has an analytic continuation to a small neighbourhood of the origin, then there exists a number α > 0 such that P (|ξ| > x) ≤ const. e^{−αx} for all numbers x > 0.

34.) If the characteristic function φ(u) = Ee^{iuξ} of a random variable ξ is integrable, i.e. ∫ |φ(u)| du < ∞, then the random variable ξ has a density function f(x). If |φ(u)| < const. |u|^{−(k+1+ε)} with some number ε > 0, then the density function f(x) has continuous and bounded k-th derivative. If |φ(u)| < const. e^{−α|u|} with some number α > 0, then the density function f(x) is analytic.

Remark 1: The statement of problem 34 can be slightly generalized. To guarantee the existence of a density function of the random variable ξ it is enough to assume that the square of the characteristic function φ(u) = Ee^{iuξ} is integrable. This statement holds because the inverse Fourier formula which expresses the density function of a random variable by means of its characteristic function in the form written down in relation (6) also holds if the characteristic function is only square integrable. Only in this case the integral in formula (6) has to be considered not as a usual Lebesgue integral. It has to be defined by the extension of the L_{2} isomorphism between a function and its Fourier transform which follows from the Parseval formula. This is a classical result of the Fourier analysis, but we do not discuss it here. It implies the result of this Remark 1.

Remark 2: We remarked in the formulation of problem 8 that to guarantee the local central limit theorem the condition about the integrability of the characteristic function of the random variables we have considered can be weakened. It is enough to assume that a sufficiently large power of the characteristic function is integrable. The exponential distribution is an example for such a distribution function whose characteristic function is not integrable, but it is square integrable (see problem 13.c). Hence only the strengthened form of the result of problem 8 yields the local central limit theorem in this case. The results of this section may explain the deeper background of this example.

Let us observe that the density function of the exponential distribution function has a discontinuity in the origin, and the non-integrability of its characteristic function is caused by this discontinuity. The convolution of the exponential distribution function with itself yields a distribution function with a smoother density function whose characteristic function is the square of the exponential distribution function. This fact may explain why the square of the Fourier characteristic function of the exponential distribution is integrable. A similar picture arises in the cases of random variables with such a density function which is sufficiently smooth except some exceptional points where the density function has a not too strong singularity.

G.) The central limit theorem.

The previous results enable us to prove the central limit theorem by means of characteristic functions.

35.) Let ξ_{1}, ξ_{2}, …, be a sequence of independent an identically distributed random variables, Eξ_{1} = 0, Eξ_{1}^{2} = 1. Put S_{n} = ξ_{1} + ⋯ + ξ_{n}. Let us show that the random variables \frac{S_{n}}{\sqrt{n}}, n = 1, 2, …, converge in distribution to the standard normal distribution function.

We want to prove the central limit theorem in the more general case when the normalized partial sums of independent but not necessarily identically distributed random variables are considered. It is worth formulating this question in an even more general setting when for all positive integers k we consider the sum of independent random variables indexed with this number k, but we assume nothing about the relation between random variables with different indices k. If the random variables indexed with the same number k are uniformly small, then under some weak conditions the sums of the random variables with the same index k have a Gaussian limit as k → ∞.

Essentially we can give the necessary and sufficient condition for the convergence of the distributions of the above sums to a Gaussian law. To formulate this statement in an explicit form we introduce the notion of triangular arrays. The central limit theorem for normalized partial sums of independent but not necessarily identically distributed random variables can be deduced as the special case of the central limit theorem for triangular arrays.

The definition of triangular arrays. A system of random variables

ξ_{1,1}, …, ξ_{1,n_{1}}
⋮  ⋮
ξ_{k,1}, …, ξ_{k,n_{k}}
⋮  ⋮

with k = 1, 2, … is called a triangular array, if the random variables ξ_{k,1}, …, ξ_{k,n_{k}} with the same first index k are independent. (We assume nothing about the relation of the random variables with different first index k.)

To prove the central limit theorem by means of characteristic function it is worth proving the following technical lemma. It gives an estimate of the difference of the function e^{it} and the sum of the first k terms of its Taylor series. 36.) For all non-negative integers k and real numbers t

|e^{it} − (1 + \frac{it}{1!} + ⋯ + \frac{(it)^{k}}{k!} )| ≤ \frac{|t|^{k+1}}{(k + 1)!}.  (11)

If we want to prove the central limit theorem for triangular arrays with the help of characteristic functions, then the following approach is natural. Let us consider the characteristic functions φ_{k,j}(·) of the random variables ξ_{k,j}, 1 ≤ j ≤ n_{k}. We have to show that the product of these characteristic functions converge to the characteristic function of a Gaussian random variable. Let us consider the logarithm of these products. Then we have to investigate the sum of the functions log φ_{k,j}(t) with a fixed index k and fixed argument t. If we have a condition which says that the random variables ξ_{k,j} are small in an appropriate sense, then it is natural to expect that φ_{j,k}(t) ∼ 1 for a fixed number t, and the error of the approximation log φ_{k,j}(t) ∼ (1 − φ_{k,j}(t)) gives a negligible error. This enables to reduce the proof to the investigation of a simpler sum. The goal of the next problem is to give a precise formulation and justification of the above heuristic argument. It makes possible to study both the necessary and sufficient conditions of the central limit theorem.

37.) Let us consider the characteristic function φ(t) of a random variable ξ with a fixed argument t. a.) If Eξ = 0 and Eξ^{2} ≤ ε with a sufficiently small positive number ε = ε(t) > 0,

then |1 − φ(t)| ≤ \frac{t^{2}}{2} Eξ^{2}, and | log φ(t) + (1 − φ(t))| ≤ t^{4} (Eξ^{2})^{2}. b.) Let ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}, be a triangular array such that Eξ_{k,j} = 0,

k = 1, 2, …, 1 ≤ j ≤ n_{k}, lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1, and let the elements of triangular

array satisfy the uniform smallness condition lim_{k→∞} ( sup_{1≤j≤n_{k}} Eξ_{k,j}^{2}) = 0. Let

φ_{k,j}(t) = Ee^{itξ_{k,j}}, −∞ < t < ∞, denote the characteristic function of the random variable ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}. The random sums S_{k} =

∑_{j=1}^{n_{k}} ξ_{k,j}, 1 ≤ k < ∞, converge in distribution to the Gaussian distribution with

expectation m and variance σ^{2} as k → ∞ if and only if

lim_{k→∞} ∑_{j=1}^{n_{k}}(φ_{k,j}(t) − 1) = −\frac{σ^{2}t^{2}}{2} + imt  (12)

for all numbers −∞ < t < ∞.

Although we are first of all interested in the question when the partial sums S_{k} of a triangular array satisfying the conditions of problem 37 converge in distribution to the standard normal distribution, it was also useful to formulate the condition for the convergence of these random sums to a general Gaussian distribution with expectation m and variance σ^{2}. This knowledge helps us to find also the necessary conditions of the central limit theorem. We shall also show such examples where the appropriately normalized partial sums of independent random variables with expectation zero converge to a Gaussian distribution with expectation m ≠ 0.

In the investigation of the central limit theorem for triangular arrays the so-called Lindeberg condition formulated below plays a fundamental role. We shall see that the

partial sums S_{k} = ∑_{j=1}^{n_{k}} ξ_{k,j} of a triangular array satisfying part b) of problem 37 converge

to the standard normal distribution function if and only if this triangular array satisfies the Lindeberg conditions.

The definition of the Lindeberg condition: Let ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}, be such a triangular array for which Eξ_{k,j} = 0, k = 1, 2, …, 1 ≤ j ≤ n_{k}, and

lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1. This triangular array satisfies the Lindeberg condition if and only

if for all positive real numbers ε > 0

lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2}I ({|ξ_{k,j}| > ε}) = 0,

where I(A) denotes the indicator function of a set A.

38.) Let ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}, be a triangular array which satisfies the relations

Eξ_{k,j} = 0, k = 1, 2, …, 1 ≤ j ≤ n_{k}, lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1 and the Lindeberg

condition. Then

a.) the triangular array ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}, also satisfies the uniform

smallness condition lim_{k→∞} ( sup_{1≤j≤n_{k}} Eξ_{k,j}^{2}) = 0.

b.) The random sums S_{k} = ∑_{j=1}^{n_{k}} ξ_{k,j}, 1 ≤ k < ∞, converge in distribution to the

standard normal distribution (i.e. to the normal distribution with expectation zero and variance 1) if k → ∞.

The above result can be reversed in the following way.

39.) Let ξ_{k,j}, k = 1, 2, …, 1 ≤ j ≤ n_{k}, be a triangular array such that Eξ_{k,j} = 0,

k = 1, 2, …, 1 ≤ j ≤ n_{k}, lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1, and it satisfies the uniform smallness

condition lim_{k→∞} ( sup_{1≤j≤n_{k}} Eξ_{k,j}^{2}) = 0. Let us also assume that the random sums

S_{k} = ∑_{j=1}^{n_{k}} ξ_{k,j}, 1 ≤ k < ∞, converge in distribution to a normal distribution function

with variance 1 and arbitrary expected value if k → ∞. Then the triangular array ξ_{k,j}, 1 ≤ k < ∞, 1 ≤ j ≤ n_{k}, also satisfies the Lindeberg condition.

The content of the Lindeberg condition is that it guarantees that the too large values (which have the same order as the square-root of the variance of the sum) have a negligible influence in the central limit theorem. The contribution of such extremely large values have a small influence both on the variance and the distribution of the sum we investigate. The next problem expresses such a fact. This result together with the result of problem 42 formulated later have the following consequence: Let us consider a triangular array which satisfies the Lindeberg condition. If we truncate the too large values (those which are greater than a fixed positive number ε > 0) and then normalize the truncated random variables in such a way that their expected value be zero, then the sum of these modified random variables have the same limit in distribution as the sums of the original random variables.

It is worth mentioning that some modification of the above argument may help to find a different proof of the central limit theorem if the Lindeberg condition holds. It can be seen that we may also truncate and normalize the elements of the k-th row of the triangular array at a level ε_{k} > 0 with such a sequence ε_{k} → 0 which converges to zero sufficiently slowly instead of a fixed number ε > 0 and we can guarantee that the new triangular array is equiconvergent with the original one. Then some not too hard calculation shows that all moments of the sums made from this truncated and normalized random variables converge to the corresponding moments of a random variable with standard normal distribution. On the other hand, it is known from the general theory that this fact implies the central limit theorem. Now we omit the details. We return to this remark in Part II. of this series of problems.

40.) Let a triangular array ξ_{k,j}, 1 ≤ j ≤ n_{k}, lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1, Eξ_{k,j} = 0, satisfy

the Lindeberg condition. Let us fix a real positive number ε > 0, and define the random variable ξ̄_{k,j} = ξ̄_{k,j}(ε) = ξ_{k,j}I(|ξ_{k,j}| < ε) − Eξ_{k,j}I(|ξ_{k,j}| < ε). Then

lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ̄_{k,j}^{2} = 1. Let us also define the partial sums S̄_{k} = ∑_{j=1}^{n_{k}} ξ̄_{k,j} and S_{k} =

∑_{j=1}^{n_{k}} ξ_{k,j}. Then the differences S_{k} − S̄_{k} converge stochastically to zero if k → ∞.

Let us emphasize that to guarantee the validity of the Lindeberg condition in problem 39 we have imposed not only the condition that the random sums S_{k} converge in distribution to a normal law. We also demanded that the limit distribution have the “right” variance 1. We also shall show examples where the triangular array ξ_{k,j}, 1 ≤

k < ∞, 1 ≤ j ≤ n_{k}, satisfies the uniform smallness condition lim_{k→∞} ( sup_{1≤j≤n_{k}} Eξ_{k,j}^{2}) =

0, it does not satisfy the Lindeberg condition, and the random sums S_{k} converge in distribution to a normal distribution. But in this example the variance of the limit distribution is less than 1. Before the discussion of such examples we present some results which give useful sufficient conditions for the validity of the Lindeberg conditions. We shall consider only normalized partial sums of independent random variables instead of triangular arrays. First we formulate the appropriate version of the Lindeberg condition for sequences of independent random variables.

The definition of the Lindeberg condition for sequences of independent random variables. Let ξ_{n}, n = 1, 2, …, be a sequence of independent random variables, for which Eξ_{n} = 0, σ_{n}^{2} = Eξ_{n}^{2} < ∞, n = 1, 2, …, and the sequence of num-

bers s_{n}^{2} = ∑_{k=1}^{n} σ_{k}^{2}, n = 1, 2, …, satisfies the relation lim_{n→∞} s_{n}^{2} = ∞. The sequence ξ_{n},

n = 1, 2, …, satisfies the Lindeberg condition if

lim_{n→∞} \frac{1}{s_{n}^{2}} ∑_{k=1}^{n} Eξ_{k}^{2}I(\{|ξ_{k}| > εs_{n}\}) = 0.

for all numbers ε > 0.

Given a sequence ξ_{n}, n = 1, 2, …, of independent random variables for which

Eξ_{n} = 0, σ_{n}^{2} = Eξ_{n}^{2} < ∞, and the sequence s_{n}^{2} = ∑_{k=1}^{n} σ_{k}^{2} satisfies the relation lim_{n→∞} s_{n}^{2} =

∞, define the triangular array ξ_{k,j}, k = 1, 2, …, j = 1, …, n_{k}, by the formulas n_{k} = k and ξ_{k,j} = \frac{ξ_{j}}{s_{k}}, if 1 ≤ j ≤ k with the help of this sequence. The original sequence of random variables ξ_{n}, n = 1, 2, … satisfies the Lindeberg condition if and only if the triangular array ξ_{k,j} defined by its help satisfies it. Hence the central limit theorem and its converse formulated for triangular arrays can be reformulated for normalized partial sums of independent random variables. (In this reformulation the uniform smallness

property of the summands have the form lim_{n→∞} ^{\frac{max_{1≤k≤n} σ_{k}^{2}}{s_{n}^{2}}} = 0.) In the next problem we formulate some properties which imply the Lindeberg condition.

41.) Let ξ_{n}, n = 1, 2, …, be a sequence of independent random variables for which

Eξ_{n} = 0, σ_{n}^{2} = Eξ_{n}^{2} < ∞, n = 1, 2, …, lim_{n→∞} s_{n}^{2} = ∞, where s_{n}^{2} = ∑_{k=1}^{n} σ_{k}^{2}. This

sequence of random variables satisfies the Lindeberg condition if one of the following properties holds.

a.) E|ξ_{k}|^{2+α} < ∞, for all numbers k = 1, 2, … with some constant α > 0, and

lim_{n→∞} ^{\frac{(∑_{k=1}^{n} E|ξ_{k}|^{2+α})^{2/(2+α)}}{s_{n}^{2}}} = 0. In particular, this condition holds if Eξ_{k}^{2} ≥ K with some constant K > 0 for all indices k = 1, 2, …, and besides the relation

E lim_{k→∞} k^{−α/2}E|ξ_{k}|^{2+α} = 0 holds.

b.) The independent random variables ξ_{n}, n = 1, 2, …, are uniformly distributed. (This means that the result of problem 35 follows from the central limit theorem formulated under general conditions.)

We want to show an example for a sequence of independent random variables with expectation zero and finite variance which satisfies the uniform smallness condition, the normalized partial sums made with the help of this sequence converge in distribution to the standard normal distribution, but the normalization is not the natural one, that is we divide the partial sums not with the square-root of their variances. Such an example shows that the normalized partial sums of independent random variables may satisfy the central limit theorem (with a non-usual normalization) also if the Lindeberg condition does not hold. Before the construction we prove a simple result which is useful also in other investigations. This result states that a sequence of random variables convergent in distribution has the same limit as its small perturbations. Let us remark that a similar result also holds for random variables taking values in more general spaces.

42.) Let two sequences of random variables S_{n} and T_{n}, n = 1, 2, …, be given such that the sequence of random variables S_{n}, n = 1, 2, …, converges in distribution to a distribution function F and the sequence T_{n}, n = 1, 2, …, converges stochastically to zero, i.e. P (|T_{n}| > ε) → 0 for all numbers ε > 0 if n → ∞. Then the sequence of random variables S_{n} + T_{n}, n = 1, 2, …, converges to the distribution function F.

43.) Let us construct a sequence of independent random variables ξ_{n}, n = 1, 2, …, for which Eξ_{n} = 0, Eξ_{n}^{2} = 1, n = 1, 2, …, and the sequence of partial sums

S_{n} = ∑_{k=1}^{n} ξ_{k}, n = 1, 2, …, satisfy one of the following statement:

a.) The sequence of normalized partial sums \sqrt{\frac{2}{n}}S_{n}, n = 1, 2, …, converge in distribution to the central normal distribution. b.) The sequence of normalized partial sums \sqrt{\frac{2}{n}}S_{n}, n = 1, 2, …, converges in distri-

bution to a normal random variable with expected value m ≠ 0 and variance 1.

The result of problem 38 implies that an example satisfying problem 43 cannot satisfy the Lindeberg condition. Indeed, if the Lindeberg condition held, then the distribution of the normalized partial sums \frac{S_{n}}{\sqrt{n}} would converge to the standard normal distribution function. Let us also remark that in the counter examples of problem 43 the limit distribution had a strictly smaller variance than the variance of the normalized partial sums. The following problem shows that no counter-example exists where the limit distribution has too large variance.

44.) Let a sequence of distribution functions F_{n}, n = 1, 2, …, converge in distribution to a distribution function F_{0}(x). Then lim inf_{n→∞} ∫ u^{2}F_{n}( du) ≥ ∫ u^{2}F_{0}( du).

H.) The multi-dimensional central limit theorem.

The multi-dimensional limit theorems can be investigated by means of the characteristic functions similarly to their one-dimensional analog. Moreover, the characteristic function method enables us to reduce the investigation of the multi-dimensional limit theorems to the one-dimensional case. This is the content of the subsequent two problems.

45.) Let Z = (Z_{1}, …, Z_{m}), n = 1, 2, …, be an m-dimensional random vector. Let us consider for all m-dimensional vectors (a_{1}, …, a_{m}) ∈ R^{m} the random variable

Z = Z(a_{1}, …, a_{m}) = ∑_{j=1}^{m} a_{j}Z_{j}. The distribution functions of all one-dimensional

random variables Z = Z(a_{1}, …, a_{m}) also detbermine the distribution of the random vector Z = (Z_{1}, …, Z_{m}).

46.) Let Z_{n} = (Z_{1,n}, …, Z_{m,n}), n = 1, 2, …, be a sequence of m-dimensional random vectors. The random vectors Z_{n} converge in distribution to an m-dimensional distribution as n → ∞ if and only if for all vectors (a_{1}, …, a_{m}) ∈ R^{m} the one-

dimensional random variables Z_{n} = Z_{n}(a_{1}, …, a_{m}) = ∑_{j=1}^{m} a_{j}Z_{j,n}, n = 1, 2, …,

converge in distribution as n → ∞. If the random vectors Z_{n} converge in distribution to a probability measure μ in the m-dimensional space, then this limit measure μ can be characterized in the following way. If Z = (Z_{1}, …, Z_{m}) is a random vector with distribution μ, and (a_{1}, …, a_{m}) ∈ R^{m} is an arbitrary m-dimensional vector,

then the distribution function of the random variable Z = Z(a_{1}, …, a_{m}) = ∑_{j=1}^{m} a_{j}Z_{j}

equals the limit of the distribution functions of the random Z_{n} = Z_{n}(a_{1}, …, a_{m}) =

∑_{j=1}^{m} a_{j}Z_{j,n}. In particular, we claim that these relations uniquely determine the

measure μ.

Let us define the notion of multi-dimensional normal (Gaussian) distributions. It will be the natural analog of the one-dimensional normal distribution. The multidimensional normal distributions with expectation zero are exactly those distributions which appear as the limit in the multi-dimensional version of the central limit theorem. Before their formal definition let us recall some notions.

The expected value of an m-dimensional random vector Z = (Z_{1}, …, Z_{m}) is the mdimensional vector M = (M_{1}, …, M_{m}) whose coordinates are the numbers M_{j} = Eξ_{j}, 1 ≤ j ≤ m. The covariance matrix of the random vector Z = (Z_{1}, …, Z_{m}) is the m × m matrix (D_{j,k}), 1 ≤ j, k ≤ m whose elements are the numbers D_{j,k} = EZ_{j}Z_{k}−EZ_{j}EZ_{k} = E(Z_{j} − EZ_{j})(Z_{k} − EZ_{k}). In our notations a vector b = (b_{1}, …, b_{m}) will mean a row-vector, and we shall denote its transpose, which is a column vector, by b^{∗}. If x = (x_{1}, …, x_{m}) ∈ R^{m} and y = (y_{1}, …, y_{m}) ∈ R^{n} are two m-dimensional vectors, then

(x, y) denotes their scalar product, i.e. (x, y) = ∑_{j=1}^{m} x_{j}y_{j}.

The definition of the multi-dimensional normal distribution. Let ξ_{j}, 1 ≤ j ≤ m, be independent random variables with standard normal distribution. Then the random vector ξ = (ξ_{1}, …, ξ_{m}) will be called an m-dimensional standard normal random vector and its distribution the m-dimensional standard normal distribution. If B = (b_{j,k}), 1 ≤ j, k ≤ m, is an m × m matrix, M = (M_{1}, …, M_{m}) is an m-dimensional vector and ξ = (ξ_{1}, …, ξ_{m}) is an m-dimensional standard normal random vector, then ξB + M is called an m-dimensional vector with normal distribution. A probability measure μ on the measurable sets of the m-dimensional Euclidean space R^{m} is called an m-dimensional normal distribution if and only if it equals the distribution of an m-dimensional random vector with normal distribution defined in the above way with an appropriate m × m matrix B and a vector M ∈ R^{m}.

Let us first characterize the multi-dimensional normal distributions.

47.) The covariance matrix Σ of all m-dimensional random vectors is positive semidefinite, i.e. for all m-dimensional vectors x = (x_{1}, …, x_{m}) xΣx^{∗} = (xΣ, x) ≥ 0. In the converse direction, for all m-dimensional vectors M = (M_{1}, …, M_{m}) ∈ R^{m} and m × m symmetric, positive semi-definite matrices Σ there exists an m-dimensional random vector with normal distribution whose expected value is this vector M and whose covariance matrix is this matrix Σ. The characteristic function of an m-dimensional normal distribution with expected value M and covariance matrix Σ equals

φ(t_{1}, …, t_{m}) = Ee^{i(t,ξ)} = Ee^{i(t_{1}ξ_{1}+···+t_{m}ξ_{m})} = exp \{−\frac{(tΣ, t)}{2} + i(t, M)\} , (13)

where t = (t_{1}, …, t_{m}), and ξ = (ξ_{1}, …, ξ_{m}) denotes an m-dimensional random vector with expected value M and covariance matrix Σ. In particular, an mdimensional normal distribution is uniquely determined by its expected value M and covariance matrix Σ.

The special form of the characteristic function of an m-dimensional characteristic function given in formula (13) has several simple but important consequences. It implies the most important properties of Gaussian distributions. Although these results do not belong to the subject of this series of problems, I discuss two problems that may be useful in such investigations.

48.) Let ξ_{j}, 1 ≤ j ≤ m, be m independent random variables with standard normal distribution, M = (M_{1}, …, M_{l}) an l-dimensional (deterministic) vector, and B a rectangular matrix of size l × m. Then (η_{1}, …, η_{l}) = (ξ_{1}, …, ξ_{m})B + M is an l-dimensional random vector with normal distribution. In particular, if we preserve only l coordinate of a a normal random vector η of dimension m, then we get a normal random vector of dimension l.

49.) Let η = (η_{1}, …, η_{m}) be such a random vector of dimension m with normal distribution whose covariance matrix Σ = (σ_{p,q}), 1 ≤ p, q ≤ m, has the following property:

The set {1, …, m} has a partition {1, …, m} = ⋃_{j=1}^{k} L_{j}, 1 ≤ j ≤ k, such that the

non zero elements of the matrix Σ are concentrated in the union of the squares L_{1} × L_{1}, …, L_{k} × l_{k}, i.e., σ_{p,q} = 0, if p ∈ L_{j}, q ∈ L_{j′}, and j ≠ j′. In this case the random vectors η̄_{j} = (η_{p}, p ∈ L_{j}), 1 ≤ j ≤ k, obtained by an appropriate grouping of the coordinates of the vector η are Gaussian random vectors, independent of each other.

Finally we formulate the multi-dimensional central limit theorem for appropriately normalized partial sums of independent vectors. We do not formulate this result in its most general form, and do not discuss its version for triangular arrays, although this would be also possible.

50.) Let ξ_{k} = (ξ_{1,k}, …, ξ_{m,k}), k = 1, 2, …, be a sequence of independent m-dimensional random vectors with expectation zero, i.e. we assume that the relation M_{k} = (Eξ_{1,k}, …, Eξ_{m,k}) = (0, …, 0) holds for all numbers k = 1, 2, . . .. Let us also assume that the random vectors ξ_{k} = (ξ_{1,k}, …, ξ_{m,k}) have a finite covariance matrix

Σ_{k} for all indices k = 1, 2, …, and the relation lim_{n→∞} \frac{1}{A_{n}^{2}} ∑_{k=1}^{n} Σ_{k} = Σ holds with an

m × m matrix Σ and with norming constants A_{n}, n = 1, 2, …, such that A_{n} → ∞ if n → ∞. If beside this all coordinates of the random vectors ξ_{k}, k = 1, …, satisfy the Lindeberg condition, i.e.

lim_{n→∞} \frac{1}{A_{n}^{2}} ∑_{k=1}^{n} Eξ_{p,k}^{2}I(|ξ_{p,k}| > εA_{n}) = 0, for all numbers p = 1, … m, (14)

for all ε > 0, then the normalized partial sums \frac{1}{A_{n}} S_{n} = \frac{1}{A_{n}} (S_{1,n}, …, S_{m,n}) =

\frac{1}{A_{n}} ∑_{k=1}^{n}(ξ_{1,k}, …, ξ_{m,k}) converge in distribution to the normal distribution function

with expected value M = (0, …, 0) and covariance matrix Σ. In particular, if ξ_{k} = (ξ_{1,k}, …, ξ_{m,k}), k = 1, 2, …, is a sequence of independent and identically distributed m-dimensional random vectors with expectation zero and finite covariance matrix Σ, then the normalized partial sums \frac{1}{\sqrt{n}}(S_{1}, …, S_{m}) = \frac{1}{\sqrt{n}} ∑_{k=1}^{n}(ξ_{1,k}, …, ξ_{m,k}) converge in distribution to the normal distribution function

with expectation zero and covariance Σ. 51.) Let ξ_{n} = (ξ_{1,n}, …, ξ_{m,n}), n = 1, 2, …, be a sequence of m-dimensional random

vectors with expectation zero and such that all random vectors ξ_{n} = (ξ_{1,n}, …, ξ_{m,n}) have a finite covariance matrix Σ_{n} for all indices n = 1, 2, . . .. Let us further

assume that the relation lim_{n→∞} \frac{1}{A_{n}^{2}} ∑_{k=1}^{n} Σ_{n} = Σ with an appropriate matrix Σ and

norming factors A_{n}, n = 1, 2, …, such that A_{n} → ∞, ha n → ∞. Besides, we also

assume that the uniform smallness condition lim_{n→∞} ^{\frac{max_{1≤j≤m} max_{1≤k≤n} Eξ_{j,k}^{2}}{A_{n}^{2}}} = 0 holds. If

the sequence of random vectors \frac{S_{n}}{A_{n}} converges in distribution to an m-dimensional normal distribution whose covariance matrix is Σ, then also the Lindeberg condition formulated in relation (14) holds.

Some additional remarks

We formulate some problems whose investigation is a natural continuation of the study of the problems in this paper. 1.) The central limit theorem for appropriately normalized partial sums of indepen-

dent random variables was deduced from the convergence of their characteristic functions. Actually not only the convergence of the characteristic functions can be proved, but also the speed of convergence can be given. Besides, the characteristic functions of the normalized partial sums can be better approximated if we add some correction terms to the characteristic function of the approximating normal distribution. These correction terms can be found by means of a natural Taylor expansion. It is natural to expect that these estimates also supply a good estimate about the accuracy of the normal approximation of the distribution function of normalized partial sums of independent random variables. Moreover, a better approximation of their characteristic functions also yields a better approximation of the distribution function of normalized partial sums of independent random variables if some appropriate correction terms are added to the normal approximation. A natural analog of the above question is to give a good estimate on the Gaussian approximation of the density function of appropriately normalized partial sums of independent random variables and to find a better approximation for this density function by means of an appropriate asymptotic expansion. Naturally, in this investigation we have to impose some additional assumptions on the distribution of the random variables whose partial sums we investigate to guarantee the existence of a nice density function. This problem about the behaviour of density functions is simpler, because there exists a relatively simple inverse Fourier transformation formula to express a density function by means of its Fourier transform. On the other hand, there is no really useful formula for the calculation of distribution functions with the help of their characteristic function in a simple, well applicable form. But by convolving the distribution function of the normalized partial sums we want to investigate with an appropriate density function (essentially concentrated in a small neighbourhood of the origin) we get a new distribution function close to the original one which has a density function, and as a consequence this modified (smoothed) distribution function can be well estimated as the integral of its density function. This approach together with some additional ideas enables us to study the problems mentioned in the previous paragraph. These problems will be the main part of the (essentially shorter) second part of this series of problems.

2.) We have seen that the appropriately normalized partial sums of independent random variables converge in distribution to a normal distribution function under some mild conditions. On the other hand, we would like to get a complete picture about all possible limit theorems for the distribution of appropriately normalized partial sums or in a more general way, for the sums of the elements in the same rows of a triangular array of random variables. It is natural to impose some kind of uniform smallness condition which should guarantee that there are no such dominating terms in the random sums we investigate whose magnitude is comparable with the magnitude of the whole sum.

A fairly complete picture can be given about all possible limit theorems for the normalized partial sums of independent random variables. In the proof the Fundamental Theorem about the convergence of distribution functions proved also in this series of problems plays a key role. It makes possible to reformulate the problem to the language of characteristic functions. The first question to be understood in this problem is the description of all possible limit theorems. This leads to the study of the fixed points of certain operators in the space of distribution functions defined by means of convolution and rescaling. This study leads to the description of the so-called infinitely divisible distributions which appear as the limits in limit theorems. Further investigations make possible also to describe in which limit theorems a certain infinitely divisible distribution appears.

These problems are solved by means of certain methods of the analysis about the study of the behaviour of the characteristic functions, but the proof also contains several probabilistic ideas. We also remark that although there are several different kind of limit theorems, the central limit theorem is the only “universal law” for the limit distribution of normalized partial sums of independent random variables, where the limit distribution function “forgets” the distribution of the single terms in the partial sums. Here we do not explain the more precise meaning of this rather loose statement. A fairly complete discussion with complete proofs of the problems mentioned in this Section 2 can be found together with the explanation of the ideas behind the proofs in the text Limit theorems and infinitely divisible distribution on my homepage. (For the time being it exists only in Hungarian.) It also contains the explanation of the statement that the central limit theorem is the only “universal law” among the limit theorems for the limit distribution of partial sums of independent variables.

3.) Let us consider a sequence ξ_{1}, ξ_{2}, …, of independent, identically distributed random variables with expectation zero and finite variance together with the partial sums S_{n} = ∑_{k=1}^{n} ξ_{n}, n = 1, 2, …, defined with their help. The central limit theo-

rem gives a good estimate on the probability P (\frac{S_{n}}{\sqrt{n}} > x) for large indices n and fixed numbers x. We may ask whether a similar good estimate can be given for the probabilities P (\frac{S_{n}}{\sqrt{n}} > x_{n}), that is we are interested in a good estimate on the probability of a similar event when the number x is replaced by a number x_{n} which may depend on n. The special case x_{n} = x\sqrt{n}, that is the investigation of the probability of the event that the average of independent, identically distributed random variables is larger than a fixed number x is a particularly important question. This problem belongs to an important part of the probability theory, called the theory of large deviations. The above mentioned problem is discussed together with some additional questions also in a series of problems The theory of large deviation; the case of partial sums of real valued random variables on my homepage. (For the time being it exists only in Hungarian). Let us remark that the estimates in the large deviations problems do not agree with the estimates suggested by a formal extension of the central limit theorem. It would be natural to try to estimate the probability P (S_{n} > nx) similarly to the estimate of the event P (S_{n} > \sqrt{n}x) in the proof of the central limit theorem. Nevertheless, this method in itself does not suffice to estimate this probability. It is worth understanding why the method of the proof of the central limit theorem is not sufficient to estimate this probability. Let us first consider the case when the distribution function P (S_{n} > nx) also has a nice density function f_{n}(x) = \frac{d}{dx}P (S_{n} > nx), and let us first estimate this density function. Then we have to investigate the integral f_{n}(x) = \frac{n}{2π} ∫ e^{−intx}φ^{n}(t) dt, where φ(t) is the characteristic function of the random variable ξ_{1}. This integral is similar to the integral appearing in the proof of the local central limit theorems. The only difference is that the factor e^{it\sqrt{n}x} in the integral investigated in the proof of the local central limit theorem is replaced by the term e^{it\sqrt{n}x} in this case. In the proof of the local central limit theorem a singular integral had to be investigated which was strongly concentrated in a small neighbourhood of the origin. In the analogous large deviation problem a similar singular integral has to be investigated. But there is a small difference which makes the latter problem more difficult. The cause of this difficulty is that although the absolute value of the complex number valued integrand whose integral has to be estimated in the large deviation problem has a strongly localized maximum in the origin, but its imaginary part has a strong fluctuation in a small neighbourhood of the origin. This fluctuation causes a strong cancellation, hence we cannot claim that the contribution of a small neighbourhood of the origin yields the main contribution to the integral we are estimating. In the proof of the local central limit theorem this difficulty did not appear, since in this case the fluctuation of the imaginary part of the integrand is not so strong in a small neighbourhood of the origin. The reason for this difference is that the term e^{−i\sqrt{n}tx} appears instead of the term e^{−intx} in the integral investigated in the proof of the local central limit theorem and \frac{dφ(t)}{dt} |_{t=0} = Eξ_{1} = 0 in this case. Such kind of problems appear often in the analysis, and an important technique, the saddle point method was worked out to investigate such problems. To apply the saddle point method let us rewrite the integral we are interested in in the form f_{n}(x) = \frac{n}{2π} ∫ e^{n(−itx+log φ(t))} dt. (In this heuristic explanation let us disregard such technical difficulties as the problem that we cannot always take logarithm of a function.) If the function n(−itx + log φ(t)) in the exponent of the integrand we study is an analytic function of the variable t, then the saddle-point method suggests to replace the domain of integration to an appropriate new curve which goes through the saddle point, i.e. through a point z which satisfies the identity

\frac{d(−izx + log φ(z))}{dz} = 0.  (∗)

The integral we are investigating does not change by this replacement. On the other hand this new integral can be better estimated since the absolute value of the integrand on this new curve has a strongly localized (local) maximum in the saddle point, and its fluctuation around the saddle point is small. The proof of the large deviation estimates for averages of independent and identically distributed random variables are based on this idea. Naturally, it is useful to combine it with some additional observations. For instance, it is enough to look for the saddle point on the imaginary axis where the imaginary part of the left-hand side of the saddle point equation (∗) is automatically zero. If the saddle point equation has no solution, then an appropriate approximation and some additional ideas are needed to solve the large deviation problems. Let us remark that in most probability text-books on the theory of large deviations the saddle point method is not discussed. In these text-books the notion of the so-called conjugated distributions are introduced, and the problem is solved with their help. On the other hand, a better understanding of the problem may help us to understand that the introduction of conjugated distributions can be interpreted as the application of the saddle point method to the investigation of large deviation problems formulated in the language of probability theory. Finally we make the following remark. The above sketched saddle point method can be applied for the investigation of large deviation problems only if the characteristic function of the random variables whose averages we investigate has an analytic continuation. One may ask whether this condition does not mean an unnecessary restriction. A detailed investigation shows that the answer to this question is negative. It turns out that this condition about the analiticity of the characteristic function has a probabilistic content, and the behaviour of the probabilities we investigate in large deviation problems heavily depends on whether the characteristic function is analytic. If this analiticity property does not hold, then the probabilities investigated in the large deviation problems have an essentially different behaviour.

Solutions.

1.) Put I = \frac{1}{\sqrt{2aπ}} ∫_{−∞}^{∞} e^{−u^{2}/2a} du. By expressing the number I^{2} as a double integral and rewriting it in polar coordinate system we get that

I^{2} = \frac{1}{2aπ} ∫_{−∞}^{∞} ∫_{−∞}^{∞} e^{−(u^{2}+v^{2})/2a} du

= ∫_{−π}^{π} ∫_{0}^{∞} \frac{1}{2π} · \frac{r}{a}e^{−r^{2}/2a} dr dφ = ∫_{−π}^{π} \frac{1}{2π} dφ = 1.

This relation implies the first statement of the problem. Let us fix a number a > 0. Put F (z) = \frac{1}{\sqrt{2aπ}} ∫_{−∞}^{∞} e^{−(u−z)^{2}/2a} du. With the substitution ū = u − z we get that

F (z) = \frac{1}{\sqrt{2aπ}} ∫_{−∞}^{∞} e^{−ū^{2}/2a} dū = 1

for all real numbers z. We can generalize this identity for all complex numbers z with the help of one of the following argument of the theory of analytic functions. First argument: Both functions F (z) and G(z) ≡ 1 are analytic function on the plane of complex numbers. As they agree for all real numbers they also agree for all complex numbers. To see that the function F (z) is really analytic, observe that it can be obtained as the limit of analytic functions in such a way that the convergence is uniform on all compact subsets of the plane of complex numbers. Then the limit of these analytic functions, i.e. the function F (z) is also analytic. Such analytic functions converging to F (z) can be obtained by approximating the integrals F (z) by the natural approximating sums of these integrals. Second argument: We get with the change of variable ū = u − z that

F (z) = \frac{1}{\sqrt{2aπ}} ∫_{−∞−Im z}^{∞−Im z} e^{−ū^{2}/2a} dū = 1.

To prove the above identity observe that lim_{|u|→∞} e^{−(u+iv)^{2}/2a} = 0, and the conver-

gence is uniform with respect to the variable v if |v| is in a bounded interval. This fact together with the result of analytic functions by which the contour integral of an analytic function (containing no singular point) on a closed curve is zero imply that the value of the above integral does not change if we replace the route of integration [−∞ − Im z, ∞ − Im z] to the line [−∞, ∞]. This implies the identity we wanted to prove.

2.) Let ξ be a random variable with Poissonian distribution with parameter λ = n, i.e. P (ξ = k) = \frac{n^{k}}{k!} e^{−n}, k = 0, 1, 2, . . .. The Fourier series corresponding to the distribution of this random variable ξ equals

P_{n}(t) = ∑_{k=0}^{∞} P (ξ = k)e^{itk} = ∑_{k=0}^{∞} \frac{n^{k}}{k!} e^{−n+ikt} = e^{−n+ne^{it}}.

This identity together with relation (2) imply relation (3) with the choice k = n. Let us observe that relation (4b) is a simple consequence of relations (3) and (4a) and the asymptotic formula \frac{1}{1+x} = 1 − x + O(x^{2}) = 1 + O(x), if |x| ≤ \frac{1}{2}. To prove formula (4a) let us first give an upper bound about the contribution of the domain {t: |t| ≥ n^{−1/3}} to the integral at the left-hand side of formula (4a). Then let us consider the restriction of this integral to the domain {t: |t| < n^{−1/3}} and let us give a good asymptotical estimate on it. To carry out the first estimate let us observe that

|e^{n(e^{it}−1−it)}| = e^{nRe (e^{it}−1−it)} = e^{n(cos t−1)} < e^{−nt^{2}/4} < e^{−n^{1/3}/4} if n^{−1/3} ≤ t ≤ π,

and this implies that

|∫_{{n^{−1/3}≤|t|≤π}} e^{n(e^{it}−1−it)} dt| = O (e^{−const. n^{1/3}}) .  (2.1)

To carry out the second estimation let us give a good asymptotical formula on the integrand of relation (4a) in a small neighbourhood of the origin by means of a Taylor expansion. We get that n(e^{it} − 1 − it) = n (−\frac{t^{2}}{2} − i\frac{t^{3}}{6} + O(t^{4})) and

e^{n(e^{it}−1−it)} = e^{−nt^{2}/2}e^{−int^{3}/6+O(nt^{4})}

= e^{−nt^{2}/2} (1 − \frac{i(\sqrt{n}t)^{3}}{6\sqrt{n}} + O (\frac{(\sqrt{n}t)^{4}}{n} ) + O (\frac{\sqrt{n}t)^{6}}{n} )) ,

if |t| ≤ n^{−1/3}. By exploiting this estimate and making the substitution of variables t̄= \sqrt{n}t we get that

∫_{−n^{−1/3}}^{n^{−1/3}} e^{n(e^{it}−1−it)} dt = \frac{1}{\sqrt{n}} ∫_{−n^{1/6}}^{n^{1/6}} e^{−t̄^{2}/2} (1 − i\frac{t̄^{3}}{6\sqrt{n}} + \frac{O (t̄^{4} + t̄^{6})}{n}) dt̄

= \frac{1}{\sqrt{n}} (∫_{−∞}^{∞} − ∫_{|t̄|>n^{1/6}}) .

On the other hand,

∫_{|t|≥n^{1/6}} e^{−t^{2}/2} (1 − i\frac{t^{3}}{6\sqrt{n}} + \frac{O(t^{4} + t^{6})}{n}) dt = O (e^{−n^{1/3}/4}) ,

and

∫_{−∞}^{∞} e^{−t^{2}/2} (1 − i\frac{t^{3}}{6\sqrt{n}} + \frac{O(t^{4} + t^{6})}{n}) dt

= ∫_{−∞}^{∞} e^{−t^{2}/2} dt − ∫_{−∞}^{∞} \frac{i}{6\sqrt{n}}t^{3}e^{−t^{2}/2} dt + O (\frac{1}{n}) = \sqrt{2π} + O (\frac{1}{n})

because of the result of the first problem, and since the function t^{3}e^{−t^{2}} is odd. These relations imply that

∫_{−n^{−1/3}}^{n^{1/3}} e^{n(e^{it}−1−it)} dt = \begin{matrix}√ & \overline{2π} \\ √ & \overline{n} \end{matrix} (1 + O (\frac{1}{n})) .  (2.2)

Then relation (4a) is a consequence of formulas (2.1) and (2.2). Relation (4d) follows similarly from formulas (4c) as formula (4a) implies relation (4b). The only difference is that now we apply a better approximation \frac{1}{1+x} = 1 − x + x^{2} − ⋯ + (−1)^{k}x^{k} + O (|x|^{k+1}) for |x| ≤ \frac{1}{2}. Also formula (4c) can be proved similarly to formula (4a). The difference is that now we consider the Taylor expansion of the functions n(e^{it} − 1 − it) and P_{n}(t) defined by the identity e^{n(e^{it}−1−it)} = e^{−nt^{2}/2}(1 + P_{n}(\sqrt{n}t)) up to the k-th and not only to the first term in the interval |t| ≤ n^{−1/3}. Since P_{n}(t) = exp \{\frac{t^{3}}{\sqrt{n}}R (\frac{t}{\sqrt{n}})\} with an analytic function R(t) bounded on the real line, these calculations enable us to give an estimate for the integrand of the integral in (4.1) with an accuracy of

e^{−nt^{2}/2}O (^{\frac{∑_{j=1}^{k}(\sqrt{n}|t|)^{j(l(j)}}{n^{(k+1)/2}}} ) = O (\frac{e^{−nt^{2}/4}}{n^{(k+1)/2}} ), where l(j) = min\{l: lj ≥ k + 1\} with

a function of the form e^{−nt^{2}/2} (1 + Q_{n}(t)). The function Q_{n}(t) can be written

as Q_{n}(t) = ∑_{j=1}^{k} \frac{Q̄_{j}(\sqrt{n}t)}{n^{j/2}}, with some polynomials Q̄_{j} which do not depend on the

parameter n, and they can be calculated explicitly. In particular, Q̄_{1}(t) = \frac{−i}{6}t^{3}. Then by carrying out the substitution t̄ = \sqrt{n}t we get the improvement (4c) of formula (4a).

3.) Let us first show that the distribution of a random variable ξ has the periodicity h if and only if |Ee^{2πiξ/h}| = 1. Indeed, if |Ee^{2πteiξ/h}| = 1, then Ee^{2πiξ/h} = e^{2πi/b} with some real number b. But this is possible if and only if the distribution of the random variable 2π \frac{ξ−b}{h} is concentrated in the points n = 0, ±1, ±2, …, i.e. the random variable ξ is concentrated on the lattice nh + b, n = 0, ±1, ±2, . . .. To see the statement in the converse direction observe that if the distribution of a random variable ξ is concentrated on a lattice nh + b, n = 0, ±1, …, of width h, then |Ee^{2πiξ}| = 1. Furthermore, if the random variable ξ concentrated on some lattice is not concentrated in a single point, then there are two numbers a and b, a ≠ b, such that P (ξ = a) > 0 and P (ξ = b) > 0. Then we have |Ee^{itx}| < 1 for all sufficiently small t > 0, and we can even state that there exists a smallest number t > 0 such that |Ee^{itξ}| = 1. Take the smallest number t > 0 such that |Ee^{itξ}| = 1. Then h = \frac{2π}{t} is the greatest number h such that the distribution of the random variable ξ is concentrated on a lattice nh + b, n = 0, ±1, ±2, …, of width h. Then the periodicity of the Fourier

series P (t) = Ee^{it(ξ−b)} = ∑_{n=−∞}^{∞} P (ξ − b = nh)e^{itnh} of the random variable ξ − b

is \frac{2π}{h}. Further, from the definition of the number h follows that |P (t)| < 1, if 0 < t < \frac{2π}{h}. Since P (−t) = P (\frac{2π}{h} − t) the above relation can be rewritten as |P (t)| < 1, if 0 < |t| < \frac{π}{h}. Clearly, P (0) = 1.

Formal differentiation by terms of the infinite sum P (t) yields that \frac{P^{(k)}(t)}{dt^{k}} =

∑_{n=−∞}^{∞} i^{k}(nh)^{k}e^{itnh}P (ξ − b = nh), and \frac{P^{(k)}(t)}{dt^{k}} |_{t=0} = ∑_{n=−∞}^{∞} i^{k}(nh)^{k}P (ξ − b = nh) =

i^{k}E(ξ − b)^{k}. Under the condition E|ξ − b|^{k} < ∞ this formal calculation is allowed. Indeed, the approximating partial sums of the k-th derivative of the function P (t)

satisfies the inequality ∑_{n=−N}^{N} |i^{k}(nh)^{k}e^{itnh}P (ξ − b = nh)| ≤ E|ξ − b|^{k}, and this

property allows (k-fold) differentiation by terms.

4.) By relation (2) and the formula written after it

P (S_{n} = k) = ∫_{−π}^{π} \frac{1}{2π} e^{−ikt}P^{n}(t) dt = ∫_{|t|< \frac{1}{ε\sqrt{n}}} + ∫_{\frac{1}{ε\sqrt{n}} <|t|<ε} + ∫_{ε<|t|<π} = I_{1}+I_{1}+I_{3},

where P (t) = ∑_{k=−∞}^{∞} e^{ikt}P (ξ_{1} = k), and ε > 0 is an arbitrary (small) positive real

number. We solve the problem if we give a good estimate for the integrals I_{1}, I_{2} and I_{3} for small ε > 0.

It is easy to bound the integral I_{3}. It follows from the result of problem 3 and the continuity of the function P (t) that sup_{ε≤|t|<π} |P (t)| < 1 with some number 0 < q =

q(ε) < 1. Hence

|I_{3}| = |∫_{ε<|t|<π} \frac{1}{2π} e^{−ikt}P^{n}(t) dt| < q^{n}

with some number 0 < q < 1. To estimate the integrals I_{1} and I_{2} we need a good bound on the function P^{n}(t) if |t| < ε. It is more convenient to work with the function log P (t) instead. (For small numbers ε > 0 this is allowed, since in this case the value of the function P (t) in the interval [−ε, ε] is in a small neighbourhood of the number 1.) Simple calculation yields that

\frac{d log P (t)}{dt} = \frac{P′(t)}{P (t)},  \frac{d log P (t)}{dt}|_{t=0} = im,
\frac{d^{2} log P (t)}{dt^{2}} = \frac{P′′(t)P (t) − P′(t)^{2}}{P^{2}(t)} ,  \frac{d^{2} log P (t)}{dt^{2}} |_{t=0} = −m_{2} + m^{2} = −σ^{2},

hence a Taylor expansion around the origin yields that

log P (t) = imt − \frac{σ^{2}}{2} t^{2} + o(t^{2}), if |t| < ε.

Péter Major

This estimate together with the relation |P^{n}(t)| = e^{nRe log P (t)} = e^{−nσ^{2}t^{2}/2+no(t^{2})} ≤ e^{−nσ^{2}t^{2}/3}, if |t| < ε and n ≥ n(ε) imply that

|I_{2}| ≤ \frac{1}{2π} ∫_{\frac{1}{ε\sqrt{n}} <|t|<ε} |P^{n}(t)| dt ≤ \frac{1}{2π} ∫_{\frac{1}{ε\sqrt{n}} <|t|<ε} e^{−nσ^{2}t^{2}/3} dt

≤ \frac{1}{π\sqrt{n}} ∫_{− \frac{1}{ε}}^{∞} e^{−σ^{2}t^{2}/3} dt ≤ \frac{e^{−σ^{2}/4ε^{2}}}{\sqrt{n}} .

Furthermore,

I_{1} = ∫− \frac{1}{ε\sqrt{n}}^{\frac{1}{ε\sqrt{n}}} \frac{1}{2π} e^{−ikt+inmt−nσ^{2}t^{2}/2+o(nt^{2})} dt

= ∫− \frac{1}{ε}^{\frac{1}{ε}} \frac{1}{2π\sqrt{n}}e^{i(mn−k)t/\sqrt{n}−σ^{2}t^{2}/2+o(t^{2})} dt

= ∫− \frac{1}{ε}^{\frac{1}{ε}} \frac{1}{2π\sqrt{n}}e^{i(mn−k)t/\sqrt{n}−σ^{2}t^{2}/2}(1 + o(t^{2})) dt

= ∫_{−∞}^{∞} \frac{1}{2π\sqrt{n}}e^{i(nm−k)t/\sqrt{n}−σ^{2}t^{2}/2} dt − ∫_{|t|> \frac{1}{ε}} \frac{1}{2π\sqrt{n}}e^{i(nm−k)t/\sqrt{n}−σ^{2}t^{2}/2} dt

+ o (\frac{1}{\sqrt{n}}) .

On the other hand

|∫_{|t|> \frac{1}{ε}} \frac{1}{2π\sqrt{n}}e^{i(nm−k)t/\sqrt{n}−σ^{2}t^{2}/2} dt| ≤ \frac{e^{−σ^{2}/4ε^{2}}}{\sqrt{n}} ,

and by completing the quadratic term in the exponent of the last formula we get that

∫_{−∞}^{∞} \frac{1}{2π\sqrt{n}}e^{i(nm−k)t/\sqrt{n}−σ^{2}t^{2}/2} dt

= \frac{e^{−(nm−k)^{2}/2nσ^{2}}}{2π\sqrt{n}} ∫_{−∞}^{∞} exp \{−\frac{σ^{2}}{2} (t − i\frac{nm − k}{\sqrt{n}} )^{2}\} dt = \frac{e^{−(nm−k)^{2}/2nσ^{2}}}{\sqrt{2πn}σ}

by the result of problem 1. These estimates imply that

|I_{1} − \frac{1}{\sqrt{2πn}σ} exp \{−\frac{(k − nm)^{2}}{2nσ^{2}} \}| ≤ const. \frac{e^{−σ^{2}/4ε^{2}}}{\sqrt{n}}

if n > n(ε). As the estimates given for the expressions I_{1}, I_{2} and I_{3} hold for all ε > 0 if n is sufficiently large, hence they imply the result of problem 4.

5.) The solution of this problem is similar to that of problem 4. Since the random variable ξ_{1} has three finite moments in the present case, hence we can make the following better approximation of the function log P (t): log P (t) = imt− \frac{σ^{2}}{2} t^{2}+O(t^{3}). Hence P^{n}(t) = e^{imnt−nσ^{2}t^{2}/2+O(nt^{3})}. Then we can solve problem 5 by making estimations similar to those in the proof of problem 4. The only essential difference is that now we define the domain of integration in the definition of the expressions in I_{1} and I_{2} in a different way. Now put I_{1} = ∫_{|t|<n^{−1/3}} and I_{2} = ∫_{n^{−1/3}≤|t|<ε}. The reason the domain of integration in the definition of expression I_{1} was chosen in the above way is that in the domain is that e^{O(nt^{3})} = 1 + O(nt^{3}) in the domain {|t| < n^{−1/3}}, hence the approximation e^{−ikt}P^{n}(t) = e^{i(mn−k)t−nσ^{2}t^{2}/2} (1 + O(nt^{3})) holds in this domain. Then by applying a natural adaptation of the calculation in problem 4 that by omitting the error term O(nt^{3}) from the integral approximating the expression I_{1} we get an error of order O (\frac{1}{n}). The error of all other remaining estimations is essentially smaller. They are of order e^{O(−const. n^{1/3})}. Hence these estimates yield the solution of problem 5.

5a.) We have to estimate the expression P (S_{n} = k) = \binom{n}{k}p^{k}(1 − p)^{n−k}. Let us first consider the case αn < k < βn with some numbers 0 < α < β < 1. By the Stirling formula

\begin{aligned}\binom{n}{k} & = \frac{n!}{k!(n − k)!} = \begin{matrix}(\frac{n}{e}) & n \\ (\frac{n−k}{e} ) & n−k & (\frac{k}{e} ) & k \end{matrix} \sqrt{\frac{n}{2πk(n − k)}} (1 + O (\frac{1}{n})) \\ & = (1 − \frac{k}{n})^{−(n−k)} (\frac{k}{n})^{−k} \frac{1}{\sqrt{2πn}\sqrt{\frac{k}{n} (1 − \frac{k}{n})}} (1 + O (\frac{1}{n})) . \end{aligned}

A Taylor expansion of the function log x around the point p yields that

p^{k} (\frac{k}{n})^{−k} = exp \{k (log p − log \frac{k}{n})\}

= exp \{−\frac{k}{p} (\frac{k}{n} − p) + \frac{k}{2p^{2}} (\frac{k}{n} − p)^{2} + O (n (\frac{k}{n} − p)^{3})\} .

Similarly,

(1 − p)^{n−k} (\frac{n − k}{n} )^{−(n−k)}

= exp \{\frac{n − k}{1 − p} (\frac{k}{n} − p) + \frac{n − k}{2(1 − p)^{2}} (\frac{k}{n} − p)^{2} + O (n (\frac{k}{n} − p)^{3})\} .

By multiplying the last two expressions and exploiting that in the formula got in such a way the coefficient of the term (\frac{k}{n} − p)^{2} is

\frac{k}{2p^{2}} + \frac{n − k}{2(1 − p)^{2}} = \frac{(k − np)^{2}}{2p(1 − p)} + (1 − 2p)\frac{(k − np)^{3}}{n^{2}}

we get that

p^{k} (\frac{k}{n})^{−k} (1−p)^{n−k} (\frac{n − k}{n} )^{−(n−k)} = exp \{−\frac{(kn − p)^{2}}{2np(1 − p)} + O (n (\frac{k}{n} − p)^{3})\} .

Since \frac{k}{n} (1 − \frac{k}{n}) = p(1 − p) (1 + O (\frac{k−np}{n})) the above calculations yield that

P (S_{n} = k) = \begin{matrix}exp & \{−\frac{(kn−p)^{2}}{2np(1−p)} + O (n (\frac{k}{n} − p)^{3}) + O (\frac{k}{n} − p) + O (\frac{1}{n})\} \\ √ & \overline{2πp(1 − p)n} \end{matrix}

Since Eξ_{1} = p, Var ξ_{1} = p(1 − p) the last formula yields an estimate stronger than we want in the case |k − np| < γn with a sufficiently small number γ > 0. Indeed, by introducing the quantity z = \frac{k−np}{\sqrt{n}} we get the demanded estimate with an error term ε(n) of the following form:

ε(n) = ε(n, z) ≤ \frac{C}{\sqrt{n}}e^{−K_{1}z^{2}} [exp \{K_{2} \frac{|z|^{3}}{\sqrt{n}} + K_{3} \frac{|z|}{\sqrt{n}} + \frac{K_{4}}{n} \} − 1]

with appropriate constants C > 0, and K_{j} > 0, j = 1, …, 4. Then we have to show that ε(z, n) ≤ \frac{const.}{n} , if |z| ≤ γ\sqrt{n}. This estimate holds for n^{1/6} > |z| < γ\sqrt{n}, since in this case ε(n, z) ≤ e^{−K_{1}z^{2}/2}. If |z| ≤ n^{1/6} then e(n, z) ≤ \frac{C}{\sqrt{n}}e^{−K_{1}z^{2}} (\frac{|z|^{3}+|z|+1}{\sqrt{n}}) ≤ \frac{const.}{n} , that is the demanded estimate holds also in this case. If |k − np| ≥ γn, then the result of problem 5a follows from the relations P (S_{n} = k) ≤ \frac{const.}{n} and e^{−(k−np)^{2}/2np(1−p)} < \frac{const.}{n^{2}} . Actually, even stronger estimates could be proved. The first estimate is a consequence of the Chebishev inequality, since P (S_{n} = k) ≤ P (|S_{n} − ES_{n}| ≥ γn) ≤ \frac{Var S_{n}}{γ^{2}n^{2}} ≤ \frac{const.}{n} . The second inequality is obvious.

6.) Let us introduce the random variables ξ̄_{j} = \frac{ξ_{j}−b}{h}, j = 1, …, n, and S̄_{n} = ∑_{j=1}^{n} ξ̄_{j}.

Then Eξ̄_{j} = \frac{m−b}{h} and Var ξ̄_{j} = \frac{σ^{2}}{h^{2}} . Since P (S_{n} = kh + nb) = P (S̄_{n} = k), and the random variable S̄_{n} is concentrated on the lattice of the integers as on the rarest lattice, statement formulated in this problem follows from the results of problems 4 and 5.

7.) It follows from formula (5) that the relation

lim_{n→∞} P (A < \frac{S_{n} − nm}{\sqrt{n}σ} < B) = ∫_{A}^{B} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du,  (2.3)

holds if −∞ < A < B < ∞, and the limit in this relation is uniform for all pairs of numbers (A, B) such that C_{1} ≤ A < B ≤ C_{2} with some fixed numbers −∞ < C_{1} < C_{2} < ∞. Indeed,

P (A < \frac{S_{n} − nm}{\sqrt{n}σ} < B) = P (A\sqrt{n}σ + nm − nb < S_{n} − nb < B\sqrt{n}σ + nm − nb)

= ∑_{k: k∈K(A,B)} P (S_{n} = kh + nb)

= \frac{h}{\sqrt{2πn}σ} ∑_{k: k∈K(A,B)} exp \{−\frac{(kh + nb − nm)^{2}}{2nσ^{2}} \} + \sqrt{n}o (\frac{1}{\sqrt{n}})

= \frac{h}{\sqrt{n}σ} ∑_{l(k,n)∈L(A,B)} \frac{1}{\sqrt{2}π} e^{−l(k,n)^{2}/2} + o(1)

where K(A, B) = \{k: A\sqrt{n}σ < kh + nb − nm > B\sqrt{n}σ\}, l(k, n) = \frac{kh−nm+nb}{\sqrt{n}σ}, and

L(A, B) = \{l(k, n) = \frac{kh − nm + nb}{\sqrt{n}σ}, k = 0, ±1, ±2, … \} ∩ (A, B),

i.e. the points of the set L(A, B) are the points of the lattice of width \frac{h}{\sqrt{n}σ} and containing the point \frac{nb−nm}{\sqrt{n}σ} which fall into the interval (A, B). This implies formula (2.3), since for a fixed number n the probability at the left-hand side is an approximating sum of the integral at the right-hand side plus an error which tends to zero as n → ∞. We prove that relation (2.3) also holds for A = −∞. Indeed, for all ε > 0 we can choose a number K = K(ε) such that ∫_{−K}^{K} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du > 1 − ε. Then

lim_{n→∞} P (|\frac{S_{n}−nm}{\sqrt{n}σ} | < K) > 1 − ε, and lim_{n→∞} P (\frac{S_{n}−nm}{\sqrt{n}σ} < −K) < ε. Then

lim sup_{n→∞} |P (\frac{S_{n} − nm}{\sqrt{n}σ} < x) − ∫_{−∞}^{x} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du|

≤ lim sup_{n→∞} |P (−K ≤ \frac{S_{n} − nm}{\sqrt{n}σ} < x) − ∫_{−K}^{x} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du| + ε ≤ ε.

Since the last relation holds for all ε > 0, it implies the statement of problem 7. 8.) Let φ(t) = ∫_{−∞}^{∞} e^{itx}f(x) dx = Ee^{itξ_{1}} denote the Fourier transform of the density

function of the random variable ξ_{1}. Then the Fourier transform of the density function of the random sum S_{n} = ξ_{1} + ⋯ + ξ_{n} equals Ee^{it(ξ_{1}+···+ξ_{n})} = (Ee^{itξ_{1}})^{n} = φ^{n}(t). Since |φ(t)| ≤ 1, hence under the conditions of problem (8) the function φ^{n}(t) is integrable for k ≥ n, and the density function f_{n}(t) of the random sum can be expressed by formula (6) if we replace the function φ(t) by φ^{n}(t). Moreover, this relation also holds if we only assume that the function φ^{k}(t) is integrable, and n ≥ k. The above calculation makes possible to prove problem 8 similarly to problem 4 with some natural modification. Now we have to estimate the integral (6) instead of the integral (2) (with the modification that we write φ^{n}(t) instead of the function φ(t) in (6). Further, because of the condition a Eξ^{2} < ∞ the Fourier transform φ(t) is twice differentiable, φ′(0) = iEξ_{1}, φ(0)′′ = −Eξ_{1}^{2}. This means that the analogs of the relations applied in the solution of problem 4 holds in this case. (Later we shall discuss the properties of the Fourier transform φ(t) in the general case.) The only essential difference in the estimation of the integral we have to investigate is that the integral I_{1} = ∫_{ε<|t|<π} e^{−ikt}P^{n}(t) dt introduced in the solution of problem 4 now we write I′_{1} = I′_{1}(x) = ∫_{ε<|t|<∞} e^{−itx}φ^{n}(t) dt. Observe that

I′_{1} ≤ ∫_{ε<|t|<∞} |φ(t)|^{n} dt ≤ sup_{ε≤|t|<∞} |φ(t)|^{n−k} ∫_{ε<|t|<∞} |φ(t)|^{k} dt

≤ const. sup_{ε≤|t|<∞} φ(t)|^{n−k},

since φ^{k}(·) is an integrable function. For a fixed number t, t ≠ 0, |φ(t)| < 1. Further, by the Riemann lemma lim_{|t|→∞} |φ(t)| = 0, and φ(t) is a continuous function.

(This series of problem also contains the proof of the Riemann lemma.) These facts imply that sup_{ε<|t|<∞} |φ(t)| < q < 1. This relation together with the previous

estimates imply that |I′_{1}| ≤ const. q^{n}. The only further difference in the solution of problem 8 when compared to the solution of problem 4 is that now in the integrals defining the expressions I_{1} and I_{2} the functions e^{−ikt}P^{n}(t) are replaced by the function e^{−itx}φ^{n}(t). These expressions can be estimated in the same way as the analogous integrals in problem 4.

9.) The solution of this problem is similar to that of problem 6, only the notations are simpler. The condition directly implies that

lim_{n→∞}(F_{n}(B) − F_{n}(A)) = ∫_{A}^{B} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du,

and the convergence is uniform for pairs of numbers (A, B) in a bounded set. Then we can show similarly to the argument in problem 6 that the number A in the above formula can be replaced by −∞.

10.) The proof is a slight modification of the proof sketched in the solution of problem 8. This modification is similar to the modification made in the solution of problem 5 compared to problem 4. Since E|ξ_{1}|^{3} < ∞, hence the approximation

log φ(t) = itEξ_{1} − \frac{t^{2}}{x} Eξ_{1}^{2} + O(t^{3})

holds in a small neighbourhood of the origin. This makes possible to get the solution of the problem by modifying the domain of integration in the expressions I_{2} and I_{3} as it was done in the solution of problem 5.

11.) Let ξ be a random variable with standard normal distribution. Then

Eξ = ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} ue^{−u^{2}/2} du = 0,

since the integrand is an odd function. On the other hand, integration by parts yields that

Eξ^{2} = ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} u^{2}e^{−u^{2}/2} du = − ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} u (\frac{d}{du}e^{−u^{2}/2}) du

= ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} e^{−u^{2}/2} du = 1.

If ξ is a Φ_{m,σ} distributed random variable, then its transform \frac{ξ−m}{σ} has standard normal distribution, ı.e. it has expectation zero and variance 1. Hence ξ has expectation m and variance σ^{2}.

12.) Let φ(t) = φ(t_{1}, …, t_{k}) = Ee^{i(t,ξ)} = Ee^{i(t_{1}ξ_{1}+···+t_{k}ξ_{k})} denote the characteristic function of a k-dimensional random vector ξ = (ξ_{1}, …, ξ_{k}), where t = (t_{1}, …, t_{k}),

and (t, ξ) = ∑_{j=1}^{k} t_{j}ξ_{j}. Then |φ(t)| ≤ E|e^{i(t,ξ)}| = 1. For arbitrary number ε > 0 there

exists a constant R = R(ε) such that P (|ξ| > R) = P ( ∑_{j=1}^{k} ξ_{j}^{2} > R^{2}) < \frac{ε}{2}. Put δ =

\frac{ε}{2R(ε)} and consider such numbers t = (t_{1}, …, t_{k}) for which |t|^{2} = ∑_{j=1}^{k} t_{j}^{2} < δ. Then

|e^{i(t,ξ)} − 1| ≤ |(t, ξ)| ≤ \frac{ε}{2} for |x| < R(ε), and |φ(t) − φ(t̄)| = |Ee^{i(t,ξ)} − Ee^{i(t̄,ξ)}| ≤ E|e^{i(t−t̄,ξ)} − 1| ≤ E|e^{i(t−t̄,ξ)} − 1|I(|ξ| ≤ R) + P (|ξ| > R) ≤ E|(t − t̄, ξ)|I(|ξ| ≤ R) + \frac{ε}{2} ≤ ε if |t − t̄| ≤ δ, where I(A) denotes the indicator function of a set A. Hence the function φ(t) is uniformly continuous. The characteristic function of a random vector aξ + m in a point t ∈ R^{k}, where a ∈ R, m ∈ R^{k} is the function Ee^{i(t,aξ+m)} = e^{(it,m)}Ee^{i(at,ξ)} = e^{(it,m)}φ(at), where φ denotes the characteristic function of the random vector ξ. If ξ_{j}, j = 1, …, n, are independent random vectors with characteristic functions φ_{j}(t), then the characteristic function of the random sum ξ_{1} +⋯+ξ_{n} in a point t ∈

R^{k} equals Ee^{i(t,ξ_{1}+···+ξ_{n})} = Ee^{i(t,ξ_{1})} ⋯ e^{i(t,ξ_{n})} = Ee^{i(t,ξ_{1})} ⋯ Ee^{i(t,ξ_{n})} = ∏_{j=1}^{k} φ_{j}(t).

13.) a.) If the random variable ξ has standard normal distribution, then

Ee^{itξ} = ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} e^{itu−u^{2}/2} du = e^{−t^{2}/2} ∫_{−∞}^{∞} \frac{1}{\sqrt{2π}} e^{−(it−u)^{2}/2} du = e^{−t^{2}/2}

by the result of problem 1. b.) If the random variable ξ has uniform distribution in the interval [0, 1], then

Ee^{itξ} = ∫_{0}^{1} e^{itu} du = \frac{e^{it} − 1}{it}.

c.) If the random variable ξ has exponential distribution with parameter λ > 0,

then

Ee^{itξ} = ∫_{0}^{∞} λe^{itu−λu} du = \frac{λ}{λ − it}.

d.) If ξ is a random variable with Cauchy distribution, then

Ee^{itξ} = ∫_{−∞}^{∞} \frac{1}{π} \frac{e^{itu}}{1 + u^{2}} du.

This integral can be calculated by means of the residium theorem in the theory of analytic functions.

The function g(z) = g(z, t) = \frac{e^{itz}}{π(1+z^{2})} is analytic in the plane of complex numbers with two poles z = ±i. The residium of the function g(z) in the point i equals e^{−t}, and in the point −i it equals e^{t}. Let us consider the following contour integral. Let us first integrate the function g(z) = g(z, t) on the interval [−R, R] and then on the half-circle |z| = R, Im z ≥ 0 if t ≥ 0 and on the half-circle |z| = R, Im z ≤ 0 if t ≤ 0. The above contour integral equals the residium of the function g(z) in the point i if t > 0 and the residium of this function in the point −i if t < 0. On the other hand the restriction of the integral to the half-circle of radius R tends to zero if R → 0. This implies that Ee^{itξ} = ∫_{−∞}^{∞} g(t, u) du = e^{−|t|}. The following argument gives another different a little bit artificial but correct proof of this statement. The characteristic function of the density function f(x) = \frac{1}{2}e^{−|x|} equals

\frac{1}{2} ∫_{−∞}^{∞} e^{−|x|+itx} dx = \frac{1}{2} (\frac{1}{1 + it} + \frac{1}{1 − it}) = \frac{1}{1 + t^{2}} .

Since the function \frac{1}{1+t^{2}} is integrable the inverse Fourier transformation formula (6) implies the desired statement.

e.) If the random variable ξ has Poissonian distribution with parameter λ > 0, then

Ee^{itξ} = e^{−λ} ∑_{k=0}^{∞} \frac{λ^{k}}{k!} e^{ikt} = exp \{λ(e^{it} − 1)\} .

f.) If the random variable ξ has binomial distribution with parameters n and p, then

Ee^{itξ} = e^{−λ} ∑_{k=0}^{n} \binom{n}{k}p^{k}e^{ikt}(1 − p)^{n−k} = (1 − p + pe^{it}) .

g.) If ξ is a random variable with negative binomial distribution with parameters n and p, then its distribution agrees with the distribution of the random sum ξ_{1} + ⋯ + ξ_{n}, where ξ_{j}, 1 ≤ j ≤ n, are independent random variables with negative binomial distribution with parameters 1 and p. (To see this property observe that a possible the random variable ξ has the following probabilistic interpretation: If we make independent experiment after each other which are successful with probability p, then ξ denotes the number of unsuccessful experiments up to the n-th successful experiment. If ξ_{j} denotes the number of the unsuccessful experiments between the j −1-th and j-th successful experiments, then we get the above representation.) Hence Ee^{itξ} = (Ee^{itξ_{1}})^{n}. On the other hand,

Ee^{itξ_{1}} = ∑_{k=0}^{∞}(1 − p)p^{k}e^{itk} = \frac{1 − p}{1 − pe^{it}} .

h.) We get with the change of variables ū = (1 − it)u that

φ_{s}(t) = \frac{1}{Γ(s)} ∫_{0}^{∞} e^{−u+itu}u^{s−1} du = \frac{1}{Γ(s)} \frac{1}{1 − it)^{s}} ∫_{0}^{∞} e^{−ū}ū^{s−1} dū = \frac{1}{(1 − it)^{s}} .

In this calculation we applied some complex analysis argument. At the change of variables step ū = (1 − it)u of the calulation the domain of integration became the line (1 − it)u, u > 0, instead of the positive abscissa axis. But we can turn back the domain of integration to the positive abscissa axis by means of a usual complex analysis argument, by which the integral of an analytic function on a closed curve equals zero. At this step we have to exploit that the function e^{−z} tends to zero fast as Re z → ∞.

14.) If f(x_{1}, …, x_{k}) and g(x_{1}, …, x_{k}) are two integrable functions, then

∞ > ∫ ∫ |f(x_{1}, …, x_{k})||g(u_{1}, …, u_{k})| dx_{1} ⋯ dx_{k}du_{1} … du_{k}

(with substitution ū_{j} = x_{j} + u_{j} j = 1, …, k,)

= ∫ ∫ |f(x_{1}, …, x_{k})||g(ū_{1} − x_{1}, …, ū_{k} − x_{k})| dx_{1} ⋯ dx_{k} dū_{1} … dū_{k}

= ∫ (∫ |f(x_{1}, …, x_{k})||g(ū_{1} − x_{1}, …, ū_{k} − x_{k})| dx_{1} ⋯ dx_{k}) dū_{1} … du_{k}

= ∫ |f| ∗ |g|(x_{1}, …, x_{k}) dx_{1} … dx_{k}.

This relation implies that the function |f ∗ g(x_{1}, …, x_{k})| ≤ |f| ∗ |g|(x_{1}, …, x_{k}) is bounded in almost all points (x_{1}, …, x_{k}) ∈ R^{k}. It also implies that f ∗ g is an integrable function. In the sequel we write x instead of (x_{1}, …, x_{k}) and u instead of (u_{1}, …, u_{k}). If μ and ν are two measures of bounded variation, then there exists a representation μ = μ_{1} − μ_{2}, ν = ν_{1} − ν_{2} such that μ_{i} and ν_{i}, i = 1, 2, are finite measures. The identity μ ∗ ν = (μ_{1} ∗ ν_{1} + μ_{2} ∗ ν_{2}) − (μ_{1} ∗ ν_{2} + μ_{2} ∗ ν_{1}) holds. Since μ_{i} ∗ ν_{j}(R^{k}) < ∞ for all indices i, j = 1, 2, this implies that μ ∗ ν is a measure of bounded variation. If the measure μ has a density function f, then we have for all measurable sets A ⊂ R^{k}

∫_{A} f ∗ ν(x) dx = ∫_{A} (∫ f(u)ν(x − du)) dx = ∫_{A} (∫ f(x − u)ν( du)) dx

= ∫ (∫_{A} f(x − u) dx) ν( du) = ∫ (∫ I(x: x ∈ A)f(x − u) dx) ν( du)

= ∫ (∫ I(v: u + v ∈ A)f(v) dv) ν( du)
= ∫ ∫ I(v: u + v ∈ A)μ( dv)ν( du)
= μ × ν ({(u, v): u + v ∈ A}) = μ ∗ ν(A)

and this means that the function f ∗ ν is the density function of the convolution μ ∗ ν of the measures μ and ν. Let us observe that the above calculations also imply that the function f ∗ ν(x) is finite in almost all points x ∈ R^{k}, moreover it is integrable. Indeed, the above calculation implies this result with the choice A = R^{k} if μ and ν are (bounded) positive measures. The general case can be reduced to this case if we decompose the measures μ and ν as the difference of two positive finite measures. (We may assume that the measures in the decomposition μ = μ_{1} −μ_{2} have density function.) If the measure μ has a density function f and the measure ν has a density function g then let us define the measures ν̄_{x}(A) = ν(x − A) for all x ∈ R^{k}. The density function of the measure ν̄_{x}(A) equals g(x − u) in the point u ∈ R^{k}, and the density function of the measure μ ∗ ν in the point x equals

∫ f(u)ν̄_{x}(du) = ∫ f(u)g(x − u) du = f ∗ g(x)

by the already proved results. 15.) It follows from the definition of the convolution that if ξ an η are independent

random variables with distributions μ and ν, then the distribution of the random sum ξ + η equals μ ∗ ν. It follows from the result of the previous problem that if the distribution μ of the random variable ξ has a density function f, then the distribution μ ∗ ν of the random variable ξ + η has a density function f ∗ ν. If also the measure ν has a density function g, then this density function equals f ∗ g. If the distribution of the random variable Z is F (x), then the distribution of the random variable Z̄ = \frac{Z−A}{B} with B > 0 equals F (Bx + A). If the random variable Z has a density function f(x), then the random variable Z̄ has a density function Bf(Bx + A). The previous results imply the statements formulated for the distribution and density function of the random sum S̄_{n}.

The relation μ ∗ ν = ν ∗ μ follows from the definition of the convolution. The statement (μ_{1} ∗ μ_{2}) ∗ μ_{3} = μ_{1} ∗ (μ_{2} ∗ μ_{3}) follows from the fact that the identity

(μ_{1} ∗ μ_{2}) ∗ μ_{3}(A) = μ_{1} ∗ (μ_{2} ∗ μ_{3})(A) = μ_{1} × μ_{2} × μ_{3}({(u, v, w): u + v + w ∈ A})

holds for all measurable sets A. The analog statements about the convolution of functions can be reduced to these statement if we represent the convolution of functions as the convolution of the density function of the corresponding signed measures. Otherwise these statement also can be simply proved by simple calculation.

16.) If μ and ν are two signed measures of bounded variation with Fourier transforms f̃(t_{1}, …, t_{k}) and g̃(t_{1}, …, t_{k}), then

f̃(t_{1}, …, t_{k})g̃(t_{1}, …, t_{k})
= ∫ e^{i(t_{1}(u_{1}+v_{1})+···+t_{k}(u_{k}+v_{k}))}μ( du_{1}, …, du_{k})ν( dv_{1}, …, dv_{k}).

The transformation T(u_{1}, …, u_{k}, v_{1}, …, v_{k}) = (u_{1}+v_{1}, …, u_{k} +v_{k}), (u_{1}, …, u_{k}) ∈ R^{k}, (v_{1}, …, v_{k}) ∈ R^{k}, is a measurable transformation from the space (R^{k} × R^{k}, B_{2k}, μ × ν) to the space (R^{k}, B_{k}, μ ∗ ν), where B_{2k} and B_{k} denote the σalgebras in the spaces R^{2k} and R^{k}. By applying the measure theoretical result which describe how measurable transformations transform integrals for the functions

h(x_{1}, …, x_{k}) = e^{it(x_{1}+···+x_{k})})

and

g(u_{1}, …, u_{k}, v_{1}, …, v_{k}) = h(T(u_{1}, …, u_{k}, v_{1}, …, v_{k})) = e^{i(t_{1}(u_{1}+v_{1})+···+t_{k}(u_{k}+v_{k}))}

with the above defined transformation T(u_{1}, …, u_{k}, v_{1}, …, v_{k}) we get from the relation written at the beginning of the solution that

f̃(t_{1}, …, t_{k})g̃(t_{1}, …, t_{k}) = ∫ e^{i(t_{1}x_{1}+···+t_{k}x_{k})}μ ∗ ν( dx_{1}, …, dx_{k}).

This implies the statement about the Fourier transform of the convolution of measures. The analogous statement about the Fourier transform of the convolution of density functions follows from this statement and the relation between the convolution of measures and their density functions.

17.) By differentiating the identity f ∗ g(x) = ∫ f(x − u)g(u) du k times we get that

\frac{df ∗ g^{k}(x)}{dx^{k}} = ∫ \frac{df^{k}(v)}{dv^{k}} |_{v=x−u} g(u) du = ∫ \frac{df^{k}(v)}{dv^{k}} |_{v=u} g(x − u) du.

The conditions of the problem allow the above successive differentiations. Further, working with the right-hand side of the last formula we can carry out l additional differentiations and get that

\frac{df ∗ g^{k+l}(x)}{dx^{k+l}} = ∫ \frac{df^{k}(v)}{dv^{k}} |_{v=u} \frac{dg^{l}(v)}{dv^{l}} |_{v=x−u} du.

If f(u) is analytic function which also satisfies the other conditions of the problem, then the function

F (z) = ∫ f(z − u)g(u) du

is an analytic continuation of the convolution f ∗g(x) to the domain {z: Im z < A}. 18.) a.) Convergence in distribution implies the convergence of the integrals:

Since F (x_{1}, …, x_{k}) → 1 if x_{j} → ∞ for all j = 1, …, k, and F (x_{1}, …, x_{k}) → 0 if x_{j} → −∞ for one of the indices 1 ≤ j ≤ k hence for all ε > 0 there exists a k-dimensional rectangle K = K(ε) such that μ_{F} (K) > 1 − ε. (Given a distribution function F in the sequel we shall denote by μ_{F} the probability measure on R^{k} induced by the distribution function F.) We also may assume, by enlarging the rectangle K if it is necessary, that the boundary of the rectangles K has zero μ_{F} measure. Indeed, the projection of the distribution function F to the j-th coordinate is a one-dimensional distribution function, and as a consequence it has at least countably many atoms (points with positive measure with respect the measure induced by this distribution) for all indices j = 1, …, k. This implies that we can choose a larger rectangle K if this is necessary whose boundary has μ_{F} measure zero. Because of the boundedness of the function f the relation

|∫_{R^{k}\K} f(x_{1}, …, x_{k}) dF (x_{1}, …, x_{k})| < const. ε

holds, and also lim sup_{n→∞} |∫_{R^{k}\K} f(x_{1}, …, x_{k}) dF_{n}(x_{1}, …, x_{k})| < const. ε for all n =

1, 2, … because of the zero μ_{F} boundary of the rectangle K and the convergence of the distribution functions F_{n} to the distribution function F. Furthermore, the function f is uniformly continuous on the rectangle K hence there exists some constant δ > 0 such that |f(x)−f(y)| < ε if |x−y| ≤ δ, and x, y ∈ K. The rectangle K can be decomposed to finitely many rectangles Δ_{j}, j = 1, …, p(K), of diameter less than δ without joint interior points, and such that all these rectangles Δ_{j} have boundaries with μ_{F} measure zero. These properties imply that lim_{n→∞} μ_{F_{n}}(Δ_{j}) = μ_{F} (Δ_{j}) for all indices j = 1, …, p(K), and because of the uniform continuity of the function f on the rectangle K

lim sup |∫_{K} f dF_{n} − ∫_{K} f dF | < ε.

The above inequalities imply that lim sup |∫ f dF_{n} − ∫ f dF | < const. ε with a const. independent of ε. Since this inequality holds for all ε > 0, it implies the statement we wanted to prove.

b.) The convergence of the integrals implies convergence in distribution: Let x = (x_{1}, …, x_{k}) be a point of continuity of the distribution function F. Then for all numbers ε > 0 there exists a number δ = δ(ε) > 0 such that the points y = (y_{1}, …, y_{k}) = (x_{1} − δ, …, x_{k} − δ) and z = (z_{1}, …, z_{k}) = (x_{1} + δ, …, x_{k} + δ) satisfy the inequalities F (y) > F (x)−ε and F (z) < F (x)+ε. There exist continuous functions f_{1}(u) and f_{2}(u) an the k-dimensional Euclidean space R^{k} which satisfy the following properties: 0 ≤ f_{i}(u) ≤ 1 for all u ∈ R^{k}, i = 1, 2. Further f_{1}(u) = 1 for u = (u_{1}, …, u_{k}) if u_{j} ≤ y_{j} for all indices j = 1, …, k, and f_{1}(u) = 0 if u_{j} ≥ x_{j} for some of the indices 1 ≤ j ≤ k. The function f_{2}(·) satisfies the following relations: f_{2}(u) = 1 if u_{j} ≤ x_{j} for all j = 1, …, k, and f_{2}(u) = 0 if u_{j} ≥ z_{j} for some 1 ≤ j ≤ k. Then

lim sup_{n→∞} F_{n}(x) ≥ lim_{n→∞} ∫ f_{1}(u) dF_{n}(u) = ∫ f_{1}(u) dF (u) ≥ F (x) − ε
lim inf_{n→∞} F_{n}(x) ≤ lim_{n→∞} ∫ f_{2}(u) dF_{n}(u) = ∫ f_{2}(u) dF (u) ≤ F (x) + ε.

Since these relations hold for all ε > 0, they imply the statement of part b.).

19.) Since ⋃_{K=1}^{∞} K(K)^{k} = R^{k}, and the rectangles K(K)^{k}, K = 1, 2, …, constitute

a monotone increasing series of sets, hence lim_{K→∞} μ(K(K)^{k}) = μ(R^{k}) = 1, i.e.

μ(K(K)^{k}) ≥ 1 − ε if K ≥ K(ε). To show that the characteristic function of the probability measure μ is determined by its characteristic function let us first observe that the integrals ∫ f(u) dμ(u) determine the measure μ if we take all continuous functions f(·) with a bounded support. Indeed, the measure μ of those rectangles P = [K_{1}, L_{1}) × ⋯ × [K_{k}, L_{k}) whose boundary has μ measure zero determine the measure μ. Besides, we claim that for all numbers ε > 0 and rectangles P there exists a function f_{ε,P}(·) such that 0 ≤ f_{ε,P}(u) ≤ 1 for all points u ∈ R^{k}, f_{ε,P}(u) = 1 if u ∈ P, and f_{ε,P}(u) = 0 if ρ(u, P) > ε. (In the sequel ρ(·, ·) denotes the usual Euclidean distance in the space R^{k}.) Then the relation μ(P) = lim_{ε→0} ∫ f_{P,ε} dμ(u) implies the above property.

A possible construction of a function f_{P,ε} with the above properties is the following: Put f_{P,ε}(u) = 1 − g_{P,ε}(u) and g_{P,ε}(u) = min (1, \frac{1}{ε}ρ(u, P)). Given a continuous function f(·) of compact support together with a sufficiently large number K > 0 for which the cube [−K, K] × ⋯ × [−K, K] contains the support of the function f(·) let us define the periodic extension of the function f(·) with period 2K by the formula f_{K}(u_{1} + 2Kj_{1}, ⋯, u_{k} + 2Kj_{k}) = f(u_{1}, ⋯, u_{k}), −K ≤ u_{j} < K, l_{j} = 0, ±1, ±2, …, j = 1, …, k. The integrals ∫ f_{K}(u) dμ(u) of these functions obtained by periodic extension determine the measure μ, since ∫ f(u) dμ(u) = lim_{K→∞} ∫ f_{K}(u) dμ(u) because of the tightness property of the measure μ formulated in the already proven part of problem 19. Finally, by Weierstrass second approximation theorem for all continuous functions f_{K}(⋯) K with period K and real number ε > 0 there exists a trigonometrical polynomial

g_{ε} = g_{ε,f_{K}} (u_{1}, ⋯, u_{k}) = ∑ c_{j_{1},...,j_{k}}^{ε}e^{iπ(j_{1}u_{1}+···+j_{k}u_{k})/K}

such that sup_{u∈R^{k}} |f_{K}(u) − g_{ε}(u)| ≤ ε. Hence

|∫ f_{K}(u) dμ(u) − ∫ g_{ε}(u) dμ(u)| ≤ ε.

On the other hand, ∫ g_{ε}(u) dμ(u) = ∑ c_{j_{1},...,j_{k}}^{ε}φ (\frac{πj_{1}}{K} , …, \frac{πj_{k}}{K} ), that is the above integral can be calculated by means of the characteristic function of the measure μ. This implies that this characteristic function determines the integrals of the form ∫ f_{K}(u) dμ(u). Hence it also determines the measure μ. The proof can be generalized without any essential modification to arbitrary signed measures μ with bounded variation.

20.) First we show that for all numbers a > 0 there exists an even density function f(u) whose Fourier transform φ(t) is sufficiently smooth, e.g. it is twice differentiable, and it equals zero outside the interval [−a, a]. Indeed, let us consider a continuously differentiable function g(u) which is concentrated in the interval [−\frac{a}{2}, \frac{a}{2}], g^{−}(u) = g(−u). Then put h(u) = g ∗ g^{−}(u), f(u) = \frac{2π}{M} ∫ e^{itu}h(u) du, where ∗ denotes convolution, and M = h(0) = ∫ |f(u)|^{2} du. We claim that this function f is a density function, and its characteristic function is the function \frac{h^{−}(u)}{M}, h^{−}(u) = h(−u), which vanishes outside the interval [−a, a]. Indeed, the function h(·) is twice differentiable, (see problem 17), hence its Fourier transform tends to zero in plus–minus infinity with order |t|^{−2} (see e.g. problem 28 of this series of problems discussed later), hence the above defined Fourier transform f(·) of the function \frac{h(u)}{M} is integrable, and we can apply the inverse Fourier transform for it. Since f(·) is en even function, this means that the function \frac{h^{−}(u)}{M} = ∫ e^{itx}f(u) du is the Fourier transform of the function f(u). In particular, \frac{h(0)}{M} = 1 = ∫ f(u) du. Finally, f(t) ≥ 0 for all numbers t ∈ R^{1}, since the Fourier transform of the function g ∗ g^{−}(·) equals ∫ e^{itu}g ∗ g^{−}(u) du = ∫ e^{itu}g(u) du ∫ e^{itu}g^{−}(u) du = |∫ e^{itu}g(u) du|^{2} ≥ 0. These properties mean that the function f(·) is a density function. (We shall return to the above problem in the second part of this series of problems where such a construction will be useful in a different context.) Let us consider an even density function f(u) whose characteristic function φ(t) is twice differentiable, and vanishes outside of a finite interval [−a, a]. Consider a number T > a, and let us define the numbers a_{k} = \frac{1}{4πT} ∫ e^{−iπtk/T} φ(t) dt, k = 0, ±1, ±2, . . .. Let us put weights a_{k} in the points \frac{πk}{T}, k = 0, ±1, ±2, . . .. We claim that in such a way we constructed a probability distribution on the lattice \frac{πk}{T}, k = 0, ±1, ±2, ⋯, whose characteristic function to the interval [−T, T] agrees with the restriction of the characteristic function φ(t) to this interval. The characteristic function of this discrete distribution is the periodic extension of the restriction of φ(t) to the interval [−T, T] with period 2T. This statement means in particular, that the above defined discrete distribution together with the distribution with density function f(·) yield an example satisfying the statement of problem 20.

The statement formulated for the discrete distribution with weights a_{k} holds, because for one hand a comparison of the definition of the number a_{k} with the inverse Fourier transformation formula expressing the function f(·) yields that

a_{k} = \frac{1}{2T} f (\frac{πk}{T} ) ≥ 0. On the other hand, the trigonometrical sum ∑_{k=−∞}^{∞} a_{k}e^{πik/T} is

the Fourier series of the function φ(t) restricted to the interval [−T, T]. In particu-

lar, φ(0) = 1 = ∑_{k=−∞}^{∞} a_{k}. (As φ(·) is a twice differentiable function, hence it equals

his Fourier series in all points.)

21.) Let us first show that if the sequence of the probability measures μ_{n}, n = 1, 2, …, is relatively compact, then it is also tight.

Let us assume indirectly this sequence of measures μ_{n}, n = 1, 2, …, is not tight. Then there exists a constant ε > 0, a subsequence μ_{n_{k}} of the sequence of probability measures μ_{n} and a sequence of positive numbers K_{n}, n = 1, 2, …, such that K_{n} → ∞, and μ_{n_{k}}([−K_{n}, K_{n}] × ⋯ × [−K_{n}, K_{n}]) < 1 − ε. We shall show that this subsequence μ_{n_{k}} of the sequence of measures μ_{n} has no sub-subsequence convergent in distribution. This means that the above formulated indirect assumption leads to contradiction.

Indeed, let us assume indirectly that the sequence of measures μ_{n_{k}} has a subsequence μ_{n_{k_{j}}} which converges in distribution to a probability measure μ. Then, there exists a constant K > 0 such that μ([−K, K] × ⋯ × [−K, K]) > 1 − \frac{ε}{2}, and the hyperplanes u_{j} = ±K, j = 1, 2, …, k, have μ measure zero. Then also the

relation lim_{j→∞} μ_{n_{k_{j}}} ([−K, K] × ⋯ × [−K, K]) = μ([−K, K] × ⋯ × [−K, K]) should

hold. But this is not possible, since the lim sup of the probabilities at the left-hand side is smaller than 1 − ε, while the right-hand side is greater than 1 − \frac{ε}{2}.

Let us show that if the sequence of measures μ_{n} is tight then it is also relatively compact.

We have to show that an arbitrary subsequence of the sequence μ_{n} has a subsubsequence convergent in distribution. For the sake of simpler notations let us denote by reindexing the elements of the subsequence again by μ_{n}. We have to show that this (also tight) sequence of probability measures μ_{n} has a subsequence convergent in distribution.

Let F_{n}(u) = F_{n}(u_{1}, …, u_{k}) denote the distribution function F_{n}(u_{1}, …, u_{k}) = μ_{n}({(v_{1}, …, v_{k}): v_{j} < u_{j}, j = 1, …, k}) determined by the measure μ_{n}. Let

u^{(p)} = (u_{1}^{(p)}, …, u_{k}^{(p)}) , p = 1, 2, …,

denote the (countable) set of points u^{(p)} ∈ R^{k} with rational coordinates with some indexing. First we show with the help of the so-called diagonal procedure that there exists an appropriate sequence of positive integers n_{j}, j = 1, 2, …, such that the limit

lim_{j→∞} F_{n_{j}} (u_{1}^{(p)}, …, u_{k}^{(p)}) = F̃ (u_{1}^{(p)}, …, u_{k}^{(p)})  (2.4)

exists for all numbers p = 1, 2, . . .. Indeed, as 0 ≤ F_{n}(u) ≤ 1, there is a subsequence n̄_{j} = (n_{j,1}) of the integers such that the limit lim_{j→∞} F_{n_{j,1}}(u^{(1)}) = F̃(u^{(1)}) exists. This sequence has a sub-

sequence n_{j,2} such that the limit lim_{j→∞} F_{n_{j,2}}(u^{(2)}) = F̃(u^{(2)}) also exists. Follow-

ing this procedure we can construct sequences n_{j,p}, j = 1, 2, …, p = 1, 2, … in such a way that the p + 1-th sequence is a subsequence of the p-th sequence, i.e. {n_{j,p+1}, j = 1, 2, … } ⊂ {n_{j,p}, j = 1, 2, … }, p = 1, 2, …, and the limit lim_{j→∞} F_{n_{j,p}}(u^{(p)}) = F̃(u^{(p)}) exists for all numbers p = 1, 2, . . .. Then the sequence

n_{j} = n_{j,j} satisfies relation (2.4). Let us introduce the function

F (u_{1}, …, u_{k}) = sup ; {u^{(p)}=(u_{1}^{(p)},...,u_{k}^{(p)}): u_{s}^{(p)}<u_{s}, s=1,...,k} F̃ (u_{1}^{(p)}, …, u_{k}^{(p)}) , (2.5)

where the function F̃ satisfies relation (2.4) with an appropriate (fixed) sequence of positive integers n_{j}, and the points u^{(p)} are the points of the k-dimensional space with rational coordinates. We claim that the above defined function F (u_{1}, …, u_{k}) is a distribution function, and the distribution functions F_{n_{j}}(u_{1}, …, u_{k}) converge in distribution to it. If we prove this statement then we complete the solution of problem 21. To show that the function F (u_{1}, …, u_{k}) is a distribution function we recall an “internal” characterization result of distribution functions which describes the distribution functions only with the help of their property. The result we recall states that a function F (u_{1}, …, u_{k}) is a distribution function if and only if it satisfies the following four properties:

(i) The function F (u_{1}, …, u_{k}) is a function continuous from the left in all of its arguments. (ii) lim_{u_{j}→∞ for all indices j=1,...,k} F (u_{1}, …, u_{k}) = 1.

(iii) lim_{u_{j}→−∞ for some index 1≤j≤k} F (u_{1}, …, u_{k}) = 0.

Finally, to formulate the last condition let us define for a function F on the space R^{k} and a rectangle K = [a_{1}, b_{1}) × ⋯ × [a_{k}, b_{k}) the number

μ(K) = μ_{F} (K) = ∑_{\begin{matrix}u_{j}= a_{j} vagy b_{j} \\ j=1,...,k \end{matrix}}(−1)^{χ(u_{1},...,u_{k})}F (u_{1}, …, u_{k}),

where χ(u_{1}, …, u_{k}) denotes the quantity of the numbers a_{j} in the sequence u_{1}, …, u_{k}. Then

(iv) μ_{F} (K) ≥ 0 for all rectangles K.

Since the function F̃(u_{1}, …, u_{k}) defined in points of rational coordinates is a monotone increasing function in all of its coordinates, the function F (·) defined in formula (2.5) also satisfies property (i). Furthermore, this monotonicity also implies that we can replace the sup by lim in formula (2.5) if we consider such sequences of indices (u_{1}^{(p)}, …, u_{k}^{(p)}), p = 1, 2, …, in the limit for which u_{j}^{(p)} < u_{j} for all

numbers 1 ≤ j ≤ k and p = 1, 2, …, and lim_{p→∞} u_{j}^{(p)} = u_{j} for all indices j = 1, …, k.

Let us consider such rectangles K(p) whose edges have rational coordinates, and the edges of these coordinates converge in a monotone increasing way to the coordinates of the corresponding edges of the rectangle K. Then μF̃(K(p)) ≥ 0, since F̃ is the limit of distribution functions. Hence μ_{F} (K) = lim_{p→∞} μF̃(K(p)) ≥ 0, i.e.

the function F satisfies property (iv). Let us observe that properties (ii) and (iii) hold if the functions F_{n} are replaced by the function F̃, and the limit is taken only in rational points. (We applied at this point of the proof the tightness of the measures μ_{n}.) This property together with formula (2.5) imply that the function F satisfies properties (ii) and (iii). Property (i) also holds. This is a simple consequence of it definition in formula (2.5). To show that the distribution functions F_{n_{j}} converge in distribution to the distribution function F let us consider a point of continuity u = (u_{1}, …, u_{k}) of the function F, and let us then choose for all numbers ε > 0 such a constant number δ = δ(ε) > 0 for which F (u) − ε ≤ F (u − δ) ≤ F (u) ≤ F (u + δ) ≤ F (u) + ε, where u ± δ = (u_{1} ± δ, …, u_{k} ± δ). Let us then choose two points r = (r_{1}, …, r_{k}) ∈ R^{k} and r̄ = (r̄_{1}, …, r̄_{k}) ∈ R^{k} with rational coordinates such that u_{j} − δ < r_{j} < u_{j} < r̄_{j} < u_{j} + δ for all indices j = 1, …, k. Then by the monotonicity properties of the function F̃(·) and the definition of the function F

F (u) − ε ≤ F (u − δ) < F̃(r) ≤ F (u) ≤ F̃(r̄) ≤ F (u + δ) ≤ F (u) + ε.

This relation together with the definition of the function F̃ and the monotonicity property of the functions F_{n_{j}} imply that

F (u) − ε ≤ lim_{j→∞} F_{n_{j}}(r) ≤ lim inf_{j→∞} F_{n_{j}}(u)

≤ lim sup_{j→∞} F_{n_{j}}(u) ≤ lim_{j→∞} F_{n_{j}}(r̄) ≤ F (u) + ε,

hence

−ε ≤ lim inf_{j→∞} F_{n_{j}}(u) − F (u) ≤ lim sup_{j→∞} F_{n_{j}(u)} − F (u) ≤ ε.

Since this relation holds for all ε > 0, it implies that lim_{j→∞} F_{n_{j}}(u) = F (u).

22.) Fix some δ > 0 and write the following identity:

\frac{1}{2δ} ∫_{−δ}^{δ} Re [1 − φ_{n}(t)] dt = ∫_{−δ}^{δ} \frac{1}{2δ} ∫_{−∞}^{∞} [1 − cos tx] dF_{n}(x) dt

= ∫_{−∞}^{∞} \frac{1}{2δ} ∫_{−δ}^{δ}[1 − cos tx] dt dF_{n}(x) = ∫_{−∞}^{∞} [\frac{t}{2δ} − \frac{sin tx}{2δx}]_{t=−δ}^{t=δ} dF_{n}(x)

= ∫_{−∞}^{∞} (1 − \frac{sin δx}{δx}) dF_{n}(x) = ∫_{−K}^{K} (1 − \frac{sin δx}{δx}) dF_{n}(x)

+ ∫_{|x|>K} (1 − \frac{sin δx}{δx}) dF_{n}(x) = I_{1,n}^{δ}(K) + I_{2,n}^{δ}(K).

(2.6)

First we show with the help of relation (2.6) that the validity of formula (10) implies that the sequence of distribution functions F_{n} is tight. Since (1 − \frac{sin δx}{δx}) ≥ 0 for all x and δ, hence the left-hand side of formula (2.6) yields an upper bound on the expression I_{2,n}^{δ}(K) for all numbers δ > 0, n ≥ 1 and K > 0. If formula (10) holds, then for all ε > 0 there exists a number δ = δ(ε) > 0 and threshold index n_{0} = n_{0}(δ, ε) such that \frac{ε}{2} ≥ ∫_{|x|>K} (1 − \frac{sin δx}{δx}) dF_{n}(x) for n ≥ n_{0}. Put K = \frac{2}{δ}. Then 1 − \frac{sin δx}{δx} ≥ \frac{1}{2} for all |x| ≥ K. Hence the previous estimate implies that \frac{ε}{2} ≥ ∫_{|x|>K} (1 − \frac{sin δx}{δx}) dF_{n}(x) ≥ \frac{1}{2}[(1 − F_{n}(K)) + F_{n}(−K)], i.e. ε ≥ [(1 − F_{n}(K)) + F_{n}(−K)] with this number K if n ≥ n_{0}. By increasing the number K > 0 if it is necessary we can achieve that the above inequality holds for all indices n ≥ 1. This means that the distribution functions F_{n}, n = 1, 2, …, are tight. Let us prove with the help of formula (2.6) that the tightness of the distribution functions F_{n} implies formula (10) and even its slightly stronger version, formula (10′) where lim sup_{n→∞} is replaced by sup_{n≥1}. Since |1 − \frac{sin δx}{δx} | ≤ 2, the tightness of

the distribution functions F_{n} makes possible to choose a number K = K(ε) > 0 for all ε > 0 such that |I_{2,n}^{δ}(K)| = |∫_{|x|>K} (1 − \frac{sin δx}{δx}) dF_{n}(x)| ≤ \frac{ε}{2} for all numbers δ > 0 and n = 1, 2, . . .. After fixing the number K = K(ε) > 0 we can choose a number δ̄ = δ̄(ε, K) > 0 such that the inequality ε ≥ 1 − \frac{sin δx}{δx} ≥ 0 holds for all numbers

|x| < K and 0 < δ < δ̄. Hence |I_{1,n}^{δ}(K)| = |∫_{−K}^{K} (1 − \frac{sin δx}{δx}) dF_{n}(x)| ≤ \frac{ε}{2}. These

estimates together with relations (2.6) imply that |\frac{1}{2δ} ∫_{−δ}^{δ} Re [1 − φ_{n}(t)] dt| ≤ ε for

all n ≥ 1 if δ ≤ δ̄(ε). The statements of problem 22 are proved. 23.) Let us consider the j-th coordinate, 1 ≤ j ≤ k, of the random vectors ξ^{(n)}, i.e. the

random variables ξ_{j}^{(n)} for all indices n = 1, 2, . . .. The characteristic function of the random variable ξ_{j}^{(n)} is the function

φ_{n}^{(j)}(t) = φ_{n}( \underbrace{0, ⋯, 0,}_{j−1 0 coordinates} t, \underbrace{0, ⋯, 0}_{n−j−1 0 coordinates}).

By the conditions of the problem the functions φ_{n}^{(j)}(t) converge to a function φ^{(j)}(t) continuous in zero in a small neighbourhood of the origin. Let us remark that φ^{(j)}(0) = lim_{n→∞} φ_{n}^{(j)}(0) = 1. Hence the continuity of the function φ^{(j)}(t) in the

origin implies that for all numbers ε > 0 there exists a threshold index δ̄ = δ̄(ε) > 0 such that for all numbers 0 < δ < δ̄

0 ≤ \frac{1}{2δ} ∫_{−δ}^{δ} Re [1 − φ_{n}^{(j)}(t)] dt < ε.

Furthermore, as lim_{n→∞} Re [1 − φ_{n}^{(j)}(t)] = Re [1 − φ^{(j)}(t)] if |t| < δ < δ̄ (we choose a

smaller threshold δ̄ > 0 if it is necessary), and 0 ≤ Re [1 − φ_{n}^{(j)}(t)] ≤ 2, it follows from Lebesgue’s dominated convergence theorem that 0 ≤ lim sup_{n→∞} \frac{1}{2δ} ∫_{−δ}^{δ} Re [1 −

φ^{(j)}(t)] dt < ε. Hence the result of problem 22 shows that the distribution functions of the random variables ξ_{n}^{(j)}, n = 1, 2, …, are tight, i.e. for all ε > 0 there exists a constant K = K(ε) > 0 such that the inequality P (|ξ_{n}^{(j)}| > K) < \frac{ε}{k} holds. Since this statement holds for all numbers j = 1, …, k, it implies that the distributions of the random vectors ξ̄_{n} = (ξ_{1}^{(n)}, …, …, ξ_{k}^{(n)}) are tight.

24.) It follows from the results of problems 21 and 23 that the sequence of distribution functions F_{n}(u_{1}, …, u_{k}), n = 1, 2, …, is relatively compact, i.e. an arbitrary subsequence of the sequence of the distribution functions F_{n} has a sub-subsequence convergent in distribution if the characteristic functions of the distribution functions F_{n} converge to a function continuous in the origin. (Moreover, it is enough to assume that this property holds for the restriction of the characteristic functions to the coordinate axes.) To see that under the conditions of the first part of problem 24 the distribution functions F_{n} converge in distribution it is enough to show that in this case all convergent subsequences of this sequence of distribution functions have the same limit. To justify this reduction of the problem let us choose a convergent subsequence F_{n_{l}} which converges to some distribution function F (u_{1}, …, u_{k}). If this distribution function F (·) were not the limit of the distribution functions F_{n}, then the distribution function F (u_{1}, …, u_{k}) would have a point of continuity u = (u_{1}, …, u_{k}) together with a constant ε > 0 and a sequence of indices n_{j}, j = 1, 2, …, such that |F_{n_{j}}(u_{1}, …, u_{k}) − F (u_{1}, …, u_{k})| > ε for all j = 1, 2, . . .. But this would mean that a convergent subsequence of the sequence of distribution functions F_{n_{j}}, j = 1, 2, …, would have a limit different of the distribution function F (u_{1}, …, u_{k}). The statement that all convergent subsequences of the distribution functions F_{n} have the same limit follows from the results of Theorem A and problem 19. Indeed, it follows from Theorem A that the characteristic function of the limit distribution function of a convergent subsequence of the distribution functions F_{n} is the limit of the characteristic functions of the distribution functions in this subsequence. The limit of these characteristic functions does not depend on which convergent subsequence we have considered. But by the result of problem 19 a distribution function is determined by its distribution function. Hence the condition that the characteristic functions of a sequence of distribution functions converge to a function continuous in the origin implies that these distribution functions converge in distribution to a distribution function, and besides the characteristic function of the limit distribution function equals the limit of the characteristic functions of the distribution functions we have considered. If a sequence of distribution functions F_{n}(u_{1}, …, u_{k}) converges in distribution to a distribution function F_{0}(u_{1}, …, u_{k}), then it follows from Theorem A that the characteristic functions φ_{n}(t_{1}, …, t_{k}) of these distribution functions converge to the characteristic function φ_{0}(t_{1}, …, t_{k}) of the distribution function F_{0} in all points (t_{1}, …, t_{k}) ∈ R^{k}. To complete the proof of the Fundamental Theorem we have still to show that this convergence is uniform in all compact subset of the Euclidean space R^{k}. To prove this statement let us observe that since the distribution functions F_{n} converge in distribution they are tight. Hence for all ε > 0 there exists a constant K = K(ε) such that a sequence of random vectors ξ_{n} = (ξ_{n}^{(1)}, …, ξ_{n}^{(k)}), n = 1, 2, …, with distribution functions F_{n} satisfy the inequality P (|ξ_{n}| > K) < \frac{ε}{3} for all indices n = 0, 1, 2, . . .. (In the further part of the proof ξ(ω), t ∈ R^{k}, u ∈ R^{k} denote points of the k-dimensional space, and (u, t), u ∈ R^{k}, t ∈ R^{k}, denotes the scalar product of the vectors u and t.) Let us choose a finite set of points T = {t^{(1)}, …, t^{(s)}} ⊂ K, s = s(K, δ), in a compact set K ⊂ R^{k} which is δ-dense in the set K, i.e. for all t ∈ K there is a point t^{(j)} ∈ T such that ρ(t, t^{(j)}) < δ. Then

|φ_{n}(t) − φ_{n}(t^{(j)})| = |Ee^{i(t,ξ_{n})} − e^{i(t^{(j)},ξ_{n})}|

≤ E |e^{i(t−t^{(j)},ξ_{n})} − 1| I (|ξ_{n}| ≤ K) + P (|ξ_{n}| > K) ≤ \frac{2ε}{3}

for all numbers n = 1, 2, . . .. Further we can choose a threshold index n_{0} = n_{0}(ε), such the inequality sup_{n≥n_{0}} sup_{t^{(j)}∈T} |φ_{n}(t^{(j)}) − φ_{0}(t^{(j)})| < \frac{ε}{3} holds. It follows from the

last inequalities that sup_{t∈K} |φ_{n}(t) − φ_{0}(t)| < ε if n ≥ n_{0}. This means that the

convergence φ_{n}(t) → φ_{0}(t) is uniform on all compact sets K. 25.) Let φ_{0}(t) denote the characteristic function of the uniform distribution in the in-

terval [−1, 1], i.e. φ_{0}(t) = ∫_{−1}^{1} \frac{1}{2}e^{itu} du = \frac{e^{it}−e^{−it}}{2it}. Let us define the characteristic functions φ_{n}(t) as the characteristic functions of the following discretizations μ_{n}, n = 1, 2, …, of the uniform distribution on the interval [−1, 1]: μ_{n} (\frac{k}{n}) = \frac{1}{2n+1}, −n ≤ k ≤ n. Then

φ_{n}(t) = \frac{1}{2n + 1} ∑_{k=−n}^{n} e^{ikt/n} = \frac{e^{i(n+1)t/n} − e^{i(−n+1)t/n}}{(2n + 1)(e^{it/n} − 1)} .

Simple calculation shows that φ_{n}(t) → φ_{0}(t) for all points t ∈ R^{1}, and the convergence is uniform in all finite intervals. On the other hand, the convergence is not uniform on the whole real line, since lim_{t→∞} φ_{0}(t) = 0, while φ_{n}(t) = 1 in the points of the form t = 2πkn, k = 0, ±1, ±2, . . .. (The background of this construction: We have approximated the characteristic function of a distribution function having a density function with the characteristic function of more and more closer discretization of this distribution function. These discretized approximations of the original distribution functions had lattice distributions. The characteristic functions of such approximating distributions converge to the characteristic function of the limit distribution by the Fundamental Theorem. Besides, the characteristic function of a distribution function with a density function tends to zero in the infinity by the Riemann lemma. On the other hand the characteristic function of a lattice valued distribution is a periodic function which has absolute value 1 in certain points.)

26.) An example for case a): Let the measures μ_{n} have uniform distribution in the

interval [−n, n]. Then φ_{n}(t) = \frac{1}{2n} ∫_{−n}^{n} e^{itu} du = \frac{e^{itn}−e^{−itn}}{2int}. Hence lim_{n→∞} φ_{n}(t) = 0

if t ≠ 0, and lim_{n→∞} φ_{n}(0) = 1.

An example for case b): Let μ_{2n}(\{n\}) = μ_{2n}(\{−n\}) = \frac{1}{2}, and μ_{2n+1} be the probability measure μ_{n} defined in case a). Then φ_{2n}(t) = \frac{1}{2}(e^{itn} + e^{−itn}), and it equals 1 in the points of the form \frac{2kπ}{n}. This means that in the points of the form t = φ (\frac{2kπ}{l} ) φ_{n_{k}}(t) = 0 for a certain subsequence n_{k}, and lim_{k→∞} φn̄_{k}(t) = 1 for a certain subsequence n̄_{k}.

27.) Let F (x) denote the distribution function of the random variable ξ. Then φ(t) = ∫ e^{itu}dF (u). By successive differentiation we get that \frac{d^{k}φ(t)}{dt^{k}} = i^{k} ∫ u^{k}e^{itu} dF (u),

in particular \frac{d^{k}φ(t)}{dt^{k}} |_{t=0} = i^{k} ∫ u^{k} dF (u) = i^{k}Eξ^{k} if the order of derivation and integration can be changed in the above calculation. The above calculation is legitim if the distribution F is concentrated in the interval [−K, K], because the integrand in the integral expressing the fraction \frac{φ(t+h)−φ(t)}{h} satisfies the relation \frac{e^{i(t+h)u}−e^{itu}}{h} = iue^{itu} + O(h), and for a fixed number t the order O(h) is uniform if u ∈ [−K, K]. If E|ξ| = ∫ |u| dF (u) < ∞, then to prove the statement of problem 27 for the first derivative we introduce the functions

G_{n}(t) = ∫_{−n}^{n} e^{iut} dF (u), H_{n}(t) = i ∫_{−n}^{n} ue^{iut} dF (u), n = 1, 2, …,

and G(t) = ∫_{−∞}^{∞} e^{iut} dF (u) és H(t) = i ∫_{−∞}^{∞} ue^{iut} dF (u). Then lim_{n→∞} G_{n}(t) = G(t),

and lim_{n→∞} \frac{dG_{n}(t)}{dt} = H(t) with H(t) = i ∫_{−∞}^{∞} ue^{iut} dF (u). We show with the help

of the above statements that the function G(t) is differentiable, and \frac{dG(t)}{dt} =

H(t). Indeed, G(t) = lim_{n→∞} G_{n}(t) = lim_{n→∞} [G_{n}(0) + ∫_{0}^{t} H_{n}(s) ds], hence the relations lim_{n→∞} G_{n}(0) = G(0), lim_{n→∞} H_{n}(s) = H(s), and the validity of the inequality |H_{n}(s)| ≤ E|ξ| for all numbers n and s together with Lebesgue dominated convergence theorem imply that G(t) = G(0) + ∫_{0}^{t} H(s) ds. Hence \frac{dG(t)}{dt} = H(t), and this is what we had to prove. The statement about the k-th derivative of the Fourier transform under the condition E|ξ|^{k} < ∞ can be proved similarly by induction with respect to the parameter k with the help of the identity \frac{dG^{k}(t)}{dt^{k}} = \frac{d}{dt}(\frac{dG^{k−1}(t)}{dt^{k−1}}). Only in this case we have to work with the functions G_{n}(t) = i^{k−1} ∫_{−∞}^{∞} u^{k−1}e^{iut} dF (u), H_{n}(t) = i^{k} ∫_{−n}^{n} u^{k}e^{iut} dF (u), n = 1, 2, …, and G(t) = i^{k−1} ∫_{−∞}^{∞} u^{k−1}e^{iut} dF (u), H(t) = i^{k} ∫_{−∞}^{∞} u^{k}e^{iut} dF (u). We prove the identity G(t) = G(0) + ∫_{0}^{t} H(s) ds also in this case. If Ee^{tξ} < ∞ with some number t > 0, then P (ξ > x) = P (e^{tξ} > e^{tx}) ≤ e^{−tx}Ee^{tξ} ≤ const. e^{−tx} for all numbers x ≥ 0. Similarly, P (ξ < −x) ≤ const. e^{−tx}, if Ee^{−tξ} < ∞. Hence P (|ξ| > x) ≤ const. e^{−tx} if Ee^{ux} < ∞ for |u| ≤ t. Conversely, if G(u) = P (|ξ| > x) ≤ const. e^{−αx}, then we get by partial differentiation that Ee^{t|ξ|} = ∫_{0}^{∞} e^{tu} dG(u) = [e^{tu}G(u)]_{0}^{∞} − ∫_{0}^{∞} te^{tu}G(u) du < ∞ in the case 0 < t < α. Hence Ee^{±tξ} ≤ 1 + Ee^{t|ξ|} < ∞. Finally, if P (|ξ| > x) ≤ const. e^{−αx}, then the function G(z) = ∫ e^{izx} dF (x) is analytic in the domain {z: |Im z| < α}, because in an arbitrary compact set in the interior of this domain the function G(z) can be represented as the uniform limit of analytic functions (finite sums approximating this integral). This function G(z) is the analytic continuation of the function φ(t) to the above domain.

28.) Let us first prove Riemann’s lemma. If g(u) = I([a, b]) is the indicator function of an interval [a, b], then ∫ e^{itu}g(u) du = \frac{e^{ibt}−e^{iat}}{it} → 0 if t → ∞ or t → −∞. This

relation also holds if g(u) = ∑_{j=1}^{k} c_{j}I([a_{j}, b_{j}]), ı.e. g(·) is a finite linear combination

of indicator functions of intervals. Such functions constitute an everywhere dense subset of the integrable functions in L_{1} norm, that is for all numbers ε > 0 and integrable function f(·) there exists a function g(·) or the above form such that ∫ |f(u)−g(u)| du < ε. This relation implies that |∫ e^{itu}f(u) du − ∫ e^{itu}g(u) du| < ε for all numbers t ∈ R^{1}. The Riemann lemma is a consequence of the above relations. If the function f(t) is k times differentiable, and the first k derivatives are integrable functions on the real line, then we get by successive partial differentiation that φ(t) = i^{k}t^{−k} ∫ e^{itu} \frac{df^{k}(s)}{ds^{k}} |_{s=u} du. This relation together with Riemann’s lemma

imply that in this case φ(t) = o(t^{−k}) if t → ±∞. If the function f(·) is analytic in a strip {z: Re z ∈ [−A, A]} around the real line and the function f(−ia + ·) is integrable for a < A, then we can write, because of the inequality |e^{i(u−ia)t}| < e^{−at} that

|φ(t)| = |∫_{−ia−∞}^{−ia+∞} e^{itu}f(u) du| ≤ e^{−at} ∫_{−∞}^{∞} |f(u − ia)| du ≤ const. e^{−at}

for t > 0. The case t < −0 can be handled similarly, only the integral has to be

replaced to the line [−∞ + ia, ∞ + ia] on the positive half-space.

29.) The relation |φ(t)| = 1 holds if and only if φ(t) = e^{ita}, that is Ee^{it(ξ−a)} = 1 with some real number a. This identity holds if and only if P (t(ξ − a) ∈ 2π{0, ±1, ±2, …, }) = 1. This means that |φ(t)| = 1 with some t ≠ 0 if the values of the random variable ξ are concentrated to a lattice \{\frac{2πk}{t} + a, k = 0, ±1, ±2, … \} of width \frac{2π}{t}. In particular, the relation |φ(t)| = 1 for all t ∈ R^{1} can hold if and only if the random variable takes a constant value with probability 1. If ξ is not a deterministic constant, and it is lattice distributed, then there is a largest h > 0 such that the distribution of ξ has period h. Indeed, in this case there are two numbers a and b, a ≠ b, such that P (ξ = a) > 0, and P (ξ = b) > 0. Then the distribution of ξ may have a period only with \frac{b−a}{k}, where k is a positive integer. Since the distribution of ξ has a period h > 0, the above statement holds.

To finish the solution of the problem it is enough to observe that the characteristic function is continuous on the real line, and the characteristic function φ(·) of a nonlattice valued random variable satisfies the inequality |φ(t)| < 1 if t ≠ 0. Hence sup_{A≤|t|≤B} |φ(t)| < 1 in this case.

30.) The characteristic function of the random variable ξ considered in this problem equals φ(t) = \frac{1}{2} (cos t + cos(\sqrt{2}t)). There exist pairs of integers (p_{n}, q_{n}), n = 1, 2, …, such that q_{n} → ∞, and |\sqrt{2}q_{n} − p_{n}| ≤ \frac{1}{q_{n}}. (Such pairs of integers (p_{n}, q_{n}) can be found as the nominator and denominator of the continued fraction of the number \sqrt{2}.) Choose t_{n} = 2πq_{n}. Then cos t_{n} = 1, and lim_{n→∞} cos(\sqrt{2}t_{n}) = 1. Hence t_{n} → ∞, and |φ(t_{n})| → 1, if n → ∞. On the other hand, φ(t) ≠ 1 if t ≠ 0.

If ξ is a random variable whose values are concentrated in a subset of the real line consisting of finitely or countably many points, then for all numbers ε > 0 there exists an integer s = s(ε) < ∞ and points u_{1}, …, u_{s} on the real line such that P (ξ ∈ \{u_{1}, …, u_{s}\}) ≥ 1 − \frac{ε}{3}. Further, by a classical (and simple) result of the number theory, by a result of Dirichlet, for all numbers N ≥ 1 there exists an integer 1 ≤ q_{N} ≤ N and a set of s integers p_{1}, …, p_{s} such that |q_{N}u_{k} − p_{k}| ≤ N^{−1/s}, for all indices k = 1, …, s. Hence, by choosing the number N sufficiently large we can achieve with the choice t = 2πq_{N} that Re e^{itu_{k}} ≥ 1 − \frac{ε}{3} for all indices 1 ≤ k ≤ s.

Then Re Ee^{itξ} ≥ ∑_{k=1}^{s} P (ξ = u_{k})(1 − \frac{ε}{3}) − \frac{ε}{3} ≥ 1 − ε. Since we can make this

construction for all ε > 0, there exists a sequence of positive integers q_{N}, N = 1, 2, …, such that the sequence t_{N} = 2πq_{N} satisfies the relation lim_{N→∞} φ(t_{N}) = 1.

Since the random variable ξ is not lattice distributed, hence sup_{2π≤t≤B} |φ_{n}(t)| ≤ q < 1

for any B > 2π with an appropriate constant q = q(B) < 1. This implies that the above constructed sequence of positive real numbers t_{N} satisfies the relation t_{N} → ∞ if N → ∞.

31.) Let us denote the distribution function of the random variable ξ by F (u), its characteristic function by φ(t) and put Re φ(t) = u(t). Then for all h ≥ 0

\frac{1 − u(h)}{h^{2}} = ∫_{−∞}^{∞} \frac{1 − cos hu}{h^{2}u^{2}}u^{2} dF (u),

and since lim_{h→0} \frac{1−cos hu}{h^{2}u^{2}} = \frac{1}{2}, \frac{1−cos hu}{h^{2}u^{2}} ≥ 0 for all numbers u ∈ R^{1} and h ∈ R^{1}, hence

lim inf_{h→0} ∫_{−∞}^{∞} \frac{1−cos hu}{h^{2}u^{2}}u^{2} dF (u) ≥ \frac{1}{2} ∫_{−∞}^{∞} u^{2} dF (u) = \frac{1}{2}Eξ^{2} by Fatou’s lemma. Hence,

to solve the problem it is enough to show that lim sup_{h→0} \frac{1−u(h)}{h^{2}} < ∞ if the function

φ(t) is twice differentiable in the origin. If the function φ(t) is twice differentiable in the origin, then the same relation holds for the function u(t). Then the derivative u′(t) = \frac{du(t)}{dt} exists in a small neighbourhood of the origin, and u′(0) = 0, as u(·) is an even function. Further, u(0) = 1, u(t) ≤ 1 for all numbers t ∈ R^{1}, hence

0 ≤ \frac{1−u(h)}{h^{2}} = \frac{u(0)−u(h)}{h^{2}} = −\frac{u′(ϑh)}{h} ≤ sup_{0≤s≤h} \frac{u′(0)−u′(s)}{s} < ∞ for small number h > 0,

if u(·) is twice differentiable in the origin, where 0 ≤ ϑ ≤ 1 is an appropriate number, and u′(·) denotes the derivative of the function u(·). Hence lim sup_{h→0} \frac{1−u(h)}{h^{2}} < ∞ in

this case.

By induction with respect to the parameter k we can see that if ξ is a random variable with distribution function F, and the 2k-th derivative of the characteristic function in the origin is finite, then the random variable ξ has finite 2k-th moment. Indeed, by the induction hypothesis there exists a distribution function F^{(k−1)}(du) = \frac{u^{(2k−2)}F (du)}{m_{2k−2}} where m_{2k−2} = ∫ u^{2k−2} dF (u). Further, the characteristic function of the distribution function F^{(k−1)} has finite second moment in the origin, since the characteristic function of this distribution function equals the 2k−2-th derivative of the characteristic function of the distribution function F multiplied by (−1)^{k−1}m_{2k−2}^{−1}, and the characteristic function of a random variable with distribution function F^{(k−1)}(u) has finite second derivative in the origin. Hence by the already proven part of this problem a random variable with distribution function F^{(k−1)} has finite second moment. This is equivalent to the statement that a random variable with distribution function F has finite 2k-th moment.

32.) The solution of this problem applies similar idea as the proof of problem 22. In problem 22 we deduced from some kind of continuity of the characteristic function of a distribution function in the origin some sort of estimate about the tail behaviour of the distribution function. In this problem we exploit that if we know more continuity about the distribution function in the neighbourhood of the origin, then we get sharper estimates about the tail behaviour of the distribution function.

Let F (x) denote the distribution function of the random variable ξ, and put u(t) = Re φ(t). The estimate |\frac{1−u(h)}{h} | = |u′(ϑh)| ≤ const. h^{α} holds with an appropriate constant 0 < ϑ < 1 under the conditions of the problem. Then the following analog of formula (2.6) holds.

const. h^{1+α} > ∫_{−h}^{h} \frac{1 − u(t)}{h} dt = ∫_{−1/2h}^{1/2h} (1 − \frac{sin hx}{hx}) dF (x)

+ ∫_{|x|> \frac{1}{2h}} (1 − \frac{sin hx}{hx}) dF (x) ≥ ∫_{|x|> \frac{1}{2h}} \frac{1}{2} dF (x)

= \frac{1}{2}P (|ξ| > \frac{1}{2h}) .

This implies that P (|ξ| > u|) ≤ const. u^{−1−α} for all numbers u > 0. Let us introduce the function G(u) = P (|ξ| > u|). Integration by parts yields that

E|ξ| = ∫_{0}^{∞} |u| dG(u) = [uG(u)]_{0}^{∞} − ∫ G(u) du < ∞.

If the characteristic function φ(t) is 2k + 1-times differentiable in a small neighbourhood of the origin, and the 2k + 1-th differential is a Lipschitz α function, α > 0, in this small neighbourhood, then let us introduce the distribution function F^{(k)}(du) = \frac{u^{2k}F (du)}{m_{k}}, where F (·) denotes the distribution function of the random variable ξ, and m_{k} = ∫ u^{2k} dF (u). The argument applied in the solution of the previous problem can be adapted to the present case. The already proven part of this problem can be applied to a random variable with distribution function F^{(k)}, and it yields that E|ξ|^{2k+1} < ∞.

33.) The relation (−1)^{k}Eξ^{2k} = \frac{d^{2k}φ(t)}{dt^{2k}} |_{t=0} = \frac{(2k)!}{2πi} ∮_{z=R} \frac{φ(z)}{z^{2k+1}} dz holds for all integers k by the result of problem 31 and the Cauchy integral formula if the circle with center in the origin and radius R is in the domain of analiticity of the function φ(z). Since sup_{z=R} |φ(z)| < ∞ the above relation implies that Eξ^{2k} ≤ (ak)^{2k} with some constant

a > 0, and P (|ξ| > x) ≤ (\frac{ak}{x} )^{2k} for all positive integers k ≥ 1. If x ≥ C_{0} with some number C_{0} > 0, then let us fix the constant k = [\frac{x}{2a}], where [u] is the greatest integer smaller than u. This implies that the inequality P (|ξ| > x) < const. e^{−αx} holds for all numbers x > 0 with an appropriate constant α > 0. Let us remark that we had to apply the above relatively complicated argument, because at the beginning of the proof we did not know that the analytic continuation of the characteristic function of a random variable ξ is always the function φ(z) = Ee^{izξ}.

34.) If the characteristic function of a random variable ξ is integrable, then the result about the inverse Fourier transform can be applied. It implies that the random variable ξ also has a density function f(x), and the identity f(x) = \frac{1}{2π} ∫ e^{−itx}φ(t) dt

holds. Also the formula \frac{d^{k}f(x)}{dx^{k}} = \frac{(−i)^{k}}{2π} ∫ t^{k}e^{−itx}φ(t) dt holds if the order of integration and differentiation can be changed in the inverse Fourier transform formula. In the solution of problem 27 we have proved that this change of order can be carried out if the functions |t|^{j}|φ(t)| are integrable for all indices 0 ≤ j ≤ k. Hence the statement of this problem holds if |φ(u)| < const. |u|^{−(k+1+ε)} with some number ε > 0. If |φ(u)| < const. e^{−α|u|} with some constant α > 0, then the function f(z) = \frac{1}{2π} ∫ e^{−itz}φ(t) dt is an analytic continuation of the density function f(x) of the random variable ξ.

35.) Let φ(t) = E^{itξ} denote the characteristic function of the random variable ξ_{1}. The normalized partial sum \frac{S_{n}}{\sqrt{n}} has the characteristic function φ^{n} (\frac{t}{\sqrt{n}}). Hence by the Fundamental Theorem about convergence in distribution and the result of problem 13a) it is enough to show that φ^{n} (\frac{t}{\sqrt{n}}) → e^{−t^{2}/2} for all numbers t ∈ R^{1} if n → ∞. On the other hand, a Taylor expansion of the function φ(t) around the point t = 0 and the result of problem 27 yield that φ(t) = 1 − \frac{t^{2}}{2} + o(t^{2}) if t = o(1),

and φ (\frac{t}{\sqrt{n}}) = 1− \frac{t^{2}}{n} +o (\frac{1}{n}) = e^{−t^{2}/2n+o(n^{−1})} if t = O(1). The last relation implies

that φ^{n} (\frac{t}{\sqrt{n}}) = e^{−t^{2}/2+o(1)} → e^{−t^{2}/2} for all fixed number t, if n → ∞.

36.) Let us introduce the function F (t) = e^{it}−(1 + \frac{it}{1!} + ⋯ + \frac{(it)^{k}}{k!} ), and let us consider

its derivatives F^{(j)}(t), j = 1, 2, …, k. Observe that F^{(j)}(0) = 0 of all 0 ≤ j ≤ k, and |F^{(k+1)}(t)| = |e^{ikt}| = 1 for all t ∈ R^{1}. We get by induction with respect to the parameter j that |F^{(j)}(t)| ≤ ∫_{0}^{t} |F^{(j+1)}(s)| ds ≤ ∫_{0}^{t} \frac{|s|^{k−j} ds}{(k−j)!} = \frac{|t|^{k+1−j}}{(k+1−j)!} for all

indices j = k + 1, k, …, 0. In particular, |F (t)| ≤ \frac{|t|^{k+1}}{(k+1)!}, and this is the statement of the problem.

37.a) By applying formula (11) with k = 1 we get that |e^{itξ} − 1 − itξ| ≤ \frac{t^{2}ξ^{2}}{2} . Taking the expected value of the expression at the left-hand side between the absolute value sign we get that |φ(t) − 1| ≤ \frac{t^{2}}{2} Eξ^{2}. If Eξ^{2} < ε with a sufficiently small number ε = ε(t) > 0, then |1 − φ(t)| ≤ \frac{1}{4}, and | log φ(t) + (1 − φ(t))| = | log (1 − (1 − φ(t))) − (1 − φ(t))| ≤ |1 − φ(t)|^{2} ≤ t^{4} (Eξ^{2})^{2}.

b.) The sequence of random variables S_{k} converge in distribution to a normal random variable with expected value m and variance σ^{2} if and only if

lim_{k→∞} ∏_{j=1}^{n_{k}} φ_{k,j}(t) = e^{−σ^{2}t^{2}/2+imt} for all t ∈ R^{1}.  (2.7)

Let us take the logarithm in the relation (2.7). We claim that formula (2.7) is equivalent to formula (2.7′) formulated below.

lim_{k→∞} ∑_{j=1}^{n_{k}} log φ_{k,j}(t) = −\frac{σ^{2}t^{2}}{2} + imt for all t ∈ R^{1}.  (2.7′)

The equivalence of relations (2.7) and (2.7′) is less obvious than it may seem at first sight. Some difficulty arises because in the space of complex numbers where we have to work the equation e^{z_{1}} = e^{z_{2}} only implies that z_{1} = z_{2} + i2kπ with some integer k, but the numbers z_{1} and z_{2} may be different. Hence although the implication (2.7′) ⇒ (2.7) needed in the proof of the central limit theorem is straightforward, the proof of the implication (2.7) ⇒ (2.7′) needed in the proof of the converse of the central limit theorem requires a more careful argument.

Before the proof of the implication (2.7) ⇒ (2.7′) let us remark that because of the uniform smallness condition for a fixed value t ∈ R^{1} the characteristic function φ_{k,j}(t) we consider is in a small neighbourhood of the number 1 for all 1 ≤ j ≤ n_{k} if the index k is sufficiently large. Hence for a sufficiently large k ≥ k_{0}(t) > 0 there is a version log φ_{k,j}(t) of the functions φ_{k,j}(t) which is in a small neighbourhood of the origin, say | log φ_{k,j}(t)| < \frac{1}{2}. We take this version of the logarithm in formula (2.7′).

To prove the implication (2.7) ⇒ (2.7′) let us first make the following observation. We have seen in the proof of problem 24 (in the proof of the Fundamental Theorem about convergence of distribution functions) that the convergence in formula (2.7) is uniform in all finite intervals. Besides, the right-hand side of formula (2.7) is separated both from zero and infinity in a finite interval. Hence we get by taking logarithm in formula (2.7) that for all ε > 0 and T > 0 there exists a k_{0} = k_{0}(ε, T) and δ = δ(ε, k, T) such that

|∑_{j=1}^{n_{k}} log φ_{k,j}(t) − (−\frac{σ^{2}t^{2}}{2} + imt) + i2πl_{k}(t)| < ε if |t| ≤ T and k ≥ k_{0}

(2.7′′)

with some integer l_{k}(t) which may depend both on k and t. We have to show that l_{k}(t) ≡ 0 in formula (2.7′′). First we prove the weaker statement that l_{k}(t) = l_{k}, i.e. this constant in formula (2.7′′) does not depend on t. Indeed, otherwise for all δ = δ_{k} > 0 a pair of constant −T ≤ s, t ≤ T, |t − s| < δ could be found such that l_{k}(t) ≠ l_{k}(s), i.e. |2πi(l_{k}(t) − l_{k}(s)| ≥ 2π. But

this is not possible, because the function g_{k}(t) = ∑_{j=1}^{n_{k}} log φ_{k,j}(t) is uniformly

continuous in the interval [−T, T]. Hence fixing a small ε > 0 we can write |g_{k}(t) − g_{k}(s)| < ε for sufficiently small δ = δ(k, ε, T) > 0. Besides, also the inequality |(−\frac{σ^{2}t^{2}}{2} + imt) − (−\frac{σ^{2}s^{2}}{2} + ims)| < ε holds if δ > 0 is sufficiently

small. Let ε < \frac{1}{3}. The above relations together with formula (2.7′′) would contradict to the assumption |2πi(l_{k}(t) − l_{k}(s)| ≥ 2π. Hence l_{k}(t) = l_{k}. Finally, it is easy to see that l_{k} = l_{k}(0) = 1. Hence relation (2.7′′) holds for all ε > 0 with l_{k}(t) ≡ 0, and this implies relation (2.7′).

Because of the uniform smallness condition of problem 37 and the already proved part a) of this problem we can write for k ≥ k_{0}(t)

|∑_{j=1}^{n_{k}} log φ_{k,j}(t) − ∑_{j=1}^{n_{k}}(φ_{k,j}(t) − 1)| ≤ t^{4} ∑_{j=1}^{n_{k}} (Eξ_{k,j}^{2})^{2} ≤ const. t^{4} max_{1≤j≤n_{k}} Eξ_{k,j}^{2},

since lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{j,k}^{2} = 1. Because of this relation and the uniform smallness condition

lim_{k→∞} |∑_{j=1}^{n_{k}} log φ_{k,j}(t) − ∑_{j=1}^{n_{k}}(φ_{k,j}(t) − 1)| = 0.

These relations also imply part b) of problem 37. 38.) Let us fix a number ε > 0. Then

Eξ_{k,j}^{2} = Eξ_{k,j}^{2}I({(|ξ_{k,j}| < ε}) + Eξ_{k,j}^{2}I({|ξ_{k,j}| ≥ ε}) ≤ ε^{2} + ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2}I({|ξ_{k,j}| ≥ ε}),

hence by the Lindeberg condition lim sup_{k→∞} sup_{1≤j≤n_{k}} Eξ_{k,j}^{2} ≤ ε^{2}. Since this formula holds for all numbers ε > 0, it implies the uniform smallness property. By formula (12) (with the choice m = 0 and σ = 1) to prove the central limit theorem it is enough to show that under the conditions of this problem

lim_{k→∞} ∑_{j=1}^{n_{k}} (φ_{k,j}(t) − 1) = lim_{k→∞} ∑_{k=1}^{n_{k}} E (e^{itξ_{k,j}} − 1 − itξ_{k,j}) → −\frac{t^{2}}{2} ,

or since lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2} = 1 it is enough to show that

lim_{k→∞} ∑_{j=1}^{n_{k}} E (e^{itξ_{k,j}} − 1 − itξ_{k,j} + \frac{t^{2}}{2} ξ_{k,j}^{2}) → 0.

By applying formula (11) for k = 2 if |tx| ≤ ε and for k = 1 if |tx| ≥ ε together with the Lindeberg condition we get that

|∑_{j=1}^{n_{k}} E (e^{itξ_{k,j}} − 1 − itξ_{k,j} + \frac{t^{2}}{2} ξ_{k,j}^{2}) I(\{|ξ_{k,j}| ≤ ε\})| ≤ ∑_{j=1}^{n_{k}} E \frac{|tξ_{k,j}|^{3}}{6} I(\{|ξ_{k,j}| ≤ ε\})

≤ ε|t|^{3} ∑_{j=1}^{n_{k}} E \frac{ξ_{k,j}^{2}}{6} ≤ const. ε,

and

lim_{k→∞} |∑_{j=1}^{n_{k}} E (e^{itξ_{k,j}} − 1 − itξ_{k,j} + \frac{t^{2}}{2} ξ_{k,j}^{2}))I(\{|ξ_{k,j}| > ε\})|

≤ lim_{k→∞} ∑_{j=1}^{n_{k}} Et^{2}ξ_{k,j}^{2}I({|ξ_{k,j}| > ε}) = 0.

Since these relations hold for all ε > 0, it implies formula (12) with the choice m = 0 and σ^{2} = 1. In such a way we have solved problem 38.

39.) If the conditions of the problem are satisfied, then lim_{k→∞} ∑_{j=1}^{n_{k}} Re (φ_{k,j}(t) − 1) = −\frac{t^{2}}{2} .

Further, since the sum of the random variables in the k-th row is almost 1 for large index k, hence

lim_{k→∞} ∑_{j=1}^{n_{k}} E (cos(tξ_{k,j}) − 1 + \frac{t^{2}ξ_{k,j}^{2}}{2} ) = 0 for all numbers t ∈ R^{1}.

Let us observe that cos u − 1 + \frac{u^{2}}{2} ≥ 0 for all numbers u ∈ R^{1}, since we have for the function F (u) = cos u − 1 + \frac{u^{2}}{2} ≥ 0, F (0) = 0, F′(0) = 0 and F′′(u) = 1 − cos u ≥ 0 for all numbers u ∈∈ R^{1}. Besides, cos u − 1 + \frac{u^{2}}{2} ≥ \frac{u^{2}}{4} if |u| > 3. The above inequalities imply that

lim_{k→∞} ∑_{j=1}^{n_{k}} \frac{t^{2}}{4} Eξ_{k,j}^{2}I (\{|ξ_{k,j}| ≥ \frac{3}{t}\}) = 0.

We get the solution of problem 39 from this relation with the choice t = \frac{3}{ε}. 40.) By the Schwarz inequality

(Eξ_{k,j}I(|ξ_{k,j}| ≤ ε)^{2} = (Eξ_{k,j}I(|ξ_{k,j}| > ε)^{2} ≤ Eξ_{k,j}^{2}I(|ξ_{k,j}| > ε).

Hence the Lindeberg condition implies that

lim_{k→∞} ∑_{j=1}^{n_{k}} (Eξ_{k,j}I(|ξ_{k,j}| ≤ ε))^{2} ≤ lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2}I(|ξ_{k,j}| > ε) = 0,

and

lim_{k→∞} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2}I(|ξ_{k,j}| ≤ ε) = 1.

These two relations imply the first statement of the problem. To prove the second statement let us first observe that E(ξ̄_{k,j} − ξ_{k,j}) = Eξ̄_{k,j} − Eξ_{k,j} = 0, 1 ≤ j ≤ n_{k}. Hence the Chebishev inequality and Lindeberg condition imply that for all ε > 0

P (|S_{k} − S̄_{k}| > ε) ≤ \frac{Var (S_{k} − S̄_{k})}{ε^{2}} = \frac{1}{ε^{2}} ∑_{j=1}^{n_{k}} Var (ξ_{k,j} − ξ̄_{k,j})

≤ \frac{1}{ε^{2}} ∑_{j=1}^{n_{k}} Eξ_{k,j}^{2}I(|ξ_{k,j}| > 0) → 0.

Since this relation holds for all ε > 0 it implies the second statement of the problem. 41.) By the Hölder inequality

∑_{k=1}^{n} Eξ_{k}^{2}I(|ξ_{k}| > εs_{n}) ≤ ∑_{k=1}^{n} (E|ξ_{k}|^{(2+α)})^{(2/α+2)} P (|ξ_{k}| > εs_{n})^{α/(2+α)}

≤ (∑_{k=1}^{n} E|ξ_{k}|^{(2+α)})^{(2/α+2)} (∑_{k=1}^{n} P (|ξ_{k}| > εs_{n}))^{α/(2+α)} .

On the other hand, by the Chebishev inequality ∑_{k=1}^{n} P (|ξ_{k}| > εs_{n}) ≤ ∑_{k=1}^{n} \frac{Eξ_{k}^{2}}{ε^{2}s_{n}^{2}} = \frac{1}{ε^{2}} .

Hence it the conditions of part a) of problem 41 hold, then

lim_{n→∞} \frac{1}{s_{n}^{2}} ∑_{k=1}^{n} Eξ_{k}^{2}I(|ξ_{k}| > εs_{n}) = 0,

i.e. these conditions imply the Lindeberg condition.

In the case considered at the end of part a.) s_{n}^{2} ≥ const. n, and ∑_{k=1}^{n} E|ξ_{k}|^{2+α} =

o (n^{(α+2)/2)}) if n → ∞, hence in this case the condition formulated in part a) of problem 41 is satisfied. If ξ_{1}, ξ_{2}, …, is a sequence of independent and identically distributed random variables, Eξ_{1} = 0, 0 < Eξ_{1}^{2} < ∞, then

\frac{1}{s_{n}^{2}} ∑_{k=1}^{n} Eξ_{k}^{2}I(|ξ_{k}| > εs_{n}) = \frac{1}{Eξ_{1}^{2}} Eξ_{1}^{2}I (|ξ_{1}| > ε\sqrt{nEξ_{1}^{2}}) → 0

if n → ∞. This means that the Lindeberg condition holds also in this case. 42.) If the point x is a point of continuity of the limit distribution function F (·), then

for all ε > 0 there exists a δ > 0 such that F (x) − \frac{ε}{2} < F (x − δ) < F (x) < F (x + δ) < F (x + δ) + \frac{ε}{2}. Since the monotone increasing function F (·) has at most countably infinite points of discontinuity, hence we may assume without violating the generality that we choose the point δ > 0 in such a way that the points x ± δ are also points of continuity of the function F (·). Then there exists an index n_{0} = n_{0}(δ, ε) such that P (S_{n} < x + δ) < F (x + δ) + \frac{ε}{4}, P (S_{n} > x − δ) < 1 − F (x − δ) + \frac{ε}{4}, and P (|T_{n}| ≥ δ) < \frac{ε}{4} if n ≥ n_{0}. Then P (S_{n} + T_{n} < x) ≤ P (S_{n} < x + δ) + P (|T_{n}| > δ) < F (x + δ) + \frac{ε}{2} < F (x) + ε if n ≥ n_{0}(ε, δ). We get in a similar way that P (S_{n} + T_{n} > x) < 1 − F (x) + ε if n ≥ n_{0}(ε, δ). Since the above statements hold for all ε > 0, they imply the statement of the problem.

43.) Let the independent random variables ξ_{n}, n = 1, 2, …, have the following distribution: P (ξ_{n} = n) = P (ξ_{n} = −n) = \frac{1}{4n^{2}} , P (ξ_{n} = 2) = P (ξ_{n} = −2) = \frac{1}{4}, and P (ξ_{n} = 0) = \frac{1}{2} − \frac{1}{2n^{2}} , n = 1, 2, . . .. Then Eξ_{n} = 0, Eξ_{n}^{2} = 1. Put X_{k} = ξ_{k}I(|ξ_{k}| ≤ 2), Y_{n} = ξ_{k}I(|ξ_{k}| > 2) for all k = 1, 2, . . .. Consider the par-

tial sums S_{n} = ∑_{k=n}^{n} X_{k} and T_{n} = ∑_{k=1}^{n} Y_{k}, n = 1, 2, . . .. Then the normalized

partial sums \sqrt{\frac{2}{n}}S_{n} converge in distribution to the standard normal distribution. Indeed, the partial sums of the random variables X_{k}, k = 1, 2, …, satisfy the conditions of the central limit theorem, and EX_{k}^{2} = \frac{1}{2}, k = 1, 2, . . .. On the other

hand, the expressions \sqrt{\frac{2}{n}}T_{n} converge stochastically to zero if n → ∞. Indeed,

∑_{k=1}^{∞} P (Y_{k} ≠ 0) < ∞, hence with probability 1 only finitely many terms Y_{k}(ω) do

not equal zero, and ∑_{k=1}^{∞} |Y_{k}(ω)| ≤ K(ω). Since \sqrt{\frac{2}{n}} ∑_{k=1}^{n} ξ_{k} = \sqrt{\frac{2}{n}}S_{n} + \sqrt{\frac{2}{n}}T_{n}, the

above calculation and the result of problem 42 imply that the above construction yields an example for the statement of part a) of problem 43. Let us make some slight modifications in the construction of the above random variables ξ_{n}. Let us put similarly to the previous construction P (ξ_{n} = 2) = P (ξ_{n} = −2) = \frac{1}{4} and P (ξ_{n} = n) = \frac{1}{4n^{2}} . Let us define further

P (ξ_{n} = \frac{1}{\sqrt{n}}) = \frac{1}{2} − \frac{1}{2n^{2}} , and P (ξ_{n} = −n − 2n^{3/2} (1 − \frac{1}{n^{2}} )) = \frac{1}{4n^{2}} ,

n = 1, 2, . . .. Then Eξ_{n} = 0, n = 1, 2, . . .. By applying the truncation technique of part a) and carrying out a natural modification of the calculation following it we get that these random variable ξ_{n}, n = 1, 2, …, yield an example for part b) of problem 43.

44.) Let us choose an arbitrary number L such that ∫ u^{2}F_{0}( du) > L. It is enough to show that lim inf_{n→∞} ∫ u^{2}F_{n}( du) ≥ L. There exists such a bounded and continu-

ous function g(u) = g_{L}(u) for which g(u) ≤ u^{2}, and ∫ g(u)F_{0}( du) ≥ L. Indeed, the function g(u) = g_{L}(u) = min(u^{2}, K) satisfies this property if we choose a sufficiently large constant K = K(L) > 0. Then the characterization of the convergence in distribution given in Theorem A implies that lim inf_{n→∞} ∫ u^{2}F_{n}( du) ≥

lim_{n→∞} ∫ g(u)F_{n}( du) = ∫ g(u)F_{0}( du) ≥ L.

45.) The distribution of the random vector Z = (Z_{1}, …, Z_{m}) is determined by its characteristic function. (See the result of problem 19.) On the other hand, the characteristic function E^{i(t_{1}Z_{1}+···+t_{m}Z_{m})} of the random vector (Z_{1}, …, Z_{m}) in the point (t_{1}, …, t_{m}) agrees with the characteristic function of the random variable Z(t_{1}, …, t_{m}) in the point 1. Hence the characteristic function and distribution function of the random vector Z is determined by the distribution of the above considered one dimensional distributions.

46.) First we show with the help of the Fundamental Theorem about the convergence of distribution functions that the random vectors Z_{n} = (Z_{1,n}, …, Z_{m,n}), n = 1, 2, …, converge in distribution to some m-dimensional distribution as n → ∞ if the onedimensional random variables Z_{n} = Z_{n}(a_{1}, …, a_{m}), n = 1, 2, …, converge in distribution for all real numbers a_{1}, …, a_{m} as n → ∞. Indeed, if the one-dimensional random variables interested in this problem convergence in distribution, then the characteristic functions of the random vectors Z_{n}, n = 1, 2, …, converge to a function φ(t_{1}, …, t_{m}) in all points (t_{1}, …, t_{m}) ∈ R^{m}, and the restriction of this limit function to the coordinates axes is continuous. Hence by the Fundamental Theorem the random vectors Z_{n}, n = 1, 2, …, also converge in distribution, and the characteristic function of the limit distribution is the above limit function. The Fundamental Theorem also implies that the convergence of the random vectors Z_{n} implies the convergence of the random variables of the random variables Z_{n}(a_{1}, …, a_{m}) in distribution. If the random vectors Z_{n} converge in distribution, then the limit distribution is determined by its characteristic function which is the limit of the characteristic functions of these random variables. Similarly, the characteristic function of the limit of the one-dimensional random variables we have considered equals the limit of the characteristic function of these random variables. These facts imply the characterization of the limit distribution μ given in this problem together with the statement that the above characterization determines the limit distribution in a unique way.

47.) Let Σ = (D_{j,k}), 1 ≤ j, k ≤ m, denote the covariance matrix of an m-dimensional random vector (Z_{1}, …, Z_{m}), i.e. let D_{j,k} = E(Z_{j} − EZ_{j})(Z_{k} − EZ_{k}), 1 ≤ j, k ≤ m.

Then the matrix Σ is symmetrical. Besides, xΣx^{∗} = ∑_{j=1}^{m} ∑_{k=1}^{m} x_{j}E(Z_{j} − EZ_{j})(Z_{k} −

EZ_{k})x_{k} = E ( ∑_{j=1}^{k} x_{j}(Z_{j} − EZ_{j}))^{2} ≥ 0 for all vectors x = (x_{1}, …, x_{m}) ∈ R^{m}, and

this means that the matrix Σ is positive semi-definite. On the other hand, if Σ is an arbitrary m × m positive semi-definite matrix, then the results of linear algebra imply that there exists a matrix B such that Σ = B^{∗}B. (The matrix B satisfying this relation is not determined in a unique way. A possible construction of a matrix B satisfying the above relation can be given in the following way: It is known from linear algebra that a symmetric matrix Σ can be represented in the form Σ = UΛU^{∗} where the matrix U is unitary and the matrix Λ is diagonal with some elements λ_{1}, …, λ_{m} in the diagonal. The matrix Σ is positive semidefinite if and only if all elements λ_{j}, 1 ≤ j ≤ m, in the above representation are non-negative. If Σ = UΛU^{∗} is a positive semi-definite matrix, then let us define the symmetric matrix B = U\sqrt{Λ}U^{∗}, where \sqrt{Λ} is the diagonal matrix with elements \sqrt{λ_{j}}, j = 1, …, m, in the diagonal. Then Σ = B^{2} = B^{∗}B.) Let Σ = (D_{j,k}), 1 ≤ j, k ≤ m, be an arbitrary m × m positive semi-definite matrix, and M = (M_{1}, …, M_{m}) ∈ R^{m} a vector in the Euclidean space R^{m}. Let ξ = (ξ_{1}, …, ξ_{m}) be an m-dimensional random variable with standard normal distribution, B = (b_{j,k}), 1 ≤ j, k ≤ m, an m × m matrix such that B^{∗}B = Σ. Let us define the m-dimensional random vector η = (η_{1}, …, η_{m}) = ξB + M. Then η has normal distribution, and we claim that it has expected value M and covariance matrix B^{∗}B = Σ. This implies that for all vectors M ∈ R^{m} and m × m positive semi-definite matrices Σ there exists an m-dimensional normally distributed random vector with expectation zero and covariance matrix Σ. Indeed, Eη = (Eη_{1}, …, Eη_{m}) = (M_{1}, …, M_{m}) = M, and the elements of the covariance matrix of η can be calculated in the following way.

E(η_{j} − Eη_{j})(η_{k} − Eη_{k}) = E (∑_{l=1}^{m} b_{j,l}ξ_{l}) (∑_{p=1}^{m} b_{k,p}ξ_{k}) = ∑_{l=1}^{p} b_{j,l}b_{k,l} = D_{j,k}

for all indices 1 ≤ j, k ≤ m, because Eξ_{l}ξ_{p} = 0, if l ≠ p, Eξ_{l}^{2} = 1. The last identity means the identity B^{∗}B = Σ in coordinate form. Let us consider an m-dimensional random vector η = (η_{1}, …, η_{m}) = ξB + M with normal distribution function, where the random vector ξ = (ξ_{1}, …, ξ_{m}) has standard normal distribution, B is an m × m matrix, M = (M_{1}, …, M_{m}) ∈ R^{m}. Put B^{∗}B = Σ = (D_{j,k}), 1 ≤ j, k ≤ m. Let us calculate the characteristic function φ(t_{1}, …, t_{m}) = Ee^{i(t_{1}η_{1}+···+t_{m}η_{m})} η of the random vector η. To calculate it let us introduce the random variable ζ = t_{1}η_{1}+⋯+t_{m}η_{m}. Then ζ is a normally distributed

random variable with expected value M̄ = M̄(t_{1}, …, t_{m}) = ∑_{k=1}^{m} t_{k}M_{k} = (M, t) and

variance σ^{2} = ∑_{j=1}^{m} ∑_{k=1}^{m} t_{j}t_{k}Eη_{j}η_{k} = ∑_{j=1}^{m} ∑_{k=1}^{m} t_{j}t_{k}D_{j,k} = tΣt^{∗}, where t = (t_{1}, …, t_{m}).

Hence the characteristic function of the random variable ζ with normal distribution equals ψ(u) = Ee^{iuζ} = e^{−u^{2}tΣt/2+i(M,t)u}. This implies that φ(t_{1}, …, t_{m}) = Ee^{i(t_{1}η_{1}+···+t_{m}η_{m})} = ψ(1) = e^{−tΣt^{∗}/2+i(M,t)}, that is relation (13) holds. It follows from formula (13) that if η is an m-dimensional random variable with normal distribution, then the characteristic function and as a consequence, the distribution function of this random vector is determined by its expectation and covariance matrix. Let us remark that there can be given two different m × m matrices B_{1} and B_{2} such that B_{1}^{∗}B_{1} = B_{2}^{∗}B_{2}. Let ξ be an m-dimensional random vector with standard normal distribution, B_{1} and B_{2} two m×m matrices such that B_{1}^{∗}B_{1} = B_{2}^{∗}B_{2}, an M ∈ R^{m} an arbitrary vector. Let us define the random vectors η_{1} = ξB_{1} + M and η_{2} = ξB_{2} + M. Then the expectation and covariance matrix, hence the distribution function of the random vectors η_{1} and η_{2} agree, although this statement is not self-evident because of the relation B_{1} ≠ B_{2}.

48.) We get similarly to the proof of formula (13) in problem 47., that the characteristic function of the random vector η = (η_{1}, …, η_{l}) equals Ee^{i(t,η)} = e^{−tΣt^{∗}/2+i(M,t)}, where t = (t_{1}, …, t_{l}), and Σ = B^{∗}B. It follows from this formula that η is a normally distributed random vector with covariance matrix Σ and expectation vector M. Given a normally distributed random vector η of dimension m, let us write it in the form η = ξB + M, where ξ is a random vector of dimension m with standard normal distribution. (It can be proved that such a representation is always possible. Actually it would be enough for us such a representation of a random vector with the same distribution as η. The possibility of such a representation follows from the definition of the normally distributed random vectors.) If we omit some of the coordinates of the random vector η and we preserve only l coordinates, then the random vector η′ we obtain after this delition of coordinates can be presented in the following way. Let us omit those rows of the vector B which have the same indices as the elements of η we omitted. Similarly let us omit the coordinates of the vector M with the same indices as the coordines omitted from η. Let us denote the matrix and vector obtained in such a way by M′ and B′. Then we have η′ = ξB′ + M′, hence η′ is normally distributed by the already proved part of the problem.

49.) I shall give two different solutions of the problem, because, this may be instructive.

First soulution: We can write up the characteristic function of the random vector η, as Ee^{i(t,η)} = e^{−tΣt^{∗}/2+i(M,t)} where t = (t_{1}, …, t_{m}), and M = (M_{1}, …, M_{m}) is the expected value of η, with the introduction of some notations in the following way. Let t_{j} denote the restriction of the vector t, and let M_{j} denote the restriction of vector M to the coordinates p ∈ L_{j}, 1 ≤ j ≤ k. Let us define similarly the matrix Σ_{j} as the restriction of the matrix Σ to the coordinates σ_{p,q}, p ∈ L_{j} and q ∈ L_{j},

1 ≤ j ≤ k. With such a notation Ee^{i(t,η)} = ∏_{j=1}^{k} e^{−t_{j}Σ_{j}t_{j}^{∗}/2+i(M_{j},t_{j})}. (We exploited

the properties of the matix Σ at this point.) Let η′_{1}, …, η′_{k} independent, normally distributed random vectors with covariance matrix Σ_{j} and expected value M_{j}, 1 ≤ j ≤ k. The characteristic function of Eη′_{j} equals Ee^{i(t_{j},η′_{j})} = e^{−t_{j}Σ_{j}t_{j}^{∗}/2+i(M_{j},t_{j})} for all indices 1 ≤ j ≤ k. Hence the characteristic function of the random vectors η′ = (η′_{1}, …, η′_{k}) and η agree. Hence the distribution of η and η′ also agree, and as a consequence the random vectors η̄_{j}, 1 ≤ j ≤ k, are independent of each other, similarly to the random vectors η′_{j}.

Second solution: Let us apply the notations introduced in the previous solution. The random vector η′ defined there is normally distributed with covariance matrix Σ and expected value M. As the distribution of a normal random vector is determined by its covariance matrix and expected value, hence the distribution of η és η′ agree. Therefore the random vectors η′_{1}, …, η′_{k} are independent, similarly to the random vectors η̄_{1}, …, η̄_{k}.

50.) Because of the result of problem 46 it is enough to show that for all vectors a = (a_{1}, …, a_{m}) ∈ R^{m} the normalized partial sums \frac{1}{A_{n}} S_{n} = \frac{1}{A_{n}} S_{n}(a_{1}, …, a_{m}) =

\frac{1}{A_{n}} ∑_{p=1}^{m} a_{p}S_{p,n}, n = 1, 2, … converge in distribution to the normal distribution

with expectation zero and variance σ^{2} = aΣa^{∗} if n → ∞. Let us observe that

\frac{1}{A_{n}} S_{n} = \frac{1}{A_{n}} ∑_{k=1}^{n} η_{k}, n = 1, 2, …, where η_{k} = ∑_{p=1}^{m} a_{p}ξ_{p,k}, k = 1, 2, . . .. Besides, the

random variables η_{k}, k = 1, 2, … are independent, Eη_{k} = 0, Eη_{k}^{2} = aΣ_{k}a^{∗}.

Let us consider separately the cases aΣa^{∗} = 0 and aΣa^{∗} > 0. If aΣa^{∗} = 0, then

\frac{1}{A_{n}^{2}} ES_{n}^{2} = \frac{1}{A_{k}^{2}} ∑_{j=1}^{k} aΣ_{j}a^{∗} → 0 if k → 0, hence \frac{S_{n}}{A_{n}} converges to zero stochastically if

n → ∞. That is, in this case the random variables \frac{1}{A_{n}} S_{n} converge in distribution to the (degenerated) normal variable with expectation zero and variance zero. In the case aΣa^{∗} > 0 let us define the triangular array η_{k,n} = \frac{η_{k}}{A_{n}\sqrt{aΣa^{∗}}}, 1 ≤ k ≤ n,

n = 1, 2, . . .. Then ∑_{k=1}^{n} Eη_{k,n}^{2} = \frac{1}{A_{n}^{2}} ∑_{k=1}^{n} \frac{aΣ_{k}a^{∗}}{aΣa^{∗}} → 1 if n → ∞. Hence in the case

aΣa^{∗} > 0 the convergence of the normalized partial sums \frac{1}{A_{n}} S_{n}, n = 1, 2, …, to the normal distribution with expectation zero and variance aΣa^{∗} follows from the central limit theorem for triangular arrays formulated in problem 38 if we show that the above defined triangular array η_{k,n}, 1 ≤ k ≤ n, n = 1, 2, …, satisfies the Lindeberg condition. This implies the statement of the problem. To prove the Lindeberg condition first we show the following inequality (2.8). Here we shall apply the notation K = max_{1≤p≤m} |a_{p}|.

E(η_{k,n}^{2}I(|η_{k,n}| > ε) ≤ \frac{K^{2}m^{2}}{A_{n}^{2}aΣa^{∗}} ∑_{p=1}^{m} Eξ_{p,k}^{2}I(|ξ_{p,k}| > ε̄A_{n})  (2.8)

for all indices 1 ≤ k ≤ n and n = 1, 2, …, where ε̄ = ε̄(k) = _{\frac{ε}{m sup_{1≤p≤m} |a_{p}|}} · \frac{1}{\sqrt{aΣa^{∗}}}. By

summing up these inequalities for all indices 1 ≤ k ≤ n, and applying formula (14) for all 1 ≤ p ≤ m, we get that the triangular array η_{k,n}, 1 ≤ k ≤ n, n = 1, 2, … satisfies the Lindeberg condition. To prove formula (2.8) let us introduce the random index p̄(k) = p̄(k, ω) which is the (smallest) number p such that |ξ_{p̄(k),k}(ω)| = max_{1≤p≤m} |ξ_{p,k}(ω)|. Let us observe

that

{ω: |η_{k,n}(ω)| > ε} ⊂ ⋃_{p=1}^{M}{ω: |ξ_{p,k}(ω)| > ε̄A_{n}},

hence I(|η_{k}(ω)| > ε) ≤ I(|ξ_{p̄(k,ω),k}(ω)| > ε̄A_{n}). Besides, |η_{k}(ω)|^{2} ≤ \frac{m^{2}K^{2}}{A_{n}^{2}aΣa^{∗}} ξ_{p̄(k),k}^{2}. This implies that

η_{k}^{2}I(|η_{k}| > ε) ≤ \frac{m^{2}K^{2}}{A_{n}^{2}aΣa^{∗}} ξ_{p̄(k),k}^{2}I(|ξ_{p̄(k,ω),k}(ω)| > ε̄A_{n})

≤ \frac{m^{2}K^{2}}{A_{n}^{2}aΣa^{∗}} ∑_{p=1}^{m} ξ_{p,k}^{2}I(|ξ_{p,k}| > ε̄A_{n}).

By taking expection in this inequality we get formula (2.8). Let us finally observe that if ξ_{k} = (ξ_{1,k}, …, ξ_{m,k}), k = 1, 2, …, is a sequence of independent and identically distributed m-dimensional random vectors with expectation zero and finite covariance matrix Σ, then this sequence of random vectors satisfies relation (14) with the choice A_{n}^{2} = n. This statement was proved in part b) of problem 41 for those coordinates p for which Eξ_{p,1}^{2} > 0. For those coordinates p for which Eξ_{p,1}^{2} = 0, ξ_{p,1} ≡ 0. Hence these coordinates can be omitted.

51.) Let us consider first those numbers p, 1 ≤ p ≤ m for which the p-th element of the diagonal of the (semi-definite) matrix Σ D_{p,p} satisfies the inequality D_{p,p} > 0. Let us define the triangular array η_{k,j} = η_{k,j}(p) = \frac{ξ_{p,j}}{A_{k}D_{p,p}}, 1 ≤ j ≤ k, k = 1, 2, . . .. Then lim_{k→∞} Eη_{k,j}^{2} = 0, and the triangular array η_{k,j}, 1 ≤ j ≤ k, k = 1, 2, … satisfies the condition of uniform smallness and the central limit theorem. Hence by the result of problem 39 the Lindeberg condition formulated in formula (14) holds for all such indices p. Since the matrix Σ is positive semi-definite, hence D_{p,p} ≥ 0 for all 1 ≤ p ≤ m. Therefore we have still consider those indices p for which D_{p,p} = 0. In this case

lim_{n→∞} \frac{1}{A_{n}^{2}} ∑_{k=1}^{n} Eξ_{p,k}^{2} = 0. Since Eξ_{p,k}^{2} ≥ Eξ_{p,k}^{2}I(|ξ_{p,k}| > εA_{n}), relation (14) also holds

for such indices p.

Appendix

The proof of the inversion formula for Fourier transforms.

Let us introduce the function f̂(u) = ∫ e^{−itu}f̃(u) du. To prove formula (6) we have to show that f̂(u) = f(u) for almost all numbers u ∈ R^{1}. This statement is equivalent to the identity ∫_{0}^{t} f(u) du = ∫_{0}^{t} f̂(u) du for all numbers t ∈ R^{1}. Since

∫_{0}^{t} f̂(u) du = \frac{1}{2π} ∫_{−∞}^{∞} ∫_{0}^{t} e^{−ius}f̃(u) ds du = \frac{1}{2π} ∫_{−∞}^{∞} \frac{e^{−itu} − 1}{−itu}f̃(u) du,  (A1)

we have to show the identity

∫_{−∞}^{∞} I_{[0,t]}(u)f(u) du = \frac{1}{2π} ∫_{−∞}^{∞} \frac{e^{−itu} − 1}{−itu}f̃(u) du,  (A2)

where I_{[0,t]}(·) is the indicator function of the interval [0, t]. The identity (A2) is a special case of an important identity of the Fourier analysis, of the Parseval formula. Let us formulate it.

Parseval formula.

∫ f(u)ḡ(u) du = \frac{1}{2π} ∫ f̃(u)¯g̃(u) du,  (A3)

where f̃(·) denotes the Fourier transform and f̄(·) the conjugate of a function f(·). Formula (A3) holds if one of the following conditions is satisfied:

a.) Both functions f and g are square integrable. b.) Both functions f̃ and g̃ are square integrable.

If one of the conditions a.) and b.) is satisfied, then also the other condition holds. In this case the identity ∫ |f(u)|^{2} du = \frac{1}{2π} ∫ |f̃(u)|^{2}, du holds (because of the Parseval formula). The transformation Tf = f → \frac{1}{\sqrt{2π}} f̃ is an automorphism in the space of square integrable function. This statement means not only the validity of the identity ∫ Tf(u)\overline{Tg}(u) du = ∫ f(u)ḡ(u) du. It also states that all square integrable functions f can be represented in the form f = Th with a square integrable function h.)

Let us finally remark that in a complete formulation of the Parseval formula the notion of the Fourier transform has to be defined for all square integrable but not necessarily integrable functions f(·). This definition can be given by means of the above mentioned L_{2} isomorphism. For all square integrable functions f(·) there exists a sequence of integrable and square integrable functions f_{n}(·) which converges to the function f(·) in the L_{2} norm of square integrable functions, i.e. ∫ |f_{n}(u) − f(u)|^{2} du → 0 if n → ∞. Then the Fourier transform f̃(·) of the function f(·) is the limit of the functions f̃_{n}(·) in the L_{2} norm. This limit always exists, and it does not depend on the choice of the sequence of functions f_{n}(·) converging to the function f(·).

In the Parseval formula formulated in this text a norming factor \frac{1}{2π} is present which does not appear in its formulation in text books. The reason of this difference is that we have chosen a different normalization in the definition of the Fourier transform. (We have omitted the factor \frac{1}{\sqrt{2π}} from the definition.)

If the Fourier transform of an integrable function f is integrable, then it is also square integrable, since it is a bounded function. Further, the Fourier transform of the function g(u) = I_{[0,t]}(u) is the square integrable function g̃(v) = ∫_{0}^{t} e^{iuv} du = \frac{e^{iv}−1}{iv}. Hence the formula (A2) (and therefore also formula (6)) is a consequence of the Parseval formula with the choice of the above functions f(·) and g(·).

We can prove that a finite measure μ with an integrable Fourier transform f̃(u) = ∫ e^{itu}μ( dt) has a density function f(·) defined by formula (6) with the help of the following smoothing argument. Let us consider for all numbers ε > 0 the Gaussian measure ν_{ε} with expectation zero and variance ε. This measure has density function φ_{ε}(u) = \frac{1}{\sqrt{2πε}}e^{−u^{2}/2ε} and Fourier transform e^{−εu^{2}/2}. Let us introduce the convolution μ_{ε} = μ ∗ ν_{ε}, i.e. μ_{ε}(A) = μ ∗ ν_{ε}(A) = ∫ μ(A − u)φ_{ε}(u) du.

The measure μ_{ε} has a density function f_{ε}(u) = ∫ φ_{ε}(u − v)μ( dv), and the Fourier transform of this measure is the integrable function f̃_{ε}(u) = e^{−εu^{2}/2}f̃(u). Hence the function f_{ε}(u) can be expressed as the inverse Fourier transform of the function f̃_{ε}(u) defined in formula (6). If ε → 0, then f_{ε}(u) → f(u), where f(u) is the function defined in formula (6), and this convergence is uniform in the variable u. On the other hand, the measure μ is the weak limit of the measures μ_{ε} if ε → 0, that is the probability measures \frac{μ_{ε}}{μ(R^{1})} converge weakly to the probability measure \frac{μ}{μ(R^{1})}. (Let us remark that

μ(R^{1}) = μ_{ε}(R^{1}).) Hence we get by taking limit ε → 0 that μ((a, b]) = ∫_{a}^{b} f(u) du if μ({a}) = μ({b}) = 0, i.e. if the points a and b are points of continuity of the measure μ. This implies that the function f is the density function of the measure μ.

By some slight modification of the above argument we can prove the above statement also in the case if μ is a signed measure with bounded variation and integrable Fourier transform. Let us remark that by refining the argument of the above limit procedure and exploiting the L_{2} isomorphism property of the Fourier transform, the above result about the density function of a measure μ and its Fourier transform can be strengthened. It is enough to assume that the Fourier transform of the (signed) measure μ is square integrable. But in this case the inverse Fourier transform defined in formula (6) has to be defined by means of the L_{2} extension of the original integral, and we cannot claim that the density function of the measure μ is continuous.

The proof of the Parseval formula. Let us first prove the Parseval formula for such simple pairs of functions (f, g) which disappear outside of a finite interval [−A, A], and which are sufficiently smooth, say they have two continuous derivatives. Then by considering the restriction of these functions to some interval [−πT, πT] ⊃ [−A, A], T π ≥ A, and the discrete version of the Parseval formula we can write that

∫ f(u)g(u) du = 2πT ∑_{k=−∞}^{∞} a_{k}(T)b̄_{k}(T),

where a_{k}(T) = \frac{1}{2πT} ∫ e^{iku/T} f(u) du = \frac{1}{2πT} f̃(\frac{k}{T} ), and b_{k}(T) = \frac{1}{2πT} g̃ (\frac{k}{T} ). But the above

expression 2πT ∑_{k=−∞}^{∞} a_{k}(T)b̄_{k}(T) is an approximating sum of the integral ∫ f̃(u)¯g̃(u) du,

and the Fourier transforms f̃(u) and g̃(u) tend to zero fast as |u| → ∞ because of the smoothness of the functions f and g. (See for instance the result of problem 28.) Hence the limit procedure T → ∞ yields formula (A3) in this special case.

The Parseval formula yields with the choice f = g the identity ∫ |f(u)|^{2} du = \frac{1}{2π} ∫ |f̃(u)|^{2} du, further the functions f for which we have proved these identity are everywhere dense in the space of square integrable functions. Hence we get proof of the Parseval formula by extending the isometry T: f → Tf = \frac{1}{\sqrt{2π}} f̃ in the L_{2} norm to the space of all square integrable functions. To complete the proof we still have to show that this extension of the transformation T maps to the whole space of square integrable functions.

To prove this missing part let us consider those functions f which are sufficiently smooth (say they are 10-times differentiable) and to zero sufficiently fast in plus minus infinity (say |f(u)| ≤ const. (1 + |u|^{100})). Since such functions constitute an everywhere dense set in the space of square integrable functions it is enough to show that they are in the image space of the operator T. We will show that the identity T\sqrt{2π}f̃^{−} = f, where f^{−}(u) = f(−u), follows from the already proved statements.

Also the function f̃ is smooth, and it tends to zero fast. (This also follows from the statements of problems 27 and 28. Actually the statement of problem 27 deals only with the Fourier transform of probability measures, but it is not difficult to see that this statement also holds for the Fourier transform of all signed measures with bounded variance. We want to exploit that the measures μ^{±}, μ^{±}(A) = ∫_{A} f^{±}(u) du, f^{+}(u) = max(f(u), 0), f^{−}(u) = − min(f(u), 0) have at least 8 moments.) Since both the functions f(u) and the indicator function I_{[0,t]}(u) of the interval [0, t] are square integrable, formula (A2) holds by the already proved part of the Parseval formula. Since the faction f̂ defined with the help of the function f̃(u) at the start of this Appendix is integrable, also formula (A1) holds. Formulas (A1) and (A2) together imply that the pair of functions (f, f̃) satisfy relation (6). But this relation is equivalent to the statement that the function f is the Fourier transform of the function 2πf̃^{−}, where f^{−}(u) = f(−u), and this is what we wanted to prove.

The proof of Weierstrass second approximation theorem.

The functions \frac{1}{(2π)^{k/2}} e^{i(j_{1}t_{1}+···+j_{k}t_{k})} constitute a complete orthonormal system in the space of square integrable functions which are periodic by 2π in all their arguments. This important result of the theory of Fourier series implies that all sufficiently smooth and in all their arguments periodic functions are the uniform limits of their Fourier series. Indeed, in this case the Fourier coefficients tend to zero fast, and this implies the uniform convergence. Since such functions are everywhere dense in the supremum norm in the space of continuous functions, this statement implies Weierstrass second approximation theorem. Nevertheless, instead of this argument we present a direct proof of Weierstrass second approximation theorem which does not apply the completeness of the trigonometrical functions in the L_{2} space. We shall prove Fejér’s theorem, more precisely its multi-dimensional version. Weierstrass second approximation theorem is a direct consequence of this result.

Fejér’s theorem. Let f(x_{1}, …, x_{k}) be a continuous function of k arguments which is periodic by 2π in all of its arguments. Let us define for all k-dimensional vectors (n_{1}, …, n_{k}) with non-negative integers the trigonometrical sum

s_{n_{1},...,n_{k}}(f)(t_{1}, …, t_{k}) = ∑_{j_{1}=−n_{1}}^{n_{1}} ⋯ ∑_{j_{k}=−n_{k}}^{n_{k}} A_{j_{1},...,j_{k}}e^{i(j_{1}t_{1}+···+j_{k}t_{k})},

where

A_{j_{1},...,j_{k}} = \frac{1}{(2π)^{k}} ∫_{−π}^{π} ⋯ ∫_{−π}^{π} e^{−i(j_{1}u_{1}+···+j_{k}u_{k})}f(u_{1}, …, u_{k}) du_{1} … du_{k}.

Let us also consider the following Cesaro means A_{n}(f), n = 1, 2, …, of the above trigonometrical sums:

A_{n}(f)(t_{1}, …, t_{k}) = \frac{1}{(n + 1)^{k}} ∑_{0≤n_{j}≤n for all indices 1≤j≤k} s_{n_{1},...,n_{k}}(f)(t_{1}, …, t_{k}).

Then lim_{n→∞} A_{n}(f)(t_{1}, …, t_{k}) = f(t_{1}, …, t_{k}), and the above convergence is uniform in all of its arguments t_{1}, …, t_{k}.

The proof of Fejér’s theorem. The proof of Fejér’s theorem is based on the following formula:

A_{n}(f)(t_{1}, …, t_{k}) = ∫_{−π}^{π} ⋯ ∫_{−π}^{π} f(u_{1}, …, u_{k})K̄_{n}(t_{1} −u_{1}, …, t_{k} −u_{k}) du_{1} … du_{k}, (A4)

where

K̄_{n}(u_{1}, …, u_{k}) = K_{n}(u_{1}) ⋯ K_{n}(u_{k}),  (A5)

and

K_{n}(u) = \frac{1}{2π(n + 1)} ∑_{k=−n}^{n}(n + 1 − |k|)e^{iuk} = \frac{sin^{2} (\frac{n+1}{2}u)}{2π(n + 1) sin^{2} (\frac{u}{2})}.  (A5′)

These relations hold, since by writing into the definitions of the expressions A_{n}(f) and s_{n_{1},...,n_{k}}(f) the definition of the Fourier coefficient A_{j_{1},...,j_{k}} we get relation (A4) together with formula (A5), where

K̄_{n}(u_{1}, …, u_{k}) = \frac{1}{(2π(n + 1))^{k}} ∑_{0≤n_{j}≤n for all indices 1≤j≤k} ∑_{|m_{j}|≤n_{j} for all indices 1≤j≤k} e^{i(m_{1}u_{1}+···+m_{k}u_{k})}

= \frac{1}{(2π(n + 1))^{k}} ∏_{j=1}^{k} ( ∑_{n_{j}=0}^{n} ∑_{m_{j}=−n_{j}}^{n_{j}} e^{im_{j}u_{j}} ) = K_{n}(u_{1}) … K_{n}(u_{k}),

and the function K_{n}(u) is defined in the middle term of formula (A5′). This sum can be written in a closed form with the help of the following calculation.

\frac{1}{2π(n + 1)} ∑_{k=−n}^{n}(n + 1 − |k|)e^{iuk} = \frac{1}{2π(n + 1)} (∑_{k=0}^{n} e^{iuk}) (∑_{k=0}^{n} e^{−iuk})

= \frac{1}{2π(n + 1)} |\frac{e^{i(n+1)u} − 1}{e^{iu} − 1}|^{2} = \frac{1}{2π(n + 1)} \begin{matrix}|e^{i(n+1)u/2} − e^{−i(n+1)u/2}| & 2 \\ |e^{iu/2} − e^{−iu/2}| & 2 \end{matrix}

= \frac{sin^{2} (\frac{n+1}{2}u)}{2π(n + 1) sin^{2} (\frac{u}{2})}.

The function K_{n}(u) defined in formula (A5′) have the following properties important for us:

(i) ∫_{−π}^{π} K_{n}(u) du = 1. This statement follows from the representation of the functions K_{n}(·) in the form of a sum.

(ii) K_{n}(u) ≥ 0 for all numbers u ∈ R^{1}. (iii) lim_{n→∞} sup_{ε≤|u|≤π} K_{n}(u) = 0 for all numbers ε > 0.

Statements (ii) and (iii) follow from the representation of the functions K_{n}(·) given in a closed form.

Since a function continuous on a compact set is uniformly continuous, there exists a constant δ = δ(ε, f) for all ε > 0 such that the continuous and in its coordinates periodic function f satisfies the inequality |f(x_{1}, …, x_{k})−f(y_{1}, …, y_{k})| < ε if |x_{j} −y_{j}| < δ for all indices j = 1, …, k. (In this relation we identify the points x_{j} + 2πl, l = 0 ± 1, ±2, …, and the inequality |x_{j} − y_{j}| < δ means that |x_{j} − y_{j} + 2πl_{l}| < δ with an appropriate integer l_{j}.) Let us introduce the notation B(δ, (t_{1}, …, t_{k})) = {(u_{1}, …, u_{k}): |u_{j} − t_{j}| < δ, −π ≤ u_{j} < π, j = 1, …, k}. Because of property (i)

A_{n}(f)(t_{1}, …, t_{k}) − f(t_{1}, …, t_{k})

= ∫_{[−π,π)^{k}} (f(u_{1}, …, u_{k}) − f(t_{1}, …, t_{k})) K_{n}(t_{1} − u_{1}) ⋯ K_{n}(t_{k} − u_{k}) du_{1} … du_{k}

= ∫_{B(δ,(t_{1},...,t_{k}))}[⋯] du_{1} … du_{k} + ∫_{[−π,π)^{k}\B(δ,(t_{1},...,t_{k}))}[⋯] du_{1} … du_{k}
= I_{1,n}(t_{1}, …, t_{k}) + I_{2,n}(t_{1}, …, t_{k}).

It follows from the definition of the set B(δ, (t_{1}, …, u_{t})), the number δ and properties (i) and (ii) that

|I_{1,n}(t_{1}, …, t_{k})| ≤ ε ∫_{[−π,π)^{k}} K_{n}(t_{1} − u_{1}) ⋯ K_{n}(t_{k} − u_{k}) du_{1} … du_{k} ≤ ε

for all indices n = 1, 2, … and points (t_{1}, …, t_{k}). On the other hand, by applying the notation sup_{(u_{1},...,u_{k})} |f(u_{1}, …, u_{k})| = L and carrying out the substitutions t_{j} − u_{j} = ū_{j}

we can show with the help of relations (i), (ii) and (iii) that

|I_{2,n}(t_{1}, …, t_{k})| ≤ 2L ∫_{[−π,π)^{k}\B(δ,(0,...,0))} K_{n}(ū_{1}) ⋯ K_{n}(ū_{k}) dū_{1} … dū_{k} → 0,

if n → ∞, since

∫_{\begin{matrix}δ<|ū_{j}|<π \\ −π≤ū_{l}<π, l≠j, 1≤l≤k \end{matrix}} K_{n}(ū_{1}) ⋯ K_{n}(ū_{k}) dū_{1} … dū_{k} = ∫_{δ<|u|<π} K_{n}(u) du

≤ 2π sup_{δ<|u|<π} K_{n}(u) → 0, ha n → ∞.

for all numbers 1 ≤ j ≤ k. Since the above estimates hold for all constants ε > 0 (together with an appropriate number δ = δ(ε, f)), they imply Fejér’s theorem.