ON THE TAIL BEHAVIOUR OF MULTIPLE RANDOM INTEGRALS AND DEGENERATE U-STATISTICS Péter Major Alfréd Rényi Mathematical Institute of the Hungarian Academy of Sciences

1. Introduction

In this work the following problem will be investigated: Fix a positive integer n, consider n independent, identically distributed random variables ξ_{1}, …, ξ_{n} on a measurable space (X, X) with some distribution μ and define their empirical distribution μ_{n} together with its normalization \sqrt{n}(μ_{n} − μ). Take a function f(x_{1}, …, x_{k}) of k variables on the k-fold product (X^{k}, X^{k}) of the space (X, X), introduce also the k-th power of the normalized empirical measure \sqrt{n}(μ_{n} − μ) on the space (X^{k}, X^{k}) and define the integral of the function f with respect to this signed product measure. This integral is a random variable, and for all u > 0 we want to give a good estimate on the probability that it is larger than u. More precisely, we take the integrals not on the whole space, we omit the diagonals x_{s} = x_{s′}, 1 ≤ s, s′ ≤ k, s ≠ s′, of the space X^{k} from the domain of integration. Such a modification of the integral seems to be natural.

We shall also be interested in the following generalized version of the above problem. Let us have a nice class of functions F of k variables on the product space (X^{k}, X^{k}) and consider the integral of all functions of this class with respect to the k-fold direct product of our normalized empirical measure. Give a good estimate on the probability that the supremum of these integrals is larger than some number u > 0.

The reader may ask why the above problems deserve a closer study. I found them important, because they may help to solve some important problems in probability theory and mathematical statistics. I met such problems when tried to adapt the method of proof about the Gaussian limit behaviour of the maximum likelihood estimate to some other problems. In the original problem the asymptotic behaviour of the solution of the so-called maximum likelihood equation has to be investigated. The study of this equation is hard in its original form. But by making an appropriate Taylor expansion of the function whose root we are looking for and throwing away its higher order terms we get an approximation whose behaviour can be simply understood. So to describe the limit behaviour of the maximum likelihood estimate it suffices to show that this approximation causes only a negligible error.

One would try to apply a similar procedure in more difficult situations. I met some non-parametric maximum likelihood problems, for instance the description of the limit behaviour of the so-called Kaplan–Meyer product limit estimate when such an approach could be applied. But in those problems it was harder to justify that the simplifying approximation causes only a negligible error. To show this the solution of the above mentioned problems were needed. In the non-parametric maximum likelihood estimate problems I met the estimation of multiple (random) integrals played a role similar to the estimation of the coefficients in the Taylor expansion in the study of the maximum likelihood estimate. Although I could apply this approach only in some special cases, I believe that it works in very general situations. But it demands some further work to show this.

The problem suggested in this work is interesting and non-trivial even in the special case k = 1. The solution of the problem in this case leads to some interesting, nontrivial generalization of the fundamental theorem of the mathematical statistics about the difference of the empirical and real distribution of a large sample.

The above mentioned problems have a natural counterpart about the behaviour of so-called U-statistics, a fairly popular subject in probability theory. The investigation of multiple random integrals and U-statistics are closely related, and it turned out that it is useful to consider them simultaneously. Hence both subjects will be discussed in this work.

Let us try to get some feeling what kind of results we can expect. It is useful to observe that for large sample size n the normalized empirical measure \sqrt{n}(μ_{n} − μ) behaves similarly to a Gaussian random measure. This suggests that in the problems we are interested in similar results should hold as in the case of multiple Gaussian integrals. Hence we may expect that the tail behaviour of the distribution of a k-fold random integral with respect to a normalized empirical measure is similar to that of the k-th power of a Gaussian random variable with expectation zero and an appropriate variance. Moreover, a similar estimate should hold for the supremum of random integrals of a class of functions under not too restrictive conditions. We may also hope that the methods of the theory of multiple Gaussian integrals can be adapted to the investigation of our problems.

The above belief is essentially correct, but there is an essential difference between the behaviour of multiple Gaussian integrals and multiple integrals with respect to a normalized empirical measure. If the variance of a multiple integral with respect to a normalized empirical measure is small, what turns out to be equivalent to the small L_{2}- norm of the function we are integrating, then the behaviour of this integral is different from that of multiple Gaussian integrals with the same variance. In this case the effect of some irregularities of the normalized empirical distribution turns out to be nonnegligible, and no good Gaussian approximation holds any longer. Hence some new methods have to be worked out and the hardest problems in our study appear at this point.

The precise formulation of the results will be contained in the main part of the work. Besides their proof I also try to explain the main ideas behind them and the notions introduced in their investigation. This work contains some new results, and also the proof of some already rather classical theorems is presented. To make the picture behind the problems more understandable I also discuss their Gaussian counterpart.

The proofs apply results from different parts of the probability theory. Papers investigating similar results refer to works dealing with quite different subjects, and this makes their reading rather hard. To overcome this difficulty I tried to work out the details and to present a self-contained discussion even at the price of a longer text. Thus I wrote down (in the main text or in the Appendix) the proof of many interesting and basic results, like results about Vapnik–Červonenkis classes, about U-statistics and their decomposition to sums of so-called degenerate U-statistics, logarithmic Sobolev inequalities, Borell’s inequality about homogeneous polynomials of Rademacher functions, etc.

I tried to give such an exposition where different parts of the problem are explained as independently of other as possible, and they can be understood in themselves.

This work was explained at the probability seminar of the University Debrecen (Hungary).

2. Motivation of the investigation. Discussion of some problems

Here I try to show by means of some examples why the solution of the problems mentioned in the introduction may be useful in the study of some important probabilistic problems. I try to give a good picture about the main ideas but do not work out all details. Actually, the elaboration of some details omitted would demand hard work. But as the discussion of this section is quite independent of the rest of the paper, these omissions cause no problem in understanding the subsequent part.

I start with a short discussion of the maximum likelihood estimate in the simplest case. We study the following problem. Let us have a class of density functions f(x, ϑ) on the real line depending on a parameter ϑ ∈ R^{1} and observe a sequence of independent random variables ξ_{1}(ω), …, ξ_{n}(ω) with a density function f(x, ϑ_{0}), where ϑ_{0} is an unknown parameter we want to estimate with the help of the above sequence of random variables.

We can carry out this estimation with the help of the maximum likelihood method. It suggests to choose the estimate ϑ̂_{n} = ϑ̂_{n}(ξ_{1}, …, ξ_{n}) of the parameter ϑ_{0} as the number where the density function of the random vector (ξ_{1}, …, ξ_{n}), i.e. the product

∏_{k=1}^{n} f(ξ_{k}, ϑ) = exp {∑_{k=1}^{n} log f(ξ_{k}, ϑ)}

takes its maximum. This point can be found as the solution of the so-called maximum likelihood equation

∑_{k=1}^{n} \frac{∂}{∂ϑ} log f(ξ_{k}, ϑ) = 0.  (2.1)

We are interested in the asymptotic behaviour of the random variable ϑ̂_{n} − ϑ_{0}, where ϑ̂_{n} is the (appropriate) solution of the equation (2.1).

The direct study of this equation is rather hard, but a Taylor expansion of the expression at the left-hand side of (2.1) around the (unknown) point ϑ_{0} helps to give a good and simple approximation of ϑ̂_{n}, and it enables us to describe the asymptotic behaviour of ϑ̂_{n} − ϑ_{0}.

This Taylor expansion yields that

∑_{k=1}^{n} \frac{∂}{∂ϑ} log f(ξ_{k}, ϑ̂_{n}) = ∑_{k=1}^{n} \frac{\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})}

+ (ϑ̂_{n} − ϑ_{0}) (∑_{k=1}^{n} ( \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} − \frac{(\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0}))^{2}}{f^{2}(ξ_{k}, ϑ̄_{0})} )) + O (n(ϑ̂_{n} − ϑ_{0})^{2})

=  ∑_{k=1}^{n} (η_{k} + ζ_{k}(ϑ̂_{n} − ϑ_{0})) + O (n(ϑ̂_{n} − ϑ_{0})^{2}) ,  (2.2)

where

η_{k} = \frac{\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} and ζ_{k} = \frac{\frac{∂^{2}}{∂ϑ^{2}} f(ξ_{k}, ϑ_{0})}{f(ξ_{k}, ϑ_{0})} − \frac{(\frac{∂}{∂ϑ}f(ξ_{k}, ϑ_{0}))^{2}}{f^{2}(ξ_{k}, ϑ̄_{0})}

for k = 1, …, n. We want to understand the asymptotic behaviour of the (random) expression on the right-hand side of (2.2). The relation

Eη_{k} = ∫ \frac{\frac{∂}{∂ϑ}f(x, ϑ_{0})}{f(x, ϑ_{0})}f(x, ϑ_{0}) dx = \frac{∂}{∂ϑ} ∫ f(x, ϑ_{0}) dx = 0

holds, since ∫ f(x, ϑ) dx = 1 for all ϑ, and differentiating this relation we get the last

identity. Similarly, Eη_{k}^{2} = −Eζ_{k} = ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0, k = 1, …, n. Hence by the

central limit theorem χ_{n} = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} η_{k} is asymptotically normal with expectation zero

and variance I^{2} = ∫ \frac{(\frac{∂}{∂ϑ} f(x,ϑ_{0}))^{2}}{f(x,ϑ_{0})} dx > 0. In the statistics literature this number I is

called the Fisher information. By the laws of large numbers \frac{1}{n} ∑_{k=1}^{n} ζ_{k} ∼ −I^{2}.

Thus relation (2.2) suggests the approximation ϑ̃_{n} = −\frac{∑_{k=1}^{n} η_{k}}{∑_{k=1}^{n} ζ_{k}} of the maximum-

likelihood estimate ϑ̂_{n}, and \sqrt{n}(ϑ̃_{n} − ϑ_{0}) is asymptotically normal with expectation zero and variance I^{2}. The random variable ϑ̃_{n} is not a solution of the equation (2.1), the value of the expression at the left-hand side is of order O(n(ϑ̃_{n} − ϑ_{0})^{2}) = O(1) in this point. On the other hand, the derivative of the function at the left-hand side is large in this point, it is greater than const. n with some const. > 0. This implies that the maximum-likelihood equation has a solution ϑ̂_{n} such that ϑ̂_{n} − ϑ̃_{n} = O (\frac{1}{n}). This has the consequence that \sqrt{n}(ϑ̂_{n} −ϑ_{0}) and \sqrt{n}(ϑ̃_{n} −ϑ_{0}) have the same asymptotic limit behaviour.

The previous method can be summarized in the following way: Take a simpler linearized version of the expression we want to estimate by means of an appropriate Taylor expansion, describe the limit distribution of this linearized version and show that the linearization causes only a negligible error.

We want to show that such a method also works in more difficult situations. But in some cases it is harder to show that the error we have committed by replacing the original expression by a simpler linearized version is negligible, and to do this we need the solution of the problems mentioned in the introduction. We shall present such an example by studying a fairly popular model of the mathematical statistics, the so-called Kaplan–Meyer method for the estimation of the empirical distribution function with the help of censored data.

The following problem is considered. Let (X_{i}, Z_{i}), i = 1, …, n, be a sequence of independent, identically distributed random vectors such that the components X_{i} and Z_{i} are also independent with distribution functions F (x) and G(x). We want to estimate the distribution function F of the random variables X_{i}, but we cannot observe the variables X_{i}, only the random variables Y_{i} = min(X_{i}, Z_{i}) and δ_{i} = I(X_{i} ≤ Z_{i}). In other words, we want to solve the following problem. There are certain objects whose lifetime X_{i} are independent and F distributed. But we cannot observe this lifetime X_{i}, because after a time Z_{i} the observation must be stopped. We also know whether the real lifetime X_{i} or the censoring variable Z_{i} was observed. We make n independent experiments and want to estimate with their help the distribution function F.

Kaplan and Meyer, on the basis of some maximum-likelihood estimation type considerations, proposed the following so-called product limit estimator S_{n}(u) to estimate the unknown survival function S = 1 − F:

1 − F_{n}(u) = S_{n}(u) = \{ \begin{matrix}∏_{i=1}^{n} (\frac{N(Y_{i})}{N(Y_{i}) + 1})^{I(Y_{i}≤u,δ_{i}=1)} & if u ≤ max(Y_{1}, …, Y_{n}) \\ 0 & if u ≥ max(Y_{1}, …, Y_{n}), δ_{n} = 1, \\ undefined & if u ≥ max(Y_{1}, …, Y_{n}), δ_{n} = 0, \end{matrix}

(2.3)

where

N(t) = #{Y_{i}, Y_{i} > t, 1 ≤ i ≤ n} = ∑_{i=1}^{n} I(Y_{i} > t).

We want to show that the above estimate (2.3) is really good. For this goal we shall approximate the random variables S_{n}(u) by some appropriate random variables. To do this first we introduce some notations.

Put

\begin{aligned}H(u) & = P (Y_{i} ≤ u) = 1 − H̄(u), \\ H̃(u) & = P (Y_{i} ≤ u, δ_{i} = 1), ˜ H̃(u) = P (Y_{i} ≤ u, δ_{i} = 0) \end{aligned}  (2.4)

and

\begin{aligned}H_{n}(u) & = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u) \\ H̃_{n}(u) & = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1), ˜ H̃_{n}(u) = \frac{1}{n} ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 0). \end{aligned}  (2.5)

Clearly H(u) = H̃(u) + ˜ H̃(u) and H_{n}(u) = H̃_{n}(u) + ˜ H̃_{n}(u). We shall estimate F_{n}(u) − F (u) for u ∈ (−∞, T] if

1 − H(T) > δ with some fixed δ > 0.  (2.6)

Condition (2.6) implies that there are more than \frac{δ}{2}n sample points Y_{j} larger than T with probability almost 1. It has exponentially small probability that this is not the case. This observation helps to show in the subsequent calculations that some events have negligibly small probability.

We introduce the so-called cumulative hazard function and its empirical version

Λ(u) = − log(1 − F (u)), Λ_{n}(u) = − log(1 − F_{n}(u)).  (2.7)

Since F_{n}(u) − F (u) = exp(−Λ(u)) (1 − exp(Λ(u) − Λ_{n}(u))) a simple Taylor expansion yields

F_{n}(u) − F (u) = (1 − F (u)) (Λ_{n}(u) − Λ(u)) + R_{1}(u),  (2.8)

and it is easy to see that R_{1}(u) = O (Λ(u) − Λ_{n}(u))^{2}). It follows from the subsequent estimations that Λ(u) − Λ_{n}(u) = O(n^{−1/2}), thus nR_{1}(u) = O(1). Hence it is enough to investigate the term Λ_{n}(u). We shall show that Λ_{n}(u) has an expansion with Λ(u) as the main term plus n^{−1/2} a term which is a linear functional of an appropriate normalized empirical distribution function plus an error term of order O(n^{−1}).

From (2.3) it is obvious that

Λ_{n}(u) = − ∑_{i=1}^{n} I(Y_{i} ≤ u, δ_{i} = 1) log (1 − \frac{1}{1 + N(Y_{i})}) .

We can get rid of the unpleasant logarithmic function in this formula by means of the relation − log(1 − x) = x + O(x^{2}) for small x which yields that

Λ_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{N(Y_{i})} + R_{2}(u) = Λ̃_{n}(u) + R_{2}(u),  (2.9)

and the error term nR_{2}(u) is exponentially small.

The expression Λ̃_{n}(u) is still inappropriate for our purposes. Since the denominators

N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) are dependent for different indices i we cannot see directly the

limit behaviour of Λ̃_{n}(u).

We try to approximate Λ̃_{n}(u) by a simpler expression. A natural approach would be to approximate the terms N(Y_{i}) in it by their conditional expectation (n−1)H̄(Y_{i}) = (n − 1)(1 − H(Y_{i})) = E(N(Y_{i})|Y_{i}). This is a too rough ‘first order’ approximation, but the following ‘second order approximation’ will be sufficient for our goals. Put

N(Y_{i}) = ∑_{j=1}^{n} I(Y_{j} > Y_{i}) = nH̄(Y_{i}) (1 + \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})})

and express the terms \frac{1}{N(Y_{i})} in the sum defining Λ̃_{n} by means of the relation \frac{1}{1+z} =

∑_{k=0}^{∞}(−1)^{k}z^{k} = 1 − z + ε(z) with the choice z = ^{\frac{∑_{j=1}^{n} I(Y_{j}>Y_{i})−nH̄(Y_{i})}{nH̄(Y_{i})}}. As |ε(z)| < 2z^{2} for

|z| < \frac{1}{2} we get that

Λ̃_{n}(u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} (1 + ∑_{k=1}^{∞} (−\frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})})^{k})

= ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})} (1 − \frac{∑_{j=1}^{n} I(Y_{j} > Y_{i}) − nH̄(Y_{i})}{nH̄(Y_{i})}) + R_{3}(u) (2.10)

= 2A(u) − B(u) + R_{3}(u),

where

A(u) = A(n, u) = ∑_{i=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)}{nH̄(Y_{i})}

and

B(u) = B(n, u) = ∑_{i=1}^{n} ∑_{j=1}^{n} \frac{I(Y_{i} ≤ u, δ_{i} = 1)I(Y_{j} > Y_{i})}{n^{2}H̄^{2}(Y_{i})}.

It can be proved by means of standard methods that nR_{3}(u) is exponentially small. Thus from (2.9) and (2.10) we get that

Λ_{n}(u) = 2A(u) − B(u) + negligible error.  (2.11)

This means that to solve our problem we have to describe the asymptotic behaviour of the random variables A(u) and B(u). We can get a better insight into their behaviour by rewriting the sum A(u) as an integral with respect to an empirical measure and the double sum B(u) as a two-fold integral with respect empirical measures. These integrals can be rewritten as sums of random integrals with respect to normalized empirical measures and deterministic measures. In such a way we get a representation of Λ_{n}(u) in the form of a sum whose terms can be well understood.

Let us write

A(u) = ∫_{−∞}^{+∞} \frac{I(y ≤ u)}{1 − H(y)} dH̃_{n}(y),
B(u) = ∫_{−∞}^{+∞} ∫_{−∞}^{+∞} \frac{I(y ≤ u)I(x > y)}{(1 − H(y))^{2}} dH_{n}(x)dH̃_{n}(y).

To rewrite the term B(u) in a form better for our purposes observe that

H_{n}(x)H̃_{n}(y) = H(x)H̃(y) + H(x)(H̃_{n}(y) − H̃(y)) + (H_{n}(x) − H(x))H̃(y)
+ (H_{n}(x) − H(x))(H̃_{n}(y) − H̃(y)).

Hence it can be written in the form B(u) = B_{1}(u) + B_{2}(u) + B_{3}(u) + B_{4}(u), where

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x) dH̃(y) ,

B_{2}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x) d (\sqrt{n}(H̃_{n}(y) − H̃(y))) ,

B_{3}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n} (H_{n}(x) − H(x))) dH̃(y) ,

B_{4}(u) = \frac{1}{n} ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} d (\sqrt{n} (H_{n}(x) − H(x))) d (\sqrt{n}(H̃_{n}(y) − H̃(y))) .

In the above decomposition of B(u) the term B_{1} is a deterministic function, B_{2}, B_{3} are linear functionals of empirical processes and B_{4} is a nonlinear functional of empirical processes. The deterministic term B_{1}(u) can be calculated explicitly. Indeed,

B_{1}(u) = ∫_{−∞}^{u} ∫_{−∞}^{+∞} \frac{I(x > y)}{(1 − H(y))^{2}} dH(x)dH̃(y) = ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)}.

Then the relations H̃(u) = ∫_{−∞}^{u} (1 − G(t)) dF (t) and 1 − H = (1 − F)(1 − G) imply that

B_{1}(u) = ∫_{−∞}^{u} \frac{dF (y)}{1 − F (y)} = − log(1 − F (u)) = Λ(u).  (2.12)

Observe that

\begin{aligned}A(u) & = ∫_{−∞}^{u} \frac{d H̃_{n}(y)}{1 − H(y)} \\ & = ∫_{−∞}^{u} \frac{dH̃(y)}{1 − H(y)} + \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)} \\ & = B_{1}(u) + B_{2}(u). \end{aligned}  (2.13)

From relation (2.11) using (2.12) and (2.13) it follows that

Λ_{n}(u) − Λ(u) = B_{2}(u) − B_{3}(u) − B_{4}(u) + negligible error.  (2.14)

Integrating B_{2} and B_{3} in the variable x and then integrating by parts B_{2} we get that

B_{2}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{d (\sqrt{n}(H̃_{n}(y) − H̃(y)))}{1 − H(y)}

= \frac{\sqrt{n} (H̃_{n}(u) − H̃(u))}{\sqrt{n}(1 − H(u))} − \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y) − H̃(y))}{(1 − H(y))^{2}} dH(y)

B_{3}(u) = \frac{1}{\sqrt{n}} ∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H_{n}(y) − H(y)) \\ (1 − H(y)) & 2 \end{matrix} dH̃(y).

Using the above forms of B_{2} and B_{3}, (2.12) we can write

\sqrt{n} (Λ_{n}(u) − Λ(u)) = \frac{\sqrt{n} (H̃_{n}(u) − H̃(u))}{1 − H(u)} − ∫_{−∞}^{u} \frac{\sqrt{n}(H̃_{n}(y) − H̃(y))}{(1 − H(y))^{2}} dH(y)

+ ∫_{−∞}^{u} \begin{matrix}\sqrt{n} & (H_{n}(y) − H(y)) \\ (1 − H(y)) & 2 \end{matrix} dH̃(y)

− \sqrt{n}B_{4}(u) + negligible error.

(2.15)

Formula (2.15) almost agrees with the statement we wanted to prove. Here we expressed the normalized error \sqrt{n} (Λ_{n}(u) − Λ(u)) as a sum of linear functionals of normalized empirical measures plus some negligible error terms and the error term \sqrt{n}B_{4}(u). So to get a complete proof it is enough to show that \sqrt{n}B_{4}(u) also yields a negligible error. But B_{4}(u) is a double integral of a bounded function (here we apply again formula (2.6)) with respect to a normalized empirical measure. Hence to bound this term we need a good estimate of multiple stochastic integrals (with multiplicity 2) and this is just the problem formulated in the introduction. The estimate we need here follows from Theorem 8.1 of the present work. Let us remark that the problem discussed here corresponds to the estimation of the coefficient of the second term in the Taylor expansion considered in the study of the maximum likelihood estimation. One may worry a little bit how to bound B_{4}(u) with the help of estimations of double stochastic integrals, since in the definition of B_{4}(u) we integrate by different normalized empirical processes in the two coordinates. But this is a not too difficult technical problem, it can be simply overcome for instance by rewriting the integral as a double integral with respect to the empirical process (\sqrt{n} (H_{n}(x) − H(x)) , \sqrt{n} (H̃_{n}(y) − H̃(y))) in the space R^{2}.

By working out the details of the above calculation we get that the linear functional B_{2}(u)−B_{3}(u) of normalized empirical processes yields a good estimate on the expression \sqrt{n}(Λ_{n}(u) − Λ(u)) for a fixed parameter u. But we want to prove somewhat more, we want to get an estimate uniform in the parameter u, i.e. to show that even the random variable sup_{u≤T} |\sqrt{n}(Λ_{n}(u) − Λ(u)) − B_{2}(u) + B_{3}(u)| is small. This can be done by making

estimates uniform in the parameter u in all steps of the above calculation. There appears only one difficulty when trying to carry out this program. Namely, we need an estimate on sup_{u} |B_{4}(u)|, i.e. we have to bound the supremum of multiple random integrals with

respect to a normalized random measure for a nice class of kernel functions. This can be done, but at this point the second problem mentioned in the introduction appears. This difficulty can be overcome by means of Theorem 8.2 of this work.

Thus we can find the limit behaviour of the Kaplan–Meyer estimate by means of an appropriate expansion. The steps of this investigation are fairly standard, the only hard part is the solution of the problems mentioned in the introduction. We expect that such a method also works in much more general situation. This may justify a detailed study of the problems considered in this work.

I finish this section with a remark of Richard Gill he made in a personal conversation after my talk on this subject at a conference. He told that this approach had given a complete proof about the limit behaviour of this estimate, but it had exploited the explicit formula given in the Kaplan–Meyer estimate. He missed the application of an argument based on the non-parametric maximum likelihood character of this estimate. This was a completely justified remark, since if we do not restrict our attention to this problem, but try to generalize it to general non-parametric maximum likelihood estimates, then we have to understand how the maximum likelihood character can be exploited. I believe that this can be done, but it demands further studies.

3. Some estimates about sums of independent random variables

We need some results about the distribution of sums of independent random variables bounded by a constant with probability one. Later only the results about sums of independent and identically distributed variables will be interesting for us, but since these results can be generalized without any effort to sums of not necessarily identically distributed random variables here we shall drop the condition about the identical distribution of the summands. We are interested in the question when these estimates give such a good bound as the central limit theorem suggests, and what can be told if this is not the case. More explicitly, we consider the following problem: Let X_{1}, …, X_{n} be independent random variables EX_{j} = 0, Var X_{j} = σ_{j}^{2}, 1 ≤ j ≤ n, and take the random

sum S_{n} = ∑_{j=1}^{n} X_{j} and its variance Var S_{n} = V_{n}^{2} = ∑_{j=1}^{n} σ_{j}^{2}. We want to get a good bound

on the probability P (S_{n} > xV_{n}). The central limit theorem would suggest that under general conditions an upper bound of the order 1−Φ(x) should hold for this probability where Φ(x) denotes the standard normal distribution function. Since the standard nor-

mal distribution function satisfies the inequality (\frac{1}{x} − \frac{1}{x^{3}} ) \frac{e^{−x^{2}/2}}{\sqrt{2π}} < 1 − Φ(x) < \frac{1}{x} \frac{e^{−x^{2}/2}}{\sqrt{2π}} for all x > 0 it is natural to ask when the probability P (S_{n} > xV_{n}) is comparable with the value e^{−x^{2}/2}. More generally, we say that we have a Gaussian type estimate for the probability P (S_{n} > xV_{n}) if it can be bounded by e^{−Cx^{2}} with some constant C separated from zero.

First we discuss Bernstein’s inequality which tells for which values x the probability P (S_{n} > xV_{n}) satisfies a Gaussian type estimate. Such an estimate holds (for sums of random variables bounded by 1) if x ≤ const. V_{n}. For x ≥ const. V_{n} Bernstein’s inequality yields almost no improvement if we have a better bound on the variance V_{n} of the sum S_{n}. Another estimate, Bennett’s inequality yields a slight improvement, and as an example presented before this result shows it cannot be essentially improved without imposing some additional conditions. The main difficulties we meet in this paper are closely related to the weakness of the estimates we have for the probability of the event that a sum of independent random variables is larger than some value when this probability does not satisfy a Gaussian type estimate because of the small variance of the sum.

Let us formulate Bernstein’s inequality. In its usual formulation a real number M is introduced and it is assumed that the terms in the sum we investigate are bounded by this number. But since the problem can be simply reduced to the special case M = 1 we shall only deal with this special case.

Theorem 3.1 (Bernstein’s inequality). Let X_{1}, …, X_{n} be independent random variables, P (|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n,

S_{n} = ∑_{j=1}^{n} X_{j} and V_{n}^{2} = Var S_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P (S_{n} > xV_{n}) ≤ exp \{−\frac{x^{2}}{2 (1 + \frac{1}{3} \frac{x}{V_{n}} )}\} for all x > 0.  (3.1)

Proof of Theorem 3.1. Let us give a good bound on the exponential moments Ee^{tS_{n}}

for some appropriate parameters t > 0. We can write Ee^{tX_{j}} = ∑_{k=0}^{∞} \frac{t^{k}}{k!} EX_{j}^{k} ≤ 1 +

\frac{t^{2}σ_{j}^{2}}{2} (1 + ∑_{k=1}^{∞} \frac{2t^{k}}{(k+2)!}) ≤ 1 + \frac{t^{2}σ_{j}^{2}}{2} (1 + ∑_{k=1}^{∞} 3^{−k}t^{k}) = 1 + \frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} ≤ exp \{\frac{t^{2}σ_{j}^{2}}{2} \frac{1}{1− \frac{t}{3}} \} if

0 ≤ t < 3. Hence Ee^{tS_{n}} = ∏_{j=1}^{n} Ee^{tX_{j}} ≤ exp \{\frac{t^{2}V_{n}^{2}}{2} \frac{1}{1− \frac{t}{3}} \} for 0 ≤ t < 3.

The above relation implies that

P (S_{n} > xV_{n}) = P (e^{tS_{n}} > e^{txV_{n}}) ≤ Ee^{tS_{n}}e^{−txV_{n}} ≤ exp \{\frac{t^{2}V_{n}^{2}}{2} \frac{1}{1 − \frac{t}{3}} − txV_{n}\}

if 0 ≤ t < 3. Choose the number t in this inequality as the solution of the equation t^{2}V_{n}^{2} \frac{1}{1− \frac{t}{3}} = txV_{n}, i.e. put t = \frac{x}{V_{n}+ \frac{x}{3}} . Then 0 ≤ t < 3, and we get that P (S_{n} > xV_{n}) ≤

e^{−txV_{n}/2} = exp \{−\frac{x^{2}}{2(1+ \frac{1}{3} \frac{x}{Vn})}\}.

If the random variables X_{1}, …, X_{n} satisfy the conditions of the Bernstein inequality then also the random variables −X_{1}, …, −X_{n} satisfy them. By applying the above

result in both cases we get that P (|S_{n}| > xV_{n}) ≤ 2 exp \{−\frac{x^{2}}{2(1+ \frac{1}{3} \frac{x}{Vn})}\} under the condi-

tions of the Bernstein inequality.

Bernstein’s inequality states that for all ε > 0 there is some sufficiently small number α(ε) > 0 such that in the case \frac{x}{V_{n}} < α(ε) P (S_{n} > xV_{n}) ≤ e^{−(1−ε)x^{2}/2}. Besides, for all fixed numbers A > 0 there is some constant C = C(A) > 0 such that in the case \frac{x}{V_{n}} < A the inequality P (S_{n} > xV_{n}) ≤ e^{−Cx^{2}} holds. This can be interpreted as a Gaussian type estimate for the probability P (S_{n} > xV_{n}).

On the other hand, if \frac{x}{V_{n}} is very large, then the Bernstein inequality yields a much worse estimate. The next example shows that this is not because of its weakness. There are sequences of independent, identically distributed random variables X_{1}, …, X_{n}

bounded by one and with expectation zero such that with the notations S_{n} = ∑_{j=1}^{n} X_{j},

σ^{2} = EX_{j}^{2}, V_{n}^{2} = ∑_{j=1} EX_{j}^{2} = nσ^{2} the probability P (S_{n} > xV_{n}) is relatively large if \frac{x}{V_{n}}

is large, it is much larger than the value suggested by the normal approximation. This example will be interesting for us mainly for the sake of some orientation. Hence I do not try to formulate it in such a general form as it could be done or to give the best possible constants in it. The method of proof shows that a wide class of examples could be constructed with similar properties. In the following discussion it will be convenient to replace the number x by y = xV_{n} = \sqrt{n}σx.

Example 3.2. Let us fix some positive integer n, real numbers y ≥ 200 and 1 > σ^{2} > 0 such that n > 16y > 64nσ^{2}. Put V_{n}^{2} = nσ^{2} and take a sequence of independent, identically distributed random variables X_{1}, …, X_{n} such that P (X_{j} = 1) = P (X_{j} =

−1) = \frac{σ^{2}}{2} , and P (X_{j} = 0) = 1 − σ^{2}. Put S_{n} = ∑_{j=1}^{n} X_{j}. Then ES_{n} = 0, Var S_{n} = V_{n}^{2},

and

P (S_{n} > y) > A exp \{−By log \frac{y}{V_{n}^{2}} \}

with some universal constants A > 0 and B > 0. We can choose for instance A = \frac{1}{2}, B = \frac{22}{5} in this inequality.

Here I shall give a proof of the statement of Example 3.2. Let me remark that in the work [23] I gave a simpler and more elementary proof of this result under the name Example 2.4.

Proof of the statement of Example 3.2. In the proof some ideas of the large deviation theory will be applied. Let us introduce the measure μ, μ(\{1\}) = μ(\{−1\}) = \frac{σ^{2}}{2} , μ({0}) = 1−σ^{2} on the real line, which is actually the distribution of the random variables X_{j}, together with its conjugates μ_{t}, μ_{t}( dx) = \frac{e^{tx}}{\frac{σ^{2}}{2} (e^{t}+e^{−t})+1−σ^{2}} μ( dx), x ∈ R^{1}, for all

real numbers t. Let μ^{(n)} denote the n-fold convolution of the measure μ and μ_{t}^{(n)} the n-fold convolution of the measure μ_{t} with itself. Then P (S_{n} > y) = μ^{(n)}((y, ∞)), and it is not difficult to see (and it is a well-known fact in the theory of large deviations) that μ^{(n)}(A) = (\frac{σ^{2}}{2} (e^{t} + e^{−t}) + 1 − σ^{2})^{n} ∫_{A} e^{−tu}μ_{t}^{(n)}( du) for all measurable sets A ⊂ R^{1}.

Let us consider the above defined measures μ_{t} and μ_{t}^{(n)} with t = log \frac{4y}{nσ^{2}} . I claim that μ_{t}^{(n)}([y, \frac{11}{5}y]) ≥ \frac{1}{2}. To show this let us consider n independent μ_{t} distributed, independent random variables ξ_{1}, …, ξ_{n}, and estimate their expected value and variance.

We have Eξ_{j} = \frac{\frac{σ^{2}}{2} (e^{t}−e^{−t})}{\frac{σ^{2}}{2} (e^{t}+e^{−t})+1−σ^{2}} for all 1 ≤ j ≤ n, and since 1 ≤ \frac{σ^{2}}{2} (e^{t} + e^{−t}) + 1 − σ^{2} ≤

1 + σ^{2}e^{t} = 1 + 4\frac{y}{n} ≤ \frac{5}{4}, and besides, we get with the help of the estimate e^{−t} =

e^{t}e^{−2t} = e^{t} (\frac{nσ^{2}}{4y} )^{2} ≤ \frac{1}{4}e^{t} the inequality \frac{3}{2} \frac{y}{n} = \frac{3}{8}σ^{2}e^{t} ≤ \frac{σ^{2}}{2} (e^{t} − e^{−t}) ≤ \frac{σ^{2}}{2} e^{t} = 2\frac{y}{n},

hence \frac{6}{5} \frac{y}{n} ≤ Eξ_{j} ≤ 2\frac{y}{n}. Similarly, Var ξ_{j} ≤ Eξ_{j}^{2} = \frac{\frac{σ^{2}}{2} (e^{t}+e^{−t})}{\frac{σ^{2}}{2} (e^{t}+e^{−t})+1−σ^{2}} ≤ 4\frac{y}{n}. The

above estimates together with the Chebishev inequality imply that μ_{t}^{(n)}([y, \frac{11}{5}y]) =

P (y ≤ ∑_{j=1}^{n} ξ_{j} ≤ \frac{11}{5}y) ≥ 1 − P (| ∑_{j=1}^{n}(ξ_{j} − Eξ_{j})| > \frac{y}{5} ) ≥ 1 − \frac{100y}{y^{2}} ≥ \frac{1}{2}. This inequality

together with the relation between the measures μ^{(n)} and μ_{t}^{(n)} imply that

P (S_{n} > y) = μ^{(n)}([y, ∞]) ≥ μ^{(n)} ([y, \frac{11}{5}y])

= (\frac{σ^{2}}{2} (e^{t} + e^{−t}) + 1 − σ^{2})^{n} ∫_{y}^{11y/5}e^{−tu}μ_{t}^{(n)}( du) ≥ e^{−11ty/5}μ_{t}^{(n)} ([y, \frac{11}{5}y])

≥ \frac{1}{2}e^{−11ty/5} = \frac{1}{2} exp \{−\frac{11}{5}y log \frac{4y}{V_{n}^{2}} \} ≥ \frac{1}{2} exp \{−\frac{22}{5}y log \frac{y}{V_{n}^{2}} \} .

In the case y > V_{n}^{2} the Bernstein inequality yields the estimate P (S_{n} > y) ≤ e^{−αy} with some universal constant α > 0, while the above example shows that we can expect at most an additional logarithmic factor in the exponent of the upper bound in an improvement of this estimate. The following result, called Bennett’s inequality shows that such an improvement is really possible.

Theorem 3.3 (Bennett’s inequality). Let X_{1}, …, X_{n} be independent random vari-

ables, P (|X_{j}| ≤ 1) = 1, EX_{j} = 0, 1 ≤ j ≤ n. Put σ_{j}^{2} = EX_{j}^{2}, 1 ≤ j ≤ n, S_{n} = ∑_{j=1}^{n} X_{j}

and V_{n}^{2} = Var S_{n} = ∑_{j=1}^{n} σ_{j}^{2}. Then

P (S_{n} > y) ≤ exp \{−V_{n}^{2} [(1 + \frac{y}{V_{n}^{2}} ) log (1 + \frac{y}{V_{n}^{2}} ) − \frac{y}{V_{n}^{2}} ]\} for all y > 0. (3.2)

As a consequence, for all ε > 0 there exists some B = B(ε) > 0 such that

P (S_{n} > y) ≤ exp \{−(1 − ε)y log \frac{y}{V_{n}^{2}} \} if y > BV_{n}^{2},  (3.3)

and there exists some positive constant K > 0 such that

P (S_{n} > y) ≤ exp \{−Ky log \frac{y}{V_{n}^{2}} \} if y > 2V_{n}^{2}.  (3.4)

Proof of Theorem 3.3. We have

Ee^{tX_{j}} = ∑_{k=0}^{∞} \frac{t^{k}}{k!} EX_{j}^{k} ≤ 1 + σ_{j}^{2} ∑_{k=2}^{∞} \frac{t^{k}}{k!} = 1 + σ_{j}^{2} (e^{t} − 1 − t) ≤ e^{σ_{j}^{2}(e^{t}−1−t)}, 1 ≤ j ≤ n,

and Ee^{tS_{n}} ≤ e^{V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. Hence P (S_{n} > y) ≤ e^{−ty}Ee^{tS_{n}} ≤ e^{−ty+V_{n}^{2}(e^{t}−1−t)} for all t ≥ 0. We get relation (3.2) from this inequality with the choice t = log (1 + \frac{y}{V_{n}^{2}} ).

(This is the place of minimum of the function −ty + V_{n}^{2}(e^{t} − 1 − t) for fixed y in the parameter t.)

Relation (3.2) and the observation lim_{u→∞} \frac{(u+1) log(u+1)−u}{u log u} = 1 with the choice u = \frac{y}{V_{n}^{2}} imply formula (3.3). Because of relation (3.3) to prove formula (3.4) it is enough to check it for 2 ≤ \frac{y}{V_{n}^{2}} ≤ B with some sufficiently large constant B > 0. In this case relation (3.4) follows directly from formula (3.2). This can be seen for instance by observing

that the expression ^{\frac{V_{n}^{2}[(1+_{\frac{y}{V ^{2}_{n}}} ) log(1+_{\frac{y}{V ^{2}_{n}}} )−_{\frac{y}{V ^{2}_{n}}} ]}{y log _{\frac{y}{V ^{2}_{n}}}}} is a continuous and positive function

of the variable \frac{y}{V_{n}^{2}} in the interval 2 ≤ \frac{y}{V_{n}^{2}} ≤ B, hence its minimum in this interval is strictly positive.

Let us make a short comparison between Bernstein’s and Bennett’s inequality. Both results deal with the estimation of the probability P (S_{n} > y), and their proofs are also very similar. In both cases first an estimate is given for the moment generating functions R_{j}(t) = Ee^{tX_{j}} of the summands X_{j}. In Bennett’s inequality a better estimate is given for them. (The worst case we have to handle is when P (X_{j} = 1) = ε_{j},

P (X_{j} = −\frac{e_{j}}{1−ε_{j}} ) = 1 − ε_{j}, and ε_{j} + \frac{ε_{j}^{2}}{1−ε_{j}} = σ_{j}^{2}. In this case the proof of Bennett’s inequality contains an almost optimal estimate, while the estimate in Bernstein’s inequality is weaker. In this estimate we are satisfied to give a good estimate for the first three coefficients in the Taylor expansion of the function R_{j}(t).) With the help of this estimate a bound is given on the probability we are interested in which depends on the parameter t. In the proof of Bennett’s inequality this parameter t is chosen optimally, while in Bernstein’s inequality only an asymptotically optimal choice is taken. As a consequence, Bennett’s inequality yields a sharper estimate. Actually Bernstein’s inequality can be deduced from it. On the other hand, Bernstein’s inequality gives a good, ‘visible’ bound for the probability P (S_{n} > y) for not too large values of the number y which suffices for our purposes, while the magnitude of the estimate given by Bennett’s inequality for small y cannot be directly seen. For large y Bennett’s yields a better estimate, but this improvement seems to have a smaller importance.

I finish this section with another estimate due to Hoeffding which later will be useful for us when we want to carry out certain symmetrization arguments.

Theorem 3.4 (Hoeffding’s inequality). Let ε_{1}, …, ε_{n} be independent random variables, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and let a_{1}, …, a_{n} be arbitrary real

numbers. Put V = ∑_{j=1}^{n} a_{j}ε_{j}. Then

P (V > y) ≤ exp \{−\frac{y^{2}}{2 ∑_{j=1}^{n} a_{j}^{2}} \} for all y > 0.  (3.5)

Remark: Clearly EV = 0 and Var V = ∑_{j=1}^{n} a_{j}^{2}, hence Hoeffding’s inequality yields such an estimate for P (V > y) which the central limit theorem suggests. This estimate holds for all real numbers a_{1}, …, a_{n}.

Remark 2: If we consider the Rademacher functions r_{k}(x), r_{k}(x) = 1 if (2j − 1)2^{−k} ≤ x < 2j2^{−k} and r_{k}(x) = −1 if 2(j − 1)2^{−k} ≤ x < (2j − 1)2^{−k}, 1 ≤ j ≤ 2^{k}, for all k = 1, 2, …, as random variables on the probability space Ω = [0, 1] with the Borel σalgebra and the Lebesgue measure as probability measure on the interval [0, 1], then they are independent random variables with the same distribution as the random variables ε_{1}, …, ε_{n} considered in Theorem 3.4. Therefore such results which deal with random variables of this type are also called results about Rademacher functions in the literature. At some points we shall also use this terminology.

Proof of Theorem 3.4. Let us give a good bound on the exponential moment Ee^{tV}

for all t > 0. We have Ee^{tV} = ∏_{j=1}^{n} Ee^{ta_{j}ε_{j}} = ∏_{j=1}^{n} \frac{(e^{a_{j} t}+e^{−a_{j} t})}{2}, and \frac{(e^{a_{j} t}+e^{−a_{j} t})}{2} =

∑_{k=0}^{∞} \frac{a_{j}^{2k}}{(2k)!}t^{2k} ≤ ∑_{k=0}^{∞} \frac{(a_{j}t)^{2k}}{2^{k}k!} = e^{a_{j}^{2}t^{2}/2}, since (2k)! ≥ 2^{k}k! for all k ≥ 0. This implies

that Ee^{tV} ≤ exp \{\frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\}. Hence P (V > y) ≤ exp \{−ty + \frac{t^{2}}{2} ∑_{j=1}^{n} a_{j}^{2}\}, and we get

relation (3.5) with the choice t = y ( ∑_{j=1}^{n} a_{j}^{2})^{−1}.

4. On the supremum of a nice class of partial sums

This section contains a result about the behaviour of the supremum of random integrals with respect to a normalized empirical measure in the special case when only one-fold integrals are considered. First we present an equivalent version of it about the supremum of a nice class of sums of independent, identically distributed random variables. We also discuss some natural problems related to them. In particular, we are interested in the question how restrictive the conditions of these results are. Also the natural Gaussian counterpart of these results will be given, but the proofs are postponed to a later section.

To formulate our results first we introduce the following notion.

Definition of L_{p}-dense classes of functions. Let us have a measurable space (Y, Y) and a set G of Y measurable real valued functions on this space. We call G an L_{p}- dense class of functions, 1 ≤ p < ∞, with parameter D and exponent L if for all numbers 1 ≥ ε > 0 and probability measures ν on the space (Y, Y) there exists a finite ε-dense subset G_{ε,ν} = {g_{1}, …, g_{m}} ⊂ G in the space L_{p}(Y, Y, ν) consisting of m ≤ Dε^{−L} elements, i.e. there exists such a set G_{ε,ν} ⊂ G for which inf_{g_{j}∈G_{ε,ν}} ∫ |g − g_{j}|^{p} dν < ε^{p} for

all functions g ∈ G. (Here the set G_{ε,ν} may depend on the measure ν, but its cardinality is bounded by a number depending only on ε.)

Now we formulate the following Theorem 4.1. Let us have a sequence of iid. random variables ξ_{1}, …, ξ_{n}, n ≥ 2, taking values on a measurable space (X, X) with some distribution μ together with an L_{2}-dense class F of functions of countable cardinality with some parameter D and exponent L ≥ 1 on the space (X, X) which satisfies the conditions

∥f∥_{∞} = sup_{x∈X} |f(x)| ≤ 1,  for all f ∈ F  (4.1)
∥f∥_{2}^{2} = ∫ f^{2}(x)μ( dx) ≤ σ^{2}  for all f ∈ F  (4.2)

with some constant σ > 0, and

∫ f(x)μ( dx) = 0 for all f ∈ F  (4.3)

Define the normalized partial sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{k=1}^{n} f(ξ_{k}) for all f ∈ F and introduce

the number β = max (\frac{log D}{log n} , 0), where D is the parameter of the L_{2}-dense class F.

There exist some constants C > 0, α > 0 and M > 0 such that the supremum of the normalized random sums S_{n}(f), f ∈ F, satisfies the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ CD exp \{−α (\frac{u}{σ} )^{2}\}

if \sqrt{n}σ^{2} ≥ u ≥ \sqrt{M}(L + β)^{3/4}σ log^{1/2} \frac{2}{σ} ,

(4.4)

with the number β defined in this theorem, and the numbers D and L in formula (4.4) agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of F can be weakened. For this goal we introduce the notion of countable approximability. For the sake of later applications it will be formulated more generally than needed in the present context.

Definition of countably approximable classes of random variables. Let a class of random variables U(f), f ∈ F, indexed by a class of functions on a measure space (Y, Y) be given. We say that this class of random variables U(f), f ∈ F, is countably approximable if there is a countable subset F′ ⊂ F such that for all numbers u > 0 the sets A(u) = {ω: sup_{f∈F} |U(f)(ω)| ≥ u} and B(u) = {ω: sup_{f∈F′} |U(f)(ω)| ≥ u} satisfy the

identity P (A(u) \ B(u)) = 0.

Clearly, B(u) ⊂ A(u). In the above definition we demanded that for all u > 0 the set B(u) should be almost as large as A(u). The following corollary of Theorem 4.1 holds.

Corollary of Theorem 4.1. Let a class of functions F satisfy the conditions of Theorem 4.1 with the only exception that instead of the condition about the countable cardinality of F it is assumed that the class of random variables S_{n}(f), f ∈ F, is countably approximable. Then the random variables S_{n}(f), f ∈ F, satisfy relation (4.4).

This corollary can be simply proved, we only have to apply Theorem 4.1 for the class F′. To do this we have to show that if F is an L_{2}-dense class with some parameter D and exponent L, and F′ ⊂ F, then F′ is also an L_{2}-dense class with the same exponent L, only with a possibly different parameter D′.

To prove this statement let us choose for all numbers 1 ≥ ε > 0 and probability measures ν on (Y, Y) some functions f_{1}, …, f_{m} ∈ F with m ≤ D (\frac{ε}{2})^{−L} elements, such

that the sets D_{j} = \{f: ∫ |f − f_{j}|^{2} dν ≤ (\frac{ε}{2})^{2}\} satisfy the relation ⋃_{j=1}^{m} D_{j} = Y. For

all sets D_{j} for which D_{j} ∩ F′ is non-empty choose a function f′_{j} ∈ D_{j} ∩ F′. In such a way we get a collection of functions f′_{j} from the class F′ containing at most 2^{L}Dε^{−L} elements which satisfies the condition imposed for L_{2}-dense classes with exponent L and parameter 2^{L}D for this number ε and measure ν.

Given a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} taking values on (X, X) let us introduce their empirical distribution on (X, X) as

μ_{n}(A)(ω) = \frac{1}{n}# \{j: 1 ≤ j ≤ n, ξ_{j}(ω) ∈ A\} , A ∈ X ,  (4.5)

and define for all measurable (and integrable) functions f the (random) integral

J_{n}(f) = J_{n,1}(f) = \sqrt{n} ∫ f(x)(μ_{n}( dx) − μ( dx)).  (4.6)

Clearly J_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n}(f(ξ_{j}) − Ef(ξ_{j})) = S_{n}(f̄) with f̄(x) = f(x) − ∫ f(x)μ( dx).

It is not difficult to see that sup_{x∈X} |f̄(x)| ≤ 2 if sup_{x∈X} |f(x)| ≤ 1, ∫ f̄(x)μ( dx) = 0,

∫ f̄^{2}(x)μ( dx) ≤ ∫ f^{2}(x)μ( dx), if F is an L_{2}-dense class of functions with parameter D and exponent L, then the class of functions F̄ consisting of the functions f̄(x) = f(x) − ∫ f(x)μ( dx), f ∈ F, is an L_{2}-dense class of functions with parameter 2^{L}D and exponent L, since ∫ (f̄− ḡ)^{2} dμ ≤ ε if f, g ∈ F, and ∫ (f − g)^{2} dμ ≤ (\frac{ε}{2})^{2}. Hence Theorem 4.1 implies the following result which can be considered as its version reformulated for integrals with respect to normalized empirical measures.

Theorem 4.1′. Let us have a sequence of iid. random variables ξ_{1}, …, ξ_{n}, n ≥ 2, with distribution μ on a measurable space (X, X) together with some class of functions F on this space which satisfy the conditions of Theorem 4.1 with the possible exception of condition (4.3). Then the estimate (4.4) remains valid if we replace the random sums S_{n}(f) in it by the random integrals J_{n}(f) defined in (4.6). Moreover, similarly to the corollary of Theorem 4.1, the countable cardinality of the set F can be replaced by the condition that the class of random variables J_{n}(f), f ∈ F, is countably approximable.

All finite dimensional distributions of the set of random variables S_{n}(f), f ∈ F, converge to a Gaussian field Z(f), f ∈ F, as n → ∞ with expectation EZ(f) = 0 and correlation EZ(f)Z(g) = ∫ f(x)g(x)μ( dx), f, g ∈ F. (Here and in the subsequent part of the paper a collection of random variables indexed by some set of parameters will be called a Gaussian field if for all finite subsets of these parameters the random variables indexed by this finite set are jointly Gaussian.) Hence we can expect that the random variables of a Gaussian field with such properties satisfy a result similar to Proposition 4.1. The following result can be considered as the Gaussian counterpart of Theorem 4.1.

Theorem 4.2. Let us fix some probability measure μ on a measurable space (X, X) together with a countable set F of square integrable functions with respect to the measure μ such that there exists a parameter D > 0 and exponent L ≥ 1 with the following property: For all ε > 0 there exist m ≤ Dε^{−L} functions f_{j} = f_{j}(ε) ∈ F, 1 ≤ j ≤ m, such that for all f ∈ F inf_{1≤j≤m} ∫ (f_{j}(x) − f(x))^{2}μ( dx) < ε^{2}. Let us also assume that

the class of functions F satisfies condition (4.2) with some 1 ≥ σ > 0. Let us consider a Gaussian field Z(f), f ∈ F, such that EZ(f) = 0, EZ(f)Z(g) = ∫ f(x)g(x)μ( dx), f, g ∈ F.

Then there exist some constants C > 0 and M > 0 (for instance C = 4 and M = 16 can be chosen) such that the inequality

P (sup_{f∈F} |Z(f)| ≥ u) ≤ C(D + 1) exp \{−\frac{1}{256} (\frac{u}{σ} )^{2}\} if u ≥ ML^{1/2}σ log^{1/2} \frac{2}{σ} (4.7)

holds with the parameter D and exponent L introduced in this theorem.

In the inequalities of the above results I did not try to find the best possible universal constants. One could choose for instance the coefficient \frac{1−ε}{2} with arbitrary small ε > 0 instead of the coefficient \frac{1}{256} in the exponent at the right-hand side of formula (4.7) if the other universal constants C > 0 and M > 0 are chosen sufficiently large in this inequality. This means that in the bound (4.7) we can get an estimate with an almost as good exponential term as in the estimate of the probability P (Z(f) > u) for a single Gaussian random variable Z(f) with EZ(f) = 0, Var Z(f) = σ^{2}. Similarly, the constant α > 0 can be chosen as α = \frac{1−ε}{2} with arbitrary small ε > 0 in formula (4.4).

The condition about the countable cardinality of the set F in Theorem 4.2 could be weakened similarly to Theorem 4.1. But here I omit the discussion of this question, since Theorem 4.2 was only introduced for the sake of a comparison between the Gaussian and non-Gaussian case. An essential difference between Theorems 4.1 and 4.2 is that in Theorem 4.1 the condition was imposed that the class of functions F has to be L_{2}-dense, while in Theorem 4.2 only a weaker version of this property was needed. In that result we only demanded that there exists a relatively small subset of F dense in the L_{2}(μ) norm. It may demand some explanation why the L_{2}-density property was imposed in Theorem 4.1, a property where also such probability measures ν are considered which seem to have no relation to the original problem. But as we shall see, the proof of Theorem 4.1 contains a conditioning argument where new conditional measures appear and the L_{2}-density property is needed to work with them. One would also like to know some results which enable us to check when this condition holds. In the next section we shall discuss a popular notion, the notion of Vapnik–Červonenkis classes and show that a Vapnik–Červonenkis class of functions bounded by 1 is L_{2}-dense.

Another difference between Theorems 4.1 and 4.2 is that the conditions of formula (4.4) contain the upper bound nσ^{2} > \sqrt{n}u, and no such condition is imposed in formula (4.7). This difference can be simply explained, since as we have seen in Section 3 in the case nσ^{2} = Var (\sqrt{n}S_{n}) ≪ \sqrt{n}u we can guarantee only a weak non-Gaussian type estimate for the single probabilities P (\sqrt{n}S_{n}(f) > \sqrt{n}u), f ∈ F. It has a similar reason why condition (4.1) about the supremum of the functions f ∈ F appeared in Theorems 4.1 and 4.1′, and no such condition was needed in Theorem 4.2.

The lower bounds for the level u were imposed in formulas (4.4) and (4.7) because of a similar reason. To understand why such a condition is needed in formula (4.7) let us consider the following example. Take a Wiener process W (t), 0 ≤ t ≤ 1, define the functions f_{s,t}(·) on the interval [0, 1] by the formula f_{s,t}(u) = 1 if s ≤ u ≤ t, f_{s,t}(u) = 0 if 0 ≤ u < s or t < u ≤ 1, and put Z(f_{s,t}) = ∫ f_{s,t}(u)W ( du) = W (t) − W (s). Given some σ > 0 let us consider the class of functions F_{σ} = {f_{s,t}: ∫ f_{s,t}^{2}(u) du = t − s ≤ σ^{2}, s and t are rational numbers}. It is not difficult to see that the above example sat-

isfies the conditions of Theorem 4.2. It is natural to expect that P ( sup_{f∈F_{σ}} Z(f) > u) ≤

e^{−const. (u/σ)^{2}}. However, this relation does not hold if u = u(σ) < (1 − ε)\sqrt{2}σ log^{1/2} \frac{1}{σ}

with some ε > 0. In such cases P ( sup_{f∈F_{σ}} Z(f) > u) → 1, as σ → 0. This can be

proved relatively simply with the help of the estimate P (Z(f_{s,t}) > u(σ)) ≥ const. σ^{1−ε} if |t − s| = σ^{2} and the independence of the random integrals Z(f_{s,t}) if the functions f_{s,t} are indexed by such pairs (s, t) for which the intervals (s, t) are disjoint. This means that in this example formula (4.7) holds only under the condition u ≥ Mσ log^{1/2} \frac{1}{σ} with M = \sqrt{2}.

Some additional work would show that a similar picture arises in the model where we consider the integrals J_{n}(f_{s,t}) of the functions from the same the class F_{σ} with respect to the normalized empirical measure of a sample of size n with uniform distribution on the interval [0, 1] instead of a Wiener process. In this example we have to impose the condition \sqrt{n}u ≥ M\sqrt{n}σ log^{1/2} \frac{1}{σ} with M = \sqrt{2} for the validity of relation (4.4). At a heuristic level it is clear that in the case of a class F with a large exponent L we have to put a larger coefficient of \sqrt{n}σ log^{1/2} \frac{2}{σ} in the condition of formula (4.4) for the validity of Theorem 4.1 or 4.1′, and a similar statement can be told about the condition (4.7) in Theorem 4.2. (I did not try to find the best possible coefficients in the conditions of relations (4.4) and (4.7), they could be improved considerably.)

In Theorem 4.1 (and in its version 4.1′) it was demanded that the class of functions F should be countable. Later this condition was replaced by a weaker condition about countable approximability. By restricting our attention to countable or countably approximable classes we could avoid some unpleasant measure theoretical problems which would have arisen if we had worked with the supremum of non-countable number of random variables which may be non-measurable. There are some papers where possibly non-measurable models are also considered with the help of some rather deep results of the analysis and measure theory. Actually, the problem we met here is the natural analog of an important problem in the theory of the stochastic processes about the smoothness property of the trajectories of an appropriate version of a stochastic process which we can get by exploiting our freedom to change all random variables on a set of probability zero.

The study of the problem in this work is simpler in one respect. Here the set of random variables S_{n}(f)(ω) or J_{n}(f)(ω), f ∈ F, are constructed directly with the help of the underlying random variables ξ_{1}(ω), …, ξ_{n}(ω) for all ω ∈ Ω separately. We are interested in when the sets of random variables constructed in this way are countably approximable, i.e. we are not looking for a possibly different, better version of them with the same finite dimensional distributions. In the next simple Lemma 4.3 we give a sufficient condition for countable approximability. Its condition can be interpreted as a smoothness type condition for the trajectories of a stochastic process indexed by the functions f ∈ F.

Lemma 4.3. Let a class of random variables U(f), f ∈ F, indexed by some set F of functions on a space (Y, Y) be given. If there exists a countable subset F′ ⊂ F of the set F such that the sets A(u) = {ω: sup_{f∈F} |U(f)(ω)| ≥ u} and B(u) = {ω: sup_{f∈F′} |U(f)(ω)| ≥

u} introduced for all u > 0 in the definition of countable approximability satisfy the relation A(u) ⊂ B(u − ε) for all u > ε > 0, then the class of random variables U(f), f ∈ F, is countably approximable.

The above property holds if for all f ∈ F, ε > 0 and ω ∈ Ω there exists a function f̄ = f̄(f, ε, ω) ∈ F′ such that |U(f̄)(ω)| ≥ |U(f)(ω)| − ε.

Proof of Lemma 4.3. If A(u) ⊂ B(u − ε) for all ε > 0, then P^{∗}(A(U) \ B(u)) ≤ lim_{ε→0} P (B(u−ε)\B(u)) = 0, where P^{∗}(X) denotes the outer measure of a not necessarily

measurable set X ⊂ Ω, since ⋂_{ε→0} B(u − ε) = B(u), and this is what we had to prove. If

ω ∈ A(u), then for all ε > 0 there exists some f = f(ω) ∈ F such that |U(f)(ω)| > u− \frac{ε}{2}. If there exists some f̄ = f̄(f, \frac{ε}{2}, ω), f ∈ F′ such that |U(f̄)(ω)| ≥ |Uf(ω)| − \frac{ε}{2}, then |U(f̄)(ω)| > u − ε, and ω ∈ B(u − ε). This means that A(u) ⊂ B(u − ε).

The question about countable approximability also appears in the case of multiple random integrals. To avoid some repetition we prove a result which also covers such cases. For this goal first we introduce the notion of multiple integrals with respect to a normalized empirical measure.

Given a measurable function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) and a sequence of independent random variables ξ_{1}, …, ξ_{n} with some distribution μ on the space (X, X) define the integral J_{n,k}(f) of the function f with respect to the k-fold product of the normalized empirical measure μ_{n} introduced in (4.5) by the formula

J_{n,k}(f) = \frac{n^{k/2}}{k!} ∫ ′ f(x_{1}, …, x_{k})(μ_{n}( dx_{1}) − μ( dx_{1})) … (μ_{n}( dx_{k}) − μ( dx_{k})),

where the prime in ∫′ means that the diagonals x_{j} = x_{l}, 1 ≤ j < l ≤ k,

are omitted from the domain of integration.  (4.8)

Lemma 4.3 enables us to prove that certain classes of random variables J_{n,k}(f), f ∈ F, indexed by some set of functions f ∈ F of k variables are countably approximable. I present an example which is very important in certain applications.

Let us consider the case when X = R^{s}, the s-dimensional Euclidean space with some s ≥ 1, and given some u = (u^{(1)}, …, u^{(s)}) ∈ R^{s}, v = (v^{(1)}, …, v^{(s)}) ∈ R^{s} such that u ≤ v, i.e. u^{(j)} ≤ v^{(j)} for all 1 ≤ j ≤ s, let B(u, v) denote the s-dimensional rectangle B(u, v) = {z: u ≤ z ≤ v}. Let us fix some function f(x_{1}, …, x_{k}), sup |f(x_{1}, …, x_{k})| ≤ 1, on the space (X^{k}, X^{k}) = (R^{ks}, B^{ks}), where B^{t} denotes the Borel σ-algebra on the Euclidean space R^{t} together with some probability measure μ on (R^{s}, B^{s}). For all vectors (u_{1}, …, u_{k}), (v_{1}, …, v_{k}) such that u_{j}, v_{j} ∈ R^{s} and u_{j} ≤ v_{j}, 1 ≤ j ≤ k, let us define the function f_{u_{1},...,u_{k},v_{1},...,v_{k}} which equals the function f on the rectangle [u_{1}, v_{1}] × ⋯ [u_{k}, v_{k}], and it is zero outside of this rectangle.

Let us consider a sequence of i.i.d. random variables ξ_{1}, …, ξ_{n} taking value in the space (R^{s}, B^{s}) with distribution μ and define the empirical measure μ_{n} and random integrals J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}}) by formulas (4.5) and (4.8), for all vectors (u_{1}, …, u_{k}), (v_{1}, …, v_{k}) such that u_{j}, v_{j} ∈ R^{s} and u_{j} ≤ v_{j}, 1 ≤ j ≤ k, with the above defined functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}. The following result will be proved.

Lemma 4.4. Let us take n iid. random variables ξ_{1}, …, ξ_{n} with values in the space (R^{s}, B^{s}). Let us define with the help of their distribution μ and the empirical distribution μ_{n} determined by them the class of random variables J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}}) introduced in formula (4.8), where the class of kernel functions F in these integrals consists of all functions f_{u_{1},...,u_{k},v_{1},...,v_{k}} ∈ (R^{sk}, B^{sk}), u_{j}, v_{j} ∈ R^{s}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, defined in the last but one paragraph. This class of random variables J_{n,k}(f), f ∈ F, is countably approximable.

Proof of Lemma 4.4. We shall prove that the definition of countable approximability is satisfied in this model if the class of functions F′ consists of those functions f_{u_{1},...,u_{k},v_{1},...,v_{k}}, u_{j} ≤ v_{j}, 1 ≤ j ≤ k, for which all coordinates of the vectors u_{j} and v_{j} are rational numbers.

Given some function f_{u_{1},...,u_{k},v_{1},...,v_{k}}, a real number 1 > ε > 0 and ω ∈ Ω let us choose a function f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}} ∈ F′ determined with some vectors ū_{j} = ū_{j}(ε, ω), v̄_{j} = v̄_{j}(ε, ω) 1 ≤ j ≤ k, with rational coordinates ū_{j} ≤ u_{j} < v_{j} ≤ v̄_{j} such that the sets K_{j} = B(ū_{j}, v̄_{j}) \ B(u_{j}, v_{j}) satisfy the relations μ(K_{j}) ≤ ε2^{−2k+1}n^{−k/2}, and ξ_{l}(ω) ∉ K_{j} for all j = 1, …, k and l = 1, …, n. Let us show that

|J_{n,k}(f_{ū_{1},...,ū_{k},v̄_{1},...,v̄_{k}})(ω) − J_{n,k}(f_{u_{1},...,u_{k},v_{1},...,v_{k}})(ω)| ≤ ε.  (4.9)

Lemma 4.3 (with the choice U(f) = J_{n,k}(f)) and relation (4.9) imply Lemma 4.4.

Relation (4.9) holds, since the expression in it can be written as the sum of the 2^{k}−1 integrals of the function f with respect to the k-fold product of the measure \sqrt{n}(μ_{n} −μ) on the domains D_{1} × ⋯ × D_{k} with the omission of the diagonals x_{j} = xj̄, 1 ≤ j, j̄ ≤ k, j ≠ j̄, where D_{j} is either the set K_{j} or B(u_{j}, v_{j}) and D_{j} = K_{j} for at least one index j. It is enough to show that the absolute value of all these integrals is less than ε2^{−k}. This follows from the observations that |f(x_{1}, …, x_{k})| ≤ 1, \sqrt{n}(μ_{n} − μ)(K_{j}) = −\sqrt{n}μ(K_{j}), μ(K_{j}) ≤ ε2^{−2k+1}n^{−k/2}, and the total variation of the signed measure \sqrt{n}(μ_{n} − μ) (restricted to the set B(u_{j}, v_{j})) is less than 2\sqrt{n}.

Let us discuss the relation of the results in this section to an important result, the so-called fundamental theorem of the mathematical statistics. In that problem a sequence of independent random variables ξ_{1}(ω), …, ξ_{n}(ω) is considered with distribution function F (x), the empirical distribution function F_{n}(x) = F_{n}(x, ω) = \frac{1}{n}#\{j: 1 ≤ j ≤ n, ξ_{j}(ω) < x} is introduced, and the difference F_{n}(x) − F (x) is considered. This result states that sup_{x} |F_{n}(x) − F (x)| tends to zero with probability one.

Observe that sup_{x} |F_{n}(x) − F (x)| = n^{−1/2} sup_{f∈F} |J_{n}(f)|, where F consists of the func-

tions f_{x}(·), x ∈ R^{1}, defined by the relation f_{x}(u) = 1 if u < x, and f_{x}(u) = 0 if u ≥ x.

Theorem 4.1′ yields an estimate for the probabilities P (sup_{f∈F} |J_{n}(f)| > u). We have

seen that the above class of functions F is countably approximable. The results of the next section imply that this class of functions is also L_{2}-dense. Otherwise it is not difficult to check this property directly. Hence we can apply Theorem 4.1 to the above

defined class of functions with σ = 1, and it yields that P (n^{−1/2} sup_{f∈F} |J_{n}(f)| > u) ≤

e^{−Cnu^{2}} if 1 ≥ u ≥ C̄n^{−1/2} with some universal constants C > 0 and C̄ > 0. (The condition 1 ≥ u can actually be dropped.) The application of this estimate for the numbers ε > 0 together with the Borel-Cantelli lemma imply the fundamental theorem of the mathematical statistics.

In short, the results of this section yield more information about the closeness the empirical distribution function F_{n} and distribution function F than the fundamental theorem of the mathematical statistics. Moreover, since these results can also be applied for other classes of functions they yield useful information about the closeness of the probability measure μ and empirical measure μ_{n}.

5. Vapnik–Červonenkis classes and L_{2}-dense classes of functions

In this section the most important notions and results will be presented about Vapnik– Červonenkis classes, and it will be explained how they help to show in some important cases that certain classes of functions are L_{2}-dense. Some proofs are put in the Appendix.

First I recall the following notions.

Definition of Vapnik-Červonenkis classes of sets and functions. Let a set S be given, and let us select a class D consisting of certain subsets of this set S. We call D a Vapnik–Červonenkis class if there exist two real numbers B and K such that for all positive integers n and subsets S_{0}(n) = {x_{1}, …, x_{n}} ⊂ S of cardinality n of the set S the collection of sets of the form S_{0}(n)∩D, D ∈ D, contains no more than Bn^{K} subsets of S_{0}(n). We shall call B the parameter and K the exponent of this Vapnik–Červonenkis class.

A class of real valued functions F on a space (Y, Y) is called a Vapnik–Červonenkis class if the collection of graphs of these functions is a Vapnik–Červonenkis class, i.e. if the sets A(f) = {(y, t): y ∈ Y, min(0, f(y)) ≤ t ≤ max(0, f(y))}, f ∈ F, constitute a Vapnik–Červonenkis class of subsets of the product space S = Y × R^{1}.

The following result which was first proved by Sauer is of fundamental importance in the theory of Vapnik–Červonenkis classes. Its proof is given in the Appendix.

Theorem 5.1 (Sauer’s lemma). Let a set S be given together with a class D of subsets of this set S. Fix some subset S_{0} = S_{0}(n) of the set S containing n point and consider the class of subsets D(S_{0}) = {S_{0} ∩ D: D ∈ D} of S_{0} consisting of the intersections of the set S_{0} with the elements of the class D. If there is some positive integer k such that all subsets F ⊂ S_{0} of cardinality k have at least one “hidden” subset not contained in the collection of sets D(S_{0}, F) = {F ∩ B; B ∈ D(S_{0})}, then D(S_{0}) contains at most \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} subsets of S.

Theorem 5.1 has the remarkable consequence that if there exists some integer k such that for all subsets S_{0}(k) of cardinality k of the set S the number of sets of the form S_{0}(k) ∩ D, D ∈ D, is less than 2^{k}, (i.e. not all subsets of S_{0}(k) can be represented in this form,) then S_{0}(n) ∩ D has at most \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} elements for all subsets S_{0}(n) of the set S with n ≥ k elements, since in this case the conditions of Theorem 5.1 hold for all n ≥ k and subset S_{0}(n) ⊂ S of S of cardinality n and this number k. This means that in this case D is a Vapnik-Červonenkis class. It can be proved that \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} ≤ 1.5\frac{n^{k−1}}{(k−1)!} if n ≥ k + 1, and this relation enables us to give an explicit estimate on the exponent and parameter of this Vapnik–Červonenkis class. Hence we have to check a seemingly much weaker property to show that a class of subsets of a set S is a Vapnik–Červonenkis class. Moreover, Theorem 5.1 implies that there are two cases. Either there is some set S_{0}(n) of cardinality n for all integers n such that D(S_{0}(n)) contains all subsets of S_{0}(n) or sup_{S_{0}⊂S,|S_{0}|=n} |D(S_{0})| tends to infinity

in polynomial order as n → ∞, where |S_{0}| and |D(S_{0})| denotes the cardinality of S_{0} and D(S_{0}).

The upper bound given for |D(S_{0})| in Theorem 5.1 appears in a natural way. If D(S_{0}) consists of the subsets of S_{0} of cardinality less than or equal to k − 1, then the above sum equals |D(S_{0})|. In such a case the conditions of Theorem 5.1 are satisfied, and the proof of Theorem 5.1 shows that this is the extreme case, this is the largest class of sets S_{0}(n) ∩ D satisfying Theorem 5.1.

The following Theorem 5.2, an important result of Richard Dudley, states that a Vapnik–Červonenkis class of functions bounded by 1 is an L_{1}-dense class of functions.

Theorem 5.2. Let f(y), f ∈ F, be a Vapnik–Červonenkis class of real valued functions on some measurable space (Y, Y) such that sup_{y∈Y} |f(y)| ≤ 1 for all f ∈ F. Then F is an

L_{1}-dense class of functions on (Y, Y). More explicitly, if F is a Vapnik–Červonenkis class with parameter B ≥ 1 and exponent K > 0, then it is an L_{1}-dense class with exponent L = 2K and parameter D = CB^{2}(4K)^{2K} with some universal constant C > 0.

Proof of Theorem 5.2. Let us fix some probability measure ν on (Y, Y) and a real number 1 ≥ ε > 0. We are going to show that the cardinality of any finite set D(ε, ν) = {f_{1}, …, f_{M}} ⊂ F such that ∫ |f_{j} − f_{k}| dν ≥ ε if j ≠ k, f_{j}, f_{k} ∈ D(ε, ν) has a cardinality M ≤ Dε^{−L} with some D > 0 and L > 0. This implies that F is an L_{1}-dense class with parameter D and exponent L. Indeed, let us take a maximal subset D̄(ε, ν) = {f_{1}, …, f_{M}} such that the L_{1}(ν) distance of any two functions in this subset is at least ε. Maximality means in this context that no function f_{M+1} can be attached to D̄(ε, ν) without violating this condition. If we show that M ≤ Dε^{−L}, then this means that D̄(ε, ν) is an ε-dense subset of F in the space L_{p}(Y, Y, ν) with no more than Dε^{−L} elements.

In the estimation of the cardinality M of D(ε, ν) we exploit the Vapnik–Červonenkis class property of F in the following way. Let us choose relatively few p points (y_{l}, t_{l}), y_{l} ∈ Y, −1 ≤ t_{l} ≤ 1, 1 ≤ l ≤ p, in the space (Y × [−1, 1]) in such a way that the set S_{0}(p) = {(y_{l}, t_{l}), 1 ≤ l ≤ p} and graphs A(f_{j}) = {(y, t): y ∈ Y, min(0, f_{j}(y)) ≤ t ≤ max(0, f_{j}(y))}, f ∈ F, f_{j} ∈ D(ε, ν) have the property that all sets A(f_{j}) ∩ S_{0}(p), 1 ≤ j ≤ M, are different. Then the Vapnik–Červonenkis class property of F implies that M ≤ Bp^{K}. Hence if we can construct a set S_{0}(p) with the above property with a relatively small number p, then we get a useful estimate on M. Such a set S_{0}(p) will be given by means of the following random construction.

Let us choose the p points (y_{l}, t_{l}), 1 ≤ l ≤ p, of the (random) set S_{0}(p) independently of each other in such a way that the coordinate y_{l} is chosen with distribution ν on (Y, Y) and the coordinate t_{l} with uniform distribution on the interval [−1, 1] independently of y_{l}. (The number p will be chosen later.) Let us fix some indices 1 ≤ j, k ≤ M and estimate the probability that the sets A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree, where A(f) denotes the graph of the function f. Consider the symmetric difference A(f_{j})ΔA(f_{k}) of the sets A(f_{j}) and A(f_{k}). The sets A(f_{j})∩S_{0}(p) and A(f_{k})∩S_{0}(p) agree if and only if (y_{l}, t_{l}) ∉ A(f_{j})ΔA(f_{k}) for all (y_{l}, t_{l}) ∈ S_{0}(p). Let us observe that for a fixed l P ((y_{l}, t_{l}) ∈ A(f_{j})ΔA(f_{k})) = \frac{1}{2}(ν×λ)(A(f_{j})ΔA(f_{k})) = \frac{1}{2} ∫ |f_{j}−f_{k}| dν ≥ \frac{ε}{2}, where λ denotes the Lebesgue measure. This implies that the probability that A(f_{j}) ∩ S_{0}(p) and A(f_{k}) ∩ S_{0}(p) agree can be bounded from above by (1 − \frac{ε}{2})^{p} ≤ e^{−pε/2}. Hence the probability that all sets A(f_{j}) ∩ S_{0}(p) are different is greater than 1 − \binom{M}{2}e^{−pε/2} ≥ 1 − \frac{M^{2}}{2} e^{−pε/2}. Choose p such that \frac{7}{4}e^{pε/2} > e^{(p+1)ε/2} > M^{2} ≥ e^{pε/2}. Then the above probability is greater than \frac{1}{8}, and there exists some set S_{0}(p) with the desired property.

The inequalities M ≤ Bp^{K} and M^{2} ≥ e^{pε/2} imply that M ≥ e^{εM^{1/K}/4B^{1/K}}, i.e. \frac{log M^{1/K}}{M^{1/K}} ≥ \frac{ε}{4KB^{1/K}}. As \frac{log M^{1/K}}{M^{1/K}} ≤ CM^{−1/2K} for M ≥ 1 with some universal constant C > 0, this estimate implies that Theorem 5.2 holds with the exponent L and parameter D given in its formulation.

Let us observe that if F is an L_{1}-dense class of functions on a measure space (Y, Y) with some exponent L and parameter D, and also the inequality sup_{y∈Y} |f(y)| ≤ 1 holds

for all f ∈ F, then F is an L_{2}-dense class of functions with exponent 2L and parameter D2^{L}. Indeed, if we fix some measure ν on (Y, Y) together with a number 1 ≥ ε > 0, and D(ε, ν) = \{f_{1}, …, f_{M}\} is an \frac{ε^{2}}{2} -dense set of F in the space L_{1}(Y, Y, ν), M ≤ 2^{L}Dε^{−2L} then for any f ∈ F we can choose some f_{j} ∈ D(ε, ν) such that ∫ (f − f_{j})^{2} dν ≤ 2 ∫ |f − f_{j}| dν ≤ ε^{2}. This means that F is really an L_{2}-dense class with the given exponent and parameter.

It is not easy to check whether a collection of subsets D of a set S is a Vapnik– Červonenkis class even with the help of Theorem 5.1. Therefore the following Theorem 5.3 which enables us to construct many non-trivial Vapnik–Červonenkis classes is of special interest. Its proof is also put in the Appendix.

Theorem 5.3. Let us consider a k-dimensional subspace G_{k} of the linear space of real valued functions defined on a set S, and define the level-set A(g) = {s: s ∈ S, g(s) ≥ 0} for all functions g ∈ G_{k}. Take the class of subsets D = {A(g): g ∈ G_{k}} of the set S consisting of the above introduced level sets. All subsets S_{0} = S_{0}(k + 1) ⊂ S of cardinality k + 1 has a “hidden” subset which is not contained in the class of subsets D(S_{0}) = {S_{0} ∩ D: D ∈ D} of S_{0} introduced in Theorem 5.1. By Theorem 5.1 this property implies that the class of sets D is a Vapnik–Červonenkis class.

Theorem 5.3 enables us to construct many interesting Vapnik–Červonenkis classes. Thus for instance the class of all half-spaces in a Euclidean space, the class of all ellipses in the plane, or more generally the level sets of k-order algebraic functions with a fixed number k constitute a Vapnik–Červonenkis class. It can be proved that if C and D are Vapnik–Červonenkis classes of subsets of a set S, then also their intersection C ∩ D = {C ∩ D: C ∈ C, D ∈ D}, their union C ∪ D = {C ∪ D: C ∈ C, D ∈ D} and complementers C^{c} = {S \ C: C ∈ C} are Vapnik–Červonenkis classes. These results are less important for us and their proofs will be omitted. We are interested in Vapnik– Červonenkis classes not for their own sake. We are going to study L_{2}-dense classes of functions, and Vapnik–Červonenkis classes make possible to find some examples. Indeed, Theorem 5.2 implies that if D is a Vapnik–Červonenkis class of subsets of a set S, then their indicator functions constitute an L_{1}-dense, hence also an L_{2}-dense class of functions. Then the results of Lemma 5.4 formulated below enable us to construct new L_{2}-dense class of functions. The description of L_{2}-dense classes of functions are interesting for us, because they appear in the conditions of the results in Section 4.

Lemma 5.4. Let G be an L_{2}-dense class of functions on some space (Y, Y) whose absolute values are bounded by one, and let f be a function on (Y, Y) also with absolute value bounded by one. Then f ·G = {f ·g: g ∈ G} is also an L_{2}-dense class of functions. Let G_{1} and G_{2} be two L_{2}-dense classes of functions on some space (Y, Y) whose absolute values are bounded by one. Then the classes of functions G_{1}+G_{2} = {g_{1}+g_{2} : g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, G_{1} · G_{2} = {g_{1}g_{2} : g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, min(G_{1}, G_{2}) = {min(g_{1}, g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}}, max(G_{1}, G_{2}) = {max(g_{1}, g_{2}): g_{1} ∈ G_{1}, g_{2} ∈ G_{2}} are also L_{2}-dense. If G is an L_{2}-dense class of functions, and G′ ⊂ G, then G′ is also an L_{2}-dense class.

The proof of Lemma 5.4 is rather straightforward. One has to observe for instance that if g_{1}, ḡ_{1} ∈ G_{1}, g_{2}, ḡ_{2} ∈ G_{2} then | min(g_{1}, g_{2}) − min(ḡ_{1}, ḡ_{2})| ≤ |g_{1} − ḡ_{1})| + |g_{2} − ḡ_{2}|, hence if g_{1,1}, …, g_{1,M_{1}} is an \frac{ε}{2}-dense subset of G_{1} and g_{2,1}, …, g_{2,M_{2}} is an \frac{ε}{2}-dense subset of G_{2} in the space L_{2}(Y, Y, ν) with some probability measure ν, then the functions min(g_{1,j}, g_{2,k}), 1 ≤ j ≤ M_{1}, 1 ≤ k ≤ M_{2} constitute an ε-dense subset of min(G_{1}, G_{2}) in L_{2}(Y, Y, ν). The last statement of Lemma 5.4 is proved after the Corollary of Theorem 4.1. The details are left to the reader.

The above results enable us to find some interesting classes of L_{2}-dense classes of functions. In particular, the indicator functions of Vapnik-Červonenkis class of sets is an L_{2}-dense class of functions, and then Lemma 5.4 enables us to construct new classes of L_{2}-dense classes of functions with their help. It is not difficult to see with the help of these results for instance that the random variables considered in Lemma 4.4 are not only countably approximable, but the class of functions f_{u_{1},...,u_{k},v_{1},...,v_{k}} taking part in their definition is L_{2}-dense.

6. The proof of Theorems 4.1 and 4.2 on the supremum of random sums

This section contains the proof of some results which can be proved by means of a simple but useful method, the so-called chaining argument. This method enables us to prove Theorem 4.2 completely, but it only helps to reduce Theorem 4.1 to a slightly simpler statement presented in Proposition 6.1. We also formulate another result in Proposition 6.2 and show that these two propositions together imply Theorem 4.1. The proof of Proposition 6.2 which is based on a symmetrization argument is left to the next section. The method of proof of Theorem 4.2 does not suffice in itself to prove Theorem 4.1, because we have relatively weak estimates about the distribution of sums of independent random variables with small variances. This does not allow to follow the chaining argument in the proof of Theorem 4.1 up to the end, we have to stop at a point. In such a way we only get a seemingly weak result, but as it turns out this is the result we need to cover that part of Theorem 4.1 which cannot be handled by means of the symmetrization method applied in the proof of Proposition 6.2. First we prove Theorem 4.2.

Proof of Theorem 4.2. Let us list the elements of F as {f_{0}, f_{1}, … } = F, and choose for all p = 0, 1, 2, … a set of functions F_{p} = {f_{a(p,1)}, …, f_{a(p,m_{p})}} ⊂ F with m_{p} ≤ (D + 1) 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(p,j)})^{2} dμ ≤ 2^{−4p}σ^{2} for all

f ∈ F, and f_{p} ∈ F_{p}. For all indices a(p, j) of the functions in F_{p}, p = 1, 2, …, define a predecessor a(p−1, j′) from the indices of the set of functions F_{p−1} in such a way that the functions f_{a(p,j)} and f_{a(p−1),j′)} satisfy the relation ∫ (f_{(p,j)} −f_{(p−1,j′)})^{2} dμ ≤ 2^{−4(p−1)}σ^{2}. With the help of the behaviour of the standard normal distribution function we can write the estimates

P (A(p, j)) = P (|Z(f_{a(p,j)}) − Z(f_{a(p−1,j′)})| ≥ 2^{−(1+p)}u) ≤ 2 exp \{−\frac{2^{−2(p+1)}u^{2}}{2 · 2^{−4(p−1)}σ^{2}} \}

= 2 exp \{−\frac{2^{2p}u^{2}}{128σ^{2}} \} 1 ≤ j ≤ m_{p}, p = 1, 2, …,

and

P (B(j)) = P (|Z(f_{a(0,j)})| ≥ \frac{u}{2}) ≤ exp \{−\frac{u^{2}}{8σ^{2}} \} , 1 ≤ j ≤ m_{0}.

The above estimates together with the relation ⋃_{p=0}^{∞} F_{p} = F which implies that

{|Z(f)| ≥ u} ⊂ ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(p, j) ∪ ⋃_{s=1}^{m_{0}} B(s) for all f ∈ F yield that

P (sup_{f∈F} |Z(f)| ≥ u) ≤ P ( ⋃_{p=1}^{∞} ⋃_{j=1}^{m_{p}} A(p, j) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{∞} ∑_{j=1}^{m_{p}} P (A(p, j)) + ∑_{s=1}^{m_{0}} P (B(s))

≤ ∑_{p=1}^{∞} 2(D + 1)2^{2pL}σ^{−L} exp \{−\frac{2^{2p}u^{2}}{128σ^{2}} \} + 2(D + 1)σ^{−L} exp \{−\frac{u^{2}}{8σ^{2}} \} .

If u ≥ ML^{1/2}σ log \frac{2}{σ} with M ≥ 16 (and L ≥ 1), then

2^{2pL}σ^{−L} exp \{−\frac{2^{2p}u^{2}}{256σ^{2}} \} ≤ (\frac{1}{2})^{−2pL} σ^{−L} (\frac{σ}{2} )^{2^{2p}M^{2}L/256} ≤ 2^{−pL} ≤ 2^{−p}

for all p = 0, 1 …, hence the previous inequality implies that

P (sup_{f∈F} |Z(f)| ≥ u) ≤ 2(D + 1) ∑_{p=0}^{∞} 2^{−p} exp \{−\frac{2^{2p}u^{2}}{256σ^{2}} \} = 4(D + 1) exp \{−\frac{u^{2}}{256σ^{2}} \} .

Theorem 4.2 is proved.

With the appropriate choice of the bound of the integrals in the definition of the sets F_{p} in the proof of Theorem 4.2 and some more calculation it can be proved that the coefficient \frac{1}{256} in the exponent of the right-hand side (4.7) can be replaced by \frac{1−ε}{2} with arbitrary small ε > 0 if the remaining constants in this estimate are chosen sufficiently large.

The proof of Theorem 4.2 was based on the fact that sufficiently good estimates can be given on the probabilities P (|Z(f)−Z(g)| > u) for all f, g ∈ F and u > 0. In the case of Theorem 4.1 we only have a weaker estimate for the corresponding probabilities, we cannot give a good estimate on the distribution of the difference S_{n}(f) − S_{n}(g) if its variance is small. As a consequence the chaining argument supplies only a weaker result in this case. This result will be given in Proposition 6.1, where the supremum of the normalized random sums S_{n}(f) is estimated on a relatively dense subset of the class of functions f ∈ F in the L_{2}(μ) norm. We present another result in Proposition 6.2 which will be proved in the next section and show that Theorem 4.1 follows from these two results.

Before the formulation of Proposition 6.1 I recall an estimate which is a simple

consequence of Bernstein’s inequality: If S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) is the normalized sum of

independent, identically random variables, P (|f(ξ_{1})| ≤ 1) = 1, Ef(ξ_{1}) = 0, Ef(ξ_{1})^{2} ≤ σ^{2}, then there exists some constant α > 0 such that

P (|S_{n}(f)| > u) ≤ 2e^{−αu^{2}/σ^{2}} if 0 < u < \sqrt{n}σ^{2}.  (6.1)

We can choose α = \frac{3}{8} in this estimate, and also could present a slightly more general version of it, but such additional information would not give a real help.

Proposition 6.1. Let us have a countable L_{2}-dense class of functions F with parameter D and exponent L, L ≥ 1, on a measurable space (X, X) whose elements satisfy the conditions (4.1), (4.2) and (4.3) with some probability measure μ on (X, X) and real number 0 < σ ≤ 1. Take a sequence of independent μ-distributed random variables

ξ_{1}, …, ξ_{n}, n ≥ 2, define the random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{l=1}^{n} f(ξ_{l}), for all f ∈ F. Let

us fix some number Ā ≥ 2. For all sufficiently large numbers M ≥ M_{0} = M_{0}(Ā) the following relation holds: For all numbers u > 0 for which nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ ML log \frac{2}{σ} a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = {f_{1}, …, f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the sets D_{j} = {f: f ∈

F, ∫ |f − f_{j}|^{2} dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m, satisfy the relation ⋃_{j=1}^{m} D_{j} = F, and the normalized

partial sums S_{n}(f), f ∈ Fσ̄, n ≥ 2, satisfy the inequality

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ 4D exp \{−α (\frac{u}{10Āσ} )^{2}\} if nσ^{2} ≥ (\frac{u}{σ} )^{2} ≥ ML log \frac{2}{σ}

(6.2)

with the constants α in formula (6.1) and the exponent L and parameter D of the L_{2}- dense class F. Besides, also the inequalities \frac{1}{4} (\frac{u}{Āσ̄} )^{2} ≥ nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2} and nσ̄^{2} ≥

\frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} hold with β = max (\frac{log D}{log n} , 0), provided that also the inequality nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(L + β)^{3/2} log \frac{2}{σ} holds. (We may assume that the sample size n is sufficiently large, so the set of numbers u for which nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(L+β)^{3/2} log \frac{2}{σ} is non-empty.)

Proposition 6.1 helps to reduce the proof of Theorem 4.1 to the case when the L_{2} norm of the functions in the class F is bounded by a relatively small number σ̄. In more detail, the proof of Theorem 4.1 can be reduced to a good estimate on the distribution of the supremum of random variables sup_{f∈D_{j}} |S_{n}(f − f_{j})| for all classes D_{j},

1 ≤ j ≤ m, by means of Proposition 6.1. We also have to know that the number m of the classes D_{j} is not too large, otherwise our estimates cannot be useful.

A result formulated in Proposition 6.2 helps us to complete the proof of Theorem 4.1. It contains some parameters, and we have to fit the constants in the estimates of Propositions 6.1 and 6.2. This was the reason to introduce the rather artificial parameter Ā ≥ 2 in Proposition 6.1 and to formulate the conditions of inequality (6.2) with a number M ≥ M_{0}(Ā) instead of a number M_{0}. We want such a formulation of Proposition 6.1 in which it can achieved for any fixed number A > 0 that the relation nσ̄^{2} ≥ A log n holds, where the number σ̄ was defined in the proof of Proposition 6.1. The last two relations in Proposition 6.1 shows that this is possible if first the number Ā and then the number M ≥ M_{0}(Ā) is chosen sufficiently large. Now we formulate Proposition 6.2 and prove Theorem 4.1 with its help.

Proposition 6.2. Let us have a probability measure μ on a measurable space (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n}, n ≥ 2, and a countable, L_{2}-dense class of functions f = f(x) on (X, X) with some parameter D and exponent L ≥ 1 which satisfies conditions (4.1), (4.2) and (4.3) with some σ > 0 such that the inequality nσ^{2} > K(L + β) log n holds with an appropriate, sufficiently large universal number K ≥ 3 and β = max (0, \frac{log D}{log n} ). Then there exists some universal constant γ > 0 and threshold index A_{0} > 0 such that the random sums S_{n}(f), f ∈ F, introduced in Theorem 4.1 satisfy the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ An^{1/2}σ^{2}) ≤ e^{−γA^{1/2}nσ^{2}} if A ≥ A_{0}.  (6.3)

(A possible choice of the parameters is: K = 4, A_{0} = 2^{10} · 10^{16} and γ = \frac{1}{2}.)

I did not try to find optimal parameters in formula (6.3). Even the exponent \frac{1}{2} of A in the exponent at its right-hand side could be improved. The result of Proposition 6.2 is similar to that of Theorem 4.1. Both of them give an estimate on a

probability of the type P (sup_{f∈F} |S_{n}(f)| ≥ u). The essential difference between them

is that in Theorem 4.1 this probability is considered for u ≤ const. n^{1/2}σ^{2} while in Proposition 6.2 the case u > const. n^{1/2}σ^{2} is looked at. Let us observe that this is the case when no good Gaussian type estimate can be given for the probabilities P (S_{n}(f) ≥ u), f ∈ F. In this case Bernstein’s inequality yields the bound

P (S_{n}(f) > An^{1/2}σ^{2}) = P (∑_{l=1}^{n} f(ξ_{l}) > xV_{n}) < e^{−const. Anσ^{2}} with x = A\sqrt{n}σ and

V_{n} = \sqrt{n}σ for each single function f ∈ F which takes part in the supremum of formula (6.3). The estimate (6.3) yields a slightly weaker estimate for the supremum of such random variables as it contains the coefficient A^{1/2} instead of A in the exponent of the estimate at the right-hand side. But also such a bound will be sufficient for us.

In Proposition 6.2 that situation is considered when the irregularities of the summands provide a non-negligible contribution to the probabilities P (|S_{n}(f)| ≥ u), and the method of proof supplies a good estimate only in this case. This makes natural to separate the proof Theorem 4.1 to the proof of two different statements given in Proposition 6.1 and 6.2.

In the proof of Theorem 4.1 Propositions 6.1 will be applied with a sufficiently large number Ā ≥ 2 and Proposition 6.2 with σ = σ̄ with the number σ̄ defined in Proposition 6.1 and the classes F = D_{j}, more precisely the classes of functions F = \{\frac{g−f_{j}}{2} : g ∈ D_{j}\} introduced in Proposition 6.1, where f_{j} is the function appearing in the definition of the class of functions D_{j}. Clearly,

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā})

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ,

(6.4)

where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 6.1. We want to show that if Ā and then M ≥ M_{0}(Ā) are chosen sufficiently large, then the second term at the right-hand side can be well bounded by means of Proposition 6.2, and Theorem 4.1 can be proved by means of this estimate.

Let us choose a number Ā_{0} in such a way that Ā_{0} ≥ A_{0} and γĀ_{0}^{1/2} ≥ \frac{1}{K} with the numbers A_{0}, K and γ in Proposition 6.2, put Ā = max(2Ā_{0}, 2), and apply Proposition 6.1 with this number Ā. Then also the inequality (\frac{u}{σ̄})^{2} ≥ 4Ā^{2}nσ̄^{2} ≥ (4Ā_{0})^{2}nσ̄^{2}, hence u ≥ 4Ā_{0}\sqrt{n}σ̄^{2} holds with the number σ̄ in Proposition 6.1. (We assume that such numbers u are considered which satisfy the condition nσ^{2} ≥ (\frac{u}{σ})^{2} ≥ M(L + β)^{3/2} log \frac{2}{σ} imposed in Proposition 6.1.) Choose such a number M ≥ M_{0}(Ā) in Proposition 6.1 (which also can be chosen as the number M in formula (4.4) of Theorem 4.1) which also satisfies the inequality \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} ≥ K(L+β) log n with the number K appearing in the conditions of Proposition 6.2. With such a choice we also have nσ̄^{2} ≥ K(L+β) log n.

Since (\frac{1}{2} − \frac{1}{2Ā}) u ≥ \frac{u}{4} ≥ Ā_{0}\sqrt{n}σ̄^{2} and Ā_{0} ≥ A_{0} Propositions 6.2 yields the estimation

P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ≤ P ( sup_{g∈D_{j}} |S_{n} (\frac{f_{j} − g}{2} )| ≥ Ā_{0}\sqrt{n}σ̄^{2})

≤ e^{−γĀ_{0}^{1/2}nσ̄^{2}} for all 1 ≤ j ≤ m, (observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j} is an L_{2}-dense class with parameter D and exponent L), hence Proposition 6.1 and formula 6.4 imply that

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ 4D exp \{−α (\frac{u}{10Āσ} )^{2}\} + Dσ̄^{−L}e^{−γĀ_{0}^{1/2}nσ̄^{2}}.  (6.5)

To get the result of Theorem 4.1 from inequality (6.5) we have to replace its second term at the right-hand side with a more appropriate expression where, in particular, we get rid of the coefficient σ̄^{−L}. The condition nσ̄^{2} ≥ K(L + β) log n implies that σ̄ ≥ n^{−1/2}, and by our choice of Ā_{0} we have γĀ_{0}^{1/2}nσ̄^{2} ≥ \frac{1}{K} nσ̄^{2} ≥ L log n ≥ 2L log \frac{1}{σ̄}, i.e. σ̄^{−L} ≤ e^{γĀ_{0}^{1/2}nσ̄^{2}/2}. By the estimates of Proposition 6.1 nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2}. The above

relations imply that σ̄^{−L}e^{−γĀ_{0}^{1/2}nσ̄^{2}} ≤ e^{−γĀ_{0}^{1/2}nσ̄^{2}/2} ≤ exp \{−\frac{γ}{128} Ā_{0}^{1/2}Ā^{−2} (\frac{u}{σ})^{2}\}. Then relation (6.5) gives that

P (sup_{f∈F} |S_{n}(f)| ≥ u) ≤ 4D exp \{−\frac{α}{(10Ā)^{2}} (\frac{u}{σ} )^{2}\} + D exp \{−\frac{γ}{128}Ā_{0}^{1/2}Ā^{−2} (\frac{u}{σ} )^{2}\} ,

and this estimate implies Theorem 4.1.

Proof of Proposition 6.1. Let us list the members of F, as f_{1}, f_{2}, …, and choose for all p = 0, 1, 2, … a set F_{p} = {f_{a(p,1)}, …, f_{a(p,m_{p})}} ⊂ F with m_{p} ≤ D 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f −f_{a(p,j)})^{2} dμ ≤ 2^{−4p}σ^{2} for all f ∈ F. For all indices a(p, j),

p = 1, 2, …, 1 ≤ j ≤ m_{p}, choose a predecessor a(p − 1, j′), j′ = j′(p, j), 1 ≤ j′ ≤ m_{p−1}, in such a way that the functions f_{a(p,j)} and f_{a(p−1,j′)} satisfy the relation ∫ |f_{a(p,j)} −

f_{a(p−1,j′)}|^{2} dμ ≤ σ^{2}2^{−4(p−1)}. Then we have ∫ (\frac{f_{a(p,j)}−f_{a(p−1,j′)}}{2} )^{2} dμ ≤ 4σ^{2}2^{−4p} and

sup_{x_{j}∈X, 1≤j≤k} |\frac{f_{a(p,j)}(x_{1},...,x_{k})−f_{a(p−1,j′)}(x_{1},...,x_{k})}{2} | ≤ 1. Relation (6.1) yields that

P (A(p, j)) = P (\frac{1}{2}|S_{n}(f_{a(p,j)} − f_{a(p−1,j′)})| ≥ \frac{2^{−(1+p)}u}{2Ā}) ≤ 2 exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\}

if 4nσ^{2}2^{−4p} ≥ (\frac{2^{p}u}{8Āσ} )^{2} , 1 ≤ j ≤ m_{p}, p = 1, 2, …,  (6.6)

and

\begin{aligned}P (B(s)) & = P (|S_{n}(f_{0,s})| ≥ \frac{u}{2Ā}) ≤ 2 exp \{−α (\frac{u}{2Āσ} )^{2}\} , 1 ≤ s ≤ m_{0}, \\ & if nσ^{2} ≥ (\frac{u}{2Āσ} )^{2} . \end{aligned}  (6.7)

Choose the integer number R, R ≥ 0, in such a way that \frac{2^{6(R+1)}}{256} (\frac{u}{Āσ} )^{2} ≥ nσ^{2} ≥ \frac{2^{6R}}{256} (\frac{u}{Āσ} )^{2}, define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R}. (As nσ^{2} ≥ (\frac{u}{σ})^{2} and Ā ≥ 2 by our conditions, there exists such a positive number R. The number R was chosen as the largest number p for which relation (6.6) holds.) Then the cardinality m of the set Fσ̄ equals m_{R} ≤ D2^{2R}σ^{−L} = Dσ̄^{−L}, and the sets D_{j} are D_{j} = {f: f ∈ F, ∫ (f_{a(R,j)} −

f)^{2} dμ ≤ 2^{−4R}σ^{2}}, 1 ≤ j ≤ m_{R}, hence ⋃_{j=1}^{m} D_{j} = F. Besides, the number R was chosen

in such a way that the inequalities (6.6) and (6.7) can be applied for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index (p, j) implies that

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(p, j) ∪ ⋃_{s=1}^{m_{0}} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P (A(p, j)) + ∑_{s=1}^{m_{0}} P (B(s)) ≤ ∑_{p=1}^{∞} 2D 2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\}

+ 2Dσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2}\} .

If the relation (\frac{u}{σ})^{2} ≥ ML log \frac{2}{σ} holds with a sufficiently large constant M (depending on Ā), then the inequalities

2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2}\} ≤ 2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2}\}

hold for all p = 1, 2, …, and

σ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2}\} ≤ exp \{−α (\frac{u}{10Āσ} )^{2}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} |S_{n}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} 2D2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2}\}

+ 2D exp \{−α (\frac{u}{10Āσ} )^{2}\} ≤ 4D exp \{−α (\frac{u}{10Āσ} )^{2}\} ,

and relation (6.2) holds. We have

2^{−4R} · \frac{2^{6R}}{256} (\frac{u}{Āσ} )^{2} ≤ nσ̄^{2} = 2^{−4R}nσ^{2} ≤ 2^{−4R} · \frac{2^{6(R+1)}}{256} (\frac{u}{Āσ} )^{2} = \frac{1}{4} · 2^{2R} (\frac{u}{Āσ} )^{2} ,

hence

\frac{1}{64} (\frac{u}{Āσ} )^{2} ≤ nσ̄^{2} ≤ \frac{1}{4} · (\frac{σ}{σ̄} ) (\frac{u}{Āσ} )^{2} = \frac{1}{4} · (\frac{σ̄}{σ} ) (\frac{u}{Āσ̄} )^{2} ≤ \frac{1}{4} (\frac{u}{Āσ̄} )^{2} ,

as we have claimed. It remained to show that nσ̄^{2} ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}}.

This inequality clearly holds under the conditions of Proposition 6.1 if σ ≤ n^{−1/3}, since in this case log \frac{2}{σ} ≥ \frac{log n}{3}, and nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2} ≥ \frac{1}{64} Ā^{−2}M(L + β)^{3/2} log \frac{2}{σ} ≥

\frac{Ā^{−2}}{192} M(L + β) log n ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} if M ≥ M_{0}(Ā) with a sufficiently large number M_{0}(Ā).

If σ ≥ n^{−1/3}, then we apply that the inequality 2^{6R} (\frac{u}{Āσ} )^{2} ≤ 256nσ^{2} implies

that 2^{−4R} ≥ 2^{−16/3} [\frac{(\frac{u}{Āσ} )2}{nσ^{2}} ]^{2/3}, and nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1/3} (\frac{u}{σ})^{4/3}. Since

nσ^{2} ≥ n^{1/3} and (\frac{u}{σ})^{2} ≥ \frac{M}{3} (L + β)^{3/2}, these estimates yield that

nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/3} (\frac{u}{σ} )^{4/3} ≥ \frac{Ā^{−4/3}}{50} n^{1/9} (\frac{M}{3} )^{2/3} (L + β) ≥ \frac{M^{2/3}(L + β) log n}{1000Ā^{4/3}}.

7. The completion of the proof of Theorem 4.1

In this section we prove Proposition 6.2 by which the proof of Theorem 4.1 is completed. First a symmetrization lemma is proved, and then with the help of this result and a conditioning argument the proof of Proposition 6.2 is reduced to the estimation of a probability which can be bounded by means of the Hoeffding inequality formulated in Theorem 3.4. Such an approach makes possible to prove Proposition 6.2.

First I formulate the symmetrization lemma we shall apply.

Lemma 7.1 (Symmetrization Lemma). Let Z_{n} and Z̄_{n}, n = 1, 2, …, be two sequences of random variables independent of each other, and let the random variables Z̄_{n}, n = 1, 2, …, satisfy the inequality

P (|Z̄_{n}| ≤ α) ≥ β for all n = 1, 2, …  (7.1)

with some numbers α ≥ 0 and β ≥ 0. Then

P ( sup_{1≤n<∞} |Z_{n}| > α + u) ≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) for all u > 0.

Proof of Lemma 7.1. Put τ = min{n: |Z_{n}| > α + u} if there exists such an index n, and τ = 0 otherwise. Then the event {τ = n} is independent of the sequence of random variables Z̄_{1}, Z̄_{2}, … for all n = 1, 2, …, and because of this independence

P (\{τ = n\}) ≤ \frac{1}{β} P (\{τ = n\} ∩ \{|Z̄_{n}| ≤ α\}) ≤ \frac{1}{β} P (\{τ = n\} ∩ \{|Z_{n} − Z̄_{n}| > u\})

for all n = 1, 2, . . .. Hence

P ( sup_{1≤n<∞} |Z_{n}| > α + u) = ∑_{n=1}^{∞} P (τ = n) ≤ \frac{1}{β} ∑_{n=1}^{∞} P (\{τ = n\} ∩ \{|Z_{n} − Z̄_{n}| > u\})

≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) .

Lemma 7.1 is proved.

We shall apply the following consequence Lemma 7.2 of the symmetrization lemma.

Lemma 7.2. Let us fix a countable class of functions F on a measurable space (X, X) together with a real number 0 < σ < 1. Consider a sequence of independent, identically distributed X-valued random variables ξ_{1}, …, ξ_{n} such that Ef(ξ_{1}) = 0, Ef^{2}(ξ_{1}) ≤ σ^{2} for all f ∈ F together with another sequence ε_{1}, …, ε_{n} of independent random variables with distribution P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, independent also of the random sequence ξ_{1}, …, ξ_{n}. Then

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) if A ≥ \frac{3\sqrt{2}}{\sqrt{n}σ} .

(7.2)

Proof of Lemma 7.2. Let us construct an independent copy ξ̄_{1}, …, ξ̄_{n} of the sequence ξ_{1}, …, ξ_{n} in such a way that all three sequences ξ_{1}, …, ξ_{n}, ξ̄_{1}, …, ξ̄_{n} and ε_{1}, …, ε_{n}

are independent. Define the random variables S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}) and S̄_{n}(f) =

\frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ̄_{j}) for all f ∈ F. The inequality

P (sup_{f∈F} |S_{n}(f)| > A\sqrt{n}σ^{2}) ≤ 2P (sup_{f∈F} |S_{n}(f) − S̄_{n}(f)| > \frac{2}{3}A\sqrt{n}σ^{2}) .  (7.3)

follows from Lemma 7.1 if we apply it for the countable sets Z_{n}(f) = S_{n}(f) and Z̄_{n}(f) = S̄_{n}(f), f ∈ F, of random variables and x = \frac{2}{3}A\sqrt{n}σ^{2}, α = \frac{1}{3}A\sqrt{n}σ^{2}, since the fields S_{n}(f) and S̄_{n}(f) are independent, and P (|S̄_{n}(f)| ≤ α) > \frac{1}{2} for all f ∈ F. Indeed, α = \frac{1}{3}A\sqrt{n}σ^{2} ≥ \sqrt{2}σ, ES̄_{n}(f)^{2} ≤ σ^{2}, thus Chebishev’s inequality implies that P (|S̄_{n}(f)| ≤ α) ≥ P (|S̄_{n}(f)| ≤ \sqrt{2}σ) ≥ \frac{1}{2} for all f ∈ F.

Let us observe that the random field

S_{n}(f) − S̄_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F,  (7.4)

and its randomization

\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j} (f(ξ_{j}) − f(ξ̄_{j})) , f ∈ F,  (7.4′)

have the same distribution. Indeed, even the conditional distribution of (7.4′) under the condition that the values of the ε_{j}-s are prescribed agrees with the distribution of (7.4) for all possible values of the ε_{j}-s. This follows from the observation that the distribution of the field (7.4) does not change if we exchange the random variables ξ_{j} and ξ̄_{j} for certain indices j, and this corresponds to considering the conditional distribution of the field in (7.4′) under the condition that ε_{j} = −1 for these indices j, and ε_{j} = 1 for the remaining ones.

The above relation together with formula (7.3) imply that

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2})

≤ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j} [f(ξ_{j}) − f̄(ξ_{j})]| ≥ \frac{2}{3}An^{1/2}σ^{2})

≤ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

+ 2P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ̄_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

= 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2}) .

Lemma 7.2 is proved.

Let me briefly explain the approach to the proof of Proposition 6.2. We have to es-

timate a probability of the form P (n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} f(ξ_{j})| > u), and by Lemma 7.2 this

can be replaced by the estimation of the probability P (n^{−1/2} sup_{f∈F} | ∑_{j=1}^{n} ε_{j}f(ξ_{j})| > \frac{u}{3})

with some independent random variables ε_{j}, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, j = 1, …, n, which are also independent of the random variables ξ_{j}. We shall bound the conditional probability of the event appearing in this modified problem under the condition that the values of the random variables ξ_{j} are prescribed. This can be done with the help of Hoeffding’s inequality formulated in Theorem 3.4 and the L_{2}-density property of the class of functions F we consider. By working out the details we are led to the estimation of the probability P (n^{−1/2} sup_{f∈F′} | ∑_{j=1}^{n} f(ξ_{j})| > u^{1+α}) with some new nice L_{2}-dense class

of bounded functions F′ and some number α > 0. This problem is very similar to the original one, but it is simpler, since the number u is replaced by a larger number u^{1+α} in it. By repeating this argument successively, in finitely many steps we get the proof of Proposition 6.2.

The above sketched argument suggests a backward induction procedure to prove Proposition 6.2. To carry out such a program first we introduce a property we want to prove.

Definition of good tail behaviour for a class of normalized random sums. Let us fix some measurable space (X, X) and a probability measure μ on it together with some integer n ≥ 2 and real number σ > 0, and consider some class F of functions f(x) on the space (X, X). Take a sequence of independent μ distributed random variables

ξ_{1}, …, ξ_{n}, and define with its help the normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}),

f ∈ F. Given some real number T > 0 we say that the set of normalized random sums S_{n}(f) determined by the class of functions F has a good tail behaviour at level T (with parameters n and σ^{2} which we shall fix in the sequel) if the inequality

P (sup_{f∈F} |S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ exp \{−A^{1/2}nσ^{2}\}  (7.5)

holds for all numbers A > T.

Now we formulate Proposition 7.3 and show that Proposition 6.2 follows from it.

Proposition 7.3. Let us fix a positive integer n ≥ 2, a real number σ > 0 and a probability measure μ on a measurable space (X, X) together with a countable L_{2}-dense class F of functions f = f(x) on the space (X, X) with some prescribed exponent L ≥ 1 and parameter D. Let us also assume that all functions f ∈ F satisfy the conditions sup_{x∈X} |f(x)| ≤ \frac{1}{4}, ∫ f^{2}(x)μ( dx) ≤ σ^{2}, and nσ^{2} > K(L + β) log n with a sufficiently large

fixed number K and β = max (\frac{log D}{log n} , 0).

If there is a number T > 1 such that for all classes of functions F which satisfy

the above conditions the class of normalized random sums S_{n}(f) = \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f(ξ_{j}), f ∈

F, defined with the help of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} have a good tail behaviour at level T, then there is a universal constant Ā_{0} such that the number T̄ = T^{3/4} also have this property provided that T ≥ Ā_{0}. We can choose for instance Ā_{0} = 64 · 10^{12} and K = 1.

Proposition 6.2 simply follows from Proposition 7.3. To show this let us first observe that the class of normalized random sums S_{n}(f), f ∈ F, has a good tail behaviour at level T_{0} = \frac{1}{4σ^{2}} if the class of functions F satisfies the conditions of Proposition 7.3.

Indeed, in this case P (sup_{f∈F} |S_{n}(f)| ≥ A\sqrt{n}σ^{2}) ≤ P (sup_{f∈F} |S_{n}(f)| > \frac{\sqrt{n}}{4}) = 0 for all

A > T_{0}. Then the repetitive application of Proposition 7.3 yields that the class of random sums S_{n}(f) has a good tail behaviour at all levels T ≥ T_{0}^{(3/4)^{j}} if T_{0}^{(3/4)^{j}} ≥ Ā_{0}, hence for T = Ā_{0}^{4/3} if the class of functions F satisfies the conditions of Proposition 7.3. If the class of functions f ∈ F satisfies the conditions of Proposition 6.2, then the class of functions F̄ = \{f̄ = \frac{f}{4} : f ∈ F\} satisfies the conditions of Proposition 7.2, (actually with σ̄ = \frac{σ}{4} , and a better parameter D for the class F), hence the class of functions S_{n}(f̄), f̄ ∈ F̄, has a good tail behaviour at level T = Ā_{0}^{4/3}. This implies that the original class of functions F satisfy formula (6.3) in Proposition 6.2 with 4K, A_{0} = 4Ā_{0}^{4/3} and γ = \frac{1}{2}, and this is what we had to show.

The proof of Proposition 7.3. Fix a class of functions F which satisfies the conditions of Proposition 7.3 together with two independent sequences ξ_{1}, …, ξ_{n} and ε_{1}, …, ε_{n} of independent random variables, where ξ_{j} is μ-distributed, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, and investigate the conditional probability

P (f, A|ξ_{1}, …, ξ_{n}) = P ( \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{6}\sqrt{n}σ^{2}| ξ_{1}, …, ξ_{n})

for all functions f ∈ F, A ≥ T and values (ξ_{1}, …, ξ_{n}) in the condition. By the Hoeffding inequality presented in Theorem 3.4

P (f, A|ξ_{1}, …, ξ_{n}) ≤ 2 exp \{−\frac{\frac{1}{36}A^{2}nσ^{4}}{2S̄^{2}(f, ξ_{1}, …, ξ_{n})}\}  (7.6)

with

S̄^{2}(f, x_{1}, …, x_{n}) = \frac{1}{n} ∑_{j=1}^{n} f^{2}(x_{j}), f ∈ F.

Let us introduce the set

H = H(A) = {(x_{1}, …, x_{n}): sup_{f∈F} S̄^{2}(f, x_{1}, …, x_{n}) ≥ (1 + A^{4/3}) σ^{2}} .  (7.7)

I claim that

P ((ξ_{1}, …, ξ_{n}) ∈ H) ≤ e^{−A^{2/3}nσ^{2}} if A ≥ T.  (7.7′)

(The set H plays the role of the small exceptional set, where we cannot provide a good estimate for P (f, A|ξ_{1}, …, ξ_{n}) for some f ∈ F.)

To prove relation (7.7′) let us consider the functions f̄ = f̄(f), f̄(x) = f^{2}(x) − ∫ f^{2}(x)μ( dx), and introduce the class of functions F′ = {f̄(f): f ∈ F}. Let us show that the class of functions F′ satisfies the conditions of Proposition 7.3, hence the estimate (7.5) holds for the class of functions F′ if A ≥ T^{4/3}.

The relation ∫ f̄(x)μ( dx) = 0 clearly holds. The condition sup |f̄(x)| ≤ \frac{1}{8} < \frac{1}{4} also holds if sup |f(x)| ≤ \frac{1}{4}, and ∫ f̄^{2}(x)μ( dx) ≤ ∫ f^{4}(x)μ( dx) ≤ \frac{1}{16} ∫ f^{2}(x) μ( dx) ≤ \frac{σ^{2}}{16} < σ^{2} if f ∈ F. It remained to show that F′ is an L_{2}-dense class with exponent L and parameter D.

To show this observe that ∫ (f̄(x) − ḡ(x))^{2}ρ( dx) ≤ 2 ∫ (f^{2}(x) − g^{2}(x))^{2}ρ( dx) + 2 ∫ (f^{2}(x) − g^{2}(x))^{2}μ( dx) ≤ 2(sup(|f(x)| + |g(x)|)^{2} (∫ (f(x) − g(x))^{2}(ρ( dx) + μ( dx)) ≤ ∫ (f(x) − g(x))^{2}ρ̄( dx) for all f, g ∈ F, f̄ = f̄(f), ḡ = ḡ(g) and probability measure ρ, where ρ̄ = \frac{ρ+μ}{2}. This means that if \{f_{1}, …, f_{m}\} is an ε-dense subset of F in the space L_{2}(X, X , ρ̄), then {f̄_{1}, …, f̄_{m}} is an ε-dense subset of F′ in the space L_{2}(X, X , ρ), and not only F, but also F′ is an L_{2}-dense class with exponent L and parameter D.

We get by applying the inductive hypothesis of Proposition 7.3 for the number A^{4/3} ≥ T^{4/3} and the class of functions F′ that

P ((ξ_{1}, …, ξ_{n}) ∈ H) = P (sup_{f∈F} (\frac{1}{n} ∑_{j=1}^{n} f̄(ξ_{j}) + \frac{1}{n} ∑_{j=1}^{n} Ef^{2}(ξ_{j})) ≥ (1 + A^{4/3}) σ^{2})

≤ P (sup_{f̄∈F̄} \frac{1}{\sqrt{n}} ∑_{j=1}^{n} f̄(ξ_{j}) ≥ A^{4/3}n^{1/2}σ^{2}) ≤ e^{−A^{2/3}nσ^{2}},

i.e. relation (7.7′) holds.

Formula (7.6) and the definition of the set H given in (7.7) yield the estimate

P (f, A|ξ_{1}, …, ξ_{n}) ≤ 2e^{−A^{2/3}nσ^{2}/144} if (ξ_{1}, …, ξ_{n}) ∉ H  (7.8)

for all f ∈ F and A ≥ T ≥ 1. (Here we used the estimate 1 + A^{4/3} ≤ 2A^{4/3}.) Let us introduce the conditional probability

P (F, A|ξ_{1}, …, ξ_{n}) = P ( sup_{f∈F} \frac{1}{\sqrt{n}} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}\sqrt{n}σ^{2}| ξ_{1}, …, ξ_{n})

for all (ξ_{1}, …, ξ_{n}) and A ≥ T. We shall estimate this conditional probability with the help of relation (7.8) if (ξ_{1}, …, ξ_{n}) ∉ H. Given some set of n points (x_{1}, …, x_{n}) in the space (X, X) let us introduce the measure ν = ν(x_{1}, …, x_{n}) on (X, X) in such a way that ν is concentrated in the points x_{1}, …, x_{n}, and ν(\{x_{j}\}) = \frac{1}{n}. If ∫ f^{2}(x)ν( dx) ≤ δ^{2} for

a function f, then |\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j})| ≤ n^{1/2} ∫ |f(x)|ν( dx) ≤ n^{1/2}δ. Since the condition

nσ^{2} ≥ K(L + β) log n in Proposition 7.3 also implies that nσ^{2} ≥ 1 (if the constant K is chosen sufficiently large), the above estimate implies that if f and g are two functions

such that ∫ (f − g)^{2}ν( dx) ≤ δ^{2} with δ = \frac{A}{6n}, then |\frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}f(x_{j}) − \frac{1}{\sqrt{n}} ∑_{j=1}^{n} ε_{j}g(x_{j})| ≤

\frac{A}{6\sqrt{n}} ≤ \frac{A}{6} \sqrt{n}σ^{2}.

Let us fix some (random) point (ξ_{1}, …, ξ_{n}) ∉ H, consider the measure ν = ν(ξ_{1}, …, ξ_{n}) corresponding to it and choose a δ̄-dense subset {f_{1}, …, f_{m}} of F in the space L_{2}(X, X , ν) with δ̄ = \frac{1}{6n} ≤ δ = \frac{A}{6n}, whose cardinality m satisfies the inequality m ≤ Dδ̄^{−L}. This is possible because of the L_{2}-dense property of the class F. (This is the point where the L_{2}-dense property of the class of functions F is ex-

ploited in its full strength.) The above facts imply that if \frac{1}{\sqrt{n}} | ∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3} \sqrt{n}σ^{2}

for some function f ∈ F, then \frac{1}{\sqrt{n}} | ∑_{j=1}^{n} ε_{j}f_{l}(ξ_{j})| ≥ \frac{A}{6} \sqrt{n}σ^{2} for some function f_{l} of

the δ̄-dense subset {f_{1}, …, f_{m}} of F with the fixed point (ξ_{1}, …, ξ_{n}) ∉ H. Hence

P (F, A|ξ_{1}, …, ξ_{n}) ≤ ∑_{l=1}^{m} P (f_{l}, A|ξ_{1}, …, ξ_{n}) with these functions f_{1}, …, f_{m}, and rela-

tion (7.8) yields that

P (F, A|ξ_{1}, …, ξ_{n}) ≤ 2D(6n)^{L}e^{−A^{2/3}nσ^{2}/144} if (ξ_{1}, …, ξ_{n}) ∉ H and A ≥ T.

This inequality together with Lemma 7.2 (under the restriction that A ≥ Ā_{0} ≥ \frac{3\sqrt{2}}{\sqrt{n}σ} ≥ 3\sqrt{2}) and estimate (7.7′) imply that

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2}) ≤ 4P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} ε_{j}f(ξ_{j})| ≥ \frac{A}{3}n^{1/2}σ^{2})

≤ 8D(6n)^{L}e^{−A^{2/3}nσ^{2}/144} + 4e^{−A^{2/3}nσ^{2}} if A ≥ T.

(7.9)

By the condition nσ^{2} ≥ K(L + β) log n = KL log n + K log(max(D, 1)), hence the first term at the right-hand side of (7.9) can be bounded as

8D(6n)^{L}e^{−A^{2/3}nσ^{2}/144}
≤ e^{−A^{1/2}nσ^{2}} · 8D6^{L}n^{L(1−A^{1/2}/3)} max(D, 1)^{−A^{1/2}/3} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}}

if A ≥ T ≥ Ā_{0} ≥ 64 · 10^{12} and K ≥ 1. (With such parameters \frac{A^{2/3}}{144} − A^{1/2} ≤ \frac{1}{3}A^{1/2}.) With such a choice of the parameters the inequality \frac{3\sqrt{2}}{\sqrt{n}σ} ≤ \frac{3\sqrt{2}}{\sqrt{K log 2}} ≤ Ā_{0} ≤ A, needed

for the validity of relation (7.2), also holds. The second term at the right-hand side of (7.9) be bounded as 4e^{−A^{2/3}nσ^{2}} ≤ \frac{1}{2}e^{−A^{1/2}nσ^{2}}. with the above choice of the numbers Ā_{0} and K.

By the above calculation formula (7.9) yields the inequality

P (\frac{1}{\sqrt{n}} sup_{f∈F} |∑_{j=1}^{n} f(ξ_{j})| ≥ An^{1/2}σ^{2}) ≤ e^{−A^{1/2}nσ^{2}}

if A ≥ T, and the constants Ā_{0} and K are chosen sufficiently large, for instance Ā_{0} = 64 · 10^{12} and K = 1 is an appropriate choice.

8. Formulation of the main results of this work

This section contains the main results of this work about multiple stochastic integrals and their supremum. Section 4 contains these results in the special case of one-fold integrals together with their version about the supremum of appropriate classes of normalized sums of independent and identically distributed random variables with zero expectation. (See Theorem 4.1 and 4.1′ and their comparison.) The results about multiple stochastic integrals also have a similar version, and they will be also presented. Here the role of sums of independent, and identically distributed random variables are taken by degenerate U-statistics of independent and identically distributed random variables. The condition that the U-statistics have to be degenerate plays the a role similar to the condition about the zero expectation of the summands when the independent sum versions of the one-fold integral results are considered. The basic notions about U-statistics needed to understand the results will also be explained. The proof of the equivalence of the results about multiple integrals and U-statistics formulated in this section requires a detailed study of the property of U-statistics, a problem which has a special interest in itself. This will be the subject of the next section.

We also formulate some results about multiple Wiener–Itô integrals which are natural analogs of the results about multiple integrals with respect to normalized empirical measures. But these results are only briefly discussed, because they do not belong to the main subject of this work, and they demand a more detailed study of multiple Wiener–Itô integrals. Finally, this section is finished with a the two-dimensional version of Example 3.2 which shows that certain conditions of the results discussed here are really necessary.

Let us consider a sequence of iid. random variables ξ_{1}, …, ξ_{n} taking values on a measurable space (X, X). Let μ denote its distribution, and introduce the empirical distribution of this sequence defined in (4.5). Given a measurable function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) introduce its integral J_{n,k}(f) with respect to the k-fold product of the normalized empirical measure \sqrt{n}(μ_{n} − μ) defined in formula (4.8). Here we define the domain of integration by deleting the diagonals x_{j} = x_{l}, 1 ≤ j < l ≤ k, from the k-fold product space (X^{k}, X^{k}). The following Theorem 8.1 can be considered as the multiple integral version of Bernstein’s inequality formulated in Theorem 3.1.

Theorem 8.1. Let us take a measurable function f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}) of a measure space (X, X) with some k ≥ 1 together with a non-atomic probability measure μ on (X, X) and a sequence of iid. random variables ξ_{1}, …, ξ_{n} with distribution μ on (X, X). Let the function f satisfy the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,  (8.1)

and

∥f∥_{2}^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}  (8.2)

with some constant 0 < σ ≤ 1. There exist some constants C = C_{k} > 0 and α = α_{k} > 0, such that the random integral J_{n,k}(f) defined by formulas (4.5) and (4.8) satisfies the inequality

P (|J_{n,k}(f)| > u) ≤ C max (exp \{−α (\frac{u}{σ} )^{2/k}\} , exp \{−α(nu^{2})^{1/(k+1)}\})  (8.3)

for all u > 0. The constants C = C_{k} > 0 and α = α_{k} > 0 in formula (8.3) depend only on the parameter k.

Theorem 8.1 can be reformulated in the following equivalent form.

Theorem 8.1′. Under the conditions of Theorem 8.1

P (|J_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\} for all 0 < u ≤ n^{k/2}σ^{k+1}  (8.3′)

with the number σ appearing in (8.2) and some universal constants C = C_{k} > 0, α = α_{k} > 0, depending only on the multiplicity k of the integral J_{n,k}(f).

Theorem 8.1 clearly implies Theorem 8.1′, since in the case u ≤ n^{k/2}σ^{k+1} the first term is larger than the second one in the maximum at the right-hand side of formula (8.3). On the other hand Theorem 8.1′ implies Theorem 8.1 also if u > n^{k/2}σ^{k+1}, since in this case Theorem 8.1′ can be applied with σ̄ = (un^{−k/2})^{1/(k+1)} ≥ σ. This

yields that P (|J_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ̄})^{2/k}\} = C exp \{−α(nu^{2})^{1/(k+1)}\} if u >

n^{k/2}σ^{k+1}.

Theorem 8.1 or Theorem 8.1′ state that the tail probability P (|J_{n,k}(f)| > u) of the k-fold random integral J_{n,k}(f) can be bounded similarly to the probability P (|const. ση^{k}| > u), where η is a random variable with standard normal distribution and σ is the number appearing in relation (8.2), provided that the level u we consider is less than n^{k/2}σ^{k+1}. (The value of the number σ^{2} in formula (8.2) is closely related to the variance of J_{n,k}(f).) At the end of this section an example is given which shows that such a condition is really needed in the above results.

Now we formulate Theorem 8.2 which is the generalization of Theorem 4.1 for multiple integrals. Here we apply the notions of L_{2}-dense classes and countably approximability introduced in Section 4.

Theorem 8.2. Let us have a non-atomic probability measure μ on a measurable space (X, X) together with a countable and L_{2}-dense class F of functions f = f(x_{1}, …, x_{k}) of k variables with some parameter D and exponent L, L ≥ 1, on the product space (X^{k}, X^{k}) which satisfies the conditions

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,  for all f ∈ F  (8.4)

and

∥f∥_{2}^{2} = Ef^{2}(ξ_{1}, …, ξ_{k}) = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} for all f ∈ F

(8.5)

with some constant 0 < σ ≤ 1. Then there exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the supremum of the random integrals J_{n,k}(f), f ∈ F, defined by formula (4.8) satisfies the inequality

P (sup_{f∈F} |J_{n,k}(f)| ≥ u) ≤ CD exp \{−α (\frac{u}{σ} )^{2/k}\}

if nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L + β)^{3/2} log \frac{2}{σ} ,

(8.6)

where β = max (\frac{log D}{log n} , 0) and the numbers D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables J_{n,k}(f), f ∈ F, is countably approximable.

To formulate that version of Theorems 8.1 and 8.2 which corresponds to the results about sums of independent random variables in the case k = 1 let us introduce the following notions:

Definition of U-statistics. Let us consider a function f = f(x_{1}, …, x_{k}) on the k-th power (X^{k}, X^{k}) of a space (X, X) together with a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n}, n ≥ k, which take their values on this space (X, X). The expression

I_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}, …, ξ_{l_{k}})  (8.7)

is called a U-statistic of order k with the sequence ξ_{1}, …, ξ_{n}, and f is called its kernel function.

To make our later notation non-ambiguous let us also consider functions of the form f(x_{u_{1}}, …, x_{u_{k}}), that is let us allow the possibility that the variables of the function f which take their values in the space (X, X) are indexed in a general way. In the case of such an indexation we define

I_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{u_{j}} ≤n, j=1,...,k \\ l_{u_{j}} ≠l_{u′_{j}} if j≠j′ \end{matrix}} f (ξ_{l_{u_{1}}} , …, ξ_{l_{u_{k}}} ) .  (8.7′)

A similar convention will be applied in the definition of decoupled U-statistics introduced later, and the following definition of degenerate U-statistics and canonical functions can also be similarly reformulated in the case of general indexation.

The degenerate U-statistics which correspond to sums of identically distributed random variables with expectation zero constitute an important subclass of the Ustatistics. We define it together with the notion of canonical kernel function which is closely related to it.

Definition of degenerate U-statistics. A U-statistic I_{n,k}(f) of order k with a sequence of iid. random variables ξ_{1}, …, ξ_{n} is called degenerate if its kernel function f(x_{1}, …, x_{k}) satisfies the relation

Ef(ξ_{1}, …, ξ_{k}|ξ_{1} = x_{1}, …, ξ_{j−1} = x_{j−1}, ξ_{j+1} = x_{j+1}, …, ξ_{k} = x_{k}) = 0
for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

Definition of canonical kernel function. A function f(x_{1}, …, x_{k}) taking values on the k-fold product of a measure space (X, X) is called a canonical function with respect to a probability measure μ on (X, X) if

∫ f(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{k})μ( du) = 0 for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

(8.8)

It is clear that a U-statistic I_{n,k}(f) with kernel function f and independent μdistributed random variables ξ_{1}, …, ξ_{n} is degenerate if and only if its kernel function is canonical with respect to the probability measure μ. Now we can formulate two results about U-statistics which are, as we shall see in the next section, equivalent to Theorems 8.1 and 8.2.

Theorem 8.3. Let us have a measurable function f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}), k ≥ 1, of a measure space (X, X) with some k ≥ 1 together with a probability measure μ on (X, X) and a sequence of iid. random variables ξ_{1}, …, ξ_{n} with distribution μ on (X, X). Let us consider the k-fold U-statistic I_{k,n}(f) with this sequence of random variables ξ_{1}, …, ξ_{n}. Assume that this U-statistic is degenerate, i.e. the kernel function f(x_{1}, …, x_{k}) of this U-statistic is canonical with respect to the measure μ. Let us also assume that the function f satisfies conditions (8.1) and (8.2) with some number 0 < σ ≤ 1. Then there exist some constants C = C_{k} > 0 and α = α_{k} > 0 such that the inequality

P (n^{−k/2}|I_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\}  (8.9)

holds for all 0 < u ≤ n^{k/2}σ^{k+1}. The constants C = C_{k} > 0 and α = α_{k} > 0 depend only on the parameter k.

Theorem 8.4. Let us have a probability measure μ on a measurable space (X, X) together with a countable and L_{2}-dense class F of functions f = f(x_{1}, …, x_{k}) of k variables with some parameter D and exponent L, L ≥ 1, on the product space (X^{k}, X^{k}) which satisfies conditions (8.4) and (8.5) with some constant 0 < σ ≤ 1. Besides these conditions let us assume that for a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} the U-statistics I_{n,k}(f) with this sequence ξ_{1}, …, ξ_{n} are degenerate for all f ∈ F, or in an equivalent form all functions f ∈ F are canonical with respect to the measure μ. Then there exist some constants C = C(k) > 0, α = α(k) > 0 and M = M(k) > 0 depending only on the parameter k such that the inequality

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ CD exp \{−α (\frac{u}{σ} )^{2/k}\}

if nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ M(L + β)^{3/2} log \frac{2}{σ} ,

(8.10)

holds, where β = max (\frac{log D}{log n} , 0) and the number D and L agree with the parameter and exponent of the L_{2}-dense class F.

The condition about the countable cardinality of the class F can be replaced by the weaker condition that the class of random variables n^{−k/2}I_{n,k}(f), f ∈ F, is countably approximable.

Let us briefly describe the Gaussian counterpart of the above results. Here some basic notions and results about multiple Wiener–Itô integrals are applied. But since the results about these Gaussian fields do not belong to the main subject of this work, they are mainly interesting for us for the sake of a comparison, most technical details will be omitted from our discussion.

Let us consider a measurable space (X, X) together with a non-atomic σ-finite measure μ on it. Let Z_{μ} be an orthogonal Gaussian random measure with counting measure μ on (X, X), i.e. assume that the random variables Z_{μ}(A), A ∈ X, μ(A) < ∞ are defined, they are jointly Gaussian, EZ_{μ}(A) = 0 for all A ∈ A, μ(A) < ∞ and EZ_{μ}(A)Z_{μ}(B) = μ(A ∩ B) for all A ∈ A, B ∈ A, μ(A) < ∞, μ(B) < ∞.

Let us observe that these relations imply that if A ∈ X, μ(A) < ∞ and B ∈ X, μ(B) < ∞ are disjoint sets, then Z_{μ}(A) and Z_{μ}(B) are independent, and Z_{μ}(A ∪ B) = Z_{μ}(A) + Z_{μ}(B) with probability 1. The last relation follows from the fact that E(Z_{μ}(A ∪ B) − Z_{μ}(A) − Z_{μ}(B))^{2} = 0 under these conditions.

If f(x_{1}, …, x_{k}) is a measurable function on (X^{k}, X^{k}) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞,

then the multiple Wiener–Itô integral Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k}) can be defined, and it satisfies similar estimates as the random integrals J_{n,k}(f). This statement will be formulated more explicitly in the following Theorem 8.5.

Theorem 8.5 Let us fix a measurable space (X, X) together with a σ-finite non-atomic measure μ on it, and let Z_{μ} be an orthogonal Gaussian random measure with counting measure μ on (X, X). If f(x_{1}, …, x_{k}) is a measurable function on (X^{k}, X^{k}) such that \frac{1}{k!} ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} with some 0 < σ < ∞, then

P (|Z_{μ,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\}  (8.11)

for all u > 0 with some constants C = C(k) and α = α(k) depending only on k. If F is a countable class of functions of k variables on (X, X) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} with some 0 < σ ≤ 1 for all f ∈ F,

and there exist some constant D > 0 and L > 0 such that a subset {f_{1}, …, f_{m}} ⊂ F can be chosen with m ≤ Dε^{−L} elements for which

min_{1≤j≤m} ∫ (f(x_{1}, …, x_{k}) − f_{j}(x_{1}, …, x_{k}))^{2} μ( dx_{1}) … μ( dx_{k}) ≤ ε for all f ∈ F,

then the inequality

P (sup_{f∈F} |Z_{μ,k}(f)| > u) ≤ C(D + 1) exp \{−α (\frac{u}{σ} )^{2/k}\} if u ≥ ML^{k/2}σ log^{k/2} \frac{2}{σ}

(8.12)

holds with some universal constants C = C(k) > 0, M = M(k) > 0 and α = α(k) > 0.

Since the above result does not belong to the main part of this work, only a sketchy proof will be presented. Nelson’s inequality, mentioned at the start of this section will be formulated and proved in the Appendix, and it will be explained how Theorem 8.5 can be proved with its help.

The above results show that multiple integrals with respect to a normalized empirical measure or degenerate U-statistics satisfy some estimates similar to multiple Wiener–Itô integrals, but they hold under more restrictive conditions. This difference between multiple integrals with respect to a normalized empirical measure and orthogonal Gaussian measures can be explained similarly to some arguments presented in Section 4 about the one-fold integral case. Here we do not repeat them, we only give an example similar to Example 3.2 which shows that the condition u ≤ n^{k/2}σ^{k+1} cannot be dropped from the conditions of Theorem 8.2. For the sake of simplicity we restrict our attention to the case k = 2.

Example 8.6. Let ξ_{1}, …, ξ_{n} be a sequence of independent, identically distributed random variables taking values on the plane R^{2} = X such that ξ_{j} = (η_{j,1}, η_{j,2}), η_{j,1} and η_{j,2} are independent, P (η_{j,1} = 1) = P (η_{j,1} = −1) = \frac{σ^{2}}{8} , P (η_{j,1} = 0) = 1 − \frac{σ^{2}}{4} , P (η_{j,2} = 1) = P (η_{j,2} = −1) = \frac{1}{2} for all 1 ≤ j ≤ n, introduce the function f(x, y) = f((x_{1}, x_{2}), (y_{1}, y_{2})) = x_{1}y_{2} + x_{2}y_{1}, x = (x_{1}, x_{2}) ∈ R^{2}, y = (y_{1}, y_{2}) ∈ R^{2}, and define the U-statistic

I_{n,2}(f) = ∑_{1≤j,k≤n, j≠k}(η_{j,1}η_{k,2} + η_{k,1}η_{j,2})

of order 2 with the above kernel function f and the sequence of independent random variables ξ_{1}, …, ξ_{n}. Then I_{n,2}(f) is a degenerate U-statistic. If u ≥ B_{1}nσ^{3} with some appropriate constant B_{1} > 0, B̄_{2}^{−1}n ≥ u ≥ B̄_{2}n^{−2} with a sufficiently large fixed number B̄_{2} > 0 and σ ≥ \frac{1}{n}, then the estimate

P (n^{−1}I_{n,2}(f) > u) ≥ exp \{−Bn^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\}  (8.13)

holds with some B > 0.

Remark: The main content of the above example is that in the case k = 2 the condition \frac{u}{σ} ≤ nσ^{2} cannot be dropped from Theorem 8.3. Let us observe that in the case u = nσ^{3} the right-hand side of (8.13) has the same order as Theorem 8.3 suggests. (In this model ∫ f^{2}(x, y)μ( dx)μ( dy) = E(2η_{j,1}η_{j,2})^{2} = σ^{2}.) If we consider the probability in (8.13) at the same level u, but with a much smaller parameter σ^{2}, then the probability at the right-hand side of (8.13) has a relatively small decrease, and the estimate of Theorem 8.3 does not hold any longer. Let me also remark that under some mild additional restrictions the estimate (8.13) can be slightly improved, the term log can be replaced by log^{2/3} in the exponent of the right-hand side of (8.13). To get this improvement some more calculation is needed, and the numbers u_{1} and u_{2} in the following calculations have to be replaced by v_{1} = 8n^{1/3}u^{2/3} log^{−1/3} (\frac{u}{nσ^{3}} ) and v_{2} = \frac{1}{4}n^{2/3}u^{1/3} log^{1/3} (\frac{u}{nσ^{3}} ).

It is simple to check that the U-statistic we considered in the above example is degenerate because of the independence properties of the model and the relation Eη_{j,1} = Eη_{j,2} = 0. In the proof of the estimate (8.13) we shall apply for one hand the results of Section 3, in particular Example 3.2 for the sequence η_{j,1}, j = 1, 2, …, n, on the other hand the following result from the theory of large deviations: If X_{1}, …, X_{n} are iid. random variables, P (X_{1} = 1) = P (X_{1} = −1) = \frac{1}{2}, then for any number 0 ≤ α < 1 there

exists some numbers C_{1} = C_{1}(α) > 0 and C_{2} = C_{2}(α) > 0 such that P ( ∑_{j=1}^{n} X_{j} > u) ≥

C_{1}e^{−C_{2}u^{2}/n} for all 0 ≤ u ≤ αn.

Proof of the statement of the example. We can write

P (n^{−1}I_{n,2}(f) > u) ≥ P (2 (∑_{j=1}^{n} η_{j,1}) (∑_{j=1}^{n} η_{j,2}) > 2nu) − P (2 ∑_{j=1}^{n} η_{j,1}η_{j,2} > nu) .

(8.14)

Because of the independence of the random variables η_{j,1} and η_{j,2} the first probability at the right-hand side of (8.14) can be bounded from below with the choice v_{1} = 8n^{1/3}u^{2/3} and v_{2} = \frac{1}{8}n^{2/3}u^{1/3} by means of Example 3.2. (The estimate of Example 3.2 can be applied with the choice y = v_{1}, since by the inequality \frac{n}{8} ≥ v_{1} ≥ nσ^{2} the conditions of Example 3.2 are satisfied), together with the large-deviation result mentioned after the remark. These estimates together yield that

P ((∑_{j=1}^{n} η_{j,1}) (∑_{j=1}^{n} η_{j,2}) > 2nu) ≥ P (∑_{j=1}^{n} η_{j,1} > v_{1}) P (∑_{j=1}^{n} η_{j,2} > v_{2})

≥ exp \{−B_{1}v_{1} log (\frac{v_{1}}{nσ^{2}} ) − B_{2} \frac{v_{2}^{2}}{n} \} ≥ exp \{−B_{3}n^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\}

with appropriate constants B_{1} > 0, B_{2} > 0 and B_{3} > 0. On the other hand by applying Bennett’s inequality, more precisely its consequence given in formula (3.4) for the sum of the random variables X_{j} = 2η_{j,1}η_{j,2} and y = nu we get the following upper bound for the second term at the right-hand side of (8.14):

P (2 ∑_{j=1}^{n} η_{j,1}η_{j,2} > nu) ≤ exp \{−B_{4}nu log \frac{u}{σ^{2}} \}

≤ exp \{−2B_{5}n^{1/3}u^{2/3} log (\frac{u}{nσ^{3}} )\} ,

since nu ≥ B̄n^{1/3}u^{2/3} ≥ B̄nσ^{2}, and the estimate (3.4) is applicable if B̄ is sufficiently large. The above estimates imply the statement of the example.

9. Some results about U-statistics

This section contains the proof of an important result about U-statistics, the so-called Hoeffding decomposition theorem which states that all U-statistics can be represented as a sum of degenerate U-statistics. Let us consider the kernel function of a U-statistic together with the kernel functions of the U-statistics in its Hoeffding decomposition. It will also be shown that the L_{2}-norm of the kernel functions of the U-statistics in the Hoeffding decomposition are bounded by the L_{2}-norm of the kernel function of the original U-statistic. Besides, if a class of U-statistics is given with an L_{2}-dense class of kernel functions (with the same underlying sequence of independent and identically distributed random variables) and the Hoeffding decomposition of all of these U-statistics is taken, then the kernel functions of the degenerate U-statistics taking part in the Hoeffding decomposition also constitute an L_{2}-dense class. Another important result of this section is a decomposition of a k-fold random integral with respect to a normalized empirical measure to the linear combination of degenerate U-statistics presented in Theorem 9.4. These results enable us to prove the equivalence of Theorem 8.1 with Theorem 8.3 and of Theorem 8.2 with Theorem 8.4. They are also useful in the proof of Theorems 8.3 and 8.4.

In the special case k = 1 Hoeffding’s decomposition means that the sum S_{n} = ∑_{j=1}^{n} ξ_{j}

of iid. random variables can be rewritten as S_{n} = ∑_{j=1}^{n}(ξ_{j} − Eξ_{j}) + ( ∑_{j=1}^{n} Eξ_{j}), i.e. the

sum of independent random variables with zero expectation plus a constant. We may consider a constant as a U-statistic of order zero. (For the sake of a simpler terminology in the sequel let us consider a constant as a degenerate U-statistic of order zero, and define I_{n,0}(c) = c for a constant c.) I wrote down this trivial calculation, because Hoeffding’s decomposition is actually an adaptation of this procedure to the general case. To understand this let us see how to adapt this construction in the case k = 2. In this case we have to consider a sum of the form I_{n,2}(f) = ∑_{1≤j,k≤n,j≠k} f(ξ_{j}, ξ_{k}).

Write f(ξ_{j}, ξ_{k}) = [f(ξ_{j}, ξ_{k}) − Ef(ξ_{j}, ξ_{k}|ξ_{k})] + Ef(ξ_{j}, ξ_{k}|ξ_{k}) = f_{1}(ξ_{j}, ξ_{k}) + f̄_{1}(ξ_{k}) with f_{1}(ξ_{j}, ξ_{k}) = f(ξ_{j}, ξ_{k}) − Ef(ξ_{j}, ξ_{k}|ξ_{k}), and f̄_{1}(ξ_{k}) = Ef(ξ_{j}, ξ_{k}|ξ_{k}) to achieve that the conditional expectation of f_{1}(ξ_{j}, ξ_{k}) for fixed ξ_{k} be zero. Repeating this procedure for the first coordinate we define f_{2}(ξ_{j}, ξ_{k}) = f_{1}(ξ_{j}, ξ_{k}) − Ef_{1}(ξ_{j}, ξ_{k}|ξ_{j}) and f̄_{2}(ξ_{j}) = Ef_{1}(ξ_{j}, ξ_{k}|ξ_{j}). Simple calculation shows that I_{n,2}(f_{2}) is a degenerate U-statistics of order 2, and the identity I_{n,2}(f) = I_{n,2}(f_{2})+I_{n,1}((n−1)(f̄_{1} −Ef̄_{1}))+I_{n,1}((n−1)((f̄_{2} − Ef̄_{2})) + n(n − 1)E(f̄_{1} + f̄_{2}) yields the decomposition of I_{n,2}(f) for sums of degenerate U-statistics.

We get the Hoeffding decomposition by working out the details of the above argument in the general case. But it is simpler to calculate the appropriate conditional expectations with the help of the kernel functions of the U-statistics. To carry out such a program in the study of U-statistics of order k we introduce the following notations.

Let us consider the k-fold product (X^{k}, X^{k}, μ^{k}) of a measure space (X, X , μ) with some probability measure μ, and define for all integrable functions f(x_{1}, …, x_{k}) and indices 1 ≤ j ≤ k the projection P_{j}f of the function f to its j-th coordinate as

P_{j}f(x_{1}, …, x_{j−1}, x_{j+1}, …, x_{k}) = ∫ f(x_{1}, …, x_{k})μ( dx_{j}), 1 ≤ j ≤ k.  (9.1)

Let us also define the operators Q_{j} = I − P_{j} as Q_{j}f = f − P_{j}f on the space of integrable functions on (X^{k}, X^{k}, μ^{k}), 1 ≤ j ≤ k. In the definition (9.1) P_{j}f is a function not depending on the coordinate x_{j}, but in the definition of Q_{j} we introduce the fictive coordinate x_{j} to make the expression Q_{j}f = f − P_{j}f meaningful. Now we can formulate the following result.

Theorem 9.1 (Hoeffding decomposition). Let f(x_{1}, …, x_{k}) be an integrable function on the k-fold product space (X^{k}, X^{k}, μ^{k}) of a space (X, X , μ) with a probability measure μ. It has the decomposition

f = ∑_{V ⊂{1,...,k}} f_{V} , with f_{V} (x_{j}, j ∈ V) = ( ∏_{j∈{1,...,k}\V} P_{j} ∏_{j∈V} Q_{j}) f(x_{1}, …, x_{k})

(9.2)

such that all functions f_{V}, V ⊂ {1, …, k}, in (9.2) are canonical with respect to the probability measure μ, and they depend on the |V | arguments x_{j}, j ∈ V.

Let ξ_{1}, …, ξ_{n} be a sequence of independent μ distributed random variables, and consider the U-statistics I_{n,k}(f) and I_{n,|V |}(f_{V}) corresponding to the kernel functions f, f_{V} defined in (9.2) and random variables ξ_{1}, …, ξ_{n}. Then

I_{n,k}(f) = ∑_{V ⊂\{1,...,k\}}(n − |V |)(n − |V | − 1) ⋯ (n − k + 1)\frac{|V |!}{k!}I_{n,|V |}(f_{V})  (9.3)

is a representation of I_{n,k}(f) as a sum of degenerate U-statistics, where |V | denotes the cardinality of the set V. (The product (n − |V |)(n − |V | − 1) ⋯ (n − k + 1) is defined as 1 for V = {1, …, k}, i.e. if |V | = k.) This representation is called the Hoeffding decomposition of I_{n,k}(f).

The proof of Theorem 9.1. Write f = ∏_{j=1}^{k}(P_{j} +Q_{j})f. By carrying out the multiplications

in this identity and applying the commutativity of the operators P_{j} and Q_{j} for different indices j we get formula (9.2). To show that the functions f_{V} in formula (9.2) are canonical let us observe that this property can be rewritten in the form P_{j}f_{V} = 0 (in all coordinates x_{s}, s ∈ V \ {j} if j ∈ V). Since P_{j} = P_{j}^{2}, and the identity P_{j}Q_{j} = P_{j} − P_{j}^{2} = 0 holds for all j ∈ {1, …, k} this relation follows from the above mentioned commutativ-

ity of the operators P_{j} and Q_{j}, as P_{j}f_{V} = ( ∏_{s∈{1,...,k}\V} P_{s} ∏_{s∈V \{j}} Q_{s}) P_{j}Q_{j}f = 0. By

applying the identity (9.2) for all terms f(ξ_{j_{1}}, …, ξ_{j_{k}}) in the sum defining the U-statistic I_{n,k}(f) and then summing them up we get relation (9.3).

The next result enables us to estimate the kernel functions of the degenerate Ustatistics in the Hoeffding-decomposition of a U-statistic by means of the properties kernel function of the original U-statistic.

Theorem 9.2. Let f(x_{1}, …, x_{k}) be a square integrable function on the k-fold product space (X^{k}, X^{k}, μ^{k}), and take its decomposition defined in formula (9.2). The inequalities

∫ f_{V}^{2} (x_{j}, j ∈ V) ∏_{j∈V} μ( dx_{j}) ≤ ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k})  (9.4)

and

sup_{x_{j}, j∈V} |f_{V} (x_{j}, j ∈ V)| ≤ 2^{|V |} sup_{x_{j}, 1≤j≤k} |f(x_{1}, …, x_{k})|  (9.4′)

hold for all V ⊂ {1, …, k}.

Let us consider an L_{2}-dense class F of functions with parameter D and exponent L on the space (X^{k}, X^{k}), take the decomposition (9.2) of all functions f ∈ F and define the classes of functions F_{V} = {2^{−|V |}f_{V} : f ∈ F} for all V ⊂ {1, …, k} with the help of the functions f_{V} taking part in this decomposition. The classes of functions F_{V} are also L_{2}-dense with the same parameter D and exponent L for all V ⊂ {1, …, k}.

Theorem 9.2 is a fairly simple consequence of Proposition 9.3 presented below. To formulate it first we introduce the following notations:

Let us consider the product (Y × Z, Y × Z) of two measurable spaces (Y, Y) and (Z, Z) together with a probability measure μ on (Z, Z) and the operator

P f(y) = P_{μ}f(y) = ∫ f(y, z)μ( dz), y ∈ Y, z ∈ Z  (9.5)

for all measurable functions f on the space Y × Z for which this integral is finite. Let I denote the identity operator on the space of functions on Y ×Z, i.e. let If(y, z) = f(y, z), and introduce the operator Q = Q_{μ} = I − P = I − P_{μ} which maps the functions f on the space Y × Z to functions on the space Y × Z given by the formula

\begin{aligned}Q_{μ}f(y, z) = (I − P_{μ})f(y, z) = f(y, z) − P_{μ}f(y, z) & = f(y, z) − ∫ f(y, z)μ( dz), \\ & y ∈ Y, z ∈ Z. \end{aligned} (9.6)

(Here, and in the sequel we shall sometimes identify a function g(y) defined on the space (Y, Y) with the function ḡ(y, z) = g(y) on the space (Y × Z, Y × Z) which actually does not depend on the coordinate z.) The following result will be proved:

Proposition 9.3. Let us consider the direct product (Y × Z, Y × Z) of two measure spaces (Y, Y) and (Z, Z) together with a probability measure μ on the space (Z, Z). Take the transformations P_{μ} and Q_{μ} defined in formulas (9.5) and (9.6). Given any probability measure ρ on the space (Y, Y) consider the product measure ρ × μ on (Y × Z, Y ×Z). Then the transformations P_{μ} and Q_{μ}, as maps from the space L_{2}(Y ×Z, Y × Z, μ × ρ) to L_{2}(Y, Y, ρ) and L_{2}(Y × Z, Y × Z, ρ × μ) respectively, have a norm less than or equal to 1, i.e.

∫ P_{μ}f(y)^{2}ρ( dy) ≤ ∫ f(y, z)^{2}ρ( dy)μ( dz),  (9.7)

and

∫ Q_{μ}f(y, z)^{2}ρ( dy)μ( dz) ≤ ∫ f(y, z)^{2}ρ( dy)μ( dz)  (9.8)

for all functions f ∈ L_{2}(Y × Z, Y × Z, ρ × μ).

If F is an L_{2}-dense class of functions f(y, z) in the product space (Y × Z, Y × Z), with parameter D and exponent L, then also the classes F_{μ} = {P_{μ}f, f ∈ F} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f − P_{μ}f), f ∈ F\} are L_{2}-dense classes with the same exponent L and parameter D in the spaces (Y, Y) and (Y × Z, Y × Z) respectively.

The following corollary of Proposition 9.3 is formally more general, but it is a simple consequence of this result. Actually we shall need this corollary.

Corollary of Proposition 9.3. Let us consider the product space (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}), a probability measure μ on the space (Z, Z) and define the transformations

P_{μ}f(y_{1}, y_{2}) = ∫ f(y_{1}, z, y_{2})μ( dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2}  (9.5′)

and

\begin{aligned}Q_{μ}f(y_{1}, z, y_{2}) & = (I − P_{μ})f(y_{1}, z, y_{2}) = f(y_{1}, z, y_{2}) − P_{μ}f(y_{1}, z, y_{2}) \\ & = f(y_{1}, z, y_{2}) − ∫ f(y_{1}, z, y_{2})μ( dz), y_{1} ∈ Y_{1}, z ∈ Z, y_{2} ∈ Y_{2} \end{aligned} (9.6′)

for the measurable functions f on the space Y_{1} × Z × Y_{2}. Then

∫ P_{μ}f(y_{1}, y_{2})^{2}ρ( dy_{1}, dy_{2}) ≤ ∫ f(y, z)^{2}(ρ × μ)( dy_{1}, dz, dy_{2}),  (9.7′)

for all probability measures ρ on (Y_{1}×Y_{2}, Y_{1}×Y_{2}), where ρ×μ is the product of the probability measure ρ on (Y_{1} × Y_{2}, Y_{1} × Y_{2}) and μ on (Z, Z), i.e. ρ × μ({y_{1}, z, y_{2}): (y_{1}, y_{2}) ∈ A, z ∈ B}) = ρ(A)μ(B) for all A ∈ Y_{1} × Y_{2}, B ∈ Z, and ρ × μ is its unique extension as a probability measure on (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}). Also the inequality

∫ Q_{μ}f(y_{1}, z, y_{2})^{2}ρ( dy_{1}, dy_{2})μ( dz) ≤ ∫ f(y_{1}, z, y_{2})^{2}ρ( dy_{1}, dy_{2})μ( dz)  (9.8′)

holds for all functions f ∈ L_{2}(Y × Z, Y × Z, ρ × μ).

If F is an L_{2}-dense class of functions f(y_{1}, z, y_{2}) in the product space (Y_{1} × Z × Y_{2}, Y_{1}×Z×Y_{2}), with parameter D and exponent L, then also the classes F_{μ} = {P_{μ}f, f ∈ F\} and G_{μ} = \{\frac{1}{2}Q_{μ}f = \frac{1}{2}(f − P_{μ}f), f ∈ F\} are L_{2}-dense classes with exponent L and parameter D in the spaces (Y_{1} ×Y_{2}, Y_{1} ×Y_{2}) and (Y_{1} ×Z ×Y_{2}, Y_{1} ×Z ×Y_{2}) respectively.

This corollary is a simple consequence of Proposition 9.3 if we apply it with (Y, Y) = (Y_{1}×Y_{2}, Y_{1}×Y_{2}) and take the natural mapping f((y_{1}, y_{2}), z) → f(y_{1}, z, y_{2}) of a function from the space (Y × Z, Y × Z) to a function on (Y_{1} × Z × Y_{2}, Y_{1} × Z × Y_{2}), and use the correspondence between the product measure ρ × μ in these spaces.

Proposition 9.3, more precisely its corollary implies Theorem 9.2, since it implies that the operators P_{s}, Q_{s}, 1 ≤ s ≤ k, applied in Theorem 9.2 do not increase the L_{2}(μ) norm of a function f, and it is also clear that the norm of P_{s} is bounded by 1 the norm of Q_{s} = I − P_{s} is bounded by 2 as an operator from L_{∞} spaces to L_{∞} spaces. The corollary of Proposition 9.3 also implies that if F is an L_{2}-dense class of functions with parameter D and exponent L, then the same property holds for the classes of functions F_{P_{s}} = \{P_{s}f: f ∈ F\} and F_{Q_{s}} = \{\frac{1}{2}O_{s}f: f ∈ F\}, 1 ≤ s ≤ k. These relations together

with the identity f_{V} = ( ∏_{s∈V} P_{s} ∏_{s∈{1,...,k}\V} Q_{s}) f imply Theorem 9.2.

Proof of Proposition 9.3. The Schwarz inequality yields that P_{μ}(f)^{2} ≤ ∫ f(y, z)^{2}μ( dz), and integrating this inequality with respect to the probability measure ρ( dy) we get inequality (9.7). Also the inequality

∫ Q_{μ}f(y, z)^{2}ρ(dy)μ(dz) = ∫ [f(y, z) − P_{μ}f(y, z)]^{2}ρ(du)μ(dz) ≤ ∫ f(y, z)^{2}ρ(dy)μ(dz) holds, and this is relation (9.8). It follows for instance from the observation that the functions f(y, z) − P_{μ}f(y, z) and P_{μ}f(y, z) are orthogonal in the space L_{2}(Y × Z, Y × Z, ρ × μ).

Let us consider an arbitrary probability measure ρ on the space (Y, Y). To prove that F_{μ} is an L_{2}-dense class with parameter D and exponent L we have to find m ≤ Dε^{L} functions f_{j} ∈ F_{μ}, 1 ≤ j ≤ m, such that inf_{1≤j≤m} ∫ (f_{j} − f)^{2} dρ ≤ ε^{2} for all f ∈ F_{μ}. But

a similar property holds in the space Y × Z with the probability measure ρ × μ. This property together with the L_{2} contraction property of P_{μ} formulated in (9.7) imply that F_{μ} is an L_{2}-dense class.

To prove that G_{μ} is also L_{2}-dense with parameter D and exponent L we have to find for all numbers 0 < ε ≤ 1 and probability measures ρ on Y × Z a subset

{g_{1}, …, g_{m}} ⊂ G_{μ} with m ≤ Dε^{−L} elements such that inf_{1≤j≤m} ∫ (g_{j} − g)^{2} dρ ≤ ε^{2} for all

g ∈ G_{μ}.

Let us consider the probability measure ρ̃ = \frac{1}{2}(ρ+ρ̄×μ) on (Y ×Z, Y×Z), where ρ̄ is the projection of the measure ρ to (Y, Y), i.e. ρ̄(A) = ρ(A×Z) for all A ∈ Y, take a class

of function F_{0}(ε, ρ̃) = {f_{1}, …, f_{m}} ∈ F, m ≤ Dε^{−L} such that inf_{1≤j≤m} ∫ (f_{j} − f)^{2} dρ̃ ≤ ε^{2}

for all f ∈ F, and put \{g_{1}, …, g_{m}\} = \{\frac{1}{2}Q_{μ}f_{1}, …, \frac{1}{2}Q_{μ}f_{m}\}. All functions g ∈ G_{μ} can be written in the form g = \frac{1}{2}Q_{μ}f with some f ∈ F, and there exists some function f_{j} ∈ F_{0}(ε, ρ̃) such that ∫ (f −f_{m})^{2} dρ̃ ≤ ε^{2}. Hence to complete the proof of Proposition 9.3 it is enough to show that ∫ \frac{1}{4}(Q_{μ}f − Q_{μ}f̄)^{2} dρ ≤ ∫ (f − f̄)^{2} dρ̃ for all pairs f, f̄ ∈ F. This inequality holds, since ∫ \frac{1}{4}(Q_{μ}f − Q_{μ}f̄)^{2} dρ ≤ ∫ \frac{1}{2}(f − f̄)^{2} dρ + ∫ \frac{1}{2}(P_{μ}f − P_{μ}f̄)^{2} dρ, and ∫ (P_{μ}f − P_{μ}f̄)^{2} dρ = ∫ (P_{μ}f − P_{μ}f̄)^{2} dρ̄ ≤ ∫ (f − f̄)^{2} d(ρ̄× μ) by formula 9.7. The above relations imply that ∫ \frac{1}{4}(Q_{μ}f − Qμf̄)^{2} dρ ≤ ∫ (f − f̄)^{2} \frac{1}{2}d (ρ + ρ̄ × μ) = ∫ (f − f̄)^{2}d ρ̃ as we have claimed.

Let us turn to the proof of the equivalence of Theorem 8.1′ with 8.3 and of Theorem 8.2 with 8.4. In Theorems 8.2 and 8.4 we can restrict our attention to the case when the class of functions F is countable, since the case of countably approximable classes can be simply reduced to this situation. Let us remark that the integration with respect to the measure μ_{n} − μ in the definition (4.8) of the integral J_{n,k}(f) means some kind of normalization, and no such normalization appears in the definition of the U-statistics I_{n,k}(f). This is the cause why degenerate U-statistics had to be considered in Theorems 8.3 and 8.4. The deduction of these results from Theorems 8.1′ and 8.2 is fairly simple if the underlying probability measure μ is non-atomic, since in this case the identity I_{n,k}(f) = J_{n,k}(f) holds for a canonical function with respect to the measure μ. Let us remark that the non-atomic property of the measure μ is needed in this argument not only because of the conditions of Theorems 8.1′ and 8.2, but since in the proof of the above relation we need the identity ∫ f(x_{1}, …, x_{k})μ( dx_{j}) = 0 in the case when the domain of integration is a set of the form X \ {x_{1}, …, x_{j−1}, x_{j+1}, …, x_{k}}.

The case of possibly atomic measures μ can be simply reduced to the case of nonatomic measures by means of the following enlargement of the space (X, X , μ). Let us introduce the product space (X̄, X̄, μ̄) = (X, X , μ)×([0, 1], B, λ), where B is the σ-algebra and λ is the Lebesgue measure on [0, 1]. Define the function f̄((x_{1}, u_{1}), …, (x_{k}, u_{k})) = f(x_{1}, …, x_{k}) on this enlarged space. Then I_{n,k}(f) = I_{n,k}(f̄), and the measure μ̄ = μ×λ is non-atomic. Hence we can deduce the estimates of Theorems 8.3 and 8.4 from Theorems 8.1′ and 8.2 by deducing them first for their counterpart in the above constructed enlarged space and the above defined functions.

The deduction of Theorems 8.1′ and 8.2 from Theorems 8.3 and 8.4 requires more work. Let us observe that an integral J_{n,k}(f) can be written as a sum of U-statistics of different order, and by applying the Hoeffding decomposition for each term in this sum we can express the integral J_{n,k}(f) as a sum of degenerate U-statistics. We show that the coefficients of the degenerate U-statistics in the above representation have relatively small coefficients. This is the content of the following Theorem 9.4. To make its content more understandable I formulated its main statement in the case of random integrals of multiplicity two in a more explicit form.

Theorem 9.4. Let us have a non-atomic measure μ on a measurable space (X, X) together with a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}, and take a function f(x_{1}, …, x_{k}) of k variables on the space (X^{k}, X^{k}) such that

∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞.

Let us consider the empirical distribution function μ_{n} of the sequence ξ_{1}, …, ξ_{n} introduced in (4.5) together with the k-fold random integral J_{n,k}(f) of the function f defined in (4.8). The identity

J_{n,k}(f) = ∑_{V ⊂{1,...,k}} C(n, k, V)n^{−|V |/2}I_{n,|V |}(f_{V}),  (9.9)

holds with the set of (canonical) functions f_{V} (x_{j}, j ∈ V) (with respect to the measure μ) defined in formula (9.2) together with some real numbers C(n, k, V), V ⊂ {1, …, k}, where I_{n,|V |}(f_{V}) denotes the (degenerate) U-statistic of order |V | with the random variables ξ_{1}, …, ξ_{n} and kernel function f_{V}. The constants C(n, k, V) in formula (9.9) satisfy the inequality |C(n, k, V)| ≤ C(k) with some constant C(k) depending only on the order k of the integral J_{n,k}(f). The relations lim_{n→∞} C(n, k, V) = C(k, V) with some appropriate constant such that 0 ≤ |C(k, V)| < ∞ and C(n, k, {1, …, k}) = 1 for V = {1, …, k} also hold.

Remark: Some considerations show that the coefficients C(n, k, V) in formula (9.9) depend only on the cardinality |V | of the set V, i.e. we can write C(n, k, V) = C(n, k, |V |). We shall not need this observation.

Theorems 8.1′ and 8.2 can be simply deduced from Theorems 8.3 and 8.4 respectively with the help of Theorem 9.4. Indeed, to deduce Theorem 8.1′ we can write with the help of formula 9.9

P (|J_{n,k}(f)| > u) ≤ ∑_{V ⊂\{1,...,k\}} P (n^{−|V |/2}|I_{n,|V |}(f_{V})| > \frac{u}{2^{k}C(k)})  (9.10)

with a constant C(k) satisfying the inequality C(n, k, |V |) ≤ C(k) for all coefficients C(n, k, |V |) in (9.9). Then we get Theorem 8.1′ from Theorem 8.3 and relations (9.4) and (9.4′) in Theorem 9.2 by which the L_{2}-norm of the functions f_{V} are bounded by the L_{2}-norm of the function f and the L_{∞}-norm of f_{V} is bounded by the 2^{|V |}- times the L_{∞}-norm or f if we estimate each term at the right-hand side of (9.10) by means of Theorem 8.3. Here we may assume that 2^{k}C(k) > 1 and let us first assume that also the inequality \frac{u}{2^{k}C(k)σ} ≥ 1 holds. In this case we get formula (8.3′) in Theorem 8.1′ by estimating each term at the right-hand side of (9.10). Observe that

exp \{−α (\frac{u}{2^{k}C(k)σ} )^{2/s}\} ≤ exp \{−α (\frac{u}{2^{k}C(k)σ} )^{2/k}\} for all s ≤ k if \frac{u}{2^{k}C(k)σ} ≥ 1. If

\frac{u}{2^{k}C(k)σ} ≤ 1, then formula (8.3′) holds with a sufficiently large C > 0.

Theorem 8.2 can be similarly deduced from Theorem 8.4 if we observe that relation (9.10) remains valid if we replace |J_{n,k}(f)| by sup_{f∈F} |J_{n,k}(f)| and |I_{n,|V |}(f_{V})| by

sup_{f_{V} ∈F_{V}} |I_{n,|V |}(f_{V})| in it, and the constant M in formula (8.6) of Theorem 8.2 is chosen

sufficiently large. The only difference is that now we have to exploit besides formulas (9.4) and (9.4′) of Theorem 9.2 the last statement of this result which tells that if F is an L_{2}-dense class of functions on a space (X^{k}, X^{k}), then the classes of functions F_{V} = {2^{−|V |}f_{V} : f ∈ F} are also L_{2}-dense classes of functions for all V ⊂ {1, …, k} with the same exponent and parameter.

In the definition of the random integrals J_{n,k}(f) we have integrated in all coordinates with respect to the signed measure μ_{n} − μ, and this means some kind of normalization. Thus it is not surprising that the tail behaviour of the distribution of J_{n,k}(f) is similar to that of certain degenerate U-statistics. Theorem 9.4 formulates such a relation. Formula (9.9) expresses the random integral J_{n,k}(f) as a linear combination of degenerate U-statistics of different order. It is similar to the Hoeffding decomposition in that respect that the functions f_{V} in formula (9.9) agree with the functions f_{V} appearing in the Hoeffding decomposition of the U-statistic I_{n,k}(f) with kernel function f. But the coefficients in the expansion (9.9) are small. On the other hand, these coefficients need not disappear. In particular, the expansion (9.9) may contain a non-zero constant term. In such a case the expected value EJ_{n,k}(f) may not equal zero, but it can be bounded by a number not depending on the sample size n. In the next example I show that there are really random integrals J_{n,k}(f) such that EJ_{n,k}(f) ≠ 0.

Let us choose a sequence of independent random variables ξ_{1}, …, ξ_{n} with uniform distribution on the unit interval, let μ_{n} denote its empirical distribution, let f = f(x, y) denote the indicator function of the unit square, i.e. let f(x, y) = 1 if 0 ≤ x, y ≤ 1, and f(x, y) = 0 otherwise. Let us consider the random integral J_{n,2}(f) = n ∫_{x≠y} f(x, y)(μ_{n}( dx)− dx)(μ_{n}( dy)−dy), and calculate its expected value EJ_{n,2}(f). By adjusting the diagonal x = y to the domain of integration and taking out the contribution obtained in this way we get that EJ_{n,2}(f) = nE(∫_{0}^{1} (μ_{n}( dx) − μ( dx))^{2} − n^{2} · \frac{1}{n^{2}} = −1. (The last term is the integral of the function f(x, y) on the diagonal x = y with respect to the product measure μ_{n}×μ_{n} which equals (μ_{n}−μ)×(μ_{n}−μ) on the diagonal.)

The above considerations and the proof of Theorem 9.4 indicate that the equivalence between Theorems 8.1′ and 8.3 or between Theorems 8.2 and 8.4 is not self-evident. It is simpler to prove Theorems 8.3 and 8.4 of these theorem pairs about degenerate Ustatistics, and this will be done in this work. On the other hand, Theorems 8.1′ and 8.2 seem to be more appropriate for applications, since here we do not have to restrict our attention to special, canonical kernel functions.

The proof of Theorem 9.4. Let us first introduce the (random) probability measures μ^{(l)}, 1 ≤ l ≤ n, concentrated in the sample points ξ_{l}, i.e. let μ^{(l)}(A) = 1 if ξ_{l} ∈ A, and

μ^{(l)}(A) = 0 if ξ_{l} ∉ A, A ∈ A. Then μ_{n} − μ = \frac{1}{n} (∑_{l=1}^{n} (μ^{(l)} − μ)), and formula (4.8) can

be rewritten as

J_{n,k}(f) = \frac{1}{n^{k/2}k!} ∑_{(l_{1},...,l_{k}), 1≤l_{j}≤n, 1≤j≤k} ∫ ′ f(x_{1}, …, x_{k})  (9.11)

(μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k})) .

To rearrange the above sum in a way more appropriate for us let us introduce the class of all partitions P = P_{k} of the set {1, 2, …, k}. For a partition P = {R_{1}, …, R_{u}}

⋃_{j=1}^{u} R_{j} = {1, …, k}, R_{j} ∩ R_{l} = ∅, 1 ≤ j < l ≤ u, the sets R_{j}, 1 ≤ j ≤ u, will be called

the components of the partition P. Given a sequence (l_{1}, …, l_{k}), 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, of length k let P_{H}(l_{1}, …, l_{k}) denote that partition of P_{k} in which two points s and t, 1 ≤ s, t ≤ k, belong to the same component if and only if l_{s} = l_{t}. For a partition P ∈ P_{k} let us define the set of sequences H(P) = H_{n}(P) as H(P) = {(l_{1}, …, l_{k}): 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, P_{H}(l_{1}, …, l_{k}) = P }.

Let us rewrite formula (9.11) in the form

J_{n,k}(f) = \frac{1}{n^{k/2}k!} ∑_{P ∈P} ∑_{(l_{1},...,l_{k}) : (l_{1},...,l_{k})∈H(P)} ∫ ′ f(x_{1}, …, x_{k})  (9.12)

(μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k})) .

Let us remember that the diagonals x_{s} = x_{t}, s ≠ t, were omitted from the domain of integration in the formula defining J_{n,k}(f). This implies that in the case l_{s} = l_{t} the measure μ^{(l_{s})}( dx_{s})μ^{(l_{t})}( dx_{t}) has zero measure in the domain of integration. We have to understand the cancellation effects caused by this relation. It will be shown that because of these cancellations the expression in formula (9.12) can be rewritten as a linear combination of degenerate U-statistics with not too large coefficients. Besides, it will be seen from the calculations that the same degenerate U-statistics I_{n,|V |}(f_{V}) appear in this representation of J_{n,k}(f) which were defined in formula (9.2). This seems to be a natural approach, but the detailed proof demands some rather unpleasant calculations.

Let us fix some partition P ∈ P and investigate the integrals in the internal sum at the right-hand side of (9.12) corresponding to the sequences (l_{1}, …, l_{k}) ∈ H(P). For the sake of better understanding let us first consider such a partition P ∈ P which has a component of the form {1, …, s} with some s ≥ 2. The products of measures by which we have to integrate in this case contain a part of length s of the form (μ^{(l)}(dx_{1}) − μ(dx_{1})) … (μ^{(l)}(dx_{s}) − μ(dx_{s})) This part of the product measure can be rewritten in the domain of integration as

∑_{j=1}^{s}(−1)^{s−1}μ( dx_{1}) … μ( dx_{j−1})μ^{(l)}( dx_{j})μ( dx_{j+1}) … μ( dx_{s}) + (−1)^{s}μ(dx_{1}) … μ(dx_{s})

= ∑_{j=1}^{s}(−1)^{s−1}μ( dx_{1}) … μ( dx_{j−1})(μ^{(l)}( dx_{j}) − μ( dx_{l}))μ( dx_{j+1}) … μ( dx_{s})

+ (−1)^{s−1}(s − 1)μ(dx_{1}) … μ(dx_{s}).  (9.13)

Here we exploit that all other terms of this product disappear in the domain of integration which does not contain the diagonals. Let us also observe that the term (−1)^{s−1}(s − 1)μ(dx_{1}) … μ(dx_{j}) appears n-times if we sum up for all 1 ≤ l ≤ n. We have assumed that s ≥ 2, since the case s = 1 is slightly different. In this case only the term μ^{(l)}(dx_{1}) − μ( dx_{1}) appears, i.e. have to put no additional term consisting only of (deterministic) measures μ.

More generally, let us fix some partition P = {R_{1}, …, R_{u}}, consider the integral corresponding to a sequence (l_{1}, …, l_{k}) ∈ H(P) in the internal sum of (9.12), and let us rewrite it as the sum of integrals with respect to product measures with components of the form μ^{(l_{s})}( dx_{s})−μ( dx_{s}) or μ( dx_{s}), where all measures μ^{(l_{s})} appearing in a product measure are different. Such a representation can be given, similarly to the argument of relation (9.13), only the notations will be more complicated. To write down what we get first we define a class of subsets T (P) of the set {1, …, k} depending on the partition P = {R_{1}, …, R_{u}} together with a subclass T̄(P) of it. Let T (P) consist of all such sets {j_{1}, …, j_{u′}} ⊂ {1, …, k}, u′ ≤ u, for which all numbers j_{1}, …, j_{u′} belong to a different component of the partition P. Let T̄(P) ⊂ T (P) consist of those sets V = {j_{1}, …, j_{u′}} ∈ T (P) which also satisfy the following additional condition: If some components R_{t} = {b_{t}}, 1 ≤ t ≤ u, of the partition P consists of only one point, then the sets V belonging to T̄(P) ⊂ T (P) contain this point b_{t}. With the help of the above quantities we can write in the case (l_{1}, …, l_{k}) ∈ H(P), similarly to the calculation in (9.13),

∫ ′ f(x_{1}, …, x_{k}) (μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k}))  (9.14)

= ∑_{V ∈T̄ (P)} α(V, P) ∫ f(x_{1}, …, x_{k}) ∏_{j∈V} (μ^{(l_{j})}( dx_{j}) − μ( dx_{j})) ∏_{j′∈{1,...,k}\V} μ( dx_{j′})

with some appropriate finite constants α(V, P). These constants could be calculated explicitly, but it is enough for us to know that they depend only on the partition P and the set V ∈ T̄(P). (Actually it was important for us to observe that we get a term with non-zero coefficient at the right-hand side of (9.14) only for V ∈ T̄(P), and the class of functions T̄(P) was introduced because of this reason. This property in the decomposition of the integral (9.14) holds, since in the case of a one-point component R_{t} = {b_{t}} of the partition P only the term μ^{(l_{b_{t}})}( dx_{b_{t}}) − μ( dx_{b_{t}}) appears in the component of product of measures in (9.14), a component of the form μ( dx_{b_{t}}) is missing.)

Let me remark that at the right-hand side of (9.14) I wrote ∫ instead of integral ∫′, i.e. I did not omit the diagonal from the domain of integration. This is allowed, since the measure μ is non-atomic, and this also has the consequence that the sample points ξ_{1}, …, ξ_{n} are different with probability 1.

Formula (9.14) can be rewritten, by expressing its right-hand side with the help of the random variables ξ_{l} instead of the measures μ^{(l)} as

∫ ′ f(x_{1}, …, x_{k}) (μ^{(l_{1})}( dx_{1}) − μ( dx_{1})) … (μ^{(l_{k})}( dx_{k}) − μ( dx_{k}))  (9.15)

= ∑_{V ∈T̄ (P)} α(V, P) (( ∏_{j′∈{1,...,k}\V} P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f) (ξ_{l_{j}}, j ∈ V).

Here Q_{μ,j} = I − P_{μ,j} is the operator Q_{μ} defined in (9.6′), with the choice Y_{1} which is the product of the first j − 1 components of X^{k}, Z is the j-th component and Y_{2} is the product of the last k − j components of the product space X^{k}. The operator P_{μ,j′} is the operator P_{μ} defined in (9.5′) with the choice of Y_{1} as the product of the first j′ − 1, Z the j-th component and Y_{2} as the procuct of the last k − j′ components of the space X^{k}. To see why formula (9.15) holds we have to understand that integration with respect to (μ^{(l_{j})}( dx_{j}) − μ( dx_{j})) means the application of the operator Q_{μ,j} and then putting the value ξ_{l_{j}} in the argument x_{j}, while integration with respect to μ( dx_{j′}) means the application of the operator P_{μ,j′}. Besides, the operators Q_{μ,j} and P_{μ,j′} are exchangeable.

Let us fix some partition P ∈ P_{k}, a set V ∈ T̄(P) and sum up the expressions at the right-hand side of (9.15) with this set V for all sequences (l_{1}, …, l_{k}) ∈ H(P). We get that

α(V, P) ∑_{(l_{1},...,l_{k})∈H(P)} ( ∏_{j′∈{1,...,k}\V}P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f(ξ_{l_{j}}, j ∈ V) = ᾱ(V, P, k, n)I_{n,|V |}(f_{V})

(9.16)

where I_{n,|V |} is a U-statistic of order |V | with the kernel function f_{V} (x_{j}, j ∈ V) =

( ∏_{j′∈{1,...,k}\V} P_{μ,j′} ∏_{j∈V} Q_{μ,j}) f with our function on f ∈ (X^{k}, X^{k}), and the coefficients

ᾱ(V, P, k, n) at the right-hand side of (9.16) (which could be calculated explicitly, but we do not need this formula) satisfy the inequality |ᾱ(V, P, k, n)| ≤ D(k)n^{β(P,V)}, where β(P, V) = u − |V | is the number of those components R_{j}, 1 ≤ j ≤ u, of the partition P for which R_{j} ∩ V = ∅, and the constant D(k) < ∞ depends only on the multiplicity k of the integral J_{n,k}(f).

To understand why ᾱ(V, P, k, n) can be bounded by D(k)n^{β(P,V)} let us observe that if we first fix the coordinates l_{j}, j ∈ V, and sum up for the remaining indices l_{j′}, j′ ∉ V, at the left-hand side of (9.16), then we get the term depending on the variables ξ_{l_{j}}, j ∈ V, in the sum defining the U-statistic I_{n,|V |}(f_{V}) multiplied by ᾱ(V, P, k). To get a good estimate on ᾱ(V, P, k, n) we have to bound the number of choices for the non-fixed coordinates l_{j′}, j′ ∉ V. For this aim let us consider the class of vectors (l_{1}, …, l_{k}) ∈ H(P). Two coordinates l_{j′} and l_{j′′} must agree if their indices j′ and j′′ belong to the same component of the partition P. Besides, if the number j is contained in such a component R_{t} of the partition P for which R_{t} ∩ V ≠ ∅, then the coordinate l_{j} of these vectors is fixed. Hence the value l_{j′} of those non-fixed coordinates whose indices j′ belong to the same component R_{t} of the partition P agree and only such components R_{t} have to be considered for which R_{t} ∩ V = ∅. This yields the upper bound n^{β(P,V)} for the number of possible choices of the indices l_{j′}, j′ ∉ V. A more careful consideration shows that the finite limit

C(k, V, P) = lim_{n→∞} n^{−β(P,V)}ᾱ(V, P, k, n),  |C(k, V, P)| < ∞  (9.17)

also exists.

We get by applying relation (9.12) and summing up relation (9.16) first for all V ∈ T̄(P) for a partition P ∈ P_{k} and then for all P ∈ P that the identity

J_{n,k}(f) = ∑_{V ⊂\{1,2,...,k\}} C(n, k, V)n^{−|V |/2} \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ for j∈V \end{matrix}} f_{V} (ξ_{l_{j}}, j ∈ V) (9.18)

holds with the functions

f_{V} (x_{j}, j ∈ V) = ( ∏_{j∈V} Q_{μ,j} ∏_{j′∈{1,...,k}\V} P_{μ,j′} ) f for all V ⊂ {1, …, k}  (9.19)

and some coefficients C(n, k, V). We shall show that these coefficients satisfy the inequality |C(n, k, V)| ≤ C(k) with some constant C(k) > 0. Besides, it is not difficult to see that the identity C(n, k, {1, …, k}) = 1 holds. To see that the estimate |C(n, k, V)| ≤ C(k) really holds, observe that n^{−|V |/2}C(n, k, |V |) can be written as a sum of finitely many terms, (the number of terms can be bounded by a number depending only on k) such that all of them can be bounded by a number of the form D(k)n^{−k/2+β(P,V)} with some partition P and the number β(P, V) introduced after formula (9.16) with some P ∈ P_{k} and V ∈ T̄(P). Hence it is enough to show that −\frac{k}{2} + β(P, V) ≤ −\frac{|V |}{2}, i.e. β(P, V) ≤ \frac{k−|V |}{2} if V ∈ T̄(P). This relation clearly holds, since β(P, V) is the number of components of a partition of a set with cardinality less than or equal to k − |V |, and all components of this partition have a cardinality at least 2.

Relation (9.18) can be rewritten as J_{n,k}(f) = ∑_{V ⊂{1,2,...,k}}C(n, k, V)n^{−|V |/2}I_{n,|V |}(f_{V}),

where I_{n,|V |}(f_{V}) is the U-statistic with the random variables ξ_{1}, …, ξ_{n} and the kernel function f_{V} defined in (9.19) agrees with the function f_{V} defined in (9.2). We have also seen that the coefficients C(n, k, V) satisfy the inequality stated in Theorem 9.4. Relation (9.17) together with the bound on the terms β(P, V) also imply that the finite limits lim_{n→∞} C(n, k, V) = C(k, V) also exist. Theorem 9.4 is proved.

I formulate two corollaries of Theorem 9.4. The first one explains the content of conditions (8.2) and (8.5) in Theorems 8.1—8.4.

Corollary 1 of Theorem 9.4. If I_{n,k}(f) is a degenerate U-statistic of order k with some kernel function f, then E (n^{−k/2}I_{n,k}(f))^{2} ≤ \frac{1}{k!} ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}), where μ is the distribution of the random variables taking part in the definition of the U-statistic I_{n,k}(f). Analogously, the k-fold multiple random integral J_{k,n}(f) satisfies the inequality E (n^{−k/2}J_{n,k}(f))^{2} ≤ C̄(k) ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) with some constant C̄(k) depending only on the order k of the integral J_{n,k}(f).

Proof of Corollary 1 of Theorem 9.4. We have

E(n^{−k/2}I_{n,k}(f))^{2} = \frac{1}{(k!)^{2}n^{k}} ∑′Ef(ξ_{l_{1}}, …, ξ_{l_{s}})f(ξ_{l′_{1}}, …, ξ_{l′_{s}}),

where the prime in ∑′ means that summation is taken for such pairs of k-tuples (l_{1}, …, l_{k}), (l′_{1}, …, l′_{k}), 1 ≤ l_{j}, l′_{j} ≤ n, for which l_{j} ≠ l_{j′} and l′_{j} ≠ l′_{j′} if j ≠ j′. The degeneracy of the U-statistic I_{n,k}(f) implies that Ef(ξ_{l_{1}}, …, ξ_{l_{s}})f(ξ_{l′_{1}}, …, ξ_{l′_{s}}) = 0 if the two k-tuples (l_{1}, …, l_{s}) and (l′_{1}, …, l′_{s}) differ. This can be seen by taking such an index l_{j} from the first k-tuple which does not appear in the second one, and by observing that the conditional expectation of the product we consider equals zero by the degeneracy condition of the U-statistic under the condition that the value of all random variables except that of ξ_{l_{j}} is fixed in this product. There remains k!n(n − 1) ⋯ (n − k + 1) terms in the sum expressing E (n^{−k/2}I_{n,k}(f))^{2} which may be non-zero, and all of them can be bounded by ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) because of our conditions and the Schwarz inequality. These estimates yield the bound given for E (n^{−k/2}I_{n,k}(f))^{2}.

We can simply get the bound for J_{n,k}(f) with the help of Theorem 9.4, formula (9.4) in Theorem 9.4 by which the L_{2}-norm of the functions f_{V} can be bounded by the L_{2}-norm of the function f and the bound given for the second moment of degenerate U-statistics n^{−|V |/2}I_{n,|V |}(f_{V}) appearing in the expansion (9.9).

In Corollary 2 the decomposition (9.9) of a random integral J_{n,2}(f) of order 2 is described in an explicit way.

Corollary 2 of Theorem 9.4. Let the random integral J_{n,2}(f) satisfy the conditions of Theorem 9.4. In this case formula (9.9) can be written in the following explicit form:

J_{n,2}(f) = \frac{1}{n}I_{n,2}(f_{\{1,2\}}) − \frac{1}{n}I_{n,1}(f_{\{1\}}) − \frac{1}{n}I_{n,1}(f_{\{2\}}) − f_{∅}  (9.9′)

with the functions

f_{{1,2}}(x, y) = f(x, y) − ∫ f(x, y)μ( dx) − ∫ f(x, y)μ( dy) + ∫ f(x, y)μ( dx)μ( dy),
f_{{1}}(x) = ∫ f(x, y)μ( dy) − ∫ f(x, y)μ( dx)μ( dy),
f_{{2}}(y) = ∫ f(x, y)μ( dx) − ∫ f(x, y)μ( dx)μ( dy)

and f_{∅} = ∫ f(x, y)μ( dx)μ( dy).

10. The proof of Theorem 8.3 about the distribution of U-statistics

This section contains the proof of Theorem 8.3 about the distribution of degenerate Ustatistics with the help of some results which are interesting in themselves. One of these results, called Borell’s inequality, gives an estimate on the moments of homogeneous polynomials of Rademacher functions, another result we need is a symmetrization type estimate which can be considered as the multivariate version of the more interesting part of the Marcinkiewicz–Zygmund inequality. Finally there is a third result we apply which compares the distribution of a U-statistics with the distribution of an appropriate modification of it. The first two results will be proved in the next section, the third one in the Appendix.

Theorem 8.3 can be considered as the generalization of Bernstein’s inequality (Theorem 3.1) for U-statistics. Bernstein’s inequality was proved by means of an estimation of the moment-generating function of the partial sums of independent and bounded random variables. This approach has to be modified in the proof of Theorem 8.3. In such cases we cannot work well with the moment generating functions, since if the sample size tends to infinity, then the normalized version of degenerate U-statistics of order k have a limit distribution F with a tail-behaviour 1 − F (x) ≥ e^{−Cx^{2/k}} with some C > 0 as x → ∞. (This is a relatively well-known result, but we shall not need it in this work.) This means that a random variable with this limit distribution has no moment generating function for k ≥ 3. On the other hand, the proof of Theorem 8.3 is relatively simple, if we have a good estimate also for the high moments of degenerate U-statistics. Such a moment estimate is formulated in the following

Proposition 10.1. Let us consider a canonical function f = f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}, μ^{k}) of a measure space (X, X , μ) together with a sequence of independent μ distributed random variables and the degenerate U-statistic I_{n,k}(f) determined by this sequence of random variables ξ_{1}, …, ξ_{n} and canonical function f. Let us also assume that the function f satisfies conditions (8.1) and (8.2) with some number 0 < σ ≤ 1.

Then there exists some constants C = C_{k} > 0 such that the moments of the Ustatistic I_{n,k}(f) defined in formula (8.7) satisfy the inequality

E (|n^{−k/2}I_{n,k}(f)|^{2M}) ≤ C_{k}^{M}M^{kM}σ^{2M} if 1 ≤ M ≤ nσ^{2}.  (10.1)

Let us consider the k-th power of a standard normal random variable η and calculate the asymptotic magnitude of the 2M-th moment E (ση^{k})^{2M} of ση^{k} for large M. We

have E (ση^{k})^{2M} = 1 · 3 ⋯ (2kM − 1)σ^{2M} = \frac{(2kM)!}{2^{kM} (kM)!}σ^{2M} ∼ (\frac{2k}{e} )^{kM} M^{kM}σ^{2M} by the Stirling formula. This means that the estimate given for the 2M-th moments of a normalized U-statistics n^{−k/2}I_{n,k}(f) in formula (10.1) has the same order as the 2M-th moment of the random variable const. ση^{k}, at least if 1 ≤ M ≤ nσ^{2}. This estimate will imply Theorem 8.3 which also can be so interpreted that P (n^{−k/2}I_{n,k}(f) > u) can be bounded by const. P (const. ση^{k} > u), at least if 0 < u ≤ n^{k/2}σ^{k+1}.

The hard part of the problem is to prove Proposition 10.1. There are methods to bound the moments of multiple Wiener–Itô integrals, and it is natural to try to adapt them to the proof of Proposition 10.1. I know of two different methods for estimating the moments of Wiener–Itô integrals. One of them is the so-called diagram formula which expresses the product of Wiener–Itô integrals as sums of appropriate new Wiener–Itô integrals, the other one is called Nelson’s inequality which yields a direct comparison between the L_{p}-norms of Wiener–Itô integrals for different parameters p. Both of them can be adapted to our case, but they demand the solution of several non-trivial technical problems. The adaptation of Nelson’s inequality seems to be the less complicated method, and this approach will be followed in this work. There is an important estimate, called Borell’s inequality which will be applied. This inequality makes a comparison between the L_{p} norms of homogeneous polynomials of independent Rademacher functions for different parameters p. Borell’s inequality in itself will be not sufficient for us, because we want to estimate more complicated objects. But we shall formulate and prove some additional results, and they will enable us together with Borell’s inequality to prove a version of Proposition 10.1 which will be sufficient for our purposes.

Borell’s inequality will be formulated below, but its proof is postponed to the next section.

Theorem 10.2 (Borell’s inequality). Let ε_{1}, …, ε_{n} be independent, identically distributed random variables P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, fix some real numbers a(l_{1}, …, l_{k}) for all indices (l_{1}, …, l_{k}) such that 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, and l_{j} ≠ l_{j′} if j ≠ j′, and define the random variable

Z = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, 1≤j≤k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a(l_{1}, …, l_{k})ε_{l_{1}} ⋯ ε_{l_{k}}.  (10.2)

The inequality

E|Z|^{p} ≤ (\frac{p − 1}{q − 1})^{kp/2} (E|Z|^{q})^{p/q} if 1 < q ≤ p < ∞  (10.3)

holds.

Remark: The most interesting special case of Borell’s inequality is when q = 2, and we shall consider only this case. Since EZ^{2} ≤ \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, 1≤j≤k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a^{2}(l_{1}, …, l_{k}), it yields that

E|Z|^{p} ≤ (p − 1)^{kp/2} (\frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, 1≤j≤k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a^{2}(l_{1}, …, l_{k}))^{p/2} if 2 ≤ p < ∞  (10.4)

We have the estimate written for EZ_{n}^{2} because

Eε_{l_{1}} ⋯ ε_{l_{k}}a(l_{1}, …, l_{k})ε_{l′_{1}} ⋯ ε_{l′_{k}}a(l′_{1}, …, l′_{k}) = 0

if the sets of arguments {l_{1}, …, l_{k}} and {l′_{1}, …, l′_{k}} do not agree. In the inequality written for EZ_{n}^{2} we have identity if all coefficients a(l_{1}, …, l_{k}) are symmetric functions of their arguments, otherwise we can only write inequality.

Borell’s inequality does not give a direct estimate for the moments EI_{n,k}(f)^{2M} of the U-statistics we are interested in. But together with a symmetrization result formulated below it enables us to prove such a recursive estimate between the 2M-th and 4M-th moments of degenerate U-statistics which implies a version of Proposition 10.1 appropriate for our goals. This additional symmetrization result we need can be considered as a multivariate version of the Marcinkiewicz–Zygmund inequality about independent random variables with zero mean. First this symmetrization result will be given. Then for the sake of a better understanding the Marcinkiewicz–Zygmund inequality will be recalled, and its relation to the result considered as its multivariate version will be explained.

To formulate a good multivariate version of the Marcinkiewicz–Zygmund inequality first we introduce a notion which is called decoupled U-statistics in the literature.

The definition of decoupled and randomized decoupled U-statistics. Let us have k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence ξ_{1}, …, ξ_{n} of independent and identically distributed random variables taking their values on a measurable space (X, X) together with a measurable function f(x_{1}, …, x_{k}) on the product space (X^{k}, X^{k}) with values in a separable Banach space. Then the decoupled U-statistic determined by the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f is defined by the formula

Ī_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) .  (10.5)

Let us have, besides the sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and function f(x_{1}, …, x_{k}) a sequence of independent random variables ε = (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the sequences of random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. We define the randomized decoupled U-statistic determined by the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f and the randomizing sequence ε_{1}, …, ε_{n} by the formula

Ī_{n,k}(f, ε) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) .  (10.6)

Now a symmetrization result will be formulated which will be applied in the proof of an appropriate version of Proposition 10.1. This result will be proved in the next section.

Proposition 10.3. Let ξ_{1}, …, ξ_{n} be a sequence of i.i.d. random variables which take their values on a measurable space (X, X) with some distribution μ, and let f(x_{1}, …, x_{k}) be a canonical function with respect to this measure μ such that E|f(ξ_{1}, …, ξ_{k})|^{p} < ∞ with some p ≥ 1. Let us have k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence ξ_{1}, …, ξ_{n} with the same distribution, and let ε = (ε_{1}, …, ε_{n}) be a sequence of independent random variables, P (ε_{l} = 1) = P (ε_{l} = −1), 1 ≤ l ≤ n which is also independent of the sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. The inequality

E|Ī_{n,k}(f)|^{p} ≤ 2^{kp}E|Ī_{n,k}(f, ε)|^{p}  (10.7)

holds for the decoupled U-statistic Ī_{n,k}(f) and its randomized version Ī_{n,k}(f, ε) defined in formulas (10.5) and (10.6) by means of the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, ε_{1}, …, ε_{n} and the kernel function f.

In Proposition 10.1 we want to bound the moments of a U-statistic, while in Proposition 10.3 we have an estimate about decoupled U-statistics Ī_{n,k}(f). This results deals with decoupled statistics, because as we shall see, its proof does not work for the original U-statistics. This causes some difficulties, but they can be overcome with the help of a result of de la Peña and Montgomery–Smith. It will be formulated more generally than it is needed in the solution of the present problem to make it applicable also in the investigations of the subsequent part of the work. For its more general formulation let us slightly generalize the notion of U-statistics, let us allow also the case when the kernel function f in formula (8.7) takes its value in a separable Banach space. The result will be formulated in Theorem 10.4, and it will be proved in the Appendix.

Theorem 10.4. (de la Peña and Montgomery–Smith) Let us consider a sequence of independent and identically distributed random variables ξ_{1}, …, ξ_{n} on a measurable space (X, X) together with k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. Let us also have a function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) which takes its values on a separable Banach space B. Define the U-statistic and decoupled U-statistic I_{n,k}(f) and Ī_{n,k}(f) with the help of the above random sequences ξ_{1}, …, ξ_{n}, ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and kernel function f. Then there exist some constants C̄ = C̄(k) > 0 and γ = γ(k) > 0 depending only on the order k of the U-statistic such that

P (∥I_{n,k}(f)∥ > u) ≤ C̄P (∥Ī_{n,k}(f)∥ > γu)  (10.8)

for all u > 0. Here ∥ · ∥ denotes the norm in the Banach space B where the function f takes its values.

More generally, if we have a countable sequence of functions f_{s}, s = 1, 2, …, taking their values in the same separable Banach-space, then

P ( sup_{1≤s<∞} ∥I_{n,k}(f_{s})∥ > u) ≤ C̄P ( sup_{1≤s<∞} ∥Ī_{n,k}(f_{s})∥ > γu) .  (10.8′)

We follow the following approach. We shall prove such a version of Proposition 10.1 and Theorem 8.3 where U-statistics are replaced by decoupled U-statistics. The proof of these results is simpler, because the arguments applied for U-statistics also work for decoupled U-statistics, and also Proposition 10.3 can be applied in this case. Theorem 8.3 can be obtained as a consequence of its version we shall prove and Theorem 10.4. More explicitly, we shall prove the following two results.

Proposition 10.1′. Let the conditions of Proposition 10.1 be satisfied with some sequence of iid. μ-distributed random variables ξ_{1}, …, ξ_{n} on a space (X, X), a function f on the product space (X^{k}, X^{k}) and a number 0 < σ ≤ 1. Take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequence ξ_{1}, …, ξ_{n}, and define with their help the decoupled U-statistic I_{n,k}(f) defined in (10.5). Then the inequality

E (|n^{−k/2}Ī_{n,k}(f)|^{2M}) ≤ C_{k}^{M}M^{kM}σ^{2M} if 1 ≤ M ≤ nσ^{2}  (10.1′)

holds with some constant C_{k} which depends only on the order k of the decoupled Ustatistic.

Theorem 8.3′. Let the conditions of Proposition 8.3 be satisfied with some sequence of iid. μ-distributed random variables ξ_{1}, …, ξ_{n} on a space (X, X), a function f on the product space (X^{k}, X^{k}) and a number 0 < σ ≤ 1. Take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequence ξ_{1}, …, ξ_{n}, and define with their help the decoupled U-statistic Ī_{n,k}(f) defined in (10.5). Then there exist some constants C = C(k) > 0 and α = α(k) > 0 such that the inequality

P (n^{−k/2}|Ī_{n,k}(f)| > u) ≤ C exp \{−α (\frac{u}{σ} )^{2/k}\}  (10.9)

holds for all 0 < u ≤ n^{k/2}σ^{k+1}.

It is clear that Theorem 8.3′ together with Theorem 10.4 imply Theorem 8.3. Let us continue our discussion with an explanation of the content of Proposition 10.3. As we have mentioned, it can be considered as a multivariate version of the Marcinkiewicz– Zygmund inequality which can be formulated in the following way:

Let ξ_{1}, …, ξ_{n} be independent random variables such that Eξ_{j} = 0, 1 ≤ j ≤ n. Then for all p ≥ 2 there exist some constants 0 < B_{p} < C_{p} < ∞ such that

B_{p}E (∑_{l=1}^{n} ξ_{l}^{2})^{p/2} ≤ E |∑_{j=l}^{n} ξ_{l}|^{p} ≤ C_{p}E (∑_{l=1}^{n} ξ_{l}^{2})^{p/2} .  (10.12)

(This inequality also has a generalization for sums of martingale differences.) The really interesting part of formula (10.12) is his right-hand side part. It is useful, because the expression at the right-hand side of (10.12) can be well estimated even without exploiting the independence of the summands. The right-hand side of (10.12) can be deduced from Borell’s inequality, more explicitly from its consequence (10.4) with k = 1 and the inequality

E |∑_{l=1}^{n} ξ_{l}|^{p} ≤ C̄_{p}E |∑_{l=1}^{n} ε_{j}ξ_{l}|^{p} .  (10.12′)

with some C̄_{p} > 0, where ε_{1}, …, ε_{n}, P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2} are independent random variables, independent also of the random sequence ξ_{1}, …, ξ_{n}. Indeed, formula

(10.4) implies that E |∑_{l=1}^{n} ε_{l}ξ_{l}|^{p} ≤ (p − 1)^{p/2}E (∑_{l=1}^{n} ξ_{l}^{2})^{p/2}. Let us also observe that

Proposition 10.3 is a multivariate generalization of formula (10.12′) with the additional (important) information that it gives a good explicit choice for the coefficient C̄_{p} in it.

We can prove with the help of Borell’s inequality such an inequality which has similar relation to Proposition 10.3 as the right-hand side inequality in formula (10.12) to formula (10.12′). We shall give this result in the following corollary, and actually we shall apply this consequence of Proposition 10.3.

Corollary of Proposition 10.3. Let the conditions of Proposition 10.3 hold with the additional restriction that the inequality E|f(ξ_{1}, …, ξ_{k})|^{p} < ∞ holds with some p ≥ 2 (i.e. p > 1 is not sufficient for us). Then also the inequality

E|Ī_{n,k}(f)|^{p} ≤ 2^{kp}p^{kp/2}EĪ_{n,k}(f^{2})^{p/2}  (10.13)

holds.

Proof of the Corollary of Proposition 10.3. Let F denote the σ-algebra generated by the random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. Then Proposition 10.3 implies that

E|Ī_{n,k}(f)|^{p} ≤ 2^{kp}E|Ī_{n,k}(f, ε)|^{p} = 2^{kp}E(E(|Ī_{n,k}(f, ε)|^{p}|F)).

On the other hand, the consequence of Borell’s inequality formulated in relation (10.4) yields that

E(|Ī_{n,k}(f, ε)|^{p}|F) = E_{ε} |\frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)})|^{p}

≤ p^{kp/2} (\frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f^{2} (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}))^{p/2} = p^{kp/2}Ī_{n,k}(f^{2})^{p/2},

where E_{ε} means that we fix the values of the random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k and take expectation with respect to the random variables ε_{j}, 1 ≤ j ≤ n. We get, by taking expectation in the last inequality, that E|Ī_{n,k}(f, ε)|^{p} ≤ p^{kp/2}EĪ_{n,k}(f^{2})^{p/2}. This inequality together with formula (10.7) imply relation (10.13).

Now we turn to the proof of Proposition 10.1′.

The proof of Proposition 10.1′. We have En^{−k}Ī_{n,k}(f)^{2} ≤ \frac{1}{k!^{2}} σ^{2} if f is a canonical function with respect to the probability measure μ, and ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}, i.e. relation (10.1′) in Proposition 10.1′ holds for M = 1 if C_{k} ≥ \frac{1}{k!^{2}} , because

Ef(ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)})f(ξ_{l′_{1}}^{(1)}, …, ξ_{l′_{k}}^{(k)}) = 0, if l_{j} ≠ l′_{j} for some index 1 ≤ j ≤ k,

and Ef^{2}(ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) ≤ σ^{2}.

First we prove relation (10.1′) in the special case M = 2^{m} with m = 0, 1, … if 1 ≤ M ≤ 2nσ^{2} and the constants C_{k} are chosen appropriately in (10.1′). We have already proved this relation for m = 0. We shall prove the inequality E(n^{−k/2}I_{n,k}(f)^{2M}) ≤ C_{k}^{M}M^{kM}σ^{2M} for all k = 1, 2, … with some appropriate constant C_{k} > 0 if M = 2^{m} and M ≤ 2nσ^{2} by induction with respect to m. In the proof formula (10.13) of the Corollary of Proposition 10.3 will be applied with the choice p = 2M. This yields the estimate

E ((n^{−k/2}Ī_{n,k}(f))^{2M}) ≤ 2^{2kM}(2M)^{Mk}E (n^{−k}Ī_{n,k}(f^{2}))^{M} .  (10.14)

The above inequality is not sufficient in its original form to carry out the inductive procedure we have in mind, since the function f^{2} appearing at its right-hand side is not canonical. But this difficulty can be overcome if we apply the Hoeffding decomposition (9.2) for the function f^{2}.

This result yields a representation of the form

f^{2}(x_{1}, …, x_{k}) = ∑_{V ⊂{1,...,k}} f_{V} (x_{s}, s ∈ V)

with some appropriate canonical functions f_{V} (x_{s}, s ∈ V) with respect to the measure μ for all V ⊂ {1, …, k}. This relation implies that similarly to U-statistics decoupled U-statistics satisfy the relation

Ī_{n,k}(f^{2}) = ∑_{V ⊂\{1,...,k\}}(n − |V |)(n − |V | − 1) ⋯ (n − k + 1)\frac{|V |!}{k!}Ī_{n,|V |}(f_{V}). (10.15)

In Theorem 9.1 the functions f_{V} appearing in formula (10.15) are described explicitly. (Here again we define the value of the product (n − |V |)(n − |V | − 1) ⋯ (n − k + 1) as 1 for |V | = k.) We do not need this formula, we only need that by formulas (9.4) and (9.4′) of Theorem 9.2 the integrals of the square of the functions f_{V} are bounded by σ^{2}, and these functions are bounded by 2^{|V |} in supremum norm, because the function f^{2}, similarly to the function f, is bounded by σ in L_{2}(μ)-norm, and it is bounded by 1 in the supremum norm. The coefficient f_{V} with V = ∅ in the constant term of the sum at the right-hand side of (10.15) has to be considered separately. It equals f_{∅} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}). This implies that 0 ≤ f_{∅} ≤ σ^{2}, an estimate which does not follow directly from Theorem 9.2.

Formula (10.15) and the triangular inequality in L_{M} norm imply the inequality

\begin{matrix}E(n^{−k}Ī_{n,k}(f^{2}))^{M} ≤ (n^{−k} ∑_{V ⊂\{1,...,k\}} (E (n^{k−|V |} \frac{(|V |)!}{k!}I_{n,|V |}(f_{V}))^{M})^{1/M} )^{M} \\ ≤ (∑_{j=0}^{k} \binom{k}{j}\frac{j!}{k!} sup_{V : V ⊂\{1,...,k\}, |V |=j} (n^{−jM/2}E(n^{−j/2}Ī_{n,j}(f_{V}))^{M})^{1/M} )^{M}. \end{matrix}

(10.16)

The function 2^{−j}|f_{V} | is bounded by 1 in the supremum norm and by 2^{−j/2}σ ≤ σ in the L_{2}(μ) norm if |V | = j, 1 ≤ j ≤ k. Our inductive hypothesis implies that the terms at the right-hand side of (10.16) can be estimated as

n^{−jM/2}E(n^{−j/2}Ī_{n,j}(f_{V}))^{M} ≤ 2^{jM}C_{j}^{M/2}σ^{M} (\frac{M}{2} )^{jM/2} n^{−jM/2}

= (2^{j}C_{j})^{M/2}σ^{2M} (\frac{M}{nσ^{2/j}} )^{jM/2} ≤ 2^{jM}C_{j}^{M/2}σ^{2M},

if |V | = j, 1 ≤ j ≤ k, and M ≤ 2nσ^{2},

since \frac{M}{nσ^{2/j}} ≤ \frac{M}{nσ^{2}} ≤ 2 in this case. (Observe that σ^{2} ≤ 1, since sup |f(x_{1}, …, x_{k})| ≤ 1.)

Besides, \binom{k}{j} \frac{j!}{k!} sup_{V : V ⊂\{1,...,k\}, |V |=j} (E(n^{−j}Ī_{n,j}(f_{V}))^{M})^{1/M} = \frac{f_{∅}}{k!} ≤ \frac{σ^{2}}{k!} in the case j = 0.

These estimates yield that

E(n^{−k}I_{n,k}(f^{2}))^{M} ≤ σ^{2M} (∑_{j=0}^{k} \frac{2^{j}}{(k − j)!}C_{j}^{1/2})^{M}

if M ≤ 2nσ^{2}, and we choose C_{0} ≥ 1. By formula (10.14) and this estimate

\begin{aligned}E(n^{−k/2}Ī_{n,k}(f))^{2M} & ≤ 2^{3kM}M^{kM}E (n^{−k}Ī_{n,k}(f^{2}))^{M} \\ & ≤ (∑_{j=0}^{k} \frac{2^{3k+j}}{(k − j)!}C_{j}^{1/2})^{M} M^{kM}σ^{2M} \end{aligned}  (10.17)

if M ≤ 2nσ^{2}. I show that with an appropriate choice of the coefficients C_{k} (which may depend only on k but not on M) the above estimate implies the inductive step. Indeed, we can choose such a sequence C_{k}, = 0, 1, 2, …, with C_{0} = 1 which satisfies the inequalities C_{k} ≥ \frac{1}{k!} and

∑_{j=0}^{k} \frac{2^{3k+j}}{(k − j)!}C_{j}^{1/2} ≤ C_{k} for all k = 1, 2, …  (10.18)

Let us choose such a sequence C_{k}, k = 0, 1, …, which satisfies these relations. Then formula (10.1′) holds for M = 1, and our inductive procedure together with relations (10.17) and (10.18) imply that it also holds for M ≤ 2nσ^{2}, i.e.

E(n^{−k/2}I_{n,k}(f))^{2M} ≤ C_{k}^{M}σ^{2M}M^{kM} if M = 2^{m} and M ≤ 2nσ^{2}.

Thus we have proved Proposition 10.1′ in the special case when M = 2^{m}, m = 0, 1, …, and M ≤ 2nσ^{2}. To estimate the moment E|n^{−k/2}Ī_{n,k}(f)|^{2M} for a general exponent 1 ≤ M ≤ nσ^{2} (the number M may be non-integer) let us consider the number M̄ = M̄(M) of the form M̄ = 2^{m} with some integer m which satisfies the relation M̄ ≤ M < 2M̄. By applying the already proved part of Proposition 10.1′ for M̄ we can write

E|n^{−k/2}Ī_{n,k}(f)|^{2M} ≤ (E|n^{−k/2}Ī_{n,k}(f)|^{2M̄})^{M/M̄} ≤ (CM̄ ; kσ^{2M̄} M̄^{kM̄})^{M/M̄}
≤ C_{k}^{M}σ^{2M}(2M)^{kM} = (2^{k}C_{k})^{M}σ^{2M}M^{kM}.

Proposition 10.1′ is proved.

The proof of Theorem 8.3′. By the Markov inequality and Proposition 10.1′

P (|n^{−k/2}Ī_{n,k}(f)| > u) ≤ \frac{E|n^{−k/2}Ī_{n,k}(f)|^{2M}}{u^{2M}} ≤ (\frac{C_{k}σ^{2}M^{k}}{u^{2}} )^{M}

if u > 0 and 1 ≤ M ≤ nσ^{2}. Let us choose M = \frac{1}{e} (\frac{u^{2}}{C_{k}σ^{2}} )^{1/k}. With this choice of the parameter M we get that

P (|n^{−k/2}I_{n,k}(f)| > u) ≤ e^{−kM} = exp \{−\frac{k}{e} C_{k}^{−1/k} (\frac{u}{σ} )^{2/k}\}  (10.19)

if \sqrt{C_{k}}e^{k/2}σ ≤ u ≤ e^{k/2}\sqrt{C_{k}}n^{k/2}σ^{k+1}. Relation (10.19) implies formula (10.9). Indeed, formula (10.9) remains valid for \sqrt{C_{k}}e^{k/2}n^{k/2}σ^{k+1} ≤ u ≤ n^{k/2}σ^{k+1} if the constant kC_{k}^{−1/k}e^{−1} in the exponent at the right-hand side is replaced by α = min(kC_{k}^{−1/k}e^{−1}, k), (here we exploit that P (|n^{−k/2}I_{n,k}(f)| > u) ≤ P (|n^{−k/2}I_{n,k}(f)| > \sqrt{C_{k}}e^{k/2}n^{k/2}σ^{k+1}) if u ≥ \sqrt{C_{k}}e^{k/2}n^{k/2}σ^{k+1}), and it holds also for 0 ≤ u ≤ \sqrt{C_{k}}e^{k/2}σ if the right-hand side is multiplied with a sufficiently large constant C.

As we have mentioned, Theorems 8.3′ and Theorem 10.4 together imply Theorem 8.3.

11. Some useful basic results

This section contains the proof of Borell’s inequality and Proposition 10.3 which can be considered as the multivariate version of the Marcinkiewicz–Zygmund inequality, more precisely of its more important part.

11 a.) The proof of Borell’s inequality formulated in Theorem 10.2.

Borell’s inequality will be proved as the consequence of the following hypercontractive inequality for Rademacher functions.

Theorem 11.1. The hypercontractive inequality for Rademacher functions. Let us consider two copies (X, X , μ) and (Y, Y, ν) = (X, X , μ) of the measure space (X, X , μ), where X = {−1, 1}, X contains all subsets of X, and μ({1}) = μ({−1}) = \frac{1}{2}. Given a real number γ > 0 let us introduce the linear operator T_{γ} which maps the real (or complex) valued functions on the space X to the real (or complex) valued functions on the space Y which is defined by the relations T_{γ}r_{0} = r_{0}, and T_{γ}r_{1} = γr_{1}, where r_{0}(1) = r_{0}(−1) = 1, and r_{1}(1) = 1, r_{1}(−1) = −1. For all n = 1, 2, … let us consider the n-fold product (X^{n}, X^{n}, μ^{n}) and (Y^{n}, Y^{n}, ν^{n}) of the spaces (X, X , μ) and (Y, Y, ν) together with the n-fold product of the operator T_{γ}^{n} of the operator T_{γ} acting between these product spaces, (i.e. T_{γ}^{n} is the linear transformation for which T_{γ}^{n}(f_{1}(x_{1}) ⋯ f_{n}(x_{n})) = T_{γ}f_{1}(x_{1}) ⋯ T_{γ}f_{n}(x_{n}) for all products of the functions f_{s}, 1 ≤ s ≤ n, on the space (X, X , μ)). The transformation T_{γ}^{n} from the space L_{q}(X^{n}, X^{n}, μ^{n}) to the space L_{p}(Y^{n}, Y^{n}, ν^{n}) has norm 1 for all n = 1, 2, … if 1 < p ≤ q < ∞, and

0 ≤ γ ≤ \sqrt{\frac{q−1}{p−1}}.

The name hypercontractive inequality was given to this result because it states not only that ∥T_{γ}^{n}f∥_{q} ≤ ∥f∥_{q} for all functions f but also the inequality ∥T_{γ}^{n}f∥_{p} ≤ ∥f∥_{q} with some 1 ≤ q < p, while ∥T_{γ}^{n}f∥_{q} ≤ ∥T_{γ}^{n}f∥_{p} if 1 ≤ q < p. It is not difficult to see that the hypercontractive inequality implies Borell’s inequality.

The proof of Borell’s inequality by means of the hypercontractive inequality. Let us define the function

f(x_{1}, …, x_{n}) = ∑_{\begin{matrix}1≤l_{j}≤n, 1≤j≤k \\ j_{s}≠l_{j′} if j≠j′ \end{matrix}} a(l_{1}, …, l_{k})r_{1}(x_{l_{1}}) ⋯ r_{1}(x_{l_{k}})

on the space (X^{n}, X^{n}, μ^{n}). Observe that T_{γ}^{n}f = γ^{k}f for this function f and all γ > 0, and E|Z|^{p} = ∥f∥_{p}^{p}, E|Z|^{q} = ∥f∥_{q}^{q}. Fix some numbers 1 < q ≤ p ≤ ∞ and put

γ = \sqrt{\frac{q−1}{p−1}}. The norm of T_{γ}^{n} as a transformation from the space L_{q}(X^{n}, X^{n}, μ^{n}) to

the space L_{p}(Y^{n}, Y^{n}, ν^{n}) is bounded by 1, i.e. ∥T_{γ}^{n}f∥_{p} = γ^{k}∥f∥_{p} ≤ ∥f∥_{q}. The above

relations imply that (E|Z|^{p})^{1/p} ≤ (\frac{q−1}{p−1})^{k/2} E|Z|^{q})^{1/q} in this case, and this is what we had to show.

The proof of the hypercontractive inequality can be reduced to a simpler statement by means of the following

Theorem 11.2. Let us consider two pairs of measure spaces (X_{1}, A_{1}, μ_{1}), (Y_{1}, B_{1}, ν_{1}) and (X_{2}, A_{2}, μ_{2}), (Y_{2}, B_{2}, ν_{2}) together with two linear operators T_{1} and T_{2} which map the space L_{q}(X_{1}, A_{1}, μ_{1}) to L_{p}(Y_{1}, B_{1}, ν_{1}) and the space L_{q}(X_{2}, A_{2}, μ_{2}) to L_{p}(Y_{2}, B_{2}, ν_{2}) respectively. Assume that 1 ≤ q ≤ p, and the norm of both operators T_{1} and T_{2} is less than or equal to 1. Then also the norm of their direct product T_{1} × T_{2} which maps the space L_{q}(X_{1} × X_{2}, A_{1} × A_{2}, μ_{1} × μ_{2}) to the space L_{p}(Y_{1} × Y_{2}, B_{1} × B_{2}, ν_{1} × ν_{2}) is less than or equal to one.

Proof of Theorem 11.2: We have to show that

\begin{matrix}∫_{Y_{1}×Y_{2}} |∑_{j=1}^{n} c_{j}T_{1}f_{j}(y_{1})T_{2}g_{j}(y_{2})|^{p} ν_{1}( dy_{1})ν_{2}( dy_{2}) \\ ≤ [∫_{X_{1}×X_{2}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})g_{j}(x_{2})|^{q} μ_{1}( dx_{1})μ_{2}( dx_{2})]^{p/q} \end{matrix}  (11.1)

for arbitrary index n, real (or complex) numbers c_{j} and functions f_{j}(·) ∈ L_{q}(X_{1}, A_{1}, μ_{1}) and g_{j}(·) ∈ L_{q}(X_{2}, A_{2}, μ_{2}), 1 ≤ j ≤ n, since relation (11.1) is equivalent to the inequality

∥(T_{1} × T_{2})f(y_{1}, y_{2}∥_{L_{p}} ≤ ∥f(x_{1}, x_{2})∥_{L_{q}} for the function f(x_{1}, x_{2}) = ∑_{j=1}^{n} c_{j}f_{j}(x_{1})g_{j}(x_{2}),

and as functions of the above form are dense in the space L_{q}(X_{1} ×X_{2}, A_{1} ×A_{2}, μ_{1} ×μ_{2}), this inequality implies that the norm of T_{1} × T_{2} is bounded by 1.

We get by integrating the left-hand side of (11.1) first by the variable y_{1} and by exploiting the condition |T_{1}| ≤ 1 that

\begin{matrix}∫_{Y_{1}×Y_{2}} |∑_{j=1}^{n} c_{j}T_{1}f_{j}(y_{1})T_{2}g_{j}(y_{2})|^{p} ν_{1}( dy_{1})ν_{2}( dy_{2}) \\ ≤ ∫_{Y_{2}} [∫_{X_{1}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})T_{2}g_{j}(y_{2})|^{q} μ_{1}( dx_{1})]^{p/q} ν_{2}( dy_{2}). \end{matrix}  (11.2)

We shall prove and apply the following result. Let a function G(u, v) be given on a product space (U × V, U × V, ρ_{1} × ρ_{2}), and let 1 ≤ s < ∞. Then

[∫_{V} [∫_{U} |G(u, v)|ρ_{1}( du)]^{s} ρ_{2}( dv)]^{1/s} ≤ ∫_{U} [∫_{V} |G(u, v)|^{s}ρ_{2}( dv)]^{1/s} ρ_{1}( du). (11.3)

It is enough to prove this estimate for the following special type of functions G(u, v). Let us consider a finite partition A_{1}, …, A_{m} of the space U, choose for all 1 ≤ j ≤ m a function G_{j}(v) on the space (V, V) and put G(u, v) = G_{j}(v) if u ∈ A_{j}, 1 ≤ j ≤ m. Such kind of functions are dense in the L_{q}(U × V, U × V, ρ_{1} × ρ_{2}) space, because such kind of functions are dense in the subspace consisting of functions of the form G(u, v) =

∑_{j=1}^{n} c_{j}f_{j}(u)g_{j}(v). If we prove inequality (11.3) for such special type of functions, then this

inequality can be generalized for general functions G(u, v) by an appropriate limiting procedure. Its details are left to the reader.

Inequality (11.3) in the special case we consider is equivalent to the triangular inequality in L_{s} spaces, s ≥ 1, (also called Minkowski inequality)

∥∑_{j=1}^{m} ρ_{1}(A_{j})|G_{j}(v)|∥_{s} ≤ ∑_{j=1}^{m} ∥ρ_{1}(A_{j})|G_{j}(v)|∥_{s},

where ∥f∥_{s} denotes the L_{s}-norm of a function f in the space (V, V, ρ_{2}).

Indeed,

∥∑_{j=1}^{m} ρ_{1}(A_{j})|G_{j}(v)|∥_{s} = ∥∫_{U} |G(u, v)|ρ_{1}( du)∥_{s} = [∫_{V} [∫_{U}|G(u, v)|ρ_{1}( du)]^{s}ρ_{2}( dv)]^{1/s},

and this is the left-hand side of formula (11.3), while

∑_{j=1}^{m} ∥ρ_{1}(A_{j})|G_{j}(v)|∥_{s} = ∑_{j=1}^{m} ρ_{1}(A_{j}) [∫_{V} |G_{j}(v)|^{s}ρ_{2}( dv)]^{1/s}

= ∫_{U} [∫_{V} |G(u, v)|^{s}ρ_{2}( dv)]^{1/s} ρ_{1}( du),

and this is the right-hand side of (11.3).

Using inequality (11.3) in our case on the space (X_{1} × Y_{2}, A_{1} × B_{2}, μ_{1} × ν_{2}) with

the choice s = \frac{p}{q}, U = X_{1}, V = Y_{2}, G(u, v) = | ∑_{j=1}^{n} c_{j}f_{j}(x_{1})T_{2}g_{j}(y_{2})|^{q}, ρ_{1} = μ_{1}, ρ_{2} = ν_{2}

we get with the help of formula (11.2) that

∫_{Y_{1}×Y_{2}} |∑_{j=1}^{n} c_{j}T_{1}f_{j}(y_{1})T_{2}g_{j}(y_{2})|^{p} ν_{1}( dy_{1})ν_{2}( dy_{2})

≤ (∫_{X_{1}} [∫_{Y_{2}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})T_{2}g_{j}(y_{2})|^{p} ν_{2}( dy_{2})]^{q/p} μ_{1}( dx_{1}))^{p/q} .

Then by exploiting that |T_{2}| ≤ 1 we get that

[∫_{Y_{2}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})T_{2}g_{j}(y_{2})|^{p} ν_{2}( dy_{2})]^{q/p} ≤ ∫_{X_{2}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})g_{j}(x_{2})|^{q} μ_{2}( dx_{2})

for all x_{1} ∈ X_{1}, and

∫_{Y_{1}×Y_{2}} |∑_{j=1}^{n} c_{j}T_{1}f_{j}(y_{1})T_{2}g_{j}(y_{2})|^{p} ν_{1}( dy_{1})ν_{2}( dy_{2})

≤ (∫_{X_{1}} [∫_{X_{2}} |∑_{j=1}^{n} c_{j}f_{j}(x_{1})g_{j}(x_{2})|^{q} μ_{2}( dx_{2})] μ_{1}( dx_{1}))^{p/q} .

By the Fubini theorem this inequality is equivalent to relation (11.1).

Theorem 11.2 enables us to reduce the proof of the hypercontractive inequality for Rademacher functions to the following simpler result.

Theorem 11.3. The reduced form of the hypercontractive inequality for Rademacher functions. Let ε be a random variable such that P (ε = 1) = P (ε = −1) = \frac{1}{2}. Then the following inequality holds for all real (or complex) numbers a, b,

and numbers 1 ≤ q ≤ p < ∞ together with some 0 ≤ γ ≤ \sqrt{\frac{q−1}{p−1}}:

E (|a + γbε|^{p})^{1/p} ≤ (E|a + bε|^{q})^{1/q}  (11.4)

Theorems 11.3 and 11.2 really imply Theorem 11.1, because Theorem 11.3 states the desired result in the special case n = 1, and then by Theorem 11.2 it holds for arbitrary n.

Even the proof of Theorem 11.3 is far from trivial. On the other hand, Leonhard Gross has made a deep and interesting investigation in his paper Logarithmic Sobolev inequalities (Amer. J. Math. 97, 1061-1083, 1975) which supplies this result as a special case of a general theory. His approach is based on the following idea. Let us consider a continuous time Markov process ξ(t), t ≥ 0, with its stationary distribution and a function f(x) on the state space of this Markov process. We can get good estimates on the moments E|f(ξ(t))|^{p} if we have an appropriate estimate on the infinitesimal operator of the Markov process he calls logarithmic Sobolev inequality. In an informal way this approach can be interpreted as a good estimate of a function by means of its derivative.

Gross applies a rather hard analysis in his proof, but if we restrict our attention to that example which leads to the proof of Theorem 11.3, then the most difficult parts of his study do not appear. Here we shall follow this approach.

Let us define the Markov process ξ_{t} describing the movement of a particle on the state space X = {−1, 1} consisting of two points, where the particle jumps from one state to the other one after exponential time with parameter λ = \frac{1}{2}. This means that the places of jumps constitute a Poisson process with parameter λ = \frac{1}{2}, and the transition probabilities of this Markov process are

p_{t}(1, 1) = p_{t}(−1, −1) = e^{−t/2} ∑_{k=0}^{∞} \frac{1}{(2k)!} (\frac{t}{2})^{2k} ,

p_{t}(1, −1) = p_{t}(−1, 1) = e^{−t/2} ∑_{k=0}^{∞} \frac{1}{(2k + 1)!} (\frac{t}{2})^{2k+1} .

(The particle remains in the same place after time t if it made an even number of jumps in the time interval [0, t], and changes his position if it made an odd number of jumps.) Let us calculate the semigroup U_{t}, t ≥ 0, of this Markov process, defined as U_{t}(f)(x) = E(f(ξ(t))|ξ(0) = x), for all x ∈ X, all functions f defined on X and parameters t ≥ 0 together with the infinitesimal operator of this Markov process Bf(x) = − \frac{dU_{t}(f)(x)}{dt} |_{t=0}. The above objects can be simply calculated in this model. Let us introduce the functions r_{0}(x) and r_{1}(x) on the state space X defined as r_{0}(1) = r_{0}(−1) = 1 and r_{1}(1) = 1, r_{1}(−1) = −1. Observe that p_{t}(1, 1) − p_{t}(1, −1) = e^{−t}, p_{t}(−1, −1) − p_{t}(−1, 1) = e^{−t}, hence U_{t}r_{1}(1) = e^{−t}r_{1}(1), U_{t}r_{1}(−1) = e^{−t}r_{1}(−1), i.e. U_{t}r_{1}(x) = e^{−t}r_{1}(x) for all t ≥ 0. On the other hand, clearly U_{t}r_{0}(x) = r_{0}(x) for all t ≥ 0. All functions f on the state space X can be written in the form f(x) = a+br_{1}(x) with some appropriate coefficients a and b, and U_{t}(a + br_{1})(x) = a + e^{−t}br_{1}(x). Clearly B(a + br_{1})(x) = br_{1}(x). Let μ, μ(1) = μ(−1) = \frac{1}{2}, denote the equilibrium state of the Markov process ξ(t). Put

∥f∥_{p} = (∫ |f(x)|^{p}μ( dx))^{1/p} = (\frac{1}{2}(|f(1)|^{p} + |f(−1)|^{p})^{1/p}. The following inequality will be proved, which is the logarithmic Sobolev inequality in the special model considered here. The notations introduced before will be preserved.

Proposition 11.4. Let us consider a function f(x) = a + br_{1}(x) on the space X = \{−1, 1\} with the probability measure μ, μ(1) = μ(−1) = \frac{1}{2}, on X such that both a and b are real numbers, and a ≥ |b|. Then

∫ f^{p}(x) ln f(x)μ( dx) ≤ \frac{p}{2(p − 1)} ∫ f^{p−1}(x)Bf(x)μ( dx) + ∥f∥_{p}^{p} ln ∥f∥_{p},
for all 1 < p < ∞.

(11.5)

(The letter B in formula (11.5) denotes the infinitesimal operator of the Markov process we consider.)

The corresponding result in Gross’ paper is slightly more general. It contains such an estimate which holds for all functions f, i.e. the condition that a and b are real numbers, and a ≥ |b| in the expansion f = a + br_{1} is not needed there. Our restriction makes the proof simpler, since this implies that the function f(x) is real-valued and f(x) ≥ 0 both for x = 1 and x = −1. Hence we do not have to work with absolute values. On the other hand, Proposition 11.4 is sufficient for us also in this restricted form. Before its proof we show that it implies Theorem 11.3.

The proof of Theorem 11.3 by means of Proposition 11.4. Let us introduce the function p(t, q) = 1 + (q − 1)e^{2t} for all q > 1, and t ≥ 0. First we prove that

[∫ |U_{t}f(x)|^{p(t,q)}μ( dx)]^{1/p(t,q)} ≤ [∫ |f(x)|^{q}μ( dx)]^{1/q} for all t ≥ 0  (11.6)

and functions f on X. (The general theory helps to find the ‘right’ definition of the function p(t, q). It is defined as the solution of the differential equation \frac{p}{2(p−1)} \frac{dp(t)}{dt} = p, p(0) = q. The coefficient \frac{p}{2(p−1)} in this equation agrees with the coefficient appearing in the logarithmic Sobolev inequality (11.5).) Let us prove inequality (11.6) first for such functions f(x) = a + br_{1}(x) for which a and b are real numbers and a ≥ |b|.

Given a function f(x) = a + br_{1}(x) with a ≥ |b| define the function F (t) = [∫ (U_{t}f(x))^{p(t,q)}μ( dx)]^{1/p(t,q)}. Observe that U_{t}f(x) = a + be^{−t}r_{1}(x), and a ≥ |b|e^{−t}. Hence to prove (11.6) it is enough to show that

\frac{d∥U_{t}(f)∥_{p(t,q)}}{dt} = \frac{dF (t)}{dt} ≤ 0 for all t > 0  (11.7)

which means that the function F (t) is monotone decreasing, and in the proof we can apply the logarithmic Sobolev inequality for the functions f_{t}(x) = U_{t}f(x). We have

\frac{dF (t)}{dt} = F (t)[−\frac{p′(t, q)}{p(t, q)} ln F (t) + \frac{p′(t, q)}{p(t, q)} \frac{∫ U_{t}f(x)^{p(t,q)} ln U_{t}f(x)μ( dx)}{∫ U_{t}f(x)^{p(t,q)}μ( dx)}

+ \frac{∫ U_{t}f(x)^{p(t,q)−1}(U_{t}f(x))′μ(d x)}{∫ U_{t}f(x)^{p(t,q)}μ( dx)}],

where G(t, ·)′ means partial derivative with respect to the variable t. Since F (t) = ∥U_{t}(f)∥_{p(t,q)}, ∫ U_{t}f(x)^{p(t,q)}μ( dx) = ∥U_{t}(f)∥_{p(t,q)}^{p(t,q)}, (U_{t}f(x))′ = −BU_{t}f(x) by the definition of the operator B,

∫ U_{t}f(x)^{p(t,q)−1}(U_{t}f(x))′μ(d x) = − ∫ U_{t}f(x)^{p(t,q)−1}B(U_{t}f)(x)μ(d x),

and \frac{p(t,q)}{p′(t,q)} = \frac{p(t,q)}{2(p(t,q)−1)} with our choice of functions, the last formula implies that the

inequality \frac{dF (t)}{dt} ≤ 0 is equivalent to the relation

− ∥U_{t}(f)∥_{p(t,q)}^{p(t,q)} ln ∥U_{t}(f)∥ + ∫ U_{t}f^{p(t,q)}(x) ln U_{t}f(x)μ( dx)

− \frac{p}{2(p − 1)} ∫ (U_{t}f)^{p(t,q)−1}(x)BU_{t}f(x)μ(( dx) ≤ 0.

But this inequality follows from the logarithmic Sobolev inequality if it is applied for the function U_{t}(f) with p̄ = p(t, q).

To prove relation (11.6) for a general function f it is enough to check that |U_{t}(f)| ≤ U_{t}(|f|), i.e. |U_{t}(f)(1)| ≤ U_{t}(|f|)(1) and |U_{t}(f)(−1)| ≤ U_{t}(|f|)(−1) for arbitrary function f and t ≥ 0, since this relation has been already proved for the function |f|. But this relation simply follows from the following calculation. If f(1) = A, f(−1) = B, then f(x) = \frac{A+B}{2} + \frac{A−B}{2} r_{1}(x), U_{t}f(x) = \frac{A+B}{2} + e^{−t} \frac{A−B}{2} r_{1}(x), i.e. U_{t}f(1) = \frac{1+e^{−t}}{2} A + \frac{1−e^{−t}}{2} B, and U_{t}f(−1) = \frac{1−e^{−t}}{2} A + \frac{1+e^{−t}}{2} B, while (U_{t}|f|)(±1) = \frac{1+e^{−t}}{2} |A| + \frac{1∓e^{−t}}{2} |B|.

Let us fix some numbers 1 < p ≤ q < ∞ and apply formula (11.6) for some function f(x) = a + br_{1}(x) with the number t which is the solution of the equation p(t, q) = p. Then e^{−t} = γ(p, q) = \sqrt{\frac{q−1}{p−1}}, U_{t}(a + br_{1}(x)) = a + γ(p, q)r_{1}(x), hence ∥a + γ(p, q)br_{1}(x)∥_{p} ≤ ∥a + br_{1}(x)∥_{q}. Given some γ ≤ γ_{p}, let us define p̄ as the solution

of the equation γ = \sqrt{\frac{q−1}{p̄−1}}. Then p̄ ≥ p, hence ∥a + γbr_{1}(x)∥_{p} ≤ ∥a + γbr_{1}(x)∥p̄ ≤ ∥a + br_{1}(x)∥_{q}, and this relation is equivalent to formula (11.4). Thus Theorem 11.3 is proved with the help of Proposition 11.4.

The proof of Proposition 11.4. Let us prove relation (11.5) first in the special case p = 2. We have to show that

∫ Bf · f dμ + \frac{1}{2} ∫ f^{2} dμ ln (∫ f^{2} dμ) − ∫ f^{2} ln f dμ ≥ 0

for a function of the form f = a + br_{1}, a ≥ |b|. Since the left-hand side of this inequality is homogeneous of order 2 it is enough to prove this inequality in the special case f = 1 + sr_{1}, |s| ≤ 1. In this case the inequality we want to prove can be written as

h(s) = s^{2} + \frac{1}{2}(1 + s^{2}) ln(1 + s^{2}) − \frac{1}{2} [(1 + s)^{2} ln(1 + s) + (1 − s)^{2} ln(1 − s)] ≥ 0.

Simple calculation shows that h′(s) = 2s+s ln(1+s^{2})−(1+s) ln(1+s)+(1−s) ln(1− s), and h′′(s) = \frac{2s^{2}}{1+s^{2}} +ln(1+s^{2})−ln(1−s^{2}) = \frac{2s^{2}}{1+s^{2}} −ln \frac{1−s^{2}}{1+s^{2}} = \frac{2s^{2}}{1+s^{2}} −ln (1 − \frac{2s^{2}}{1+s^{2}} ) ≥ 0 for all 0 ≤ s ≤ 1. This means that the function h(s) convex. On the other hand h(0) = h′(0) = 0. These relations imply that h(s) ≥ 0 for all 0 ≤ s ≤ 1 as we have claimed.

In the general case p > 1 let us apply inequality (11.5) in the already proven case p = 2 for the function f^{p/2}. We get that \frac{p}{2} ∫ f^{p}(x) ln f(x)μ( dx) ≤ ∫ f^{p/2}(x)Bf^{p/2}(x)μ(dx)+ \frac{p}{2}∥f∥_{p}^{p} ln ∥f∥_{p}. Hence to prove Proposition 11.4 in the general case it is enough to show that

∫ f^{p/2}(x)Bf^{p/2}(x)μ( dx) ≤ \frac{p^{2}}{4(p − 1)} ∫ f^{p−1}(x)Bf(x)μ( dx)

for a function f(x) = a + br_{1}(x) such that a ≥ |b|.

The expressions in the last inequality can be simply calculated. As

\frac{1}{2^{p/2−1}} f^{p/2}(x) = [(\frac{a + b}{2})^{p/2} + (\frac{a − b}{2})^{p/2}] + [(\frac{a + b}{2})^{p/2} − (\frac{a − b}{2})^{p/2}] r_{1}(x),

\frac{1}{2^{p/2−1}} Bf^{p/2}(x) = [(\frac{a + b}{2})^{p/2} − (\frac{a − b}{2})^{p/2}] r_{1}(x),

and

\frac{1}{2^{p−2}} f^{p−1}(x) = [(\frac{a + b}{2})^{p−1} + (\frac{a − b}{2})^{p−1}] + [(\frac{a + b}{2})^{p−1} − (\frac{a − b}{2})^{p−1}] r_{1}(x)

this inequality, more precisely its version we get by multiplying it by 2^{−(p−2)} can be rewritten as

[(\frac{a + b}{2})^{p/2} − (\frac{a − b}{2})^{p/2}]^{2}

≤ \frac{p^{2}}{4(p − 1)} [(\frac{a + b}{2})^{p−1} − (\frac{a − b}{2})^{p−1}] (\frac{a + b}{2} − \frac{a − b}{2})

or

(∫_{u}^{v} t^{(p−2)/2} dt)^{2} ≤ ∫_{u}^{v} t^{p−2} dt · ∫_{u}^{v} 1 dt

with u = \frac{a−|b|}{2} and v = \frac{a+|b|}{2}. But the last formula is a simple consequence of the Schwarz inequality. Proposition 11.4 is proved.

Remark: Theorem 11.3 is sharp in the following sense. The transformation T_{γ}, T_{γ}(a + br_{1}(x)) = a + γbr_{1}(x) as a transformation from the L_{q}(X, X , μ) space to the space

L_{p}(X, X , μ) with 1 < q < p has a norm greater then 1 if γ > \sqrt{\frac{q−1}{p−1}}. To see this let us compare the L_{q} norm of 1 + δr_{1}(x) with the L_{p}-norm of T_{γ}r_{1}(x) = 1 + γδr_{1}(x) for a small parameter δ > 0. We have ∥1 + δr_{1}(x)∥_{q} = [\frac{1}{2} ((1 + δ)^{q} + (1 − δ)^{q})]^{1/q} =

[1 + \frac{q(q−1)}{2}δ^{2} + O(δ^{3})]^{1/q} = 1+ \frac{q−1}{2}δ^{2}+O(δ^{3}). Similarly, ∥1+γδr_{1}(x)∥_{p} = 1+ \frac{p−1}{2}γ^{2}δ^{2}+

O(δ^{3}), and these relations imply the above remark.

11 b.) The proof of Proposition 10.3.

Proof of Proposition 10.3. Let us use the notation introduced in the formulation of Proposition 10.3, and take another k independent copies ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequences ξ_{1}, …, ξ_{n} which are also independent of the sequence ε_{1}, …, ε_{n} appearing in the formulation of Proposition 10.3. Let F denote the σ-algebra generated by the random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and let us introduce the notation ξ_{l}^{(j,1)} = ξ_{l}^{(j)}, ξ_{l}^{(j,−1)} = ξ̄_{l}^{(j)}, 1 ≤ l ≤ n and 1 ≤ j ≤ k. Let V_{k} denote the set of ±1 sequences of length k, and for a v ∈ V_{k} let m(v) denote the number of the digits −1 in the sequence v = (v(1), …, v(k)). Observe that E (f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})| F) = 0 if the ±1 sequence (v(1), …, v(k)) contains at least one coordinate −1, (this is the point of the proof where we exploit the canonical property of the function f), and

Ef (ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{k}}^{(k,1)}| F) = f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) for all indices 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k.

These relations together with the Jensen-inequality for conditional expectations imply that

|Ī_{n,k}(f)|^{p} = |E (\frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)}  ∑_{\begin{matrix}1≤l_{r}≤n, r=1,...,k \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})| F)|^{p}

≤ E (| \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}1≤l_{r}≤n, r=1,...,k \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})|^{p}| F) .

Hence

E|Ī_{n,k}(f)|^{p} ≤ E |\frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}1≤l_{r}≤n, r=1,...,k \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})|^{p} . (11.8)

Let us introduce the random variables

Ĩ_{n,k}(f) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}1≤l_{r}≤n, r=1,...,k \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})  (11.9)

and

Ĩ_{n,k}(f, ε) = \frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}1≤l_{r}≤n, r=1,...,k \\ l_{r}≠l_{r′} if r≠r′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))}) . (11.9′)

Let us recall that the number m(v) in these formula denotes the number of the digits −1 in the ±1 sequence v of length k, i.e. it counts how many random variables ξ_{l_{j}}^{(j,1)},

1 ≤ j ≤ k, were replaced by the ‘secondary copy’ ξ_{l_{j}}^{(j,−1)} in the corresponding terms of the sum in (11.9) or (11.9′).

I claim that the above defined two random variables Ĩ_{n,k}(f) and Ĩ_{n,k}(f, ε) have the same distribution. This statement will be formulated in a slightly more general form which will be useful in the further part of this work.

Lemma 11.5. Let us consider a (non-empty) class of functions F of k variables f(x_{1}, …, x_{k}) on the space (X^{k}, X^{k}) together with the random variables Ĩ_{n,k}(f) and Ĩ_{n,k}(f, ε) defined in formulas (11.9) and (11.9′) for all f ∈ F. The joint distributions of the set of random variables {Ĩ_{n,k}(f); f ∈ F} and {Ĩ_{n,k}(f, ε); f ∈ F} agree.

The proof of Lemma 11.5. We even claim that fixing an arbitrary sequence u = (u(1), …, u(n)), u(l) = ±1, 1 ≤ l ≤ n, of length n, the conditional distribution of the field {Ĩ_{n,k}(f, ε); f ∈ F} under the condition that (ε_{1}, …, ε_{n}) = u = (u(1), …, u(n)) agrees with the distribution of the field of {Ĩ_{n,k}(f); f ∈ F}.

Indeed, the random variables Ĩ_{n,k}(f), f ∈ F, defined in (11.9) are functions of a random vector consisting of coordinates (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}), 1 ≤ l ≤ n, 1 ≤ j ≤ k, and the distribution of this random vector does not change if we replace the coordinates (ξ_{l}^{(j)}, ξ̄_{l}^{(j)}) = (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}), by (ξ̄_{l}^{(j)}, ξ_{l}^{(j)}) = (ξ_{l}^{(j,−1)}, ξ_{l}^{(j,1)}), for those indices (j, l) for which u(l) = −1 (independently of the value of the parameter j) and do not modify these random vectors for those coordinates (l, j) for which u(l) = 1. Replacing the original vector in the definition of the expression Ĩ_{n,k}(f) in (11.9) for all f ∈ F by this modified vector we carry out a measure preserving transformation. On the other hand, the random field we get in such a way has the same distribution as the conditional distribution of the random field Ĩ_{n,k}(f, ε), f ∈ F, with the elements defined in (11.9′) under the condition that (ε_{1}, …, ε_{n}) = u with u = (u(1), …, u(n)).

To prove the last statement let us observe that the conditional distribution of the random field Ĩ_{n,k}(f, ε), f ∈ F, under the condition (ε_{1}, …, ε_{n}) = u is the same as that of the random field we obtain by putting u_{l} = ε_{l}, 1 ≤ l ≤ n, in all coordinates ε_{l} of the random variables Ĩ_{n,k}(f, ε). On the other hand, the random variables we get in such a way agree with the random variables we get by carrying out the above described transformation for the random variables Ĩ_{n,k}(f), only the terms in the sums defining these random variables are listed in a different order.

Relation (11.8) and the agreement of the distribution of the random variables Ĩ_{n,k}(f) in (11.9) and Ĩ_{n,k}(f) (11.9′) imply that

E|Ī_{n,k}(f)|^{p} ≤ E |\frac{1}{k!} ∑_{v∈V_{k}}(−1)^{m(v)} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}}ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))})|^{p} .

(11.10)

Let us define for all v = (v(1), …, v(k)) ∈ V_{k} the random variable

Ī_{n,k,v}(f, ε) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,v(1))}, …, ξ_{l_{k}}^{(k,v(k))}) , v ∈ V_{k}.

The distribution of the random variables Ī_{n,k,v}(f, ε) agree with that of Ī_{n,k}(f, ε) introduced in (10.6) for all v ∈ V_{k}. Hence relation (11.10) implies that

E|Ī_{n,k}(f)|^{p} ≤ E | ∑_{v∈V_{k}}(−1)^{m(v)}Ī_{n,k,v}(ε, f)|^{p}

≤ 2^{(k−1)p} ∑_{v∈V_{k}} E|Ī_{n,k,v}(f, ε)|^{p} = 2^{kp}E|Ī_{n,k}(f, ε)|^{p}.

Proposition 10.3 is proved.

12. Reduction of the main result in this work

The main result of this paper is Theorem 8.4 or its multiple integral version Theorem 8.2. It can be considered as the multivariate version of Theorem 4.1, and its proof is also based on a similar argument. Following the method of the proof of Theorem 4.1 first we prove a multivariate version of Proposition 6.1 in Proposition 12.1 and reduce Theorem 8.4 to a simpler statement formulated in Proposition 12.2.

The hard part of the problem is the proof of Proposition 12.2. In the first step of its proof we reduce it with the help of Theorem 10.4 (proved by de la Peña and Montgomery–Smith) to an analogous result formulated in Proposition 12.2′, where the U-statistics to be investigated are replaced by their decoupled U-statistics counterpart introduced in Section 10. The proof of this result is simpler, because here we have more independence. It is based on a symmetrization argument, similar to the proof of Proposition 6.2. The details of this symmetrization argument will be explained in the next section. This section contains only an important preliminary result needed in this argument, a multi-dimensional variant of Hoeffding’s inequality (Theorem 3.4) formulated in Theorem 12.3. It yields an estimate about the distribution of homogeneous polynomials of Rademacher functions.

The first result of this Section, Proposition 12.1, can be proved in almost the same way as its simplified version Proposition 6.1. The only essential difference between their proof is that Bernstein’s inequality applied in the proof of Proposition 6.1 is replaced now by its multivariate version Theorem 8.3. Theorem 12.1 can be considered as the result we can get by means of the Theorem 8.3 and the chaining argument. Its main content, formulated in relation (12.1) states that given a nice class of functions F it has a subclass Fσ̄ of relatively small cardinality which is also a relatively dense subclass of F in the L_{2} norm, and the supremum of the U-statistics with kernel functions from Fσ̄ can be well bounded. To get an applicable result we also need some estimates on the number σ̄ which measures how dense the subclass Fσ̄ in F is. Such estimates are contained at the end of this Proposition.

In the formulation of Proposition 12.1 we introduce, similarly to Proposition 6.1, two parameters Ā > 2^{k} and M = M(Ā, k), and this may seem at first sight unnatural. But the introduction of these parameters turned out to be useful, they help, similarly to the analogous problem in Section 6 to fit the parameters in Propositions 12.1 and 12.2 as we want to apply them simultaneously.

Proposition 12.1. Let us have the k-fold power (X^{k}, X^{k}) of a measurable space (X, X) with some probability measure μ on (X, X) and a countable L_{2}-dense class F of functions f(x_{1}, …, x_{k}) of k variables on (X^{k}, X^{k}) with parameter D and exponent L, L ≥ 1, such that all functions f ∈ F are canonical with respect to the measure μ, and they satisfy conditions (8.4) and (8.5) with some real number 0 < σ ≤ 1. Take a sequence of independent μ-distributed random variables ξ_{1}, …, ξ_{n}, n ≥ max(k, 2), and consider the (degenerate) U-statistics I_{n,k}(f), f ∈ F, defined in formula (8.7). Let us fix some number Ā ≥ 2^{k}.

For all numbers M = M(k, Ā) which are chosen sufficiently large in dependence of Ā and k the following relation depending on the numbers Ā and M holds: For all numbers u > 0 for which nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ ML log \frac{2}{σ} a number σ̄ = σ̄(u), 0 ≤ σ̄ ≤ σ ≤ 1, and a collection of functions Fσ̄ = {f_{1}, …, f_{m}} ⊂ F with m ≤ Dσ̄^{−L} elements can be chosen in such a way that the sets D_{j} = {f: f ∈ F, ∫ |f − f_{j}|^{2} dμ ≤ σ̄^{2}}, 1 ≤ j ≤ m,

satisfy the relation ⋃_{j=1}^{m} D_{j} = F, and the (degenerate) U-statistics I_{n,k}(f), f ∈ F_{σ̄(u)},

satisfy the inequality

P ( sup_{f∈F_{σ̄(u)}} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ 2CD exp \{−α (\frac{u}{10Āσ} )^{2/k}\}

if nσ^{2} ≥ (\frac{u}{σ} )^{2/k} ≥ ML log \frac{2}{σ}

(12.1)

with the constants α = α(k), C = C(k) appearing in formula (8.9) of Theorem 8.3 and the exponent L and parameter D of the L_{2}-dense class F.

The inequalities 4 (\frac{u}{Āσ̄} )^{2/k} ≥ nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k} and nσ̄^{2} ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} also

hold, provided that nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ M(L + β)^{3/2} log \frac{2}{σ} with β = max (\frac{log D}{n} , 0).

Proof of Proposition 12.1. Let us list the elements of the countable set F as f_{1}, f_{2}, . . .. For all p = 0, 1, 2, … let us choose, by exploiting the L_{2}-density property of the class F, a set F_{p} = {f_{a(p,1)}, …, f_{a(p,m_{p})}} ⊂ F with m_{p} ≤ D 2^{2pL}σ^{−L} elements in such a way that inf_{1≤j≤m_{p}} ∫ (f − f_{a(p,j)})^{2} dμ ≤ 2^{−4p}σ^{2} for all f ∈ F. For all indices a(j, p),

p = 1, 2, …, 1 ≤ j ≤ m_{p}, choose a predecessor a(j′, p − 1), j′ = j′(j, p), 1 ≤ j′ ≤ m_{p−1}, in such a way that the functions f_{a(j,p)} and f_{a(j′,p−1)} satisfy the relation ∫ |f_{a(j,p)} −

f_{a(j′,p−1)}|^{2} dμ ≤ σ^{2}2^{−4(p−1)}. Then we have ∫ (\frac{f_{a(j,p)}−f_{a(j′,p−1)}}{2} )^{2} dμ ≤ 4σ^{2}2^{−4p} and

sup_{x_{j}∈X, 1≤j≤k} |\frac{f_{a(j,p)}(x_{1},...,x_{k})−f_{a(j′,p−1)}(x_{1},...,x_{k})}{2} | ≤ 1. Theorem 8.3 yields that

\begin{aligned}P (A(j, p)) & = P (n^{−k/2}|I_{n,k}(f_{a(j,p)} − f_{a(j′,p−1)})| ≥ \frac{2^{−(1+p)}u}{Ā}) \\ & ≤ C exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\} if 4nσ^{2}2^{−4p} ≥ (\frac{2^{p}u}{8Āσ} )^{2/k} , \\ & 1 ≤ j ≤ m_{p}, p = 1, 2, …, \end{aligned}  (12.2)

and

P (B(s)) = P (n^{−k/2}|I_{n,k}(f_{0,s})| ≥ \frac{u}{2Ā}) ≤ C exp \{−α (\frac{u}{2Āσ} )^{2/k}\} , 1 ≤ s ≤ m,
if nσ^{2} ≥ (\frac{u}{2Āσ} )^{2/k} .  (12.3)

Introduce an integer R = R(u), R > 0, which satisfies the relations

2^{(4+2/k)(R+1)} (\frac{u}{Āσ} )^{2/k} ≥ 2^{2+6/k}nσ^{2} ≥ 2^{(4+2/k)R} (\frac{u}{Āσ} )^{2/k} ,

and define σ̄^{2} = 2^{−4R}σ^{2} and Fσ̄ = F_{R} (i.e the class of functions F_{p} introduced before with p = R). (As nσ^{2} ≥ (\frac{u}{σ})^{2/k} and Ā ≥ 2^{k} by our conditions, there exists such a positive integer R.) Then the cardinality m of the set Fσ̄ is clearly not greater

than Dσ̄^{−L}, and ⋃_{j=1}^{m} D_{j} = F. Besides, the number R was chosen in such a way that the

inequalities (12.2) and (12.3) hold for 1 ≤ p ≤ R. Hence the definition of the predecessor of an index a(j, p) implies that

P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ P ( ⋃_{p=1}^{R} ⋃_{j=1}^{m_{p}} A(j, p) ∪ ⋃_{s=1}^{m} B(s))

≤ ∑_{p=1}^{R} ∑_{j=1}^{m_{p}} P (A(j, p)) + ∑_{s=1}^{m} P (B(s)) ≤ ∑_{p=1}^{∞} CD 2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\}

+ CDσ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2/k}\} .

If the condition (\frac{u}{σ})^{2/k} ≥ ML^{3/2} log \frac{2}{σ} holds with a sufficiently large constant M (depending on Ā), then the inequalities

2^{2pL}σ^{−L} exp \{−α (\frac{2^{p}u}{8Āσ} )^{2/k}\} ≤ 2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2/k}\}

hold for all p = 1, 2, …, and

σ^{−L} exp \{−α (\frac{u}{2Āσ} )^{2/k}\} ≤ exp \{−α (\frac{u}{10Āσ} )^{2/k}\} .

Hence the previous estimate implies that

P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā}) ≤ ∑_{p=1}^{∞} CD2^{−p} exp \{−α (\frac{2^{p}u}{10Āσ} )^{2/k}\}

+ CD exp \{−α (\frac{u}{10Āσ} )^{2/k}\} ≤ 2CD exp \{−α (\frac{u}{10Āσ} )^{2/k}\} ,

and relation (12.1) holds. We have

nσ̄^{2} = 2^{−4R}nσ^{2} ≤ 2^{−4R} · 2^{(4+2/k)(R+1)−2−6/k} (\frac{u}{Āσ} )^{2/k} = 2^{2−4/k} · 2^{2R/k} (\frac{u}{Āσ} )^{2/k}

= 2^{2−4/k} · (\frac{σ}{σ̄} )^{1/k} (\frac{u}{Āσ} )^{2/k} = 2^{2−4/k} · (\frac{σ̄}{σ} )^{1/k} (\frac{u}{Āσ̄} )^{2/k} ,

hence nσ̄^{2} ≤ 4 (\frac{u}{Āσ̄} )^{2/k}. Besides, as nσ^{2} ≥ 2^{(4+2/k)R−2−6/k} (\frac{u}{Āσ} )^{2/k}, R ≥ 1,

nσ̄^{2} = 2^{−4R}nσ^{2} ≥ 2^{−2−6/k} · 2^{2R/k} (\frac{u}{Āσ} )^{2/k} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k} .

It remained to show that nσ̄^{2} ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}}.

This inequality clearly holds under the conditions of Proposition 12.1 if σ ≤ n^{−1/3}, since in this case log \frac{2}{σ} ≥ \frac{log n}{3}, and nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k} ≥ \frac{1}{64} Ā^{−2/k}M(L + β)^{3/2} log \frac{2}{σ} ≥ \frac{1}{192} Ā^{−2/k}M(L + β) log n ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} if M = M(Ā, k) is chosen sufficiently large.

If σ ≥ n^{−1/3}, then the inequality 2^{(4+2/k)R} (\frac{u}{Āσ} )^{2/k} ≤ 2^{2+6/k}nσ^{2} holds. Hence

2^{−4R} ≥ 2^{−4(2+6/k))/(4+2/k)} [\frac{(\frac{u}{Āσ} )2/k}{nσ^{2}} ]^{4/(4+2/k)}, and

nσ̄^{2} = 2^{−4R}nσ^{2} ≥ \frac{2^{−16/3}}{Ā^{4/3}} (nσ^{2})^{1−γ} [(\frac{u}{σ} )^{2/k}]^{γ} with γ = \frac{4}{4 + \frac{2}{k}} ≥ \frac{2}{3}.

Since nσ^{2} ≥ (\frac{u}{σ})^{2/k} ≥ \frac{M}{3} (L + β)^{3/2}, and nσ^{2} ≥ n^{1/3}, the above estimates yield that (nσ^{2})^{1−γ} [(\frac{u}{σ})^{2/k}]^{γ} ≥ (nσ^{2})^{1/3} [(\frac{u}{σ})^{2/k}]^{2/3}, and nσ̄^{2} ≥ \frac{Ā^{−4/3}}{50} (nσ^{2})^{1/3} [(\frac{u}{σ})^{2/k}]^{2/3} ≥ \frac{Ā^{−4/3}}{50} n^{1/9} (\frac{M}{3} )^{2/3} (L + β) ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}}.

Now we formulate a multivariate analog of Proposition 6.2 in Proposition 12.2 and show that Propositions 12.1 and 12.2 imply Theorem 8.4.

Proposition 12.2. Let us have a probability measure μ on a measurable space (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n} and a countable L_{2}-dense class F of canonical kernel functions f = f(x_{1}, …, x_{k}) (with respect to the measure μ) with some parameter D and exponent L on the product space (X^{k}, X^{k}) such that all functions f ∈ F satisfy conditions (8.4) and (8.5) with some 0 < σ ≤ 1, and consider the (degenerate) U-statistics I_{n,k}(f) with the random sequence ξ_{1}, …, ξ_{n} and kernel functions f ∈ F. There exists a sufficiently large constant K = K(k) together with some numbers C̄ = C̄(k) > 0, γ = γ(k) > 0 and threshold index A_{0} = A_{0}(k) > 0 depending only on the order k of the U-statistics such that if nσ^{2} > K(L+β) log n with β = max (\frac{log D}{log n} , 0), then the degenerate U-statistics I_{n,k}(f), f ∈ F, satisfy the inequality

P (sup_{f∈F} |n^{−k/2}I_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ C̄e^{−γA^{1/2k}nσ^{2}} if A ≥ A_{0}.  (12.4)

We shall prove formula (8.10) by applying Proposition 12.2 with the choice σ = σ̄ = σ̄(u) defined in Proposition 12.1 and the classes F = D_{j}, more precisely the classes F = \{\frac{g−f_{j}}{2} : g ∈ D_{j}\} of functions introduced also in Proposition 12.1, where f_{j} is the function appearing in the definition of the class of functions D_{j}. Clearly,

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ P ( sup_{f∈Fσ̄} n^{−k/2}|I_{n,k}(f)| ≥ \frac{u}{Ā})

+ ∑_{j=1}^{m} P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u) ,

(12.5)

where m is the cardinality of the set of functions Fσ̄ appearing in Proposition 12.1. We want to show that if first Ā and then M ≥ M_{0}(Ā) are chosen sufficiently large in Proposition 12.1, then the second term at the right-hand side of formula (12.5) can be well bounded by means of Proposition 12.2, and Theorem 8.4 can be proved by means of this estimate.

To carry out this program let us choose a number Ā_{0} in such a way that Ā_{0} ≥ A_{0} and γĀ_{0}^{1/2k} ≥ \frac{1}{K} with the numbers A_{0}, K and γ in Proposition 12.2, put Ā = max(2^{k+2}Ā_{0}, 2^{k}), and apply Proposition 12.1 with this number Ā. Then by Proposition 12.1 and the choice of the numbers Ā and Ā_{0} also the inequality (\frac{u}{σ̄})^{2/k} ≥ \frac{Ā^{2/k}}{4} nσ̄^{2} ≥ (4Ā_{0})^{2/k}nσ̄^{2} holds, hence u ≥ 4Ā_{0}n^{k/2}σ̄^{k+1} with the number σ̄ in Proposition 12.1. This implies that (\frac{1}{2} − \frac{1}{2Ā}) u ≥ \frac{u}{4} ≥ Ā_{0}n^{k/2}σ̄^{k+1}, Ā_{0} ≥ A_{0}, and by replacing the expression (\frac{1}{2} − \frac{1}{2Ā}) u by Ā_{0}n^{k/2}σ̄^{k+1} in the probabilities of the sum in the second term at the right-hand side of (12.5) we enlarge them.

The numbers u considered in these estimations satisfy the condition nσ^{2/k} ≥ (\frac{u}{σ})^{2/k} ≥ M(L + β)^{3/2} log \frac{2}{σ} imposed in Proposition 12.1 with some appropriately chosen constant M. Choose the number M ≥ M(Ā, k) in Proposition 12.1 (which also can be chosen as the number M in formula (8.10) of Theorem 8.4) in such a way that it also satisfies the inequality \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} ≥ K(L + β) log n with the number K appearing in the conditions of Proposition 12.2. With such a choice the inequality nσ̄^{2} ≥ \frac{M^{2/3}(L+β) log n}{1000Ā^{4/3}} ≥ K(L + β) log n holds, and Proposition 12.2 can be applied to bound the terms in the sum at the right-hand side of (12.5). It yields the estimate

P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ (\frac{1}{2} − \frac{1}{2Ā}) u)

≤ P ( sup_{g∈D_{j}} n^{−k/2} |I_{n,k} (\frac{f_{j} − g}{2} )| ≥ Ā_{0}n^{k/2}σ̄^{k+1}) ≤ C̄e^{−γĀ_{0}^{1/2k}nσ̄^{2}}

for all 1 ≤ j ≤ m. (Observe that the set of functions \frac{f_{j}−g}{2} , g ∈ D_{j} is an L_{2}-dense class with parameter D and exponent L.) Hence Proposition 12.1 (relation (12.1) together with the inequality m ≤ Dσ̄^{−L}) and formula 12.4 with A = Ā_{0} imply that

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u) ≤ 2CD exp \{−α (\frac{u}{10Āσ} )^{2/k}\} + C̄Dσ̄^{−L}e^{−γĀ_{0}^{1/2k}nσ̄^{2}}.

(12.6)

To get the result of Theorem 8.4 from inequality (12.6) we have to replace its second term at the right-hand side with a more appropriate expression where, in particular, we get rid of the coefficient σ̄^{−L}. The condition nσ̄^{2} ≥ K(L + β) log n implies that σ̄ ≥ n^{−1/2}, and by our choice of Ā_{0} we have γĀ_{0}^{1/2k}nσ̄^{2} ≥ \frac{1}{K} nσ̄^{2} ≥ L log n ≥ 2L log \frac{1}{σ̄}, i.e.

σ̄^{−L} ≤ e^{γĀ_{0}^{1/2k}nσ̄^{2}/2}. By the estimates of Proposition 12.1 nσ̄^{2} ≥ \frac{1}{64} (\frac{u}{Āσ} )^{2/k}. The above

relations imply that σ̄^{−L}e^{−γĀ_{0}^{1/2k}nσ̄^{2}} ≤ e^{−γĀ_{0}^{1/2k}nσ̄^{2}/2} ≤ exp \{−\frac{γ}{128} Ā_{0}^{1/2k}Ā^{−2/k} (\frac{u}{σ})^{2/k}\}. Hence relation (12.6) yields that

P (sup_{f∈F} n^{−k/2}|I_{n,k}(f)| ≥ u)

≤ 2CD exp \{−\frac{α}{(10Ā)^{2}} (\frac{u}{σ} )^{2/k}\} + C̄D exp \{−\frac{γ}{128}Ā_{0}^{1/2k}Ā^{−2/k} (\frac{u}{σ} )^{2/k}\} ,

and this estimate implies Theorem 8.4.

Thus to complete the proof of Theorem 8.4 it is enough to prove Proposition 12.2. It turned out to be useful to apply an approach similar to the proof of Theorem 8.3. In the proof of Theorem 8.3 first an appropriate counterpart of this result was proved, where the U-statistics were replaced by their decoupled U-statistics analogs defined in formula (10.5), and then the desired result was deduced from this estimate and Theorem 10.4. Similarly, Proposition 12.2 will be deduced from the following result.

Proposition 12.2′. Let a class of functions f ∈ F on the k-fold product (X^{k}, X^{k}) of a measurable space (X, X), a probability measure μ on (X, X) together with a sequence of independent and μ distributed random variables ξ_{1}, …, ξ_{n} satisfy the conditions of Proposition 12.2. Let us take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the random sequence ξ_{1}, …, ξ_{n}, and consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with their help by formula (10.5). Then there exists a sufficiently large constant K = K(k) together with some number γ = γ(k) > 0 and threshold index A_{0} = A_{0}(k) > 0 depending only on the order k of the decoupled U-statistics we consider such that if nσ^{2} > K(L + β) log n with β = max (\frac{log D}{log n} , 0), then the (degenerate) decoupled U-

statistics Ī_{n,k}(f), f ∈ F, satisfy the following version of inequality (12.4):

P (sup_{f∈F} |n^{−k/2}Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ e^{−γA^{1/2k}nσ^{2}} if A ≥ A_{0}.  (12.7)

It is clear that Proposition 12.2′ and Theorem 10.4, more explicitly formula (10.8′) in it imply Proposition 12.2. The proof of Proposition 12.2 is based on a symmetrization argument which will be explained in the next section. Here an important ingredient of it will be proved, the multivariate version of Hoeffding’s inequality formulated in Theorem 3.4.

Theorem 12.3. The multivariate version of Hoeffding’s inequality. Let ε_{1}, …, ε_{n} be independent random variables, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n. Fix a positive integer k, and define the random variable

Z = ∑_{(j_{1},...,j_{k}) : 1≤j_{l}≤n for all 1≤l≤k j_{l}≠j_{l′} if l≠l′} a(j_{1}, …, j_{k})ε_{j_{1}} ⋯ ε_{j_{k}}  (12.8)

with the help of some real numbers a(j_{1}, …, j_{k}) which are given for all sets of indices such that 1 ≤ j_{l} ≤ n, 1 ≤ l ≤ k, and j_{l} ≠ j_{l′} if l ≠ l′. Put

S^{2} = ∑_{(j_{1},...,j_{k}) : 1≤j_{l}≤n for all 1≤l≤k j_{l}≠j_{l′} if l≠l′} a^{2}(j_{1}, …, j_{k})  (12.9)

Then

P (|Z| > u) ≤ C exp \{−B (\frac{u}{S} )^{2/k}\} for all u ≥ 0  (12.10)

with some constants B > 0 and C > 0 depending only on the parameter k. Relation (12.10) holds for instance with the choice B = \frac{k}{2e(k!)^{1/k}} and C = e^{k}.

Proof of Theorem 12.3. We get with the help of formula (10.4) (which is a consequence of Borell’s inequality) that

E|Z|^{q} ≤ (q − 1)^{kq/2} (EZ^{2})^{q/2} ≤ q^{kq/2} (EZ^{2})^{q/2} = q^{kq/2}S̄^{q}

with

S̄^{2} = ∑_{1≤j_{1}<j_{2}···<j_{k}≤n} ( ∑_{π∈Π_{k}} a((j_{π(1)}, …, j_{π(k)}))^{2} ,

where Π_{k} denotes the set of all permutations of the set {1, …, k}. Observe that

( ∑_{π∈Π_{k}} a(j_{π(1)}, …, j_{π(k)}))^{2} ≤ k! ∑_{π∈Π_{k}} a^{2}(j_{π(1)}, …, j_{π(k)}) for all 1 ≤ j_{1} < ⋯ j_{k} ≤ n,

hence S̄^{2} ≤ k!S^{2}, and E|Z|^{q} ≤ q^{kq/2}(k!)^{q/2}S^{q} with the number S^{2} defined in (12.9). Thus the Markov inequality implies that

P (|Z| > u) ≤ (q^{k/2} \frac{\sqrt{k!}S}{u} )^{q} for all u > 0 and q ≥ 2.

Choose the number q as the solution of the equation q (\frac{\sqrt{k!}S}{u} )^{2/k} = \frac{1}{e}. Then we get that

P (|Z| > u) ≤ exp \{−B (\frac{u}{S} )^{2/k}\} with B = \frac{k}{2e(k!)^{1/k}}, provided that q = \frac{1}{ek!^{1/k}} (\frac{u}{S} )^{2/k} ≥

2, i.e. B (\frac{u}{S} )^{2/k} ≥ k. By multiplying the above upper bound with C = e^{k} we get such an estimate for P (|Z| > u) which holds for all u > 0.

Remark: The result of Theorem 12.3 will be good enough for our purposes, although the constants B and C we have given in formula (12.10) are not optimal. Thus Theorem 3.4 yields that in the special case k = 1 the estimate (12.10) holds with B = \frac{1}{2} and C = 1 (and not only with B = \frac{1}{2e} and C = e). The reason for this relative weakness of Theorem 12.3 is that the moment estimate given for a homogeneous polynomial of Rademacher functions in formula (10.4) is not always sharp. In Theorem 16.6 I present (without proof) an improved version of Theorem 12.3 which yields the estimate (12.10) with the right constant C in the exponent. The proof can be found in paper [22]. It is based on a sharp estimate on the moments EZ^{2M} for large positive integers M formulated in Theorem 16.7. This estimate can be considered as the improvement of Bernstein’s inequality in a most important special case.

13. The strategy of the proof for the main result of this paper

We have reduced the proof of the main result of this paper, the proof of Theorem 8.4 to that of Proposition 12.2′. It is a multivariate version of Proposition 6.2, and also its proof is based on similar ideas. In particular, a multivariate version of Lemma 7.2 will be proved, which means some kind of randomization. In this result we consider a class of decoupled, degenerate U-statistics Ī_{n,k}(f) together with a class of randomized, decoupled U-statistics Ī_{n,k}(f, ε) defined in formulas (10.5) and (10.6) respectively with the same countable class of functions f ∈ F and want to bound the

probability P (n^{−k/2} sup_{f∈F} Ī_{n,k}(f)| > u) with the help of a probability of the form

P (n^{−k/2} sup_{f∈F} Ī_{n,k}(f, ε)| > Bu) with some appropriate universal constant B > 0.

To carry out such a program we introduce 2k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)} and ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)}, 1 ≤ j ≤ n, of the sequence of random variables ξ_{1}, …, ξ_{n} we have at the start. We shall work with these 2k copies and a sequence of independent random variables ε = (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, independent also of the sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)} and ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)}, 1 ≤ j ≤ k. Given some function f(x_{1}, …, x_{k}) of k variables in the k-fold power (X^{k}, X^{k}) of some measurable space (X, X) let us consider the random sums Ĩ_{n,k}(f) and Ĩ_{n,}(f, ε) defined in formulas (11.9) and (11.9′) with the help of the above random sequences. We shall use Lemma 11.5 which states that given a class of functions of k-variables f ∈ F, the joint distribution of the random variables Ĩ_{n,k}(f) and Ĩ_{n,}(f, ε) agree.

As we shall see later, Lemma 11.5 enables us to reduce the multivariate version of Lemma 7.2 we would like to prove to an appropriate bounding of the distribution of sup_{f∈F} Ī_{n,k}(f) by that of sup_{f∈F} Ĩ_{n,k}(f). In the proof of Lemma 7.2 we met a simple special

case of this problem, and it could be solved by means of the Symmetrization Lemma (Lemma 7.1). In the general case Lemma 7.1 is not sufficient for our purposes, since we have to work with not necessarily independent random variables. Hence we prove a generalized version of it.

Lemma 13.1 (Generalized version of the Symmetrization Lemma.) Let Z_{n} and Z̄_{n}, n = 1, 2, …, be two sequences of random variables on a probability space (Ω, A, P). Let a σ-algebra B ⊂ A be given on the probability space (Ω, A, P) together with a Bmeasurable set B and two numbers α > 0 and β > 0 such that the random variables Z_{n}, n = 1, 2, …, are B measurable, and the inequality

P (|Z̄_{n}| ≤ α|B)(ω) ≥ β for all n = 1, 2, … if ω ∈ B  (13.1)

holds. Then

P ( sup_{1≤n<∞} |Z_{n}| > α + u) ≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) + (1 − P (B)) for all u > 0.

(13.2)

Proof of Lemma 13.1. Put τ = min{n: |Z_{n}| > α + u) if there exists such an n, and τ = 0 otherwise. Then

P (\{τ = n\} ∩ B) ≤ ∫_{\{τ=n\}∩B} \frac{1}{β} P (|Z̄_{n}| ≤ α|B) dP = \frac{1}{β} P (\{τ = n\} ∩ \{|Z̄_{n}| ≤ α\} ∩ B)

≤ \frac{1}{β} P (\{τ = n\} ∩ \{|Z_{n} − Z̄_{n}| > u\}) for all n = 1, 2, ….

Hence

P ( sup_{1≤n<∞} |Z_{n}| > α + u) − (1 − P (B)) ≤ P ({ sup_{1≤n<∞} |Z_{n}| > α + u} ∩ B)

= ∑_{n=1}^{∞} P (\{τ = n\} ∩ B) ≤ \frac{1}{β} ∑_{n=1}^{∞} P (\{τ = n\} ∩ \{|Z_{n} − Z̄_{n}| > u\})

≤ \frac{1}{β} P ( sup_{1≤n<∞} |Z_{n} − Z̄_{n}| > u) .

Thus Lemma 13.1 is proved.

The main difficulty we meet when we try to prove Proposition 12.2′ instead of its simpler version, Proposition 6.2 is that now we have to check an estimate of the form (13.1) with some appropriately chosen random variables Z_{n}, σ-algebra B and set B instead of the estimate (7.1) applied in the proof of Proposition 6.2. The cause of this difference is that now we have to work with not completely independent random variables. In the symmetrization argument needed in the proof of Proposition 6.2 we could simply check inequality (7.1) by calculating the variance of the random variables we were working with. On the other hand, to check inequality (13.1) in the symmetrization argument we want to apply in the present case we shall bound the conditional variance of certain random variables, and we can only state that this conditional variance is relatively small with great probability.

In the proof of Proposition 12.2′ we formulate and prove a multivariate version of the definition of good tail behaviour for a class of normalized random sums, where the normalized random sums are replaced by degenerate decoupled U-statistics. It is enough to prove the good tail-behaviour of decoupled U-statistics introduced below by means of an appropriate induction, and Proposition 12.2′ follows from it. But to carry out such a program we have to formulate and check another property which will be called the good tail behaviour for a class of integrals of decoupled U-statistics. This property helps us to carry out the induction procedure needed in the proof of Proposition 12.2′. Its introduction and proof corresponds to the symmetrization argument formulated in Lemma 7.2 in the proof of Proposition 6.2. The above mentioned two properties will be proved simultaneously. Before their formulation I make some comments about the idea behind the introduction of the property ‘good tail behaviour for a class of integrals of decoupled U-statistics’.

In the introduction of this property we consider a class of functions f(x_{1}, …, x_{k}, y) depending on a parameter y ∈ Y, but in all further applications we shall apply this property with the choice Y = X^{l} and ρ = μ^{l} (i.e. with the l-th power of the space X and the probability measure μ on it) with some integer l. The property ‘good tail behaviour for a class of integrals of decoupled U-statistics’ with the above choice will be useful for us for the following reason.

We shall consider the expression introduced in formula (11.9), which is some sort of linear combination of decoupled U-statistics, and want to bound the inner sums at the right-hand side of this expression. More explicitly, we consider those inner sum terms for which m(v) = l ≥ 1, i.e. for which the original sample elements are replaced by their independent copies in l ≥ 1 coordinates. We want to calculate the conditional variance of such sums under the condition that the values of the elements of the original sample are prescribed. The property of good tail behaviour for a class of integrals of decoupled U-statistics helps us in getting a good estimate for these expressions. If we want to bound the conditional variance of such an inner sum where the original sample elements are replaced in l coordinates, then the application of the property of good tail behaviour for a class of integrals of U-statistics with k − l instead of k parameters and with the choice (Y, Y, ρ) = (X^{l}, X^{l}, μ^{l}) will be useful. By applying this property with such a choice together with the canonical property of the function f(x_{1}, …, x_{k}) we shall work with we can prove the estimate we need.

Let me also remark that the estimate (13.5) we have imposed in the definition of the property of ‘good tail behaviour for a class of integrals of U-statistics’ is fairly natural. We have applied the natural normalization, and with such a normalization it is natural to expect that the distribution of sup_{f∈F} n^{−k}H_{n,k}(f) behaves similarly to that of

const. (ση^{2})^{k}, where η is a standard normal random variables. Formula (13.5) expresses such a behaviour, only the power of the number A in the exponent at the right-hand side was chosen in a non-optimal way.

Naturally, we want to prove the property of good tail behaviour for a class of integrals of decoupled U-statistics under appropriate, not too restrictive conditions. Let me remark that in the conditions of Proposition 13.3 we want to prove we have imposed besides formula (13.6) a fairly weak condition (13.7). Most difficulties arise in the proof because we want to work with this condition. Here we did not demand that the L_{2}-norm of the functions f(x_{1}, …, x_{k}, y) should be small for all parameters y. We only assumed that some average of these L_{2}-norms expressed in formula (13.7) are small. Now I formulate the definition of the properties we shall work with.

Definition of good tail behaviour for a class of decoupled U-statistics. Let us have some measurable space (X, X) and a probability measure μ on it. Let us consider some class F of functions f(x_{1}, …, x_{k}) on the k-fold product (X^{k}, X^{k}) of the space (X, X). Fix some positive integer n ≥ k and positive number 0 < σ ≤ 1, and take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ-distributed random variables ξ_{1}, …, ξ_{n}. Let us introduce with the help of these random variables the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined in formula (10.5). Given some real number T > 0 we say that the set of decoupled U-statistics determined by the class of functions F has a good tail behaviour at level T (with parameters n and σ^{2} which we fix in the sequel) if the inequality holds:

P (sup_{f∈F} |n^{−k/2}Ī_{n,k}(f)| ≥ An^{k/2}σ^{k+1}) ≤ exp {−A^{1/2k}nσ^{2}} for all A > T. (13.3)

We shall also introduce the following property:

Definition of good tail behaviour for a class of integrals of decoupled Ustatistics. Let us have a product space (X^{k} × Y, X^{k} × Y) with some product measure μ^{k} ×ρ, where (X^{k}, X^{k}, μ^{k}) is the k-fold product of some probability space (X, X , μ), and (Y, Y, ρ) is some other probability space. Fix some positive integer n ≥ k and positive number σ > 0, and consider some class F of functions f(x_{1}, …, x_{k}, y) on the product space (X^{k} × Y, X^{k} × Y, μ^{k} × ρ). Take k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent, μ-distributed random variables ξ_{1}, …, ξ_{n}. For all f ∈ F and y ∈ Y let us define the decoupled U-statistics Ī_{n,k}(f, y) = Ī_{n,k}(f_{y}) by means of these random variables ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, the kernel function f_{y}(x_{1}, …, x_{k}) = f(x_{1}, …, x_{k}, y) and formula (10.5). Define with the help of these U-statistics Ī_{n,k}(f, y) the random integrals

H_{n,k}(f) = ∫ Ī_{n,k}(f, y)^{2}ρ( dy), f ∈ F.  (13.4)

Choose some real number T > 0. We say that the set of random integrals H_{n,k}(f), f ∈ F, have a good tail behaviour at level T (with parameters n and σ^{2} which we fix in the sequel) if

P (sup_{f∈F} n^{−k}H_{n,k}(f) ≥ A^{2}n^{k}σ^{2k+2}) ≤ exp {−A^{1/(2k+1)}nσ^{2}} for A > T. (13.5)

Now I formulate those two inductive statements in Propositions 13.2 and 13.3 which imply that the above introduced properties of good tail behaviour for a class of decoupled U-statistics and good tail behaviour for a class of integrals of decoupled U-statistics hold under fairly general conditions. Proposition 12.2′ can be obtained as a relatively simple consequence of these results.

Proposition 13.2. Let us fix a positive integer n ≥ k, a real number 0 < σ ≤ 2^{−(k+1)} and a probability measure μ on a measurable space (X, X) together with a countable L_{2}-dense class F of canonical kernel functions f = f(x_{1}, …, x_{k}) (with respect to the measure μ) on the k-fold product space (X^{k}, X^{k}) which has exponent L ≥ 1 and parameter D. Let us also assume that all functions f ∈ F satisfy the condi-

tions sup_{x_{j}∈X,1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 2^{−(k+1)}, ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2},

and nσ^{2} > K(L + β) log n with an appropriately chosen fixed number K = K(k) with β = max (\frac{log D}{log n} , 0).

There exists some real number A_{0} = A_{0}(k) > 1 such that if all classes of functions F satisfying the above conditions the sets of decoupled U-statistics determined Ī_{n,k}(f), f ∈ F, have a good tail behaviour at level T^{4/3} for some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Proposition 13.3. Fix some positive integer n ≥ k and real number 0 < σ ≤ 2^{−(k+1)}, and let us have a product space (X^{k}×Y, X^{k}×Y) with some product measure μ^{k}×ρ, where (X^{k}, X^{k}, μ^{k}) is the k-fold product of some probability space (X, X , μ), and (Y, Y, ρ) is some other probability space. Let us have a countable L_{2}-dense class F of canonical functions f(x_{1}, …, x_{k}, y) on the product space (X^{k} × Y, X^{k} × Y, μ^{k} × ρ) with some exponent L ≥ 1 and parameter D. Let us also assume that the functions f ∈ F satisfy the conditions

sup_{x_{j}∈X,1≤j≤k,y∈Y} |f(x_{1}, …, x_{k}, y)| ≤ 2^{−(k+1)}  (13.6)

and

∫ f^{2}(x_{1}, …, x_{k}, y)μ( dx_{1}) … μ( dx_{k})ρ( dy) ≤ σ^{2} for all f ∈ F.  (13.7)

Let the inequality nσ^{2} > K(L + β) log n hold with a sufficiently large, appropriately chosen number K = K(k) and β = max (\frac{log D}{log n} , 0).

There exists some number A_{0} = A_{0}(k) > 1 such that if for all classes of functions F which satisfy the above conditions the random integrals H_{n,k}(f), f ∈ F, defined in (13.4) have a good tail behaviour at level T^{(2k+1)/2k} with some T ≥ A_{0}, then they also have a good tail behaviour at level T.

Remark: In the conditions of Proposition 13.3 the notion of canonical functions appeared in a slightly more general form than it was defined in formula (8.8). We say that a function f(x_{1}, …, x_{k}, y) on the product space (X^{k} × Y, X^{k} × Y, μ^{k} × ρ) is canonical if

∫ f(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{k}, y)μ( du) = 0
for all 1 ≤ j ≤ k, x_{s} ∈ X, s ≠ j and y ∈ Y

and

∫ f(x_{1}, …, x_{k}, y)ρ( dy) = 0 for all x_{j} ∈ X, 1 ≤ j ≤ k.

It is not difficult to deduce Proposition 12.2′ from Proposition 13.2. Indeed, let us observe that the set of decoupled U-statistics determined by a class of functions F satisfying the conditions of Proposition 13.2 has a good tail-behaviour at level T_{0} = σ^{−(k+1)}, since under the conditions of this Proposition the probability at the left-hand side of (13.3) equals zero for A > σ^{−(k+1)}. Then we get from Proposition 13.2 by induction with respect to the number j, that this set of decoupled U-statistics has a good tail-behaviour also for all T ≥ T_{0}^{(3/4)^{j}} = σ^{−(k+1)(3/4)^{j}} for j = 0, 1, 2, … if σ^{−(k+1)(3/4)^{j}} ≥ A_{0}. (Observe that σ < 1 under the conditions of Proposition 13.2, since σ^{2} ≤ 2^{−2(k+1)} in this case.) This implies that if a class of functions F satisfies the conditions of Proposition 13.2, then the set of decoupled U-statistics determined by this class of functions has a good tail-behaviour at level T = A_{0}^{4/3}, i.e. at a level which depends only on the order k of the decoupled U-statistics. This result implies Proposition 12.2′, only we have to apply it not directly for the class of functions F appearing in it, but these functions have to be multiplied by a sufficiently small positive number depending only on k.

Similarly to the above argument an inductive procedure yields a corollary of Proposition 13.3 formulated below. Actually, we shall need this corollary of Proposition 13.3.

Corollary of Proposition 13.3. If the class of functions F satisfies the conditions of Proposition 13.3, then there exists a constant Ā_{0} = Ā_{0}(k) > 0 depending only on k such that the class of integrals H_{n,k}(f), f ∈ F defined in formula (13.4) have a good tail behaviour at level Ā_{0}.

The main difficulty in the proof of Proposition 13.2 appears as we try to apply the symmetrization procedure corresponding to Lemma 7.2 in the one-variate case. This difficulty can be overcome by means of Proposition 13.3, more precisely its corollary. It helps us to estimate the conditional variances of the decoupled U-statistics we have to handle in the proof of Proposition 13.2. The proof of Propositions 13.2 and 13.3 apply similar arguments, and they will be proved simultaneously. These results will be proved by means of the following inductive procedure. First Propositions 13.2 and then Proposition 13.3 are proved for k = 1. If Propositions 13.2 and 13.3 are already proved for all k′ < k for some number k, then first we prove Proposition 13.2 and then Proposition 13.3 for this number k.

14. A symmetrization argument

The proof of Propositions 13.2 and 13.3 apply similar ideas to the proof of Proposition 6.2, but here some additional technical difficulties have to be overcome. As a first step we prove two results formulated in Lemma 14.1A and 14.1B. They can be considered as a symmetrization argument analogous to Lemma 7.2 applied in the proof of Propositions 6.2. Lemma 14.1A will be needed in the proof of Proposition 13.2 and Lemma 14.1B in the proof of Proposition 13.3. This section contains the proof of these results.

Lemma 14.1A is a natural multivariate version of Lemma 7.2. In this result the probability we want to estimate in Proposition 13.2 is bounded by means of the distribution of the supremum of homogeneous polynomials of Rademacher functions of order k (the order of the decoupled U-statistic we investigate), and such an expression can be investigated similarly to the proof of Proposition 6.2 by means of the multi-dimensional version of Hoeffding’s inequality given in Theorem 12.3. The case of Lemma 14.1B is more complicated. The probability we want to investigate in Proposition 13.3 will be bounded by the distribution of the supremum of some random variables W̄ (f), f ∈ F, which will be defined in formula (14.8). The expressions W̄ (f) are squares of random polynomials of Rademacher functions. It is useful to study them more closely. This will be done in the proof of corollary of Lemma 14.1B which yields a more appropriate bound for the expression we want to estimate in Proposition 13.3. We shall apply this corollary in the sequel.

The proof of Lemmas 14.1A and 14.1B is similar to that of Lemma 7.2. First we introduce an independent copy ξ̄_{n}^{(j)}, …, ξ̄_{n}^{(j)} of the k sequences ξ_{n}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, and construct with their help some appropriate expressions which have the same distribution as the randomized sums we shall work with in the proof of Lemmas 14.1A and 14.1B. This statement will be formulated and proved in Lemmas 14.2A and 14.2B. These results enable us to reduce the problems we are interested in to some simpler questions which can be studied with the help of Lemmas 14.3A and 14.3B. In Lemma 14.3A the conditional variance of a random variable is estimated under some appropriate conditions. This estimate together with the generalized form of the Symmetrization Lemma enable us to prove Lemma 14.1A. Lemma 14.1B can be proved similarly, but here we need an estimate about the conditional distribution of a more complicated expression. This estimate can be proved with the help of Lemma 14.3B. In Lemma 14.3B the conditional expectation of the absolute value of an appropriate expression is bounded.

Now we formulate the main results of this section.

Lemma 14.1A. Let F be a class of functions on the space (X^{k}, X^{k}) which satisfies the conditions of Proposition 13.2 with some probability measure μ. Let us have k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n}, and a sequence of independent random variables ε = (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is independent also of the random sequences ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k. Consider the decoupled U-statistics Ī_{n,k}(f), f ∈ F, defined with the help of these random variables by formula (10.5) together with their randomized version

Ī_{n,k}^{ε}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)}) , f ∈ F.  (14.1)

Then there exists some constant A_{0} = A_{0}(k) > 0 such that the inequality

\begin{aligned}P (sup_{f∈F} n^{−k/2} |Ī_{n,k}(f)| > An^{k/2}σ^{k+1}) & < 2^{k+1}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) \\ & + 2^{k}n^{k−1}e^{−A^{1/(2k−1)}nσ^{2}/k} \end{aligned}

(14.2)

holds for all A ≥ A_{0}.

To formulate Lemma 14.1B first we have to introduce some new quantities. We introduce them, because we want to adapt the symmetrization argument of Lemma 11.5 to the case when we work with a function f(x_{1}, …, x_{k}, y) depending on a parameter y, and we have to introduce some new notions in this new situation. Some of the quantities introduced below will be used somewhat later. The quantities Ī_{n,k}^{V}(f, y) introduced in (14.3) will depend on the sets V ⊂ {1, …, k}, and they are the natural adaptations of the inner sum terms in formula (14.9). Such expressions are needed when we want to formulate that version of the symmetrization result of Lemma 11.5 which is needed in the proof of Proposition 13.3. Their randomizations Ī_{n,k}^{(V,ε)}(f, y), introduced in formula (14.6), correspond to the inner sum terms in formula (11.9′). We also introduce the integrals of these expressions in formulas (14.4) and (14.7).

Let us consider a class F of functions f(x_{1}, …, x_{k}, y) ∈ F on a space (X^{k} ×Y, X^{k} × Y, μ^{k} ×ρ) which satisfies the conditions of Proposition 13.3. Let us take 2k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, ξ̄_{1}^{(j)}, …, ξ̄_{n}^{(j)}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{k} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous random sequences. Let us introduce the notation ξ_{l}^{(j,1)} = ξ_{l}^{(j)} and ξ_{l}^{(j,−1)} = ξ̄_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k. For all subsets V ⊂ {1, …, k} of the set {1, …, k} let |V | denote the cardinality of this set, and define for all functions f(x_{1}, …, x_{k}, y) ∈ F and V ⊂ {1, …, k} the decoupled U-statistics

Ī_{n,k}^{V}(f, y) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y) ,  (14.3)

where δ_{j} = ±1, 1 ≤ j ≤ k, δ_{j} = 1 if j ∈ V, and δ_{j} = −1 if j ∉ V, together with the random variables

H_{n,k}^{V}(f) = ∫ Ī_{n,k}^{V}(f, y)^{2}ρ( dy), f ∈ F.  (14.4)

Put

Ī_{n,k}(f, y) = Ī_{n,k}^{{1,...,k}}(f, y), H_{n,k}(f) = H_{n,k}^{{1,...,k}}(f),  (14.5)

i.e. Ī_{n,k}(f, y) and H_{n,k}(f) are the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) with V = {1, …, k} which means that these expressions are defined with the help of the random variables ξ_{l}^{(j,1)}, 1 ≤ j ≤ k, 1 ≤ l ≤ n.

Let us also define the ‘randomized version’ of the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) as

Ī_{n,k}^{(V,ε)}(f, y) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y) , f ∈ F, (14.6)

and

H_{n,k}^{(V,ε)}(f) = ∫ Ī_{n,k}^{(V,ε)}(f, y)^{2}ρ( dy), f ∈ F,  (14.7)

where δ_{j} = 1 if j ∈ V, and δ_{j} = −1 if j ∈ {1, …, k} \ V.

Let us also introduce the random variables

W̄ (f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{|V |}Ī_{n,k}^{(V,ε)}(f, y)]^{2} ρ( dy), f ∈ F  (14.8)

With the help of the above notations we can formulate Lemma 14.1B.

Lemma 14.1B. Let F be a set of functions on (X^{k} × Y, X^{k} × Y) which satisfies the conditions of Proposition 13.3 with some probability measure μ^{k} × ρ. Let us have 2k independent copies ξ_{1}^{j,±1}, …, ξ_{n}^{j,±1}, 1 ≤ j ≤ k, of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{n} together with a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n, which is independent also of the previously considered sequences.

Then there exists some constant A_{0} = A_{0}(k) > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, (this property was defined in Section 13 in the definition of good tail behaviour for a class of integrals of decoupled U-statistics), then the inequality

\begin{aligned}P (sup_{f∈F} |H_{n,k}(f)| > A^{2}n^{2k}σ^{2(k+1)}) & < 2P (sup_{f∈F} |W̄ (f)| > \frac{A^{2}}{2} n^{2k}σ^{2(k+1)}) \\ & + 2^{2k+1}n^{k−1}e^{−A^{1/2k}nσ^{2}/k} \end{aligned} (14.9)

holds with the random variables H_{n,k}(f) introduced in the second identity of relation (14.5) and with W̄ (f) defined in formula(14.8) for all A ≥ T.

We formulate a corollary of Lemma 14.1B which can be better applied than the original lemma. The inconvenience in Lemma 14.B arises, because at the right-hand side of formula (14.9) we have a probability depending on sup_{f∈F} |W̄ (f)|, and W̄ (f) is a too

complicated expression. It equals the integral of the square of homogeneous polynomials of Rademacher functions (with random coefficients) depending on a parameter y with respect to this parameter. We have to understand better the structure of W̄ (f). Hence we shall rewrite it by means of relations (14.10) and (14.11) in a somewhat complicated, but more explicit form. These formulas enable us to find such a corollary of Lemma 14.B which is more appropriate for us. To work out the details first we introduce some diagrams.

Let G = G(k) denote the set of all diagrams consisting of two rows, such that each row is the set {1, …, k}, and the diagrams of G contain some edges {(j_{1}, j′_{1}) …, (j_{s}, j′_{s})}, 0 ≤ s ≤ k, connecting some point (vertex) of the first row with some point (vertex) of the second row. The vertices j_{1}, …, j_{s} which are end points of some edge in the first row are all different, and the same relation holds also for the vertices j′_{1}, …, j′_{s} in the second row. Given some diagram G ∈ G let e(G) = {(j_{1}, j′_{1}) …, (j_{s}, j′_{s})} denote the set of its edges, and let v_{1}(G) = {j_{1}, …, j_{s}} be the set of those vertices in the first row and v_{2}(G) = {j′_{1}, …, j′_{s}} the set of those vertices in the second row of the diagram G from which an edge of G starts.

Given some diagram G ∈ G and two sets V_{1}, V_{2} ⊂ {1, …, k}, we define with the help of the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}, ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k, and ε = (ε_{1}, …, ε_{n}) taking part in the definition of the random variables W̄ (f) the following random variables H_{n,k}(f|G, V_{1}, V_{2}):

H_{n,k}(f|G, V_{1}, V_{2}) = ∑_{l_{1},...,l_{k}, l′_{1},...,l′_{k} 1≤l_{j}≤n, l_{j}≠l_{j′} if j≠j′, 1≤j,j′≤k, 1≤l′_{j}≤n, l′_{j}≠l′_{j′} if j≠j′, 1≤j,j′≤k, l_{j}=l′_{j′} if (j,j′)∈e(G), l_{j}≠l′_{j′} if (j,j′)∉e(G)} ∏_{j∈{1,...,k}\v_{1}(G)}ε_{l_{j}} ∏_{j∈{1,...,k}\v_{2}(G)}ε_{l′_{j}}

\frac{1}{k!^{2}} ∫ f(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)f(ξ_{l′_{1}}^{(1,δ̄_{1})}, …, ξ_{l′_{k}}^{(k,δ̄_{k})}, y)ρ( dy)

(14.10)

where δ_{j} = 1 if j ∈ V_{1}, δ_{j} = −1 if j ∉ V_{1}, and δ̄_{j} = 1 if j ∈ V_{2}, δ̄_{j} = −1 if j ∉ V_{2}. (Let us observe that if the graph G contains s edges, then the product of the ε-s in (14.10) contains 2(k − s) terms and the number of terms in the sum (14.10) is of order n^{2k−s}.) As the Corollary of Proposition 14.1B will indicate in the proof of Proposition 13.3 the expression H_{n,k}(f|G, V_{1}, V_{2}) has to be estimated. This can be done by means of Theorem 12.3, the multivariate version of Hoeffding’s inequality. But the estimate we get in such a way has to be rewritten in a form appropriate for our inductive procedure. This will be done in the next section.

We shall prove that the identity

W̄ (f) = ∑_{G∈G, V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|}H_{n,k}(f|G, V_{1}, V_{2})  (14.11)

holds.

To prove this identity let us write first

W̄ (f) = ∑_{V_{1},V_{2}⊂{1,...,k}}(−1)^{|V_{1}|+|V_{2}|} ∫ Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y)ρ( dy).

Then let us express the products Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y) by means of formula (14.6). Let

us rewrite this product as a sum of products of the form \frac{1}{k!^{2}} ∏_{j=1}^{k} ε_{l_{j}}f(⋯) ∏_{j=1}^{k} ε_{l′_{j}}f(⋯)

and let us define the following partition of the terms in this sum. The elements of this partition are indexed by the diagrams G ∈ G, and if we take a diagram G ∈ G with the set of edges e(G) = {(j_{1}, j′_{1}), …, (j_{s}, j′_{s})}, then the term of this sum determined by the indices l_{1}, …, l_{k}, l′_{1}, …, l′_{k} belongs to the element of the partition indexed by this diagram G if and only if l_{j_{u}} = l′_{j′_{u}} for all 1 ≤ u ≤ s, and no more numbers between the indices l_{1}, …, l_{k}, l′_{1} …, l′_{k} may agree. Since ε_{l_{ju}} ε_{l′_{j′_{u}}} = 1 for all 1 ≤ u ≤ s and all other ε_{l_{j}} and ε_{l′_{j}} are different for a term of the sum in the element of the partition indexed by

the diagram G we get by integrating the product Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y) with respect to the measure ρ that

∫ Ī_{n,k}^{(V_{1},ε)}(f, y)Ī_{n,k}^{(V_{2},ε)}(f, y)ρ( dy) = ∑_{G∈G} H_{n,k}(f|G, V_{1}, V_{2})

for all V_{1}, V_{2} ∈ {1, …, k}. The last two relations imply formula (14.11).

Since the number of terms in the sum of formula (14.11) is less than 2^{4k}k!, this relation implies that Lemma 14.1B has the following corollary:

Corollary of Lemma 14.1B. Let a set of functions F satisfy the conditions of Proposition 13.3. Then there exists some constant A_{0} = A_{0}(k) > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

P (sup_{f∈F} |H_{n,k}(f)| > A^{2}n^{k}σ^{2(k+1)})

≤ 2 ∑_{G∈G, V_{1},V_{2}⊂\{1,...,k\}} P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{k}σ^{2(k+1)})

+ 2^{2k+1}n^{k−1}e^{−A^{1/2k}nσ^{2}/k}

(14.12)

holds with the random variables H_{n,k}(f) and H_{n,k}(f|G, V_{1}, V_{2}) defined in formulas (14.5) and (14.10) for all A ≥ T.

In the proof of Lemmas 14.1A and 14.1B we apply the result of the following Lemmas 14.2A and 14.2B.

Lemma 14.2A. Let us take 2k independent copies

ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k,

of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{k} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous sequences.

Let F be a class of functions which satisfies the conditions of Proposition 13.2. Introduce with the help of the above random variables for all sets V ⊂ {1, …, k} and functions f ∈ F the decoupled U-statistic

Ī_{n,k}^{V}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})})  (14.13)

and its ‘randomized version’

Ī_{n,k}^{(V,ε)}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ε_{l_{1}} ⋯ ε_{l_{k}}f (ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}) , f ∈ F,  (14.13′)

where δ_{j} = ±1, and δ_{j} = 1 if j ∈ V, and δ_{j} = −1 if j ∈ {1, …, k} \ V.

Then the sets of random variables

S(f) = ∑_{V ⊂{1,...,k}}(−1)^{|V |}Ī_{n,k}^{V}(f), f ∈ F  (14.14)

and

S̄(f) = ∑_{V ⊂{1,...,k}}(−1)^{|V |}Ī_{n,k}^{(V,ε)}(f), f ∈ F  (14.14′)

have the same joint distribution.

Lemma 14.2B. Let us take 2k independent copies

ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, 1 ≤ j ≤ k,

of a sequence of independent μ distributed random variables ξ_{1}, …, ξ_{k} together with a sequence of independent random variables (ε_{1}, …, ε_{n}), P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the previous sequences. Let F be a class of functions of k variables satisfying the conditions of Proposition 13.3. For all functions f ∈ F and V ∈ {1, …, k} consider the decoupled U-statistics Ī_{n,k}^{V}(f, y) defined by for-

mula (14.3) with the help of the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} and ξ_{1}^{(j,−1)}, …, ξ_{n}^{(j,−1)}, and define with their help the random variables

W (f) = ∫ [ ∑_{V ⊂{1,...,k}}(−1)^{|V |}Ī_{n,k}^{V}(f, y)]^{2} ρ( dy), f ∈ F.  (14.15)

Then the random vectors {W (f): f ∈ F} defined in (14.15) and {W̄ (f): f ∈ F} defined in (14.8) have the same distribution.

Proof of Lemmas 14.2A and 14.2B. Lemma 14.2A agrees actually with the already proved result Lemma 11.5, only the notation is different. The proof of Lemma 14.2B is also similar to the proof of 11.5. We can state that even the following stronger statement holds. For any ±1 sequence (u_{1}, …, u_{n}) of length n the conditional distribution of the random field W̄ (f), f ∈ F, under the condition (ε_{1}, …, ε_{n}) = (u_{1}, …, u_{n}) agrees with the distribution of the random field W (f), f ∈ F. To see this relation let us first observe that the conditional distribution of the field W̄ (f) under this condition agrees with the distribution of the random field we get by replacing the random variables ε_{l} by u_{l} for all 1 ≤ l ≤ n in formulas (14.6) and (14.8). Besides, we get by replacing the vectors (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}) by (ξ_{l}^{(j,−1)}, ξ_{l}^{(j,1)}) for those indices (j, l) for which u(l) = −1 (independently of the value of the parameter j) and not modifying these vectors with coordinates (l, j) such that u(l) = 1 a measure preserving transformation of the distribution of the random vector consisting of the random variables (ξ_{l}^{(j,1)}, ξ_{l}^{(j,−1)}), 1 ≤ l ≤ n, 1 ≤ j ≤ k, and this implies that also the distribution of the field W (f), f ∈ F, agrees with the distribution of the field we obtain by carrying out the above transformation in the elements of the field W (f), f ∈ F. These facts imply Lemma 14.2B.

Now we formulate and prove Lemma 14.3A.

Lemma 14.3A. Let us consider a class of functions F satisfying the conditions of Proposition 13.2 with parameter k, and the random variables Ī_{n,k}^{V}(f), f ∈ F, V ⊂

{1, …, k}, defined in formula (14.1). Let B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k) denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)} , 1 ≤ j ≤ k, i.e. by the random sequences with second coordinate 1 in their upper index taking part in the definition of the random variables Ī_{n,k}^{V}(f). For all V ∈ {1, …, k}, V ≠ {1, …, k}, there exists a number A_{0} = A_{0}(k) > 0 such that the inequality

P (sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}) < n^{k−1}e^{−A^{1/(2k−1)}nσ^{2}/k}. (14.16)

holds for all A ≥ A_{0}.

Proof of Lemma 14.3A. Let us first consider the case V = ∅. In this case we can write E (Ī_{n,k}^{∅}(f)^{2}| B) = E (Ī_{n,k}^{∅}(f)^{2}) ≤ \frac{n^{k}}{k!} σ^{2} ≤ n^{2k}σ^{2k+2} for all f ∈ F. In the above calculation we have exploited that the functions f ∈ F are canonical, and this implies certain orthogonalities, and also the inequality nσ^{2} ≥ 1 holds. The above relation implies that for V = ∅ the probability at the left-hand side of (14.16) equals zero if the number A_{0} is chosen sufficiently large, i.e. the inequality (14.16) holds in this case.

To avoid some complications in the notation let us first restrict our attention to sets of the form V = {1, …, u} with some 1 ≤ u < k, and prove relation (14.16) for such sets. For this goal let us introduce the random variables

Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,u \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{u}}^{(u,1)}, ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)})

for all f ∈ F, i.e. we fix the last k − u coordinates ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)} of the random variable Ī_{n,k}^{V}(f) and sum up with respect the first u coordinates. Then we can write

\begin{aligned}E (Ī_{n,k}^{V}(f)^{2}| B) & = E ((  ∑_{\begin{matrix}1≤l_{j}≤n j=u+1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}))^{2}| B) \\ & = ∑_{\begin{matrix}1≤l_{j}≤n j=u+1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) . \end{aligned}  (14.17)

The last relation follows from the identity

E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})Ī_{n,k}^{V}(f, l′_{u+1}, …, l′_{k})| B) = 0

if (l_{u+1}, …, l_{k}) ≠ (l′_{u+1}, …, l′_{k}), which relation holds, since f is a canonical function. It follows from relation (14.17) that

{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}}

⊂ ⋃_{\begin{matrix}1≤l_{j}≤n j=u+1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} \{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) (ω) > \frac{A^{2}n^{2k}σ^{2k+2}}{2^{(3k+3)}n^{k−u}} \} .

(14.18)

The probability of the events in the union at the right-hand side of (14.18) can be estimated with the help of the corollary of Proposition 13.3 with parameter u < k instead of k. (We may assume that Proposition 13.3 holds for u < k.) We claim that this corollary yields that

P (sup_{f∈F} E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}| B) > \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} ) ≤ e^{−A^{1/(2u+1)}(n−u)σ^{2}}. (14.19)

Indeed, introduce the space (Y, Y, ρ) = (X^{k−u}, X^{k−u}, μ^{k−u}), the k − u-fold power of the measure space (X, X , μ), and for the sake of simpler notations write y = (x_{u+1}, …, x_{k}) for a point y ∈ Y. Let us consider a class of functions f ∈ F which satisfies the conditions of Proposition 13.2 and let us prove for it relation (14.16). Let us introduce for this goal the class of those function F̄ on the space (X^{u} × Y, X^{u} × Y, μ^{u} × ρ) which can be written in the form f̄(x_{1}, …, x_{u}, y) = f(x_{1}, …, x_{k}) with y = (x_{u+1}, …, x_{k}) and some function f(x_{1}, …, x_{k}) ∈ F. The class of functions F̄ satisfies the conditions of Proposition 13.3 with parameter u < k, hence we may apply by our inductive hypothesis the Corollary of Proposition 13.3 for this class of functions. We shall apply this Corollary for decoupled U-statistics with sample size n−u which is defined with the u independent sequences of independent μ-distributed random variables we define as ξ_{l}^{(j,1)}, 1 ≤ j ≤ u, l ∈ {1, …, n}\{l_{u+1}, …, l_{k}} where the set of numbers {l_{u+1}, …, l_{k}} is the set of indices appearing in formula (14.19). With such a choice we get that

\begin{aligned}P (sup_{f̄∈F̄}(n − u)^{−u}H_{n−u,u}(f̄) ≥ A^{2}(n − u)^{u}σ^{2u+2}) & ≤ e^{−A^{1/(2u+1)}(n−u)σ^{2}} \\ & for A > A_{0}(u), \end{aligned} (14.20)

where

H_{n−u,u}(f̄) = ∫ I_{n,u}(f̄, y)^{2}ρ( dy) = \frac{k!}{u!}E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k})^{2}|B)  (14.21)

with the function f ∈ F for which the identity f̄(x_{1}, …, x_{u}, y) = f(x_{1}, …, x_{k}) holds with y = (x_{u+1}, …, x_{k}).

It is not difficult to deduce formula (14.19) from relations (14.20) and (14.21). It is enough to replace the level \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} in the probability at the left-hand side of (14.19) by A^{2}(n − u)^{2u}σ^{2u+2} < \frac{A^{2}n^{k+u}σ^{2k+2}}{2^{(3k+3)}} . The last inequality really holds if the constant K in the condition nσ^{2} > K log n in Proposition 13.2 is chosen sufficiently large.

Relations (14.18) and (14.19) imply that

P (sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) > 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}) ≤ n^{k−u}e^{−A^{1/(2u+1)}(n−u)σ^{2}}.

Since e^{−A^{1/(2u+1)}(n−u)σ^{2}} ≤ e^{−A^{1/(2k−1)}nσ^{2}/k} if u ≤ k − 1 and n ≥ k inequality (14.16) holds for a set V of the form V = {1, …, u}, 1 ≤ u < k.

The case of a general set V ⊂ {1, …, k}, 1 ≤ |V | < k, can be handled similarly, only the notation becomes more complicated. Moreover, the case of general sets V can be reduced to the case of sets of form we have already considered. Indeed, given some set V ⊂ {1, …, k}, 1 ≤ |V | < k, let us define a new class of function F_{V} we get by applying a rearrangement of the indices of the arguments x_{1}, …, x_{k} of the functions f ∈ F in such a way that the arguments indexed by the set V are the first |V | arguments of the functions f_{V} ∈ F_{V}, and put V̄ = {1, …, |V |}. Then the class of functions F_{V} also satisfies the condition of Proposition 13.2, and we can get relation (14.16) with the set V by applying it for the set of function F_{V} and set V̄.

Now we prove Lemma 14.1A. It will be proved with the help of Lemma 14.2A, the generalized symmetrization lemma 13.1 and Lemma 14.3A.

Proof of Lemma 14.1A. We show with the help of the generalized symmetrization lemma, Lemma 13.1, and Lemma 14.3A that

\begin{aligned}P (sup_{f∈F} n^{−k/2} |Ī_{n,k}(f)| > An^{k/2}σ^{k+1}) & < 2P (sup_{f∈F} |S(f)| > \frac{A}{2}n^{k}σ^{k+1}) \\ & + 2^{k}n^{k−1}e^{−A^{1/(2k−1)}nσ^{2}/k} \end{aligned} (14.22)

with the function S(f) defined in (14.14). To prove relation (14.22) introduce the

random variables Z(f) = I_{n,k}^{{1,...,k}}(f) and Z̄(f) = − ∑_{V ⊂{1,...,k}, V ≠{1,...,k}}(−1)^{|V |}Ī_{n,k}^{V}(f)

for all f ∈ F, the σ-algebra B considered in Lemma 14.3A and the set

B = ⋂_{\begin{matrix}V ⊂\{1,...,k\} \\ V ≠\{1,...,k\} \end{matrix}} \{ω: sup_{f∈F} E (Ī_{n,k}^{V}(f)^{2}| B) (ω) ≤ 2^{−(3k+3)}A^{2}n^{2k}σ^{2k+2}\} .

Observe that S(f) = Z(f) − Z̄(f), f ∈ F, B ∈ B, and by Lemma 14.3A the inequality 1 − P (B) ≤ 2^{k}n^{k−1}e^{−A^{1/(2k−1)}nσ^{2}/k} holds. Hence to prove relation (14.22) it is enough to apply Lemma 13.1 and to show that

P (|Z̄(f)| > \frac{A}{2}n^{k}σ^{k+1}|B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.  (14.23)

But P (Ī_{n,k}^{|V |}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}|B) (ω) ≤ 2^{−(k+1)} for all functions f ∈ F and sets V ⊂ {1, …, k}, V ≠ {1, …, k}, if ω ∈ B by the ‘conditional Chebishev inequality’, hence relations (14.23) and (14.22) hold.

Lemma 14.1A follows from relation (14.22), Lemma 14.2A and the observation that the random vectors {Ī_{n,k}^{(V,ε)}(f)}, f ∈ F, defined in (14.13′) have the same distribution for all V ∈ {1, …, k} as the random vector Ī_{n,k}^{ε}(f), defined in formula (14.1). Hence

P (sup_{f∈F} |S(f)| > \frac{A}{2}n^{k}σ^{k+1}) ≤ 2^{k}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) .

Lemma 14.1A is proved.

Lemma 14.1B will be proved with the help of the following version Lemma 14.3B of Lemma 14.3A.

Lemma 14.3B. Let us consider a class of functions F satisfying the conditions of Proposition 13.3 and the random variables Ī_{n,k}^{V}(f, y), f ∈ F, V ⊂ {1, …, k}, defined in

formula (14.3). Let B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k) denote the σ-algebra generated by the random variables ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}, 1 ≤ j ≤ k, i.e. by the random variables with second argument 1 in their upper index taking part in the definition of the random variables Ī_{n,k}^{V}(f, y) and H_{n,k}^{V}(f) introduced in formulas (14.3) and (14.4).

a) For all V ∈ {1, …, k}, V ≠ {1, …, k}, there exists a number A_{0} = A_{0}(k) > 0 such that the inequality

P (sup_{f∈F} E(H_{n,k}^{V}(f)|B) > 2^{−(4k+4)}A^{(2k−1)/k}n^{2k}σ^{2k+2}) < n^{k−1}e^{−A^{1/2k}nσ^{2}/k}.

(14.24)

holds for all A ≥ A_{0}.

b) Given two subsets V_{1}, V_{1} ⊂ {1, …, k} of the set {1, …, k} define the integrals of random expressions

H_{n,k}^{(V_{1},V_{2})}(f) = ∫ |Ī_{n,k}^{V_{1}}(f, y)Ī_{n,k}^{V_{2}}(f, y)|ρ( dy), f ∈ F,  (14.25)

with the help of the functions Ī_{n,k}^{V}(f, y) defined in (14.3). If at least one of the sets V_{1} and V_{2} is not the set {1, …, k}, then there exists some number A_{0} = A_{0}(k) > 0 such that if the integrals H_{n,k}(f), f ∈ F, determined by this class of functions F have a good tail behaviour at level T^{(2k+1)/2k} for some T ≥ A_{0}, then the inequality

P (sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2}) < 2n^{k−1}e^{−A^{1/2k}nσ^{2}/k} (14.26)

holds for all A ≥ T.

Proof of Lemma 14.3B. Part a) of Lemma 14.3B can be proved in almost the same way as Lemma 14.3A. Hence I only briefly explain the main step of the proof. In the case V = ∅ E(H_{n,k}^{V}(f)|B) = E(H_{n,k}^{V}(f)), hence it is enough to show that E(H_{n,k}^{V}(f)) ≤

\frac{n^{k}σ^{2}}{k!} ≤ \frac{n^{2k}σ^{2k+2}}{k!} for all f ∈ F under the conditions of Proposition 13.3. (Here we exploit in particular that the functions of the class F are canonical.) The case of a general set V, V ≠ ∅ can be reduced to the case V = {1, …, u} with some 1 ≤ u < k.

Given a set V = {1, …, u} let us define the random variables

Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,u \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f (ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{u}}^{(u,1)}, ξ_{l_{u+1}}^{(u+1,−1)}, …, ξ_{l_{k}}^{(k,−1)}, y)

for all f ∈ F. We can show by exploiting the canonical property of the functions f ∈ F that

E (H̄_{n,k}^{V}(f)^{2}| B) = ∑_{\begin{matrix}1≤l_{j}≤n j=u+1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y)^{2}| B) ρ( dy),

and the proof of part a) of Lemma 14.3B can be reduced to the inequality

P (sup_{f∈F} ∫ E (Ī_{n,k}^{V}(f, l_{u+1}, …, l_{k}, y)^{2}ρ( dy)| B) > \frac{A^{(2k−1)/k}n^{k+u}σ^{2k+2}}{2^{(4k+4)}} )
≤ e^{−A^{(2k−1)/2(2u+1)k}(n−u)σ^{2}}.

This inequality can be proved, similarly to relation (14.19) in the proof of Lemma 14.3A with the help of the Corollary of Proposition 13.3. Only here we have to work in the space (X^{u} × Ȳ , X^{u} × Ȳ, μ^{u} × ρ̄) where Ȳ = X^{k−u} × Y, Ȳ = X^{k−u} × Y, ρ̄ = μ^{k−u} × ρ with the class of function F so that we identify a function f(x_{1}, …, x_{k}, y) ∈ F with f(x_{1}, …, x_{u}, ȳ) = f(x_{1}, …, x_{k}, y) so that ȳ = (x_{u+1}, …, x_{k}, y). I omit the details.

Part b) of Lemma 14.3B will be proved with the help of Part a) and the inequality

sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) ≤ (sup_{f∈F} E(H_{n,k}^{V_{1}}(f)|B))^{1/2} (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B))^{1/2}

which follows from the Schwarz inequality applied for integrals with respect to conditional distributions. Let us assume that V_{1} ≠ {1, …, k}. The last inequality implies that

P (sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) > 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2})

≤ P (sup_{f∈F} E(H_{n,k}^{V_{1}}(f)|B) > 2^{−(4k+4)}A^{(2k−1)/k}n^{2k}σ^{2k+2})

+ P (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2})

Hence the estimate (14.24) together with the inequality

P (sup_{f∈F} E(H_{n,k}^{V_{2}}(f)|B) > A^{(2k+1)/k}n^{2k}σ^{2k+2}) ≤ n^{k−1}e^{A^{1/2k}nσ^{2}}  (14.27)

imply relation (14.26). Relation 14.27 follows from Part a) of Lemma 14.3B if V_{2} ≠ {1, …, n} and A ≥ A_{0} with a sufficiently large number A_{0} (in this case the level A^{(2k+1)/k}n^{2k}σ^{2k+2} can be replaced by the larger number 2^{−(4k+2)}A^{(2k−1)/k}n^{2k}σ^{2k+2} in the probability of formula (14.27)) and from the conditions of Part b) of Lemma 14.3B if V_{2} = {1, …, k}. Indeed, in this case we may apply the estimate (13.5) for this probability, since A^{(2k+1)/2k} ≥ T^{(2k+1)/2k}, and this implies relation (14.27).

Now we turn to the proof of Lemma 14.1B.

Proof of Lemma 14.1B. By Lemma 14.2B it is enough to prove that relation (14.9) holds if the random variables W̄ (f) are replaced in it by the random variables W (f) defined in formula (14.15). We shall prove this by applying the generalized form of the symmetrization lemma, Lemma 13.1 with the choice of Z(f) = H_{n,k}^{(V̄ ,V̄)}(f), V̄ =

{1, …, k}, Z̄(f) = W (f) − Z(f), f ∈ F, B = B(ξ_{1}^{(j,1)}, …, ξ_{n}^{(j,1)}; 1 ≤ j ≤ k), and the set

B = ⋂_{\begin{matrix}(V_{1},V_{2}) : V_{j}∈\{1,...,k\}, j=1,2 \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}} \{ω: sup_{f∈F} E(H_{n,k}^{(V_{1},V_{2})}(f)|B)(ω) ≤ 2^{−(2k+2)}A^{2}n^{2k}σ^{2k+2}\} .

By Lemma 14.3B the inequality 1 − P (B) ≤ 2^{k+1}n^{k−1}e^{A^{1/2k}nσ^{2}/k} holds, and to prove Lemma 14.1B with the help of Lemma 13.1 it is enough to show that

P (|Z̄(f)| > \frac{A^{2}}{2} n^{2k}σ^{2(k+1)}| B) (ω) ≤ \frac{1}{2} for all f ∈ F if ω ∈ B.

To prove this relation observe that

E(|Z̄(f)||B) ≤ ∑_{\begin{matrix}(V_{1},V_{2}) : V_{j}∈\{1,...,k\}, j=1,2 \\ V_{1}≠\{1,...,k\} or V_{2}≠\{1,...,k\} \end{matrix}} E(H_{n,k}^{(V_{1},V_{2})}(f)|B) ≤ \frac{A^{2}}{4} n^{2k}σ^{2k+2} if ω ∈ B

for all f ∈ F. Hence the ‘conditional Markov inequality’ implies that

P (|Z̄(f)| > \frac{A^{2}}{2} n^{2k}σ^{2k+2}| B) ≤ \frac{1}{2} if ω ∈ B and f ∈ F.

Lemma 14.1B is proved.

15. The proof of the main result

In this section we prove Proposition 13.2 and thus complete the proof of the main result of this work, of Theorem 8.4 or of its equivalent version Theorem 8.2. Proposition 13.2 will be proved with the help of the symmetrization Lemma 14.1A. In the proof of this symmetrization lemma we have also applied the corollary of Proposition 13.3 (for orders u < k if we want to prove Proposition 13.2 for decoupled U-statistics of order k.) Hence to complete the proof of Proposition 13.2 we also have to prove Proposition 13.3. This section contains the proof of both results. First we prove Proposition 13.2.

A.) The proof of Proposition 13.2.

The proof of Theorem 13.2 is similar to the proof of Proposition 6.2. It applies an induction procedure with respect to the parameter k. In the proof of Proposition 13.2 for parameter k we may assume that Propositions 13.2 and 13.3 hold for u < k. In the proof we

want to give a good estimate on the probability P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1})

appearing in the estimate (14.2) of Lemma 14.1A. To estimate this probability we introduce (using the notation of Proposition 13.2) the functions

S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f^{2} (x_{l_{1}}^{(1)}, …, x_{l_{k}}^{(k)}) , f ∈ F,

(15.1)

with x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, 1 ≤ j ≤ k. Then we estimate the probability we are interested in with the help of this quantity similarly to the argument applied in the solution of the corresponding problem in the proof of Proposition 6.2.

Fix some number A > T and define the set H ⊂ X^{kn}

\begin{aligned}H = H(A) & = \{(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k): \\ & sup_{f∈F} S_{n,k}^{2}(f)(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2}\}. \end{aligned} (15.2)

We want to show that

P ({ω: (ξ_{l}^{(j)}(ω), 1 ≤ j ≤ n, 1 ≤ j ≤ k) ∈ H}) ≤ 2^{k}e^{−A^{2/3k}nσ^{2}} if A ≥ T.  (15.3)

Relation (15.3) will be proved by means of the Hoeffding decomposition (Theorem 9.1) of the U-statistics with kernel functions f^{2}(x_{1}, …, x_{k}), f ∈ F, and by the estimation of the sum this decomposition yields. More explicitly, write (applying formula (9.2) in Theorem 9.1)

f^{2}(x_{1}, …, x_{k}) = ∑_{V ⊂{1,...,k}} f_{V} (x_{j}, j ∈ V)  (15.4)

with f_{V} (x_{j}, j ∈ V) = ∏_{j∉V} P_{j} ∏_{j∈V} Q_{j}f^{2}(x_{1}, …, x_{k}), where P_{j} is the projection defined

in formula (9.1) and Q_{j} = I − P_{j} is also the same operator as the operator Q_{j} in formula (9.2).

The functions f_{V} appearing in formula (15.4) are canonical (with respect to the measure μ), and the identity S_{n,k}^{2}(f)(ξ_{l}^{(j)} 1 ≤ l ≤ n, 1 ≤ j ≤ k) = Ī_{n,k}(f^{2}) holds for all

f ∈ F. By applying the Hoeffding decomposition (15.4) for each term f^{2}(ξ_{l_{1}}^{(1)} …, ξ_{l_{k}}^{(k)}) in the expression I_{n,k}(f^{2}) we get that

\begin{matrix}P (sup_{f∈F} S_{n,k}^{2}(f)(ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) > 2^{k}A^{4/3}n^{k}σ^{2}) \\ ≤ ∑_{V ⊂\{1,...,k\}} P (\frac{|V |!}{k!} sup_{f∈F} n^{k−|V |}|Ī_{n,|V |}(f_{V})| > A^{4/3}n^{k}σ^{2}) \end{matrix}  (15.5)

with the functions f_{V} in (15.4). We want to give a good estimate for all terms in the sum at the right-hand side in (15.5). For this goal first we show that the classes of functions {f_{V} : f ∈ F} satisfy the conditions of Proposition 13.2 for all V ⊂ {1, …, k}.

The functions f_{V} are canonical for all V ⊂ {1, …, k}. It follows from the conditions of Proposition 13.2 that |f^{2}(x_{1}, …, x_{k})| ≤ 2^{−2(k+1)} and

∫ f^{4}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ 2^{−(k+1)}σ^{2}.

Hence relations (9.4) and (9.4′) of Theorem 9.2 imply that | sup_{x_{j}∈X,j∈V} f_{V} (x_{j}, j ∈ V)| ≤

2^{−(k+2)} ≤ 2^{−(k+1)} for all V ⊂ {1, …, k} and ∫ f_{V}^{2} (x_{j}, j ∈ V) ∏_{j∈V} μ( dx_{j}) ≤ 2^{−(k+1)}σ^{2} ≤

σ^{2} for all V ⊂ {1, …, k}. Finally, to check that the class of functions F_{V} = {f_{V} : f ∈ F} is L_{2}-dense with exponent L and parameter D observe that for all probability measures ρ on (X^{k}, X^{k}) and pairs of functions f, g ∈ F ∫ (f^{2} − g^{2})^{2} dρ ≤ 2^{−2k} ∫ (f − g)^{2} dρ. This implies that if {f_{1}, …, f_{m}}, m ≤ Dε^{−L}, is an ε-dense subset of F in the space L_{2}(X^{k}, X^{k}, ρ), then the set of functions {2^{k}f_{1}^{2}, …, 2^{k}f_{m}^{2}} is an ε-dense subset of the class of functions F′ = {2^{k}f^{2} : f ∈ F}, hence F′ is also an L_{2}-dense class of functions with exponent L and parameter D. Then by Theorem 9.2 the class of functions F_{V} is also L_{2}-dense with exponent L and parameter D for all sets V ⊂ {1, …, k}.

For V = ∅, the function f_{V} is constant, f_{V} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2} holds, and I_{|V |}(f_{|V |})| = f_{V} ≤ σ^{2}. Therefore the term corresponding to V = ∅ in the sum at the right-hand side of (13.5) equals zero if A_{0} ≥ 1 in the conditions of Proposition 13.2. I claim that the terms corresponding to sets V, 1 ≤ |V | ≤ k, in these sums satisfy the inequality

P (sup_{f∈F} |Ī_{n,|V |}(f_{V})| > A^{4/3}n^{|V |}σ^{2})

≤ P (sup_{f∈F} |Ī_{n,|V |}(f_{V})| > A^{4/3} \frac{k!}{|V |!}n^{|V |}σ^{|V |+1}) ≤ e^{−A^{2/3k}nσ^{2}} if 1 ≤ |V | ≤ k. (15.6)

The first inequality in (15.6) holds, since σ^{|V |+1} ≤ σ^{2} for |V | ≥ 1, the second inequality follows from the inductive hypothesis if |V | < k, since it yields the upper bound e^{−(A^{4/3}k!/|V |!)^{1/2|V |}nσ^{2}} ≤ e^{−A^{2/3k}nσ^{2}} if A_{0} = A_{0}(k) in Proposition 13.2 is sufficiently large, and in the case V = {1, …, k} it follows from the inequality A ≥ T and the assumption that U-statistics determined by a class of functions satisfying the conditions of Proposition 13.2 have a good tail behaviour at level T^{4/3}. Relations (15.5) and (15.6) together with the estimate in the case V = ∅ imply formula (15.3).

By conditioning the probability P (|Ī_{n,k}^{ε}(f)| > 2^{−(k+2)}An^{k/2}σ^{k+1}) with respect to

the random variables ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k we get with the help of the multivariate version of Hoeffding’s inequality (Theorem 12.3) that

P (|Ī_{n,k}^{ε}(f)| > 2^{−(k+2)}An^{k}σ^{k+1}| ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)

≤ C exp \{−B (\frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{2k+4}S_{n,k}^{2}(x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)/k!})^{1/k}\}  (15.7)

≤ Ce^{−2^{−3−4/k}BA^{2/3k}k!nσ^{2}} for all f ∈ F if (x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) ∉ H.

Given some points x_{l}^{(j)} ∈ X, 1 ≤ l ≤ n, 1 ≤ j ≤ k, define the probability measures ρ_{j} = ρ_{j, (x_{l}^{(j)}, 1≤l≤n)}, 1 ≤ j ≤ k, uniformly distributed on the set x_{l}^{(j)}, 1 ≤ l ≤ n, i.e.

let ρ_{j}(x_{l}^{(j)}) = \frac{1}{n}, 1 ≤ l ≤ n. Let us also define the product ρ = ρ_{1} × ⋯ × ρ_{k} of these measures. If f is a function on (X^{k}, X^{k}) such that ∫ f^{2} dρ ≤ δ^{2} with some δ > 0, then

|f(x_{l_{j}}^{(j)}, 1 ≤ j ≤ k)| ≤ δn^{k/2} for all vectors (l_{1}, …, l_{k}), 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k,

and this implies that P (|Ī_{n,k}^{ε}(f)| > δn^{3k/2}| ξ_{l}^{(j)} = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) = 0 for

such a function f. Take the numbers δ̄ = An^{−k/2}2^{−(k+2)}σ^{k+1} and δ = 2^{−(k+2)}n^{−k−1/2} ≤ δ̄. (The inequality δ ≤ δ̄ holds, since A ≥ A_{0} ≥ 1, and σ ≥ n^{−1/2}.) Choose a δ-dense set {f_{1}, …, f_{m}} in the L_{2}(X^{k}, X^{k}, ρ) space with m ≤ Dδ^{−L} ≤ 2^{(k+2)L}n^{β+(k+1)L/2} elements. Then the above estimates, the δ-dense property of the set of functions {f_{1}, …, f_{m}} in L_{2}(X^{k}, X^{k}, ρ) and formula (15.7) imply that

P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}| ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k)

≤ ∑_{s=1}^{m} P (|Ī_{n,k}^{ε}(f_{s})| > 2^{−(k+2)}An^{k}σ^{k+1}| ξ_{l}^{(j)}(ω) = x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (15.8)

≤ C2^{(k+2)L}n^{β+(k+1)L/2}e^{−2^{−3−4/k}BA^{2/3k}nk!σ^{2}} if {x_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k} ∉ H.

Relations (15.3) and (15.8) imply that

\begin{matrix}P (sup_{f∈F} |Ī_{n,k}^{ε}(f)| > 2^{−(k+1)}An^{k}σ^{k+1}) \\ ≤ C2^{(k+2)L}n^{β+(k+1)L/2}e^{−2^{−3−4/k}BA^{2/3k}nσ^{2}} + 2^{k}e^{−A^{2/3k}k!nσ^{2}} if A ≥ T. \end{matrix} (15.9)

Proposition 13.2 follows from the estimates (14.2) and (15.9) if the constant A_{0} together with the constant K in the condition nσ^{2} ≥ K(L + β) log n are chosen sufficiently large. In this case these estimates yield an upper bound less than e^{−A^{1/2k}nσ^{2}} for the probability at the left-hand side of (13.3).

Now we turn to the proof of Proposition 13.3.

B.) The proof of Proposition 13.3.

Because of formula (14.12) in the corollary of Lemma 14.1B to prove Proposition 13.3 i.e. inequality (13.5) it is enough to choose the parameter A_{0} in Proposition 13.3 for which A > T ≥ A_{0} sufficiently large and to show that

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)}) ≤ 2^{k+1}e^{−A^{1/2k}nσ^{2}}

for all G ∈ G and V_{1}, V_{2} ∈ {1, …, k} if A ≥ A_{0}

(15.10)

with the random variables H_{n,k}(f|G, V_{1}, V_{2}) defined in formula (14.10). Let us first prove formula (15.10) in the case when |e(G)| = k, i.e. when all vertices of the diagram G are end-points of some edge, and the expression H_{n,k}(f|G, V_{1}, V_{2}) contains no ‘symmetryzing term’ ε_{j}. In this case we apply a special argument to prove relation (15.10).

If G is such a diagram for which |e(G)| = k, then we can show by means of the Schwarz inequality that

\begin{aligned}|H_{n,k}(f|G, V_{1}, V_{2})| & ≤ \frac{1}{k!} (  ∑_{\begin{matrix}l_{1},...,l_{k}, 1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1),δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)ρ( dy))^{1/2} \\ & \frac{1}{k!} (  ∑_{\begin{matrix}l_{1},...,l_{k},1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ̄_{1})}, …, ξ_{l_{k}}^{(k,δ̄_{k})}, y)ρ( dy))^{1/2} , \end{aligned}

(15.11)

where δ_{j} = 1 if j ∈ V_{1}, δ_{j} = −1 if j ∉ V_{1}, and δ̄_{j} = 1 if j ∈ V_{2}, δ̄_{j} = −1 if j ∉ V_{2}. Indeed, in this case the sum of integrals in (14.10) can be rewritten in a natural way as the integral of the product of two functions on the product space (I_{n}^{k} × Y, I_{n}^{k} × Y, λ_{n}^{k} × ρ), where I_{n} = {1, …, n}, I_{n} is the σ-algebra of all subsets of I_{n}, and λ_{n} is the counting measure on I_{n}. Then the Schwarz inequality for this product yields formula (15.11). (Observe that the coordinates l_{1}, …, l_{k} determine the coordinates l′_{1}, …, l′_{k} in the summation (14.10) if |e(G)| = k.)

By formula (15.11)

\{ω: sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})(ω)| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)}\}

⊂ \{ω: sup_{f∈F} ∑_{\begin{matrix}l_{1},...,l_{k},1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}}∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1})}(ω), …, ξ_{l_{k}}^{(k,δ_{k})}(ω), y)ρ( dy) > \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}} \}

∪ \{ω: sup_{f∈F} ∑_{\begin{matrix}l_{1},...,l_{k},1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}}∫ f^{2}(ξ_{l_{1}}^{(1,δ̄_{1})}(ω), …, ξ_{l_{k}}^{(k,δ̄_{k})}(ω), y)ρ( dy) > \frac{A^{2}n^{2k}σ^{2(k+1)}k!}{2^{4k+1}} \} ,

and

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)})

≤ 2P (sup_{f∈F} \frac{1}{k!} ∑_{\begin{matrix}l_{1},...,l_{k},1≤l_{j}≤n, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} h_{f}(ξ_{l_{1}}^{(1,1)}, …, ξ_{l_{k}}^{(k,1)}) > \frac{A^{2}n^{2k}σ^{2(k+1)}}{2^{4k+1}} )

(15.12)

with the functions h_{f}(x_{1}, …, x_{k}) = ∫ f^{2}(x_{1}, …, x_{k}, y)ρ( dy), f ∈ F. (In this upper bound we could get rid of the terms δ_{j} and δ̄_{j}, i.e. on the dependence of the expression H_{n,k}(f|G, V_{1}, V_{2}) on the sets V_{1} and V_{2}, since the probability of the events in the previous formula do not depend on these terms.)

I claim that

P (sup_{f∈F} |Ī_{n,k}(h_{f})| ≥ An^{k}σ^{2}) ≤ 2^{k}e^{−A^{1/2k}nσ^{2}} for A ≥ A_{0}  (15.13)

if the constants A_{0} and K are chosen sufficiently large in Proposition 13.3. Relation (15.13) together with the relation A\frac{n^{2k}σ^{2(k+1)}}{2^{4k+1}} ≥ n^{k}σ^{2} imply that the probability at the right-hand side of (15.12) can be bounded by 2^{k+1}e^{−A^{1/2k}nσ^{2}}, and the estimate (15.10) holds in the case |e(G)| = k.

Relation (15.13) is similar to relation (15.5), and the proof of the latter formula helps to carry out the proof in the present case. Indeed, it follows from the conditions of Proposition 13.3 that 0 ≤ ∫ h_{f}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}, and it is not difficult to check thatsup |h_{f}(x_{1}, …, x_{k})| ≤ 2^{−2(k+1)}, and the class of functions H = {2^{k}h_{f}, f ∈ F} is an L_{2}-dense class with exponent L and parameter D. This means that by applying the Hoeffding decomposition of the functions h_{f}, f ∈ F, similarly to formula (15.4) we get such sets of functions (h_{f})_{V}, f ∈ F, for all V ⊂ {1, …, k} which satisfy the conditions of Proposition 13.2. Hence a natural adaptation of the estimate given for the expression at the right-hand side of (15.5) yields the proof of formula (15.13). We only have to replace S_{n,k}(f) by I_{n,k}(h_{f}), I_{n,|V |}(f_{V}) by I_{n,|V |}((h_{f})_{V}) and the levels 2^{k}A^{4/3}n^{k}σ^{2} and A^{4/3}n^{k}σ^{2} by An^{k}σ^{2} and 2^{−k}An^{k}σ^{2}. Let us observe that each term of the upper bound we get in such a way can be directly bounded, since our inductive hypothesis the result of Proposition 13.2 holds also for k.

In the case e(G) < k formula (15.10) will be proved with the help of the multivariate version of Hoeffding’s inequality, Theorem 12.3. In the proof of this case an expression, analogous to S_{n,k}^{2}(f) defined in formula (15.1) will be introduced and estimated for all sets V_{1}, V_{2} ⊂ {1, …, k} and diagrams G ∈ G such that |e(G)| < k. To define this expression first some notations will be introduced.

Let us consider the set J_{0}(G) = J_{0}(G, k, n),

J_{0}(G) = {(l_{1}, …, l_{k}, l′_{1}, …, l′_{k}): 1 ≤ l_{j}, l′_{j} ≤ n, 1 ≤ j ≤ k, l_{j} ≠ l_{j′} if j ≠ j′,
l′_{j} ≠ l′_{j′} if j ≠ j′, l_{j} = l′_{j′} if (j, j′) ∈ e(G), l_{j} ≠ l′_{j′} if (j, j′) ∉ e(G)},

the set of those sequences (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) which appear as indices in the summation in formula (14.10). Let us introduce a partition of J_{0}(G) appropriate for our purposes.

For this aim let us first define the sets M_{1} = M_{1}(G) = {j(1), …, j(k − |e(G)|)} = {1, …, k} \ v_{1}(G), j(1) < ⋯ < j(k − |e(G)|), and M_{2} = M_{2}(G) = {j̄(1), …, j̄(k − |e(G|)} = {1, …, k} \ v_{2}(G), j̄(1) < ⋯ < j̄(k − |e(G|), the sets of those vertices of the first and second row of the diagram G in increasing order from which no edges start. Let us also introduce the set V (G) = V (G, n, k),

V (G) = {(l_{j(1)}, …, l_{j(k−|e(G)|)}, l′_{j̄(1)}, …, l′_{j̄(k−|e(G)|)}): 1 ≤ l_{j(p)}, l′_{j̄(p)} ≤ n,

1 ≤ p ≤ k − |e(G)|, l_{j(p)} ≠ l_{j(p′)}, l′_{j̄(p)} ≠ l′_{j̄(p′)} if p ≠ p′, 1 ≤ p, p′ ≤ k − |e(G)|,

l_{j(p)} ≠ l′_{j̄(p′)}, 1 ≤ p, p′ ≤ k − |e(G)|}.

The set V (G) consists of such vectors which can be obtained as the restriction of some vector (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) ∈ J_{0}(G) to the coordinates indexed by the elements of the set M_{1}∪M_{2}. The elements of V (G) can be characterized as such vectors, whose coordinates indexed by the set M_{1} ∪ M_{2}, take different integer values between 1 and n. Given a vector v ∈ V (G) put v = (v_{1}, v_{2}), and let v_{1} = {v(r), 1 ≤ r ≤ k − |e(G)|}, and v_{2} = {v̄(r), 1 ≤ r ≤ k − |e(G)|}, denote the set of coordinates of v indexed by the elements of the set M_{1} and M_{2} respectively. For all vectors v ∈ V (G) define the set

E_{G}(v) = {(l_{1}, …, l_{k}, l′_{1}, …, l′_{k}): 1 ≤ l_{j} ≤ n, 1 ≤ l′ ; j̄ ≤ n, for 1 ≤ j, j̄ ≤ k,

l_{j} ≠ l_{j′} if j ≠ j′, l′ ; j̄ ≠ l′_{j̄′} if j̄ ≠ j̄′,

l_{j} = l′ ; j̄ if (j, j̄) ∈ e(G) and l_{j} ≠ l′ ; j̄ if (j, j̄) ∉ e(G),

l_{j(r)} = v(r), l′_{j̄(r)} = v̄(r), 1 ≤ r ≤ k − |e(G)|}, v ∈ V (G), where {j(1), …, j(k − |e(G)|)} = M_{1}, {j̄(1), …, j̄(k − |e(G)|)} = M_{2}, v = (v^{(1)}, v^{(2)}) with v^{(1)} = (v(1), …, v(k − |e(G)|)) and v^{(2)} = (v̄(1), …, v̄(k − |e(G)|)) in the last line of this definition. The elements ℓ = (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) of the set E_{G}(v) for some v ∈ V (G) can be characterized in the following way: For j ∈ M_{1} the coordinate l_{j} agrees with the corresponding element of v^{(1)}, for j̄ ∈ M_{2} the coordinate l′ ; j̄ agrees with the corresponding element of v^{(2)}. The indices of the remaining coordinates of ℓ can be partitioned into pairs (j_{s}, j̄_{s}), 1 ≤ s ≤ |e(G)| in such a way that (j_{s}, j̄_{s}) ∈ e(G). The identity l_{j_{s}} = l′ ; j̄_{s} holds for all these pairs, and these values l_{j_{s}} = l′ ; j̄_{s} must be different for different indices s. Otherwise, they can be chosen freely in the set {1, …, n}\{v^{(1)}, v^{(2)}}.

The sets E_{G}(v), v ∈ V (G), constitute a partition of the set J_{0}(G), and we can rewrite with their help the random variables H_{n,k}(f|G, V_{1}, V_{2}) defined in (14.10) as

\begin{matrix}H_{n,k}(f|G, V_{1}, V_{2}) = ∑_{v=(l_{j(1)},...,l_{j(k−|e(G)|)},l′_{j̄(1)},...,l′_{j̄(k−|e(G)|})∈V (G)} ∏_{s=1}^{k−|e(G)|} ε_{l_{j(s)}} ∏_{s=1}^{k−|e(G)|} ε_{l′_{j̄(s)}} \\ ∑_{(l_{1},...,l_{k},l′_{1}...,l′_{k})∈E_{G}(v)} \frac{1}{k!^{2}} ∫ f(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)f(ξ_{l′_{1}}^{(1,δ̄_{1})}, …, ξ_{l′_{k}}^{(k,δ̄_{k})}, y)ρ( dy) \end{matrix}

(15.14)

where δ_{j} = 1 if j ∈ V_{1}, δ_{j} = −1 if j ∉ V_{1}, and δ̄_{j} = 1 if j ∈ V_{2}, δ̄_{j} = −1 if j ∉ V_{2}. The inequality

P (S^{2}(F|G, V_{1}, V_{2}) > A^{8/3}n^{2k}σ^{4}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0} and e(G) < k

(15.15)

will be proved for the random variable

S^{2}(F|G, V_{1}, V_{2}) = sup_{f∈F} \frac{1}{k!^{2}} ∑_{v∈V (G)}(  ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)} ∫ f(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)

f(ξ_{l′_{1}}^{(1,δ̄_{1})}, …, ξ_{l′_{k}}^{(k,δ̄_{k})}, y)ρ( dy))^{2},  (15.15′)

where δ_{j} = 1 if j ∈ V_{1}, δ_{j} = −1 if j ∉ V_{1}, and δ̄_{j} = 1 if j ∈ V_{2}, δ̄_{j} = −1 if j ∉ V_{2}. The random variable S^{2}(F|G, V_{1}, V_{2}) defined in (15.15′) plays a similar role in the proof of Proposition 13.3 as the random variable sup_{f∈F} S_{n,k}^{2}(f) in the proof of Proposition 13.2,

where S_{n,k}^{2}(f) was defined in formula (15.1).

To prove formula (15.15) let us first fix some v ∈ V (G) and let us observe that, similarly to the proof of relation (15.11), the Schwarz inequality implies the relation

(  ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)} ∫ f(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)f(ξ_{l′_{1}}^{(1,δ̄_{1})}, …, ξ_{l′_{k}}^{(k,δ̄_{k})}, y)ρ( dy))^{2}

≤ ( ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)ρ( dy))

( ∑_{(l̄_{1},...,l̄_{k},l̄′_{1},...,l̄′_{k})∈E_{G}(v)} ∫ f^{2}(ξ(1,δ̄_{1}) ; l̄′_{1}, …, ξ(k,δ̄_{k}) ; l̄′_{k}, y)ρ( dy))

for all v ∈ V (G). Summing up these inequalities for all v ∈ V (G) we get that

S^{2}(F|G, V_{1}, V_{2}) ≤ sup_{f∈F} ∑_{v∈V (G)} \frac{1}{k!} (  ∑_{(l_{1},...,l_{k},l′_{1},...,l′_{k})∈E_{G}(v)}∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)ρ( dy))

\frac{1}{k!} (  ∑_{(l̄_{1},...,l̄_{k},l̄′_{1},...,l̄′_{k})∈E_{G}(v)} ∫ f^{2}(ξ(1,δ̄_{1}) ; l̄′_{1}, …, ξ(k,δ̄_{k}) ; l̄′_{k}, y)ρ( dy)) (15.16)

≤ sup_{f∈F} \frac{1}{k!} ( ∑_{\begin{matrix}(l_{1},...,l_{k}), 1≤l_{j}≤n, 1≤j≤k, \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)ρ( dy))

sup_{f∈F} \frac{1}{k!} (  ∑_{\begin{matrix}(l̄_{1},...,l̄_{k}), 1≤l̄_{j}≤n, 1≤j≤k, \\ l̄_{j}≠l̄_{j′} if j≠j′ \end{matrix}} ∫ f^{2}(ξ(1,δ̄_{1}) ; l̄_{1}, …, ξ(k,δ̄_{k}) ; l̄_{k}, y)ρ( dy))

To check the second inequality of formula (15.16) let us first observe that it can be reduced to the simpler relation, where the expression sup_{f∈F} is omitted at each place. The

simplified inequality we get after the omission of the expressions sup can be checked by carrying out the term by term multiplication between the products of sums appearing in (15.16). We get at both sides of the inequality sums consisting of terms of the form

\frac{1}{k!^{2}} ∫ f^{2}(ξ_{l_{1}}^{(1,δ_{1})}, …, ξ_{l_{k}}^{(k,δ_{k})}, y)ρ( dy) ∫ f^{2}(ξ(1,δ̄_{1}) ; l̄_{1}, …, ξ(k,δ̄_{k}) ; l̄_{k}, y)ρ( dy)  (15.17)

and we have to check that if a term of this form appears in the middle term of the simplified version formula of (15.16), then it appears with coefficient 1, and it also appears at the right-hand side of this formula. To see this observe that each term of the form (15.17) which appears in the middle term determines uniquely the index v = (v_{1}, v_{2}) ∈ V (G) in the outer sum in the middle term for which the product of the inner sums yields this term. Indeed, the coordinates of this vector v = (v_{1}, v_{2}) (which depends only on the indices in M_{1} ∪ M_{2}) is such that v_{1} agrees with the coordinates of the vector l = (l_{1}, …, l_{k}) at indices in M_{1} and v_{2} agrees with the coordinates of (l̄_{1}, …, l̄_{k}) at indices in M_{2}. Besides, all terms of the form (15.17) which appear at the left-hand side also appear at the right-hand of this expression.

Relation (15.16) implies that

P (S^{2}(F|G, V_{1}, V_{2})) > A^{8/3}n^{2k}σ^{4}) ≤ 2P (sup_{f∈F} Ī_{n,k}(h_{f}) > A^{4/3}n^{k}σ^{2})

with h_{f}(x_{1}, …, x_{k}) = ∫ f^{2}(x_{1}, …, x_{k}, y)ρ( dy). (Here we exploited that in the last formula S^{2}(F|G, V_{1}, V_{2}) is bounded by the product of two random variables whose distributions do not depend on the sets V_{1} and V_{2}.) Thus to prove inequality (15.15) it is enough to show that

2P (sup_{f∈F} Ī_{n,k}(h_{f}) > A^{4/3}n^{k}σ^{2}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}} if A ≥ A_{0}.  (15.18)

Actually formula (15.18) follows from the already proven formula (15.13), only the parameter A has to be replaced by A^{4/3} in it.

With the help of relation (15.15) the proof of Proposition 13.3 can be completed similarly to that of Proposition 13.2. It follows from the generalized version of Hoeffding’s inequality Theorem 12.3 and the definition of the random variable H_{n,k}(f|G, V_{1}, V_{2}) given in the form (15.14) that

P (|H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+2}k!}n^{2k}σ^{2(k+1)}| ξ_{l}^{j,±1}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ Ce^{−B2^{−(4+2/k)}A^{2/3k}nσ^{2}} if S^{2}(F|G, V_{1}, V_{2})(ω) ≤ A^{8/3}n^{2k}σ^{4} for all f ∈ F,

and G ∈ G such that |e(G)| < k, and V_{1}, V_{2} ∈ {1, …, k} if A ≥ A_{0}.

(15.19)

Indeed, in this case the conditional probability considered in (15.19) can be bounded by

C exp \{−B (\frac{A^{4}n^{4k}σ^{4(k+1)}}{2^{8k+4}(k!)^{2}S^{2}(F|G,V_{1},V_{2})/(k!)^{2}} )^{1/2j}\} ≤ C exp \{−B (\frac{A^{4/3}n^{2k}σ^{4k}}{2^{8k+4}} )^{1/2j}\}, where

2j = 2k − 2|e(G)|, the number of vertices of the diagram G from which no edges start. Since j ≤ k, nσ^{2} ≥ 1, and also \frac{A^{4/3}}{2^{8k+4}} ≥ 1 if A_{0} is chosen sufficiently large the above calculation implies relation (15.19).

Let us show that also the inequality

P ( sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)}| ξ_{l}^{j,±1}, 1 ≤ l ≤ n, 1 ≤ j ≤ k) (ω)

≤ Cn^{(3k+1)L/2+β}e^{−BA^{2/3k}nσ^{2}/2^{(4+2/k)}(k!)^{1/k}} if S^{2}(F|G, V_{1}, V_{2}))(ω) ≤ A^{8/3}n^{2k}σ^{4}

for all G ∈ G such that |e(G)| < k, and V_{1}, V_{2} ∈ {1, …, k} if A ≥ A_{0}

(15.20)

holds.

To prove formula (15.20) let us fix an elementary event ω ∈ Ω which satisfies the relation S^{2}(F|G, V_{1}, V_{2})(ω) ≤ A^{8/3}n^{2k}σ^{4}, two sets V_{1}, V_{2} ⊂ {1, …, k}, and a diagram G such that |e(G)| < k, consider the points x_{l}^{(j,±1)} = x_{l}^{(j,±1)}(ω) = ξ_{l}^{(j,±1)}(ω), 1 ≤ l ≤ n, 1 ≤ j ≤ k, and introduce with their help the following probability measures: For all 1 ≤ j ≤ k define the probability measures ν_{j}^{(1)} which are uniformly distributed on

the points x_{l}^{(j,δ_{j})}, 1 ≤ l ≤ n, and ν_{j}^{(2)} which are uniformly distributed on the points

x_{l}^{(j,δ̄_{j})}, 1 ≤ l ≤ n, i.e. let ν_{j}^{(1)} (\{x_{l}^{(j,δ_{j})}\}) = \frac{1}{n} and ν_{j}^{(2)} (\{x_{l}^{(j,δ̄_{j})\}}) = \frac{1}{n}, 1 ≤ l ≤ n,

1 ≤ j ≤ k, where δ_{j} = 1 if j ∈ V_{1}, δ_{j} = −1 if j ∉ V_{1}, and similarly δ̄_{j} = 1 if j ∈ V_{2} and δ̄_{j} = −1 if j ∉ V_{2}. Let us consider the product measures α_{1} = ν_{1}^{(1)} × ⋯ × ν_{k}^{(1)} × ρ, α_{2} = ν_{1}^{(2)} × ⋯ × ν_{k}^{(2)} × ρ on the product space (X^{k} × Y, X^{k} × Y), where ρ is that probability measure on (Y, Y) which appears in Proposition 13.3, and define the measure α = \frac{α_{1}+α_{2}}{2} . Given two functions f ∈ F and g ∈ F we give an upper bound for |H_{n,k}(f|G, V_{1}, V_{2})(ω) − H_{n,k}(g|G, V_{1}, V_{2})(ω)| if ∫ (f − g)^{2} dα ≤ δ^{2} with some δ > 0. (This bound does not depend on the ‘randomizing terms’ ε_{l}(ω) in the definition of the random variable H_{n,k}(·|G, V_{1}, V_{2}).)

In this case ∫ (f − g)^{2} dα_{j} ≤ 2δ^{2}, and

∫ |f(x_{l_{1}}^{(1,δ_{1})}, …, x_{l_{k}}^{(k,δ_{k})}, y) − g(x_{l_{1}}^{(1,δ_{1})}, …, x_{l_{k}}^{(k,δ_{k})}, y)|^{2}ρ( dy) ≤ 2δ^{2}n^{k},
∫ |f(x_{l_{1}}^{(1,δ_{1})}, …, x_{l_{k}}^{(k,δ_{k})}, y) − g(x_{l_{1}}^{(1,δ_{1})}, …, x_{l_{k}}^{(k,δ_{k})}, y)|ρ( dy) ≤ \sqrt{2}δn^{k/2}

for all 1 ≤ l ≤ k, and 1 ≤ l_{j} ≤ n, and the same result holds if all δ_{j} is replaced by δ̄_{j}, 1 ≤ j ≤ k. Since |f| ≤ 1, |g| ≤ 1 if f, g ∈ F, the condition ∫ (f − g)^{2} dα ≤ δ^{2} implies that

∫ |f(ξ_{l_{1}}^{(1,δ_{1})}(ω), …, ξ_{l_{k}}^{(k,δ_{k})}(ω), y)f(ξ_{l′_{1}}^{(1,δ̄_{1})}(ω), …, ξ_{l′_{k}}^{(k,δ̄_{k})}(ω), y)ρ( dy)

− g(ξ_{l_{1}}^{(1,δ_{1})}(ω), …, ξ_{l_{k}}^{(k,δ_{k})}(ω), y)g(ξ_{l′_{1}}^{(1,δ̄_{1})}(ω), …, ξ_{l′_{k}}^{(k,δ̄_{k})}(ω), y)ρ( dy)| ≤ 2\sqrt{2}δn^{k/2}

for all vectors (l_{1}, …, l_{k}, l′_{1}, …, l′_{k}) which appear as an index in the summation in (14.10). Hence

|H_{n,k}(f|G, V_{1}, V_{2})(ω) − H_{n,k}(g|G, V_{1}, V_{2})(ω)| ≤ 2\sqrt{2}δn^{5k/2}

if f, g ∈ F, ∫ (f −g)^{2} dα < δ^{2} and the originally fixed ω ∈ Ω is considered. (The measure α is defined by means of this ω.)

Put δ̄ = \frac{A^{2}n^{−k/2}σ^{2(k+1)}}{2^{(4k+7/2)}k!} , and δ = n^{−(3k+1)/2} ≤ δ̄ ( the inequality δ ≤ δ̄ holds, since σ ≥ \frac{1}{\sqrt{n}} and we may assume that A ≥ A_{0} is sufficiently large), choose a δ-dense subset {f_{1}, …, f_{m}} ⊂ F in the L_{2}(X^{k} × Y, X^{k} × Y, α) space with m ≤ Dδ^{−L} ≤ n^{(3k+1)L/2+β} elements. Relation (15.19) for these functions together with the above estimates yield formula (15.20).

It follows from relations (15.15) and (15.20) that

P (sup_{f∈F} |H_{n,k}(f|G, V_{1}, V_{2})| > \frac{A^{2}}{2^{4k+1}k!}n^{2k}σ^{2(k+1)}) ≤ 2^{k+1}e^{−A^{2/3k}nσ^{2}}

+ Cn^{(3k+1)L/2+β}e^{−B2^{−(4+2/k)}A^{2/3k}nσ^{2}} if A ≥ A_{0}

for all V_{1}, V_{2} ⊂ {1, …, k} also in the case |e(G)| ≤ k − 1. This inequality implies that relation (15.10) holds also in this case if the constants A_{0} and K are chosen sufficiently large in Proposition 13.3. Proposition 13.3 is proved.

16. The improvement of some results in Section 8

In this section I present an improvement of Theorems 8.3 and 8.5. I shall explain the picture behind these results and the some ideas of the proofs. But the detailed proofs which are based on some results called the diagram formulas for the products of multiple Wiener–Itô integrals and degenerate U-statistics are omitted. These diagram formulas present an identity which enables us to express the product of Wiener–Itô integrals or degenerate U-statistics as a sum of such objects. I omitted the proofs because they heavily depend on the diagram formula, a technique not discussed in this work. The interested reader can find the detailed proofs in my papers [21] and [22].

The main result discussed in this section is the following

Theorem 16.1. Let ξ_{1}, …, ξ_{n} be a sequence of iid. random variables on a space (X, X) with some distribution μ. Let us consider a function f(x_{1}, …, x_{k}) canonical with respect to the measure μ on the space (X^{k}, X^{k}) which satisfies conditions (8.1) and (8.2) with some 0 < σ^{2} ≤ 1 together with the degenerate U-statistic I_{n,k}(f) with this kernel function. There exist some constants A = A(k) > 0 and B = B(k) > 0 depending only on the order k of the U-statistic I_{n,k}(f) such that

P (k!n^{−k/2}|I_{n,k}(f)| > u) ≤ A exp \{−\frac{u^{2/k}}{2σ^{2/k} (1 + B (un^{−k/2}σ^{−(k+1)})^{1/k})}\} (16.1)

for all 0 ≤ u ≤ n^{k/2}σ^{k+1}.

Theorem 16.1 states in particular that if 0 < u ≤ εn^{k/2}σ^{k+1} with a sufficiently small ε > 0, then P (k!n^{−k/2}|I_{n,k}(f)| > u) ≤ A exp \{−\frac{1−Cε^{1/k}}{2} (\frac{u}{σ})^{2/k}\} with some universal constants A > 0 and C > 0 depending only on the order k of the U-statistic I_{n,k}(f). This result is very similar to Theorem 8.3. Both theorems yield an estimate on the probability P (k!n^{−k/2}|I_{n,k}(f)| > u) for 0 ≤ u ≤ n^{k/2}σ^{k+1}, but in the present result we also get a good estimate on the constant α in formula (8.9) for 0 ≤ u ≤ εn^{k/2}σ^{k+1}. At first sight this additional result does not seem an essential improvement, but actually it expresses an important property of the estimate (16.1). To understand this it is worth while to compare Theorem 16.1 with Bernstein’s inequality formulated in Theorem 3.1.

Theorem 3.1 implies the estimate

P (n^{−1/2}|I_{n,1}(f)| > u) ≤ 2e^{−Cu^{2}/σ^{2}} if 0 ≤ u ≤ nσ^{2}  (16.2)

for the degenerate U-statistic I_{n,1}(f) of order 1 with a kernel function f, (i.e. for a sum of iid. random variables Ef(ξ_{1}) = 0) if the relations sup |f(x)| ≤ 1 and Ef(ξ_{j}) = 0 and Ef^{2}(ξ_{j}) ≤ σ^{2} hold. Besides, relation (16.2) also holds with a constant of the form C = \frac{1−O(ε)}{2} if 0 ≤ u ≤ εnσ^{2}. On the other hand, Example 3.2 shows an example (formulated with a different normalization) with a function f and a sequence of iid. random variables ξ_{1}, ξ_{2}, … satisfying the above conditions such that

P (n^{−1/2}I_{n,1}(f) > u) ≥ A exp \{−B (\frac{u}{σ} )^{2} · \frac{\sqrt{n}σ^{2}}{u} log \frac{u}{\sqrt{n}σ^{2}} \}

if u ≫ nσ^{2}. This means that in the special case k = 1 the probability P (n^{−1/2}|I_{n,1}(f)| > u) has a Gaussian type estimate for 0 ≤ u ≤ const. nσ^{2}, and such an estimate does not hold for u ≫ nσ^{2}. Besides, in the smaller interval 0 ≤ u ≤ εnσ^{2} we can say more. In this case the relation (16.2) holds with such a constant C which almost agrees with the number \frac{1}{2}, i.e. the upper bound we get for k = 1 almost agrees with the quantity suggested by a formal application of the central limit theorem.

I want to explain that Theorem 16.1 states a similar result for degenerate Ustatistics of any order k ≥ 1. To understand this let us first recall that a sequence of normalized degenerate U-statics n^{−k/2}I_{n,k}(f), n = 1, 2, …, defined with the help of a sequence of iid. random variables ξ_{1}, ξ_{2}, … taking values on some measurable space (X, X) with distribution μ and a function f(x_{1}, …, x_{k}) of k variables canonical with respect to μ and such that

σ^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞

has a limit distribution as n → ∞. Moreover, this limit can be expressed explicitly as the distribution of the Wiener–Itô integral

Z_{μ,k}(f) = \frac{1}{k!} ∫ f(x_{1}, …, x_{k})μ_{W} ( dx_{1}) … μ_{W} (dx_{k}),  (16.3)

where μ_{W} is the white noise with counting measure μ, i.e. μ_{W} (A), A ∈ X, is a Gaussian field indexed by the measurable subsets of the space X such that Eμ_{W} (A) = 0 and Eμ_{W} (A)μ_{W} (B) = μ(A ∩ B) for all A, B ∈ X. (The definition of Wiener–Itô integrals can be found e.g. in [17].) Hence it is natural to expect that in the estimates about the distribution of degenerate U-statistics the distributions of Wiener–Itô integrals play a role similar to the Gaussian distributions in the case k = 1. Therefore we are interested in good estimates on the distribution of Wiener–Itô integrals. The next result supplies such an estimate. As Theorem 16.1 was an improvement of Theorem 8.3, the next result is an improvement of the first estimate in Theorem 8.5 presented in formula (8.11).

Theorem 16.2. Let us consider a σ-finite measure μ on a measurable space together with a white noise μ_{W} with counting measure μ. Let us have a real-valued function f(x_{1}, …, x_{k}) on the space (X^{k}, X^{k}) which satisfies relation (8.2). Take the random integral Z_{μ,k}(f) introduced in formula (16.3). This random integral satisfies the inequality

P (k!|Z_{μ,k}(f)| > u) ≤ C exp \{−\frac{1}{2} (\frac{u}{σ} )^{2/k}\} for all u > 0  (16.4)

with an appropriate constant C = C(k) > 0 depending only on the multiplicity k of the integral.

In Theorem 16.2 we gave only an upper bound for the distribution of Wiener– Itô integrals. The following example shows that there are cases when this estimate is essentially sharp.

Example 16.3. Let us have a σ-finite measure μ on some measure space (X, X) together with a white noise μ_{W} on (X, X) with counting measure μ. Let f_{0}(x) be a real valued function on (X, X) such that ∫ f_{0}(x)^{2}μ( dx) = 1, and take the function f(x_{1}, …, x_{k}) = σf_{0}(x_{1}) ⋯ f_{0}(x_{k}) with some number σ > 0 and the Wiener–Itô integral Z_{μ,k}(f) introduced in formula (16.3).

Then the relation ∫ f(x_{1}, …, x_{k})^{2} μ( dx_{1}) … μ( dx_{k}) = σ^{2} holds, and the random integral Z_{μ,k}(f) satisfies the inequality

P (k!|Z_{μ,k}(f)| > u) ≥ \frac{C̄}{(\frac{u}{σ})^{1/k} + 1} exp \{−\frac{1}{2} (\frac{u}{σ} )^{2/k}\} for all u > 0  (16.5)

with some constant C̄ > 0.

Proof of the statement of Example 16.3. We may restrict our attention to the case k ≥ 2. Itô’s formula (see e.g. [17]) states that the random variable k!Z̄_{μ,k}(f) can be expressed as k!Z_{μ,k}(f) = σH_{k} (∫ f_{0}(x)μ_{W} ( dx)) = σH_{k}(η), where H_{k}(x) is the k-th Hermite polynomial with leading coefficient 1, and η = ∫ f_{0}(x)μ_{W} ( dx) is a standard normal random variable. Hence we get by exploiting that the coefficient of x^{k−1} in the polynomial H_{k}(x) is zero that P (k!|Z_{μ,k}(f)| > u) = P (|H_{k}(η)| ≥ \frac{u}{σ}) ≥ P (|η^{k}| − D|η^{k−2}| > \frac{u}{σ}) with a sufficiently large constant D > 0 if \frac{u}{σ} > 1. There exist such positive constants A

and B that P (|η^{k}| − D|η^{k−2}| > \frac{u}{σ}) ≥ P (|η^{k}| > \frac{u}{σ} + A (\frac{u}{σ})^{(k−2)/k}) if \frac{u}{σ} > B.

Hence

P (k!|Z_{μ,k}(f)| > u) ≥ P (|η| > (\frac{u}{σ} )^{1/k} (1 + A (\frac{u}{σ} )^{−2/k})) ≥ \frac{C̄ exp \{−\frac{1}{2} (\frac{u}{σ})^{2/k}\}}{(\frac{u}{σ})^{1/k} + 1}

with an appropriate C̄ > 0 if \frac{u}{σ} > B. Since P (k!|Z_{μ,k}(f)| > 0) > 0, the above inequality also holds for 0 ≤ \frac{u}{σ} ≤ B if the constant C̄ > 0 is chosen sufficiently small. This means that relation (16.5) holds.

Let us remark that if f(x_{1}, …, x_{k}) = σf_{0}(x_{1}) … f_{0}(x_{k}) is a function on the space (X^{k}, X^{k}) such that ∫ f_{0}(x)μ( dx) = 0, ∫ f_{0}^{2}(x)μ( dx) = 1, sup |f_{0}(x)| ≤ 1, 0 < σ ≤ 1, and we have a sequence of iid. random variables, ξ_{1}, ξ_{2}, … with distribution μ, then the U-statistics I_{n,k}(f), n = 1, 2, …, are degenerate, and they satisfy the conditions of Theorem 16.1. Besides, they converge in distribution to the Wiener–Itô integral Z_{μ,k}(f) as n → ∞ which satisfies the conditions of example (16.3). Hence the U-statistics I_{n,k}(f) satisfy relation (16.1), and also the inequality

lim_{n→∞} P (k!n^{−k/2}|I_{n,k}(f)| > u) ≥ \frac{C̄ exp \{−\frac{1}{2} (\frac{u}{σ})^{2/k}\}}{(\frac{u}{σ})^{1/k} + 1}

holds with an appropriate C̄ > 0 if \frac{u}{σ} > B. This means that for not too large values of u, more explicitly if u ≤ εn^{k/2}σ^{k+1} with a small number ε > 0, the estimate given in Theorem 16.1 is essentially sharp. Let me also remark that Example 8.6 shows such a degenerate U-statistic of order k = 2 for which an estimate similar to that of Theorem 8.3 cannot hold for n ≫ n^{k/2}σ^{k+1}. We have presented such an example only for k = 2, but similar examples can be given for all k ≥ 1.

This means that Theorem 16.1 shows a similar picture about the distribution of degenerate U-statistics of order k for all k ≥ 1 as Bernstein’s inequality shows in the case k = 1. We have a good estimate on the distribution P (n^{−k/2}I_{n,k}(f) > u) of a degenerate U-statistic with a kernel function f satisfying relations (8.1) and (8.2) in the domain 0 ≤ u ≤ n^{k/2}σ^{k+1}. Such an estimate is already proved in Theorem 8.3, but Theorem 16.1 says more in an interval of the form 0 ≤ u ≤ εn^{k/2}σ^{k+1} with a small ε > 0. The limit theorems about degenerate U-statistics give an upper bound for the coefficient α in the exponent of formula (8.9) in Theorem 8.3, and Theorem 16.1 states that the estimate (8.9) holds with an almost as good coefficient α in the interval 0 ≤ u ≤ εn^{k/2}σ^{k+1} as this upper bound suggests.

The proof of the above results are based, similarly to the proof of Theorems 8.3 and 8.5, on some good estimates on high moments of degenerate U-statistics I_{n,k}(f) and of Wiener–Itô integrals Z_{n,k}(f). The result of Theorem 16.2 follows from the following

Proposition 16.4. Let the conditions of Theorem 16.2 be satisfied for a multiple Wiener–Itô integral Z_{μ,k}(f) of order k. Then, with the notations of Theorem 16.2, the inequality

E (k!|Z_{μ,k}(f)|)^{2M} ≤ 1 · 3 · 5 ⋯ (2kM − 1)σ^{2M} for all M = 1, 2, …  (16.6)

holds.

By the Stirling formula Proposition 16.4 implies that

E(k!|Z_{μ,k}(f)|)^{2M} ≤ \frac{(2kM)!}{2^{kM}(kM)!}σ^{2M} ≤ A (\frac{2}{e})^{kM} (kM)^{kM}σ^{2M}  (16.7)

for all numbers A > \sqrt{2} if M ≥ M_{0} = M_{0}(A). The following Proposition 16.5 states a similar, but weaker inequality for the moments of normalized degenerate U-statistics.

Proposition 16.5. Let us consider a degenerate U-statistic I_{n,k}(f) of order k with sample size n and with a kernel function f satisfying relations (8.1) and (8.2) with some 0 < σ^{2} ≤ 1. Fix a positive number η > 0. There exists some universal constants A = A(k) > \sqrt{2}, C = C(k) > 0 and M_{0} = M_{0}(k) ≥ 1 depending only on the order of the U-statistic I_{n,k}(f) such that

E (n^{−k/2}k!I_{n,k}(f))^{2M} ≤ A (1 + C\sqrt{η})^{2kM} (\frac{2}{e})^{kM} (kM)^{kM} σ^{2M}
for all integers M such that kM_{0} ≤ kM ≤ ηnσ^{2}.

(16.8)

The constant C = C(k) in formula (2.3) can be chosen e.g. as C = 2\sqrt{2} which does not depend on the order k of the U-statistic I_{n,k}(f).

Let us remark that formula (16.6) can be reformulated as E(k!|Z_{μ,k}(f)|)^{2M} ≤ E(ση^{k})^{2M}, where η is a standard normal random variable. Theorem 16.2 states that the tail distribution of k!|Z_{μ,k}(f)| satisfies an estimate similar to that of σ|η|^{k}. This follows simply from Proposition 16.4 and the Markov inequality P (k!|Z_{μ,k}(f)| > u) ≤ \frac{E(k!|Z_{μ,k}(f)|)^{2M}}{u^{2M}} with an appropriate choice of the parameter M.

Proposition 16.5 states that in the case M_{0} ≤ M ≤ εnσ^{2} the inequality

E (n^{−k/2}k!I_{n,k}(f))^{2M} ≤ E((1 + β(ε))ση^{k})^{2M}

holds with a standard normal random variable η and a function β(ε), 0 ≤ ε ≤ 1, such that β(ε) → 0 if ε → 0, and β(ε) ≤ C with some universal constant C = C(k) if 0 ≤ ε ≤ 1. This means that certain high but not too high moments of n^{−k/2}k!I_{n,k}(f) behave similarly to the moments of k!Z_{μ,k}(f). As a consequence, we can prove a similar, but slightly weaker estimate for the distribution of n^{−k/2}k!I_{n,k}(f) as for the distribution of k!Z_{μ,k}(f). Actually this is done in the proof of Theorem 16.1.

Estimate (16.8) is very similar to the bound (10.1) formulated in Proposition (10.1). The main difference is that here we get the estimate

E (n^{−k/2}k!I_{n,k}(f))^{2M} ≤ C^{M}(kM)^{kM}σ^{2M}  (16.9)

with a good constant C, at least if M ≤ εnσ^{2} with a small number ε > 0. The method of proof of Theorem 8.3 presented in this paper cannot yield such a good estimate. The main problem with this method is that it applies a symmetrization argument (this is done in the proof of the Marcinkiewicz–Zygmund inequality), in which we bound the moments of the random variable we are investigating by the moments of a random variable with constant times larger variance. Such a step in the proof does not allow to get the estimate (16.9) with a good constant C > 0.

On the other hand, the estimation of the moments of a degenerate U-statistics by means of the diagram formula yields a better estimate of the moments. The idea behind this approach is that in calculating the even moments E (I_{n,k}(f))^{2M} of a degenerate U-statistics by means of the diagram formula we have to work with some terms which also appear in the calculation of the moments E(Z_{μ,k}(f))^{2M} of the Wiener–Itô integral Z_{μ,k}(f), but we also have to handle some additional terms. It must be checked that the contribution of these additional terms is not too large. This is the case if M ≤ nσ^{2} with σ^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}), and an even better estimate can be given about the contribution of these terms if M ≥ εnσ^{2} with a small ε > 0.

Let me finally remark that the above method can also give an improvement of the multivariate version of the Hoeffding inequality (Theorem 12.3). The proof of the following inequality can be found in [22].

Theorem 16.6. The multivariate version of Hoeffding’s inequality. Let ε_{1}, …, ε_{n} be independent random variables, P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤ n. Fix a positive integer k, and define the random variable

Z = ∑_{(j_{1},...,j_{k}) : 1≤j_{l}≤n for all 1≤l≤k j_{l}≠j_{l′} if l≠l′} a(j_{1}, …, j_{k})ε_{j_{1}} ⋯ ε_{j_{k}}  (16.10)

with the help of some real numbers a(j_{1}, …, j_{k}) which are given for all sets of indices such that 1 ≤ j_{l} ≤ n, 1 ≤ l ≤ k, and j_{l} ≠ j_{l′} if l ≠ l′. Put

S^{2} = ∑_{(j_{1},...,j_{k}) : 1≤j_{l}≤n for all 1≤l≤k j_{l}≠j_{l′} if l≠l′} a^{2}(j_{1}, …, j_{k})  (16.11)

Then

P (k!|Z| > u) ≤ C exp \{−\frac{1}{2} (\frac{u}{S} )^{2/k}\} for all u ≥ 0  (16.12)

with some constant C > 0 depending only on the parameter k.

We may assume that the coefficients a(j_{1}, …, j_{k}) in formulas (16.10) and (16.11) are symmetric functions of their arguments, i.e. a(j_{1}, …, j_{k}) = a(j_{π(1)}, …, j_{π(k)}) for all permutations π ∈ Π_{k} of the set {1, …, k}. If these coefficients a(j_{1}, …, j_{k}) do not have not this symmetry property, then we can replace them with their symmetrizations a_{Sym}(j_{1}, …, j_{k}) = \frac{1}{k!} ∑_{π∈Π_{k}} a(j_{π(1)}, …, j_{π(k)}. In such a way we do not modify the value

of the random variable Z, and decrease the value of the number S^{2}. With such a choice of the coefficients we have EZ = 0 and Var Z = k!S^{2}.

The main advantage of this result with respect to Theorem 12.3 is that formula (16.12) holds with the right constant in the exponent at the right-hand side. The proof is based on good moment estimates of the random variable Z defined in (16.10). I formulate this result which may be interesting in itself.

Theorem 16.7 The random variable Z defined in formula (16.10) satisfies the inequality

EZ^{2M} ≤ 1 · 3 · 5 ⋯ (2kM − 1)S^{2M} for all M = 1, 2, …  (16.13)

with the constant S defined in formula (16.11).

It is worth while to compare formula (16.13) with the estimate that Borell’s inequality yields for this problem. By applying Borell’s inequality with the choice q = 2 and p = 2M we get that EZ^{2M} ≤ (2M − 1)^{kM}E(Z^{2})^{M} = (2M − 1)^{kM} (k!S^{2}) M. Since (2M − 1)^{2M} = (2M)^{kM} (1 − \frac{1}{2M} )^{kM} ∼ e^{−k/2}(2M)^{kM} for large values M, hence Borell’s inequality yields the inequality EZ^{2M} ≤ const. (2M)^{kM}S^{2M} · (k!)^{M} for large exponents M. On the other hand, Theorem 16.7 together with the Stirling formula yield the estimate EZ^{2M} ≤ const. (2M)^{kM}S^{2M} · (\frac{k}{e} )^{kM}. It can be seen that k! > (\frac{k}{e} )^{k} for all k ≥ 1. This means that Theorem 16.7 yields an improvement of the Borell’s inequality in the special case discussed above. This estimate is only a special case of Borell’s inequality, but this is its most important special case.

17. An overview of the results in this work

I discuss briefly the problems investigated in this work and recall some basic results related to them. I also give a list of works where they can be found. Besides, I discuss some background problems and results which may explain the motivation for the study presented here.

I met the main problem considered in this work when tried to adapt the method of proof of the central limit theorem for maximum-likelihood estimates to some more difficult questions about so-called non-parametric maximum likelihood estimate problems. The Kaplan–Meyer estimate for the empirical distribution function with the help of censored data investigated in the second section is such a problem. It is not a maximumlikelihood estimate in the classical sense, but it can be considered as a non-parametric maximum likelihood estimate. Indeed, since in the estimation of a distribution function with the help of censored data the class of possible candidates for being the distribution function we are looking for is too large, there is no dominating measure with respect to which all of them have a density function. As a consequence, the classical principle of the maximum-likelihood estimate cannot be applied in this case. A natural way to overcome this difficulty is to choose a smaller class of distribution functions, to compare the probability of the appearance of the sample we observe with respect to all distribution functions of this class and to choose that distribution function as our estimate for which this probability takes its maximum. The Kaplan–Meyer estimate can be found on the basis of this principle in the following way: Let us estimate the distribution function F (x) of the censored data simultaneously with the distribution function G(x) of the censoring data. (We have a sample of size n and know which sample elements are censored and which are censoring data.) Let us consider the class of such pairs of estimates (F_{n}(x), G_{n}(x)) of the pair (F (x), G(x)) for which the distribution function F_{n}(x) is concentrated in the censored sample points and the distribution function G_{n}(x) is concentrated in the censoring sample points; more precisely, let us also assume that if the largest sample point is a censored point, then the distribution function G_{n}(x) of the censoring data takes still another value which is larger than any sample point, and if it is a censoring point then the distribution function F_{n}(x) of the censored data takes still another value larger than any sample point. (This modification at the end of the definition is needed, since if the largest sample points is from the class of censored data, then the distribution G(x) of the censoring data in this point must be strictly less than 1, and if it is from the class of censoring data, then the value of the distribution function F (x) of the censored data must be strictly less than 1 in this point.) Let us take this class of pairs of distribution functions (F_{n}(x), G_{n}(x)), and let us choose that pair of distribution functions of this class as the (non-parametric maximum likelihood) estimate with respect to which our observation has the greatest probability.

The above extremal problem for the pairs of distribution functions (F_{n}(x), G_{n}(x)) can be solved explicitly, and it yields the estimate of F_{n}(x) written down in formula (2.3). (The function G_{n}(x) satisfies a similar relation, only the random variables X_{j} and Y_{j} and the events δ_{j} = 1 and δ_{j} = 0 have to be replaced in it.) Then, as I have indicated, a natural analog of the linearization procedure in the maximum likelihood estimate also works in this case, and there is only one really hard part of the proof. We need a good estimate on the distribution of the integral of a function of two variables with respect to the product of a normalized empirical measure with itself. Moreover, we also need a good estimate on the distribution of the supremum of a class of integrals, when the elements of an appropriate class of functions are integrated with respect to the above product measure. The main subject of this work is to solve the above problems in a more general setting, when not only two-fold, but also k-fold integrals are considered with arbitrary number k ≥ 1.

The proof of this work for the limit behaviour of the Kaplan–Meyer estimate applied the explicit form of this estimate. It would be interesting to find such a modification of this proof which exploits that the Kaplan–Meyer estimate is the solution of an appropriate extremal problem. We may expect that such a proof can be generalized to a general result about the limit behaviour for a wide class of non-parametric maximum likelihood estimates. Such a consideration is behind the remark of Richard Gill I quoted at the end of Section 2. I hope that such a program can be realized, but at the present time I cannot do this.

A detailed proof together with a sharp estimate on the speed of convergence for the limit behaviour of the Kaplan–Meyer estimate based on the ideas presented in Section 2 is given in paper [24]. Paper [25] explains more about its background, and it also discusses the solution of some other non-parametric maximum likelihood problems. The results about multiple integrals with respect to a normalized empirical distribution function needed in these works were proved in [17]. The results of [18] are completely satisfactory for the study in [24], but they also have some drawbacks. They do not show that if the random integrals we are considering have small variances, then they satisfy better estimates. Besides, if we consider the supremum of random integrals of an appropriate class of functions, then these results can be applied only in very special cases. Moreover, the method of proof of [18] did not allow a real generalization of its results, hence I had to find a different approach when tried to generalize them.

I do not know of other works where the distribution of multiple random integrals with respect to a normalized empirical distribution is studied. On the other hand, there are some works where the distribution of (degenerate) U-statistics is investigated. The most important results obtained in this field are contained in the book of de la Peña and Giné Decoupling, From Dependence to Independence [6]. The problems about the behaviour of degenerate U-statistics and multiple integrals with respect to a normalized empirical distribution function are closely related, but the explanation of their relation is far from trivial. I return to this question later.

Even the study of the one-dimensional version of the problems studied here, i.e. the description of the behaviour of one-fold integrals or classes of one-fold integrals contains several hard problems which have to be investigated closely to have a good understanding of the subject. In the one-dimensional case it is fairly simple to prove that the problems about the behaviour of one-fold integrals with respect to a normalized empirical measure and about the behaviour of normalized sums of independent random variables are equivalent. I start this work with the description of the case of (classes of) one-fold integrals or of sums of independent random variables. This question has a fairly big literature. I would mention first of all the books A course on empirical processes [9], Real Analysis and Probability [10] and Uniform Central Limit Theorems [11] written by R. M. Dudley. These books contain a much more detailed description of the empirical processes than the present work together with a lot of interesting results.

The first problem studied here deals with the tail behaviour of sums of independent and bounded random variables with expectation zero. This question is considered in Section 3 where the proof of two already classical results, that of Bernstein’s and Bennett’s inequalities is explained. (These results are proved e.g. [4]). We are also interested in the question when these results give an estimate suggested by the central limit theorem. Bernstein’s inequality provides such an estimate if the variance of the sum is not too small. (The results in Section 3 tell explicitly when this variance should be considered too small.) If the variance of the sum is too small, then Bennett’s inequality provides a slight improvement of Bernstein’s inequality. On the other hand, Example 3.2 shows that in the unpleasant case when this variance is too small Bennett’s inequality is essentially sharp. I inserted this example to the text, because it may help to understand better the content of Bernstein’s and Bennett’s inequality. I have not found similar examples in the literature.

The estimate on the distribution of a sum of independent random variables if this sum has a small variance is weak because of the following reason. In this case the probability that the sum will be larger than a given value may be much larger than the (rather small) value suggested by the central limit theorem because of the appearance of some irregularities with relatively large probability. The hardest problems we have to cope with in the solution of the problems of this work are closely related to the weak estimates for sums of independent random variables if the variance of the sums are small and to the weak estimates in some similar problems. The weakness of these estimates imply that in the study of the problems we are interested in the method of proof for their Gaussian counterpart cannot be adapted completely, some new ideas are needed. We have overcome this difficulty by applying a symmetrization argument. The last result of Section 3, Hoeffding’s inequality presented in Theorem 3.4 is an important ingredient of this symmetrization argument. It is also a classical result whose proof can be found for instance in [15].

In Section 4 I formulated the one-variate version of our main result about the supremum of the integrals of a class F of functions with respect to a normalized empirical measure together with an equivalent statement about the distribution of the supremum

of a class of random sums ∑_{j=1}^{n} f(ξ_{j}) defined with the help of a sequence of i.i.d. random

variables ξ_{1}, …, ξ_{n} and a class of functions f ∈ F satisfying some appropriate conditions. These results are given in Theorems 4.1 and 4.1′. Also a Gaussian version of them is presented in Theorem 4.2 about the distribution of the supremum of a Gaussian field with some appropriate properties.

In the above mentioned results we have imposed the condition that the class of functions F or what is equivalent the set of random variables whose supremum we estimate is countable. In the proofs this condition is really exploited. On the other hand, in some important applications we also need results about the supremum of a possibly non-countable set of random variables. Hence I introduced the notion of countably approximable classes of random variables and proved that in the results of this work the condition about countability can be replaced by the weaker condition that the class of random variables whose supremum is taken is countably approximable. R. M. Dudley worked out a different method to handle the supremum of possibly noncountably many random variables, and generally his method is applied in the literature. The relation between these two methods deserves some discussion.

Let us first recall that if a class of random variables S_{t}, t ∈ T, indexed by some index set T is given, then a set A can be measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, only if there exists a countable subset T′ = T′(A) ⊂ T such that the set A is also measurable with respect to the smaller σ-algebra generated by the random variable S_{t}, t ∈ T′. Besides, if the finite dimensional distributions of the random variables S_{t}, t ∈ T, are given, then by the results of classical measure theory also the probability of the events measurable with respect to the σ-algebra generated by these random variables S_{t}, t ∈ T, is determined. But there are rather few other events whose probabilities are determined by the finite dimensional distributions of the random variables S_{t}, t ∈ T. On the other hand, if T is a non-countable set, then the events

{sup_{t∈T} S_{t} > u} are not measurable with respect to the above σ-algebra, hence generally

we cannot speak of their probabilities. To overcome this difficulty Dudley worked out a theory which enabled him to work also with outer measures. His theory is based on some rather deep results of the analysis. It can be found for instance in his book [11].

I restricted my attention to the case when after the completion of the probability measure P we can also speak of the real (and not only outer) probabilities

P (sup_{t∈T} S_{t} > u). I tried to find appropriate conditions under which these probabili-

ties really exist. More explicitly, we are interested in the case when for all u > 0 there exists some set A = A_{u} measurable with respect to the σ-algebra generated by the random variables S_{t}, t ∈ T, such that the symmetric difference of the sets A_{u} and

{sup_{t∈T} S_{t} > u} is contained in a set measurable with respect to the σ-algebra generated

by the random variables S_{t}, t ∈ T, and has probability zero. In such a case we can define

also the probability P (sup_{t∈T} S_{t} > u) as P (A_{u}). This approach led me to the definition

of countable approximable classes of random variables. Its validity enables us to speak about the probability of the event that the supremum of the random variables we are interested in is larger than some fixed value. I also proved a simple but useful result in Lemma 4.3, which provides a condition for the validity of this property.

The problem we met here is not an abstract, technical difficulty. Indeed, the distribution of such a supremum can become different if we modify each random variable at a set of probability zero, although the joint distribution of the random variables we consider remains the same after such an operation. Hence, if we are interested in the supremum of a non-countable set of random variables with described joint distribution we have to describe more explicitly which version of this set of random variables we consider. It is natural to look for such an appropriate version of the random field S_{t}, t ∈ T, whose ‘trajectories’ S_{t}(ω), t ∈ T, have nice properties for all elementary events ω ∈ Ω. Lemma 4.3 can be interpreted as a result in this spirit. The condition given for the countable approximability of a class of random variables at the end of this lemma can be considered as a smoothness condition about the ‘trajectories’ of the random field we consider. This approach shows some analogy to some important problems in the theory of stochastic process when a regular version of a stochastic process is considered and the smoothness properties of its trajectories are investigated.

In our problems the version of the set of random variables S_{t}, t ∈ T, we shall work with appears in a simple and natural way. In these problems we have finitely many random variables ξ_{1}, …, ξ_{n} at the start, and all random variables S_{t}(ω), t ∈ T, we are considering can be defined individually for each ω as a functional of these random variables ξ_{1}(ω), …, ξ_{n}(ω). We take the version of the random field S_{t}(ω), t ∈ T, we get in such a way and want to show that it is countably approximable. In Section 4 we have proved this property in an important model, probably in the most important model in possible applications we are interested in. In more complicated situations when our random variables are defined not as a functional of finitely many sample points, for instance in the case when we define our set of random variables by means of integrals with respect to a Gaussian field it is harder to find the right regular version of our sets of random variables. In this case the integrals we consider are defined only with probability 1, and we have to make some extra work to find their right version. At any rate, in the problems we are interested in our approach is satisfactory for our purposes, and it is simpler than that of Dudley; we do not have to follow his rather difficult technique. On the other hand, I must admit that I do not know the precise relation between the approach of this work and that of Dudley.

In Section 4 the notion of L_{p}-dense classes, 1 ≤ p < ∞, is also introduced. The notion of L_{2}-dense classes plays an important role in the formulation Theorems 4.1 and 4.1′. The notion of L_{2}-dense classes can be considered as a version of the ε-entropy discussed at many places in the literature. On the other hand, there seems to be no unique definition of ε-entropy in the literature. I introduced the term of L_{2}-dense classes, because this seems to be the appropriate notion in the study of this work. To apply the results related to L_{2}-dense classes we also need some knowledge about how to check it in concrete models. For this goal I discussed here Vapnik–Červonenkis classes, a popular and important notion of modern probability theory. Several books and papers, (see e.g. the books [11], [28], [30] and the references in them) deal with this subject. An important result in this field is Sauer’s lemma, (Lemma 5.2) which together with some other results, like Lemma 5.3 imply that the classes of sets or functions are in many several interesting models Vapnik–Červonenkis classes.

I put these results to the Appendix, partly because they can be found in the literature, partly because in our investigation Vapnik–Červonenkis classes play a different and less important role than at other places. In our discussion Vapnik–Červonenkis classes are applied to show that certain classes of functions are L_{2}-dense. A result of Dudley formulated in Lemma 5.2 implies that a Vapnik–Červonenkis class of functions with absolute value bounded by a fixed constant is an L_{1}, hence also an L_{2}-dense class of functions. The proof of this important result which seems to be less known even among experts of this subject than it should be is contained in the main text. Dudley’s original result was formulated in the special case when the functions we consider are indicator functions of some sets, but its proof contained all important ideas needed in the proof of Lemma 5.2.

Theorem 4.2, which is the Gaussian counterpart of Theorems 4.1 and 4.1′ is proved in Section 6 by means of a natural and important technique, called the chaining argument. We apply an inductive procedure, during which an appropriate sequence of finite subsets of our set of random variables is defined, and try to give a good estimate on the supremum of these subsets of our random variables. The subsets we consider are denser and denser subsets of the original set of random variables, and if they are constructed in a clever way, then we get the result we want to prove by means of a limiting procedure. In such a way we get a relatively simple proof of Theorem 4.2, but this method is not strong enough to supply a complete proof of Theorem 4.1. The cause of the weakness of the method in this case is that we cannot give a good estimate on the probability that a sum of independent random variables is greater than a prescribed value if these random variables have too small variances. The chaining argument supplies a result much weaker than that what we want to prove under the conditions of Theorem 4.1. Lemma 6.1 contains the result the chaining argument yields under the conditions of Theorem 4.1. In Section 6 still another result, Lemma 6.2 is formulated, and it is also shown that Lemmas 6.1 and 6.2 together imply Theorem 4.1. The proof is not difficult, despite of some non-attractive details. We have to check that the parameters in Lemmas 6.1 and 6.2 can be fitted to each other.

Lemma 6.2 is proved in Section 7. It is based on a symmetrization argument. This proof applies the ideas of a paper of Kenneth Alexander [1], and although its presentation is essentially different of Alexander’s approach, it can be considered as a version of his proof.

A similar problem should also be mentioned at this place. M. Talagrand wrote a series of papers about concentration inequalities, and this research was also continued by some other authors. I would mention the works of M. Ledoux [16] and P. Massart [26]. Concentration inequalities give a bound about the difference of the supremum of a set of appropriately defined random variables from its expected value; they express how strongly this supremum is concentrate around its expected value. Such results are closely related to Theorem 4.1, and the discussion of their relation deserves some attention. A typical concentration inequality is the following result of Talagrand [29].

Theorem 17.1. (Theorem of Talagrand.) Consider n independent and identically distributed random variables ξ_{1}, …, ξ_{n} with values in some measurable space (X, X). Let F be some countable family of real-valued measurable functions of (X, X) such that

∥f∥_{∞} ≤ b < ∞ for every f ∈ F. Let Z = sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i}) and v = E(sup_{f∈F} ∑_{i=1}^{n} f^{2}(ξ_{i})).

Then for every positive number x,

P (Z ≥ EZ + x) ≤ K exp \{−\frac{1}{K′} \frac{x}{b} log (1 + \frac{xb}{v})\}

and

P (Z ≥ EZ + x) ≤ K exp \{−\frac{x^{2}}{2(c_{1}v + c_{2}bx)}\} ,

where K, K′, c_{1} and c_{2} are universal positive constants. Moreover, the same inequalities hold when replacing Z by −Z.

Theorem 17.1 yields, similarly to Theorem 4.1, an estimate about the distribution of the supremum for a class of sums of independent random variables. It can be considered as a generalization of Bernstein’s and Bennett’s inequalities when the distribution of the supremum of partial sums is estimated. A remarkable feature of this result is that it assumes no condition about the structure of the class of functions F (like the condition of L_{2}-dense property of the class F imposed in Theorem 4.1.) On the other

hand, the estimates in Theorem 17.1 contain the quantity EZ = E (sup_{f∈F} ∑_{i=1}^{n} f(ξ_{i})).

Such an expectation of some supremum appears in all concentration inequalities. As a consequence, they are useful only if we can bound the expected value of an appropriate supremum. This is a hard question in the general case, and this is the reason why I preferred a direct proof of Theorem 4.1 without the application of concentration inequalities. Let me remark that the condition u ≥ const. σ log^{1/2} \frac{2}{σ} with some appropriate constant which cannot be dropped from Theorem 4.1 is related to the fact that the expected value of the supremum of the normalized random sums considered in Theorem 4.1 has such a magnitude.

The main results of this work are presented in Section 8. Theorem 8.3 which contains an estimate about the distribution of a degenerate U-statistic was first proved in a paper of Giné and Arcones in [2], its equivalent version about the multiple integrals with respect to a normalized empirical measure formulated in Theorem 8.1 in my paper [19]. The equivalence of these two results is not self-evident. Later I proved an improved version of Theorem 8.3 in paper [21]. This result is formulated in Theorem 16.1, and it is also compared with Theorem 8.3. It is also explained that Theorem 16.1 could be considered the multivariate version of Bernstein’s inequality with more right than Theorem 8.3. Here I omitted its proof which applies a technique (diagram formulas for the calculation of products of multiple random integrals or degenerate U-statistics) not discussed in this work. Here Theorem 8.3 was proved by means of a symmetrization argument. The explanation of such a proof was simpler in the present work, because it applies such methods which were worked out in the investigation of other problems. On the other hand, some arguments can be posed against such a proof. The application of symmetrization arguments in the proof of Theorem 8.3 also has some drawbacks. In certain problems, like the problem of Theorem 8.3, this method cannot supply a really sharp result. Some mathematicians working in this field seem not to be aware of this fact.

It may be interesting to mention that the problem of Theorem 8.3 has a natural generalization worth of a closer study. We can consider such generalized U-statistics in which the underlying random variables ξ_{1}, …, ξ_{n} are independent, but they need not be identically distributed, and the U-statistic also may have a more general form. Namely, we can take a class of kernel functions f = {f_{l_{1},...,l_{k}}(x_{1}, …, x_{k})} on the space (X^{k}, X^{k}) with such an indexation that 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k, and l_{j} ≠ l_{j′} if j ≠ j′, and define with the help of these independent random variables and class of kernel functions the generalized U-statistic

I_{n,k}(f) = ∑_{\begin{matrix}1≤l_{j}≤n, 1≤j≤k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}}(ξ_{l_{1}}, …, ξ_{l_{k}}).  (17.1)

One can also naturally define generalized degenerate U-statistics. We call a generalized U-statistic degenerate if for all sets of indices (l_{1}, …, l_{k}) in the sum (17.1) and for all 1 ≤ j ≤ k

E(f_{l_{1},...,l_{k}}(ξ_{l_{1}}, …, ξ_{l_{k}})|ξ_{l_{s}}, s ∈ {1 …, k} \ {j}) ≡ 0.

Generalized degenerate U-statistics can be considered as the natural multivariate generalizations of sums of independent random variables, just as degenerate U-statistics are the natural multivariate generalizations of sums of iid. random variables. One would also try to generalize Theorem 8.3 to an estimation about the distribution of generalized degenerate U-statistics. One may hope that the method of proof of Theorem 8.3 can also be applied for the study of generalized degenerate U-statistics, just as the distribution of sums independent random variables can be investigated similarly to the sums of iid. random variables. Probably, the methods worked out for the study of the problems related to Theorem 8.3 are helpful, but in the study of generalized degenerate U-statistics first some special questions have to be clarified. We have to find the right form of the estimation about the distribution of a generalized degenerate U-statistic. In particular, it must be clarified which are the natural quantities by which we should express this estimate.

It is natural to expect that generalized degenerate U-statistics I_{n,k}(f) of order k (without normalization) satisfy the inequality

P (|I_{n,k}(f)| > u) < A exp \{−C (\frac{u}{V_{n}} )^{2/k}\}  (17.2)

with some universal constants A = A(k) > 0 and C = C(k) > 0 in a relatively large interval for the parameter u, where V_{n}^{2} denotes the variance of I_{n,k}(f). An essential problem is to find a relatively good constant C and to determine the interval 0 < u < D_{n}, where the estimate (17.2) holds. Theorem 8.3 states that in the case of classical degenerate U-statistics (17.2) holds in the interval [0, D_{n}] with D_{n} = const. n^{k}σ^{k+1}, where σ^{2} = Ef(ξ_{1}, …, ξ_{k})^{2}. For k = 1 this means that relation (1.9) holds in the interval 0 ≤ u ≤ V_{n}^{2}. But it is not clear what corresponds in the case of generalized degenerate U-statistics to the right end-point D_{n} = const. n^{k}σ^{k+1} of the interval where the estimate (17.2) should hold. (The variance of a degenerate U-statistic of order k is of order n^{k}σ^{2}.)

Theorems 8.2 and 8.4 yield an estimate about the supremum of (degenerate) Ustatistics or of multiple random integrals with respect to a normalized empirical measure when the class of kernel functions in these U-statistics or random integrals satisfy some conditions. They were proved in my paper [20]. Earlier Arcones and Giné proved a weaker form of this result in paper [3]. The Gaussian version of Theorem 8.1 or 8.3 given in Theorem 8.5 was proved much earlier. My lecture note [17] also contains a proof of this result. The second statement of Theorem 8.5 about the supremum of Wiener–Itô integrals can be simply proved. Section 8 also contains an example which shows in particular that the probability P (n^{−1}I_{n,2}(f) > u) can be bounded for a degenerate U-statistic I_{n,2}(f) of order 2 by the estimate given in Theorem 8.3 only if u ≤ const. nσ^{3}, i.e. this condition of Theorem 8.3 (in the case k = 2) cannot be dropped. Similar examples could be constructed for all k ≥ 1. The paper of Arcones and Giné [2] contains another example explained by Talagrand to the authors which also has a similar consequence.

On the other hand, this example does not exclude the possibility to prove such a multi-dimensional version of Hoeffding’s inequality Theorem 3.3 which provides a slight improvement of Theorems 8.1 and 8.3 similarly to the improvement of Bernstein’s inequality provided by Hoeffding’s inequality. Moreover, we can also expect such a strengthened form of Theorems 8.2 and 8.4 (or of Theorem 4.2 in the one-dimensional case) which takes into account the above improvements if the supremum of a nice class of random integrals or degenerated U-statistics is considered. There is a hope that some refinement of the methods of the present work would supply such results. However, here we did not study this problem.

Theorems 9.2 and 9.3 deal with the properties of degenerate U-statistics. This subject deserves special attention. Degenerate U-statistics can be considered as the multivariate version of sums of independent and identically distributed random variables with expectation zero. Similarly, if f is a canonical function with respect to a measure μ and put independent μ-distributed random variables into its arguments, then the random variables we get in such a way can be considered as the multivariate version of random variables with expectation zero. The background of several proofs about the behaviour of U-statistics can be better understood with the help of the above remark. I tried to explain for instance that the proof about the Hoeffding decomposition of U-statistics (Theorem 9.1) is actually a natural adaptation of the decomposition of a random variable to the sum of a random variable with expectation zero plus the expected value of the random variable.

Hoeffding’s decomposition is a fairly well-known result which can be found for instance in the Appendix of [12]. Theorem 9.1 slightly differs from the formulation of Hoeffding’s decomposition one usually meets in the literature. It can be exploited that a U-statistic does not change if we replace its kernel function by its symmetrized version. Besides, the value of the U-statistics I_{n,|V |}(f_{V}) do not change if we replace the kernel function f_{V} (x_{j_{1}}, …, x_{j_{|V |}}), V = {j_{1}, …, j_{|V |}}, by f_{V} (x_{1}, …, x_{|V |}) in the Hoeffding decomposition (9.3) of the U-statistic I_{n,k}(f), and f_{V} (x_{1}, …, x_{|V |}) is also a canonical function. The above observations enable us to unify the contribution of all terms I_{n,|V |}(f_{V}) with |V | = l for some 0 ≤ l ≤ k into one non-degenerate Ustatistics of order l. Generally, the formula obtained in such a way is called the Hoeffding decomposition in the literature. Nevertheless, we have applied Theorem 9.1 in this work, because this form of the Hoeffding’s decomposition was more convenient for us.

In our investigations it is important to know that if a function satisfies a good L_{2}-norm or L_{∞}-norm estimate, then the elements of its Hoeffding decomposition also have this property, and if a class of function is L_{2}-dense, then the same relation holds for the classes of functions in the Hoeffding decomposition of the functions in this class. This is the content of Propositions 9.2 and 9.3. The estimates on the L_{2}-norm given in formulas (9.7) and (9.8) are actually reformulations of some well-known facts about the properties of conditional expectations.

Theorem 9.4 enables us to reduce the estimates about multiple random integrals with respect to normalized empirical measures to estimates about degenerate U-sta-tistics. Such random integrals are actually sums of U-statistics, and we can apply for each of these U-statistics the Hoeffding decomposition. Besides, as we consider multiple integrals with respect to a normalized empirical measure we can expect that a lot of cancellations appear during the calculation by which we express our random integral in the form of linear combination of degenerate U-statistics. We get such a representation which enables us to reduce the estimates we want to prove about multiple random integrals to analogous estimates about degenerate U-statistics. This is the main content of Theorem 9.4 which can be considered as an analog of the Hoeffding decomposition for multiple stochastic integrals with respect to normalized empirical measures. This representation of a multiple stochastic integral as a linear combination of degenerate U-statistics of different order also contains degenerate U-statistics of low order. But as a consequence of the cancellation effects these U-statistics are multiplied with small coefficients. The proof of Theorem 9.4 is based on a good “book-keeping” of the different contributions to the integral J_{n,k}(f). An essential, although less spectacular step of this “book-keeping” procedure is to express the terms we are working with by means of the (signed) measures μ and μ^{(l)} − μ, i.e. the measures μ^{(l)} have to be replaced by their normalizations μ^{(l)} − μ. The calculations needed in the proof are quite natural, but unfortunately they contain some unpleasant and complicated technical details.

Theorem 9.4 also has the consequence that the second moment of the multiple random integral of a function with respect to a normalized empirical measure can be bounded by constant times the L_{2}-norm of the kernel function we integrate. The representation of the stochastic integrals given in Theorem 9.4 may also contain a non-zero constant term. This has the unexpected consequence that the expected value of a multiple random integral with respect to a normalized empirical measure can be non-zero. Our random integrals may show such an unusual behaviour because the numbers of sample points falling to disjoint sets are not independent random variables. But the dependence between such random variables is very weak, and the expected value of the random integrals we consider is sufficiently small.

From the pair of Theorems 8.1 and 8.3 I have proved only Theorem 8.3, since its proof is simpler, and by the results of Section 9 Theorem 8.1 follows from it. The proof of Theorem 8.3 is different from its original proof published in paper [2]. First a good estimate is presented about the moments of the degenerate U-statistics in Proposition 10.1. Theorem 8.3 can be deduced from this estimate. Actually the proof is different, first a version Theorem 8.3′ of Theorem 8.3 is proved, where an analogous estimate is proved for degenerate decoupled U-statistic. The adjective ‘decoupled’ refers to the fact that we put independent copies of a sequence of iid. random variables in different coordinates of the kernel function of the U-statistic. The study of decoupled U-statistics is a popular subject of some authors. In particular, the main subject of the book [6] is a comparison of the properties of U-statistics and decoupled U-statistics.

The study of decoupled U-statistics is simpler than that of usual U-statistics, because the arguments applied in the study of usual U-statistics can be applied for them, and they also satisfy a multivariate version of the Marcinkiewicz–Zygmund inequality. On the other hand, the Marcinkiewicz–Zygmund inequality does not hold for usual Ustatistics, at least the proofs I know of do not work for them. We can prove with the help of the multivariate version of the Marcinkiewicz–Zygmund and Borell’s inequality an estimate about the moments of degenerate U-statistics formulated in Proposition 10.1′. Proposition 8.3′ can be deduced from Proposition 10.1′, and by a result of de la Peña and Montgomery–Smith formulated in Theorem 10.4 Theorem 8.3′ implies Theorem 8.3. The results applied in the proof of Theorem 8.3 are proved in Section 11. Let me also remark that Proposition 10.1 is not proved in this text, since we chose such an approach where we do not need it. On the other hand, it follows from the results of this work and some other standard results about U-statistics not discussed in the present work.

I have mentioned the possibility of another proof of Theorem 8.3 on the basis of the methods of the theory of Wiener–Itô integrals to this problem. In [19] I gave a proof of Theorem 8.1 by means of the so-called diagram method. Let me also remark that the method of paper [21] which yields an improvement of Theorem 8.3 presented in Theorem 16.1 is actually a refinement of the method in [19]. Both in paper [19] and in the present work the main step of the proof consists of finding a good estimate on the moments of the random variables we are investigating. It is enough to estimate the moments of the type M = 2^{m}, where m is a positive integer. For m = 1 such an estimate is known, and we can get an estimate for m > 1 by means of a recursive procedure. A similar approach is applied in [19] and in the present work. The main difference between them is in the form of the recursive inequality between the moments of the random variables we work with and the way we prove them.

I found the result about the multivariate version of the Marcinkiewicz–Zygmund equation in the book [6], but the proof of the result given here is different. Only the proof about the upper estimate of the p-th moment of decoupled U-statistics is written down. There is also an estimate in the opposite direction, but such a result would be interesting for us only for the sake of some orientation. Theorem 10.4 was proved by de la Peña and Montgomery–Smith in their paper [7]. I formulated their result for separable Banach space valued random variables, just as they did it. Such a general formulation of the results is very popular in the literature, but here the discussion of Banach space valued random variables had a different cause. I also wanted to prove formula (10.8′), a result which is actually not contained in paper [7]. (Book [6] contains this result, but the proof is left to the reader.) The simplest way to get this statement was to prove the original result in Banach spaces, and to apply it in appropriate L_{∞} spaces. Paper [7] also contains some kind of converse result of Theorem 10.4, but as we do not need it I omitted its discussion.

This work contains the proof of de la Peña and Montgomery–Smith for Theorem 10.4, but I have explained it in my own style. In particular, I worked out some details where the author gave only a very short explanation. This proof is given in the Appendix.

The proof of Borell’s inequality is closely related to that of Nelson’s inequality. Edward Nelson published the inequality named after him in his paper [27]. He also showed that the general inequality presented in Appendix C can be reduced to the inequality given in formula (C1) or in Proposition C2 of this work. This reduction follows actually from our Theorem 11.2. However, this observation did not help him to find a proof, and finally he gave a proof without its application. Borell’s inequality can also be reduced to a one-dimensional statement formulated in Theorem 11.3. This seems to be a simple inequality, but its proof is surprisingly hard. Actually in this paper it is enough to prove this inequality in the special case q = 2 and p = 2k, k = 1, 2, . . .. Actually, as I mentioned in Theorem 16.6, Borell’s inequality can be proved in this special case with better constants. (See paper [22].)

In the proof of Theorem 11.3 I have followed the paper of Leonhard Gross Logarithmic Sobolev inequalities [13]. Gross has worked out a general theory and he could prove both Nelson’s and Borell’s inequality (more precisely an estimate which simply implies this result) with its help. Gross’ method and results are interesting, because they are very useful in several parts of the mathematics. (See e.g [16] or [14].) Let me also remark that similar results and ideas also appeared in an earlier work of A. Bonami [5].

Gross introduced a so-called logarithmic Sobolev inequality related to Markov processes and showed that it implies another inequality, which is in the case of a Wiener process Nelson’s inequality, while we can define such a simple Markov process for which the logarithmic Sobolev inequality corresponding to it yields the proof of Theorem 11.3. This Markov process is explicitly described in Section 11, and the logarithmic Sobolev inequality corresponding to it is also formulated and proved there. Actually Gross showed that each logarithmic Sobolev inequality is equivalent to the inequality he proved as its consequence. On the other hand, the proof of the logarithmic Sobolev inequalities is less difficult than a direct proof of the inequalities he has obtained as their consequence.

The name ‘logarithmic Sobolev inequality’ has the following explanation. Generally one calls ‘Sobolev inequality’ such inequalities where for some pairs of numbers 1 ≤ q < p < ∞ we prove a bound on the L_{p}-norm of a function if we have an estimate on its L_{q}-norm together with the L_{q}-norm of some partial derivatives of this function. In the logarithmic Sobolev inequalities the integral of a function of the form |f|^{p} log |f| is bounded by means of the integral of |f|^{p} and the integral of a differential type operator of this function f which is closely related to the infinitesimal operator of a Markov process.

The proof of Borell’s inequality presented here is due to Leonhard Gross. We have also shown in the Appendix that from this estimate and the central limit theorem Nelson’s inequality can be deduced. In this proof we have applied some basic facts about Wiener–Itô integrals which we did not discuss in detail. The most important results we have used here are the so-called Itô’s formula for Wiener–Itô integrals and the diagram formula. All these results can be found in my lecture note [17]. Borell’s inequality was applied in the proof of Theorem 8.3′. We also proved another result with its help which plays an important role in our study. This is the multivariate version of Hoeffding’s inequality in Theorem 12.3. This result is a simple consequence of Borell’s inequality, but I did not find it in the literature. Paper [22] contains an improved version of this estimates presented in Theorem 16.6.

Sections 12 — 15 deal with the proof of Theorem 8.4 about the tail-behaviour of the supremum of a class of degenerate U-statistics under appropriate conditions. This result was proved in my paper [20]. The proof of this result is similar to that of its onevariate version Theorem 4.1, but some additional difficulties have to be overcome. We have formulated some results in Propositions 12.1 and 12.2 which are the multivariate analogs of Propositions 6.1 and 6.2, and Theorem 8.4 can be proved as their consequence. Proposition 12.1 can be proved similarly to Proposition 6.1, and also the deduction of Theorem 8.4 from Propositions 12.1 and 12.2 is similar to the argument applied in the proof of Theorem 4.1.

The hard part of the problem is to prove Proposition 12.2. By means of the results of de la Peña and Montgomery–Smith it can be reduced to a version formulated in Proposition 12.2′, where degenerate U-statistics are replaced by degenerate decoupled U-statistics. This result is proved by means of a refinement of the argument of the proof of Proposition 6.2. The main difficulty appears as we want to find the multivariate analog of the symmetrization argument made by means of the Symmetrization Lemma, Lemma 7.1 and Lemma 7.2 in the one-variate case. In the proof of Theorem 4.1 we could carry out a symmetrization procedure by investigating the difference of two independent copies of the random sums we have considered. In the proof of Proposition 12.2′ a more sophisticated construction has to be applied. This construction actually appeared in the proof of Theorem 8.3, and Lemma 11.5 explains its most important properties.

In the proof of Proposition 12.2′ Lemma 7.1 is not sufficient for us in its original form. We need a generalization of this result, and this is given in Lemma 13.1. The proof of Lemma 13.1 is not hard. The real difficulty arises when we want to apply it in our case. Then as we want to check formula (13.1) we have to bound some non-trivial conditional probabilities. In the analog relation, in formula (7.1) of Lemma 7.1 it was enough to bound a usual probability, and this was simple. But as we want to adapt this method in the multivariate case we have to bound an appropriate conditional variance. This demands much more work, and the hardest new steps of the proof were introduced to overcome this difficulty.

Proposition 12.2′ was proved by means of an inductive procedure formulated in Proposition 13.2, which is the multivariate analog of Proposition 6.2. But because of the problems we meet in carrying out the symmetrization procedure the arguments of Proposition 7.2 are not sufficient in this case. Hence another statement is introduced in Proposition 13.3. Propositions 13.2 and 13.3 can be proved simultaneously by means of an appropriate inductive procedure. The proof is based on a refinement of the arguments in the proof of Proposition 6.2. We also have to exploit our knowledge about the properties of Hoeffding’s decomposition.

Appendix A.

The proof of some results about Vapnik–Červonenkis classes

Proof of Theorem 5.1. (Sauer’s lemma) Let F_{1}, …, F_{m} be the subsets of cardinality k of the set S_{0}(n), m = \binom{n}{k}. By the conditions of the theorem all sets F_{j}, 1 ≤ j ≤ m, have a “hidden” subset H_{j} ⊂ F_{j} such that the class of sets D(S_{0}, F_{j}) = {F_{j} ∩ B; B ∈ D(S_{0})} does not contain the set H_{j}. Let us denote by C_{0} = C_{0}((F_{1}, H_{1}), …, (F_{m}, H_{m})) the class of subsets of S_{0}(n) we get by taking first all subsets of S_{0}(n) and then omitting all subsets of the form H_{j} ∪G_{j} with some G_{j} ⊂ S_{0} \F_{j}, 1 ≤ j ≤ m. The subsets omitted in the definition of C_{0} do not belong to D(S_{0}), thus C_{0} contains all elements of D(S_{0}), and it is enough to show that C_{0} contains no more than \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k−1} subsets of S_{0}(n). If H_{j} = F_{j} for all “hidden” subsets H_{j}, 1 ≤ j ≤ m, then C_{0} contains the subsets of S_{0}(n) with cardinality at most k − 1, and we have to show that this is the extreme case.

Let us choose some element s ∈ S_{0}, and define similarly to the class C_{0} a new class C_{1} = C_{1}((F_{1}, H̄_{1}), …, (F_{m}, H̄_{m})) with the difference that instead of the “hidden” subsets H_{j} of F_{j} taking part in the definition of C_{0} we work with the sets H̄_{j} we get by augmenting H_{j} with the element s if it is possible, i.e. in the definition of C_{1} H_{j} is replaced by H̄_{j} = (H_{j} ∪ {s}) ∩ F_{j}. Given a set B ⊂ S_{0} we can say that B ∈ C_{0} if and only if B ∩ F_{j} ≠ H_{j} for all 1 ≤ j ≤ m, and B ∈ C_{1} if and only if B ∩ F_{j} ≠ H̄_{j} for all 1 ≤ j ≤ m. We want to show that C_{1} has more elements than C_{0}. Theorem 5.1 can be deduced from this statement, because by iterating this procedure for enlarging the “hidden” subsets H_{j} of the sets F_{j} for all s ∈ S_{0} we get that the class C_{0} has the greatest cardinality in the case when H_{j} = F_{j} for all 1 ≤ j ≤ k.

Let us define the map T (B) = B \ {s} for all sets B ⊂ S_{0}(n). We shall show that T (·) is an injection of C_{0} \ C_{1} to C_{1} \ C_{0}. This implies that the cardinality of C_{1} is larger than that of C_{0} just as we claimed. To prove the above property of T (·) first we check that a) if B ∈ C_{0} \ C_{1} then s ∈ B. This implies that different elements of C_{0} \ C_{1} have different images under the map T. We also check that b) if B ∈ C_{0} \ C_{1}, then T (B) ∈ C_{1} \ C_{0}, i.e. b1) T (B) ∈ C_{1} and b2) T (B) ∉ C_{0}.

If B ∈ C_{0} \ C_{1} then B ∩ F_{j} ≠ H_{j} for all 1 ≤ j ≤ m, and B ∩ F_{j} = H̄_{j} for some j. This means that B ∩ F_{j} ≠ H_{j} and B ∩ F_{j} = H̄_{j} for some index j. This is only possible if s ∉ H_{j}, s ∈ F_{j} and s ∈ B, i.e. property a) holds. Besides, T (B) ∩ F_{j} = H̄_{j} \ {s} = H_{j} for such an index j which means that property b2) holds. To check property b1) we have to show that if B ∈ C_{0} \ C_{1}, then (B \ {s}) ∩ F_{j} ≠ H̄_{j} for all 1 ≤ j ≤ m. This relation clearly holds for such indices j for which s ∈ F_{j}, since in this case s ∈ H̄_{j}. If s ∉ F_{j}, then the condition B ∈ C_{0} implies that B ∩ F_{j} ≠ H_{j}, and H̄_{j} = H_{j} and B ∩ F_{j} = (B \ {s}) ∩ F_{j} because of the relation s ∉ F_{j}. These relations imply that (B \ {s}) ∩ F_{j} ≠ H̄_{j} also in this case.

The proof of Theorem 5.3 Let us fix an arbitrary set F = {s_{1}, …, s_{k+1}} of the set S, and consider the set of vectors G_{k}(F) = {(g(s_{1}), …, g(s_{k+1})): g ∈ G_{k}} of the k + 1dimensional space R^{k+1}. By the conditions of the Theorem G_{k}(F) is an at most kdimensional subspace of R^{k+1}. Hence there exists a non-zero vector a = (a_{1}, …, a_{k+1}) such that ∑_{j=1}^{k+1} a_{j}g(s_{j}) = 0 for all g ∈ G_{k}. We may assume that A = A(a) = {j: a_{j} <

0, 1 ≤ j ≤ k + 1} is a non-empty set by multiplying the vector a by −1 if it is necessary. Thus we can write

∑_{j∈A} a_{j}g(s_{j}) = ∑_{j∈{1,...,k+1}\A}(−a_{j})g(s_{j}),  for all g ∈ G_{k}.  (A1)

Put B = {s_{j}, j ∈ A}. Then B ⊂ F, and we claim that B ≠ {g: g(s) ≥ 0} ∩ F for all g ∈ G_{k}. Indeed, if there were some g ∈ G_{k} such that B = {g: g(s) ≥ 0} ∩ F, then the left-hand side of the equation (A1) would be strictly positive and its right-hand side would be non-positive for this g ∈ G_{k}, and this is a contradiction.

Thus Theorem 5.1 implies that for all subsets S_{0}(n) of S with n ≥ k + 1 elements and the class of subsets D of S introduced in the formulation of Theorem 5.3 S_{0}(n) ∩ D has at most \binom{n}{0} + \binom{n}{1} + ⋯ + \binom{n}{k} elements. Hence D is a Vapnik–Červonenkis class.

Appendix B. The proof of Theorem 10.3

(A result of de le Peña and Montgomery–Smith)

The proof of Theorem 10.3. We concentrate our efforts to prove relation (10.8). Formula (10.8′) can be obtained as a relatively simple consequence of this result. The proof of formula (10.8) will be made by means of an inductive procedure. To carry out it we have to formulate and prove our statement in a more general form where such generalized U-statistics are considered for which different kernel functions may appear in each term of the sum. More explicitly, let ℓ = ℓ(n, k) denote the set of all sequences l = (l_{1}, …, l_{k}) of length k such that 1 ≤ l_{j} ≤ n, 1 ≤ j ≤ k. Let us fix a class of functions {f_{l_{1},...,l_{k}}(x_{1}, …, x_{k}), (l_{1}, …, l_{k}) ∈ ℓ} which map the space (X^{k}, X^{k}) to a separable Banach space B. Let us denote this class of functions by f(ℓ), and define similarly to the U-statistics and decoupled U-statistics the generalized U-statistics and generalized decoupled U-statistics by the formulas

I_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}} (ξ_{l_{1}}, …, ξ_{l_{k}})

and

Ī_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(1)}, …, ξ_{l_{k}}^{(k)})

(with the same random variables ξ_{l} and ξ_{l}^{(j)}, 1 ≤ l ≤ n, 1 ≤ j ≤ k as before.) The following generalization of relation (10.8) will be proved.

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ AP (∥Ī_{n,k}(f(ℓ))∥ > γu)  (10.8b)

with some constants A = A(k) and γ = γ(k) depending only on the order of these U-statistics.

To prove relation (10.8b) first we verify the following statement.

Let us take two independent copies ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} of our original sequence of random variables ξ_{1}, …, ξ_{n} and introduce for all sets V ⊂ {1, …, k} the function α_{V} (j), 1 ≤ j ≤ k, defined as α_{V} (j) = 1 if j ∈ V and α_{V} (j) = 2 if j ∉ V. Let us define with the help of these quantities the decoupled generalized U-statistics

I_{n,k,V} (f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}}f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(α_{V} (1))}, …, ξ_{l_{k}}^{(α_{V} (k))}) for all V ⊂ \{1, …, k\}.

(B1)

The following inequality will be proved: There are some constants C_{k} > 0 and D_{k} > 0 depending only on the order k of the generalized U-statistic I_{n,k}(f(ℓ)) such that for all numbers u > 0

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ ∑_{V ⊂{1,...,k}, 1≤|V |≤k−1} C_{k}P (D_{k}∥I_{n,k,V} (f(ℓ))∥ > u) .  (B2)

Here |V | denotes the cardinality of the set V, and the condition 1 ≤ |V | ≤ k − 1 in the summation of formula (B2) means that we omit the sets V = ∅ and V = {1, …, k} from the summation, i.e. the cases when either α_{V} (j) = 1 for all 1 ≤ j ≤ k or α_{V} (j) = 2 for all 1 ≤ j ≤ k are not considered in this sum. Formula (10.8b) can be deduced from formula (B2) by means of a relatively simple inductive argument. In the proof of formula (B2) we shall apply the following simple lemma.

Lemma B1. Let ξ and η be two independent and identically distributed random variables taking values on a separable Banach space B. Then

3P (|ξ + η| > \frac{2}{3}u) ≥ P (|ξ| > u) for all u > 0.

Proof of Lemma B1. Let ξ, η and ζ three independent, identically distributed random variables taking values in B. Then

3P (|ξ + η| > \frac{2}{3}u) = P (|ξ + η| > \frac{2}{3}u) + P (|ξ + ζ| > \frac{2}{3}u) + P (| − (η + ζ)| > \frac{2}{3}u)
≥ P (|ξ + η + ξ + ζ − η − ζ| > 2u) = P (|ξ| > u).

To prove formula (B2) let us introduce the random variable

T_{n,k}(f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n,s_{j}=1 or s_{j}=2, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})}) = ∑_{V ⊂\{1,...,k\}} Ī_{n,k,V} (f(ℓ)),

(B3) and observe that the random variables I_{n,k}(f(ℓ)), I_{n,k,∅}(f(ℓ)) and I_{n,k,{1,...,k}}(f(ℓ)) are identically distributed and the last two random variables are independent of each other. Hence Lemma B1 yields that

P (∥I_{n,k}(f(ℓ))∥ > u) ≤ 3P (∥I_{n,k,∅}(f(ℓ)) + I_{n,k,\{1,...,k\}}(f(ℓ))∥ > \frac{2}{3}u)

= 3P (∥T_{n,k}(f(ℓ)) − ∑_{V : V ⊂\{1,...,k\}, 1≤|V |≤k−1} I_{n,k,|V |}(f(ℓ))∥ > \frac{2}{3}u)

≤ P (3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ > u)  (B4)

+ ∑_{V : V ⊂{1,...,k}, 1≤|V |≤k−1} P (3 · 2^{k−1}∥I_{n,k,|V |}(f(ℓ))∥ > u).

To deduce relation (B2) from relation (B4) we need a good estimate on the probability P (3 · 2^{k−1}∥T_{n,k}(f(ℓ))∥ > u). We shall compare the distribution of ∥T_{n,k}(f(ℓ))∥ with that of ∥I_{n,k,V} (f(ℓ))∥ for an arbitrary set V ⊂ {1, …, k} and get an estimate which is sufficient to prove relation (B2). To carry out this program first we prove the following lemmas.

Lemma B2. Let us consider a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, which is also independent of the random variables ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} appearing in the definition of the decoupled U-statistics I_{n,k,V} (f(ℓ)) defined in formula (B1). Let us define with their help the sequences of random variables η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)} whose elements (η_{l}^{(1)}, η_{l}^{(2)}) = (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})), 1 ≤ l ≤ n, are given as

(η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (\frac{1 + ε_{l}}{2} ξ_{l}^{(1)} + \frac{1 − ε_{l}}{2} ξ_{l}^{(2)}, \frac{1 − ε_{l}}{2} ξ_{l}^{(1)} + \frac{1 + ε_{l}}{2} ξ_{l}^{(2)}) ,

i.e. let (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(1)}, ξ_{l}^{(2)}) if ε_{l} = 1, and (η_{l}^{(1)}(ε_{l}), η_{l}^{(2)}(ε_{l})) = (ξ_{l}^{(2)}, ξ_{l}^{(1)}) if ε_{l} = −1, 1 ≤ l ≤ n. Then the joint distribution of the pair of sequences of random variables ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} agrees with that of the pair of sequences η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)}.

Let us fix some V ⊂ {1, …, k}, and introduce the random variable

Ī_{n,k,V} (f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}} (η_{l_{1}}^{(α_{V} (1))}, …, η_{l_{k}}^{(α_{V} (k))}) ,  (B5)

where similarly to formula (B1) α_{V} (j) = 1 if j ∈ V, and α_{V} (j) = 2 if j ∉ V. Then the identity

2^{k}Ī_{n,k,V} (f(ℓ))  (B6)

= \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n,s_{j}=1 or s_{j}=2, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} (1 + ε_{l_{1},s_{1},V}^{(1)} ) ⋯ (1 + ε_{l_{k},s_{k},V}^{(k)} )f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})})

holds, where ε_{l,1,V}^{(j)} = ε_{l}, ε_{l,2,V}^{(j)} = −ε_{l} if j ∈ V, and ε_{l,1,V}^{(j)} = −ε_{l}, ε_{l,2,V}^{(j)} = ε_{l} if j ∉ V, 1 ≤ l ≤ n.

In the proof of relation (B2) we need besides Lemma B2 another result given in Lemma B4. Before the formulation of Lemma B4 we present Lemma B3 whose result will be used in its proof.

Lemma B3. Let Z be a random variable in a separable Banach space B with expectation zero, i.e. let Eκ(Z) = 0 for all κ ∈ B′. Then P (∥v + Z∥ ≥ ∥x∥) ≥ inf_{κ∈B′} \frac{(Eκ(Z))^{2}}{4Eκ(Z)^{2}} for all

v ∈ B. Here B′ denotes the (Banach) space of all (bounded) linear transformations on B to the real line.

Lemma B4. Let us consider a sequence of independent random variables ε_{1}, …, ε_{n}, P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, 1 ≤ l ≤ n, a polynomial of order k of these random variables with some coefficients a(l_{1}, …, l_{s}), 1 ≤ s ≤ k, 1 ≤ l_{s} ≤ n, from some separable Banach space B. Let us assume that the coefficients of this polynomial satisfy the relation a(l_{1}, …, l_{s}) = 0 if l_{p} = l_{q} with some 1 ≤ p < q ≤ s, and the constant term is zero. The inequality

P (∥v + ∑_{s=1}^{k} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,s \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}} ∥ > ∥v∥) ≥ c_{k}  (B7)

holds for all v ∈ B with some constant c_{k} > 0 depending only on the order k of this polynomial.

The proof of Lemma B2. Let us consider the conditional joint distribution of the sequences of random variables η_{1}^{(1)}, …, η_{n}^{(1)} and η_{1}^{(2)}, …, η_{n}^{(2)} under the condition that the random vector ε_{1}, …, ε_{n} takes the value of some prescribed ±1 series of length n. Observe that this conditional distribution agrees with the joint distribution of the sequences ξ_{1}^{(1)}, …, ξ_{n}^{(1)} and ξ_{1}^{(2)}, …, ξ_{n}^{(2)} for all possible conditions. This fact implies the statement about the joint distribution of the sequences η_{l}^{(1)}, η_{l}^{(2)}, 1 ≤ l ≤ n.

To prove identity (B6) let us fix a set M ⊂ {1, …, n} and consider the case when ε_{l} = 1 if l ∈ M and ε_{l} = −1 if l ∉ M. Observe that for all fixed sequences 1 ≤ l_{1}, …, l_{k} ≤ n, l_{j} ≠ l_{j′} if j ≠ j′

f_{l_{1},...,l_{k}} (η_{l_{1}}^{(α_{V} (1))}, …, η_{l_{k}}^{(α_{V} (k))}) = f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(β_{V,M} (1,l_{1}))}, …, ξ_{l_{k}}^{(β_{V,M} (k,l_{k}))}) ,

where β_{V,M}(j, l) = 1 if j ∈ V and l ∈ M or j ∉ V and l ∉ M), and β_{V,M}(j, l) = 2 otherwise. On the other hand,

∑_{s_{j}=1 or s_{j}=2, j=1,...,k}(1 + ε_{l_{1},s_{1},V}^{(1)} ) ⋯ (1 + ε_{l_{k},s_{k},V}^{(k)} )f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})})

= 2^{k}f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(β_{V,M} (1,l_{1}))}, …, ξ_{l_{k}}^{(β_{V,M} (k,l_{k}))}) ,

since the product (1 + ε_{l_{1},s_{1},V}^{(1)} ) ⋯ (1 + ε_{l_{k},s_{k},V}^{(k)} ) equals either zero or 2^{k}, and ε_{l_{j},s_{j},V}^{(j)} = 1

if β_{V,M}(j, l_{j}) = s_{j}, and ε_{l_{j},s_{j},V}^{(j)} = −1 if β_{V,M}(j, l_{j}) ≠ s_{j}.

Summing up these identities for all 1 ≤ l_{1}, …, l_{k} ≤ n such that l_{j} ≠ l_{j′} if j ≠ j′ we get identity (B6).

The proof of Lemma B3. Let us first observe that if ξ is a real valued random variable with zero expectation, then P (ξ > 0) ≥ \frac{(E|ξ|)^{2}}{4Eξ^{2}} since (E|ξ|)^{2} = 4(E(ξI(\{ξ > 0\}))^{2} ≤ 4P (ξ > 0)Eξ^{2} by the Schwarz inequality, where I(A) denotes the indicator function of the set A.

Given some v ∈ B let us choose a linear operator κ such that ∥κ∥ = 1 and κ(v) = ∥v∥. Such an operator exists by the Banach–Hahn theorem. Observe that {ω: ∥v + Z(ω)∥ ≥ ∥v∥} ⊃ {ω: κ(v + Z(ω)) ≥ κ(v)} = {ω: κ(Z(ω)) ≥ 0}. Besides, Eκ(Z) = 0. Hence we can apply the above proved inequality for ξ = κ(Z), and it yields that P (∥v + Z∥ ≥ ∥v∥) ≥ \frac{Eκ(Z)^{2}}{4(Eκ(Z))^{2}} . Lemma B3 is proved.

Proof of Lemma B4. Take the class of random polynomials

Y = ∑_{s=1}^{k} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,s \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} b(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}},

where ε_{l}, 1 ≤ l ≤ n, are independent random variables with P (ε_{l} = 1) = P (ε_{l} = −1) = \frac{1}{2}, and the coefficients b(l_{1}, …, l_{s}), 1 ≤ s ≤ k, are arbitrary real numbers. It is enough to show that there exists a constant c_{k} depending only on the order k of these polynomials such that the inequality

(E|Y |)^{2} ≥ 4c_{k}EY^{2}.  (B8)

holds for all of these polynomials Y. Indeed, formula (B7) follows from relation (B8) and Lemma B3 with c_{k} ≥ inf_{κ} \frac{(Eκ(Z))^{2}}{4Eκ(Z)^{2}} if we apply them for the vector v ∈ B in formula (B7) and

Z = ∑_{s=1}^{k} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,s \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} a(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}},

and the infimum is taken for all bounded linear operators κ on the Banach space B. But this inequality follows from relation (B8).

To prove relation (B8) first we compare the moments EY^{2} and EY^{4}. Let us introduce the random variables

Y_{s} = ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,s \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} b(l_{1}, …, l_{s})ε_{l_{1}} ⋯ ε_{l_{s}} 1 ≤ s ≤ k,

and observe that because of Borell’s inequality (Theorem 10.2) and the uncorrelatedness of the random variables Y_{s}, 1 ≤ s ≤ k,

EY^{4} = (∑_{s=1}^{k} Y_{s})^{4} ≤ k^{3} ∑_{s=1}^{k} EY_{s}^{4} ≤ k^{3}3^{3k/2} ∑_{s=1}^{k}(EY_{s}^{2})^{2}

≤ k^{3}3^{3k/2} (∑_{s=1}^{k} EY_{s}^{2})^{2} = k^{3}3^{3k/2}(EY^{2})^{2}.

This estimate together with the Hölder inequality yield that EY^{2} = E(Y^{4})^{1/3}|Y |^{2/3} ≤ (EY^{4})^{1/3}(E|Y |)^{2/3} ≤ k3^{k/2}(EY^{2})^{1/3}(E|Y |)^{2/3}, i.e. EY^{2} ≤ k^{3/2}3^{3k/4}(E|Y |)^{2}, and relation (B8) holds with 4c_{k} = k^{−3/2}3^{−3k/4}. Lemma B4 is proved.

Let us turn back to the estimation of the probability P (3 · 2^{k−1}∥T_{n,k}(f)∥ > u). Let us introduce the σ-algebra F = B(ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n) generated by the random variables ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n, and fix some set V ⊂ {1, …, k}. We claim that there exists some constant c_{k} > 0 that the random variable Ī_{n,k,V} (f(ℓ)) defined in formula (B5) satisfies the inequality

P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F) ≥ c_{k} with probability 1.  (B9)

Indeed, formula (B6) and the independence of the random sequences ε_{l,V}, ξ_{l}^{(1)} and ξ_{l}^{(2)}, 1 ≤ l ≤ n yield that

P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F)

= P_{ε_{V}} (∥\frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n,s_{j}=1 or s_{j}=2, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} (1 + ε_{l_{1},s_{1},V}^{(1)} ) ⋯ (1 + ε_{l_{k},s_{k},V}^{(k)} )f_{l_{1},...,l_{k}}(ξ_{l_{1}}^{(s_{1})}, …, ξ_{l_{k}}^{(s_{k})})∥

> ∥T_{n,k}(f(ℓ))∥),  (B10)

where P_{ε_{V}} means that we fix the values of the random variables ξ_{l}^{(1)}, ξ_{l}^{(2)}, 1 ≤ l ≤ n and take the probability with respect to the remaining random variables ε_{l,s,V}^{(j)} , 1 ≤ j ≤ k, 1 ≤ l ≤ n, and s = 1 or s = 2. Let us observe that the probability considered at the right-hand side of (B10) is a polynomial of order k of the random variables ε_{1}, …, ε_{n}. (The terms ε_{l_{j},s_{j},V}^{(j)} taking part in it equal either ε_{l_{j}} or −ε_{l_{j}} depending on the parameters j and s_{j}.) Besides, the constant term of this polynomial equals T_{n,k}(f). Hence this probability can be bounded by means of Lemma B4, and this result yields relation (B9).

Relation (B9) implies that

P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ ≥ 3 · 2^{k−1}u)

≥ P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ ≥ ∥T_{n,k}(f(ℓ))∥, ∥T_{n,k}(f(ℓ))∥ ≥ 3 · 2^{k−1}u)
= ∫_{{ω: ∥T_{n,k}(f(ℓ))(ω)∥≥3·2^{k−1}u}} P (∥2^{k}Ī_{n,k,V} (f(ℓ))∥ > ∥T_{n,k}(f(ℓ))∥|F) dP
≥ c_{k}P (∥T_{n,k}(f(ℓ))∥ ≥ 3 · 2^{k−1}u)

The last inequality with the choice of any set V ⊂ {1, …, k}, 1 ≤ |V | ≤ k − 1, together with relation (B4) imply formula (B2).

To formulate the inductive hypothesis we need to prove formula (10.8b) with the help of relation (B2) first we introduce the following quantities. Let W = W(k) denote the set of all partitions of the set {1, …, k}. Let us fix k independent copies ξ_{1}^{(j)}, …, ξ_{n}^{(j)}, 1 ≤ j ≤ k, of the sequence of random variables ξ_{1}, …, ξ_{n}. Given a partition W = (V_{1}, …, V_{s}) ∈ W(k) let us introduce the function s_{W} (j), 1 ≤ j ≤ k, which tells for all arguments j the index of that element of the partition W which contains the point j, i.e. the function s_{W} (j), 1 ≤ j ≤ k, is defined by the relation j ∈ V_{s_{W} (j)}. Let us define (actually generalizing the notion introduced in formula (B1)) the notion of generalized decoupled U-statistics corresponding to a partition W ∈ W(k) as

I_{n,k,W} (f(ℓ)) = \frac{1}{k!} ∑_{\begin{matrix}1≤l_{j}≤n, j=1,...,k \\ l_{j}≠l_{j′} if j≠j′ \end{matrix}} f_{l_{1},...,l_{k}} (ξ_{l_{1}}^{(s_{W} (1))}, …, ξ_{l_{k}}^{(s_{W} (k))}) for all W ∈ W(k).

Given a partition W = (V_{1}, …, V_{s}) let us call the number s of the elements of this partition the rank both of the partition W and of the generalized decoupled U-statistic I_{n,k,W} (f(ℓ)).

Relation (10.8b) will be proved by induction with respect to the order k of the U-statistics. This induction assumption clearly holds for k = 1, so when we prove it for k we may assume that it holds for all k′ < k. We prove it by first showing the following statement. Fix the number k. For all numbers 2 ≤ j ≤ k there exist some constants C(k, j) > 0 and δ(k, j) > 0 such that for all generalized decoupled U-statistics I_{n,k,W} (f(ℓ)) of order k

P (∥I_{n,k,W} (f(ℓ))∥ > u) ≤ C(k, j)P (∥Ī_{n,k}(f(ℓ))∥ > δ(k, j)u)

for all 2 ≤ j ≤ k if the rank of W equals j.

(B11)

(In relation (B11) we compare the distribution of some generalized decoupled U-statis-tics with that of the decoupled U-statistic Ī_{n,k}(f(ℓ)).) We shall prove this statement by means of a backward induction with respect to the rank j of the generalized decoupled U-statistics.

Relation (B11) clearly holds for j = k with C(k, k) = 1 and δ(k, k) = 1. To prove it for generalized decoupled U-statistics of rank 2 ≤ j < k first we make the following observation. If the rank j of the partition W = (U_{1}, …, U_{j}) satisfies the relation 2 ≤ j ≤ k − 1, then it contains an element with cardinality strictly less than k and strictly greater than 1. For the sake of simpler notation let us assume that the element U_{j} of this partition is such an element, and U_{j} = {s, …, k} with some 2 ≤ s ≤ k−1. The investigation of general U-statistics of rank j, 2 ≤ j ≤ k − 1 can be reduced to this case by a reindexation of the random arguments in the U-statistics if it is necessary. Let us consider the partition W̄ = (U_{1}, …, U_{j−1}, {s}, …, {k}) and the generalized decoupled U-statistic I_{n,k,W̄} (f(ℓ)) corresponding to this partition W̄. We show that our inductive hypothesis implies the inequality

P (∥I_{n,k,W} (f(ℓ))∥ > u) ≤ Ā(k)P (∥I_{n,k,W̄} (f(ℓ))∥ > γ̄(k)u)  (B12)

with Ā(k) = sup_{j≤k−1} A(j), γ(k) = inf_{j≤k−1} γ(j) if the rank j of W is such that 2 ≤ j ≤ k −1.

To prove relation (B12) let us define the σ-algebra F generated by the random variables appearing in the first s − 1 coordinates of these generalized U-statistics. We show that relation (10.8b) for U-statistics of order k − s + 1 ≤ k − 1 yields that P (∥I_{n,k,W} (f(ℓ))∥ > u|F) ≤ Ā(k)P (∥I_{n,k,W̄} (f(ℓ))∥ > γ̄(k)u|F) with probability 1. This inequality follows from our inductive hypothesis, since the conditional probabilities we compare here are generalized U-statistics and generalized decoupled U-statistics of order k − s + 1 we get by putting substituting the (known) first s − 1 coordinates in the generalized U-statistics I_{n,k,W} (f(ℓ)) and I_{n,k,W̄} (f(ℓ)). Then taking expectation at both sides of this inequality we get relation (B12). As the rank of W̄ is strictly greater than the rank of W relation (B12) together with our backward inductive assumption imply relation (B11) for all 2 ≤ j ≤ k.

Inequality (10.8b) is a simple consequence of relations (B2) and (B11). Indeed, the probability P (∥I_{n,k}(f(ℓ))∥ > u) is bounded in formula (B2) by such an expression, where some linear combination of the probabilities are considered that certain generalized decoupled U-statistics of order k and rank 2 are larger than uD_{k}^{−1}. Each of these terms can be bounded by means of relation (B11), and in such a way we get relation (10.8b).

We prove formula (10.8′) first in the simpler case when the supremum of finitely many functions is taken. Let us have M functions f_{1}, …, f_{M}, and to prove relation (10.8′) in this case let us apply formula (10.8) with the function f = (f_{1}, …, f_{M}) taking values in the separable Banach space B_{M} consisting of the points (v_{1}, …, v_{M}), v_{j} ∈ B, 1 ≤ j ≤ M, with the norm ∥(v_{1}, …, v_{M})∥ = sup_{1≤j≤m} ∥v_{j}∥. The application of

formula (10.8) with this choice yields formula (10.8′) in this case. Let us emphasize that the constants appearing in this estimate do not depend on the number M. Since the distribution of the random variables sup_{1≤s≤M} ∥I_{n,k}(f_{s})∥ converge to sup_{1≤s<∞} ∥I_{n,k}(f_{s})∥, the

distribution of the random variables sup_{1≤s≤M} ∥Ī_{n,k}(f_{s})∥ converge to sup_{1≤s<∞} ∥Ī_{n,k}(f_{s})∥ as

M → ∞, we get the proof of relation (10.8′) in the general case by taking limit M → ∞ in this relation.

Appendix C.

Nelson’s inequality and its application

In this part of the Appendix I formulate and prove Nelson’s inequality and briefly indicate how it can be applied in the proof of Theorem 8.5, i.e. in the Gaussian counterpart of Theorems 8.3 and 8.4. As the latter problem does not belong to the main subject of the work, the detailed explanation of some background results I shall apply in the proof will be omitted. In particular, I do not discuss the basic results about the properties of multiple Wiener–Itô integrals. These results can be found for instance in my lecture note Multiple Wiener–Itô integrals.

There are several equivalent formulations of Nelson’s inequality. First I present its terminologically simplest form. Before its formulation let me recall that the Hermite polynomials H_{k}(x), k = 0, 1, 2, …, are those polynomials which constitute an orthogonal system with respect to the normal density function φ(x) = \frac{1}{\sqrt{2π}} e^{−x^{2}/2}. To fix their normalization, let us make the agreement that H_{k}(x) is a polynomial of order k, and the coefficient of its leading term x^{k} equals 1.

Theorem C1. (Nelson’s inequality). Let (Y, Y, ν) = (R^{∞}, B^{∞}, ν^{∞}) be the direct product of infinite many copies of the space (R, B, λ_{φ}), where R denotes the real line, B is the Borel σ-algebra on it, λ_{φ} is the measure determined by the standard normal distribution function, i.e. the probability measure which is absolutely continuous with respect to the Lebesgue measure with density function φ(y) = \frac{1}{\sqrt{2π}} e^{−y^{2}/2}.

Given a number γ > 0 introduce the operator T_{γ} on (Y, Y) by defining it first on polynomials by the formula

T_{γ} (∑ c_{l_{1},j_{1},...,l_{s},j_{s}}H_{l_{1}}(y_{j_{1}}) ⋯ H_{l_{s}}(y_{j_{s}}))
= ∑ γ^{l_{1}+···+l_{s}}c_{l_{1},j_{1},...,l_{s},j_{s}}H_{l_{1}}(y_{j_{1}}) ⋯ H_{l_{s}}(y_{j_{s}}),

where all finite sums of the above form are considered, and H_{l}(·) denotes the Hermite polynomial of order l. Let us extend this linear operator to general functions on the space (Y, Y) in the natural way.

Fix two numbers 1 < q ≤ p < ∞ and a number γ ≤ \sqrt{\frac{q−1}{p−1}}. Then the operator T_{γ} defined above, considered as a linear operator from the space L_{q}(Y, Y, ν) to L_{p}(Y, Y, ν) is a contraction, i.e. ∥T_{γ}(f)∥_{p} ≤ ∥f∥_{q} for all functions f ∈ L_{q}(Y, Y, ν).

By Theorem 11.2 Nelson’s inequality can be reduced to the following one-dimen-sional inequality

∫_{−∞}^{∞} |∑_{l=0}^{s} c_{l}γ^{l}H_{l}(x)|^{p} \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx ≤ [∫_{−∞}^{∞} |∑_{l=0}^{s} c_{l}H_{l}(x)|^{q} \frac{1}{\sqrt{2π}} e^{−x^{2}/2} dx]^{p/q} (C1)

for all finite polynomials ∑_{l=0}^{s} c_{l}H_{l}(x) if 1 < q ≤ p < ∞, and γ ≤ \sqrt{\frac{q−1}{p−1}}.

We shall prove inequality (C1) in a seemingly more complicated equivalent form in the following Proposition C2. Proposition C2 can be proved by means of the hypercontractive inequality for Rademacher functions, the central limit theorem and by some basic results in the theory of multiple Wiener–Itô integrals.

Proposition C2. Let us consider a Wiener process W (t) on the interval [0, 1], and consider multiple Wiener–itô integrals with respect to it. The inequality

E |∑_{l=0}^{s} c_{l}γ^{l} ∫ W ( dx_{1}) ⋯ W ( dx_{l})|^{p} ≤ [E |∑_{l=0}^{s} c_{l} ∫ W ( dx_{1}) ⋯ W ( dx_{l})|^{q}]^{p/q} (C2)

holds for all numbers s and coefficients c_{l}, 0 ≤ l ≤ s if 1 < q ≤ p < ∞, and γ ≤ \sqrt{\frac{q−1}{p−1}}.

Remark: Relations (C1) and (C2) are equivalent. To show this observe that by Itô’s formula for multiple Wiener–Itô integrals ∫ W ( dx_{1}) ⋯ W ( dx_{l}) = H_{l} (∫ W ( dx)). Besides, the random variable ξ = ∫ W ( dx) = W (1) − W (0) has standard normal distribution, and formula (C2) can be rewritten with its help as

E |∑_{l=0}^{s} c_{l}γ^{l}H_{l}(ξ)|^{p} ≤ [E |∑_{l=0}^{s} c_{l}H_{l}(ξ)|^{q}]^{p/q}

which is clearly equivalent to relation (C1).

The proof of Proposition C2. First we want to show a version of formula (C2) where the multiple Wiener–Itô integrals are replaced by appropriate approximations of these integrals. For this goal let us consider m independent, normally distributed random variables ξ_{1}, …, ξ_{m} with expectation zero and variance \frac{1}{m}. We shall prove with the help of the hypercontractive inequality for Rademacher functions and the central limit theorem (more precisely a slight generalization of it) the following inequality:

E |∑_{l=0}^{s} c_{l}γ^{l} ∑_{\begin{matrix}1≤j_{1},...,j_{l}≤m \\ j_{u}≠j′_{u} if u≠u′, 1≤u,u′≤m \end{matrix}} ξ_{j_{1}} ⋯ ξ_{j_{l}} |^{p} ≤ [E |∑_{l=0}^{s} c_{l} ∑_{\begin{matrix}1≤j_{1},··· ,j_{l}≤m \\ j_{u}≠j′_{u} if u≠u′, 1≤u,u′≤m \end{matrix}} ξ_{j_{1}} … ξ_{j_{l}} |^{q}]^{p/q} (C3)

for all s and coefficients c_{s}, 1 ≤ s ≤ l, if 1 < q ≤ p < ∞, and γ ≤ \sqrt{\frac{q−1}{p−1}}.

To prove relation (C3) let us choose for all n = 1, 2, … a sequence of independent random variables ε_{1}, …, ε_{mn} such that P (ε_{j} = 1) = P (ε_{j} = −1) = \frac{1}{2}, 1 ≤ j ≤

mn, and define the random variables Z_{j}^{(n)} = \frac{1}{\sqrt{mn}} ∑_{k=(j−1)n+1}^{jn} ε_{j}, 1 ≤ j ≤ m. The

hypercontractive inequality for Rademacher functions implies that

E |∑_{l=0}^{s} c_{l}γ^{l} ∑_{\begin{matrix}1≤j_{1},...,j_{l}≤m \\ j_{u}≠j′_{u} if u≠u′, 1≤u,u′≤m \end{matrix}} Z_{j_{1}}^{(n)} ⋯ Z_{j_{l}}^{(n)}|^{p} ≤ [E |∑_{l=0}^{s} c_{l} ∑_{\begin{matrix}1≤j_{1},...,j_{l}≤m \\ j_{u}≠j′_{u} if u≠u′, 1≤u,u′≤m \end{matrix}} Z_{j_{1}}^{(n)} ⋯ Z_{j_{l}}^{(n)}|^{q}]^{p/q}

(C4)

By the central limit theorem the random vectors (Z_{1}^{(n)}, …, Z_{m}^{(n)}) converge in distribution to the random vector (ξ_{1}, …, ξ_{m}) as n → ∞. This convergence in distribution also can be expressed as the relation

lim_{n→∞} Ef(Z_{1}^{(n)}, …, Z_{m}^{(n)}) = Ef(ξ_{1}, …, ξ_{m})  (C5)

for all bounded and continuous functions f on R^{m}. Moreover, it can be proved that the distribution of the random vectors Z_{j}^{(n)} converge to zero sufficiently fast at infinity, more explicitly for all K > 0 there exists some constant C = C(K) > 0 such that P (|Z_{m}^{(n)}| > x) ≤ Cx^{K} for all x ≥ 1 and n = 1, 2, . . .. This fact implies that relation (C5) also holds for continuous functions f such that |f(x)| ≤ C(1 + |x|)^{K}) with some constant C > 0 and K > 0, where x = (x_{1}, …, x_{m}) and |x| is the length of the vector x. This strengthened form of (C5) enables us to take the limit n → ∞ in formula (C4) and to get relation (C3) in such a way.

Let us apply formula (C3) with the choice ξ_{j} = W (\frac{j}{m}) − W (\frac{j−1}{m}), 1 ≤ j ≤ m. Observe that for all indices l the inner sums at both sides of this expression are approximative sums for the Wiener–Itô integral ∫ W ( dx_{1}) ⋯ W ( dx_{l}). Hence it is natural to expect that by applying the limiting procedure m → ∞ in formula (C3) with the above choice of the random variables ξ_{j} we get relation (C2). This belief is correct, only its justification requires the application of some deeper results from the theory of Wiener–Itô integrals. We need some estimate which states that also the high moments of a Wiener– Itô integral with a small kernel function are small. We can apply the following result. If h is such a function in [0, 1]^{l} for which ∫ |h(x_{1}, …, x_{l})|^{2} dx_{1} … dx_{l} < ε with some ε > 0, then also the inequality E |∫ h(x_{1}, …, x_{l})W ( dx_{1}) … W ( dx_{l})|^{2K} ≤ C(K, l)ε^{K} holds for all K = 1, 2, … with some constant C(K, l) depending only on K and l. But the proof of this estimate demands some deeper results about Wiener–Itô integrals. (In my lecture note about Wiener–Itô integrals this result is proved as a consequence of the so-called diagram formula.) By applying this limiting procedure we get the proof of (C2). In such a way we have proved Proposition C2 which, as we have shown, implies Theorem C1.

Now I formulate a version of Nelson’s inequality presented in the language of Wiener–Itô integrals.

Theorem C3. Let us fix a measurable space (X, X) together with a countable nonatomic measure μ on it, and let Z_{μ} be an orthogonal Gaussian random measure with counting measure μ on (X, X). (See the definition of counting measure before the formulation of Theorem (8.5).) For the sake of simplicity let us assume that the space L_{2}(X, X , μ) is separable.

Let us have a sequence of measurable functions f_{k}(x_{1}, …, x_{k}) on (X^{k}, X^{k}) of real constant c_{k}, k = 1, 2, …, and also a constant c_{0} such that

c_{0}^{2} + ∑_{k=1}^{∞} \frac{c_{k}^{2}}{k!} ∫ f_{k}^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) < ∞.  (C6)

Then

\begin{matrix}E |c_{0} + ∑_{k=1}^{∞} γ^{k} \frac{c_{k}}{k!} ∫ f_{k}(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k})|^{p} \\ ≤ [E |c_{0} + ∑_{k=1}^{∞} \frac{c_{k}}{k!} ∫ f_{k}(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k})|^{q}]^{p/q} . \end{matrix}  (C7)

if 1 < q ≤ p < ∞, and γ ≤ \sqrt{\frac{q−1}{p−1}}.

Inequality (C7) means in particular that if the right-hand side is finite then the left-hand side is also finite.

The proof of Theorem C3. Theorem C3 will be proved as the consequence of Theorem C1 and Itô’s formula for multiple Wiener–Itô integrals. Let us choose a complete orthonormal system ψ_{1}(x), ψ_{2}(x), ⋯ in the space L_{2}(X, X , μ), and define the random variables ξ_{n} = ∫ ψ(x)Z_{μ}( dx). Then ξ_{1}, ξ_{2}, … is a sequence of independent random variables with standard normal distribution, and U(ω) = (ξ_{1}(ω), ξ_{2}(ω), …) is a measure preserving transformation of the probability space (Ω, A, P) where the orthonormal Gaussian random measure Z_{μ} is defined to the space (Y, Y, ν) introduced in the formulation of Theorem C1. We express the Wiener–Itô integrals

V_{k} = ∫ f_{k}(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k}), 1 ≤ k < ∞,

by means of Itô’s formula as a function of the Hermite polynomials of the random variables ξ_{j}, 1 ≤ j < ∞, and then we deduce Theorem C3 from Theorem C1 by means of the above introduced measure preserving transformation U.

To carry out this program let us expand the function f_{k}(x_{1}, …, x_{k}) by means of the complete orthonormal system consisting of the products ψ_{j_{1}}(x_{1}) ⋯ ψ_{j_{k}}(x_{k}) 1 ≤ j_{s} < ∞, j = 1, …, k, in the space (X^{k}, X^{k}, μ^{k}). We can write

f_{k}(x_{1}, …, x_{k}) = ∑_{s=1}^{k} ∑_{\begin{matrix}(j_{1},...,j_{s}), (l_{1},...,l_{s}) \\ j_{u}≥1, l_{u}≥1, 1≤u≤s, j_{1}+···+j_{s}=k \\ l_{u}≠l_{u′} if u≠u′, 1≤u,u′≤s \end{matrix}} d_{j_{1},...,j_{s},l_{1},...,l_{s}}F_{j_{1},...,j_{s},l_{1},...,l_{s}}(x_{1}, …, x_{k})

with some appropriate coefficients d_{j_{1},...,j_{s},l_{1},...,l_{s}} and

F_{j_{1},...,j_{s},l_{1},...,l_{s}}(x_{1}, …, x_{k}) = ∏_{u=1}^{s} ψ_{l_{u}}(x_{J(u−1)+1})ψ_{l_{u}}(x_{J(u−1)+1}) ⋯ ψ_{l_{u}}(x_{J(u)}),

where J(0) = 0 and J(u) = j_{1} + ⋯ + j_{u}, 1 ≤ u ≤ s.

By Itô’s formula ∫ F_{j_{1},...,j_{s},l_{1},...,l_{s}}(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k}) = ∏_{u=1}^{s} H_{j_{u}}(ξ_{l_{u}}),

and ∫ γ^{k}F_{j_{1},...,j_{s},l_{1},...,l_{s}}(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k}) = ∏_{u=1}^{s} γ^{j_{u}}H_{j_{u}}(ξ_{l_{u}}).

By summing up the above inequalities we get that

V_{k} = U^{∗} (∑_{s=1}^{k} ∑_{\begin{matrix}(j_{1},...,j_{s}), (l_{1},...,l_{s}) \\ j_{u}≥1, l_{u}≥1, 1≤u≤s, j_{1}+···+j_{s}=k \end{matrix}} d̄_{j_{1},...,j_{s},l_{1},...,l_{s}}H_{j_{1}}(y_{l_{1}}) ⋯ H_{j_{s}}(y_{l_{s}}))

and

γ^{k}V_{k} = U^{∗} (∑_{s=1}^{k} ∑_{\begin{matrix}(j_{1},...,j_{s}), (l_{1},...,l_{s}) \\ j_{u}≥1, l_{u}≥1, 1≤u≤s, j_{1}+···+j_{s}=k \end{matrix}} d̄_{j_{1},...,j_{s},l_{1},...,l_{s}}γ^{j_{1}}H_{j_{1}}(y_{l_{1}}) ⋯ γ^{j_{s}}H_{j_{s}}(y_{l_{s}}))

with some coefficients d̄_{j_{1},...,j_{s},l_{1},...,l_{s}}, where U^{∗} denotes the operator from the space of functions on (Y, Y, ν) to the space of functions on (Ω, A, P) induced by the measure preserving transformation U. Summing up these identities for all k = 0, 1, 2, … and exploiting the measure preserving property of the transformation U we get that Theorem C1 implies Theorem C3. (Let me remark that condition (C6) was imposed only to guarantee that the infinite sum of the Wiener–Itô integrals we considered really exists.)

The proof of formula (8.11) in Theorem 8.5 is fairly simple with the help of Nelson’s inequality.

The proof of formula (8.11). Let us observe that relation (B7) with q = 2, p = 2M yields that for a k-fold Wiener–Itô integral

\begin{matrix}E |\frac{1}{k!} ∫ f(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k})|^{2M} \\ ≤ (2M − 1)^{kM} [E |\frac{1}{k!} ∫ f(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k})|^{2}]^{M} \\ = (2M − 1)^{kM} (\frac{1}{k!} ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}))^{M} if M ≥ 1. \end{matrix} (C8)

In the last line of formula (C8) we exploited the following simple but basic relation of the theory of Wiener–Itô integrals:

E (\frac{1}{k!} ∫ f(x_{1}, …, x_{k})Z_{μ}( dx_{1}) … Z_{μ}( dx_{k}))^{2} = \frac{1}{k!} ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}).

Relation (C8) and the Markov inequality imply that under the conditions of Theorem 8.5

P (|Z_{μ,k}(f)| > u) ≤ \frac{EZ_{μ,k}(f)^{2M}}{u^{2M}} ≤ (\frac{(2M)^{k}σ^{2}}{u^{2}} )^{M}

if M ≥ 1. We get with the choice M = \frac{1}{2e} (\frac{u}{σ})^{2/k} that

P (|Z_{μ,k}(f)| > u) ≤ exp \{−\frac{k}{2e} (\frac{u}{σ} )^{2/k}\} if u > (2e)^{k/2}σ.

By choosing a sufficiently large A ≥ 1 at the right-hand side of this inequality we get that formula (8.11) holds for all u ≥ 0.

The second inequality (8.12) of Theorem 8.5 can be proved in the same way as Theorem 4.1 in the one-dimensional case. No difficulty arises during the proof. The main point is that inequality (8.11) holds for all u > 0, hence the chaining argument applied in the proof of Theorem 4.1 supplies the proof also in this case. I omit the details.

Let me finally remark that Leonhard Gross in his paper Logarithmic Sobolev inequalities also gave a proof of the Nelson inequality by means of the hypercontractive inequality for Rademacher functions. He showed that the central limit theorem enables us to prove that the logarithmic Sobolev inequality holds not only for the Markov process considered in Section 11, but also for Wiener processes. This result together with the general theory he presents imply an inequality which is equivalent to our formula (C1).

References:

1.) Alexander, K. (1987) The central limit theorem for empirical processes over Vapnik–

Červonenkis classes. Ann. Probab. 15, 178–203 2.) Arcones, M. A. and Giné, E. (1993) Limit theorems for U-processes. Ann. Probab.

21, 1494–1542 3.) Arcones, M. A. and Giné, E. (1994) U-processes indexed by Vapnik–Červonenkis

classes of functions with application to asymptotics and bootstrap of U-statistics with estimated parameters. Stoch. Proc. Appl. 52, 17–38

4.) Bennett, G. (1962) Probability inequality for the sum of independent random vari-

ables. J. Amer. Statist. Assoc. 57, 33-45 5.) Bonami, A. (1970) Étude des coefficients de Fourier des fonctions de L^{p}(G). Ann.

Inst. Fourier 20, 335–402 6.) de la Peña, V. H. and Giné, E. (1999) Decoupling. From dependence to inde-

pendence. Springer series in statistics. Probability and its application. Springer Verlag, New York, Berlin, Heidelberg

7.) de la Peña, V. H. and Montgomery–Smith, S. (1995) Decoupling inequalities for

the tail-probabilities of multivariate U-statistics. Ann. Probab., 23, 806–816 8.) Dudley, R. M. (1978) Central limit theorems for empirical measures. Ann. Probab.

6, 899–929 9.) Dudley, R. M. (1984) A course on empirical processes. Lecture Notes in Mathe-

mematics 1097, 1–142 Springer Verlag, New York 10.) Dudley, R. M. (1989) Real Analysis and Probability. Wadsworth & Brooks, Pacific

Grove, California 11.) Dudley, R. M. (1998) Uniform Central Limit Theorems. Cambridge University

Press, Cambridge U.K. 12.) Dynkin, E. B. and Mandelbaum, A. (1983) Symmetric statistics, Poisson processes

and multiple Wiener integrals. Annals of Statistics 11, 739–745 13.) Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. 97, 1061–1083 14.) Guionnet, A. and Zegarlinski, B. (2003) Lectures on Logarithmic Sobolev inequal-

ities. Lecture Notes in Mathematics 1801 1–134 2. Springer Verlag, New York 15.) Hoeffding, W. (1963) Probability inequalities for sums of bounded random vari-

ables. J. Amer. Math. Society 58, 13–30 16.) Ledoux, M. (1996) On Talagrand deviation inequalities for product measures.

ESAIM: Probab. Statist. 1. 63–87. Available at http://www.emath./fr/ps/. 17.) Major, P. (1981) Multiple Wiener–Itô integrals. Lecture Notes in Mathematics

849, Springer Verlag, Berlin, Heidelberg, New York, 18.) Major, P. (1988) On the tail behaviour of the distribution function of multiple

stochastic integrals. Probability Theory and Related Fields, 78, 419–435 19.) Major, P. (2005) An estimate about multiple stochastic integrals with respect to a

normalized empirical measure. Submitted to Studia Scientarum Mathematicarum Hungarica.

20.) Major, P. (2005) An estimate on the maximum of a nice class of stochastic integrals.

Submitted to Probability Theory and Related Fields, 21.) Major, P. (2005) On a multivariate version of Bernstein’s inequality submitted to

Ann. Probab. 22.) Major, P. (2005) A multivariate generalization of Hoeffding’s inequality. Submitted

to Ann. Probab. 23.) Major, P. (2005) On the tail behaviour of multiple random integrals and U-sta-

tistics, on the supremum of classes of such quantities, and some related questions. (An overview work submitted to xxx)

24.) Major, P. and Rejtő, L. (1988) Strong embedding of the distribution function under

random censorship. Annals of Statistics, 16, 1113–1132 25.) Major, P. and Rejtő, L. (1998) A note on nonparametric estimations. In the con-

ference volume to the 65. birthday of Miklós Csörgő. 759–774 26.) Massart, P. (2000) About the constants in Talagrand’s concentration inequalities

for empirical processes. Ann. Probab. 28, 863–884 27.) Nelson, E. (1973) The free Markov field. J. Functional Analysis 12, 211–227 28.) Pollard, D. (1984) Convergence of Stochastic Processes. Springer Verlag, New York 29.) Talagrand, M. (1996) New concentration inequalities in product spaces. Invent.

Math. 126, 505–563 30.) Vapnik, V. N. (1995) The Nature of Statistical Learning Theory. Springer Verlag,

New York

Content

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1
2. Motivation of the investigation. Discussion of some problems …  3
3. Some estimates about sums of independent random variables … 10
4. On the supremum of a nice class of partial sums . . . . . . . . . . . . . . . 15
5. Vapnik–Červonenkis classes and L_{2}-dense classes of functions . . 23

6. The proof of Theorems 4.1 and 4.2 on the supremum of random

sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

7. The completion of the proof of Theorem 4.1 . . . . . . . . . . . . . . . . . . . 33
8. Formulation of the main results of this work . . . . . . . . . . . . . . . . . . . 40
9. Some results about U-statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
10. The proof of Theorem 8.3 about the distribution of U-statistics  60
11. Some useful basic results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
12. Reduction of the main result in this work . . . . . . . . . . . . . . . . . . . . . . 79
13. The strategy of the proof for the main result of this paper . . . . . 87
14. A symmetrization argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
15. The proof of the main result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
16. The improvement of some results in Section 8. . . . . . . . . . . . . . . . . . 115
17. An overview of the results in this work . . . . . . . . . . . . . . . . . . . . . . . . 121

Appendix A. The proof of some results about Vapnik–Červonenkis

classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Appendix B. The proof of Theorem 10.3. (A result of de le Peña

and Montgomery–Smith) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Appendix C. Nelson’s inequality and its application . . . . . . . . . . . . 143
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148