An estimate about multiple stochastic integrals with respect to a normalized empirical measure

Péter Major

Alfréd Rényi Mathematical Institute of the Hungarian Academy of Sciences Budapest, P.O.B. 127 H–1364, Hungary, e-mail: major@renyi.hu

AMS classification numbers 60E15, 60F10, secondary 60H05 keywords: multiple random integrals, degenerate U-statistics, Wiener–Itô integrals, diagram formula for product of multiple random integrals

Supported by the Hungarian Scientific Research Foundation (OTKA) Nr. 03786

Summary: Let a sequence of iid. random variables ξ_{1}, …, ξ_{n} be given on a measurable space (X, X) with distribution μ together with a function f(x_{1}, …, x_{k}) on the product space (X^{k}, X^{k}). Let μ_{n} denote the empirical measure defined by these random variables and consider the random integral

J_{n,k}(f) = \frac{n^{k/2}}{k!} ∫ ′ f(u_{1}, …, u_{k})(μ_{n}( du_{1}) − μ(du_{1})) … (μ_{n}( du_{k}) − μ( du_{k})),

where prime means that the diagonals are omitted from the domain of integration. A good bound is given on the probability P (|J_{n,k}(f)| > x) for all x > 0 which is similar to the estimate in the analogous problem we obtain by considering the Gaussian (multiple) Wiener–Itô integral of the function f. The proof is based on an adaptation of some methods of the theory of Wiener–Itô integrals. In particular, a sort of diagram formula is proved for the random integrals J_{n,k}(f) together with some of its important properties, a result which may be interesting in itself. The relation of this paper to some results about U-statistics is also discussed.

1. Introduction, formulation of the main results

The investigation of the questions discussed in this paper was motivated by some problems of non-parametric maximum likelihood estimates. (See [5], [6].) The following problem is studied here: Let μ be a probability measure on a measurable space (X, X), and take a sequence ξ_{1}, …, ξ_{n} of independent, identically distributed (X, X) valued random variables with distribution μ. Let us introduce the empirical measure μ_{n},

μ_{n}(A) = \frac{1}{n}#\{j: ξ_{j} ∈ A, 1 ≤ j ≤ n\}, A ∈ X

of this sample ξ_{1}, …, ξ_{n}, and given a function f(x_{1}, …, x_{k}) on the k-fold product space (X^{k}, X^{k}) define the integral J_{n,k}(f) of the function f with respect to the k-fold product of the normalized empirical measure μ_{n} by the formula

J_{n,k}(f) = \frac{n^{k/2}}{k!} ∫ ′ f(u_{1}, …, u_{k})(μ_{n}( du_{1}) − μ( du_{1})) … (μ_{n}( du_{k}) − μ( du_{k})),

where the prime in ∫′ means that the diagonals u_{j} = u_{l}, 1 ≤ j < l ≤ k,

are omitted from the domain of integration.  (1.1)

Let us consider constants as functions of zero variable, and define J_{n,0}(c) = c for a constant c. Given a function f(x_{1}, …, x_{k}) of k variables we want to give a good estimate on the probability P (|J_{n,k}(f)| > x) for all x ≥ 0. Our main result is the following

Theorem 1. Let f = f(x_{1}, …, x_{k}) be a measurable function on the space (X^{k}, X^{k}, μ^{k}) with some k ≥ 1 such that

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,

and

∥f∥_{2}^{2} = Ef^{2}(ξ_{1}, …, ξ_{k}) = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2}  (1.2)

with some constant σ > 0. Let us also assume that the measure μ on (X, X) is nonatomic. Then there exist some constants C = C_{k} > 0 and α = α_{k} > 0, such that the random integral J_{n,k}(f) defined by formula (1.1) satisfies the inequality

P (|J_{n,k}(f)| > x) ≤ C max (exp \{−α (\frac{x}{σ} )^{2/k}\} , exp \{−α(nx^{2})^{1/(k+1)}\})  (1.3)

for all x > 0. These constants C = C_{k} > 0 and α = α_{k} > 0 depend only on the parameter k.

It may be useful to reformulate this result in the following equivalent form:

Theorem 1′. Under the conditions of Theorem 1

P (|J_{n,k}(f)| > x) ≤ C exp \{−α (\frac{x}{σ} )^{2/k}\} for x ≤ n^{k/2}σ^{k+1}

with the number σ appearing in (1.2) and some universal constants C = C_{k} > 0, α = α_{k} > 0, depending only on the multiplicity k of the integral J_{n,k}(f).

Theorem 1 clearly implies Theorem 1′, since in the case x ≤ n^{k/2}σ^{k+1} the first term is larger than the second one in the maximum at the right-hand side of formula (1.3). On the other hand Theorem 1′ implies Theorem 1 also if x > n^{k/2}σ^{k+1}, since in this case Theorem 1′ can be applied with σ̄ = (xn^{−k/2})^{1/(k+1)} ≥ σ. This yields that

P (|J_{n,k}(f)| > x) ≤ C exp \{−α (\frac{x}{σ̄} )^{2/k}\} = C exp \{−α(nx^{2})^{1/(k+1)}\} if x > n^{k/2}σ^{k+1}.

As it will be seen later, the expression ∥f∥_{2}^{2} considered in formula (1.2) has the same order as the variance of the random integral J_{n,k}(f). Beside this, if η is a random

variable with standard normal distribution, then P (|ση^{k}| > x) = P (|η| > (\frac{x}{σ} )^{1/k}) ≤

const. exp \{− (\frac{x}{σ} )^{2/k}\}, and this inequality is essentially sharp. Thus the results of

Theorem 1 or Theorem 1′ state that the tail probability P (|J_{n,k}(f)| > x) of the k-fold random integral J_{n,k}(f) behaves similarly to P (|const. ση^{k}| > x), where the random variable η has standard normal distribution and σ is the variance of J_{n,k}(f), provided that the level x we consider is less than n^{k/2}σ^{k+1}. It can be shown that such a condition on the level x is really needed.

Actually, I am interested in the following more general problem: Let a nice class of functions f ∈ F be given on the space (X^{k}, X^{k}), and let us give a good bound on

the probability P (sup_{f∈F} |J_{n,k}(f)| ≥ x). In a subsequent paper [7] I shall show that for

nice classes F of functions, for instance if F is a so-called Vapnik–Červonenkis class of functions bounded by 1, a bound similar to that in Theorem 1 can be given for this maximum. But to show this first Theorem 1 of this paper has to be proved.

I met such a problem when I tried to apply the method of proof for the existence of a Gaussian limit of the maximum likelihood estimates to certain non-parametric problems. (See [5] and [6].) The proof in the parametric case contains a simple but important linearization step. In this step the function appearing in the maximum likelihood equation is replaced by its Taylor expansion around the (unknown) parameter up to the first term, and it is shown that such a linearization causes only a negligibly small error. In my attempts to adapt this argument to the non-parametric case the result of Theorem 1 had to be applied. Actually this estimate was needed only in the case k = 2. With its help a bound can be obtained which corresponds to the estimate of the second coefficient in the Taylor expansion of the classical maximum likelihood equation. Also the problem discussed in paper [7] appears in certain non-parametric estimation problems. For instance in the estimation of a distribution function with the help of some observations such a problem appears in a natural way. In this case we have to bound the difference between the distribution function and its estimate for all numbers x, and this requires a different linearization for all numbers x. In this problem we want to estimate the supremum of the error in all points x, and this requires a bound on the supremum of a class of random integrals J_{n,2}(f).

Earlier I could only prove a weaker version of Theorem 1 in paper [4] and applied this result. In that paper I could not prove a better estimate for random integrals J_{n,k}(f) with a small variance. I do not know of other papers where the distribution of the random integral J_{n,k}(f) was investigated directly. On the other hand, some deep and interesting results were proved about the tail-behaviour of U-statistics, more precisely about the tail-behaviour of so-called degenerated U-statistics with canonical kernel functions (see [1], [2]), and they are very close to our results. It may be worthwhile to discuss the relation between them in more detail. To do this first I recall some notions about U-statistics.

Let us consider a function f = f(x_{1}, …, x_{k}) defined on the k-th power (X^{k}, X^{k}) of a space (X, X) together with a sequence of independent and identically distributed random variables ξ_{1}, ξ_{2}, … which take their values on this space (X, X), and let μ denote their distribution. We define with their help the U-statistic I_{n,k}(f)

I_{n,k}(f) = \frac{1}{k!} ∑_{\begin{matrix}1≤j_{s}≤n, s=1,...,k \\ j_{s}≠j_{s′} if s≠s′ \end{matrix}} f (ξ_{j_{1}}, …, ξ_{j_{k}}) .  (1.4)

(The function f in this formula will also be called a kernel function in the sequel.)

A real valued function f = f(x_{1}, …, x_{k}) on the k-th power (X^{k}, X^{k}) of a space (X, X) is called a canonical kernel function (with respect to a probability measure μ on the space (X, X)) if

∫ f(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{k})μ( du) = 0 for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

In an equivalent form this means that if ξ_{j}, 1 ≤ j ≤ k, are independent random variables with distribution μ, then E(f(ξ_{1}, …, ξ_{k})|ξ_{1} = x_{1}, …, ξ_{j−1} = x_{j−1}, ξ_{j+1} = x_{j+1}, …, ξ_{k} = x_{k}) = 0 for all 1 ≤ j ≤ k and x_{s} ∈ X, s ≠ j.

The above definition slightly differs from that given in the literature, where generally it is assumed that the kernel function in the U-statistic is symmetric. But this difference is not essential. It is not difficult to see that I_{n,k}(f) = I_{n,k}(Sym f), and our definition is equivalent to the usual one in the case of symmetric kernel functions.

The definition of the integral J_{n,k}(f) contains a certain kind of regularization, since integration is taken with respect to the k-th power of the signed measure μ_{n} − μ in it. On the other hand, the definition of U-statistics contains no such regularization. This has the consequence that only U-statistics with a canonical kernel function satisfy an estimate similar to Theorem 1. Theorem 1 has the following consequence.

Theorem 2. If f = f(x_{1}, …, x_{k}) is a canonical function on the space (X^{k}, X^{k}, μ^{k}),

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1

and

∥f∥_{2}^{2} = Ef^{2}(ξ_{1}, …, ξ_{k}) = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2},

then there exist some constants C = C_{k} > 0 and α = α_{k} > 0 such that the U-statistic I_{n,k}(f) defined in formula (1.4) satisfies the inequality

P (n^{−k/2}|I_{n,k}(f)| > x) ≤ C max (exp \{−α (\frac{x}{σ} )^{2/k}\} , exp \{−α(nx^{2})^{1/(k+1)}\}) (1.5)

for all x > 0, where the constants C = C_{k} > 0 and α = α_{k} > 0 depend only on the parameter k.

I shall show that Theorem 2 is a simple consequence of Theorem 1. It is also possible to prove Theorem 2 directly and to deduce Theorem 1 from it. But such a deduction demands more work. A basic result of the theory of U-statistics, the Hoeffding decomposition may help, since it enables one to represent general U-statistics (the random integral J_{n,k}(f) also can be considered as an U-statistic) as a sum of Ustatistics of different orders with canonical kernel functions. But to deduce Theorem 1 from Theorem 2 some additional work is still needed. Here I do not deal with this problem, but the end of Section 2 in [7] indicates what kind of calculations are needed to work out the details.

Part c) of Theorem 2 in [1] gives an estimate equivalent to our Theorem 2. It states (with the notations of the present paper) that under the conditions of Theorem 2

P (n^{−k/2}|I_{n,k}(f)| > x) ≤ c_{1} exp \{−\frac{c_{2}x^{2/k}}{σ^{2/k} + (x^{1/k}n^{−1/2})^{2/(k+1)}} \}  (1.6)

with some universal constants c_{1} and c_{2} depending only on the parameter k. In [1] this estimate is called a new Bernstein-type inequality. It is formally different from our Theorem 2, but actually they are equivalent if we do not specify the universal constants in these estimates. It is simple to see this if the two cases when the first and the second term is dominating in the denominator at the right-hand side of the expression in (1.6) are considered separately.

2. The idea of the proof

We shall prove Theorem 1 by first giving a good estimate on certain even moments of the random variable J_{n,k}(f) and then by applying the Markov inequality P (|J_{n,k}(f)| > x) ≤

\frac{EJ_{n,k}(f)^{2M}}{x^{2M}}. In the proof we need a good bound on the moments EJ_{n,k}(f)^{2M}. We do not have to estimate all even moments, but a good estimate is needed for a sufficiently large class of moments. It is enough to bound the moments of the form M = 2^{m}, m = 0, 1, 2, …, and this is done in Proposition 1. The estimate of Proposition 1 is contained in formula (3.1). This formula presents an upper bound for EJ_{n,k}(f)^{2M} in the form of the maximum of two terms, where the first term has the same order as the 2M-th moment of a random variable const. ση^{k} with a random variable η with standard normal distribution, and the second correction term is needed to make relation (3.1) valid also for small values of σ.

The main problem of this paper is how to prove Proposition 1. To overcome this difficulty let us consider an analogous problem when the moments of a multiple stochastic integral with respect to a Gaussian process (with independent increments) are estimated and try to adapt the method applied there. An important result about stochastic integrals, the diagram formula (see e.g. [3]), enables us to prove a good estimate for the moments of multiple stochastic integrals with respect to a Gaussian process. It expresses the product of two multiple stochastic integrals as the sum of certain multiple stochastic integrals with different multiplicities. Here also a constant term may appear which can be considered as a stochastic integral of zero order. The name diagram formula refers to the fact that the kernel functions of the stochastic integrals appearing in this identity are defined with the help of certain diagrams. The diagram formula enables us to express the 2M-th power of a multiple Gaussian stochastic integral as the sum of certain stochastic integrals and to bound its expected value. I want to show that an appropriate adaptation of this method can yield the proof of Proposition 1.

To carry out such a program we need a result analogous to the diagram formula where the product of two multiple stochastic integrals J_{n,k_{1}}(f) and J_{n,k_{2}}(g) with respect to a normalized empirical measure is expressed as the sum of multiple stochastic integrals. Such a result is proved in Lemma 2, and this also will be called the diagram formula. The main difference between Lemma 2 and the corresponding result in the Gaussian case is that now some new terms appear. The kernel functions of the multiple stochastic integrals in this new version of the diagram formula will be defined with the help of an object I shall call coloured diagram.

This diagram formula enables us to bound the 4M = 2^{m+2}-th moment of multiple stochastic integrals if we have a good bound on their 2M = 2^{m+1}-th moment. This suggests to try to give a bound on the moments EJ_{n,k}(f)^{2^{m}} by means of an inductive procedure with respect the parameter m. To do this first we have to estimate the first and second moment of a multiple stochastic integral J_{n,k}(f). Such an estimate is given in Lemmas 1 and 3. They are slightly different from the formulas that appear in the study of Gaussian multiple stochastic integrals. The expected value of the stochastic integral J_{n,k}(f) may be non-zero, and instead of an exact identity I can only give an upper bound for its second moment. The cause of this difference is that, unlike in the Gaussian case, the (normalized) empirical measures of disjoint sets are not independent. But the dependence between them is very weak, therefore the expected value EJ_{n,k}(f) is so small that its non-zero value causes no problem for us. The bound we give for this expected value does not depend on the sample size n, at least if n ≥ \frac{k}{2} . On the other hand, the estimation we gave in Lemma 1 depends on the multiplicity of the integral J_{n,k}(f). We need such a bound, since we estimate the moments of the random integrals J_{n,k}(f) simultaneously for all k = 1, 2, . . ..

The diagram formula reduces the estimate of the 4M = 2^{m+2}-th moment of a multiple random integral to the estimate of the 2M = 2^{m+1}-th moment of a sum of multiple random integrals. If we do not try to get the best possible constant α in Theorem 1 then it is enough to give a sufficiently good estimate on the 2M-th moment of each term in this sum. To get such an estimate the following analytical problem has to be solved: Given two functions f(x_{1}, …, x_{k_{1}}) and g(x_{1}, …, x_{k_{2}}) let us express the product of the random integrals J_{n,k_{1}}(f)J_{n,k_{2}}(g) by means of the diagram formula and give a good estimate on the L_{2}-norm of the kernel functions appearing at the right-hand side of this expression. The solution of this problem is a most essential part of our proof.

Such an estimate is given in formulas (3.14) and (3.15) which can be proved in a natural way by means of the Schwarz inequality. Formula (3.14) yields an estimate for the L_{2} norm of general kernel functions appearing in the diagram formula, and formula (3.15) provides a better bound in the case of those kernel functions which also appear in the Gaussian case. But these estimates alone are not sufficient for our purposes. It turned out that in certain cases the L_{2} norm of the kernel functions defined in the diagram formula can be better bounded. Moreover, to get a sufficiently good estimate for the moments bounded in Proposition 1 we have to exploit this improvement.

This leads us to the introduction of the notion of (r, σ^{2}) dominated functions and the formulation of Lemma 4 which contains the most important properties of functions with such a property. After Lemma 4 I also formulated Proposition 2. This is a sharpened version of Proposition 1 which yields a better estimate for the expected value EJ_{n,k}(f)^{2M} if the variance σ^{2} of J_{n,k}(f) is small and f is an (r, σ^{2}) dominated function with some large r. Proposition 2 was introduced not because the estimate of Proposition 1 was not good enough for us, but because the inductive procedure we apply in the proof works only for Proposition 2. To carry out this procedure we must have a sufficiently good estimate for each term appearing in the intermediate steps, and only Proposition 2 provides a good estimate for them.

To prove Proposition 2 still we need a technical result formulated in Lemma 5. By working out the arguments sketched above the expected value EJ_{n,k}(f)^{2M} can be estimated by a sum with terms possessing a good upper bound. We have to show that these bounds also yield a sufficiently good estimate for the sum itself. This is not a trivial problem, and Lemma 5 helps us to overcome this difficulty.

This paper consists of five sections. In Sections 1 and 2 the main results of the paper were presented and the method of the proof was explained. The lemmas and propositions needed in the proof are formulated in Section 3, and also Theorems 1 and 2 are proved there. Lemmas 1—4 are proved in Section 4. They describe some important properties of the random integrals J_{n,k}(f) defined in formula (1.1). I also put Lemma 4 to Section 4, although it contains a result about a deterministic transform of some functions, so formally this result is not connected to the properties of the random integrals J_{n,k}. Nevertheless, it can be put among the important results about such random integrals, because it plays an essential role in the estimation of the high moments of the terms appearing in the diagram formula. Finally, Section 5 contains the proof of Proposition 2 together with the rather technical Lemma 5 needed in its proof. Theorem 1 was proved as a consequence of this Proposition.

3. The proof of Theorems 1 and 2 with the help of some lemmas

Theorem 1 will be proved with the help of the following result.

Proposition 1. If f = f(x_{1}, …, x_{k}) is a function of k variables,

∥f∥_{∞} = sup_{x_{j}∈X, 1≤j≤k} |f(x_{1}, …, x_{k})| ≤ 1,

and

∥f∥_{2}^{2} = ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k}) ≤ σ^{2},

the measure μ is non-atomic, then there exist some constants C_{k} such that for all numbers of the form M = 2^{m}, where m is a non-negative integer, and n = 1, 2, …

EJ_{n,k}(f)^{2M} ≤ (C_{k}σ^{2}M^{k})^{M} · max (1, (\frac{M}{nσ^{2}} )^{M}) .  (3.1)

The constant C_{k} depends only on the number of variables k in the function f.

Proposition 1 states that typically the 2M-th moment of the random variable J_{n,k}(f) behaves like the 2M-th moment of const. ση^{k}, where η is a random variable with standard normal distribution. But such an estimate holds only if nσ^{2} is not too small. Formula 3.1 also contains a constant C_{k} not given explicitly. The parameter α in the exponent of the estimate in Theorem 1 depends on its value.

To prove Proposition 1 we give an upper bound for the first and second moment of a random integral J_{n,k}(f) if f is a bounded function with k variables and prove some kind of diagram formula which expresses the random integral J_{n,k}(f)^{2} or more generally the product of random integrals J_{n,k_{1}}(f)J_{n,k_{2}}(g) with bounded functions f and g of k_{1} and k_{2} arguments as the sum of appropriate random integrals J_{n,k̄}(·) of some functions which have 0 ≤ k̄ ≤ k_{1} + k_{2} arguments.

To express a product J_{n,k_{1}}(f)J_{n,k_{2}}(g) of random integrals with functions f and g of k_{1} and k_{2} arguments respectively in the desired form we introduce some diagrams and coloured diagrams and define some functions with their help.

Given two positive integers k_{1} > 0 and k_{2} > 0 let us consider a set

N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})}

of pairs of integers such that 1 ≤ j_{s} ≤ k_{1}, k_{1} + 1 ≤ j′_{s} ≤ k_{1} + k_{2}, 1 ≤ s ≤ l, and j_{s} ≠ j_{s′}, j′_{s} ≠ j′_{s′} if s ≠ s′. Let us assume that 1 ≤ j_{1} < j_{2} < ⋯ < j_{l} ≤ k_{1} to introduce an explicit ordering among the elements of the set N. We define the diagram B(N) = B(N , k_{1}, k_{2}) in the following way: The diagram B(N) has two rows, the first row consists of the vertices 1, …, k_{1} and the second row of the vertices k_{1}+1, …, k_{1}+k_{2}. The diagram B(N) contains the edges connecting the vertex j_{s} from the first row with the vertex j′_{s} in the second row, (j_{s}, j′_{s}) ∈ N, 1 ≤ s ≤ l. Given a diagram B(N) and a subset N_{1} ⊂ N we define the coloured diagram B(N , N_{1}) = B(N , N_{1}, k_{1}, k_{2}) as the diagram B(N) whose edges are coloured in the following way: An edge (j_{u}, j′_{u}) has colour 1 if (j_{u}, j′_{u}) ∈ N_{1}, and colour −1 if (j_{u}, j′_{u}) ∈ N \ N_{1}.

Let us also define the following operators on the space of functions. Given an integrable function h(x_{1}, …, x_{s}) on the product space (X^{s}, X^{s}, μ^{s}) with some s = 1, 2, …, define the operators P_{j} = P_{j,s}, 1 ≤ j ≤ s, as

P_{j}h(x_{1}, …, x_{j−1}, x_{j+1}, …, x_{s}) = ∫ h(x_{1}, …, x_{j−1}, u, x_{j+1}, …, x_{s})μ( du). (3.2)

Let us fix a pair of integers (j, j′), 1 ≤ j ≤ k_{1}, k_{1} + 1 ≤ j′ ≤ k_{1} + k_{2} and a function h(x_{s(1)}, …, x_{s(r)}) on an appropriate product of the space (X, X , μ) whose variables are indexed by some numbers 1 ≤ s(1) < s(2) < ⋯ < s(r) ≤ k_{1} + k_{2}, and {j, j′} ⊂ {s(1), …, s(r)}. In this case we define the function R_{j,j′}h as

\begin{matrix}R_{j,j′}h(x_{s(1)}, …, x_{s(p−1)}, x_{s(p)}, x_{s(p+1)}, …, x_{s(q−1)}, x_{s(q)}, x_{s(q+1)}, …, x_{s(r)}) \\ = h(x_{s(1)}, …, x_{s(p−1)}, x_{s(p)}, x_{s(p+1)}, …, x_{s(q−1)}, x_{s(p)}, x_{s(q+1)}, …, x_{s(r)}) \\ if s(p) = j, and s(q) = j′ \end{matrix} (3.3)

In words: The application of the operator R_{j,j′} means that the argument x_{j′} is replaced by the argument x_{j}.

Given two functions f(x_{1}, …, x_{k_{1}}) and g(x_{1}, …, x_{k_{2}}) on the spaces (X^{k_{1}}, X^{k_{1}}) and (X^{k_{2}}, X^{k_{2}}) respectively define the function

f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) = f(x_{1}, …, x_{k_{1}})g(x_{k_{1}+1}, …, x_{k_{1}+k_{2}})  (3.4)

and given a diagram B(N) = B(N , k_{1}, k_{2}) with edges N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})} put

\overline{f ∘ g}_{B(N)}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−l)}) = ∏_{t=1}^{l} R_{j_{t},j′_{t}}f ∘ g(x_{1}, …, x_{k_{1}+k_{2}})  (3.5)

with s(u) = u if 1 ≤ u ≤ k_{1} and to define s(u) in the case k_{1} + 1 ≤ u ≤ k_{1} + k_{2} − l let us list the set {k_{1} + 1, …, k_{1} + k_{2}} \ {j′_{1}, …, j′_{l}} = {v(1), …, v(k_{2} − l)}, k_{1} + 1 ≤ v(1) < ⋯ < v(k_{2} − l) ≤ k_{1} + k_{2} in increasing order and put s(u) = v(u − k_{1}). In words, the following construction was made. Take the product of the functions f and g, and replace the argument x_{j′_{s}} by x_{j_{s}} for all edges (j_{s}, j_{s′}) of the diagram B(N), and then the indices of the variables are not changed. The numbers s(u) were introduced to list the indices of the variables in an increasing order.

Given a coloured diagram B(N , N_{1}) = B(N , N_{1}, k_{1}, k_{2}) with

N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})} and N_{1} = {(j_{u(1)}, j′_{u(1)}), …, (j_{u(p)}, j′_{u(p)})},

where {u(1), …, u(p)} ⊂ {1, …, l} define

\overline{f ∘ g}_{B(N ,N_{1})}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−l−p)}) = ∏_{v=1}^{p} P_{j_{u(v)}} ∏_{t=1}^{l} R_{j_{t},j′_{t}}f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) (3.6)

with the help of the operators P_{j} and R_{j,j′} defined in (3.2) and (3.3), where the operators at the right-hand side are applied from right to left order and then the indices u(·) at the left-hand side are defined in the following way: Let us list the sets {w(1), …, w(k_{1} − p)} = {1, …, k_{1}} \ {j_{u(1)}, …, j_{u(p)}}, 1 ≤ w(1) < ⋯ < w(k_{1} − p), and {v(1), …, v(k_{2} − l)} = {k_{1} + 1, …, k_{1} + k_{2}} \ {j′_{1}, …, j′_{l}}, k_{1} + 1 ≤ v(1) < ⋯ < v(k_{2} − l) ≤ k_{1} + k_{2}, in increasing order. Then we define s(u) = w(u) if 1 ≤ u ≤ k_{1} −p, and s(u) = v(u−k_{1} +p) if k_{1} − p + 1 ≤ u ≤ k_{1} + k_{2} − l − p. In words, take the function \overline{f ∘ g}_{B(N)} and apply the projection operator P_{u} for those indices u which are end-points of edges with colour 1 in the first row of the coloured diagram B(N , N_{1}) i.e. which are end-points of edges in N_{1} in the first row. After this procedure these variables x_{u} disappear. The remaining variables are preserved together with their indices. The numbers s(u) were introduced again to list these variables in increasing order.

To formulate the diagram formula we define the function f ∘g_{B(N ,N_{1})} with the help of the function \overline{f ∘ g}_{B(N ,N_{1})} in the following way.

f ∘ g_{B(N ,N_{1})}(x_{1}, …, x_{k_{1}+k_{2}−l−p}) = \overline{f ∘ g}_{B(N ,N_{1})}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−l−p)}). (3.7)

Here we reindexed the variables of \overline{f ∘ g} and listed them in an increasing order. Now I formulate some lemmas needed in the proof of Proposition 1.

Lemma 1. Let f(x_{1}, …, x_{k}) be an integrable function on a space (X^{k}, X^{k}, μ^{k}) where μ^{k} is the k-th power of the probability measure μ appearing in formula (3.1). Let the measure μ be non-atomic. Then

EJ_{n,k}(f) = B_{n,k} ∫ f(u_{1}, …, u_{k})μ( du_{1}) … μ( du_{k})  (3.8)

with

B_{n,k} = \frac{1}{k!n^{k/2}} ∑_{s=1}^{k}(−1)^{k−s}\binom{n}{s}s! ∑_{(r_{1},...,r_{s}) r_{j} is integer r_{j}≥1, 1≤j≤s, r_{1}+···+r_{s}=k}(r_{1} − 1) ⋯ (r_{s} − 1)B(r_{1}, …, r_{s}),

(3.9) where B(r_{1}, …, r_{s}) equals the number of partitions of the set {1, …, k} to disjoint sets with cardinalities r_{1}, …, r_{s}. The above constants B_{n,k} satisfy the estimate

|B_{n,k}| ≤ \frac{C^{k}}{k^{k/2}}  (3.10)

with some universal constant C > 0 for all n ≥ \frac{k}{2} .

Lemma 2. (Diagram formula) Let f = f(x_{1}, …, x_{k_{1}}) and g = g(x_{1}, …, x_{k_{2}}) be two square integrable functions on the product spaces (X^{k_{1}}, X^{k_{1}}, μ^{k_{1}}) and (X^{k_{2}}, X^{k_{2}}, μ^{k_{2}}) respectively, and assume that the measure μ is non-atomic. Then the identity

J_{n,k_{1}}(f)J_{n,k_{2}}(g) = ∑_{l=0}^{min(k_{1},k_{2})}∑_{p=0}^{l} \frac{(k_{1} + k_{2} − l − p)!}{(k_{1} − l)!(k_{2} − l)!(l − p)!p!}n^{−(l−p)/2}J_{n,k_{1}+k_{2}−l−p}(f ∘g_{l,p}).

(3.11)

holds with the functions f ∘ g_{l,p} = f ∘ g_{l,p,k_{1},k_{2}} of k_{1} + k_{2} − l − p variables, 0 ≤ p ≤ l ≤ min(k_{1}, k_{2}), defined by the formula

\begin{aligned}f ∘ g_{l,p}(x_{1}, …, x_{k_{1}+k_{2}−l−p}) & = \frac{(k_{1} − l)!(k_{2} − l)!(l − p)!p!}{k_{1}!k_{2}!} \\ & ∑_{B(N ,N_{1})∈B(l,p)} f ∘ g_{B(N ,N_{1})}(x_{1}, …, x_{k_{1}+k_{2}−l−p}) \end{aligned} (3.12)

where the functions f ∘ g_{B(N ,N_{1})} are defined by formulas (3.2), (3.3), (3.6) and (3.7), and the set B(l, p) = B(l, p, k_{1}, k_{2}) consists of those coloured diagrams B(N , N_{1}) = B(N , N_{1}, k_{1}, k_{2}) with k_{1} vertices in the first and k_{2} vertices in the second row for which |N | = l, and |N_{1}| = p. (We apply here and in the sequel the convention that |A| denotes the cardinality of a set A.)

Let us remark that the norming constant \frac{(k_{1} − l)!(k_{2} − l)!(l − p)!p!}{k_{1}!k_{2}!} in (3.12) is the

reciproque of the cardinality of the set B(l, p, k_{1}, k_{2}).

Lemma 3. Let f(x_{1}, …, x_{k}) be a square integrable function on the space (X^{k}, X^{k}), and let μ be a non-atomic measure on (X, X). Then for all n ≥ k

EJ_{n,k}(f)^{2} ≤ \frac{C^{k}}{k^{k}} ∥f∥_{2}^{2}  (3.13)

with some universal constant C > 0.

The diagram formula formulated in Lemma 2 enables us to estimate EJ_{n,k}(f)^{2^{m+1}} if we have an estimate for the expectation EJ_{n,k̄}(g)^{2^{m}} for all bounded functions g (where also the number of arguments in the function g may be arbitrary). This may enable us to prove Proposition 1 by induction. But to prove a sufficiently good estimate, first we have to prove a good bound on the L_{2} norm of functions of the form f ∘ f_{B(N ,N_{1})}(x_{1}, …, x_{2k−p}) with some coloured diagram B(N , N_{1}). We will show that

∥f ∘ g_{B(N ,N_{1})}∥_{2}^{2} ≤ ∥f∥_{2}∥g∥_{2} for any diagram B(N , N_{1}) if ∥f∥_{∞} ≤ 1 and ∥g∥_{∞} ≤ 1.

(3.14)

Moreover, in the special case when |N | = |N_{1}| this estimate can be improved. Namely, we shall show that

∥f ∘ g_{B(N ,N_{1})}∥_{2}^{2} ≤ ∥f∥_{2}^{2}∥g∥_{2}^{2} if |N | = |N_{1}|.  (3.15)

Let us call a diagram B(N , N_{1}) Gaussian if N = N_{1}, and non-Gaussian in the other case. The introduction of such a terminology is natural, since in the diagram formula for multiple stochastic integrals with respect to Gaussian processes those diagrams appear (with the same integrals attached to them) which we called Gaussian. The most difficult part of our investigation is to check the contribution of the integrals corresponding to non-Gaussian diagrams.

To prove formula (3.14) let us take two functions f(x_{1}, …, x_{k_{1}}) and g(x_{1}, …, x_{k_{2}}) whose supremum norm is bounded by 1 and consider the function \overline{f ∘ g}_{B(N)} with some diagram N defined by formulas (3.3), (3.4) and (3.5). The Schwarz inequality yields that

∥\overline{f ∘ g}_{B(N)}∥_{2}^{2} ≤ (∫ f^{4}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k_{1}}))^{1/2}

(∫ g^{4}(x_{1}, …, x_{k_{2}})μ( dx_{1}) … μ( dx_{k_{2}}))^{1/2}

≤ (∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k_{1}}))^{1/2}

(∫ g^{2}(x_{1}, …, x_{k_{2}})μ( dx_{1}) … μ( dx_{k_{2}}))^{1/2} = ∥f∥_{2}∥g∥_{2}

if ∥f∥_{∞} ≤ 1 and ∥g∥_{∞} ≤ 1.

The operators P_{j} are contractions in L_{2} norm. Hence the above estimate together with formulas (3.5), (3.6) and (3.7) imply that

∥f ∘ g_{B(N ,N_{1})}∥_{2}^{2} = ∥\overline{f ∘ g}_{B(N ,N_{1})}∥_{2}^{2} ≤ ∥\overline{f ∘ g}_{B(N)}∥_{2}^{2} ≤ ∥f∥_{2}∥g∥_{2},

and this is relation (3.14).

To prove formula (3.15) let us first give a pointwise estimate on \overline{f ∘ g}_{B(N ,N)}^{2} with the help of the Schwarz inequality. We get by using formulas (3.2), (3.3), (3.4) and (3.6) that

\begin{aligned}\overline{f ∘ g}_{B(N ,N)}^{2}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−l−p)}) & ≤ ∫ f^{2}(x_{1}, …, x_{k_{1}}) ∏_{s=1}^{l} μ( dx_{j_{s}}) \\ & ∫ g^{2}(x_{k_{1}+1}, …, x_{k_{1}+k_{2}}) ∏_{s=1}^{l} μ( dx_{j′_{s}}), \end{aligned} (3.16)

where N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})}. Beside this, the arguments of the function at the left-hand side of (3.16) are the elements of the set {s(1), …, s(k_{1} + k_{2} − l − p)} = {1, …, k_{1} + k_{2}} \ {j_{1}, …, j_{l}, j′_{1}, …, j′_{l}}. The important point is that the estimate at the right-hand side of (3.16) is the product of two functions with different arguments. We get by integrating both sides of inequality (3.16) that

∥\overline{f ∘ g}_{B(N ,N)}∥_{2}^{2} ≤ ∥f∥_{2}^{2}∥g∥_{2}^{2}.

This inequality together with the identity ∥\overline{f ∘ g}_{B(N ,N)}∥_{2}^{2} = ∥f ∘ g_{B(N ,N)}∥_{2}^{2} imply formula (3.15).

Lemma 3 gives a bound on EJ_{n,k}(f)^{2} for a function f with the help of ∥f∥_{2}^{2}, and the application of Lemma 2 with the choice f = g enables us to bound EJ_{n,k}(f)^{4M} if we have a bound for the expressions of the form EJ_{n,k̄}(g)^{2M}. In such a way we can prove good bounds on the moments EJ_{n,k}(f)^{2^{m}} by induction. To carry out such a program we need a good bound on the L_{2} norm of the terms at the right-hand side (3.12) in the special case f = g. Formulas (3.14) and (3.15) supply such estimates, but they are not always good enough for our purposes.

These estimates are sufficient to bound EJ_{n,k}(f)^{2M} for M = 1, 2, but do not suffice already in the case M = 4. Indeed, to give a good bound on EJ_{n,k}(f)^{8} with the help of Lemma 2 we also have to give a good bound on EJ_{n,2k}(g)^{4} with g(x_{1}, …, x_{2k}) = f(x_{1}, …, x_{k})f(x_{k+1}, …, x_{2k}). (This term is f ∘ f_{B(N ,N_{1})} with N = N_{1} = ∅.) To estimate this expression (again with the help of Lemma 2) we also have to bound EJ_{n,4k−1}(g_{1})^{2} with the function g_{1}(x_{1}, …, x_{4k−1}) = g(x_{1}, …, x_{2k})g(x_{1}, x_{2k+2}, …, x_{4k}) (here we consider the diagram B(N , N_{1}) with N = {(1, 2k + 1)} and N_{1} = ∅), and to estimate EJ_{n,4k−1}(g_{1})^{2} we have to bound L_{2} norm of the function g_{1}.

Formulas (3.14) and (3.15) imply that ∥g∥_{2}^{2} ≤ σ^{4} and ∥g_{1}∥_{2}^{2} ≤ σ^{4} if ∥f∥_{2}^{2} ≤ σ^{2}. But a more careful analysis shows that the function g_{1} also satisfies the sharper estimate ∥g_{1}∥_{2}^{2} ≤ σ^{6}. Indeed, the function g is the product of two functions with different arguments, and the L_{2} norm of both terms is bounded by σ. Further considerations show that the function g_{1} can be written as the product of three terms whose L_{2} norm are bounded by σ, and this implies the above statement.

If ∥f∥_{2}^{2} ≤ σ^{2} with a small number σ, then to get a sufficiently good estimate for EJ_{n,k}(f)^{2M} for M ≥ 4 with the help of Lemma 2 a sufficiently good bound has to be proved about L_{2} norm of certain functions. The estimates (3.14) and (3.15) are not always sufficient for this goal. In the previous example we could get some improvement by exploiting the special structure of certain functions. It worked, because the function whose L_{2} norm we had to bound could be represented as the product of functions with different arguments and small L_{2} norms. An appropriate refinement of this argument would also supply a useful estimate for the L_{2} norm of such functions which can be bounded by the product of functions with different arguments and small L_{2} norm. To get sufficiently good estimates which lead to the proof of Proposition 1 we have to exploit the improvements following from the above sketched observations. Such considerations lead to the introduction of the following definition.

The definition of (r, σ^{2}) dominated functions. Let us consider a function f = f(x_{u_{1}}, …, x_{u_{k}}), x_{u_{j}} ∈ X, 1 ≤ j ≤ k, of k variables. We say that this function is (r, σ^{2}) dominated with 1 ≤ r < ∞, 0 < σ ≤ 1 if the set {u_{1}, …, u_{k}} can be partitioned to

r disjoint sets B_{l} = {j_{1}^{(l)}, …, j_{s_{l}}^{(l)}}, 1 ≤ l ≤ r, ⋃_{l=1}^{r} B_{l} = {u_{1}, …, u_{k}}, and for all

1 ≤ l ≤ r such functions h_{l} = h_{l}(x_{j_{1}^{(l)}}, …, x_{j_{s_{l}}^{(l)}}) can be defined whose arguments are indexed by the elements of the set B_{l}, and which satisfy the inequalities

|f(x_{u_{1}}, …, x_{u_{k}})| ≤ ∏_{l=1}^{r} h_{l}(x_{j_{1}^{(l)}}, …, x_{j_{s_{l}}^{(l)}}) for all (x_{u_{1}}, …, x_{u_{k}})

and ∥h_{l}∥_{∞} ≤ 1, ∥h_{l}∥_{2}^{2} ≤ σ^{2}, 1 ≤ l ≤ r.

In particular, we allow that some of the sets B_{l} be empty in the above partition. If the set B_{l} is empty, then the function h_{l} has to be constant, and 0 ≤ h_{l} ≤ σ.

Let us remark that a function f is (1, σ^{2}) dominated if and only if ∥f∥_{∞} ≤ 1, and ∥f∥_{2} ≤ σ.

The following Lemma 4 which uses the above definition plays a most important role in the proof of Proposition 1. We formulate it in a form more general than we need. We shall consider two functions f and g, although we shall apply this result only in the special case when f = g.

Lemma 4. Let f be a function on the product space (X^{k_{1}}, X^{k_{1}}, μ^{k_{1}}) and g a function on the product space (X^{k_{2}}, X^{k_{2}}, μ^{k_{2}}) with some integers k_{1} ≥ 1, k_{2} ≥ 1. Let us take a coloured diagram B(N , N_{1}) = B(N , N_{1}, k_{1}, k_{2}) with |N | = l and |N_{1}| = p and consider the function f ∘ g_{B(N ,N_{1})} defined by formulas (3.2), (3.3), (3.6) and (3.7). Let f be an (r_{1}, σ^{2}) and g an (r_{2}, σ^{2}) dominated function. If the relation r_{1} + r_{2} ≥ l − p + 1 holds, then the function f ∘ g_{B(N ,N_{1})} is (r_{1} + r_{2} − (l − p), σ^{2}) dominated. Beside this, the function f ∘ g_{B(N ,N_{1})} is (1, σ^{(r_{1}+r_{2})/2}) dominated for all 0 ≤ p ≤ l ≤ min(k_{1}, k_{2}).

It is clear that if f is an (r_{1}, σ^{2}) and g is an (r_{2}, σ^{2}) dominated function, then f ∘ g is an (r_{1} + r_{2}, σ^{2}) dominated function. Lemma 4 essentially states that if we calculate the function f ∘ g_{B(N ,N_{1})} with the help of the function f ∘ g, then the edges of N_{1} (with colour 1) do not diminish and the edges in N \ N_{1} diminish only with 1 the first coordinate in the (r_{1} + r_{2}, σ^{2}) dominance property.

With the help of Lemmas 1—4 the following Proposition 2 can be proved which yields a more detailed description on the moments of the random integrals J_{n,k}(f) than Proposition 1.

Proposition 2. Let f = f(x_{1}, …, x_{k}) be a measurable (r, σ^{2/r}) dominated function of k variables, k ≥ 1, with some r ≥ 1. Put M = 2^{m}. Let us assume that the measure μ appearing in formula (3.1) is non-atomic. Then for all m = 0, 1, … such that kM ≤ n

EJ_{n,k}(f)^{2M} ≤ (\frac{C(k, m)M^{k}σ^{2}}{k^{k}} )^{M} · max (1, (\frac{kM}{nσ^{2/r}} )^{M min(k,r)})  (3.17)

with some constants C(k, m) ≥ 1 which do not depend on the parameters n and σ, and they satisfy the inequality sup_{0≤m<∞} C(k, m) ≤ C_{k} < ∞ for all k = 1, 2, . . ..

Remark: The condition kM ≤ n in Proposition 2 is of technical character which probably can be omitted. We imposed this condition to avoid the investigation of such degenerate random integrals J_{n,k}(f) where the number of variables of the function f is larger than the sample size n. To avoid such a situation during the inductive procedure leading to the proof of Proposition 2 we had to impose the condition kM ≤ n. This restriction does not cause a serious problem in the proof of Theorem 1.

Proposition 1 follows from Proposition 2 if we apply it for (r, σ^{2/r}) dominated func-

tions f = f(x_{1}, …, x_{k}) with r = 1 and choose C = C_{k} = \frac{1}{k^{k−1}} sup_{0≤m≤∞} C(k, m). Actu-

ally Proposition 2 implies Proposition 1 only under the additional condition kM ≤ n. But Proposition 1 can be proved in the case n ≤ kM with the help of the following very rude estimate: Since the measure μ_{n}−μ is the difference of two probability measures and ∥f∥_{∞} ≤ 1, hence ∥J_{n,k}f∥_{∞} ≤ 2^{k}n^{k/2} and EJ_{n,k}(f)^{2M} ≤ 2^{2kM}n^{kM} ≤ 2^{2kM}(kM)^{kM} ≤ n^{−M}2^{2kM}(kM)^{(k+1)M} = C̄_{k}^{M}n^{−M}M^{(k+1)M} with C̄_{k} = 4^{k}k^{k+1} if n ≤ kM. This estimate means that the estimate (3.1) remains valid with a possibly new constant C_{k} if we also allow the case n ≤ kM. (Here we use the second term in the maximum at the right-hand side of (3.1).) Hence it is enough to prove Proposition 2.

Proposition 2 will be proved by induction with respect to m. For m = 0 Lemma 3 implies the result we need. If we know the bound (3.17) for some m with a constant C(k, m), then this bound together with Lemma 4 enable us to give a good bound on the 2^{m}-th moment of the expressions J_{n,2k−(l−p)}(f ∘f_{l,p}) with the functions f ∘f_{l,p} appearing in formula (3.12). Then we can prove formula (3.17) for m + 1 with an appropriate constant C(k, m + 1) by means of the diagram formula presented in Lemma 2. The calculations made during this proof show that constant C(k, m + 1) appearing in this estimate depends only on the parameters k and m, i.e. it does not depend on the sample size n or the variance σ^{2}. Moreover, an important but rather technical result formulated below in Lemma 5 will enable us to show that the norming constants C(k, m) can be chosen relatively small. By exploiting the existence of some constants C̄(k, m) with the properties formulated in Lemma 5 we can show that in Proposition 2 the constants C(k, m) can be chosen in such a way that they also satisfy the inequality

sup_{0≤m≤∞} C(k, m) < ∞.

Lemma 5. There exists a set of positive real numbers C̄(k, m), C̄(k, 0) = 1, m = 0, 1, …, k = 0, 1, …, which satisfy the inequality

C̄(k, m + 1)^{2} ≥ 2^{2l(4−m)} \frac{(2k)^{2k−l+p}(2k − l − p)^{3l−p−2k}}{(2l)^{2l}} C̄(2k − l − p, m)
for all k = 0, 1, 2, …, m = 0, 1, 2, … and 0 ≤ p ≤ l ≤ k,

(3.18)

and

sup_{m≥0} C̄(k, m) ≤ D(k) < ∞ for all k = 0, 1, 2, …  (3.19)

with some constants D(k) > 0, k = 0, 1, 2, . . .. The numbers C̄(k, m) can be given in the form C̄(k, m) = C̄(m)^{k}, where C̄(m), m = 0, 1, 2, …, is a monotone increasing sequence such that C̄(0) = 1, and lim_{m→∞} C̄(m) < ∞.

We finish Section 3 by proving Theorems 1 and 2 with the help of Proposition 1. The proof of Theorem 1. By Proposition 1 and the Markov inequality

P (|J_{n,k}(f)| > x) ≤ (\frac{C_{k}σ^{2}M^{k}}{x^{2}} )^{M} · max (1, (\frac{M}{nσ^{2}} )^{M})  (3.20)

for all positive integers M of the form M = 2^{m}. Let us assume for a while that the number x satisfies the inequalities \frac{x}{σ} ≥ 2^{(k+1)/2}C_{k}^{1/2} and nx^{2} ≥ 2^{k+3}C_{k}. Then the number M = 2^{m} can be chosen in such a way that the inequality \frac{1}{2^{k+1}} ≤ \frac{C_{k}σ^{2}M^{k}}{x^{2}} < \frac{1}{2},

i.e. \frac{1}{(2^{k+1}C_{k})^{1/k}} (\frac{x}{σ} )^{2/k} ≤ M < \frac{1}{(2C_{k})^{1/k}} (\frac{x}{σ} )^{2/k} holds. Let us choose such a number

M if \frac{1}{(2C_{k})^{1/k}} (\frac{x}{σ} )^{2/k} ≤ nσ^{2}, i.e. x ≤ \sqrt{2C_{k}}n^{k/2}σ^{k+1}. In this case M ≤ nσ^{2}, the

first term is dominating in the maximum at the right-hand side of formula (3.20), and P (|J_{n,k}(f)| > x) ≤ 2^{−M} ≤ exp \{−α (\frac{x}{σ} )^{2/k}\} with some appropriate α > 0.

In the case x ≥ \sqrt{2C_{k}}n^{k/2}σ^{k+1} let us choose M = 2^{m} in such a way that \frac{1}{2^{(k+3)}} ≤ \frac{C_{k}M^{k+1}}{nx^{2}} ≤ \frac{1}{4}. (There is a number M = 2^{m} satisfying this inequality, because by our temporary conditions nx^{2} ≥ 2^{k+3}C_{k}, i.e. \frac{C_{k}M^{k+1}}{nx^{2}} ≤ \frac{1}{2^{k+3}} for M = 1.) With the above choice of M the inequality \frac{M}{nσ^{2}} ≥ 2^{−(k+2)/(k+1)} holds, since \frac{1}{2^{k+3}} ≤ \frac{C_{k}M^{k+1}}{nx^{2}} ≤ \frac{1}{2} (\frac{M}{nσ^{2}} )^{k+1} for x ≥ \sqrt{2C_{k}}n^{k/2}σ^{k+1}. Hence, by formula (3.20) and the inequality

\frac{1}{2^{(k+3)}} ≤ \frac{C_{k}M^{k+1}}{nx^{2}} ≤ \frac{1}{4}

P (|J_{n,k}(f)| > x) ≤ (\frac{C_{k}σ^{2}M^{k}}{x^{2}} )^{M} (\frac{2^{(k+2)/(k+1)}M}{nσ^{2}} )^{M} = (\frac{2^{(k+2)/(k+1)}C_{k}M^{k+1}}{nx^{2}} )^{M}

≤ (\frac{2^{(k+2)/(k+1)}}{4} )^{M} ≤ exp \{−α(nx^{2})^{1/(k+1)}\}

with some appropriate α > 0.

The above calculations show that formula (1.3) holds if \frac{x}{σ} ≥ 2^{(k+1)/2}C_{k}^{1/2} and nx^{2} ≥ 2^{k+3}C_{k}. By choosing the constant C > 0 sufficiently large in formula (1.3) we can achieve that the right-hand side of (1.3) is greater than 1 if one of the above inequalities does not hold. With such a choice of the constant C we get that formula (1.3) holds for all x ≥ 0.

Now we turn to the proof of Theorem 2.

The proof of Theorem 2. Let us first consider the case when the measure μ is nonatomic. Let us take a canonical function f(x_{1}, …, x_{k}) of k variables. In this case we can write

n^{−k/2}I_{n,k}(f) = \frac{n^{k/2}}{k!} ∫ ′ f(u_{1}, …, u_{k})μ_{n}( du_{1}) … μ_{n}(du_{k})

= \frac{n^{k/2}}{k!} ∫ ′ f(u_{1}, …, u_{k})(μ_{n}( du_{1}) − μ( du_{1})) … (μ_{n}(du_{k}) − μ(du_{k})) = J_{n,k}(f). (3.21)

(Let us remark that we have exploited at this point that the measure μ is non-atomic. The canonical property of the function f implies that ∫ f(u_{1}, …, u_{k})μ( du_{j}) = 0 for any 1 ≤ j ≤ k and u_{s} ∈ X, j ≠ s. But we need the fact that this relation remains valid if we omit the points u_{j} = u_{s}, j ≠ s, from the domain of integration.) Theorem 1 and formula (3.21) imply Theorem 2 in the case when μ is non-atomic.

The general case can be reduced to the case of non-atomic measure μ with the help of the following construction. Given a space (X, X) with a measure μ on it together with a sequence of independent random variables ξ_{1}, …, ξ_{n} and a canonical function f(x_{1}, …, x_{k}) on the product space (X^{k}, X^{k}) consider the space (Y, B, λ) where Y is the unit interval, B is the Borel σ-algebra and λ is the Lebesgue measure on it. Then define the product space (X̄, X̄, μ̄) as X̄ = X × Y, X̄ = X × A and μ̄ = μ × λ. Define the function f̄ on the k-th power of this new space (X̄^{k}, X̄^{k}) as f̄((x_{1}, y_{1}), …, (x_{k}, y_{k})) = f(x_{1}, …, x_{k}), i.e. the function f̄ does not depend on the new coordinates. Finally, let us consider a sequence of independent random variables η_{1}, …, η_{n} with uniform distribution on the unit interval [0, 1] which are independent also of the random variables ξ_{1}, …, ξ_{n}. Define the random variables (ξ̄_{1}, …, ξ̄_{n}) = ((ξ_{1}, η_{1}), …, (ξ_{n}, η_{n})), and let μ̄_{n} denote their empirical distribution function. Then I_{n,k}(f) = I_{n,k}(f̄), and the functions f and f̄ are simultaneously canonical. This means that the result of Theorem 2 can be applied for the function f̄, hence it also holds for the function f.

4. On some basic properties of the random integrals J_{n,k}

First we prove Lemma 1 which expresses the expected value of the random integrals J_{n,k}(f).

The proof of Lemma 1. Let us prove formula (3.8) first in the special case when f(x_{1}, …, x_{k}) = I_{A_{1}}(x_{1}) ⋯ I_{A_{k}}(x_{k}), where A_{1}, …, A_{k} are disjoint measurable subsets of X and I(A) denotes the indicator function of the set A. Let us introduce the (random) signed measures ν_{l}, 1 ≤ l ≤ n, on (X, X)

ν_{l}(A) = ν_{l}(A, ω) = \{ \begin{matrix}1 − μ(A) & if ξ_{l}(ω) ∈ A \\ −μ(A) & if ξ_{l}(ω) ∉ A \end{matrix} for A ∈ X and 1 ≤ l ≤ n.

Then μ_{n} − μ = \frac{1}{n} ∑_{l=1}^{n} ν_{l} and

EJ_{n,k} (I_{A_{1}}(x_{1}) ⋯ I_{A_{k}}(x_{k})) = \frac{n^{−k/2}}{k!} E ( ∏_{m=1}^{k} (∑_{l=1}^{n} ν_{l}(A_{m}))) .  (4.1)

To calculate the expression in (4.1) first we fix some set of indices D = {m_{1}, …, m_{r}} ⊂ {1, …, k} of cardinality r together with a number 1 ≤ l ≤ n and show that

E (∏_{t=1}^{r} ν_{l}(A_{m_{t}})) = (r − 1)(−1)^{r−1} ∏_{t=1}^{r} μ(A_{m_{t}}).  (4.2)

Indeed, let us consider separately the cases when ξ_{l} ∈ A_{m_{t}} for an m_{t} ∈ D and the case when ξ_{l} ∉ A_{m_{t}} for all m_{t} ∈ D, and write

P (ν_{l}(A_{m_{t}}) = 1 − μ(A_{m_{t}})) = P (ν_{l}(A_{m_{t}}) = 1 − μ(A_{m_{t}}), ν_{l}(A_{m_{q}}) = −μ(A_{m_{q}}) if 1 ≤ q ≤ r and q ≠ t) = μ(A_{m_{t}})

for all 1 ≤ t ≤ r, and P (ν_{l}(A_{m_{t}}) = −μ(A_{m_{t}}) for all 1 ≤ t ≤ r)) = 1− ∑_{t=1}^{r} μ(A_{m_{t}}). Hence

E (∏_{t=1}^{r} ν_{l}(A_{m_{t}})) = ∑_{t=1}^{r} μ(A_{m_{t}}) ( ∏_{1≤q≤r, q≠t}(−μ(A_{m_{q}})) + ∏_{q=1}^{r}(−μ(A_{m_{q}})))

+ (1 − ∑_{t=1}^{r} μ(A_{m_{t}})) ∏_{q=1}^{r}(−μ(A_{m_{q}})) = (r − 1)(−1)^{r−1} ∏_{t=1}^{s} μ(A_{m_{t}}).

Take a partition D_{1}, …, D_{s} of the set {1, …, k} such that |D_{u}| = r_{u}, 1 ≤ u ≤ s, with some prescribed numbers r_{u} and some integers l_{u}, 1 ≤ l_{u} ≤ n, 1 ≤ u ≤ s, such that l_{u} ≠ l_{u′} if u ≠ u′. The random vectors (ν_{l_{u}}(A_{1}), …, ν_{l_{u}}(A_{k})) are independent for u = 1, …, s. This fact together with relation (4.2) imply that

\begin{aligned}E (∏_{u=1}^{s} ∏_{v∈D_{u}} ν_{l_{u}}(A_{v})) & = ∏_{u=1}^{s} E ( ∏_{v∈D_{u}} ν_{l_{u}}(A_{v})) = ∏_{u=1}^{s}(r_{u} − 1)(−1)^{r_{u}−1} ∏_{j=1}^{k} μ(A_{j}) \\ & = (−1)^{k−s} ∏_{u=1}^{s}(r_{u} − 1) ∫ f(u_{1}, …, u_{k})μ( du_{1}) … μ( du_{k}) \end{aligned}

(4.3)

with f(u_{1}, …, u_{k}) = I_{A_{1}}(u_{1}) ⋯ I_{A_{k}}(u_{k}). We get formula (3.8) together with the constant defined in (3.9) in the special case we have considered by means of relations (4.1) and (4.3) if we observe that by carrying out the multiplication at the right-hand side of (4.1) we get a sum with terms of the forms defined in (4.3). (To get a term of the form given in (4.3) we prescribe which measures ν_{u_{l}}, ≤ u_{l} ≤ n, 1 ≤ l ≤ k, we take, together with the partition of the set {1, …, k} which tells for which indices l the measure ν_{u_{l}} is chosen. We can choose \binom{n}{s}s!B(r_{1}, …, r_{s}) such terms, where B(r_{1}, …, r_{s}) denotes the number of partitions of the set {1, …, k} to subsets of size r_{1}, r_{2}, … and r_{s}.)

Relation (3.8) also holds for any linear combination of functions of the above type. Since the set of such linear combinations is dense in the L_{1} norm in the space (X^{k}, X^{k}, μ^{k}) to prove formula (3.8) for general functions f it is enough to show that

E|J_{n,k}(f)| ≤ C(n, k)∥f∥_{1} for all functions f of k variables  (4.4)

with an appropriate constant C(n, k) < ∞. Let us emphasize that since in the problem considered here the dimension k of the function f and the size n of the sample ξ_{1}, …, ξ_{n} are fixed an arbitrary large constant C(n, k) would suffice in (4.4).

Moreover, relation (4.4) can be simplified. By carrying out the multiplications of the measures μ_{n}( du_{j}) − μ( du_{j}) in formula (1.1) we get a sum with finitely many terms, and it is enough to prove the bound given in (4.4) for each of these terms. Moreover, by carrying out the integration with respect the deterministic measures μ( du_{j}) we get that it is enough to show that

E ∫ ′|f(u_{1}, …, uk̄)|μ_{n}( du_{1}) … μ_{n}( duk̄) ≤ C(n, k)∥f∥_{1}
for all functions f of k̄ ≤ k variables.  (4.5)

Even this relation can be reduced with the help of the triangular inequality in the L_{1} norm and the fact that the linear combinations of the indicator functions of measurable sets are dense in the space L_{1}(Xk̄, X k̄, μk̄). It is enough to prove the following simpler statement: For any k̄ ≤ k and measurable set A ∈ (Xk̄, X k̄)

E ∫′_{A} μ_{n}( du_{1}) … μ_{n}(du_{k}) ≤ C(n, k)μk̄(A).  (4.6)

This statement is obvious, since we can express the integral at the left-hand side of (4.6) as nk̄E ∫′_{A} μ_{n}( du_{1}) … μ_{n}(du_{k}) = n(n − 1) ⋯ (n − k̄ + 1)μk̄(A), and this implies relation (4.6). The above identity holds, since to calculate the integral at the lefthand side of (4.6) we have to count the number of such k̄ tuples (ξ_{j_{1}}, …, ξ_{jk̄}) for which (ξ_{j_{1}}, …, ξ_{jk̄}) ∈ A and apply the identity P ((ξ_{j_{1}}, …, ξ_{jk̄}) ∈ A) = μk̄(A). (Let us remark that at this point we heavily exploited that in the definition of the random integrals J_{n,k} we deleted the diagonals from the domain of integration.)

To complete the proof of Lemma 1 it is enough to check inequality (3.10). For this aim let us observe that because of the factors r_{u} − 1 in (3.9) we may assume in this formula that r_{u} ≥ 2 for all 1 ≤ u ≤ s, hence s ≤ [\frac{k}{2} ], where [x] denotes the integer part

of the number x. Beside this, we shall apply the rather rude inequality ∏_{u=1}^{s}(r_{u} − 1) ≤ 4^{k}

if r_{u} ≥ 1, 1 ≤ u ≤ s and ∑_{u=1}^{s} r_{u} = k. Thus we get that

|B_{n,k}| = |\frac{1}{k!n^{k/2}} ∑_{s=1}^{k}(−1)^{k−s}\binom{n}{s}s!  ∑_{(r_{1},...,r_{s}) r_{j} is integer r_{j}≥1, 1≤j≤s s_{1}+···+r_{s}=k}(r_{1} − 1) ⋯ (r_{s} − 1)B(r_{1}, …, r_{s})|

≤ \frac{4^{k}}{k!n^{k/2}} \begin{matrix}[ & \frac{k}{2} & ] \\ ∑ \\ s=1 \end{matrix} n(n − 1) ⋯ (n − s + 1) ∑_{(r_{1},...,r_{s}) r_{j} is integer r_{j}≥1, 1≤j≤s r_{1}+···+r_{s}=k} B(r_{1}, …, r_{s}).

(4.7)

To estimate the above expression B(n, k) let us first show that the inner sum at the right-hand side of this expression satisfies the inequality

∑_{(r_{1},...,r_{s}) r_{j} is integer r_{j}≥1, 1≤j≤s r_{1}+···+r_{s}=k} B(r_{1}, …, r_{s}) ≤ 2^{k}s^{k−s}, for all 1 ≤ s ≤ k.  (4.8)

Indeed, the expression at the left-hand side of (4.8) equals the number of partitions of the set {1, …, k} into s non-empty sets. This can be bounded with the help of the following construction: Choose a set {u_{1}, …, u_{s}}, 1 ≤ u_{1} < u_{2} < ⋯ u_{s} ≤ k, what can be done in \binom{k}{s} way. Let us construct a partition D_{1}, …, D_{s} of the set \{1, …, k\} so that u_{j} ∈ D_{s}, 1 ≤ j ≤ s, and distribute the remaining k − s points of {1, …, k}among the sets D_{j}, 1 ≤ j ≤ s. This can be done in s^{k−s} way. Beside this \binom{k}{s} ≤ 2^{k}. We have defined no more than 2^{k}s^{k−s} partitions. I claim that we have constructed all possible partitions of s elements. Indeed, let us put the elements of a partition D_{1}, …, D_{s} in such an order that j < j′ if the smallest element ū_{j} of D_{j} is smaller than the smallest element ū_{j′} of D_{j′}. This partition will be obtained if the set {ū_{1}, …, ū_{s}} is chosen in the first step of the above construction, and then the remaining terms are appropriately distributed among the sets of partition.

Relations (4.7), (4.8) and the condition 1 ≤ k ≤ \frac{n}{2} yield that

|B_{n,k}| ≤ \frac{8^{k}}{k!n^{k/2}} \begin{matrix}[ & \frac{k}{2} & ] \\ ∑ \\ s=1 \end{matrix} n^{s}s^{k−s} ≤ \frac{C̄^{k}}{k^{k}} \begin{matrix}[ & \frac{k}{2} & ] \\ ∑ \\ s=1 \end{matrix} n^{s−k/2}s^{k−s} ≤ \frac{(2C̄)^{k}}{k^{k/2}} \begin{matrix}[ & \frac{k}{2} & ] \\ ∑ \\ s=1 \end{matrix} (\frac{s}{k} )^{k−s} ≤ \frac{C^{k}}{k^{k/2}}

with some appropriate constants C̄ > 0 and C > 0. This means that formula (3.10) holds. Lemma 1 is proved.

Now we turn to the proof of the diagram formula described in Lemma 2. Let me remark that similar results were already proved for random integrals with respect to a Poisson process, (see [8]). These two problems and their possible proofs are similar, although in the present problem some additional technical difficulties appear because of the absence of independence for the empirical measure μ_{n} of disjoint sets.

The proof of Lemma 2. Let us consider first the case when f(x_{1}, …, x_{k_{1}}) = ∏_{j=1}^{k_{1}} I_{A_{j}}(x_{j}),

g(x_{1}, …, x_{k_{2}}) = ∏_{j′=1}^{k_{2}} I_{B_{j′}} (x_{j′}), the sets A_{1}, …, A_{k_{1}} are disjoint and the same relation

holds for the sets B_{1}, …, B_{k_{2}}. (On the other hand, the sets A_{j} ∩ B_{j′} may be nonempty. We denote again the indicator function of a set A by I_{A}(x).) Let us choose a

parameter ε > 0 which we shall let go to zero and consider a partition A_{j} = ⋃_{s=1}^{u(j)} A(j, s) =

⋃_{s=1}^{u(j,ε)} A(j, s), 1 ≤ j ≤ k_{1}, B_{j′} = ⋃_{t=1}^{v(j′)} B(j′, t) = ⋃_{t=1}^{v(j′,ε)} B(j′, t), 1 ≤ j′ ≤ k_{2}, of the sets A_{j}

and B_{j′} such that μ(A(j, s)) ≤ ε, μ(B(j′, t)) ≤ ε, for all 1 ≤ j ≤ k_{1}, 1 ≤ j′ ≤ k_{2}, 1 ≤ s ≤ u(j), 1 ≤ t ≤ v(j′), and beside this either A(j, s) = B(j′, t) or A(j, s) ⋂ B(j′, t) = ∅ for all pairs (j, s) and (j′, t). We can write

J_{n,k_{1}}(f)J_{n,k_{2}}(g) = \frac{n^{(k_{1}+k_{2})/2}}{k_{1}!k_{2}!} ∑_{\begin{matrix}(s_{1},...,s_{k_{1}} ) : \\ 1≤s_{j}≤u(j), \\ 1≤j≤k_{1} \end{matrix}} ∑_{\begin{matrix}(t_{1},...,t_{k_{2}} ) : \\ 1≤t_{j′} ≤v(j′), \\ 1≤j′≤k_{2} \end{matrix}}∏_{j=1}^{k_{1}} (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))

∏_{j′=1}^{k_{2}} (μ_{n}(B(j′, t_{j′})) − μ(B(j′, t_{j′}))) .  (4.9)

Let B(N) = B(N , k_{1}, k_{2}) denote the set of all diagrams with first row {1, …, k_{1}} and second row {1, …, k_{2}}. We rewrite formula (4.9) with the help of these diagrams by putting the products in the sum in (4.9) into certain groups in dependence on the behaviour of the indices defining them. Put

J_{n,k_{1}}(f)J_{n,k_{2}}(g) = \frac{n^{(k_{1}+k_{2})/2}}{k_{1}!k_{2}!} ∑_{B(N)∈B(N)} Z_{N}  (4.10)

with

\begin{aligned}Z_{N} = Z_{N} (ε) = ^{∑N} ; \begin{matrix}(s_{1},...,s_{k_{1}} ), (t_{1},...,t_{k_{2}} ) : \\ 1≤s_{j}≤u(j), 1≤j≤k_{1}, \\ 1≤t_{j′} ≤v(j′), 1≤j′≤k_{2}, \end{matrix} & ∏_{j=1}^{k_{1}} (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j}))) \\ & ∏_{j′=1}^{k_{2}} (μ_{n}(B(j′, t_{j′})) − μ(B(j′, t_{j′}))) , \end{aligned} (4.11)

where ^{∑N} means that summation is taken for such pairs of vectors (s_{1}, …, s_{k_{1}}) and (t_{1}, …, t_{k_{2}}) for which A(j, s_{j}) = B(j′, t_{j′}) if (j, j′) is an edge of the diagram B(N) and A(j, s_{j}) ∩ B(j′, t_{j′}) = ∅ if (j, j′) is not an edge of the diagram B(N). If there are no vectors (s_{1}, …, s_{k_{1}}) and (t_{1}, …, t_{k_{2}}) with such a property, then we define Z_{N} = 0.

Let us consider a diagram B(N) ∈ B(N) with edges N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})} and consider an approximation Z̄_{N} of Z_{N} which we obtain in the following way: If (j, j′) ∈ N then we replace the term (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))(μ_{n}(B(j′, t_{j′})) − μ(B(j′, t_{j′}))) =

(μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))^{2} by \frac{1}{n}μ_{n}(A(j, s_{j})) and do not change the other terms. (The following heuristic argument may lead to the introduction of such an approximation. The expression (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))^{2} can be well approximated by μ_{n}(A(j, s_{j}))^{2}, the other terms being negligible. On the other hand, μ_{n}(A(j, s_{j}))^{2} ∼ \frac{1}{n}μ_{n}(A(j, s_{j})), since the probability of the event that at least two sample points ξ_{j} fall in a set A(j, s_{j}) of small μ measure is negligibly small. Hence μ_{n}^{2}(A(j, s_{j})) − \frac{1}{n}μ_{n}(A(j, s_{j})) = 0 with probability almost 1.) More explicitly, we define Z̄_{N} for all B(N) ∈ B(N) as

Z̄_{N} = Z̄_{N} (ε) = ^{∑N} ; \begin{matrix}(s_{1},...,s_{k_{1}} ), (t_{1},...,t_{k_{2}} ) : \\ 1≤s_{j}≤u(j), 1≤j≤k_{1}, \\ 1≤t_{j′} ≤v(j′), 1≤j′≤k_{2} \end{matrix} ∏_{j: j∈\{1,...,k_{1}\}\\{j_{1},...,j_{l}\}} (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))

∏_{m=1}^{l} (\frac{1}{n}μ_{n}(A(j_{m}, s_{j_{m}})))
∏_{j′: j′∈{1,...,k_{2}}\{j′_{1},...,j′_{l}}} (μ_{n}(B(j′, t_{j′})) − μ(B(j′, t_{j′}))) ,  (4.12)

I claim that for all B(N) ∈ B(N) the inequality

|Z_{N} (ε) − Z̄_{N} (ε)| ≤ C(k_{1} + k_{2}, n)ε  (4.13)

holds with some constant C(k_{1} + k_{2}, n) < ∞ which does not depend on ε. We do not need a good estimate for this constant. What is important for us is that relations (4.10) and (4.13) have the consequence

J_{n,k_{1}}(f)J_{n,k_{2}}(g) = lim_{ε→0} \frac{n^{(k_{1}+k_{2})/2}}{k_{1}!k_{2}!} ∑_{B(N)∈B(N)} Z̄_{N} (ε).  (4.14)

Before proving relation (4.13) let us show that formulas (4.14) and (4.12) imply relations (3.11) and (3.12) in the special case we are considering. To do this we express Z̄_{N} (ε) (which as we shall see actually does not depend on the parameter ε) in an appropriate form. To do this let us write the terms in the middle product of (4.12) in the form \frac{1}{n}μ_{n}(A(j_{m}, s_{j_{m}})) = \frac{1}{n} ((μ_{n}(A(j_{m}, s_{j_{m}})) − μ(A(j_{m}, s_{j_{m}}))) + μ(A(j_{m}, s_{j_{m}}))), and let us rewrite the terms Z̄_{N} as

Z̄_{N} = n^{−l} ∫ ∏_{t=1}^{l} R_{j_{t},j′_{t}}f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) ∏_{j: j∈{1,...,k_{1}}\{j_{1},...,j_{l}}} (μ_{n}(dx_{j}) − μ( dx_{j}))
∏_{j′: j′∈{1,...,k_{2}}\{j′_{1},...,j′_{l}}} (μ_{n}(dx_{j′}) − μ( dx_{j′}))  (4.15)

∏_{s=1}^{l} ((μ_{n}( dx_{j_{s}}) − μ( dx_{j_{s}})) + μ( dx_{j_{s}})) ,

where (j_{1}, j′_{1}), … (j_{l}, j′_{l}) are the edges of the diagram B(N), and the function f ∘ g and operators R_{j_{t},j′_{t}} are defined in formulas (3.4) and (3.3).

Given a diagram B(N) let us introduce the class of coloured diagrams C(B(N)) consisting of all coloured diagrams B(N , N_{1}) we obtain by colouring the edges of the diagram of B(N) by +1 or −1. By carrying out the multiplications in formula (4.15) by expressing the last line of this expression as the sum of μ_{n}( dx_{j_{s}}) − μ( dx_{j_{s}}) and μ( dx_{j_{s}}) we can rewrite Z̄_{N} with the help of the coloured diagrams in C(B(N)) as

Z̄_{N} = ∑_{p=1}^{l} ∑_{\begin{matrix}B(N ,N_{1})∈C(B(N)), \\ and |N_{1}|=p \end{matrix}} n^{−l} \frac{(k_{1} + k_{2} − l − p)!}{n^{(k_{1}+k_{2}−l−p)/2}}J_{n,k_{1}+k_{2}−l−p}(\overline{f ∘ g}_{B(N ,N_{1})})

The last relation together with formula (4.14) yield that

J_{n,k_{1}}(f)J_{n,k_{2}}(g) = ∑_{l=0}^{min(k_{1},k_{2})}∑_{p=0}^{l} ∑_{B(N ,N_{1})∈B(l,p)} \frac{(k_{1} + k_{2} − l − p)!}{n^{(l−p)/2}k_{1}!k_{2}!}J_{n,k_{1}+k_{2}−l−p}(f ∘ g_{B(N ,N_{1})}).

This relation is equivalent to the diagram formula described in formulas (3.11) and (3.12). Hence to prove Lemma 2 in the special case considered now it is enough to prove formula (4.13).

First we formulate an inequality and show that formula (4.13) follows from it. To formulate it let us introduce the following five (random) functions ρ^{(m)}, 1 ≤ m ≤ 5, defined on the measurable sets A ∈ X of the space (X, X). Put ρ^{(1)}(A) = μ_{n}(A), ρ^{(2)}(A) = −μ(A), ρ^{(3)}(A) = −2μ(A)μ_{n}(A), ρ^{(4)}(A) = μ(A)^{2} and ρ^{(5)}(A) = μ_{n}(A)^{2} − \frac{1}{n}μ_{n}(A) for all A ∈ X. Let us fix some positive integer r and r disjoint subsets C_{j} ∈ X of the space X, 1 ≤ j ≤ r, such that μ(C_{j}) ≤ ε for all 1 ≤ j ≤ r with some r ≥ 1. Let us choose some sequence of integers m(j), 1 ≤ j ≤ r, such that 1 ≤ m(j) ≤ 5 for all 1 ≤ j ≤ r and m(j) ≥ 3 for at least one of these indices. I claim that

E ∏_{j=1}^{r} |ρ^{m(j)}(C_{j})| ≤ εC̄(r, n) ∏_{j=1}^{r} μ(C_{j})  (4.16)

for the above defined functions ρ^{(m)} with some appropriate constant C̄(r, n). (One could get rid of the dependence of n in the constant C̄(r, n) but such an improvement of the inequality (4.16) has no great importance for us.)

First I show that relation (4.13) can be proved with the help of the estimate (4.16). For this aim I rewrite the expressions Z_{N} and Z̄_{N} with the help of the above defined functions ρ^{(m)}. To do this let us first fix a pair of indices (s_{1}, …, s_{k_{1}}) and (t_{1}, …, t_{k_{2}}) in the outer sum in formulas (4.11) and (4.12), and let us express the products corresponding to it in formulas (4.11) and (4.12) by means of the functions ρ^{(m)}. In this calculation let us first consider those pairs (j, j′), 1 ≤ j ≤ k_{1}, 1 ≤ j′ ≤ k_{2} together with the indices s_{j} and t_{j′} for which (j, j′) ∈ N, and let us express their contribution to the products we consider in the desired form. In formula (4.11) the expression (μ_{n}(A(j, s_{j}))−μ(A(j, s_{j})))(μ_{n}(B(j′, t_{j′}))−μ(B(j′, t_{j′}))) = (μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})))^{2} has to be considered, and it can be rewritten as ρ^{(5)}(A(j, s_{j})) + \frac{1}{n}ρ^{(1)}(A(j, s_{j})) + ρ^{(3)}(A(j, s_{j}))+ρ^{(4)}(A(j, s_{j})). Similarly, in formula (4.12) we have to take the expression \frac{1}{n}μ_{n}(A(j, s_{j})), and it can be rewritten as \frac{1}{n}ρ^{(1)}(A(j, s_{j})). The terms corresponding to the remaining indices j which are not end-points of some edge in the first row of the diagram N, have the form μ_{n}(A(j, s_{j})) − μ(A(j, s_{j})) both in formula (4.11) and (4.12), and they can be rewritten as ρ^{(1)}(A(j, s_{j})) + ρ^{(2)}(A(j, s_{j})). Similarly, the terms corresponding to some j′ which is not an end-point of an edge in the second row of the diagram N have the form μ_{n}(B(j′, t_{j′})) − μ(B(j′, t_{j′})) both in (4.11) and (4.12), and they can be rewritten as ρ^{(1)}(B(j′, t_{j′})) + ρ^{(2)}(B(j′, t_{j′})). By taking the product of the appropriate expressions we can express each term in the sum in formulas (4.11) and (4.12) by means of the quantities ρ^{(m)}.

Let us consider the difference Z_{N} − Z̄_{N} rewritten in the above form as the function of the terms ρ^{m(j)}(A_{j}, s_{j}), 1 ≤ j ≤ k_{1} and ρ^{m(j′)}(B_{j′}, t_{j′}), j′ ∈ {1, …, k_{2}} \ {j′_{1}, …, j′_{l}}. In such a way we can express Z_{N} − Z̄_{N} as the linear combination of some terms of the form

∏_{j=1}^{k_{1}} ρ^{m(j)}(A(j, s_{j})) ∏_{j′∈{1,...,k_{2}}\{j′_{1},...,j′_{l}}} ρ^{m(j′)}(B(j′, t_{j′})),  (4.17)

with coefficients between zero and 1, where {j′_{1}, …, j′_{l}} is the set consisting of the endpoints of the edges of the diagram B(N) in the second row. Beside this, it follows from the above construction that in the terms described in (4.17) m(j) ≥ 3 for at least one of the indices j_{1}, …, j_{l}, where j_{1}, …, j_{l} are the end-points of the edges of the diagram N in the first row. Since by our assumption the inequalities μ(A(j, s)) ≤ ε and μ(B_{j′}, t) ≤ ε hold for all terms we consider, formula (4.16) can be applied with the choice r = k_{1} + k_{2} − l to estimate all terms of the form (4.17). It yields that

\begin{matrix}E |∏_{j=1}^{k_{1}} ρ^{m(j)}(A(j, s_{j})) ∏_{j′∈\{1,...,k_{2}\}\\{j′_{1},...,j′_{l}\}} ρ^{m(j′)}(B(j′, t_{j′}))| \\ ≤ εC̄(k_{2}, n) ∏_{j=1}^{k_{1}} μ(A(j, s_{j})) ∏_{j′∈\{1,...,k_{2}\}\\{j′_{1},...,j′_{l}\}} μ(B(j′, t_{j′})) \\ = εC̄(k_{1} + k_{2}, n)μ^{k_{1}+k_{2}−l} (∏_{j=1}^{k_{1}} A(j, s_{j}) × ∏_{j′∈\{1,...,k_{2}\}\\{j′_{1},...,j′_{l}\}} B(j′, t_{j′})) , \end{matrix} (4.18)

where μ^{k_{1}+k_{2}−l} denotes the k_{1} + k_{2} − l-fold direct product of the measure μ with itself on the space (X^{k_{1}+k_{2}−l}, X^{k_{1}+k_{2}−l}), and the product at the right-hand side of (4.18) denotes the direct product of the corresponding sets.

The estimate (4.13) follows from inequality (4.18) if we sum up this estimate for all such terms of the form (4.17) which appear in the difference Z_{N} − Z̄_{N} when it is written in the above described form. We can see this if we observe that any two sets A(j, s) and B(j′, t) appearing in one of these expressions are disjoint unless they are both functions of the same type A(·, ·) or B(·, ·), and their arguments (j, s_{j}) or (j′, t_{j′})

agree. This fact implies that those sets ∏_{j=1}^{k_{1}} A(j, s_{j}) × ∏_{j′∈{1,...,k_{2}}\{j′_{1},...,j′_{l}}} B(j′, t_{j′}) at

the right-hand side of (4.18) whose μ^{k_{1}+k_{2}−l} measures have to be summed up to get an estimate for the difference E|Z_{N} − Z̄_{N} | are either disjoint or they coincide. Since each term appears less than 5^{k_{1}+k_{2}} times in one of these estimates this means that formula (4.16) implies formula (4.13) with the choice C(k_{1} + k_{2}, n) = 5^{k_{1}+k_{2}} C̄(k_{1} + k_{2}, n).

To prove formula (4.16) let us express the functions ρ^{(1)}(C_{j}), ρ^{(3)}(C_{j}) and ρ^{(5)}(C_{j}) with the help of the measures ν̄_{l} defined as

ν̄_{l}(A) = ν̄_{l}(A, ω) = \{ \begin{matrix}1 & if ξ_{l}(ω) ∈ A \\ 0 & if ξ_{l}(ω) ∉ A \end{matrix} for all A ∈ X and 1 ≤ l ≤ n.

We can write ρ^{(1)}(C_{j}) = \frac{1}{n} ∑_{l_{j}=1}^{n} ν̄_{l_{j}}(C_{j}), ρ^{(3)}(C_{j}) = −\frac{2}{n}μ(C_{j}) ∑_{l_{j}=1}^{n} ν̄_{l_{j}}(C_{j}), and

ρ^{(5)}(C_{j}) = \frac{1}{n^{2}} ∑_{l_{j}=1}^{n} ν̄_{l_{j}}(C_{j}) − (\frac{1}{n} ∑_{l_{j}=1}^{n} ν̄_{l_{j}}(C_{j}))^{2} = −\frac{1}{n^{2}} ∑_{1≤l_{j},l′_{j}≤n, l_{j}≠l_{j′}} ν̄_{l_{j}}(C_{j})ν̄_{l_{j′}} (C_{j}).

Let us observe that only pairs (l_{j}, l_{j′}) with l_{j} ≠ l_{j′} are considered at the right-hand side of the last formula, since ν̄_{l}(C) − ν̄_{l}^{2}(C) = 0 for all C ∈ X.

Let us insert these formulas to the left-hand side of formula (4.16), carry out the multiplications, and let us bound the absolute value of the sum obtained in such a way as the sum of the absolute value of these terms. All these terms are products of expressions of the form ν̄_{l_{j}}(C_{j}), ν̄_{l_{j′}} (C_{j}) or μ(C_{j}). Let us observe that only those products have to be considered for which the indices l_{j} and l_{j′} of the measures ν_{l_{j}} and ν_{l_{j′}} are all different for different indices j. Otherwise the product we consider equals zero. We can see this by observing that the sets C_{j} are disjoint. Therefore if l_{j_{1}} = l_{j_{2}} for some j_{1} ≠ j_{2}, then either ν̄_{l_{j_{1}}} (C_{j_{1}}) = 0 or ν̄_{l_{j_{2}}} (C_{j_{2}}) = 0. On the other hand, the measures ν̄_{l} are independent for different indices l, and this implies that the terms we have to bound are the products of r independent random variables (some of these variables may be constant). Since the sum we have to bound contains less than n^{r} terms to prove (4.16) it is enough to show that the expectation of the absolute value of these terms can

be bounded by const. ε ∏_{j=1}^{r} μ(C_{j}). Since Eν̄_{l_{j}}(C_{j}) = Eν̄_{l_{j′}} (C_{j}) = μ(C_{j}), the product

whose expected value has to be estimated is the product of r such independent random variables whose expected values can be bounded μ(C_{j}), 1 ≤ j ≤ r. Moreover, there is at least one index j for which m(j) ≥ 3 in the expression (4.16). This property together with the condition μ(C_{j}) ≤ ε imply that the product whose expected value we have to estimate contains at least one factor whose expected value is less than 2εμ(C_{j}). This means that the inequality needed for the proof of relation (4.16) holds.

Let us observe that both sides of formula (3.11) is a bilinear form of the functions f and g. This fact implies that it holds not only for those pairs (f, g) of functions we considered before, but also for any finite linear combinations of such functions f and g. Such linear combinations are dense in the spaces L_{2}(X^{k_{1}}, X^{k_{1}}, μ^{k_{1}}) and L_{2}(X^{k_{2}}, X^{k_{2}}, μ^{k_{2}}) respectively. Hence to complete the proof of Lemma 2 it is enough to show that if a sequence of functions f_{n} and g_{n} satisfy the relations ∥f_{n} − f∥_{2} → 0 and ∥g_{n} − g∥_{2} → 0 with some functions f and g, then

lim_{n→∞} E|J_{n,k_{1}}(f_{n})J_{n,k_{2}}(g_{n}) − J_{n,k_{1}}(f)J_{n,k_{2}}(g)| = 0  (4.19)

and

lim_{n→∞} E|J_{n,k_{1}+k_{2}−l−p}(f_{n} ∘ g_{n})_{B(N ,N_{1})} − J_{n,k_{1}+k_{2}−l−p}(f ∘ g)_{B(N ,N_{1})}| = 0  (4.20)

for all coloured diagrams B(N , N_{1}) ∈ B(l, p), 0 ≤ p ≤ l ≤ min(k_{1}, k_{2}).

To prove relation (4.19) it is enough to prove that E(J_{n,k_{1}}(f_{n}) − J_{n,k_{1}}(f))^{2} = E((J_{n,k_{1}}(f_{n} − f))^{2} → 0 if ∥f_{n} − f∥_{2} → 0, and E(J_{n,k_{2}}(g_{n} − g))^{2} → 0 if ∥g_{n} − g∥_{2} → 0. Let us prove the inequality E(J_{n,k}(h))^{2} ≤ C(n, k_{1})∥h∥_{2}^{2} with some constant C(k, n) for a function h of k variables. The relation we want to prove is a consequence of this inequality. On the other hand, this inequality can be deduced from some arguments presented in the proof of Lemma 1. Indeed, similarly to the argument presented there we can reduce the statement to be proved to the inequality ∫ h̄^{2}(x_{1}, …, x_{k})μ_{n}( du_{1}) … μ_{n}( duk̄) ≤ C(n, k)∥h̄∥_{2}^{2} with an appropriate constant C(n, k) for any function h̄ of k̄ ≤ k variables such that ∥h̄∥_{2}^{2} < ∞. But this statement is contained in formula (4.5). We only have to apply it for the integrable function h̄^{2}.

Relation (4.20) can be deduced from the results proved at the end of Lemma 1 in an even simpler way. Because of the results proved there it is enough to check that ∥(f_{n} ∘ g_{n})_{B(N ,N_{1})} − (f ∘ g)_{B(N ,N_{1})}∥_{1} → 0 if ∥f_{n} − f∥_{2}^{2} → 0 and ∥g_{n} − g∥_{2}^{2} → 0. On the other hand, it follows from the Schwarz inequality and the contraction property of the operator P defined in (3.2) in the L_{2} norm that

∥(f ∘ g_{n})_{BN ,N_{1})} − (f ∘ g)_{B(N ,N_{1})}∥_{1} = ∥(f ∘ (g_{n} − g))_{B(N ,N_{1})}∥_{1} ≤ ∥f∥_{2}^{2}∥g − g_{n}∥_{2}^{2},

hence ∥(f ∘ g_{n})_{B(N ,N_{1})} − (f ∘ g)_{B(N ,N_{1})}∥_{1} → 0. Similarly,

∥(f_{n} ∘ g_{n})_{B(N ,N_{1})} − (f ∘ g_{n})_{B(N ,N_{1})}∥_{1} → 0.

These inequalities imply relation (4.20). Lemma 2 is proved.

Lemma 3 which gives an estimate on the second moment of a random integral J_{n,k}(f) can be proved as a consequence of Lemmas 1 and 2.

The proof of Lemma 3. By Lemma 2 we have

EJ_{n,k}(f)^{2} = ∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!}n^{−(l−p)/2}EJ_{n,2k−l−p}(f ∘ f_{l,p})

with the functions f ∘ f_{l,p} defined in formula (3.12). Hence by formulas (3.8) and (3.10) in Lemma 1

EJ_{n,k}(f)^{2} ≤ ∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!}n^{−(l−p)/2} \frac{C^{2k−l−p}}{(2k − l − p)^{(2k−l−p)/2}} ∥f ∘f_{l,p}∥_{1} (4.21)

with some constant C > 0. We claim that ∥f ∘ f_{l,p}∥_{1} ≤ ∥f∥_{2}^{2}. Since f ∘ f_{l,p} is the average of certain functions of the form f ∘ f_{B(N ,N_{1})} with some diagram B(N , N_{1}) it is enough to show that ∥f ∘ f_{B(N ,N_{1})}∥_{1} ≤ ∥f∥_{2}^{2} for an arbitrary diagram B(N , N_{1}). We get similarly to the proof of formula (3.15) by considering first functions of the form \overline{f ∘ f}_{B(N)} defined in formula (3.5) and applying the Schwarz inequality for them that

∥\overline{f ∘ f}_{B(N)}∥_{1} ≤ ∫ f^{2}(x_{1}, …, x_{k})μ( dx_{1}) … μ( dx_{k_{1}}) = ∥f∥_{2}^{2}

for an arbitrary diagram N. Since the operators P_{j} defined in formula (3.2) are contractions also in L_{1} norm, the last inequality implies that the relation ∥f ∘f_{B(N ,N_{1})}∥_{1} = ∥\overline{f ∘ f}_{B(N ,N_{1})}∥_{1} ≤ ∥\overline{f ∘ f}_{B(N)}∥_{1} ≤ ∥f∥_{2}^{2} holds, as we claimed. By formula (4.21), the inequality ∥f ∘ f_{l,p}∥_{1} ≤ ∥f∥_{2}^{2} and the Stirling formula we get that

EJ_{n,k}(f)^{2} ≤ C^{k} ∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)^{(2k−l−p)/2}}{(k − l)^{2(k−l)}(l − p)^{l−p}p^{p}} n^{−(l−p)/2}∥f∥_{2}^{2}  (4.22)

with a possibly new constant C > 0.

Since the right-hand side of (4.22) contains less than k^{2} terms and n ≥ k this relation enables us to reduce the proof of Lemma 3 to the estimate

(2k − l − p)^{(2k−l−p)/2} ≤ C̄^{k}(k − l)^{2(k−l)}(l − p)^{l−p}p^{p}k^{(l−p)/2}k^{−k}

with an appropriate C̄ ≥ 0 for all 0 ≤ p ≤ l ≤ k. Since (k − l)^{k−l}p^{p}(l − p)^{l−p} ≤ 3^{−k}k^{k} (this follows from instance from the convexity of the function x log x if we take logarithm), to prove the last inequality it is enough to show that

(\frac{2k − l − p}{2})^{(2k−l−p)/2} ≤ C̃^{k}(k − l)^{(k−l)}k^{(l−p)/2} if 0 ≤ p ≤ l ≤ k

with some C̃ > 0. By dividing both sides of the last inequality by k^{(2k−l−p)/2} and after this taking the k-th root of both sides we get that the last inequality is equivalent to the statement

(\frac{2k − l − p}{2k})^{(2k−l−p)/2k} ≤ C̃ (\frac{k − l}{k} )^{(k−l)/k} if 0 ≤ p ≤ l ≤ k.  (4.23)

Inequality (4.23) holds. This inequality holds since the left-hand side can be bounded by sup_{0≤u≤1} u^{u} ≤ 1 from above, and the right-hand side by C̃ inf_{0≤u≤1} u^{u} ≥ C̃e^{−1/e} from

below. Lemma 3 is proved.

The proof of Lemma 4. Let us first investigate the main result of Lemma 4 which states that the function f ∘ g is (r_{1} + r_{2} − (l − p), σ^{2}) dominated under appropriate conditions. Let us fix a coloured diagram B(N , N_{1}) = B(N , N_{1}, k_{1}, k_{2}) with

N = {(j_{1}, j′_{1}), …, (j_{l}, j′_{l})} and N_{1} = {(j_{u(1)}, j′_{u(1)}), …, (j_{u(p)}, j′_{u(p)})},  (4.24)

where \{u(1), …, u(p)\} ⊂ \{1, …, l\}. It is enough to prove that the function \overline{f ∘ g}_{B(N ,N_{1})} defined in formula (3.6) is (r_{1} + r_{2} − (l − p), σ^{2}) dominated. We have certain freedom to change the order of the operators R_{j,j′} and P_{j} in the definition of this function. By exploiting this freedom we can rewrite the function \overline{f ∘ g}_{B(N ,N_{1})} as

\begin{matrix}\overline{f ∘ g}_{B(N ,N_{1})}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−l−p)}) \\ = ∏_{s=1}^{l−p} R_{j_{v(s)},j′_{v(s)}} ∏_{t=1}^{p} P_{j_{u(t)}}R_{j_{u(t)},j′_{u(t)}}f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) \end{matrix}  (4.25)

with {v(1), …, v(l − p)} = {1, …, l} \ {u(1), …, u(p)}, i.e. the operators R_{j_{v(s)},j′_{v(s)}} are indexed by the edges

M = N \ N_{1} = {(j_{v(1)}, j′_{v(1)}), …, (j_{v(l−p)}, j′_{v(l−p)})}  (4.26)

of the diagram B(N , N_{1}). Beside this, we can choose the order of the operators R_{j_{v(s)},j′_{v(s)}} in an arbitrary way. Let me recall that in our notations the operators written after each other are applied in right to left order.

Lemma 4 will be proved with the help of an inductive procedure described below. To carry it out let us introduce some notations. Let us consider a function F (x_{s(1)}, …, x_{s(m)}) with the set of indices S = {s(1), …, s(m)} ⊂ {1, …, k_{1} + k_{2}} together with a diagram B(M) = B(M, S) consisting of two rows {s(1), …, s(r)} and {s(r + 1), …, s(m)}, where the index r is defined by the relations s(r) ≤ k_{1} < s(r + 1). Beside this, the diagram B(M, S) has some edges M = {(j_{1}, j′_{1}), …, (j_{q}, j′_{q})} with 0 ≤ q ≤ min(r, m − r), 1 ≤ j_{1} < ⋯ j_{q} ≤ s(r), {j_{1}, …, j_{q}} ⊂ {s(1), …, s(r)}, {j′_{1}, …, j′_{q}} ⊂ {s(r + 1), …, s(m)} and j′_{u} ≠ j′_{u′} if u ≠ u′, 1 ≤ u, u′ ≤ q. We introduced the diagram B(M, S) to indicate for which pairs (j, j′) we want to apply the operator R_{j,j′} for the function F during the inductive procedure leading to the proof of Lemma 4.

Let us call a partition B_{1}, …, B_{r}, B_{u} = {s(d_{(u,1)}), …, s(d_{(u,v(u))})}, 1 ≤ u ≤ r, of the set S = {s(1), …, s(m)} together with a set of functions h_{1}, …, h_{r}, such that the function h_{u} = h_{u} (x_{s(d_{(u,1)})}, …, x_{s(d_{(u,v(u))})}) depends only on the variables indexed by

the elements of the set B_{u} for all 1 ≤ u ≤ r, and ∥h_{u}∥_{∞} ≤ 1, ∥h_{u}∥_{2}^{2} ≤ σ^{2}, 1 ≤ u ≤ r, an (r, σ^{2}) dominating system of the function F if also the relation

|F (x_{s(1)}, …, x_{s(m)})| ≤ ∏_{u=1}^{r} h_{u} (x_{s(d_{(u,1)})}, …, x_{s(d_{(u,v(u))})}) for all x_{s(l)} ∈ X, 1 ≤ l ≤ m

holds. Let us call this (r, σ^{2}) dominating system regular (with respect to the function F and the diagram B(M) attached to it) if the two end-points j_{i} and j′_{i} of the edges of the diagram B(M) are contained in different sets B_{u(i)} and B_{u′(i)} of the partition B_{1}, …, B_{r} for all 1 ≤ i ≤ q. We call an (r, σ^{2}) dominating system super-regular if all elements B_{1}, …, B_{r} of the partition of this system are contained either in the set {1, …, k_{1}} or in the set {k_{1} +1, …, k_{1} +k_{2}}. A super-regular (r, σ^{2}) dominating system is clearly regular with respect to any diagram B(M), since all edges of a diagram B(M) are going between the vertices of the sets {1, …, k_{1}} and {k_{1} + 1, …, k_{1} + k_{2}}.

It follows from the conditions of Lemma 4 that the function f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) defined in formula (3.4) has a super-regular (r_{1} + r_{2}, σ^{2}) dominating system. Indeed, the function f(x_{1}, …, x_{k_{1}}) can be dominated by a partition B_{1}, …, B_{r_{1}} of the set {1, …, k_{1}} together with some functions h_{1}, …, h_{r_{1}} such that the indices of the arguments of the function h_{u} are contained in B_{u}, 1 ≤ u ≤ r_{1}, and the function g (x_{k_{1}+1}, …, x_{k_{1}+k_{2}}) by a partition B_{r_{1}+1}, …, B_{r_{1}+r_{2}} of the set {k_{1} + 1, …, k_{1} + k_{2}} together with some functions h_{r_{1}+1}, …, h_{r_{1}+r_{2}} such that the indices of the arguments of the function h_{u} are contained in B_{u}, r_{1} + 1 ≤ u ≤ r_{1} + r_{2}. Beside this ∥h_{u}∥_{∞} ≤ 1 and ∥h_{u}∥_{2}^{2} ≤ σ^{2} for all 1 ≤ u ≤ r_{1} + r_{2}. The union of these sets and functions yield a super-regular (r_{1} + r_{2}, σ^{2}) dominating system of the function f ∘ g consisting of the sets B_{1}, …, B_{r_{1}+r_{2}} and the functions h_{1}, …, h_{r_{1}+r_{2}}.

We shall prove Lemma 4 with the help of the above observation, formula (4.25) and some properties of the operators P_{j} and R_{j,j′}. First we show by induction that

the function ∏_{t=1}^{p} P_{j_{t}}R_{j_{t},j′_{t}}f ∘ g(x_{1}, …, x_{k_{1}+k_{2}}) = \overline{f ∘ g}_{B(N_{1},N_{1})} also has a super-regular

(r_{1} + r_{2}, σ^{2}) dominating system.

We shall prove the following statement which implies the above relation. Let a function G(x_{s_{1}}, …, x_{s_{m}}) have an (r, σ^{2}) super-regular dominating system and take two indices 1 ≤ j, ≤ k_{1} < j′ ≤ k_{1} + k_{2} such that {j, j′} ⊂ {s_{1}, …, s_{m}}. Then the function P_{j}R_{j,j′}G also has an (r, σ^{2}) super-regular dominating system. This statement enables us to show with the help of a simple induction that together with the function f ∘ g

also the function \overline{f ∘ g}_{B(N_{1},N_{1})} = ∏_{t=1}^{p} P_{j_{t}}R_{j_{t},j′_{t}}f ∘ g has a super-regular (r_{1} + r_{2}, σ^{2})

dominating system.

To prove the above statement let us consider a super-regular (r, σ^{2}) dominating system of the function G consisting of a partition B_{1}, …, B_{r} of the set {s_{1}, …, s_{m}} and some functions h_{l}, 1 ≤ l ≤ r, depending on the arguments indexed by the set B_{l}. To construct a super-regular dominating system of the function P_{j}R_{j,j′}G let us introduce the following notations. Let u_{1} and u_{2} denote those indices for which j ∈ B_{u_{1}} and j′ ∈ B_{u_{2}}, and let us denote the elements of these sets as B_{u_{1}} = {j, v(1), …, v(m)}, B_{u_{2}} = {j′, v′(1), …, v′(m′)}. Then we define B̄_{u} = B_{u}, h̄_{u} = h_{u} if u ≠ u_{1} and u ≠ u_{2}, B̄_{u_{1}} = B_{u_{1}} \ {j}, B̄_{u_{2}} = B_{u_{2}} \ {j′}, and

h̄_{u_{1}}(x_{v(1)}, …, x_{v(m)}) = (∫ h_{u_{1}}^{2}(z_{j}, x_{v(1)}, …, x_{v(m)})μ( dz_{j}))^{1/2} ,

h̄_{u_{2}}(x_{v′(1)}, …, x_{v′(m′)}) = (∫ h_{u_{2}}^{2}(z_{j′}, x_{v′(1)}, …, x_{v′(m′)})μ( dz_{j′}))^{1/2}

if u = u_{1} or u = u_{2}. We claim that the above defined sets B̄_{l} and functions h̄_{l}, 1 ≤ l ≤ r, supply a super-regular (r, σ^{2}) dominating system of the function P_{j}R_{j,j′}G. To show this let us first observe that because of the Schwarz inequality

\begin{matrix}|P_{j}R_{j,j′}h_{u_{1}}(x_{j}, x_{v(1)}, …, x_{v(m)})h_{u_{2}}(x_{j′}, x_{v′(1)}, …, x_{v′(m′)})| \\ ≤ h̄_{u_{1}}(x_{v(1)}, …, x_{v(m)})h̄_{u_{2}}(x_{v′(1)}, …, x_{v′(m′)}). \end{matrix}  (4.27)

Since the product of the functions h_{l}, 1 ≤ l ≤ r, gives an upper bound of the function G, formula (4.27) and the definition of the functions h̄_{l} imply that the product of the functions h̄_{l} yields an upper bound for the function P_{j}R_{j,j′}G. The functions h̄_{u_{j}}, j = 1, 2, satisfy also the relation ∥h̄_{u_{j}}∥^{2} = ∥h_{u_{j}}∥^{2} ≤ σ^{2}. It is not difficult to see with the help of these relations that the sets B̄_{l} and h̄_{l}, 1 ≤ l ≤ r, yield a super-regular (r, σ^{2}) dominating system of the function R_{j.j′}P_{j}G.

Thus we have proved that, with the notation (4.24) and (4.26), the function \overline{f ∘ g}_{B(N_{1},N_{1})}(x_{s(1)}, …, x_{s(k_{1}+k_{2}−2p)}) together with the diagram B(M) attached to it, where M = N \ N_{1} = {(j_{v(1)}, j′_{v(1)}), …, (j_{v(l−p)}, j′_{v(l−p)})}, and the indices of the arguments of the function \overline{f ∘ g}_{B(N_{1},N_{1})} are

{s(1), …, s(k_{1} − p)} = {1, …, k_{1}} \ {j_{u(1)}, …, j_{u(p)}},
{s(k_{1} − p + 1), …, s(k_{1} + k_{2} − 2p)} = {k_{1} + 1, …, k_{1} + k_{2}} \ {j′_{u(1)}, …, j′_{u(p)}},

has a regular (r_{1} + r_{2}, σ^{2}) dominating system. We want to prove that the function

\overline{f ∘ g}_{B(N ,N_{1})} = ∏_{m=1}^{l−p} R_{j_{v(m)},j′_{v(m)}} \overline{f ∘ g}_{B(N_{1},N_{1})}  (4.28)

has an (r_{1} + r_{2} − (l − p), σ^{2}) dominating system. Let us recall that |M| = l − p and the operators R_{j_{v(m)},j′_{v(m)}} considered in formula (4.28) are commutative.

The above statement will be proved by the successive application of a statement formulated below. Let a function G(x_{s(1)}, …, x_{s(m)}) and a diagram B(M) attached to it have a regular (r, σ^{2}) dominating system consisting of some partition B_{1}, …, B_{r} of the set {s(1), …, s(m)} and some functions h_{l}, 1 ≤ l ≤ r, depending on variables indexed by the set B_{l}. Let us consider two sets B_{l} and B_{l′} such that there are edges of the diagram M whose end-points are in the sets B_{l} and B_{l′}. By a new enumeration of the sets of the partition if it is needed we may assume that l = r − 1 and l′ = r. Let M_{0} = {(s(1), s(1)′), …, (s(q), s(q)′)}, q ≥ 1, denote the set of all edges of M which connect some points of the sets B_{r−1} and B_{r}. Here we do not specify which element of a pair (s(t), s(t)′) is in the set B_{r−1} and which element is in the set B_{r}, 1 ≤ t ≤ q.

We claim that the function ∏_{t=1}^{q} R_{s(t),s(t)′}G together with the diagram B(M \ M_{0}) has

a regular (r − 1, σ^{2}) dominating system. (Let us recall that the diagram attached to the function G has the role to list the pairs (j, j′) for which we want to apply the operator R_{j,j′}.) More explicitly, we can get a regular (r − 1, σ^{2}) dominating system (B̄_{l}, h̄_{l}), 1 ≤ l ≤ r − 1, in the following way. Put B̄_{l} = B_{l} and h̄_{l} = h_{l}, 1 ≤ l ≤ r − 2,

B̄_{r−1} = B_{r−1} ∪ B_{r} \ {s(1)′, …, s(q)′} and h̄_{r−1} = ∏_{t=1}^{q} R_{s(t),s(t)′}h_{r−1}h_{r}. It is rather

straightforward to check that the above sets B̄_{l} and functions h̄_{l}, 1 ≤ l ≤ r−1 constitute a regular (r−1, σ^{2}) dominating system. The only statement which demands some special consideration is the inequality ∥h̄_{r−1}∥_{2}^{2} ≤ σ^{2}. We can see with the help of the Schwarz inequality (similarly to the corresponding argument applied in the proof of formula (3.14)) that

∥h̄_{r−1}∥_{2}^{2} = ∥∏_{t=1}^{q} R_{s(t),s(t)′}h_{r−1}h_{r}∥_{2}^{2} ≤ ∥h_{r−1}∥_{2}∥h_{r}∥_{2} ≤ σ^{2}.

By applying the above fact for the function at the right-hand side of (4.28) s ≤ l−p times we get that the function \overline{f ∘ g}_{B(N ,N_{1})} is (r_{1} + r_{2} − s, σ^{2}) dominated with some s ≤ l−p. (At each step we take into consideration at least one new operator R_{j_{v(m)},j′_{v(m)}} in formula (4.28), so the procedure finishes in s ≤ l − p steps.) This property implies that the function \overline{f ∘ g}_{B(N ,N_{1})} has an (r_{1} + r_{2} − s, σ^{2}) dominating system with some number s ≤ l − p. To give a complete proof for the main result of Lemma 4 let us observe that in the case of a ‘too good estimate’ when s < l − p we can construct an (r, σ^{2}) dominating system of the function \overline{f ∘ g}_{B(N ,N_{1})} with r = r_{1} + r_{2} − (l − p). We get such a system by taking the union of some sets B_{l} of an (r̄, σ^{2}) dominating system if r̄ > r, and by considering the product of the functions h_{l} corresponding to them.

Finally, the last statement of Lemma 4 is a simple consequence of formula (3.14). Indeed, if f is (r_{1}, σ^{2}) and g is (r_{2}, σ^{2}) dominated, then ∥f∥_{2}^{2} ≤ σ^{2r_{1}}, ∥g∥_{2}^{2} ≤ σ^{2r_{2}}, hence ∥f ∘ g_{B(N ,N_{1})}∥_{2}^{2} ≤ σ^{r_{1}+r_{2}} by relation (3.14). On the other hand, ∥f∥_{∞} ≤ 1 and ∥g∥_{∞} ≤ 1, hence ∥f ∘ g_{B(N ,N_{1})}∥_{∞} ≤ 1. These inequalities mean that f ∘ g_{B(N ,N_{1})} is an (1, σ^{(r_{1}+r_{2})/2}) dominated function as we claimed.

5. The proof of Proposition 2 together with a technical lemma

First I prove Proposition 2 with the help of Lemma 5 formulated in Section 2. Then I also prove Lemma 5.

The proof of Proposition 2 with the help of Lemma 5. By Lemma 3 formula (3.17) holds for m = 0 and all k = 1, 2, … with the choice C(k, 0) = C^{k} with some C > 0. Let us assume that it holds for some m and all k ≥ 1 with appropriate norming constants C(k, m). Then I prove it by induction for m + 1 with appropriate norming constants C(k, m + 1).

To do this let us fix some function f(x_{1}, …, x_{k}) of k variables, consider the functions f ∘ f_{l,p} defined in formula (3.12) and give a good bound for the 2M = 2^{m+1}-th moment of the random integrals J_{n,2k−l−p}(f ∘ f_{l,p}). Then the 4M = 2^{m+2}-th moment of the random integral J_{n,k}(f) (in the case n ≥ 2kM) can be estimated by means of these bounds and the diagram formula.

Let f = f(x_{1}, …, x_{k}) be a function of k variables, k ≥ 1, which is (r, σ^{2/r}) dominated with some r ≥ 1. I show with the help of Lemma 4 and formula (3.17) with the parameter M = 2^{m} that the inequality

\begin{matrix}E (n^{−(l−p)/2}J_{n,2k−l−p}(f ∘ f_{l,p}))^{2M} ≤ (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M} \\ · \frac{2^{−2kM}}{(2k)^{(l−p)M}M^{2lM}} max (1, (\frac{2kM}{nσ^{2/r}} )^{2M min(k,r)}) , 0 ≤ p ≤ l ≤ k \end{matrix} (5.1)

holds if 2kM ≤ n. This formula also holds in the degenerated case p = l = k with the choice C(0, m) = 1. Indeed, in this case 2k − p − l = 0, hence f ∘ f_{k,k} is a constant, and EJ_{n,0}(f ∘ f_{k,k})^{2M} = (f ∘ f_{k,k})^{2M} ≤ (σ^{2})^{2M}. The last inequality follows from the definition of the function f ∘ f_{k,k} = f ∘ f_{B(N ,N)}, |N | = k, given in formula (3.12) and the estimate |f ∘ f_{B(N ,N_{1})}| ≤ ∥f∥_{2}^{2} ≤ σ^{2} if N = N_{1}, |N | = k, (k is the number of variables of the function f). The last inequality follows from relation (3.15). These relations imply that |f ∘ f_{k,k}| ≤ σ^{2}. We had to consider this case separately, because our inductive hypothesis (3.17) is formulated only for functions with k ≥ 1 variables.

The integral J_{n,2k−l−p}(f ∘ f_{l,p}) is the average of the random integrals J_{n,2k−l−p}(f ∘ f_{B(N ,N_{1})}) with the functions f_{B(N ,N_{1})}(x_{1}, …, x_{2k−l−p}) such that B(N , N_{1}) ∈ B(l, p). Hence the triangular inequality for the L_{2M} norm in the probability space (Ω, A, P) where the random variables ξ_{j} are living implies that it is enough to prove such a version of formula (5.1) where the random integral J_{n,k}(f ∘f_{l,p}) at the left-hand side is replaced by an arbitrary random integral J_{n,2k−l−p}(f ∘ f_{B(N ,N_{1})}) such that B(N , N_{1}) ∈ B(l, p).

Let us apply formula (3.17) for a random integral J_{n,2k−l−p}(f ∘f_{B(N ,N_{1})}) such that B(N , N_{1}) ∈ B(l, p). (In this case (2k−l−p)M ≤ n, and our inductive hypothesis allows the application of formula (3.17).) Let us consider the cases 2r ≥ l−p+1 and 2r ≤ l−p separately. In the first case the function f ∘ f_{B(N ,N_{1})} is (2r − l + p, σ^{2/r}) or in other notation (2r − l + p, σ̄^{2/(2r−l+p)}) dominated with σ̄ = σ^{(2r−l+p)/r} by Lemma 4. Let us apply relation (3.17) with the choice k̄ = 2k − l − p, r̄ = 2r − l + p and σ̄ = σ^{(2r−l+p)/r} in this case. Observe that σ^{2/r} = σ̄^{2/r̄}. We get that

E (n^{−(l−p)/2}J_{n,2k−l−p}(f ∘ f_{B(N ,N_{1})}))^{2M} ≤ (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}

· \frac{2^{−2kM}}{M^{(l+p)M}} \frac{1}{(nσ^{2/r})^{(l−p)M}} max (1, (\frac{2kM}{nσ^{2/r}} )^{M min(2k−l−p,2r−l+p)})

= (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}

· \frac{2^{−2kM}}{M^{2lM}(2k)^{(l−p)M}} max ((\frac{2kM}{(nσ^{2/r}} )^{(l−p)M} , (\frac{2kM}{nσ^{2/r}} )^{M min(2k−2p,2r)})

≤ (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}

· \frac{2^{−2kM}}{M^{2lM}(2k)^{(l−p)M}} max (1, (\frac{2kM}{nσ^{2/r}} )^{2M min(k,r)}) ,  (5.2)

since the exponents (l − p)M and min(2k − 2p, 2r)M in the last but one inequality of formula (5.2) satisfy the inequality 0 ≤ (l − p)M ≤ min 2M(k, r), and 0 ≤ min(2k − 2p, 2r)M ≤ min 2M(k, r).

In the second case when 2r ≤ l − p we know by Lemma 4 that the function f ∘ f_{B(N ,N_{1})} is (1, σ^{2}) dominated. In this case we apply formula (3.17) with the k̄ = 2k − l − p, r̄ = 1 and σ̄ = σ. We get that

E (n^{−(l−p)/2}J_{n,2k−l−p}(f_{B(N ,N_{1})}))^{2M} ≤ (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}

· \frac{2^{−2kM}}{M^{(l+p)M}} \frac{1}{σ^{2M}n^{(l−p)M}} max (1, (\frac{2kM}{nσ^{2}} )^{M})

= (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}  (5.3)

· \frac{2^{−2kM}}{M^{(l+p)M}} max ((\frac{1}{n^{(l−p)}σ^{2}} )^{M} , (\frac{2kM}{n^{l−p+1}σ^{4}} )^{M})

I claim that

max (\frac{1}{n^{(l−p)}σ^{2}} , \frac{2kM}{n^{l−p+1}σ^{4}} ) ≤ \frac{1}{(2kM)^{l−p}} max (1, (\frac{2kM}{nσ^{2/r}} )^{2r}) .  (5.4)

Relation (5.4) together with formula (5.3) and the relation k ≥ r if 2r ≤ l − p yield that in this case also the inequality

E (n^{−(l−p)/2}J_{n,2k−l−p}(f ∘ f_{B(N ,N_{1})}))^{2M} ≤ (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{M} ((2M)^{k}σ^{2})^{2M}

· \frac{2^{−2kM}}{M^{2lM}} \frac{1}{(2k)^{(l−p)M}} max (1, (\frac{2kM}{nσ^{2/r}} )^{2M min(r,k)})  (5.5)

holds. Relations (5.2) and (5.5) give a bound on E (n^{−(l−p)/2}J_{n,2k−l−p}(f_{B(N ,N_{1})}))^{2M} in both cases 2r ≥ l − p + 1 and 2r ≤ l − p, thus they imply formula (5.1). Hence to complete the proof of formula (5.1) it is enough to check relation (5.4).

Actually I prove a stronger inequality than formula (5.4). I show that if the inequality 2kM ≤ n holds, and this was assumed among the conditions of relation (5.1),

then the left-hand side of (5.4) can be bounded by \frac{1}{(2kM)^{l−p}} (\frac{2kM}{nσ^{2/r}} )^{2r}, i.e. by the

second term of the maximum at the right-hand side of (5.4). This follows from the following estimations:

\frac{1}{n^{l−p}σ^{2}} = \frac{1}{(2kM)^{l−p}} (\frac{2kM}{nσ^{2/r}} )^{2r} (\frac{n}{2kM} )^{2r−(l−p)} σ^{2} ≤ \frac{1}{(2kM)^{l−p}} (\frac{2kM}{nσ^{2/r}} )^{2r}

and

\frac{2kM}{n^{l−p+1}σ^{4}} = (\frac{2kM}{nσ^{2/r}} )^{2r} \frac{1}{(2kM)^{l−p}} (\frac{n}{2kM} )^{2r−(l−p)−1} ≤ (\frac{2kM}{nσ^{2/r}} )^{2r} \frac{1}{(2kM)^{l−p}} ,

since 2kM ≤ n, 2r − (l − p) ≤ 0 and σ^{2} ≤ 1.

The diagram formula (3.11) together with the triangular inequality in the L_{2M} norm yield that

\begin{matrix}E(J_{n,k}(f)^{4M} = E (∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!}n^{−(l−p)/2} · J_{n,2k−l−p}(f ∘ f_{l,p}))^{2M} \\ ≤ (∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!} (E (n^{−(l−p)/2}J_{n,2k−l−p}(f ∘ f_{l,p})^{2M})^{1/2M})^{2M} . \end{matrix}

(5.6)

Relations (5.1) and (5.6) imply the inequality

\begin{matrix}E(J_{n,k}(f)^{4M} ≤ ((2M)^{k}σ^{2})^{2M} max (1, (\frac{2kM}{nσ^{2/r}} )^{2M min(k,r)}) \\ · (∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!} (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{1/2} \frac{(2k)^{−(l−p)/2}}{2^{k} · M^{l}} )^{2M} \end{matrix} (5.7)

for the random integral J_{n,k}(f) of an (r, σ^{2/r}) dominated function f. By writing the term M^{l} in the form 2^{ml} in (5.7) and comparing formulas (3.17) and (5.7) we can see that relation (3.17) also holds for the new parameter m + 1 with some new (positive) constants C(k, m + 1), k = 1, 2, …, if we can choose them in such a way that they satisfy the inequalities

\frac{C(k, m + 1)}{k^{k}} ≥ ∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!} (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{1/2} \frac{(2k)^{−(l−p)/2}}{2^{k} · 2^{lm}} (5.8)

for all k = 1, 2, . . .. We show with the help of Lemma 5 that it is possible to choose such constants C(k, m), k = 0, 1, …, m = 0, 1, … which satisfy both relation (5.8) and the inequalities sup_{0≤m<∞} C(k, m) ≤ C_{k} < ∞ for all k = 0, 1, 2, …, and beside this also

the inequalities C(0, m) ≥ 1, m = 1, 2, …, and C(k, 0) ≥ C^{k}, k = 0, 1, 2, …, hold for all k = 0, 1, … with a prescribed number C > 0. (The last two inequalities are needed to satisfy relation (5.1) in the degenerate case k = 0 and relation (3.17) for the starting step m = 0.) The proof of the above statement completes the proof of Proposition 2.

To find a sequence C(n, k) satisfying relation (5.8) first I show that a sequence C̄(n, k) satisfying Lemma 5 also satisfies the system of inequalities

C̄(k, m + 1)^{2} ≥ (\frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!})^{2} \frac{k^{2k}}{(2k − l − p)^{2k−l−p}} \frac{(2k)^{−(l−p)}}{2^{2k} · 2^{2lm}} C̄(2k − l − p, m)
for all 0 ≤ k < ∞, and 0 ≤ m < ∞ and 0 ≤ p < l ≤ k.

(5.9)

(Formula (5.9) is a weakened version of (5.8) where it is demanded that if the double sum at the right-hand side of (5.8) is replaced by the maximum of these terms, then this new expression should be smaller than the left-hand side.)

Indeed, to show that a sequence satisfying Lemma 5 also satisfies (5.9) observe that by the estimate \binom{n}{m} ≤ 2^{n} for all 0 ≤ m ≤ n and the rather rough bound 2^{2l}l! ≥ l^{l}

\frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!} ≤ (2k − l − p)^{l−p}\binom{2k − 2l}{k − l}\frac{1}{l!}\binom{l}{p}

≤ (2k − l − p)^{l−p}2^{2(k−l)} \frac{2^{2l}}{l^{l}} 2^{l} = 2^{2l} \frac{2^{2k}(2k − l − p)^{l−p}}{(2l)^{l}} ,

and this estimation implies that inequality (5.9) follows from the relation

C̄(k, m + 1)^{2} ≥ \frac{1}{2^{2lm}} \frac{2^{4l}(2k)^{2k−l+p}(2k − l − p)^{3l−p−2k}}{(2l)^{2l}} C̄(2k − l − p, m)

i.e. from inequality (3.18) imposed in Lemma 5.

We claim that the numbers C(k, m) = AB^{k}C̄(k, m) defined with the help of the above numbers C̄(k, m) and sufficiently large constants A > 1 and B > 1 satisfy formula (5.8) together with the additional properties we have imposed. Indeed, we get with such a choice that

∑_{l=0}^{k} ∑_{p=0}^{l} \frac{(2k − l − p)!}{(k − l)!^{2}(l − p)!p!} (\frac{C(2k − l − p, m)}{(2k − l − p)^{2k−l−p}} )^{1/2} \frac{(2k)^{−(l−p)/2}}{2^{k} · 2^{lm}}

≤ \frac{1}{k^{k}} ∑_{l=0}^{k} ∑_{p=0}^{l} [AB^{2k−l−p}C̄(k, m + 1)^{2}]^{1/2} = \frac{C(k, m + 1)}{\sqrt{A}k^{k}} ∑_{l=0}^{k} ∑_{p=0}^{l} B^{−(l+p)/2}

≤ \frac{C(k, m + 1)}{\sqrt{A}k^{k}} (∑_{j=1}^{∞} B^{−j/2})^{2} ≤ \frac{C(k, m + 1)}{k^{k}}

if we choose the number A = A(B) sufficiently large. Because of Lemma 5 the numbers C(k, m) also satisfy the relation sup_{0≤m<∞} C(k, m) ≤ C_{k} < ∞ with an appropriate

constant C_{k}. Beside this, if we choose B ≥ C, then because of the relation C̄(k, 0) = 1 formulated in Lemma 5 also the relation C(k, 0) ≥ C^{k} holds for all k = 0, 1, . . .. The relation C(0, m) = AC̄(0, m) = A ≥ 1 also holds. The proof of Proposition 2 with the help of Lemma 5 is finished.

The proof of Lemma 5. Let us introduce the numbers

A(l, p, k, m) = 2^{2l(4−m)} \frac{(2k)^{2k−l+p}(2k − l − p)^{3l−p−2k}}{(2l)^{2l}} , 0 ≤ p ≤ l ≤ k, m ≥ 0.

To construct a sequence C̄(k, m) which satisfies Lemma 5 it is enough to show that there exist such numbers D(m) ≥ 1, m = 0, 1, …, which satisfy the inequalities D(m)^{2k} ≥

A(l, p, k, m) for all 0 ≤ p ≤ l ≤ k, m = 0, 1, …, and ∏_{m=0}^{∞} D(m) < ∞. Indeed, if there

exists such a sequence D(m), then the numbers C̄(k, 0) = 1, C̄(k, m) = ( ∏_{p=0}^{m−1} D(p))^{k}

for m ≥ 1, k ≥ 0, satisfy Lemma 5, since the above defined numbers satisfy the inequality

C̄(k, m + 1)^{2} ≥ C̄(2k − l − p, m)D(m)^{2k} ≥ A(l, p, k, m)C̄(2k − l − p, m)

for all 0 ≤ p ≤ l ≤ k.

To show that we can define numbers D(m) with the above required properties let us consider the extension of the sequence A(l, p, k, m) to the real valued function

A_{k,m}(u, v) = 2^{2(4−m)v} \frac{(2k)^{2k+u−v}(2k − u − v)^{3v−u−2k}}{(2v)^{2v}} , 0 ≤ u ≤ v ≤ k,

and let us estimate its maximum. To get a simpler estimate let us bound the function A_{k,m}(u, v) in the following way. Observe that

A_{k,m}(u, v) ≤ (\frac{2k}{2k − u − v} )^{v−u} A_{k,m}(u, v) = 2^{2(4−m)v} \frac{(2k)^{2k}}{(2k − u − v)^{2k−2v}(2v)^{2v}}

≤ 2^{2(4−m)v} \frac{(2k)^{2k}}{(2k − 2v)^{2k−2v}(2v)^{2v}}

for 0 ≤ u ≤ v ≤ k, and this inequality implies that

sup_{0≤u≤v≤k} A_{k,m}(u, v) ≤ sup_{0≤v≤k} B_{k,m}(v)

with

B_{k,m}(v) = 2^{2(4−m)v} \frac{(2k)^{2k}}{(2k − 2v)^{2k−2v}(2v)^{2v}} .

Simple differentiation of the function log B_{k,m}(v) shows that the function B_{k,m}(v) takes

its maximum in the point v̄ = \frac{k}{2^{(m−4)} + 1}, and B_{k,m}(v̄) = (1 + 2^{4−m})^{2k}.

The above calculations imply that the numbers D(m) = 1 + 2^{4−m}, m = 0, 1, 2, …,

satisfy the inequality sup_{0≤p≤l≤k} A(l, p, k, m) ≤ D(m)^{2k}. The relation ∏_{m=0}^{∞} D(m) < ∞

also holds. Hence, as the argument at the beginning of the proof shows the numbers

C̄(k, 0) = 1, C̄(k, m) = ( ∏_{p=0}^{m−1} D(p))^{k} for m ≥ 1 satisfy the properties demanded in

Lemma 5.

References:

1.) Arcones, M. A. and Giné, E. (1993) Limit theorems for U-processes. Ann. Probab.

21, 1494–1542 2.) Arcones, M. A. and Giné, E. (1994) U-processes indexed by Vapnik–Červonenkis

classes of functions with application to asymptotics and bootstrap of U-statistics with estimated parameters. Stoch. Proc. Appl. 52, 17–38

3.) Major, P. (1981) Multiple Wiener–Itô integrals. Lecture Notes in Mathematics

849, Springer–Verlag, Berlin Heidelberg, New York, 4.) Major, P. (1988) On the tail behaviour of the distribution function of multiple

stochastic integrals. Probability Theory and Related Fields, 78, 419–435 5.) Major, P. and Rejtő, L. (1988) Strong embedding of the distribution function under

random censorship, Annals of Statistics, 16, 1113–1132 6.) Major, P. and Rejtő, L. (1998) A note on nonparametric estimations. In the con-

ference volume to the 65. birthday of Miklós Csörgő.759–774 7.) Major, P. (2003) An estimate on the maximum of a nice class of stochastic integrals.

submitted to Probability Theory and Related Fields, 8.) Surgailis, D. (1984) On multiple Poisson stochastic integrals and associated Markov

semigroups. Probab. Math. Statist. 3. no. 2 217-239